3145-0238 Supporting Statement Part B

Section B
Introduction
B.1. Respondent Universe and Sampling Methods
The IIP Program Monitoring Clearance’s goal is to count and describe the universe of IIPfunded research and education projects. The statistical method employed in each collection
is that of a census of all IIP-funded projects under the corresponding program for which the
collection is being prepared. Data collection is expected to involve all awardees in the
program.
The table below shows the total universe and sample size for each of the collections.
Table 4. Respondent Universe and Sample Size of ENG Program Monitoring Clearance
Collections
Collection Title

Universe of
Respondents

Sample Size

Grant Opportunities for Academic Liaison
with Industry (GOALI)

200

200

Innovation Corps (I-Corps) Longitudinal
Collection

700

700

Innovation Corps (I-Corps) Pre-Course
Survey Questionnaire

800

800

Innovation Corps (I-Corps) Post-Course
Survey Questionnaire

800

800

Partnerships For Innovation: Accelerating
Innovation Research (PFI:AIR)

200

200

Partnerships For Innovation: building
Innovation Capacity (PFI:BIC)

30

30

Small Business Innovation Research
(SBIR)/Small Business Technology
Transfer (STTR)

900

900

SBIR Baseline Monitoring Survey

800

800

10

B.2. Information Collection Procedures/Limitations of the Study
The data collections in this clearance are expected to use web-based instruments but some
could use interviews, either in person or by phone. Each respondent will provide answers
once a year during the life of the award. Respondents post-award will be invited to report
voluntarily up to four times over the course of 10 years after the award has expired.
IIP understands the limitations of the Program Monitoring Clearance, particularly in terms
of using the data to determine program effectiveness. Data collected under this clearance are
for monitoring purposes; evaluation studies are cleared under separate OMB requests.
However, monitoring systems covered by this request will be explicitly identified as a source
of data for independent program evaluations. IIP Program Monitoring Clearance data are not
used to determine the ultimate effectiveness of research, but they are a key element in IIP’s
efforts to manage its program portfolio, to report on agency activities and goals, and to lay
the groundwork for future evaluations.
B.2.1. Statistical Methodology for Stratification and Sample Selection
Each of the collections in this clearance request is a census, in which the sample size is the
universe. Details on the size of the universe in each collection are included in the burden
estimate and in section B.1 above. A census approach to data collection is critical for
monitoring of scientific research, particularly fundamental research, due to the uniqueness
of each project. The merit review process for each program elicits unique and transformative
projects in their contribution and methods. Each project asks a different research question
and uses different experimental and theoretical approaches. As such, would be impractical
to consider sampling methods that will yield a representative population of the universe of
NSF funded research awards.
B.2.2. Estimation Procedure
Not applicable
B.2.3. Degree of Accuracy Needed for the Purpose Described in the Justification
Not applicable
B.2.4. Unusual Problems Requiring Specialized Sampling Procedures
Not applicable
B.2.5. Justification for Data Collection Cycles
In post-award monitoring systems, IIP endeavors to collect data on indicators of outcomes
and impacts of investments in research that are unlikely to be realized over the course of the
award. These data may include indicators such as publications, patents, and licensing
activities, student career choices after participating in the funded research, and technologies
11

Developed from discoveries made by fundamental research, for example. In many cases,
particularly in the case of fundamental research, the most important outcomes of research
investments are not expected to be realized for several years after the award has ended, due
to the inherent time lag in the transition from discovery to application of research findings.
As such, we propose to collect data on these outcomes and impacts of our research
investments for up to 10 years post-award. These collections for programs in IIP which are
often focused on translation or commercialization of research findings, the important
indicators are expected to appear sooner after the award ends. However, due to the burden
on the PIs and our expectation that certain outcomes and impacts are more likely to occur at
less frequent intervals post-award, in most cases we propose to collect data at 1-year, 3-year,
and 5-year intervals post-award, with a fourth data point collected at 10 years post-award
for some programs.
B.3. Methods for Maximizing the Response Rate and Addressing Issues of
Nonresponse
All potential collections during the life of the award included in this clearance may become
part of the reporting required of awardees for specific solicitations or programs. In those
specific cases, a high response rate is expected. The pre-and post-survey questionnaires for
the I-Corps program will be implemented before and after the teams of grantees undergo
training. A high-response rate is also expected in this case.
For post-award monitoring, participation is entirely voluntary. Although there is no penalty
for non-participation with data collection requests outside of the life of the award, many
respondents are eager to communicate their achievements to NSF program staff in general,
so we foresee no obstacles to achieving a high response rate even outside of the life of the
award. The table below shows the expected response rates for each of the individual
collections based on NSF’s experience with other monitoring systems.
The voluntary nature of the response will be clearly communicated to respondents in each
instance.
Table 5. Expected Response Rates for ENG Program Monitoring Clearance Collections
Collection Title

Expected Response Rate

Grant Opportunities for Academic Liaison
with Industry (GOALI)

80%

Innovation Corps (I-Corps) Longitudinal
Collection

80%

Innovation Corps (I-Corps) Pre-Course
Survey Questionnaire

90%

12

Innovation Corps (I-Corps) Post-Course
Survey Questionnaire

90 %

Partnerships For Innovation: Accelerating
Innovation Research (PFI:AIR)

80%

80%

Partnerships For Innovation: building
Innovation Capacity (PFI:BIC)
Small Business Innovation Research
(SBIR)/Small Business Technology Transfer
(STTR)

80%

SBIR Baseline Monitoring Survey

90 %

For web-based collection systems, a series of e-mail messages and phone calls, including
introductory emails alerting the respondent to the data that will be collected will also be used
to follow up with respondents.
B.4. Tests of Procedures or Methods
Test methods used to improve the questions in the ENG IIP Program Monitoring Clearance
include feedback from PIs, both as data are collected and during meetings and conferences;
review by NSF staff; and testing performed by the data collection system developers. These
monitoring collections are based on data collection methods currently used by other NSF
groups, and many of the items and response categories follow formats that are already in
place.
B.6. Contact Information for Individuals Responsible for Data Collections
Yuen Lau
Science Analyst
Division of Industrial Innovation and Partnerships
Directorate for Engineering
National Science Foundation
Alexandria, VA 22314
703-292-2342
[email protected]

13

Appendix I – Crosswalk
ENG PROGRAM MONITORING CLEARANCE CROSSWALK
Overview of Types of Data Elements Found in the Tasks Under This Request

Data Elements

GOALI

I-Corps

Name (of PI, co-PI, trainees, etc.)

X

X

Contact Information (email address of PI, co-PI, trainees, etc.)

X

X

Name of Student’s Advisor/Project Supervisor
Field/Area of Study/Student Major

X
X

X
X

Student Educational Data (e.g., year in school, expected/actual
graduation date, GPA, degree held or anticipated – graduate,
undergraduate, master’s, PhD, etc.)

X

Financial Support Received From Other Sources (e.g., amount and
term of support, counts of students receiving support, sometimes
with demographic data)
Partner Organizations or Collaborators
Number of Project Participants (sometimes with demographic data)
Educational and Professional Development Activities of Staff
and/or Participants
Other Project Activities (e.g., outreach, broadening participation
activities, etc.)
Health of Partnerships (IBM 7 Keys to Project Management
indicators)

I-Corps
PRE
X
X

I-Corps
POST
X
X

X

X

PFI:AIR

PFI:BIC

SBIR

X

X

X

X

X

X

X
X

X
X

X
X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Data Elements

GOALI

I-Corps

I-Corps
PRE

I-Corps
POST

Degrees Granted/Received (sometimes with demographic data)
Project Products/Outputs (e.g., proposals submitted, proposals awarded,
presentations given, publications, citations, patents awarded, other
activities that are not already reported in RPPR)
Best Practices Described
Changes in Practices Implemented at Respondent Institutions or
Elsewhere/Impact on Faculty, Students, Scientific Community
Project Goals/Targets Established and/or Described and/or Achieved
Outreach/Dissemination Activities Conducted at or beyond endpoint of
award
Partner Organizations/Collaborative Projects
Materials Used/Changes Made in Curricula at any level (K12,
undergraduate, graduate)
Student Career Goals
PI or Trainee recognition, promotions, and awards
Other Trainee/Student Data
Other Faculty Data
Other Activities/Additional Information
New research directions emerging from funded research
Community/relationships between academia-industry, engineeringpersons with disabilities, engineering-social/behavioral sciences, etc.
Licensing activities related to technology/services/processes
Career pathways of each trainee trained under awards
Technology/service/process/partnership model adopted by others in
research community
New conferences, societies, committees, journals, disciplines
established at intersections of transdisciplinary funded research
Businesses or start-up companies formed around funded research
Additional awards at NSF or other agencies spawned from the funded
research
I-Corps course satisfaction

X

X

X

PFI:BI
C
X

X

X

X

X

PFI:AIR

SBIR
X
X

X
X
X

X

X

X

X

X

X

X

X

X

X

X

X

X

X
X
X
X
X
X

X
X
X
X

X
X
X

X

X

X

X

X

X
X
X
X
X
X

X
X
X
X
X
X

X

X

X
X

X
X

X

X

X
X
X
X

X

X
X

X

X

X

X

X

X

X

X

X

X

X

X