NIH SBIR Program - Evaluation Report - dated January 23, 2009

2008 National Survey to Evaluate the NIH SBIR Program - January 23, 2009.pdf

NIST Generic Clearance for Program Evaluation Data Collections

NIH SBIR Program - Evaluation Report - dated January 23, 2009

OMB: 0693-0033

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0693-0033 can be found here:

Document [pdf]

Download: pdf | pdf

Final Report

January 23, 2009

National Survey to Evaluate the NIH SBIR Program

Final Report

Jo Anne Goodnight
Office of Extramural Research
National Institutes of Health
Bethesda, MD
Susan Pucie
National Heart, Lung and Blood Institute
National Institutes of Health
Bethesda, MD

January 23, 2009

Stephanie Karsten
Lynne Firester
Georgine Pion
April Smith
Maura Kephart
Humanitas, Inc.
Silver Spring, MD

National Survey to Evaluate the NIH SBIR Program
Final Report

TABLE OF CONTENTS

ACKNOWLEDGEMENTS.....................................................................................x

EXECUTIVE SUMMARY .....................................................................................xi

1. INTRODUCTION
1.1 Background ..................................................................................................... 1-2

1.1.1 2002 National Survey to Evaluate the NIH SBIR Program .......................................1-2

1.1.2 Subsequent Performance Monitoring and Evaluation ..............................................1-3

1.2 Methodology for the 2008 NIH Survey .......................................................... 1-4

1.2.1
1.2.2
1.2.3
1.2.4
1.2.5
1.2.6
1.2.7

Questionnaire ............................................................................................................1-5

Sampling ...................................................................................................................1-5

Contact Database Update.........................................................................................1-7

Survey Implementation..............................................................................................1-8

Survey Response ......................................................................................................1-8

Data Cleaning and Coding ........................................................................................1-9

Analytic Strategies and Reporting Practices...........................................................1-10

1.3 Strengths and Limitations of the Evaluation.............................................. 1-11

2. CHARACTERISTICS OF SURVEY RESPONDENTS
2.1 Usability and Eligibility of Potential Respondent Small Businesses......... 2-1

2.2 Spokesperson Characteristics ..................................................................... 2-2

2.3 Awardee Small Businesses............................................................................ 2-3

2.3.1 Nonrespondent and Respondent Awardee Comparison ..........................................2-3

2.3.2 Awardee Small Business Respondents ....................................................................2-5

2.4 SBIR-Supported Projects and Products ....................................................... 2-8

3. STIMULATING TECHNOLOGICAL INNOVATION
3.1 NIH SBIR Awardees Stimulated Technological Innovation........................ 3-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

TABLE OF CONTENTS (continued)
3.2 Performance on NIH SBIR Product Development Targets.......................... 3-2

3.2.1 New or Improved Products in Health-Related Fields................................................3-2

3.2.2 Technical Articles and Related Activities in Support of the NIH Mission ..................3-2

3.2.3 Patents and Other Indicators of Stimulating Technological Innovation ....................3-6

3.3 Performance on Receipt of Additional SBIR Awards .................................. 3-9

3.4 Comparison with 1997-2001 Phase II Awardees ........................................ 3-10

4. INCREASED USE OF SMALL BUSINESS CONCERNS
4.1 NIH SBIR Awardees Met Federal Research and Development

Needs ................................................................................................................ 4-1

4.2 Performance on Contributions to Knowledge ............................................. 4-2

4.3 Performance on Dissemination of Information ............................................ 4-4

4.3.1 NIH Dissemination to SBIR Phase II Awardees.......................................................4-4

4.3.2 Awardee Dissemination of Health-Related Information ...........................................4-4

4.4 Satisfaction with the SBIR Application, Review, Award, and

Post-Award Processes .................................................................................. 4-9

4.5 Comparison of Two Awardee Cohorts’ Performance in Increasing the

Use of Small Business to Meet Federal R&D Needs................................. 4-10

5. COMMERCIALIZATION OF INNOVATIONS
5.1 NIH SBIR Awardees Commercialized Innovations ...................................... 5-1

5.2 Performance in Commercializing SBIR-Funded Products ......................... 5-2

5.2.1
5.2.2
5.2.3
5.2.4
5.2.5
5.2.6
5.2.7

Product Sales ...........................................................................................................5-2

Cumulative Sales .....................................................................................................5-5

FDA Approval of SBIR-Funded Products.................................................................5-9

Licensing of SBIR-Funded Projects .......................................................................5-12

Other Evidence of Commercialization....................................................................5-12

Receipt of Additional Non-SBIR Funding ..............................................................5-13

Summary of Commercialization Efforts .................................................................5-18

iii

J anuary 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

TABLE OF CONTENTS (continued)
5.3 Growth of Awardee Small Businesses ..................................................... 5-20

5.4 Comparison of Two Awardee Cohorts’ Performance in

Commercializing Innovations .................................................................. 5-23

6. CONCLUSIONS AND RECOMMENDATIONS
6.1 Awardees Met or Exceeded NIH Performance Targets .............................. 6-1

6.2 Enhancements to NIH SBIR Program Monitoring and Evaluation

Should be Considered ................................................................................... 6-3

6.2.1 Update the SBIR Program Evaluation Framework ..................................................6-3

6.2.2 Expand Program Monitoring Capabilities.................................................................6-4

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

LIST OF EXHIBITS
Exhibit 1-1 NIH Institutes and Centers within the 2008 Survey Sample ........................ 1-6

Exhibit 2-1 Characteristics of Survey Respondents ....................................................... 2-3

Exhibit 2-2 Selected Characteristics of Nonrespondent and Respondent SBIR

Awardee Small Businesses...................................................................................... 2-4

Exhibit 2-3 Awardee Small Business Respondents by State ......................................... 2-5

Exhibit 2-4 Years in Operation at the Time of the SBIR Award for

Responding Small Businesses................................................................................. 2-6

Exhibit 2-5 Major Field of Business Reported by SBIR-Funded Awardees

for FYs 2002-2006 ................................................................................................... 2-7

Exhibit 2-6 Type of Planned Product for SBIR Funded Awardees................................. 2-9

Exhibit 2-7 Technological Innovations by Type of Planned Product ............................ 2-10

Exhibit 2-8 Current Project Status by Fiscal Year of Award......................................... 2-11

Exhibit 2-9 Current Project Status by Type of Planned Product .................................. 2-12

Exhibit 2-10 Current Project Status by NIH IC Sponsor............................................... 2-14

Exhibit 2-11 Reasons for Discontinued Projects.......................................................... 2-15

Exhibit 2-12 Discontinued Projects by Product Characteristic

and Awardee’s Receipt of Additional SBIR Awards ............................................... 2-16

Exhibit 3-1 Summary of Outcomes Associated with Stimulating Technological

Innovation................................................................................................................. 3-1

Exhibit 3-2 Summary of Publications, Presentations, and Awards

Associated with SBIR-Supported Projects ............................................................... 3-3

Exhibit 3-3 Awardees with Presentations, Publications, and Awards

by the Current Status of Their SBIR-Supported Project........................................... 3-4

Exhibit 3-4 Awardees with Publications, Presentations, and Awards

by the Type of SBIR-Supported Product.................................................................. 3-5

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

LIST OF EXHIBITS (continued)
Exhibit 3-5 Awardees’ Patents, Copyrights, and Trademarks........................................ 3-6

Exhibit 3-6 Awardees with One or More Patents, Copyrights, and Trademarks

by Type of SBIR-Supported Project ......................................................................... 3-7

Exhibit 3-7 Reported Patents, Pending Patent Applications, Copyrights, and

Trademarks by Status of SBIR-Funded Project ....................................................... 3-8

Exhibit 3-8 Additional Related Phase I and Phase II SBIR Awards ............................... 3-9

Exhibit 3-9 Comparison of NIH SBIR Awardees’ Performance

in Stimulating Technological Innovation ................................................................ 3-11

Exhibit 4-1 Summary of Outcomes Associated with

Using Small Business Concerns to Meet Federal Research and

Development Needs................................................................................................. 4-1

Exhibit 4-2 Medical, Societal, or Technological Outcomes of the

SBIR-Funded Projects ............................................................................................. 4-2

Exhibit 4-3 Target Population Groups that Use or Are Likely to Use the

SBIR-Supported Product.......................................................................................... 4-5

Exhibit 4-4 Target Populations Currently Using or Likely to Use SBIR-Funded

Products .................................................................................................................. 4-6

Exhibit 4-5 Anticipated Size of Most Important Target Populations

for Ongoing Projects ................................................................................................ 4-8

Exhibit 4-6 Awardee Satisfaction with the NIH Application, Review, Award,

and Post-Award Processes...................................................................................... 4-9

Exhibit 4-7 Comparison of NIH SBIR Awardees’ Performance in Increasing

Use of Small Business to Meet Federal R&D Needs ............................................. 4-11

Exhibit 5-1 Summary of Outcomes Associated with Commercializing Innovations ....... 5-1

Exhibit 5-2 Number and Percent of Awardees That Expected Sales by

Type of Product........................................................................................................ 5-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

LIST OF EXHIBITS (continued)
Exhibit 5-3 Current Status of Sales by Fiscal Year of Phase II Award........................... 5-4

Exhibit 5-4 Realization of Sales upon Project Completion by Type of SBIR-Supported

Product..................................................................................................................... 5-5

Exhibit 5-5 Amount of Cumulative Sales for Completed SBIR-Funded Projects ........... 5-6

Exhibit 5-6 Percent of Companies that Achieved Cumulative Sales

by Type of Product ................................................................................................... 5-7

Exhibit 5-7 Estimated Cumulative Sales for Awardees Realizing Sales by Awarding

NIH Institute or Center ............................................................................................. 5-8

Exhibit 5-8 Summary of SBIR Projects’ Sales Performance for FYs 2002-2006

as of December 2007............................................................................................... 5-9
Exhibit 5-9 Percent of SBIR-Funded Products Requiring FDA Approval by Type
of Product............................................................................................................... 5-10

Exhibit 5-10 FDA Approval Status by Type of SBIR-Funded Product ......................... 5-11

Exhibit 5-11 Stage of the Approval Process for SBIR-Funded Projects

that Submitted Applications to the FDA ................................................................. 5-12

Exhibit 5-12 Percent of Awardees Engaged in Marketing Activities by

Current Project Status............................................................................................ 5-14

Exhibit 5-13 Percent of Awardees that Received Additional Non-SBIR

Funding by Type of SBIR-Supported Product........................................................ 5-15

Exhibit 5-14 Percent of Awardees that Reported Receipt of Additional

Non-SBIR Funding by Awarding NIH Institute or Center ....................................... 5-16

Exhibit 5-15 Percent of Awardees that Reported Receipt of Additional

Non-SBIR Funding by Funding Source and Most Important Funding
Source.................................................................................................................... 5-17

Exhibit 5-16 Number and Type of Commercialization-Related Outcomes

Achieved by Awardees........................................................................................... 5-19

Exhibit 5-17 Number of Full-Time Employees for Awardees ....................................... 5-21

Exhibit 5-18 Awardees’ Reported Growth-Related Activities ....................................... 5-22

vii

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

LIST OF EXHIBITS (continued)
Exhibit 5-19 Comparison of SBIR Awardees’ Performance in

Commercializing Innovations ................................................................................. 5-24

Exhibit 6-1 Summary of Performance for FYs 2002-2006 SBIR Phase II Awardees..... 6-2

viii

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

APPENDICES

Evaluation Framework
A1
Program Goals and Measures

A2
Map of Survey Items to Goals

Survey Instrument

Email Messages and Telephone Follow-up Briefing
C1
Contact Update Email
C2
Email Verification Message
C3
Advance Email
C4
First Email Cover Letter
C5
Thank You/Reminder Email
C6
Second Email Cover Letter
C7
Personal Jo Anne Goodnight Email
C8
Incomplete Email
C9
Promise to Complete Email
C10 Promise to Complete Last Call Email
C11 Final Notice for the SBIR Survey Email
C12 Telephone Follow-up Briefing

Response Rate – Final Disposition of Sample
D1 Response Rate
D2 Usability Rates for Remaining Institutes and Centers

Survey Sample

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

ACKNOWLEDGEMENTS

Contract support for the survey was provided from NIH Evaluation Set-Aside funds. Humanitas, Inc.,
planned the survey methodology, implemented the questionnaire online, conducted the follow-up with
nonresponders, and cleaned and analyzed the survey data.
Guidance on survey topics and questions was provided by an internal NIH staff working group whose
membership included:
•
•
•
•
•
•
•
•

Kay Etzler, Office of the Director, Division of Special Programs
Frankie Giles, National Heart, Lung and Blood Institute
Jo Anne Goodnight, Office of the Director, Division of Special Programs
Margaret Grabb, Ph.D. National Institute of Mental Health
Rosemarie Hunziker, Ph.D. National Institute of Biomedical Imaging and Bioengineering
Matthew Portnoy, Ph.D., National Institute of General Medical Sciences
Susan Pucie, National Heart, Lung and Blood Institute
Michael Weingarten, National Cancer Institute

Finally, special thanks to the SBIR awardees who so willingly participated in the study to describe their
experiences with the SBIR program and their project outcomes.

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXECUTIVE SUMMARY
Established in 1982 through the Small Business Innovation Development Act, the Small Business
Innovation Research (SBIR) program offers qualified small business concerns the opportunity to propose
and develop innovative ideas with commercial potential while meeting the specific research and
development (R&D) needs of the Federal government. SBIR legislation currently requires agencies with
extramural R&D budgets that exceed $100 million to set aside 2.5 percent of their extramural R&D funds
for an SBIR program. Among the 11 Federal agencies that currently participate in the SBIR Program, the
National Institutes of Health (NIH) is the largest civilian agency and the second largest participating
Federal agency overall. Since the inception of the program in 1982, NIH has invested over $5 billion in
research support to small business concerns.
This report summarizes the findings from a 2008 survey of small businesses that competed successfully
for SBIR Phase II awards in fiscal years (FYs) 2002-2006. This new survey was a logical next step in the
NIH SBIR program evaluation and monitoring effort. A 2002 national survey to assess the outcomes of
the NIH SBIR program had described NIH SBIR program performance of Phase II awardees funded from
FYs 1992 through 2001. The 2008 survey included a new group of awardees. Both surveys addressed the
same major evaluation issues:
• To what degree has the NIH SBIR program stimulated technological innovation?
• Has the NIH SBIR program’s use of small business concerns satisfied Federal R&D needs?
• Has the NIH SBIR program increased private sector commercialization of innovations derived
from Federal R&D funding?

Survey Response
To leverage NIH expenditures to date and also to promote comparability of results, the design and
execution of the 2008 national survey to assess the outcomes of the NIH SBIR program was based on the
same guiding principles used for the 2002 survey.
The survey was administered to all 918 eligible awardee small businesses that received an NIH SBIR
Phase II award in FYs 2002-2006. There were 719 respondent companies, and the final response rate
was 78 percent.
Respondents and nonrespondents were similar in terms of the Institute/Center (IC) that sponsored the
Phase II award, the fiscal year of the award, and the geographic region in which the small business was
located. Coupled with the high response rate, these similarities reduce the potential threat of
nonrespondent bias to the survey results.
The largest percentage of respondents (32 percent) was located in the West, and the smallest percentage
(16 percent) in the Midwest. Twenty-eight percent were located in the Northeast, and the remaining 24
percent were in the South.
Over one-fourth of respondents (28 percent) received their Phase II award in FY 2006; between 17 and 19
percent of the companies received their award in each of the other calendar years covered by the survey,
FYs 2002-2006. The National Cancer Institute (NCI) supported 15 percent of all respondents, and the
National Heart, Lung and Blood Institute (NHLBI) supported 13 percent. The National Institute

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

of General Medical Sciences (NIGMS); the National Institute of Diabetes, Digestive, and Kidney
Diseases (NIDDK); the National Institute on Aging (NIA); and the National Institute of Neurological
Disorders and Stroke (NINDS) each awarded funds to another 6 to 8 percent of the companies that
responded to the survey. The remaining awardee companies were funded by one of the other 16
participating ICs.
Overall, 41 percent of the respondents reported that their SBIR-funded product, process, or service was
currently in the development phase at the time of the survey. Another 20 percent said that the SBIRfunded product, process, or service was being commercialized, and 22 percent stated that it was in use by
the target population(s). Ten percent said that product, process, or service production was “on hold”,
awaiting the receipt of additional funding, or inactive for another reason. Eight percent of the awardee
companies reportedly had discontinued work on the project. (Numbers total more than 100 due to
rounding).

Summary of Findings
Three of the four program goals were the survey focus – stimulate technological innovation (Goal 1), use
small business concerns to meet R&D needs (Goal 2), and increase the commercialization of innovations
(Goal 4). The exhibit on the following page displays these three goals, related performance indices and
measures promulgated by the NIH SBIR program Evaluation Framework for each, and the 2008 survey
finding about the percentage of awardees that achieved each performance index.

xii

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Summary of 2008 Survey Findings:

Performance of NIH SBIR Phase II Awardees in FYs 2002-2006*

Finding

NIH SBIR Performance Goal
NIH SBIR Performance Index

NIH SBIR Performance Measure

1.0 Stimulate Technological Innovation
1.1 Whether or not sales have occurred, NIH SBIR awardees produce new or improved
products, processes, usages, and/or services in support of the NIH mission.

Percent of awardees developing new or improved products

Percent of awardees having published one or more technical articles on new or improved
SBIR supported products

Percent of awardees having obtained one or more patents relevant to the core technology
supported by the Phase II award
1.2 NIH SBIR awardees receive additional Phase I or Phase II awards that relate to the core
technology.

Percent of awardees receiving additional Phase I or Phase II awards that related to the
core technology

82%
53%

31%

58%

2.0 Use Small Business Concerns to Meet Federal R&D Needs
2.1 NIH awardees make contributions to knowledge in health promotion, disease prevention,
diagnosis, health care, and amelioration and cure of disease.

Percent of SBIR awardees that have contributed to increases in knowledge regarding
health promotion, disease prevention, diagnosis, health care, and amelioration and cure of
disease
2.2 NIH awardees are able to obtain and disseminate health-related information

2.3

Percent of awardees that have disseminated or plan to disseminate SBIR supported
technology and information among populations using and receiving health and health care
resources
NIH SBIR awardees express satisfaction with the usefulness of the NIH SBIR program

Percent of awardees, depending on the program component, that are completely or mostly
satisfied with the SBIR application, review, award, and post-administration of the program

82%

74 – 90%

4.0 Increase the Commercialization of Innovations
4.1 Companies with NIH SBIR awards commercialize new or improved products, processes,
usages, and/or services in health-related fields.

Percent of SBIR-supported products, processes, usages, and/or services that yield sales

Percent of awardees that have successfully executed licensing arrangements for their
SBIR-supported product

25%

Percent of awardees reported commercializing the core technology or information
supported by their Phase II award

61%

Percent of awardees obtaining additional non-SBIR funding for their Phase II product

4.2 Companies with NIH SBIR awards grow their companies.

Percent of awardees that believed that the SBIR Program had had an impact on their
ability to grow their companies in terms of hiring additional personnel
*The survey assessed performance for Goals 1, 2, and 4 from the Evaluation Framework for the NIH
SBIR program.

xiii

33%

36%

82%

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Customer satisfaction responses indicated that nearly all SBIR awardees (96 percent) were satisfied with
their experiences in obtaining information about the Phase II program. Between 88 and 90 percent
expressed satisfaction with the application, award, and post-award administration components of the
Phase II program.
The 2008 survey instrument was essentially identical to that of the 2002 survey, so it was possible to
compare outcomes for respondents to both surveys. For such comparisons, the respondent cohort for the
2002 survey was restricted to awardees that received grant funding during the last five-year period
surveyed, FYs 1997 through 2001. This made the elapsed time from award receipt to survey
administration as comparable as possible because the 2008 survey cohort also received grant funding for a
five-year period, FYs 2002-2006.
Performance of the two cohorts did not significantly differ for the majority of outcomes. The earlier
cohort did have significantly higher percentages of awardees that published one technical article (68
percent versus 43 percent); made conference presentations (79 percent versus 73 percent); and were
granted at least one patent for an SBIR-supported product (39 percent versus 31 percent).

xiv

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

1. INTRODUCTION

The Small Business Innovation Research (SBIR) program, first authorized in 1982, occupies a key niche
in the nation’s scientific research and development arena. The program encourages small businesses to
explore their technological potential and provides the incentive to profit from commercializing products,
processes and services developed with SBIR support.
Through a competitive, three-phase award system, the program provides qualified small business
concerns with opportunities to propose and develop innovative ideas that meet the mission or specific
research and development (R&D) needs of the Federal government.
• Phase I. The objective of Phase I is to establish the technical merit, feasibility, and potential for
commercialization of the proposed research or R&D efforts, and to determine the quality of
performance of the small business awardee organization prior to providing further Federal
support in Phase II. Support under Phase I normally may not exceed $100,000 for total costs for a
period normally not to exceed six months.
• Phase II. The objective of Phase II is to continue the R&D efforts initiated in Phase I. Funding is
based on the results achieved in Phase I and the scientific and technical merit and commercial
potential of the project proposed in Phase II. Only Phase I awardees are eligible for a Phase II
award. Support for SBIR Phase II awards normally may not exceed $750,000 in total costs for a
period normally not to exceed two years.
• Phase III. The objective of Phase III, where appropriate, is for the small business concern to
pursue with non-SBIR/STTR funds the commercialization objectives resulting from the research
and R&D activities of Phases I and II. Phase III may involve follow-on non-SBIR funded R&D
or production contracts for products, processes or services intended for use by the U.S.
Government.
The National Institutes of Health (NIH) is the largest civilian agency and the second largest participating
agency of the 11 Federal agencies participating in the SBIR program. Since the inception of the program
in 1982, NIH has invested over $5 billion in research support to small business concerns. The NIH SBIR
program is intended to foster the agency’s mission to “uncover new knowledge that will lead to better
health for everyone,” particularly in translating research results into tangible products and services that
will improve human health.
In late 2006, NIH SBIR program administrators from a cross-section of the Institutes and Centers (ICs)
decided to sponsor a survey of Phase II awardees whose awards were granted in fiscal years (FYs) 20022006. This survey was planned as a next step in documenting NIH SBIR program outcomes. The first
step was the 2002 National Survey to Assess the Outcomes of the NIH SBIR Program, which described
NIH SBIR program performance from FYs 1992-2001. For both surveys, the major evaluation questions
addressed targeted performance related to three program goals:
• To what degree has the NIH SBIR program stimulated technological innovation?
• Has the NIH SBIR program’s use of small business concerns satisfied Federal R&D needs?

Page 1-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

• Has the NIH SBIR program increased private sector commercialization of innovations derived
from Federal R&D funding?
This report documents the 2008 national survey to assess the outcomes of the NIH SBIR program:
• Chapter 1 discusses background issues that influenced and prompted the study and also

summarizes the study methodology.

• Chapter 2 notes characteristics of potential and actual survey respondents.
• Chapters 3-5 describe study findings about each of the three key evaluation questions.
• Chapter 6 presents study conclusions and recommendations.

1.1

Background

Before 2003, there was little formal analysis of either the overall SBIR program or the one administered
by the NIH, with the exception of a few studies by the Government Accountability Office (GAO), the
Office of the Inspector General (OIG), and the Small Business Administration (SBA). These early
studies raised questions about program performance, prompting a series of recent evaluations of both the
Federal program and its NIH component.
1.1.1

2002 National Survey to Evaluate the NIH SBIR Program

Beginning in the late 1990s, NIH SBIR program administrators from a cross-section of ICs took a
heightened interest in the success of the NIH SBIR program. By 2001, they formalized consensus about
how to evaluate the program in a manner that would assess responsiveness to Congressional intent for the
SBIR program. They produced an Evaluation Framework document that reflects their agreement about
standards, indices, and measures for describing how well the NIH SBIR program is performing in
meeting all Congressional program goals. The Framework is, to their knowledge, the first such guide for
any SBIR program about why and how to evaluate overall performance.
The NIH administrators then decided to survey NIH awardees to learn about program performance. They
developed a draft survey instrument to query NIH awardees about performance of three of four program
goals. A focus group of nine principal investigators (PIs) who had won Phase I and Phase II SBIR
awards provided feedback about the questionnaire, the most survey appropriate respondents, the response
burden, the best metrics for measuring successful award outcomes, and motivations for participating in
the survey.
Based on the focus group findings, NIH revised the survey instrument and then sponsored an online
pretest to further assess its potential. The online pretest and subsequent telephone debriefings with nine
PIs whose small businesses had won Phase II awards prompted further refinements to planned
approaches. The final survey questionnaire used in 2002 incorporated those changes and clarifications. It
also conformed to the Paperwork Reduction Act standards to minimize respondent burden. Appendix A,
Evaluation Framework, shows both the framework and a map of survey questionnaire items to the NIH
SBIR program objectives.
In 2002, the NIH implemented the first National Survey to Assess the Outcomes of the NIH SBIR
Program. This survey was a census of all small business awardees that had been awarded NIH SBIR
Page 1-2

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Phase II grants in FYs 1992-2001. Each awardee response described experience on one Phase II award.
For the 32 percent of the businesses that received more than one Phase II award during the study period,
the survey focus was a single award chosen through random selection. Due to vigorous follow-up effort to
maximize survey participation, 758 of the 905 eligible awardee companies surveyed completed the
questionnaire, yielding an 85 percent response rate.
Survey results indicated that NIH SBIR awardees stimulate technological innovation and contribute
significantly to the body of scientific knowledge related to health issues. In addition, NIH SBIRsupported technologies generate substantial commercial activity. Respondents judged the NIH SBIR
program to be quite valuable, allowing them to further develop their businesses and pursue high-risk
ideas. The primary study recommendation was to continue building on this systematic assessment by
maintaining a performance monitoring system to document NIH SBIR program progress toward
achieving performance outcome targets. 1
1.1.2

Subsequent Performance Monitoring and Evaluation

Since 2003, the NIH SBIR/STTR program has been developing a Performance Outcomes and Data
System (PODS) to respond to the 2002 survey recommendation about monitoring SBIR performance.
PODS affords authorized NIH personnel easy access via the NIH Intranet to SBIR awardee information,
the most recently reported data measuring award outcomes, and user-defined lists and tables of awardee
information and outcome measures.
PODS contains data from the 2002 survey and from subsequent voluntary updates by survey respondents.
Seventy-four percent of the original survey respondents—or 568 awardees—have participated in at least
one of five voluntary update opportunities since the 2002 survey. Considering that about 280 of the
original 758 awardee respondents had either discontinued their projects or were no longer in business by
2007, the response is especially impressive.
NIH relies on PODS updates to document the continued achievements of SBIR awardees (FYs 19922001) over time:
• The number of awardees with FDA-approved projects increased 51 percent, from 59 in 2002 to
89 in March 2007.
• The estimated cumulative sales to date achieved by awardees’ commercialized projects increased
over 200 percent, from $821 million in 2002 to $1.95 billion in March 2007.
• The number of awardees receiving additional non-SBIR funding or capital increased 33 percent,
from 281 in 2002 to 375 in March 2007.

“National Survey to Evaluate the NIH SBIR Program”; available from
http://grants.nih.gov/grants/funding/sbir_report_2003_07.pdf; Internet; accessed 10 December 2008.

Page 1-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Building on the results of NIH’s 2002 survey, the National Research Council (NRC), an arm of the National Academy of Sciences, expanded the NIH assessment to include all agencies that participated in the
SBIR program. 2 This evaluation included an online survey of a probability sample of FY 1992 through
2001 awardees that incorporated many of the same questions as the 2002 NIH instrument; a survey of
Phase I awardees; case studies of awardee companies; and interviews with NIH staff, grantees, and other
stakeholders. 3 The NRC survey conclusions mirrored those of the 2002 NIH survey. NRC commended
the NIH SBIR program, both for its meaningful achievements and also for its evaluation efforts. The
report further recommended that the SBIR program receive more funding for administration, evaluation,
and testing of new initiatives to enhance commercialization and program operations.

1.2

Methodology for the 2008 NIH Survey

By 2006, the NIH decided to sponsor a second national survey of Phase II SBIR awardees. The purpose
of this initiative was to describe the performance of a second group of awardees—those whose NIH SBIR
awards were made from FYs 2002-2006. Learning about the performance of these awardees would be a
logical follow on to the 2002 survey. It could enable program sponsors to add contact and performance
data about a new cohort to PODS. Doing so could improve the NIH SBIR program’s ability to assess
performance of more recent program activities.
The 2008 survey implementation mirrored that of the 2002 NIH survey implementation.
• Use the Evaluation Framework to develop the questionnaire to ensure that the data include
metrics for answering all study questions.
• Conduct a census of awardees rather than a survey, if the sample universe for the study period is
not too large.
• Update the contact information before fielding the survey to maximize respondent participation.
• Construct the survey to facilitate response and minimize respondent burden.
• Use information technology for implementing the survey online to reduce respondent burden.
• Maximize the response rate to the survey to minimize nonresponse bias.
• Clean and code the survey data to produce an accurate data file.
The following sections discuss the implementation of these survey principles and refer to sections in the
appendices that contain additional information.

Charles W. Wessner, ed. An Assessment of the Small Business Innovation Research Program. (Washington, D.C.: National
Research Council, 2007).

The NRC survey sampled individual awards rather than awardee companies. Of the 1,127 awards for which the PI was asked to
complete the questionnaire, 496 responded, resulting in a response rate of 44 percent.

Page 1-4

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

1.2.1

Questionnaire

Appendix B, Survey Instrument, contains a copy of the survey questionnaire. With the exception of very
minor revisions to the wording of several response categories and the addition of several options, the
2002 and 2008 survey questionnaires are the same. The minor modifications were based on responses to
several items in the 2002 survey for which more than a few respondents supplied the same specific
answers in the “Other” response category. These same responses became new categories in the 2008
survey questionnaire.
To encourage response, the questionnaire design enabled respondents to answer all items within 15 to 30
minutes, including any time needed to retrieve information or consult other sources. The survey
questionnaire was as precise as possible, used consistent terminology, and continually referenced the
randomly selected award that was the focus of the survey. The intent was to have respondents answer all
questions in terms of the “referenced SBIR award” and the “product, process, or service” planned and
developed under the supported “project.” Since selecting a response makes it easier for most respondents
to answer a question, the questionnaire tended to provide close-ended response categories. Sometimes it
was appropriate also to provide respondents with the opportunity to choose an “Other (please specify)”
open-ended response to encourage their answering a question when they did not wish to select a closedend response. In keeping with accepted guidelines for online surveys, 4 the questionnaire prompted
respondents to answer positively or negatively to each response for lists of optional answers.
1.2.2

Sampling

Like the 2002 NIH survey, the 2008 National Survey to Evaluate the NIH SBIR Program is a census of
all Phase II NIH SBIR awardee recipients, funded during the study period. This sample includes awards
from 23 NIH ICs, as shown in Exhibit 1-1 on the following page.

Don A. Dillman. Mail and Internet Surveys: The Tailored Design Method (New York: John Wiley and Sons, 2000), 398-399.

Page 1-5

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 1-1

NIH Institutes and Centers within the 2008 Survey Sample

Awarding NIH Institute or Center
Abbreviation
NCI
NCCAM
NCMHD
NCRR
NEI
NHGRI
NHLBI
NIA
NIAAA
NIAID
NIAMS
NIBIB
NICHD
NIDA
NIDCD
NIDCR
NIDDK
NIEHS
NIGMS
NIMH
NINDS
NINR
NLM

Title
National Cancer Institute
National Center for Complementary and Alternative Medicine
National Center on Minority Health and Health Disparities
National Center for Research Resources
National Eye Institute
National Human Genome Research Institute
National Heart, Lung, and Blood Institute
National Institute on Aging
National Institute on Alcohol Abuse and Alcoholism
National Institute of Allergy and Infectious Diseases Extramural Activities
National Institute of Arthritis and Musculoskeletal and Skin Diseases
National Institute of Biomedical Imaging and Bioengineering
National Institute of Child Health and Human Development
National Institute on Drug Abuse
National Institute on Deafness and Other Communication Disorders
National Institute of Dental and Craniofacial Research
National Institute of Diabetes and Digestive and Kidney Diseases
National Institute of Environmental Health Sciences
National Institute of General Medical Sciences
National Institute of Mental Health
National Institute of Neurological Disorders and Stroke
National Institute of Nursing Research
National Library of Medicine

Just as in 2002, awardee respondents answered survey questions about one randomly selected single
award. This sampling approach helped minimize respondent burden and provided a single, consistent
focus for respondents’ assessments.
The sampling approach uses these definitions:
• Unit of Study. The unit of study for the survey is the award within the awardee.
• Award. The award is a single NIH SBIR Phase II grant awarded from FYs 2002-2006.
• Awardee. The awardee is the small business that received the grant.
• Project. The project is the R&D undertaken and supported by the award.
• Respondents. The eligible survey respondents for each awardee, in order of desirability, are: (1)
the Principle Investigator (PI) on the SBIR award application, (2) the replacement PI, if the
original one was no longer employed by the awardee, and (3) the business official who signed for
the awardee company on the award application, if there was no replacement PI.
• Study Period. The study period included in this evaluation is FYs 2002-2006.

Page 1-6

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

• Stratification. There was no stratification of the sample by funding institute or IC (an initial plan
prior to the 2002 survey) because of the decision to conduct a census. Post-sample stratification
using various criteria, such as grouping the smaller ICs, is part of the data analysis.
• Usable Sample Units. Usable sample units are existing, nonduplicate awardee small businesses
located within the U.S.
• Eligible Respondents. Eligible respondents are employed by the awardee small business, reside
within the U.S., and are capable of interviewing in English.
1.2.3

Contact Database Update

NIH databases of contact information for awardees and respondents were from one to six years old, so a
contact database update was the initial phase of the survey implementation. The purpose was to: (1)
confirm that the small business still existed, (2) learn the status of the business and resolve any name
changes resulting from mergers and acquisitions, (3) update the business’s email address and telephone
number, (4) determine the eligible respondent, and (5) update the respondent’s business email address and
telephone number.
An initial contact email message addressed to the PI on the SBIR award application asked the PI to update
information about the status of the awardee business and its contact information using an online form. If
the contact information was correct, the PI could merely reply to the email message with the word
“COMPLETE” in the subject or body of the email reply.
For nonresponders to the update request, contact methods included emails, telephone calls, web searches,
and research using databases available through the NIH eRA Commons. Contact activities attempted to
reach each nonresponder and obtain: (1) updated email contact information for the awardee business, (2)
the name and email address for the eligible respondent, and (3) when appropriate, information that the
awardee business was no longer in existence (and could therefore be classified as unusable sample).
Appendix C, Email Messages and Telephone Follow-up Briefing, contains copies of the update email
message, the online response form, and all other communications used during survey fielding and followup with nonrespondents.
The result of the update phase was a database of 1,037 unique awardee small businesses with current (or
presumed current) contact information for the 89 percent (928) of the awardees deemed usable sample
units, inasmuch as they were still operating and located in the U.S. Nearly all of these small businesses
(918) could be deemed eligible sample. Chapter 2 presents details about analyses of usability and
eligibility aspects of the quality of the survey sample data.
Appendix D, Response Rate—Final Disposition of Sample, describes the final disposition of sample. For
additional information about the response rate, sample disposition, and online calculator, please see this
appendix. It also includes a final disposition of the sample.
Appendix E, Survey Sample, contains more detailed information about the SBIR awardee database, the
selection procedure, and possible limitations to the database. While limitations conceivably could result
from the random procedure used to select the single award for awardees with multiple awards,
comparison confirms that the randomly selected awards are indeed representative of the universe of
awards.

Page 1-7

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

1.2.4

Survey Implementation

Online access and use of the Internet are now common for the respondent population, so the survey was
administered online only. (The 2002 survey had offered online, mail, and telephone response options.)
Online implementation minimized respondent burden—for most respondents, it is easier and less time
consuming to click on the appropriate answer than to check it on paper, or to type in an open-ended
response than to write it out. Online surveys typically elicit quicker response than do paper ones, and
they reduce data entry errors and costs associated with key-entering data.
The automated survey process included security and confidentiality safeguards. The survey was
implemented using Secure Socket Layer (SSL) encryption technology. The access process required
respondents to input their user name and unique password before they could begin the survey.
Respondents could save their responses, leave the online survey, and return to the survey later to enter
more responses.
The online implementation allowed each respondent to complete only a single survey. Submitting the
survey stored the survey responses in a secure database.
All survey communications were sent using personalized email messages. Using email allows
prospective respondents to receive communications nearly instantaneously and to respond to the survey
merely by clicking the embedded link to the online implementation. The communications included the
following.
•
•
•
•
•
•
•
•
•

Initial update email to update contact information
Email verification message
Advance email about the upcoming survey
First cover letter email
Reminder/thank you email
Second cover letter email to nonresponders
Thank-you for promising to respond email
Incomplete email to those who started the online survey but did not submit it
Last call email to nonresponders from NIH SBIR/STTR Program Coordinator

All but the first two email messages included the survey link, the user name and password, and telephone
numbers to call for additional information about the survey or for assistance with any technical problems.
1.2.5

Survey Response

As in 2002, the 2008 survey had a goal of achieving participation from 80 percent of the usable and
eligible awardees. This is an extremely high response rate, which is typically achieved only when
respondents believe the survey has high interest and importance or when respondents receive other valued
incentives. Past NIH experience indicated that the 2008 survey’s awardee spokespersons would view this
survey as important and worthy of participation.
In addition to the multiple email communications itemized in Section 1.2.4, a professional telephone
center promoted survey response. Supervisors and interviewers at the center received training about the
background and goals of this assessment and how to administer a brief telephone script approved by NIH.

Page 1-8

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Interviewers made up to nine attempts (staggered across days and time periods) to contact all
nonresponders, update awardee and respondent contact information, and elicit a promise to complete the
online survey. After the telephone center elicited a promise to complete, all those who agreed to
complete an online survey received new email messages containing the survey link, user name, and password.
An online response rate calculator, accessible at any time by authorized NIH personnel, monitored the
survey response rate on a live, continuous basis. In addition to showing the current response rate, it
tracked the number of usable and eligible potential respondents, the number of completed surveys, and
the status of various incomplete and promised surveys.
The 2008 survey achieved a 78 percent response rate (719 respondents from the pool of 918 eligible
awardees). This rate is relatively high but not as high as the 85 percent rate achieved in the 2002 survey.
The lower response rate may be due to two factors. First, interviewers who contacted nonresponders were
not told to administer the survey on the telephone either on the spot or at a convenient appointment time
for the survey recipient. (In the 2002 survey, interviewers were told to administer the survey, and 11
percent of respondents completed telephone interviews.) Second, online surveys are no longer the
novelty that they were in 2002.
1.2.6

Data Cleaning and Coding

Minimal cleaning is required for data from an online survey because the logic used by the implementation
program controls the skipping of questions, allows input of only displayed response codes, and checks the
ranges of uncategorized numeric data. Online implementation minimizes errors and typos that can occur
from the manual input of data from paper questionnaires.
Two survey coding schemes facilitated data analysis:
• Coding to categorize the narrative responses to open-ended questions
• Construction of subgroups of survey participants to facilitate comparing responses among key
subgroups, using statistical criteria to identify differences 5
The number of respondents for an individual IC often was too small (i.e., less than 30 awardees) to allow
precise estimates of outcomes. Constructing two clusters of the small ICs made it possible to compare
these ICs’ performance in certain areas of interest to project sponsors. Assignment of ICs to one or the
other cluster depended on the type of products that were supported by the Phase II award. One cluster
comprised ICs where more than half of the sponsored projects’ product development primarily relied on
biological and physical sciences research (e.g., drugs, biologics, and genomics). The second cluster
included the remaining ICs. ICs in the second cluster were those whose projects were being developed by
behavioral, social science, or statistical research techniques (e.g., educational materials, psychological

The appropriate use of statistical tests requires that the data are robust—that is, that they do not violate the test’s assumptions.
Often, these assumptions involve having subgroups of sufficient size to avoid confounding normal sampling variability with
significant differences between the groups.

Page 1-9

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

assessments and interventions, and statistical software); ICs whose projects were fairly evenly distributed
among both types also were included in this cluster.
IC groups constructed for the 2002 and 2008 surveys are not comparable. One reason is that they use
different grouping schemes. While the 2008 groups are constructed on the basis of project type, the 2002
survey groups were constructed on the basis of number of awards made by the IC. Another difference is
that the 2008 survey describes performance of ICs that either did not sponsor SBIR awards in 2002
(National Center on Minority Health and Health Disparities and National Institute of Biomedical Imaging
and Bioengineering) or had no eligible respondents from ICs whose awardees responded to the 2002
survey (National Library of Medicine).
1.2.7

Analytic Strategies and Reporting Practices

The analysis compared responses for two categories of respondents.
• Subgroups of special interest to NIH. Subgroups of interest included fiscal year of award, IC
that sponsored the award, type of SBIR-supported product, and current status of the Phase II
project. For comparisons that involved two categorical variables, chi square tests determined if
there was a relationship between the two variables (e.g., type of product and current project
status). 6 For outcomes that were interval-level variables (such as number of publications) t-tests
and analyses of variance determined if differences between subgroups were statistically
significant.
• 2002 and 2008 survey populations. In reporting attainment of performance indices and
measures, the narrative describes performance of the entire 2002 and 2008 survey respondent
populations. Comparisons of other types of performance take account of the fact that the 2002
survey describes a ten-year award period (FYs 1992-2001), while the 2008 survey describes a
five-year award period (FYs 2002-2006). To construct more equivalent groups in terms of the
award period being described, these comparisons describe (1) only the 2002 survey participants
whose awards were made in the last five-year period, FYs 1997-2001 and (2) all 2008 survey
respondents.
These general practices are used in reporting the survey results.
• Descriptive statistics. In reporting descriptive statistics summaries for a major index and
associated measure in the Evaluation Framework, the values for means, medians, and standard
deviations are rounded to the nearest tenth. Data are summarized for each major index and
associated measure in the Evaluation Framework. For categorical variables, percentages are
rounded to the nearest percentage point; this may result in the percentages for all categories of a
variable not summing to 100 percent.

The chi-square test statistic only indicates that there is a statistically significant relationship between two variables; it does not
provide information about the nature of the relationship. To determine this, we used adjusted residuals, which compare the
frequencies that were observed with the frequencies that would be expected if the two variables were not related.

Page 1-10

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

• Key comparisons. In reporting key comparisons of respondent subgroups, results of the relevant
statistical tests are reported (1) in the notes to graphs and tables that present the result of
statistical analyses, and (2) in the text itself, when comparing other variables of interest.

1.3

Strengths and Limitations of the Evaluation

The 2008 survey collected information about a set of awardees that had not yet been surveyed—those
small business awardees that received SBIR Phase II grants from FYs 2002-2006. It generated a written
report and a standard data set. These products add to the NIH SBIR program’s body of knowledge for a
new set of awardees. They provide information that can facilitate understanding of program performance
over time.
One strength of the study described in this report is the relatively high 78 percent response rate. Another
positive aspect of the study is the fact that the 2002 and 2008 surveys were essentially identical, thus,
allowing comparison of the two studies and their results.
Certain limitations exist, of course. One major limitation is that all data on outcomes are based solely on
information reported by the awardees themselves. Another limitation is that it was not possible to make
valid comparisons of FYs 1997-2001 and FYs 2002-2006 outcomes for outcomes of interest that could
only be answered with data from two questions that were revised in 2008.
• Question 18 queried awardees about the current status of their SBIR-funded project. Ten percent
of 2008 respondents selected a response option, “on hold,” to describe the current status of their
SBIR-supported project. This option was not available to 2002 respondents. Because current
project status was used in conjunction with other questions about key outcomes (developing new
or improved products; contributing to increases in health knowledge, research tools, and health
promotion; and disseminating SBIR-supported technology and information to populations using
and receiving health and health care resources), comparisons of these outcomes between the two
cohorts of awardees were not possible.
• Question 23 asked for the dollar range of total cumulative sales through December 2007 for the
product developed under their Phase II award. The response option in the 2002 survey
(“$5,000,000-$49,999,999”) was divided into two separate categories in the 2008 survey
(“$5,000,000-$24,999,999” and “$25,000,000-$49,999,999”). This revision precluded making
comparisons of the two survey cohorts for estimated total and average cumulative sales in those
categories; estimates are derived using the midpoints of response categories, and the response
categories are now different for the two cohorts.
One other limitation is that small businesses that were liquidated, had merged with another company, or
had been acquired by another business were defined as unusable and thus not included in the survey
sample. If these events had indeed occurred for these unusable awardees, performance on some outcomes
(such as commercialization and sales) may have been even stronger than that described in this report.

Page 1-11

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

2. CHARACTERISTICS OF SURVEY RESPONDENTS
This chapter describes the characteristics of survey respondents. It presents information about the
usability and eligibility of the pool of potential respondents, awardees who received NIH SBIR grants
from FYs 2002-2006. Then it discusses characteristics of the respondent spokespersons who completed
the survey questionnaire, the awardee small businesses with which they are associated, and the products
whose development was supported by an NIH SBIR Phase II award from FYs 2002-2006.

2.1

Usability and Eligibility of Potential Respondent Small Businesses

NIH databases provided contact information about a pool of potential respondents—1,037 small
businesses that were awarded a Phase II SBIR grant from FYs 2002-2006. Approximately 89 percent of
the entire sample of potential respondents was deemed both usable and eligible.
Usability. Pre-survey searching strategies determined that 933 awardee businesses were potentially
usable for the survey because they were still operating and located in the U.S. One hundred and nine of
these awardees were unusable for one of these reasons:
• Awardee business reportedly had been liquidated or dissolved (22 or 2 percent)
• Awardee business reportedly had merged with or been acquired by other companies (45 or 4
percent)
• Valid contact information about awardee businesses was not available to the survey team, even
after extensive tracking efforts (37 or 4 percent)
An additional five awardees completed the survey questionnaire and were deemed unusable because they
reported having merged with or been bought by another company.
The usable sample for the 2008 survey was 928, or 89 percent of the total available sample of 1,037
awardees. 7 The geographic locations of usable and unusable awardees did not differ significantly.
Unusable awardee companies were significantly more likely to have been awarded a Phase II award in the
earlier years of the survey period than were usable awardees. Whereas the usability rates were between
81 and 82 percent for FYs 2002 and 2003, they ranged from 91 to 97 percent for FYs 2003 through 2006
(chi square statistic=45.6, df=4, p < 0.0001).
Usability rates differed somewhat among awardees of different Institutes and Centers (ICs). Between 94
and 98 percent of Phase II awardees funded by the National Center for Research Resources (NCRR),
National Institute on Aging (NIA), National Institute of Child and Human Development (NICHD), and
the National Institute of Digestive, Diabetic, and Kidney Diseases (NIDDK) were usable. The same was

Being purchased by or merging with another company can be regarded as a successful outcome for a Phase II awardee. The
decision to designate these companies as unusable is primarily based on the substantial difficulty in obtaining recent contact
information for both the company and the PI. Exclusion of these companies, however, always introduces the issue of potential
bias in the survey results. Given the high usability rate in the 2008 study, the degree of bias is most likely extremely small.

Page 2-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

true for the cluster of ICs that included the National Institute on Alcohol and Alcoholism (NIAAA), the
National Institute on Drug Abuse (NIDA), the National Institute of Nursing Research (NINR), and the
National Center for Minority Health Disparities (NCMHD).
The National Heart, Lung, and Blood Institute (NHLBI) and the National Institute on Allergy and
Infectious Diseases (NIAID) had the lowest usability rates (83 percent). For the remaining ICs, usability
rates fell between 87 and 91 percent. 8
Eligibility. Of the 928 usable awardee businesses, nearly all (918) were determined to be eligible. The
ten companies that were ineligible were so designated because the Principal Investigator (PI) was
unavailable (either due to death or departure from the company), and no other individual with adequate
knowledge to complete the questionnaire could be identified. 9

2.2

Spokesperson Characteristics

Exhibit 2-1 on the next page displays selected characteristics of individual spokesperson respondents who
completed the survey. Approximately 80 percent of the individuals who completed the 2008
questionnaire were the original PIs listed on the Phase II SBIR award application. Another 11 percent
were individuals who later had been designated as the replacement PI, typically as a result of the initial PI
leaving the company. As such, a total of 91 percent of respondents were directly responsible for the work
under the award at some point during the award period. The remaining 9 percent of respondents were
either the business official listed on the SBIR application or another contact at the company; these
individuals agreed to complete the survey because the original PI could not be located.
Survey respondents often performed many roles in the awardee business. Approximately 36 percent
functioned in only one capacity—as owners (13 percent), part of the managerial team (13 percent),
employees (9 percent), and, in a few cases, shareholders (3 percent). The largest group of respondents
(40 percent) reported serving in multiple roles whereby they not only owned the company but also served
in a managerial role and were shareholders. Substantially smaller percentages held two roles.

See Appendix D, Response Rate—Final Disposition of Sample.

Statistical comparisons were not performed, given the small number of ineligible respondents.

Page 2-2

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-1

Characteristics of Survey Respondents

Characteristic

Percent

Respondent role
Initial PI

574

Replacement PI

Business official

Other company contact

Owner only

Owner and manager

281

Management only

Manager and shareholder

Employee only

Employee and shareholder

Shareholder only

Other (e.g., former employee)

Relationship to awardee company
Owner

Owner and shareholder
Owner, manager, and shareholder
Part of management

Employee

Note. Information on respondent role was provided by 716 awardees; the corresponding
figure for relationship to the awardee company was 701. The percentages for respondent
roles are column percentages and may not sum to 100 percent due to rounding.
Most respondents were either the original PI or a replacement PI. Respondents tended to
report multiple relationships within the awardee small business; most typical were owner, and
part of management.

2.3

Awardee Small Businesses

This section compares characteristics of awardee small businesses that were eligible for the survey. It
then provides additional detail about the characteristics of eligible awardees who responded.
2.3.1

Nonrespondent and Respondent Awardee Comparison

Approximately 78 percent of eligible respondent awardee companies participated in the 2008 survey.
Exhibit 2-2 on page 2-4 shows fiscal year of award, sponsoring IC, and region of the country for
respondent and nonrespondent awardees.

Page 2-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-2

Selected Characteristics of Nonrespondent and Respondent

SBIR Awardee Small Businesses

Characteristic

Nonrespondents
(N = 199)

Respondents
(N = 719)

2002

122

2003

131

2004

135

2005

130

2006

201

154

110

NIA

NIAAA, NIDA, NINR, and NCMHD

NIAID

NICHD

NIDDK

NIGMS

Fiscal Year of Award

Sponsoring IC
NBIB, NCRR, NHGRI, NIAMS, NIDCD,
NIDCR, NIEHS, NCCAM, and NLM
NCI

NIMH

NINDS

NHLBI

Midwest

114

Northeast

203

South

174

West

228

U.S. Region

Note. The percentages are column percentages and may not sum to 100 percent due to rounding. The full titles of the
sponsoring ICs are shown in Exhibit 1-1 on page 1-6.
Non-respondents and respondents were similar in terms of the IC that sponsored the Phase II award, the fiscal year of
the award, and the geographic region in which the small business was located.

As shown by Exhibit 2-2, over one-fourth of the 719 respondent awardees received their Phase II award
in FY 2006. Between 17 and 19 percent of the companies each received their award in one of the earlier
years included in the evaluation (FYs 2002-2006).
The NCI supported the largest number of awards during this five-year period, funding 110 companies or
15 percent of all respondents. The NHLBI supported the second largest number, making Phase II awards
Page 2-4

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

to 96 companies (13 percent). Six ICs each funded another 6 to 8 percent of the companies that
responded to the survey; these included NIAID, National Institute of General Medical Sciences (NIGMS),
NIDDK, NICHD, NIA, and National Institute of Neurological Disorders and Stroke (NINDS). The
remaining awardee companies were funded by other ICs.
The largest percentage of respondent companies (32 percent) was located in the West, and the smallest
percentage (16 percent) in the Midwest. Twenty-eight percent were in the Northeast, and the remaining
24 percent were in the South.
Respondents and nonrespondents were reasonably similar in terms of which IC sponsored the award.
They also were similar in terms of the year in which the award was made and region of the country. This
suggests that the threat of nonresponse bias is very small or nonexistent for survey responses that are
strongly related to one or more of these three characteristics.
2.3.2

Awardee Small Business Respondents

Exhibit 2-3 shows a detailed picture of the geographical distribution of responding awardee small
businesses, identifying the number of respondents by state. California by far had the heaviest
concentration of respondents (132), followed by Massachusetts (72) and Maryland (51). Twenty-five
percent of the states had between one and two awardees, and two states (Alaska and West Virginia)
received no Phase II awards from FYs 2002-2006 and thus had no awardees that participated in the
survey.
EXHIBIT 2-3

Awardee Small Business Respondents by State

0
1

The largest numbers of awardee respondents were in California, Massachusetts, and Maryland. Five states had
only one respondent. Alaska and West Virginia received no awards during FYs 2002-2006 and therefore had no
respondents to the 2008 survey.
Page 2-5

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Twenty-four percent of awardee small businesses were founded in 2000 or later, and more than one-third
(37 percent) began operations between 1995 and 1999. Sixteen percent were founded during the early
1990s, 12 percent between 1985 and 1989, and 7 percent between 1980 and 1984. Only a small
percentage (5 percent) were established prior to 1980.
Looking at this another way, a significant percentage of awardee companies were starting operations at
the time of the award. Exhibit 2-4 shows the distribution of awardee companies by years in operation at
time of award. Awardee respondents to the 2008 survey reported a range of years in operation at the time
of award from 1-2 to 26 or more years.
EXHIBIT 2-4

Years in Operation at the Time of the SBIR Award

for Responding Small Businesses

Percent of awardee companies

0
1-2

Years in operation at tim e of aw ard

26 or
m ore

Note. The total number of awardee businesses that provided this information was 711. Percentages may not
sum to 100 percent due to rounding.
Phase II awardee businesses tended to be “young” in terms of how long they had been in operation at the time of
award. Approximately 30 percent had been operating for five years or less.

Overall, 54 percent of awardee businesses had been in operation for eight years or less at the time of the
Phase II award.
There were no significant differences among awardees in the average age when examined by fiscal year
of the award or type of product. On the other hand, significant differences did appear for different types
of awardee businesses. Biotechnology, pharmaceutical, diagnostic, medical devices, and computer
software/hardware companies were significantly likely to be “younger” than either instrumentation
companies or companies in the other category (i.e., healthcare, engineering, chemical, and environmental
businesses).

Page 2-6

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Company age at the time of the award was similar among the ICs, with one exception. Small businesses
that were supported by the NIAAA and NIDA had been in operation significantly longer than those
supported by other ICs. This is primarily a function of the fact that a larger percentage of NIAAA and
NIDA awards were in businesses that specialized in medical education and health promotion—a group
that also tended to have been in existence longer (mean of 11.1 years at the time of the award) than other
types of businesses.
When asked about their major field of business, the 612 companies that responded to this question represented a varied group of companies (see Exhibit 2-5). Two areas were predominant—biotechnology (24
percent) and medical devices (21 percent). Between five and nine percent of all businesses were in each
of six fields, including medical education and health promotion, informatics, research, and statistics,
pharmaceuticals, diagnostics, computer hardware and software (HW/SW), and instrumentation. Much
smaller percentages were in as healthcare, chemicals, and environmental fields.

Percent of awardee companies

EXHIBIT 2-5

Major Field of Business Reported by SBIR-Funded Awardees

for FYs 2002-2006

40
35
30
25

20
15
10

4
1

io
te
ch
M
no
ed
lo
ic
gy
al
de
In
vi
st
ce
ru
s
m
en
Ph
t
at
ar
io
m
n
ac
eu
t ic
al
D
s
ia
C
g
M
om
no
ed
In
f
s
ic
pu
or
tic
al
m
te
s
ed
at
rH
ic
uc
W
s
at
/S
an
io
W
d
n/
re
he
s
ea
al
th
rc
pr
h
om
ot
io
En
n
gi
ne
er
in
H
g
ea
lth
ca
re
Ch
em
En
ic
al
vi
s
ro
nm
en
ta
l
O
th
er

Major field of business
Note. The total number of awardee businesses that provided this information was 612. Percentages
may not sum to 100 percent due to rounding.
By far, biotechnology and medical devices were the most common major fields of business for Phase II
SBIR-funded awardee respondents.

Page 2-7

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Of the 719 awardee companies that completed the survey, 107 (15 percent) did not report their major field
of business. Including the missing responses in the analysis, did not noticeably change the percentages or
relative ranking of the different major fields, although the amount of missing data suggests caution in
interpreting these percentages. It is not clear why 15 percent of respondents failed to answer the question
about major field of business. Because of the amount of missing data, this variable is not used in analyses
of responses to the three key study questions.

2.4

SBIR-Supported Projects and Products

The SBIR program supported a diverse group of projects and their product-related activities. 10 The
majority (70 percent) of the projects involved the development of a totally new product, and one-fifth (20
percent) focused on improving an existing product. Another 7 percent sought to develop a combination
of new products, and 3 percent intended to create a new use for an existing product. These percentages
did not differ by the year of the award or the sponsoring IC. They also were nearly identical to those
found in the 2002 survey.
As shown by Exhibit 2-6 on the next page, when asked about the specific type of product that was the
focus of the award, 22 percent of the awardees reported having received SBIR funding to develop medical
devices. Computer hardware and software, research tools, diagnostic materials and devices, and drugs
each comprised between 12 and 14 percent of the projects.
Educational materials were identified by approximately 7 percent of the small businesses, as were
measurement tools. Noticeably smaller percentages (2 to 3 percent) of the companies funded work was in
genomics, non-drug therapeutics, environmental tools, and chemicals and chemical processes.

For the ease of the reader, henceforth we use the term “product” throughout this report to represent the products, processes, and
services that were planned by awardees.

Page 2-8

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-6

Type of Planned Product for SBIR Funded Awardees

Percent of awardee companies

35
30
25

20
14

5
3

0
c
l
g
ic
al
ls
ol
ic
gi
ce
ru
ol
oo
ic
to
ut
lo
ria
D
vi
om
to
lt
e
m
e
o
e
h
n
i
a
t
t
e
t
e
a
B
ap
rc
l/ d
en
G
al
ch
en
er
ea
te
m
lm
ic
ria
g
h
m
s
e
a
u
t
e
d
u
e
n
r
n
p
e
at
R
dr
ro
io
M
ug
su
m
om
nvi
at
dr
ea
o
C
n
c
it c
N
E
M
u
s
on
N
Ed
no
g
ia
D
ce
vi
e
d

/S
W
H
r

Type of product, process, or service

Note. These percentages are based on a total of 716 awardees who provided information on type of product.
Percentages may not sum to 100 percent due to rounding. In addition, a very small percentage (0.1 percent) of
the awards planned a product other than the ones identified.
Phase II awards sponsored a diverse group of products. The most common were medical devices. Five types
of products—medical devices, computer hardware and software, research tools, diagnostic materials/devices,
and drugs—accounted for nearly three-quarters (73 percent) of all planned products.

Among the different types of planned products, there were differences in whether the project sought to
develop one or more new products or improve an existing product. As Exhibit 2-7 shows (on the next
page) projects that focused on drugs most often described their goal as developing a “totally new” product
(90 percent). This also was the case for companies working on biologics (77 percent), educational
materials (73 percent), and medical devices (71 percent). In fact, regardless of the type of technological
innovation, at least 50 percent or more of the companies labeled their product as “totally new,” with one
exception. Companies whose award was for a non-drug chemical or chemical process were more likely
to see their intended product as an improvement to an already existing product.

Page 2-9

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-7

Technological Innovations by Type of Planned Product

Drug
Medical device
Biologic

Type of product

Genomic
Diagnostic
Non-drug therapeutic
Research tool
Measurement tool
Educational materials
Computer HW/SW
Environmental, ergonomic, or assistive tool
Chemical or chemical process
0

100

Percent of awardee companies
Totally new product

Improvement to an existing product

Combination of products or other

New use for an existing product

Note. The total number of awardee businesses that provided this information was 715. For each type of product,
the percentages may not sum to 100 percent due to rounding. The relationship between the type of technological
innovation and the planned product was statistically significant (chi-square statistic=69.8, df = 33, p < 0.0002).
Most product types were described as “totally new” at least 50 percent of the time. The one exception was
products involving chemicals or chemical processes, which were most frequently described (46 percent) as
supporting improvements to an existing product.

Overall, 41 percent of the companies reported that their SBIR-funded product was currently in the
development phase at the time of the survey. Another 20 percent indicated that it was being
commercialized, and 22 percent stated that it was being used by the target population(s). Ten percent said
that the product was “on hold,” awaiting the receipt of additional funding or inactive for another reason.
Eight percent of the companies had actually discontinued work on the project. (Numbers total more than
100 percent due to rounding).
Not surprisingly, the time that had elapsed since initial receipt of SBIR Phase II funding was significantly
related to the current status of the project (see Exhibit 2-8 on the next page). Projects funded in FY 2006,
having been underway for only about two years, were most likely to still be in the development stage (58
percent) and least likely to be discontinued (1 percent). However, only 18 percent of the projects funded
in FY 2002, six years after the start of the award, were still under development, and 14 percent had been
discontinued. Similarly, the percentage of projects whose products were currently in use by the target
Page 2-10

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

population was reasonably small for FY 2006 awards (12 percent). This figure tripled for projects that
were first funded in FY 2002 (36 percent).
EXHIBIT 2-8

Current Project Status by Fiscal Year of Award

100

P ercent of projects

43
40

18 19

18
13 14

21
17
13

21 22
15

11
8

0
2002

2003

2004

2005

2006

Fiscal year of award
Under development

Being commercialized

In use by target population

On hold

Discontinued

Note. The total N of respondents was 716. For each fiscal year of the award, percentages may not sum to 100 percent due
to rounding. Fiscal year of the award and current project status were significantly related (chi-square statistic=99.4, df =16,
p < 0.001).
The longer the elapsed time since the start date of the award, the more likely it was that projects were under development.
Projects with relatively longer elapsed time, since the start date of the award were more likely to be in use by the target
population(s) than were projects funded in FY 2006.

Another factor affecting project status was the type of product being developed. As shown in Exhibit 2-9,
on the next page, awardees reported that the status of their SBIR product was either (1) under
development, (2) in the commercialization stage, (3) in use by the target population; (4) “on hold,” or (5)
discontinued. Environmental, ergonomic, and assistive tools, along with genomics, were significantly
more likely than other product types to be in use at the time of the survey. Drugs and medical devices
were significantly more likely to be under development.
Educational materials were most likely to be in the commercialization stage (41 percent) at the time of the
survey. Drugs (4 percent) and chemicals (8 percent) were least likely to be in the commercialization
stage. Chemical products were said to have been discontinued more often (25 percent) than other types of
products.

Page 2-11

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

100

Percent of projects

EXHIBIT 2-9
Current Project Status by Type of Planned Product

4
0

Drugs (n=77)

20
15

8 7

Medical devices
(n=160)

20 20

Biologics (n=40)

14 15 14

13
7

Genomics (n=15)

Diagnostics (n=86)

7 7

Non-drug
therapeutics (n=14)

7 7

Research tools
(n=97)

Type of planned product

Percent of projects

Under development

In commercialization stage

In use by the target population(s)

"On hold"

Discontinued

100

26 26 24

27 23 32

8
11

31
8

Measurement tools
(n=50)

Educational materials
(n=51)

Computer HW/SW
(n=101)

Environmental,
ergonomic, or
assistive tools (n=12)

Chemicals and

chemical processes

(n=12)

Type of planned product

Under development

In commercialization stage

In use by the target population(s)

"On hold"

Discontinued

Note. A total of 715 companies provided this information. For each type of product, percentages may not sum to 100 percent due to rounding. Current project status
and type of planned product were significantly associated (chi-square statistic=145, df = 27, p < 0.0001).
The type of product was significantly related to current project status. Environmental, ergonomic, and assistive tools, along with genomics, were significantly more likely
to be already in use at the time of the survey. Drugs and medical devices were significantly more likely to be under development.

Page 2-12

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Finally, differences in current project status surfaced, depending on the IC that sponsored the Phase II
award (see Exhibit 2-10 on the next page). The National Institute of Mental Health (NIMH), the NICHD,
and the cluster of ICs that included the NIAAA, the National Institute on Drug Abuse (NIDA), NINR,
and NCMHD were the most likely to have their products in use at the time of the survey. Nearly onethird of these ICs’ awardees reported that their SBIR-funded project had been completed and their
product made available to the respective target population(s). These percentages were significantly larger
than were those for projects sponsored by the NCI, NHLBI, NIAID, NIGMS, and NIDDK, which ranged
from 14 to 17 percent. Such differences might be expected: the ICs with lower percentages are more
likely to support projects to develop drugs and medical devices—two product types that take the longest
to reach the market because of such factors as the need for FDA approval.
The discontinued rates for Phase II projects were typically between 8 and 10 percent. A significantly
lower rate occurred for the NIGMS when compared with that for all other ICs, excluding the NIMH,
combined (chi-square statistic=5.33, df=1, p < 0.03). Although the NIMH also experienced a visibly
lower rate (3 percent), this difference was not significantly different.
Although the percentage discontinued SBIR-funded projects was only 8 percent overall, it is useful to
explore the extent to which these projects shared any distinctive characteristics from those of projects that
were still ongoing. Exhibit 2-11 on page 2-15 shows the various reasons for abandoning work on the
project for the 54 discontinued projects. As shown, 59 percent of the awardees reported that there was
insufficient funding for the planned work. The second most frequent reason cited by 44 percent of
awardees was that the business itself had shifted priorities. Between 24 and 30 percent cited problems
involving too high a level of risk, the non-competitive nature of the product, and a market demand that
was too small. Smaller percentages believed that the project was abandoned at least in part because of the
PI’s departure from the company, the inability to obtain FDA approval, or that products were licensed to
another company.

Page 2-13

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-10

Current Project Status by NIH IC Sponsor

Current Project Status
NIH IC Sponsor

Total
N

Under
development

Under
commercialization

In use

“On hold”

Discontinued

NBIB, NCRR,
NHGRI, NIAMS,
NIDCD, NIDCR,
NIEHS, NCCAM
and NLM

154

NCI

109

NHLBI

NIAID

NIGMS

NIDDK

NICHD

NIA

NINDS

NIMH

NIAAA, NIDA,
NINR, and
NCMHD

Note. The total number of respondents was 716. Percentages are row percentages and may not sum to 100 percent due to rounding. Sponsoring
IC and project status (under development, under commercialization, in use, and “on hold” or discontinued) were significantly associated (chi-square
statistic= 57.6, df = 33, p < 0.005). The full titles of the sponsoring ICs are shown in Exhibit 1-1 on page 1-6.

There were significant differences in the current project status of awardee companies, depending on the IC that funded the project. The NIMH, the
NICHD, and the cluster of ICs that included the NIAAA, NIDA, NINR, and NCMHD were the most likely to have their products in use at the time of
the survey. Compared to all other ICs, the NIGMS, NIMH, and NIA were significantly less likely to have any discontinued or “on hold” projects.

Page 2-14

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-11

Reasons for Discontinued Projects

Percent of discontinued projects

100

80
59
60
44
40

24
17

th
er

ce
ns
ed

an
ot

FD
A

ap
pr
ov
he
al
rc
om
pa
ny

fa
ile
d

ef
t

Id
ea

PI
l

In
su
C
ffi
om
ci
en
pa
tf
ny
un
sh
di
ift
ng
ed
pr
Le
io
ve
ri
Pr
lo
tie
od
fr
s
uc
is
k
tw
to
as
M
o
ar
hi
n
ke
ot
gh
td
co
em
m
pe
an
tit
d
iv
w
e
as
to
o
sm
al
l

Reason for discontinuing project
Note. These percentages are based on a total of 54 respondents who provided reasons as to why their SBIR-funded
project was discontinued. Percentages do not sum to 100 percent, given that individuals could indicate multiple
reasons.
The most common reported reason for discontinuing work on an SBIR-funded project was insufficient funding (59
percent). A shift in the priorities of the company also was mentioned as a contributing factor by 44 percent of
respondents). Thirty percent indicated that the level of risk was too high and about one-quarter (24 and 26 percent)
believed the demand was too small or the product was not competitive.

Exhibit 2-12, on the next page shows that the relationship between product characteristics and
discontinuing work was only marginally significant (p < 0.06). Whereas 7 percent of companies
developing new products reported discontinuing work on the SBIR-funded project, this was true of 11
percent of companies that were working on improvements to or new uses for existing products.
The decision to discontinue a project was significantly related to receipt of additional SBIR awards. As
shown by Exhibit 2-12, companies that had no related SBIR awards were nearly twice as likely as those
that had competed successfully for these awards to have discontinued their project (10 versus 6 percent).

Page 2-15

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 2-12

Discontinued Projects by Product Characteristic

and Awardee’s Receipt of Additional SBIR Awards

Project was discontinued
Yes

No
%

Product characteristic
Developing one or more new products

510

Improving or finding new use for an existing product

146

Yes

385

261

Receipt of additional SBIR awards

Note. A total of 710 awardees provided information on both current project status and product characteristic.
The corresponding number for receipt of additional SBIR awards and project status was 699. Percentages are
row percentages. The relationship between product characteristic and project status was only marginally
significant (chi-square statistic=3.45, df=1, p < 0.06) whereas the relationship between receipt of additional
SBIR awards and project status was statistically significant (chi-square statistic=5.29, df=1, p < 0.02).
Companies that received additional SBIR Phase I or Phase II awards were significantly less likely to have
abandoned work on their Phase II project then were small businesses that had not competed successfully for
such additional awards.

Seldom did awardees attribute only one reason to their decision to abandon work on their product. Only
24 percent of this group cited only one factor, 33 percent identified two reasons, and 43 percent listed
three or more motivations. Certain rationales also tended to occur jointly. The belief that a product was
not sufficiently competitive was significantly related to also viewing the level of risk as too high (the
correlation between these two variables was 0.36, p < 0.008). Having insufficient funding also was
significantly correlated with the departure of the PI (the correlation was 0.27, p < 0.04). In both cases,
although these correlations were statistically significant, they were fairly modest in size.

Page 2-16

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

3. STIMULATING TECHNOLOGICAL INNOVATION
This chapter presents findings about attainment of the first NIH SBIR program goal, stimulating
technological innovation. It begins with a summary of findings about goal attainment. Then it presents
analyses of each related index and overall performance of SBIR awardees over time.

3.1

NIH SBIR Awardees Stimulated Technological Innovation

Exhibit 3-1 summarizes the survey findings about awardee outcomes for stimulating technological
innovation. Phase II awardees reported exceeding specified levels for both indices in the Evaluation
Framework—developing products in support of the NIH mission and receiving additional awards that
relate to the core technology.
EXHIBIT 3-1

Summary of Outcomes Associated with Stimulating Technological Innovation

Goal 1
Stimulate technological innovation

Index 1.1
Regardless of sales, 40 percent or more of
NIH SBIR awardees develop products in
support of the NIH mission

Measure 1.1.1

82 percent of the SBIR-supported

projects that responded produce new

or improved products in health-related

fields

Index 1.2

10 percent of NIH SBIR awardees receive

additional Phase I or Phase II awards that

relate to the core technology

Measure 1.2.1

57 percent of the responding awardees have

received additional Phase I or II awards that

are related to the core technology supported

by their Phase II award. This includes 1,615

awards.

Measure 1.1.2

53 percent of the SBIR supported

projects that responded have

published one or more technical

articles and have produced 1,397

technical publications

Measure 1.1.3

31 percent of the SBIR supported

projects that responded have been

granted one or more patents for a total

of 586 patents.

The next sections describe awardees’ reported performance in attaining each performance index
benchmark at the time of survey measurement.

Page 3-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

3.2

Performance on NIH SBIR Product Development Targets

Survey respondents exceeded the NIH SBIR performance target expressed by Index 1.1, which is
“Regardless of sales, 40 percent or more of NIH SBIR awardees develop products in support of the NIH
mission.” This section describes performance on the three related measures from the NIH SBIR program
Evaluation Framework.
3.2.1

New or Improved Products in Health-Related Fields

Measure 1.1.1 describes the number of SBIR-supported projects that produce new or improved products
in health-related fields, which is one indicator of alignment with the NIH mission. The number of
respondents who said they are developing and commercializing new or improved products in healthrelated fields is 587 (82 percent of 719 respondents). These included 290 projects with products under
development, 143 projects in the midst of the commercialization stage, and 154 projects that have made
their products available for commercial use.
3.2.2

Technical Articles and Related Activities in Support of the NIH Mission

Measure 1.1.2 describes the number of technical articles generated by NIH SBIR projects. This measure
provides important information about mission support because progress toward stimulating technological
innovation is accomplished when contributions are made to the body of scientific and technological
knowledge. These contributions include publishing articles in peer-reviewed scientific journals and
authoring or co-authoring articles that appear in industry or technical publications.
In addition to querying survey respondents about publications, the questionnaire also asked about other
indicators of contribution to the body of scientific and technological knowledge in support of the NIH
mission: conference presentations and honorary awards. Conference presentations can be precursors to
formal publications; they promote public awareness of the SBIR-supported product, and in turn increase
the product’s visibility and possibly its quality. Visibility and quality benefits also can accrue when
project personnel receive honorary awards that relate to their SBIR project activities.
As shown by Exhibit 3-2 (on the next page), slightly more than half (53 percent) of awardee respondents
indicated that they had generated one or more publications that were associated with the project, and nearly
three-quarters (73 percent) made one or more conference presentations. Thirteen percent received at least one
award.

Page 3-2

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 3-2
Summary of Presentations, Publications, and Awards
Associated with SBIR-Supported Projects
Conference
Presentations

Publications

Awards

Awardee companies
Number

519

375

Percent

Total

2,771

1,397

148

Mean

0.9

2.0

0.2

Standard deviation

7.4

4.9

0.7

Median

2.0

1.0

0.0

Items

Note. The total number of awardees who responded to survey queries about these
items was 709.
Nearly three-quarters (73 percent) of all awardees reported giving one or more
conference presentations. Fifty-three percent reported producing one or more
publications. Approximately 13 percent reported receiving at least one award.

The NIH SBIR program awards generated 1,397 publications, 2,771 conference presentations, and 148
awards. Median numbers per reporting awardee were 1, 2, and 0, respectively.
Whether awardees reported publications, presentations, and awards related to the SBIR-funded project
was significantly related to the current status of the project. Exhibit 3-3 on the following page shows that
larger percentages of companies whose products were being commercialized or in use by the target
population(s) indicated having one or more publications, presentations, and awards. Fifty-eight percent of
awardees who were in the midst of commercializing their product had published at least one technical
article. Sixty-nine percent of awardees whose product was commercially available and in use published
at least one technical article. The corresponding percentage was 46 percent for awardees who were still
involved in product development.
Regardless of project status, a sizable percentage of SBIR projects make contributions to the scientific
knowledge base. Noticeable levels of activity in authoring technical articles occurred even for projects
that were “on hold” or discontinued (45 and 42 percent, respectively).

Page 3-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Percent of awardee companies

EXHIBIT 3-3

Awardees with Presentations, Publications, and Awards

by the Current Status of their SBIR-Supported Project

100

82
84

18
21

0
Presentations

Publications

Awards

Type of contribution
Under developm ent

Com m ercialization stage

On hold

Discontinued

In use by target population(s)

Note. Current project status was significantly related to all three ways of contributing to the knowledge base. For
publications, the chi-square was 26.3 (df = 4, p < 0.001). For presentations, the chi-square statistic was 28.8 (df
= 4, p < 0.0001), and for awards, it was 19.1 (df = 4, p < 0.0007). The total number of awardees who responded
to survey queries about these outcomes was 709.
Awardees that were commercializing their product or had made it already available were significantly more likely
to have published one or more technical articles and to have received one or more awards than were awardees
whose projects were “on hold” or discontinued. Awardees whose projects were “on hold” or discontinued had
published at least one technical article (45 and 42 percent, respectively). Even higher percentages of these more
“inactive” projects reported making one or more conference presentations.

As also shown by Exhibit 3-3, a pattern similar to that observed for publications occurred with regard to
conference presentations and awards. The strongest performance was exhibited by awardees who had moved
past the product development stage and had ongoing or completed projects.
Overall, when compared to publications, the percent of awardees with at least one product-related conference
presentation was noticeably higher, and the percent with one or more awards was dramatically smaller,
regardless of the current status of the SBIR-funded project.
The elapsed time between the receipt of SBIR Phase II funds was significantly related to publications (chisquare statistic = 27.2, df = 4, p < 0.0001). Essentially, the more time that had elapsed, the more likely it was
for a company to have published one or more scientific articles. Whereas 68 percent of FY 2002 awardees
indicated one or more publications, the corresponding percentages for FYs 2003-2006 awardees were 60, 57,
44, and 42 percent, respectively. In contrast, there was no relationship between when the small business
received the SBIR Phase II award and whether it had since presented at a conference or received an honorary
award that was project-related.

Page 3-4

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Having one or more publications, presentations, or awards was not significantly related to the
Institute/Center (IC) groupings. Conference presentations and awards were not associated with the type
of product being sponsored by the Phase II award.
Publications and type of product were related. Exhibit 3-4 shows awardees that were working on non-drug
therapeutics were the most likely to have one or more publications (86 percent). Between 61 and 66 percent of
companies working on measurement and assessment tools, diagnostic materials and devices, and biologics,
and 53 to 58 percent of awardees developing new or improved drugs, genomics, research tools, and
environmental tools reported that they had produced one or more publications. Nearly half of the awardees
receiving SBIR support for medical devices, software and hardware, and chemical products reported having
one or more publications. Awardees planning on creating educational materials were the least likely to have
published, with 35 percent reporting having one or more publications related to the award.
EXHIBIT 3-4

Awardees with Publications, Presentations, and Awards

by Type of SBIR-Supported Product

Non-drug therapeutics (n=14)

Type of product

66
66
62

15
58

8
56

Drugs (n=75)

Research tools (n=96)

100

86
86

Diagnostics (n=86)

12
53

7
47

Medical devices (n=157)

19
46

Chem icals (n=13)

Softw are or hardw are (n=101)

Biologics (n=39)

Genom ics (n=15)

Percent
50
60

Measurem ent tools (n=50)

Environm ental tools (n=12)

31
46
10

67
69
69

Educational m aterials (n=51)

Publications

Presentations

Aw ards

Note. Type of product and publication activity were significantly related (chi-square statistic=24.1, df = 11, p < 0.01). For
presentations, the relationship with product type was only marginally significant (chi-square statistic=17.6, df = 11, p < 0.09). The
total number of awardees who responded to survey queries about these outcomes was 709.
Awardees with projects developing new or improving existing non-drug therapeutics were significantly more likely to have
published one or more technical articles that were related to their Phase II project than were awardees working on other types of
products. Awardees developing non-drug therapeutic products, chemicals or chemical processes also were significantly more
likely to have received recognition in terms of an award. Regardless of the type of product, large majorities of awardees (between
66 and 87 percent) had presented at one or more scientific conferences.

Page 3-5

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

3.2.3

Patents and Other Indicators of Stimulating Technological Innovation

Measure 1.1.3 quantifies patents, another indicator of the degree to which NIH SBIR projects stimulate
technological innovation in support of the agency’s mission. The survey provided information on other
indicators of technological innovation stimulus as well—copyrights and trademarks.
As shown by Exhibit 3-5, 31 percent of survey respondents received one or more patents associated with
their SBIR Phase II project. This figure is consistent with the earlier findings of the National Research
Council (NRC) (33 percent). 11 Another 41 percent had a pending patent application; 18 percent reported
having at least one copyright; and 27 percent reported having obtained at least one trademark.
EXHIBIT 3-5

Awardees’ Patents, Copyrights, and Trademarks

Patents

Pending Patent
Applications

Copyrights

Trademarks

Total

Awardee
companies
Number

217

289

127

191

491

Percent

31%

41%

18%

27%

69%

Total

586

614

517

306

2,023

Mean
Standard
deviation
Median

0.8

0.9

0.7

0.4

2.9

3.1

1.7

5.6

0.9

7.2

0.0

1.0

Items

Note. The Total column on the far right of this table reports the number and percent of awardees with one or more patents,
pending patent applications, copyrights, and trademarks. It also reports the sum of these four items and the mean and
median number per project. The number of awardees who responded to survey queries about these outcomes was 709.
Nearly one-third of awardees reported having obtained one or more patents for their SBIR-supported project. Whether
holding a patent or not, 41 percent had a patent application pending. Eighteen percent of the projects reported having one
or more copyrights, and 27 percent reported at least one trademark associated with their SBIR product. The majority (69
percent) had at least one patent, pending patent application, copyright, or trademark related to their SBIR project.

Across all 709 responding awardees, SBIR-funded work resulted in 586 patents, 614 pending patent
applications, 517 copyrights, and 306 trademarks. Although the average and median numbers of these
items per awardee are either 0 or less than 1, it is important to keep mind that patents, copyrights, and
trademarks are not relevant outcomes for all awardees. Failure to generate these outcomes may be due
partly to the current status of the project, or to the fact that some types of products may have no patent,
copyright, or trademark potential.

National Research Council, 2007. An Assessment of the Small Business Innovation Research Program. Charles W. Wessner,
ed. Washington, DC: National Academy Press.

Page 3-6

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Exhibit 3-6 shows that the type of SBIR-funded product and receipt of one or more patents, copyrights,
and trademarks are related. For example, whereas patents were granted to 48 percent of projects that
involved the development and improvement of drugs, very few such projects reported having obtained
trademarks (12 percent), and none had obtained a copyright. In contrast, no educational materials
products were given patents, but 69 percent had one or more copyrights—the largest percentage among
different product types for this outcome.
EXHIBIT 3-6

Awardees with One or More Patents, Copyrights, and Trademarks

by Type of SBIR-Supported Product

Non-drug therapeutics (n=14)

Typ e of pr oduc t

15
58

8
56
11

Research tools (n=96)

12
53

7
47

Medical devices (n=157)

100

86
86

Drugs (n=75)

19
46

Chemicals (n=13)

Educational materials (n=51)

66
66

Diagnostics (n=86)

Software or hardware (n=101)

Percent
50

Biologics (n=39)

Genomics (n=15)

Measurement tools (n=50)

Environmental tools (n=12)

31
46
10

67
69
69

Publications

Presentations

Awards

Note. Type of product and having one or more SBIR project-related patents were significantly associated (chi-square
statistic=76.4, df = 11, p < 0.0001). This also was true for copyrights and product type (chi-square statistic=180.9, df = 11,
p < 0.0001) as well as trademarks and product type (chi-square statistic=24.3, df = 11, p < 0.02. The total number of
awardees who responded to survey queries about each of these outcomes was 709.
The extent to which awardees had obtained at least one patent, copyright, or trademark was highly dependent on the type
of product supported by the award. Between 48 and 58 percent of environmental tools, biologics, and medical devices had
received at least one patent. Educational materials (69 percent) were most likely to have received copyrights. Non-drug
therapeutics (57 percent) were most likely to have received trademarks.

Page 3-7

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

As Exhibit 3-7 illustrates, the current status of the Phase II project was significantly related to all intellectual
property outcomes, with the exception of patents. The patterns of differences for pending patent applications,
copyrights, and trademarks were different. Copyrights were significantly more likely for projects where the
product was being commercialized or was actually available for use. Thirty-eight percent of these projects
reported receiving one or more copyrights related to the SBIR award, as compared to 9 percent of projects
where the product was still being developed, 4 percent of projects that were “on hold,” and 8 percent of
discontinued projects. The same was true for projects that had reached the commercialization stage; here, 26
percent indicated receipt of at least one copyright. The largest percentage of projects that possessed related
trademarks (38 percent) were those in the commercialization stage. The corresponding figures for products
under development, in use, “on hold”, or discontinued were 20, 22, 16, and 9 percent, respectively.
EXHIBIT 3-7

Reported Patents, Pending Patent Applications, Copyrights, and Trademarks

by Status of SBIR-Funded Project

100

P er cent of awardee companies

90
80
70
60
51
50
40
30

39 40
34

32
26

26
22

20
9

9
4

22
16
9

0
Patents

Pending Patents

Copyrights

Trademarks

Type of contribution
Under development

Commercialization stage

On hold

Discontinued

In use by target population(s)

Note. A total of 708 awardee small businesses provided this information. Current project status and having at

least one pending patent application were significantly related (chi square statistic=35.7, df=4, p < .0001, p <

0.03). The same was true for current project status and both copyrights (chi square statistic=77.3, df=4, p <

.0001) and trademarks (chi square statistic=44.5, df=4, p < .0001)

The current status of the project and having been granted at least one patent were unrelated. Projects whose

products were under development were significantly more likely to have at least one pending patent application.

Copyrights were significantly more likely for projects that were in the commercialization stage or whose products

were in use by the target population. Projects in the commercialization stage were significantly more likely to

have one or more trademarks.

Page 3-8

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Current status of project and receipt of one or more patents were unrelated. Fifty-one percent of projects that
were in the midst of product development reported pending patent applications. This percentage is
significantly higher than those for all other status categories.
The receipt of one or more patents was significantly related to the fiscal year of the award (chi-square
statistic=11.3, df=4, p < 0.03). Whereas 38 percent of awardees who received their Phase II award in FYs
2002 or 2003 reported having obtained one or more patents, this was true for 30 percent of FY 2004 awards,
24 percent of FY 2005 awards, and 25 percent of FY 2006 awards. Such results are not unexpected, given that
less time had elapsed for the more recently funded projects to have developed their product to the point where
a patent application is appropriate. In fact, the start date of the Phase II award and having one or more pending
patent applications also were significantly related (chi-square statistic=10.8, df=4, p < 0.03); here, the
percentage of pending applications was higher for more recently awarded projects (e.g., 48 percent of 2006
awardees versus 30 percent of FY 2002 awardees). In contrast, obtaining a copyright or getting a trademark
was not related to fiscal year of the award.

3.3

Performance on Receipt of Additional SBIR Awards

The second major index of degree of technological innovation identified by the Evaluation Framework
quantifies the number of awardees who received additional Phase I or Phase II SBIR awards related to the core
technology or product referenced in the survey. Exhibit 3-8 shows that 408 awardees (58 percent) who
responded to survey questions about this matter reported having competed successfully for one or more
additional Phase I or Phase II awards for products related to the supported project.
As also shown by Exhibit 3-8, a larger percentage of awardee companies obtained Phase I awards (58
percent) than Phase II awards (40 percent). This differential success is to be expected. Companies are
likely to have more than one idea or product under development at any given time, not all Phase I
awardees request Phase II funding, and not all Phase II funding requests are granted.
EXHIBIT 3-8
Additional Related Phase I and Phase II SBIR Awards

Phase I
Award

Phase II
Award

Either Phase I or
Phase II Award

Awardee Companies
Number with at least
one award
Percent

405
58

Phase I
Awards

283

408

Phase II
Awards

All Awards

Awards
Total

1,408

567

1,615

Mean

1.5

0.8

2.3

Standard deviation

1.4

3.4

Median

1.0

0.0

1.0

Note. A total of 699 awardee small businesses provided information on whether they had
received a Phase I or Phase II award, and 604 provided data on the actual number of
awards received.
Most awardees had successfully competed for one or more Phase I or Phase II awards
related to their SBIR-supported project.

Page 3-9

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Forty-two percent of awardee companies that competed successfully for additional SBIR awards also reported
success in obtaining non SBIR awards for their Phase II funded project. In contrast, only 28 percent of
awardees who had not competed successfully for additional SBIR awards reported successfully obtaining non
SBIR awards. This difference in performance was statistically significant (chi-square=14.1, df=1, p < 0.0002).

3.4

Comparison with 1997-2001 Phase II Awardees

Exhibit 3-9, on the next page, summarizes the performance of FYs 2002-2006 Phase II awardees on the
two indices for the goal of stimulating technological innovation. It also compares their performance with
that of the FYs 1997-2001 Phase II awardees who responded to the 2002 survey.
The FYs 2002-2006 Phase II awardees met or exceeded each of the two numerical indices identified by
the SBIR program. This also was true for FYs 1997-2001 awardees for the one index that could be
compared—receipt of additional SBIR Phase I or Phase II awards. 12
Further study is needed to determine reasons for the significant differences in performance that were
observed between the two cohorts:
• A decrease in the total number of technical articles authored by FYs 2002-2006 awardees as

compared to their FYs 1997-2001 counterparts (t-test statistic=2.69, df=544, p < 0.005).

• A decline for recent awardees in the percentage who obtained at least one patent for their SBIR
funded product (chi square statistic=9.7, df=1, p < 0.002).
• A decline in the percentage of FYs 2002-2006 awardees who published (chi square statistic=28.0,
df=1, p < 0.0001).
• A decline in the percentage of recent awardees with scientific conference presentations (chi

square statistic=5.4, df=1, p < 0.02), which also may be related to the lower percentage who

published.

As Ruhm and Link found for a sample of 1992-2001 Phase II awardees, involvement by faculty, graduate
students, and other academic personnel did distinguish awardees who commercialized their product from
those who did not. 13 This also may be true with regard to publications, given that academic institutions
are more likely to emphasize such outcomes. These types of variables could be explored in future studies.

As noted in Chapter 1, the question used to assess performance on the index involving the development of new or improved
products was not identical in the 2002 and 2008 surveys, preventing comparison between 1997-2001 and 2002-2006 awardees.

Albert N. Link and Christopher J. Ruhm. Bringing Science to Market: Commercializing from NIH SBIR Awards. (Working
Paper 14057). (Cambridge, MA: National Bureau of Economic Research, June 2008).

Page 3-10

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 3-9

Comparison of NIH SBIR Awardees’ Performance

in Stimulating Technological Innovation

Phase II Award Start
Performance Index
Performance Measure
1.1

Number of new or improved SBIR
supported products in health-related
fields

1.1.2
1.1.3

1.2.1

FYs 2002- 2006
Total
Percent
N
N

Whether or not sales have occurred, 40%
or more of NIH SBIR awardees produce
new or improved products, processes,
usages, and/or services in support of the
NIH mission

1.1.1

1.2

FYs 1997- 2001
Total
Percent
N
N

587

719

Number of technical articles on new or
improved SBIR-supported products

369

545

375

709

Number of patents for new or improved
SBIR-supported products

213

546

217

709

298

535

408

699

10% or more of NIH SBIR awardees receive
additional Phase I or Phase II awards that
relate to the core technology
Number and percent of awardees who
received additional related Phase I or
Phase II awards

Note. Performance indices and measures are posited by the Evaluation Framework (Appendix A). The FY 1997-2001 results are
based on analyses of this subset of NIH SBIR program 2002 survey data to allow comparisons for this report. Further study is
needed to determine reasons for apparent differences in performance. Differences in question wording between the 2002 and
2008 surveys did not allow comparison of performance for Performance Index 1.1.1.
Comparisons of performance for awardees that received SBIR awards between FYs 1997-2001 and those that received awards
between FYs 2002-2006 showed both similarities and differences in performance for performance indices 1.1 and 1.2 and
associated measures. For technical articles and patents, performance levels did differ. Significantly larger percentages of
awardees in the earlier cohort had published at least one technical article (68 percent) and obtained one or more patents (39
percent). The corresponding percentages for FY 2002-2006 awardees were 53 and 31 percent, respectively. Both cohorts far
exceeded the stated benchmark of 10 percent for receiving additional SBIR funding that was related to the core technology
supported by the Phase II award. Their level of performance also was quite similar—56 percent of FY 1997-2001 awardees and
58 percent of FY 2002-2006 awardees.

Page 3-11

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

4. INCREASED USE OF SMALL BUSINESS CONCERNS

This chapter presents findings about attainment of the second NIH SBIR program goal addressed by the
survey, using small business concerns to meet Federal research and development (R&D) needs. It opens
with a summary of findings about goal attainment. Then it presents analyses of each related index and
overall performance of SBIR awardees over time.

4.1

NIH SBIR Awardees Met Federal Research and Development Needs

Exhibit 4-1 displays the survey findings about awardee outcomes related to using small business concerns
to meet Federal R&D needs. Phase II awardees reported exceeding specified benchmarks for all three
related indices in the Evaluation Framework—contributing specified types of knowledge, obtaining and
disseminating health-related information, and expressing satisfaction with the usefulness of the NIH SBIR
program.
EXHIBIT 4-1

Summary of Outcomes Associated with

Using Small Business Concerns to Meet Federal Research and Development Needs

Goal 2
Increase the use of small business concerns to meet Federal R&D needs

Index 2.1

50 percent or more of NIH SBIR awardees make

contributions to knowledge in health promotion,

disease prevention, diagnosis, health care, and

amelioration and cure of disease

Index 2.2
50 percent or more of NIH SBIR awardees are able to
obtain and to disseminate health-related information

Measure 2.2.1

96 percent of awardee respondents give

high rankings to the usefulness of

outreach and informational services

Measure 2.1.1

82 percent of the responding awardees

contribute increases in health knowledge,

research tools, and education

Measure 2.2.2

82 percent of responding companies

anticipate or are disseminating SBIR-

supported technology and information to

populations using and receiving health

and health care resources

Index 2.3

50 percent or more of NIH SBIR awardees express

satisfaction with the usefulness of the NIH SBIR

program

Measure 2.3.1

74-90 percent of awardee respondents,

depending on the program component,

are completely or mostly satisfied with

the SBIR application, review, award, and

post-administration of the program.

Page 4-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Awardees’ reported performance in attaining each performance index at the time of survey measurement
is described in the following sections.

4.2

Performance on Contributions to Knowledge

Survey respondents reported having far exceeded the Index 2.1 benchmark that 50 percent or more of
awardees contribute to knowledge in health promotion, disease prevention, diagnosis, health care, and
amelioration and cure of disease. They provided strong evidence of having provided the increases in
health knowledge, research tools, and education specified by Measure 2.1.1.
To identify these contributions, the survey asked the awardees themselves to select which of 11 specific
medical, societal, and technological outcomes were associated with their funded project. This list was
developed by NIH personnel as those contributions most relevant to health promotion as well as
prevention, diagnosis, treatment, and cure of disease. Awardees were first asked to choose the outcomes
most relevant to their product; they were then to identify the one outcome that was most important.
Exhibit 4-2 lists the specific outcomes included in the 2008 survey instrument. As shown by the exhibit,
projects typically had multiple outcomes. In fact, only 11 percent of respondents identified only one of
the 11 outcomes listed; the median number of outcomes was four.
EXHIBIT 4-2

Medical, Societal, or Technological Outcomes of SBIR-Funded Projects

Treating disease or disability
Im proving research tools

Detecting disease or disability

Outcome

Preventing disease or disability

Diagnosing disease or disability

Reducing the cost of m edical care

Developing health inform ation for the general public

Developing inform ation for health professionals

Fostering new research collaborations

Training reesearch investigators

Other outcom es

36
62
25
39
54
28

8
0

11
20

100

Percent
Most im portant outcom e

Outcom es related to the project

Note. A total of 714 awardee small businesses provided information about whether each specific outcome applied to their SBIRfunded project. Data on the most important outcome were provided by 712 awardees.
SBIR-supported projects, in general, had multiple outcomes. Most outcomes were associated with health care, enhancing the
quality of research, or both. Respondents cited treating disease and disability and improving research tools as the most important
outcomes.

Page 4-2

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Most common were two outcomes associated with providing health care—reducing the cost of medical
care (62 percent) and treating disease or disability (53 percent)—and two aimed at enhancing the quality
of research—improving research tools (62 percent) and fostering new research collaborations (54
percent). The next most frequent outcomes all were targeted at health care issues; between 30 and 40
percent of awardee companies viewed disease prevention, detection, diagnosis and developing
information for health professionals as outcomes of their projects. Another research outcome—training
new investigators—was viewed as relevant by 28 percent of awardees.
When awardees were asked to identify the most important outcome, addressing healthcare and research
needs remained top priorities. Twenty-nine percent of SBIR-funded small businesses indicated that
treating disease and disability was the most important outcome associated with their project, and another
18 percent attached the same level of importance to improving research tools. Other outcomes were
deemed most important by smaller percentages of awardees. For example, reducing the cost of medical
care, although chosen as one outcome by 62 percent of respondents, was viewed as the most important
outcome by only 5 percent of the projects. Similarly, more than one half (54 percent) believed that
fostering research collaborations was an outcome of their SBIR award, but only a very small minority (1
percent) believed it to be the most important outcome. Overall, 92 percent believed that the most
important outcome for their SBIR-funded project was one of these healthcare or research outcomes.
It is interesting to note that significant differences repeatedly appeared in the relationship between the
current project status and both (1) whether a specific outcome was attributed to an SBIR-funded project,
and (2) which outcome was identified as most important. Projects still in the developmental stage were
more than twice as likely to identify treating disease as the most important outcome (45 percent) than
were completed projects and those being commercialized (20 percent for each) When compared to
completed projects, projects that were developing their products also were significantly less likely to
believe their most important outcome was the improvement of research tools (chi-square statistic=80.8,
df=10, p < 0.001). Whereas only 11 percent chose this as their primary outcome, this was true for 24
percent of products that already had been made available for use by the target population(s).
Such differences in perspectives might be expected, given that different types of products have different
timelines for product development. As reported in Chapter 2, current project status was related to type of
product, and certain types of products (e.g., specific types of research tools) may require less time to
develop and thus be available more quickly. In addition, views of outcomes most likely change as
products move further along the product development pathway. During the product development phase,
goals are broad, a host of ideas are continuously tested, and modifications are made. By the time a
product is ready for commercialization, however, developers have a much clearer idea about the product’s
capabilities and expected outcomes.
Performance assessment for this index focused on awardees that are still in the development phase,
commercializing their product, or completed their project and making it available to consumers. This was
the method used in assessing performance in the 2002 survey. A total of 587 projects (82 percent) fell into
these categories.
Technical articles and conference presentations also represent contributions to health-related knowledge.
However, the Evaluation Framework includes these types of contributions as a measure of the extent to
which SBIR awardees develop new and improved products consistent with the NIH mission (Goal 1)
rather than as a measure of contributions to knowledge (Goal 2). As reported in Chapter 1, sizable
percentages of awardees have published one or more technical articles and presented at scientific
conferences. This includes both projects in the developmental and commercialization stages, those that
have made their product available for use by the target populations(s), and projects that have been

Page 4-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

discontinued. Given that publications and presentations are not included as a measure for Goal 2, both
estimates of the percentage of projects that make contributions to health-related knowledge most likely
would increase if publications, presentations, and patents also were defined as relevant to include under
Measure 2.1.1.

4.3

Performance on Dissemination of Information

The second index for assessing performance on the overall goal of using small business concerns to meet
Federal R&D needs focuses on the dissemination of health-related information. The Evaluation
Framework describes dissemination as having two distinct aspects. First, the NIH should disseminate
information to their awardee companies. Second, Phase II awardees are responsible for disseminating
health-related information to populations using and receiving health care resources.
4.3.1

NIH Dissemination to SBIR Phase II Awardees

Measure 2.2.1 quantifies and describes awardee perception of the usefulness of the NIH SBIR/STTR
program office’s outreach and informational services. When asked if they were aware that they could
contact NIH staff for more information about the program or assistance with the application, review,
award, and award management, an overwhelming 96 percent of awardees responded affirmatively. There
were no significant differences in these perceptions among awardees with different award start dates,
different sponsoring Institutes/Centers (ICs), or whether they had received other related SBIR awards.
4.3.2

Awardee Dissemination of Health-Related Information

Measure 2.2.2 quantifies and describes the awardees who anticipate or are disseminating SBIR-supported
technology and information to populations using and receiving health and health care resources. The
2008 survey asked awardees to identify the potential or actual target populations for their product as well
as which target population was most important. Similar to the measure for contributions to knowledge,
the 17 specific target populations, as well as the four broader target groups under which these target
populations fell, were identified by NIH personnel.
The four broad target groups of key interest were: (1) hospital inpatients, outpatients, or staff; (2) other
healthcare practitioners, such as physicians, nurses, home care providers and emergency medical
technicians; (3) laboratories, and (4) other populations, including the general public, worksites, municipal
workers, and educational institutions. Awardees typically reported that their SBIR-supported product was
likely to be used by one or more major target population group.
Exhibit 4-3 on the next page shows which major target population groups were cited as using or likely to
use the SBIR-supported product. Sixty-five percent said that hospital inpatients, outpatients, or staff were
using or likely to use their product. Nearly two-thirds (66 percent) said SBIR-supported products were
used or likely to be used by other healthcare practitioners. Fifty-six percent were intended for use by
research and/or diagnostic laboratories. Nearly three-quarters (74 percent) were targeted to other
populations. As also shown by Exhibit 4-3, awardees reported that the most important target population
was least frequently other healthcare practitioners (15 percent) and most often the other three categories
(27 to 31 percent).

Page 4-4

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 4-3

Target Population Groups that Use or Are Likely to Use the SBIR-Supported Product

Tar get pop ulat ion group

Hospital outpatients,
inpatients, or personnel

Laboratories

Other healthcare
practitioners

Other populations

100

Percent of awardee companies
Most important target population

Target population

Note. A total of 658 awardee small businesses provided information about whether their SBIR-funded project was directed at

each of the four target population groups. Data on the most important target population group were reported by 651

awardees.

SBIR-supported products are directed at four broad target population groups: hospital outpatients, inpatients, or personnel;

laboratories; other healthcare practitioners; and other populations. Between 56 and 74 percent of awardees stated that their

product focused on each of these four different target population groups. When asked about their product’s most important

target population, 31 percent of awardees identified one in the hospital outpatients, inpatients, or personnel target group. The

corresponding percentages for laboratories, other healthcare practitioners, and other populations were 27, 15, and 27 percent,

respectively.

As shown by Exhibit 4-4 (on the next page), many awardees said that their SBIR projects target several
specific populations within the target population groups. The median number of specific populations selected
was five. Eight percent of awardees identified only one specific target population for their SBIR-supported
project.
Of the 16 specific target populations listed in the survey, all but five (research laboratories, outpatients,
medical practitioners, schools and universities, and municipal workers) were identified as targets by nearly
two-thirds or more of awardees. Worksites, emergency medical services, and homecare providers were cited
as a target population most frequently, and more than 80 percent of the time. Medical practitioners and
research laboratories were cited as a target population least frequently, and less than 50 percent of the time.

Page 4-5

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 4-4

Target Populations Currently Using or Likely to Use SBIR-Funded Products

Research laboratories
Outpatients

Medical practitioners

General public

Inpatients

T arget po pu lat ion

Diagnostic laboratories

Schools and universities

Other com paies or technologies

Hospital personnel

Other health services

Educators

Worksites

Em ergency m edical services

Hom ecare providers

Police, fire, and other m unicipal w orkers

Military m edical services

Other target population

68
58
67
68
70
71
87
87
83
58
66
88

7
0

90 100

Percent
Most im portant target population

A target population

Note. A total of 658 awardee small businesses provided information about whether each specific target population applied to their
SBIR-funded project. Data on the most important outcome was reported by 651 awardees.
Of the 16 specific target populations listed in the survey, all but five were identified as a target population by nearly two-thirds or
more of awardees. The two specific target populations that were identified as the most important by the largest percentages of
respondents were research laboratories and outpatients.

Page 4-6

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

For the hospital populations, outpatients, inpatients, or personnel category, awardees most frequently
identified outpatients as their most important target population (19 percent). For laboratories, awardees
most often stated that their most important population was research laboratories (22 percent). For other
healthcare practitioners, medical practitioners emerged as the key target population (12 percent). Finally,
when considering other populations, the general public was the key group, chosen by 11 percent of
respondents. Other specific populations were the most important target for smaller groups of awardee
businesses (1 to 9 percent).
Current project status was related to both whether awardees identified a target population group for their
SBIR-funded project and which segment was most important. Although no differences surfaced with
regard to research and diagnostic laboratories, they did appear for the other target population groups. The
primary disparities occurred between “on hold” projects and those at more active stages. Whereas 31
percent of the products in development, 25 percent of projects in the commercialization stage, and 30
percent of completed projects saw hospitals as a target population, this was true for only 14 percent of
projects that were “on hold.” Only with regard to other target populations did a different pattern emerge.
Here, 50 percent of projects in the midst of developing their products identified a specific target
population such as the general public, worksites, or schools and universities, compared to between 16 and
18 percent of projects at other stages.
When one considers only the most important specific target population, 41 percent of projects in the
developmental stage viewed hospitals as their most important consumers. However, only 17 percent of
completed projects, 23 percent of projects in the commercialization stage, and 34 percent of projects “on
hold” held similar opinions.
Exhibit 4-5, on the following page, presents SBIR awardees’ estimates of the anticipated size of the
primary target population for their products. All awardees estimate that the largest target population is
the general public; 72 percent of the companies that are gearing their product toward this group judge its
size as equal to or exceeding 500,000. The same percentage of companies whose most important target
population is other health service practitioners also estimate these groups to be of similar size, as do the
69 percent of projects with medical practitioners as their key constituency. Between 52 and 56 percent of
awardees who identified diagnostic labs, outpatients, and other populations also expect these groups to be
relatively large.
Multiplying the number of respondents in each size category by the midpoint of each category yields a
rough estimate of the size of the total population being served, for most size categories. Estimates for the
category of 500,000 or more used the start point of 500,000 because the category’s upper limit was not
specified. This yields 145.2 million as the approximate total size of all populations using and receiving
products relevant to health prevention, detection, diagnosis, and treatment as well as biomedical research.
This is clearly a “guesstimate” for several reasons. In addition to using the midpoint of each size
category, this figure does not take into account that several SBIR projects can have products that are
aimed at the same group(s) of individuals, and it relies on the information provided by respondents,
whose actual knowledge of the size of the target population may vary.

Page 4-7

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 4-5
Anticipated Size of Most Important Target Populations for Ongoing Projects

Anticipated Size of Population
Target Population
Group

Total

Under 10,000
%

N
Hospitals
Outpatients
Inpatients
Hospital personnel
Laboratories
Research laboratories
Diagnostic laboratories
Other healthcare
practitioners
Medical practitioners
Other health services
Other populations
General public
Educators
Schools and universities
Other companies or
technologies
Total

108

10
10

10,000 – 49,999
%

8
9
2

56
13

14
11

7
10
26

4
1
5
8

6
5
4
7

520

106

99
23

69
18
60
10
19

3
55
3

7
0

8
17
15

10
50
21

200,000 – 499,999

9
19
22

53
13

50,000 – 199,999

19
8
2
10
2

18
15
15
10
9

12
11

8
2
3
1
1

5
10
5

500,000 or more
%

N
61
22

56
42
46

9
4
0

8
8
0

3
5

3
22

13
12

13
52

6
1

9
6

38
13

69
72

4
2
2

7
20
11

43
1
7
27

72
10
37
56

243

Note. Due to small sample sizes, home care providers and emergency medical services were combined with other health services populations. For the same reason, worksites, police and
other municipal workers, and other companies also were included under Other Populations that are not the general public, educators, or educational institutions.
More than two-thirds of projects that identified the general public, medical practitioners or other health services professionals anticipated their target populations to total at least 500,000. This
also was true for 52 percent of projects whose major target population was diagnostic laboratories and 56 percent of projects with products primarily aimed at outpatients. Nearly threequarters (74 percent) of awardees working on core technologies for research laboratories, and 60 percent of awardees with projects aimed at educators, expected their most important target
population to be less than 50,000.

Page 4-8

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

4.4

Satisfaction with the SBIR Application, Review, Award, and Post-Award
Processes

Index 2.3 describes awardees’ satisfaction with selected aspects of the SBIR program. Measure 2.3.1
benchmarks degree of satisfaction with the NIH SBIR application, review, award, and post-administration
processes.
Exhibit 4-6 summarizes awardees’ judgments about the adequacy of the instructions for preparing
applications, the review and award processes, and post-award administration. For each, levels of
satisfaction were high. Between 88 and 90 percent were completely or mostly satisfied with the quality of
the instructions for preparing applications, the award process, and the administration after their grant had
been awarded.
EXHIBIT 4-6

Awardee Satisfaction with the NIH Application, Review, Award,

and Post-Award Processes

Percent of awardee companies

SBIR progra m component

Instructions for
preparing
applications

Review process

Aw ard process

Post-aw ard
adm inistration

Com pletely or m ostly satisfied

Mixed

Com pletely or m ostly dissatisfied

Note. A total of 658 awardee small businesses provided information about whether each specific outcome
applied to their SBIR-funded project. Data on the most important outcome was reported by 652 grantees.
The total numbers of awardees who rated each aspect ranged from 704 to 708. Respondents used a 5-point
rating scale where 1 was “completely satisfied”, 2 was “mostly satisfied”, 3 was “mixed”, 4 was “mostly
dissatisfied”, and 5 was “completely dissatisfied.” The numbers shown are percentages and may not sum to
100 percent due to rounding.
SBIR Phase II awardees expressed high levels of satisfaction with the application, review, award, and postaward administrative components of the program. Between 88 and 94 percent were completely or mostly
satisfied with all components except the review process, which completely or mostly satisfied 74 percent.

Page 4-9

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

The one component that generated a somewhat lower level of satisfaction was the review process, but
even here 74 percent were completely or mostly satisfied and only 4 percent were completely or mostly
dissatisfied. Twenty-one percent held more mixed views. This expression of ambivalence is not uncommon for the recipients of NIH funding in an environment that is increasingly competitive and where even
successful awardees often have to revise and resubmit their applications one or more times before
securing funding.
Satisfaction levels did not differ significantly by fiscal year of award or by IC grouping. Although overall
satisfaction levels remained high, they did depend somewhat, however, on whether awardees had won
other related SBIR awards. Whereas 92 percent of small businesses that had no additional SBIR funding
were satisfied with the award process, this was true for 86 percent of awardees who had won other SBIR
grants (chi-square statistic=7.5, df=2, p < 0.03).
Averaging the satisfaction ratings across the four components for each respondent yielded an index of
satisfaction for each respondent. The average rating was 1.75 (standard deviation = 0.60). Using a 5point rating scale that was used where 1 = “completely satisfied” and 2 = “mostly satisfied”, indicates that
awardees’ overall satisfaction level is slightly higher than “mostly satisfied.” This represents a very
positive assessment.
Awardees were asked whether the project funded by the award would have been pursued if SBIR support
had been unavailable. Only 12 percent of companies believed that the project would still have been undertaken, and 67 percent thought that it never would have been initiated. Twenty percent were uncertain
about what would have happened. Again, no significant differences in responses were found, depending
on the fiscal year of the award, the receipt of other related SBIR awards, IC grouping, or type of product
sponsored by the award.
Finally, respondents indicated how important SBIR support was or will be in the R&D of the planned
product. Again, the overwhelming majority (89 percent) believed it was very important, and 9 percent
judged it as important. Only 1 percent said that it was somewhat important, and less than 1 percent each
deemed SBIR support as not important or not very important. Again, there were no significant
differences in responses by type of product, fiscal year, IC grouping, or receipt of other SBIR awards.

4.5

Comparison of Two Awardee Cohorts’ Performance in Increasing the Use
of Small Business to Meet Federal R&D Needs

Exhibit 4-7, on the following page, presents data about awardees performance in increasing the use of
small business to meet Federal R&D needs. The exhibit describes performance for three indices and their
associated measures for the Phase II awardees in FYs 1997-2001 and FYs 2002-2006.
As shown by the exhibit, the FY 2002-2006 awardees exceeded the 50 percent benchmark for index 2.1—
making contributions to knowledge in health promotion, disease prevention, diagnosis, health care, and
amelioration and cure of disease. As was noted in Chapter 1, comparisons of these awardees’
performance with those of the earlier survey cohort could not be made for this index due to a difference in
question wording between the two survey instruments.

Page 4-10

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 4-7

Comparison of NIH SBIR Awardees’ Performance in

Increasing Use of Small Business to Meet Federal R&D Needs

Phase II Award Start
Performance Index

FYs 1997-2001

Performance Measure

Percent

Total N

567

719

Number and percent of awardee
companies that gave high rankings to
the usefulness of outreach and
informational services

528

559

674

705

Number of disseminations of SBIRsupported technology and information
among populations using and
receiving health and health care
resources

587

719

Number and percent of awardees who
have or anticipate disseminations of
SBIR-supported technology and
information among populations using
and receiving health and health care
resources

587

719

414-498

74-89

554-560

526-637

74-90

704-708

N
2.1

2.1.1

2.2

2.2.1

2.2.2

2.3

2.3.1

FYs 2002-2006
Percent

Total N

50 percent or more of NIH SBIR
awardees make contributions to
knowledge in health promotion,
disease prevention, diagnosis, health
care, and amelioration and cure of
disease
Number and percent of SBIRsupported contributions from ongoing
projects that yield increases in health
knowledge, research tools, and
education
50 percent or more of NIH SBIR
awardees are able to obtain and
disseminate health-related information

50 percent or more of NIH awardees
express satisfaction with the
usefulness of the NIH SBIR program
Numbers and percents of awardee
companies experiencing high levels of
satisfaction with the SBIR application,
review, award, and post-award
administration of the NIH SBIR
program

Note. The FYs 1997-2001 results are based on analyses of this subset of NIH SBIR program 2002 survey data to allow comparisons
for this report. Further study is needed to determine reasons for apparent differences in performance. The 2008 survey added a
response option that affected more than one outcome measure; therefore, it was not possible to compare performance in terms of the
percents of awardees that make health-related contributions or who have or anticipate making disseminations of SBIR-supported
technology and information.
Comparisons of performance between awardees for FYs 1997-2001 and FYs 2002-2006 were limited to two measures. For both
measures, the percentages were very similar. Between 94 and 96 percent of awardees gave high rankings to outreach and
informational service for measure 2.2.1. Between 74 and 90 percent of the two cohorts expressed high levels of satisfaction with the
SBIR application, review, award, and post-award administration of the program for measure 2.3.1.

Page 4-11

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

The SBIR program consistently outperformed the index for disseminating information to its awardees.
Ninety-six percent of FYs 2002-2006 awardees gave high marks to these efforts, similar to the 94 percent
obtained for the earlier cohort of respondents.
FYs 2002-2006 awardees also exceeded the third index regarding awardee satisfaction with the SBIR
program. From 74 to 90 percent expressed high levels of satisfaction with the application, review, award,
and post-award administration processes. These results were consistent with those for the earlier cohort.

Page 4-12

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

5. COMMERCIALIZATION OF INNOVATIONS

This chapter presents findings about attainment of the third NIH SBIR program goal addressed by the survey
and the fourth goal in the Evaluation Framework, commercializing innovations. It opens with a summary of
findings about goal attainment. Then it presents analyses of each related index and overall performance of
SBIR awardees over time.

5.1

NIH SBIR Awardees Commercialized Innovations

Exhibit 5-1 displays the survey findings about awardee outcomes related to commercializing innovations.
Phase II awardees reported attainment of benchmarks that bode well for success.
EXHIBIT 5-1

Summary of Outcomes Associated with Commercializing Innovations

Goal 4
Increase commercialization of innovations

Index 4.1
40 percent or more of NIH SBIR awardees
commercialize new or improved products in healthrelated fields

Measure 4.1.1
33 percent of the awardee respondents reported
that their SBIR-supported products yielded sales

Measure 4.1.2
The total dollar volume of sales of SBIR-funded
products for responding awardees was
$395.5 million

Index 4.2
40 percent of NIH SBIR awardees grow their
companies

Measure 4.2.1

The median number of years respondent

companies with NIH SBIR awards had existed was

8 years

Measure 4.2.2

The median number of full-time employees in

Phase II awardee companies that responded was

8 individuals

Measure 4.1.3
21 percent of the SBIR-funded products that
required FDA approval received this approval

Measure 4.1.4
26 percent of the SBIR-supported products
executed licensing arrangements
Measure 4.1.5
64 percent of the SBIR-funded projects showed
other evidence of commercialization

Measure 4.1.6
36 percent of the awardee respondents obtained
additional developmental funding for SBIRsupported innovations from other sources

Page 5-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Awardees’ reported performance in attaining each performance index at the time of survey measurement
is described in the following sections.

5.2

Performance in Commercializing SBIR-Funded Products

This section reports on reported awardee attainment of six related measures in the realms of sales, FDA
approval, licensing arrangements, other evidence of commercialization, and additional developmental
funding. In assessing performance for Index 4.1, commercialization is defined as accomplishing one or
more of these outcomes. By this definition, 76 percent of the FYs 2002-2006 awardees that provided the
necessary information commercialized new or improved products in health-related fields.
5.2.1

Product Sales

Overall, 581 (88 percent) of the awardees with ongoing projects anticipated sales upon completion of
their projects. This is roughly the same percentage as for the 2002 survey, when 85 percent of awardees
expected sales.
As shown by Exhibit 5-2 on the next page, large majorities of awardees, regardless of the type of SBIR
funded product, expected or had achieved sales. With only a few exceptions, whether sales were
expected did not dramatically differ among companies working on different types of products; the
percentage foreseeing sales ranged between 88 and 94 percent for all but three types of products. The
percentages expecting sales for companies working on drugs and those developing diagnostic materials or
devices were, however, significantly lower than companies with other types of projects. Here, 80 to 81
percent of these awardees expected sales (chi-square statistic = 16.0, df=1, p < 0.001). In addition, 67
percent of companies whose SBIR-supported project involved biologics foresaw sales, and this
percentage was significantly lower than those mentioned for any other type of product (p < 0.0001).
The survey asked awardees who anticipated sales upon completion of their projects whether their expectations had been realized. When project status is not taken into account, 33 percent or 189 SBIRsupported new or improved products yielded sales; 59 percent stated that sales were still anticipated; and
8 percent described other situations, in which sales were uncertain because the project was “on hold.”

Page 5-2

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-2

Number and Percent of Awardees that Expected Sales

by Type of Product

Total
N

Type of product

Expectations for Sales
N

Drugs

Percent

149

138

Biologics

Genomics

Research tools

Software or hardware

Educational materials

Diagnostic materials or devices

Measurement or assessment tools

Environmental tools

Chemicals

Non-drug therapeutics

662

581

Medical devices

Total

Note. Data are for projects that are under development, being commercialized, already in
use by the target population or on hold. No projects that had been discontinued expected
sales.

Overall, large majorities of awardees expected sales at the start of their SBIR-supported
projects. Awardees developing drugs, biologics, and diagnostics expected significantly lower
sales (68 to 81 percent) than did awardees working on other types of products (85 to 94
percent).

When the elapsed time between the start date of the Phase II award is considered, one gains more insight
regarding the potential of awardees’ sales expectations being fulfilled. As Exhibit 5-3 on the next page
shows, between 36 and 46 percent of awardees whose Phase II award began in FYs 2002-2005 reported
that sales had been realized, compared with 24 percent for awardees whose award began in FY 2006.

Page 5-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-3

Current Status of Sales by Fiscal Year of Phase II Award

Start date of
Phase II award
(fiscal year)

Status of Expected Sales
Total N

Realized

Anticipated

N (%)

2002

42 (46)

50 (54)

2003

107

33 (31)

74 (69)

2004

103

35 (34)

68 (66)

2005

107

38 (36)

69 (64)

2006

171

41 (24)

130 (76)

Note. Data are for 580 ongoing projects that reported expected sales. Award date
and status of sales were significantly related (chi-square = 13.6, df = 4, p < 0.009).
Among awardee companies that anticipated sales, the elapsed time from the start
date of the Phase II award to survey administration was significantly related to
whether sales were realized. Awardees that received the Phase II SBIR award in
FY 2002 were the most likely to have realized their sales expectations; those that
received the award in FY 2006 were least likely to have realized sales.

The characteristics of SBIR-funded products were examined more closely for the ongoing projects that
started between FYs 2002-2006 that expected sales (see Exhibit 5-4 on the next page). The type of
product was significantly related to current sales status. Half or nearly half of the projects that produced
chemicals, environmental tools, or non-drug therapeutics, research tools, and measurement tool products
had already realized sales. However, this was true for only 22 to 25 percent of projects with medical
device, biologic, genomic, and diagnostic products. Only 5 percent of projects working on drug products
had achieved expected sales. These differences may well be a function of several factors that can increase
the time required to make a product commercially available, such as requiring FDA approval.

Page 5-4

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-4

Realization of Sales upon Project Completion by Type of SBIR-Supported Product

Ty pe of S BIR -su pport ed produc t

Environmental tools, chemicals, and non-drug
therapeutics

Resesarch tool

Measurement tools

48
44

Educatio nal materials

Computer HW/SW
Diagnostics

Biologics and genomics

Medical devices

22
5

Drugs

100

Percent
Note. Data are for 579 ongoing projects that reported whether expected sales had been realized. Status of sales was
significantly related to type of product (chi-square statistic=52.4, df=8, p < 0.0001)
As would be expected, there were significant differences between whether sales had been realized and the type of product
supported by the Phase II award. Projects involving the development or improvement of environmental tools, measurement
tools, and non-drug therapeutics; research tools; educational materials; and computer hardware and software were most likely
to have realized sales.

5.2.2

Cumulative Sales

Measure 4.1.2 indicates commercialization attainment levels by quantifying the total dollar volume of
sales of SBIR-supported products. Of the 189 awardees who realized sales, 184 provided information
about the amount of total cumulative sales through December 2007. As shown in Exhibit 5-5 on the next
page, 66 percent of awardees reported cumulative sales totaling less than $1 million. Sixteen percent
indicated sales of at least $1 million but less than $5 million. Seven percent reported sales of $5 million
or more; typically, these sales did not quite reach $25 million.

Page 5-5

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-5

Amount of Cumulative Sales for Completed SBIR-Funded Projects

Note. Data are for the 184 ongoing projects that reported realizing sales and provided information on the amount
of cumulative sales as of December 2007. Percents may not total 100 percent due to rounding.

The large majority (66 percent) of awardees indicated that their cumulative sales totaled less than $1 million.

Sixteen percent reported cumulative sales of at least $1 million but less than $5 million. Seven percent

generated sales of $5 million or more.

An estimate of cumulative total sales can be obtained by using the midpoint of each sales category and the
start point of the final unbounded category ($50 million or more). This yields an estimate of cumulative
total sales through December 2007 is $395.5 million, with an average amount per project of
approximately $2.15 million.
Certain types of products generated more sales dollars than others (see Exhibit 5-6 on the next page). The
total cumulative sales across all projects that focused on research tools nearly reached $100 million. For
medical devices, the corresponding figure was approximately $73 million. In contrast, educational
materials generated the lowest sales figures; 20 products produced a total of $4.8 million.

Page 5-6

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-6

Percent of Companies that Achieved Cumulative Sales

by Type of Product

Amount of cumulative sales through December 2007 (in thousands)
Type of product

Total
N

$50
or
less

$50$99

$100$499

$500$999

$1,000$4,499

$5,000$24,999

$25,000$49,999

$50,000
or More

Estimated
total
across all
products

Percent of Projects
%

$18,750

Medical devices

$73,075

Biologics and
genomics

$57,225

Research tools

$99,400

Software or
hardware

$53,350

Educational
materials

$4,800

Diagnostic materials
or devices

$8,175

Measurement or
assessment tools

$24,200

Other

$56,475

184

$395,450

Drugs

Total, all products

Note. Data are for the 184 ongoing projects that reported realizing sales and provided information on the amount of cumulative sales as of
December 2007. Percentages are row percentages and may not total to 100 percent due to rounding. The estimated total sales across each
product type was calculated, using the midpoint of each sales category; for the category “50,000 or more”, the bottom limit ($50 million) was
used.
Certain types of products generated more sales dollars than others. Most awardees reported estimated sales of $50,000 or less. The highest
estimated cumulative sales total was $99.4 million for companies that developed research tools. The next highest estimate—for projects that
were developing medical devices—was $73 million. Across all products, the cumulative sales totaled an estimated $395.5 million.

Page 5-7

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Exhibit 5-7 presents sales for each Institute/Center (IC) that reported sales figures. Comparing
cumulative sales among (1) different IC groupings and (2) new products and those improving or
identifying another use for an existing product indicated that there is no statistically significant
relationship between the two variables.
EXHIBIT 5-7

Estimated Cumulative Sales for Awardees Realizing Sales

by Awarding NIH Institute or Center

Awardees realizing
sales
% of all
awardees
funded
by this IC

Awarding NIH Institute or Center
N
NCI
NCRR
NCCAM
NEI
NHGRI
NHLBI
NIA
NIAAA
NIAID
NIAMS
NIBIB
NICHD
NIDA
NIDCD
NIDCR
NIDDK
NIEHD
NIGMS
NIMHI
NINDS

Total Sales
(in
thousands
of dollars)

Mean sales
(in
thousands
of dollars)

National Cancer Institute
National Center for Research Resources
National Center on Complementary and Alternative
Medicine
National Eye Institute
National Human Genome Research Institute

21
17

26
41

50,725
d
39,725

2,536.3
d
2,482.8

825

412.5

4
5

33
45

3,775
90,825

1,258.3
18,165.0

National Heart, Lung, and Blood Institute
National Institute on Aging
National Institute on Alcohol Abuse and Alcoholism
National Institute of Allergy and Infectious Diseases
National Institute of Arthritis, Musculoskeletal and Skin
Disease

13
13
5
13

17
38
56
30

7,900
a
6,900
400
a
20,900

607.7
a
575.0
80.0
a
1,741.7

625

208.3

8,325

520.3

3,100

1033.3

3,825

765.0

1,650

412.5

56,425

4,340.4

675

135.0

21
13
12

40
46
35

41,425
13,650
43,700

1,972.6
1,050.0
3,641.7

189

395,450

2,092.3

National Institute of Biomedical Imaging and
Bioengineering
National Institute of Child Health and Human
Development
National Institute on Drug Abuse
National Institute on Deafness and Communication
Disorders
National Institute of Dental and Craniofacial Research
National Institute on Diabetes, Digestive, and Kidney
Diseases
National Institute of Environmental and Health
Sciences
National Institute of General Medical Sciences
National Institute of Mental Health
National Institute of Neurological Disorders and Stroke

Total

Note. Data on realizing sales are for the 189 ongoing projects that provided this information. Information on cumulative sales as of December
2007 was provided by 184 of these respondents. Total sales and mean sales were estimated by using the midpoint of the categories shown in
Exhibit 5-6 (the start point was used for the final, unbounded category). None of the Phase II projects funded by the National Institute of
Nursing Research (NINR) and the National Center on Minority Health and Health Disparities (NCMHD) had yet realized sales.
a

Based on 12 respondents

Based on 20 respondents

Based on 3 respondents

Based on 16 respondents

The percent of awardees that realized sales, the estimated total cumulative sales, and the average sales for SBIR-funded projects varied
among ICs. NHGRI had the highest mean sales, an estimated $18 million. The four ICS with the next highest mean sales were NIDDK ($4.3
million), NINDS ($3.6 million, NCI ($2.5 million), and NCRR ($2.5 million).

Page 5-8

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Exhibit 5-8 summarizes the extent to which FYs 2002-2006 projects achieved sales.
EXHIBIT 5-8

Summary of SBIR Projects’ Sales Performance for FYs 2002-2006

as of December 2007

N of projects

Percent

Total N

Ongoing projects that expected sales

581

719

Ongoing projects that expected and had
realized sales

189

581

N of awardees

Estimated total cumulative sales to date

184

Estimated total
$395,450,000

Average per awardee
$2,092.300

Among the 719 respondents, 581, or 82, percent were ongoing and expected sales. Of this group, 189, or 33 percent,
stated that sales already had been realized. As of December 2007, 184 of the awardees who had realized sales
estimated total cumulative sales at $395.5 million, resulting in an average total of slightly more than $2 million in sales per
awardee.

5.2.3

FDA Approval of SBIR-Funded Products

Measure 4.1.3 quantifies the number of SBIR-funded products that have received FDA approval, a
paperwork intensive “must” for certain product types that delays the awardee’s ability to commercialize
in other ways, such as sales and marketing. Three hundred and nineteen respondents indicated that their
products required FDA approval. Exhibit 5-9, on the next page, shows that all projects that involved drug
development required FDA approval, as did substantial majorities of projects whose product was a
medical device (83 percent), a biologic (75 percent), or diagnostic material or device (62 percent). For
the remainder of products, FDA approval was less likely to be applicable.

Page 5-9

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-9
Percent of SBIR-Funded Products Requiring FDA Approval by Type of Product
100
100
90

83
75

Percen t

60
50
40

30
20

0
D

g
ru
ed
M

ce
vi
e
d
al
ic

c
gi
lo
o
i
D

tic
os
n
g
ia
nNo

ug
dr

tic
eu
p
a
er

l
ic
W
oo
/S
om
tt
n
W
n
e
e
H
G
er
em
ut
ur
p
s
ea
om
M
C

al
ic
em
Re

ch
ar
se

ol
to

n
io
at
c
u
Ed

m
al

l
ta
en
nm
iro
v
En

ls
ia
er
t
a

Type of product
Note. The number of awardees that indicated whether FDA approval was required and also identified the type of SBIR product
totaled 716. As would be expected, the type of product and the need for FDA approval were significantly related (chi-square
statistic=421.2, df=11, p < 0.0001).
All drug development products required FDA approval, as did 83 percent of medical device products, 75 percent of biologic products,
and 62 percent of diagnostic products.

As shown by Exhibit 5-10 (on the next page), among the 716 awardees, 319 (45 percent) indicated that
their projects needed FDA approval. Among the 319 respondents whose SBIR-funded projects required
FDA approval, slightly less than one-third (31 percent) had been submitted for FDA review. Of the 98
products that had been submitted, 68 percent had been approved. Most likely to receive approval were
diagnostics, measurement tools, non-drug therapeutics, and computer hardware and software, all of which
had approval rates of 75 percent or higher. Approval rates were somewhat lower (between 57 and 68
percent) for drugs, medical devices and biologics.

Page 5-10

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-10

FDA Approval Status by Type of SBIR-Funded Product

Type of product
Total N

N
Drug

Status of FDA Approval Process

Required FDA
approval

Submitted for FDA
review
N
%

Received FDA
approval
N
%

100

160

133

Biologic

Diagnostic

Non-drug therapeutic

100

Measurement tool

100

Genomic

13
3

68%

Medical device

Computer HW/SW

101

Chemical (non-drug)

Research tool

Educational materials

Environmental, ergonomic, or
assistive tool

716

319

45%

31%

All product types

Note. Three respondents provided no data on whether their product required FDA approval. The results are ordered from
highest to lowest in terms of the percent of SBIR-funded projects requiring FDA approval.
At least 75 percent of diagnostics, non-drug therapeutics, measurement tools, and software or hardware products that
required FDA approval had been approved.

As Exhibit 5-11 on the following page shows, 38 percent of the projects that had begun the FDA review
process had received approval for marketing. Another 32 percent had received approval for use in
clinical trials and had (1) Investigational New Drug (IND) status, (2) been granted an exemption from
IND, or (3) been judged a non significant risk. Only a small minority (2 percent) had completed the
process but not been approved.

Page 5-11

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-11

Stage of the FDA Approval Process for SBIR-Funded Projects

that Submitted Applications to the FDA

Percent

30
25
20
15

0
Obtained
approval
for
marketing

Obtained
approval
for use in
clinical
trials

Applied for Applied for
marketing clinical trial
approval
approval

Review
ongoing

Not
approved

Stage of the FDA approval process

Note. Data are for 98 projects that reported having submitted applications to the FDA.
For awardees that submitted applications to the FDA, 38 percent received approval for marketing, and 32
percent received approval for use in clinical trials. Only a small minority (2 percent) were not approved.

5.2.4

Licensing of SBIR-Funded Projects

Measure 4.1.4 quantifies the number of SBIR-supported products that executed licensing arrangements,
another indicator of commercialization. Among all projects, 26 percent (173 projects) reported that they
had done so. This percentage did not differ significantly by the start date of the Phase II award or by the
type of product, but significant differences did appear in terms of current project status (chi-square
statistic=40.4, df=8, p < 0.0001). As would be expected, discontinued projects and those that had been
placed “on hold” were the least likely to have licensed their products (13 and 14 percent, respectively). In
some cases, the failure to obtain a license was one of the reasons for not further pursuing
commercialization.
Awardee small businesses that were still developing their products also were less likely to have
successfully completed licensing negotiations (19 percent). Significantly larger percentages of projects
that were involved in commercialization (36 percent) or had made their products commercially available
(40 percent) had secured licenses for their products.
5.2.5

Other Evidence of Commercialization

Measure 4.1.5 describes and quantifies other evidence of commercialization—various types of marketing
activities, public stock offerings, and giving trade or commercial names to the SBIR-funded product.

Page 5-12

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Approximately 88 percent of companies (488) reported at least one ongoing or completed marketing
activity. Overall, about 23 percent of companies had completed their marketing plan, 20 percent had
completed hiring marketing staff, and 20 percent had completed test marketing activities. Eleven percent
reported that their publicity and advertising campaigns had been completed.
Exhibit 5-12 on the next page shows the status of marketing activities for awardees with ongoing projects
by current status of project. The shaded percentages identify their most typical level of involvement in
marketing activities, depending on the current status of their project. Between 64 and 66 percent of
awardees who were still developing their products had not yet become involved in most marketing efforts,
with the exception of preparing a marketing plan. In contrast, companies that were commercializing their
SBIR-funded projects were more likely to be in the midst of preparing marketing plans (60 percent),
publicizing and advertising their product (54 percent), and carrying out test marketing (50 percent). For
awardees who had already made their product available for use, the majority had completed all marketing
activities except ongoing publicity and advertising. When projects were “on hold,” little attention was
being paid to marketing activities.
Only 11 companies (2 percent) with ongoing projects reported a pubic offering of their stock. Nine of
these companies were listed on NASDAQ, one was on AMEX, and another was listed on another
exchange. Six of these 11 companies reported biotechnology as their major field of business, three were
pharmaceutical companies, one was primarily associated with diagnostic devices and materials, and one
was a medical devices company. Over half (6) of these companies had more than 50 employees.
Approximately 57 percent of awardees (366) provided a trade name for their SBIR-supported product.
5.2.6

Receipt of Additional Non-SBIR Funding

Measure 4.1.6 describes and quantifies awardees’ performance in obtaining additional developmental
funding for SBIR-supported innovations from non-SBIR funding sources. Approximately 36 percent of
companies received additional non-SBIR funding for their project. As would be expected, companies
whose projects were “on hold” or discontinued were significantly less likely to have obtained such nonSBIR support. Whereas 38 and 40 percent of products in the developmental and commercialization
stages and 44 percent of products in use reported the receipt of additional non-SBIR funding, this was
true for only 19 and 9 percent of “on hold” and discontinued projects, respectively (chi square
statistic=34.9, df=4, p < 0.0001).
Supplemental financial support also was more likely for certain types of products. As Exhibit 5-13 on
page 5-15 shows, between 47 and 55 percent of products that were biologics, environmental tools, drugs,
and genomics received additional non-SBIR funding. As a group, the percentage was significantly higher
than that for medical devices (43 percent had obtained additional non-SBIR funding). Between 31 and 34
percent of projects that involved diagnostic, research tool, and chemical products obtained such support as
did 29 and 27 percent of projects focused on non-drug therapeutics and computer hardware and software.
Again, the combined percentage for these products was significantly lower than for medical devices and
the previously mentioned cluster of products. The least likely to get additional funding were
measurement tools and educational materials.

Page 5-13

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-12

Percent of Awardees Engaged in Marketing Activities by Current Project Status

Project Status
Marketing Activity

Under development
Preparation of marketing
plan
Hiring of marketing staff
Publicity and advertising
Test marketing
Commercialization stage
Preparation of marketing
plan
Hiring of marketing staff
Publicity and advertising
Test marketing
In use by target
population(s)
Preparation of marketing
plan

Hiring of marketing staff
Publicity and advertising
Test marketing
“On hold”
Preparation of marketing
plan
Hiring of marketing staff
Publicity and advertising
Test marketing

Total
N

Status of involvement in marketing activities
Not yet
Planned
Ongoing
Complete
planned

%
%
%
%

N
N
N
N

274

237
250
246

155
164
157

65
66
64

45
51
32

19
20
13

23
30
4

10
12
2

14
5
13

6
2
5

133

113
128
122

31
27
19

27
21
16

29
24
24

26
19
20

27
69
61

24
54
50

26
8
18

23
6
15

140

122
141
123

16
10
7

13
7
6

7
2
2

6
1
2

36
79
31

30
56
25

52
36
68

47
52
49

43
40
39

92
77
80

1
4
3

2
8
6

2
7
5

4
14
10

1
1
2

2
2
4

Note. Percentages are row percentages and may not sum to 100 percent due to rounding. Shaded percentages refer to the current
status of individual marketing activities for the majority of projects in the status category.
Awardees’ involvement in marketing efforts depended on current project status. Awardees that were still developing their product or
had placed efforts “on hold”, typically had not yet planned marketing activities. In contrast, awardees that were in the
commercialization stage tended to have ongoing marketing activities. Awardees that had made their product available for use
typically were still running advertising and publicity campaigns, but had completed many other marketing activities.

Page 5-14

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-13

Percent of Awardees that Received Additional Non-SBIR Funding by Type

of SBIR-Supported Product

100
90
80

Perce nt

70
55

20
10
0

ol
to
l
ta
en
m
n
ro
vi

c
gi
lo
o
i

ug
Dr

al
ls
ic
ic
ol
ol
ic
W
ce
ic
st
m
to
to
ut
/S
vi
ria
o
e
o
m
t
e
e
h
n
W
e
d
at
g
en
ap
H
en
rc
al
ch
G
ia
er
ea
er
lm
em
g
ic
D
t
h
s
t
a
r
d
u
e
n
r
e
pu
R
su
ug
io
-d
M
ea
at
om
dr
on
c
M
C
N
on
du
N
E

Type of product
Note. Data are for 716 projects that provided information. Obtaining addition non-SBIR funding and type of product were
significantly related (chi square statistic=31.4, df = 11, p < 0.001)
Between 47 and 55 percent of awardees that were working on biologics, environmental tools, drugs, and genomics had
obtained additional non-SBIR funding for the core technology supported by the Phase II award. Slightly more than two-fifths (43
percent) of medical devices had been successful in securing this type of funding. For the remainder of product types, the
percentages with additional support ranged from 22 to 34 percent.

Page 5-15

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Of the 259 projects receiving non-SBIR support, the large majority (83 percent) attributed such support
partly to having received SBIR funding.
Significant differences also occurred among awarding NIH ICs (Exhibit 5-14). Awardees whose Phase II
award was sponsored by the NIAID were significantly more likely to obtain additional non-SBIR support
than all other Institutes or Centers except NIDDK. On the other hand, companies with awards from the
NIA were significantly less likely to report obtaining additional monies, with only one-fifth of each
indicating such support. This also was true for the cluster of ICs that included NIAAA, NIDA, NINR,
and NCMHD.
EXHIBIT 5-14

Percent of Awardees that Reported Receipt of Additional Non-SBIR Funding

by Awarding NIH Institute or Center

NIAID
46

NIDDK
41

Awar ding N IH Institute or C enter

NHLBI
38

NIGMS
35

NCI
32

NICHD

NIMH
25

NINDS
20

NIA

NIAAA, NIDA, NINR, and NCMHD

All other ICs
0

Percent

Note. Data are based on responses from 717 awardees. There were significant differences in the percentages of
awardees who received additional non-SBIR funding by IC (chi square statistic=19.4, df = 10, p < 0.0352). See Appendix
D for a full list of all other ICs and Exhibit 1-1 for the titles of all ICs.
Projects sponsored by different ICs differed significantly in terms of receipt of additional non-SBIR funding for their SBIRsupported project. Companies whose Phase II award was sponsored by the NIAID were significantly more likely to obtain
additional non-SBIR support than were those funded by all other ICs, except for the NIDDK.

Page 5-16

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Awardees reported different opinions about the importance to these additional funding streams. When
asked which source of funds had been most important, 18 percent identified monies from their own
company, and another 16 percent identified another private company (Exhibit 5-15). Another program
within the NIH or at another Federal agency was the most important additional source for 12 percent of
awardees. Thirteen percent perceived the monies that they had obtained from U.S. venture capital
companies as the most important additional funding source.
EXHIBIT 5-15

Percent of Awardees that Reported Receipt of Additional Non-SBIR Funding

by Funding Source and Most Important Funding Source

Aw ardee's ow n
com pany
Other private com pany

Non-SBIR Federal funds

Non-SBIR funding source

U.S. venture capital

Private individual
investor

Personal funds

State or local
governm ent funds

44
40
32

7
10

College or university

Foreign venture capital
institution

0
7
7

All other
0

Percent
Most im portant funding source

Funding source

Note. Percentages are based on responses from 257 awardees who received additional non-SBIR funding.
Awardees most often said that the most important non-SBIR funding source was their own company (18 percent), which also was
the most frequently cited non-SBIR funding source (75 percent). Sixteen percent of awardees cited other private companies as the
most important funding source, with 12 and 13 percent respectively citing non-SBIR Federal funds and U.S. venture capital most
important.

Page 5-17

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

Funding from the awardee company was least likely for projects “on hold” or discontinued (chi-square
statistic=7.30, df=3, p < 0.0629). Between 78 and 81 percent of companies whose product was being
developed, in the commercialization stage, or in use, reported investment by the business itself or
additional non-SBIR funding. This was true for only about half of companies with discontinued or
inactive projects. Although the percentages are much smaller, this pattern also occurred with regard to
foreign venture capital funding. No discontinued or inactive projects had this type of additional funding.
Only 3 percent of products in use had foreign venture capital monies, but 12 percent of companies that
were still developing or commercializing their product reported this source (chi-square statistic=6.97,
df=3, p < 0.0729). Additional investment from the individual PI was significantly more likely for projects
in the commercialization stage or in use (45 and 54 percent, respectively) as compared to projects that
were still in the developmental phase (33 percent) or inactive (11 percent) (chi-square statistic=15.1, df=3,
p < 0.0018). No statistically significant differences were found for the other sources.
5.2.7

Summary of Commercialization Efforts

As shown by Exhibit 5-1 and discussed throughout this chapter, the Evaluation Framework includes five
measures describing how NIH SBIR awardees commercialize their products—by realizing sales,
obtaining FDA approval, executing licensing arrangements, having ongoing or completed marketing
efforts, and obtaining additional non-SBIR funds for project support. Seventy-six percent of the FY 2002
through FY 2006 awardees reported commercializing new or improved products in health-related fields in
one or more of these ways.
Identifying the numbers and types of commercialization outcomes reported by awardees can be useful in
providing a deeper understanding of the nature of Phase II awardees’ commercialization. Exhibit 5-16 on
the next page provides this information for the 543 awardees who reported. Results should be interpreted
by keeping in mind that the total number of measures that any awardee can achieve depends on the
product being developed. For example, some products do not require FDA approval or may not need
licensing agreements.
As shown by Exhibit 5-16, at the time of the survey, approximately 39 percent of the respondents
reported having satisfied only one measure. The most frequently-cited accomplishment was marketing
efforts (21 percent).
Twenty-nine percent of the awardees mentioned progress in two measures of commercialization. The
most frequent mentions of two measures cited (1) realizing sales and marketing efforts (54 awardees, or
10 percent of the respondents), and (2) marketing activities and receipt of non-SBIR funds (9 percent).
Twenty-two percent reported progress on three measures. The most frequent mention of three measures
cited realizing sales, marketing efforts, and obtaining additional non-SBIR funds (40 awardees, or 7
percent of the respondents). Much smaller percentages of awardees had achieved four measures (9
percent) and all five measures (1 percent).

Page 5-18

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-16

Number and Type of Commercialization-Related Outcomes

Achieved by Awardees

Commercialization Measures
Realized
sales

Obtained FDA
approval

Executed
licensing
agreement(s)

Had ongoing or
completed
marketing efforts

Obtained
additional
non-SBIR
funds

◙
◙
◙
◙
◙
Achieved one of the five measures
◙
◙
◙

◙
◙
◙
◙
◙
◙

◙
◙
◙
◙
◙

◙
◙

Achieved two of the five measures
◙
◙
◙
◙
◙

◙
◙

◙
◙
◙

◙
◙
◙
◙
◙

◙
◙

◙

◙
◙
◙
◙
◙

Achieved three of the five measures
◙
◙
◙
◙

◙
◙
◙
◙

◙
◙

◙
◙
◙
◙
◙

◙
◙

◙
◙
◙

Achieved four of the five measures
◙

◙

Achieved all five measures

◙

7
9
24
115
56

1
2
4
21
10

211

1
54
2
1
6
3
24
16
50

<1
10
<1
<1
1
1
4
3
9

157

1
7
30
2
40
3
1
9
26

<1
1
6
<1
7
1
<1
2
5

119

2
27
10
2
9

<1
5
2
<1
2

Note. Percentages are based on responses from 543 awardees. Percentages may not total to 100 percent or to the percent for the
general category (e.g., achieved three measures) due to rounding.
Awardees were considered to have commercialized a new or improved product if they reported achieving at least one of the five
commercialization measures. Thirty-nine percent of the responding awardees had satisfied at least one measure—most typically
having ongoing or completed marketing efforts (21 percent). Twenty-nine percent of the responding awardees had achieved two of
the five measures, with the highest percentage (10 percent) having realized sales and having ongoing or completed marketing
efforts. Twenty-two percent of respondents realized three measures—the same two as the largest group in the two measure
category, and also obtaining additional non-SBIR funds. The 9 percent who achieved four of the five measures most typically
accomplished all but FDA approval (5 percent). Six awardees, or 1 percent of those reporting, achieved all five measures.

Page 5-19

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

5.3

Growth of Awardee Small Businesses

As described in Chapter 2, a significant percentage of awardee companies were in their early years of
operation at the time of the award. Approximately 30 percent had been operating for five or fewer years.
One-third (33 percent) had been in existence for 6-10 years, 16 percent for 11-15 years, and the remainder
(21 percent) for more than 15 years. The distribution of years in operation was skewed to the right. The
average years in existence was 10 years (standard deviation = 8), and the median was 8 years.
• One indicator of company size is the number of employees. As shown in Exhibit 5-17 on the
next page, 62 percent of awardee companies had 10 or fewer full-time employees. Twenty-three
percent employed 11 to 25 individuals, and 14 percent had between 26 and 100 employees. Only
a small minority (7 percent) had more than 100 employees. The distribution of full-time
employees was again skewed to the right, with a mean of 36 employees but a median that was
much smaller (8 employees).
Eighty-one percent of the respondents reported that granting of the SBIR award had allowed them to hire
additional personnel. Sixteen percent perceived the Phase II award to have no influence on staff hiring,
and 3 percent were uncertain.
Awardees also were asked to indicate whether their companies experienced various growth-related
activities because of the product developed with SBIR support. Exhibit 5-18, on page 5-22, shows the
percentage of awardee companies that responded positively to these questions. The most frequent event
was establishing a strategic partnership or alliance with another entity; nearly one-third (32 percent) of
awardees reported that such relationships had been developed. The second most frequent activity
involved executing a licensing agreement; here, slightly more than one-quarter (26 percent) of awardees
had either licensed the SBIR-funded product themselves or sold the licensing rights to another company.
In addition, about one-fifth (21 percent) of respondents reported private placement with angels, venture
capitalists, or relatives. Joint ventures and debt financing occurred for 15 and 17 percent of awardee
businesses, respectively.

Page 5-20

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-17

Number of Full-Time Employees for Awardees

100
90
80
70

Percent

60
50
40

23
18

20
10

14
7

0
1

2- 5

6 - 10

11 - 25

26 - 100

More than
100

Number of full-time employees

Note. Percentages are based on 711 awardees. For three companies that did not report this
information; data were extracted from Dunn and Bradstreet profiles available on www.manta.com.

Slightly more than three-fifths (62 percent) of Phase II awardee companies had 10 or fewer fulltime employees. The median number was eight.

Much lower percentages of awardees set up one or more spin-off companies (9 percent), merged their
company with another business (4 percent), or sold their company (2 percent). These respondents could
have been eliminated from the analysis because these outcomes made them technically ineligible to
respond, given the criteria used for inclusion in the survey target population. However, in order to remain
consistent with the analytic strategies used for the 2002 survey of Phase II awardees, these awardees were
included in the analyses of survey responses.
Only 2 percent of awardee small businesses reported a public offering of their stock. The stock
exchanges and ticker symbols for these 11 companies were:
•
•
•

New York Stock Exchange: CLRA
NASDAQ: BDSI, CASM, DCGN, IDIX, IMMC, MBRX, PANC, RPROS, RYND
Other: MIIS.OB

Page 5-21

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-18

Awardees’ Reported Growth-Related Activities

Strategic partnership or alliance

Growth-related activities

Executed licensing agreem ent

Private placem ent (angels, venture capital, or
relatives)

Joint venture (academ ic or com m ercial)

Debt financing

Set up one or m ore spin-off com panies

Merged com pany

Sold com pany

Public offering

2
0

Percent

Note. Depending on the specific activity, between 654 and 681 awardee companies provided information.
The most commonly reported growth-related activity was the development of strategic partnerships or alliances (32

percent). The next most frequently mentioned growth-related activities were successfully negotiating licensing

arrangements (26 percent) and private placements (21 percent).

The questionnaire asked awardees about the extent to which receiving an SBIR award had an impact on
other activities that are relevant to company growth. Eighty-six percent believed it had affected their
credibility or visibility for finding partners. Fifty-three percent responded that it had an impact on raising
additional capital.
Although less directly related to company growth, 94 percent said that the award had an effect on their
ability to pursue a high-risk idea or action that might otherwise not have been undertaken. This is a
strong testimonial to the role of the SBIR program in encouraging innovation.

Page 5-22

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

5.4

Comparison of Two Awardee Cohorts’ Performance in Commercializing
Innovations

Exhibit 5-19 on the next page shows the outcomes of FYs 1997-2001 and FYs 2002-2006 awardees on
the indices identified for commercialization. For the major index stating that companies with SBIR
awards commercialize new or improved products, 67 percent of the earlier cohort of awardees had some
involvement in commercialization as reported in 2002. The corresponding percentage for the later cohort
of awardees was 61 percent as reported in 2008. This difference was not statistically significant.
With respect to the other measures reported in Exhibit 4-7, the performances of each cohort were nearly
identical. The percentage of the later cohort of awardee companies that had executed licensing
arrangements was somewhat higher than the earlier cohort—25 versus 20 percent—but again, this was
not a significant difference.
In terms of growing their companies, the median numbers of years that awardees were in operation were
both 8 years. The reported median number of employees in the later cohort was 10, versus 8 employees
reported for the earlier survey cohort.

Page 5-23

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 5-19

Comparison of SBIR Awardees’ Performance in

Commercializing Innovations

Phase II Award Start
Performance Index
Performance Measure

FYs 1997- 2001
%

4.1

40 percent of companies with NIH
SBIR awards commercialize new or
improved products, processes, and or
a
services in health-related fields

4.1.1

Number and percent of SBIRsupported products that yield sales
(of those that expect sales)

4.1.2

Estimated dollar volume of sales of
SBIR-supported products (in millions)

4.1.3

4.1.4
4.1.5
4.1.6

4.2

Number and percent of SBIRsupported products receiving FDA
approval (of those that required
approval)
Number and percent of awardees
executing licensing arrangements
Number and percent of awardees
with other evidence of
commercialization
Number and percent of awardees
receiving additional non-SBIR
funding
40 percent of companies with NIH
b
SBIR awards grow their companies

FYs 2002- 2006

Total N

377

566

541

717

146

442

189

580

$395.5

189

319

116

509

173

664

341

527

391

607

214

257

717

465

558

583

714

Median

Total N

Median

Total N

4.2.1

Median number of years companies with
NIH SBIR awards have existed (median)

557

711

4.2.2

Median number of full-time employees

557

711

Note. The FYs 1997-2001 results are based on analyses of a subset of the 2002 survey data. Total cumulative sales and the
percent obtaining FDA approval were not compared for the two cohorts, given that the response categories for the questions in
the 2008 survey were modified slightly from those listed in the 2002 questionnaire.
a

This percentage refers to the percent of awardee respondents who did one or more of the following: realized sales; obtained
FDA approval (if required); executed licensing arrangements; showed other evidence of commercialization (had
ongoing/completed marketing activities or a public stock offering); or received additional non-SBIR funding for their core
technology sponsored by the Phase II award.

This percentage refers to the percent of awardee respondents who stated that one of the SBIR awards related to their product
had on impact on their hiring of additional personnel.

Performance of the two cohorts was quite similar for measures related to both indices, Index 4.1, percent of companies that
commercialized, and Index 4.2, percent of companies with NIH-SBIR awards who reported growing their companies. The one
difference between the two cohorts was that a significantly higher percentage of FY 2002-2006 awardees reported some
evidence of commercialization (74 percent) compared to the earlier cohort (67 percent).

Page 5-24

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

6. CONCLUSIONS AND RECOMMENDATIONS

The 2008 national survey to assess the outcomes of the NIH SBIR program addresses three questions
about performance by NIH SBIR Phase II awardees from FYs 2002-2006:
• To what degree do small businesses supported through the NIH SBIR program stimulate

technological innovation?

• Has the NIH SBIR program’s use of small business concerns satisfied Federal research and
development (R&D) needs?
• Has the NIH SBIR program increased private sector commercialization of innovations derived
from Federal R&D funding?
The questions focus on how well NIH has done in attaining Congressionally-mandated SBIR program
goals 1, 2 and 4. Information from the 2008 survey supplements that obtained in the similar 2002
National Institutes of Health survey and the 2006 National Research Council assessment.
This chapter presents conclusions from the 2008 survey and offers recommendations for future NIH SBIR
program evaluation.

6.1

NIH SBIR Awardees Met or Exceeded NIH Performance Targets

Respondents to the 2008 national survey to assess the outcomes of the NIH SBIR program reported
exceeding all NIH targets related to the performance issues that were the study focus.
As shown by Exhibit 6-1 on the next page, at least 82 percent of awardees:
• Developed products that were consistent with the NIH mission
• Contributed to knowledge in fields of interest to NIH
• Demonstrated ability to obtain and disseminate health-related information to target populations
• Expressed satisfaction with the usefulness of the NIH SBIR program in terms of the application,
award, and post-award administration program
• Grew their companies
For 74 percent, there is evidence that commercialization has been initiated at some level. Eighty-two
percent of respondents reported that the SBIR program had had an impact on their hiring of additional
personnel. Fifty-eight percent of awardees received additional Phase I or Phase II awards that relate to
the core technology developed by the award described by their survey responses.

Page 6-1

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

EXHIBIT 6-1

Summary of Performance for FYs 2002-2006 SBIR Phase II Awardees*

NIH SBIR Performance Goal
NIH SBIR Performance Index

NIH Performance Index
Benchmark

Awardee
Performance

1.1 Regardless of sales SBIR awardees develop products
in support of the NIH’s mission

40%

82%

1.2 SBIR awardees received additional Phase I or Phase II
awards that relate to the core technology

10%

58%

2.1 SBIR awardees make contributions to knowledge in
health promotion, disease prevention, diagnosis, health
care, and amelioration and cure of disease

50%

82%

2.2 SBIR awardees are able to obtain and disseminate
health-related information

50%

82%

2.3 SBIR awardees express satisfaction with the
usefulness of the NIH SBIR program

50%

74 – 90%

4.1 SBIR awardees commercialize new or improved
products in health-related fields

40%

74%

4.2 SBIR awardees grow their companies

40%

82%

1. Stimulate Technological Innovation

2. Use Small Business Concerns to Meet Federal R&D
Needs

4. Increase Commercialization of Innovations

The 2008 national survey to assess the outcomes of the NIH SBIR program assessed performance for three of the
program’s four goals. Awardee performance exceeded performance benchmarks for all of the performance indices that
were examined.
*The survey assessed performance for Goals 1, 2, and 4 for the SBIR program.

In addition to exceeding these numerical benchmarks, substantial percentages of SBIR awardees achieved
several other related accomplishments of keen interest to Federal SBIR policy makers and practitioners.
These accomplishments include:
• Fifty-three percent published one or more technical articles about new or improved SBIRsupported products, and 31 percent obtained one or more patents relevant to the core technology
supported by the Phase II award.
• One-third (33 percent) of the projects realized sales, for a cumulative sales total of $395.5
million.
• One-quarter (25 percent) successfully executed licensing arrangements for their SBIR-supported
product.
• Over one-third (36 percent) obtained additional non-SBIR funding related to their Phase II core
technology.

Page 6-2

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

6.2 Enhancements to NIH SBIR Program Monitoring and Evaluation Should be
Considered
The NIH SBIR program has been in the vanguard of Federal programs in focusing scarce resources on
monitoring and evaluation—two activities strongly endorsed by recent SBIR surveys and both legislative
and executive branches of the Federal government. The 2008 survey implementation experience suggests
that it may now be timely to consider two types of enhancements—updating the NIH SBIR program
Evaluation Framework and expanding NIH SBIR program monitoring capabilities.
6.2.1 Update the SBIR Program Evaluation Framework
Evaluation frameworks can be valuable documents that accurately reflect up-to-date consensus about the
attributes of program success and how and when they should be measured. Frameworks provide a guide
about how to generate information that can be predicted to furnish credible evidence of program
performance to program policy makers and managers alike.
The NIH SBIR program Evaluation Framework was developed in 2001 by an NIH-wide committee of
SBIR administrators who wished to develop consensus on the attributes of a successful NIH SBIR
program. In 2006, its suitability for use in structuring assessment of 2008 survey results was reaffirmed
by a new NIH-wide committee. Revisiting the suitability of the NIH SBIR program Evaluation
Framework would be beneficial to confirm current consensus about performance aims, given the new
understanding of program performance generated by the NIH and NRC evaluations. At least these types
of reviews should be conducted.
• Performance measure suitability. Some current measures may not be the most suitable ones for
truly assessing attainment of a performance aim. Thus, this dimension of the measures should
also be examined.
• Performance measure focus. Assessing program performance should focus on measuring those
outcomes that indisputably relate to the effects of the SBIR award. Evaluation Framework
measures should be reviewed to ensure that they all meet this criterion.
• Performance benchmark level reasonableness. There are large disparities between some stated
performance index benchmark levels in the current Evaluation Framework and awardees’
performance. This can be interpreted as quite positive—the program more than exceeded its
expectations and selected the most talented and capable small businesses to support. However,
review of performance index benchmarks is warranted, given the much higher than required
levels of performance described by both the 2002 and 2008 surveys.
A committee of SBIR administrators from throughout the NIH could be convened to review the existing
framework prior to instituting additional performance assessments and data collection efforts. To broaden
the perspective of the group, it may be useful to ask professional evaluators—possibly including those
from other Federal agencies—to review and comment on the draft NIH Framework. The review could
address these issues:
• Does the current Evaluation Framework address all relevant performance questions? If not, what
should be added? What variables best measure success in those areas of performance?

Page 6-3

January 23, 2009

National Survey to Evaluate the NIH SBIR Program
Final Report

• Does the current Evaluation Framework address any performance questions that are no longer
key issues for the NIH? Which should be deleted?
• Do Evaluation Framework performance standards and indices accurately reflect current NIH
thinking about the degree of attainment that constitutes success? If not, how should they be
revised?
• Could alternate measurement approaches be more accurate in describing results?
6.2.2

Expand Program Monitoring Capabilities

The NIH SBIR program’s ability to evaluate and report on outcome attainment relies in part on its ability
to continuously monitor program performance. Several expansions to the program’s current monitoring
capabilities may well be worthwhile investments for NIH to consider as ways to leverage the investment
in the 2008 survey and other recent evaluations.
One priority issue is whether to continue the Program Outcomes Data System (PODS) beyond its current
end date of March 2009. PODS development was funded primarily by NIH evaluation funds, but now the
system must have other support for maintenance and ongoing operations. PODS provides ready access to
automated contact and outcome data from the 2002 survey, voluntary 2002 survey updates, and the 2008
survey. The NIH SBIR program has found PODS useful for monitoring and disseminating up-to-date
status and longitudinal data about awardees and award results. PODS is a system whose purpose and
architecture matches the need for regular monitoring and data collection on outcomes for all types of
Federal programs by providing up-to-date, usable, automated data.
If the decision is made to continue PODS, it may be useful to consider strategies for generating high
levels of participation in updates by SBIR awardees. Participation rates to update requests have been
lower than ideal, in part because Federal regulations dictate that participation must be voluntary. NIH
SBIR administrators may wish to explore additional options for improving their ability to collect a
standard data set for all awardees over time. They have already tested the option of having awardees
submit Phase II final progress reports online in a format that facilitates storing the data in PODS. Initial
attempts to do this indicate that awardees are enthusiastic about this submission method and that it may
well be a solution to ensuring participation by all awardees over time.
If monitoring capabilities are expanded, early on NIH OER will need to develop procedures for handling
multiple SBIR awards per awardee and analyzing their impact on product development. Both the 2002
and 2008 surveys randomly selected a single SBIR award and its product as the focus of each survey.
The subsequent updates followed up on just this single award per awardee. This is an idealized situation,
created to minimize respondent burden in the surveys.

Page 6-4

January 23, 2009

File Type	application/pdf
File Modified	0000-00-00
File Created	2017-09-26