Sbir_a 11.07

The Small Business Innovation Research (SBIR) Program, Office of the Director (OD), National Institutes of Health (NIH) is submitting this request for OMB reinstatement with changes and approval of the information collection, Second National Survey to Evaluate the NIH SBIR Program. OMB approved the information collection associated with the initial National Survey to Evaluate the NIH SBIR Program on March 15, 2002 (OMB No. 0925-0499, expiration April 30, 2003).

Background

The United States depends heavily on scientific research and development that result in innovation. In 1982, Congress enacted the Small Business Innovation Development Act of 1982 (P.L. 97-219), establishing the Small Business Innovation Research (SBIR) program. The Act authorized the SBIR program to promote and support technological innovation and to enhance the ability of small businesses to transfer research results into new products, processes, and services. The Act identified four goals for the SBIR program:

Stimulate technological innovation
Use small businesses to meet Federal research and development needs
Foster and encourage participation by minority and disadvantaged persons in technological innovation
Increase private sector commercialization of innovations derived from Federal research and development

The NIH SBIR program is tailored to meeting these goals within the context of the NIH mission to uncover new knowledge that will lead to better health for everyone.

Congress reauthorized the SBIR program in 1992, placing specific emphasis on the program goal of commercialization of Federal research and development. Through the Small Business Reauthorization Act of 2000 (P.L. 106-544), Congress extended the SBIR Program through September 30, 2008.^¹

Reasons for This Evaluation

Since the program’s inception, the NIH has invested more than $5 billion in research support for more than 24,000 projects to small businesses under the SBIR program. The NIH SBIR set-aside in FY 2007 is $580.7 million. With overall coordination provided by the Office of Extramural Programs (OEP) within the Office of Extramural Research (OER), the SBIR program is a trans-NIH activity. Each NIH Institute/Center is required to set aside a certain percentage (2.5 percent of its extramural R&D funds) to fund SBIR projects.

The NIH Office of the Director (OD), Office of Extramural Research (OER), Office of Extramural Programs (OEP), is seeking OMB approval to reinstate with changes a prior approved collection to conduct a second national survey to evaluate the outcomes of the NIH Small Business Innovation Research program. The SBIR program provides research support to small businesses for innovative technology. OMB approved the information collection associated with the initial National Survey to Evaluate the NIH SBIR Program on March 15, 2002 (OMB No. 0925-0499, expiration April 30, 2003). The first National Survey to Evaluate the NIH SBIR Program, collected data from SBIR Phase II awardees funded between FY 1992 and FY 2001. Through this First National Survey, NIH was able to obtain data demonstrating significant SBIR programmatic results including:

73% of the 768 awardee respondents reported commercializing new or improved products, processes, usages, and/or services in health-related fields.
48 drugs and medical devices developed by SBIR awardees received FDA approval;
281 awardees received additional funding from non-SBIR sources; and
436 awardees engaged in ongoing or completed marketing activities.

NIH seeks OMB approval to reinstate this information collection with changes, with the primary objective to assess the extent to which the SBIR program goals continue to be met, particularly those dealing with the commercialization of research products, processes, or services and the uncovering of new knowledge that will lead to better health for everyone. With additional outcome data, NIH can more accurately assess the results of its large financial investment in funding innovative research conducted by small business concerns. Findings will help NIH to (1) understand if innovative projects supported through the NIH SBIR program are being commercialized and if so, to classify the types of products, processes, or services that are derived through SBIR funding; (2) determine if other measures of success defined within the NIH mission are being achieved; and (3) enhance NIH’s administration of the SBIR program and the support that it provides to small business concerns.

Overall, the NIH will use the evaluation results to assess the outcomes from NIH-supported SBIR awards. The evaluation results will provide the NIH OD with the information necessary to make quality improvements to the SBIR program and enhance program performance in generating significant outcomes. The Government Performance and Results Act of 1993 (GPRA) mandates that Federal programs improve their effectiveness and public accountability by focusing on results. The OMB developed the Program Assessment Rating Tool (PART) to monitor compliance with the GPRA and to rate Federal programs for their effectiveness and ability to show results. It is anticipated that results from a second national survey will assist NIH in demonstrating that it is meeting its GPRA goals for the NIH SBIR program.

A.2 Purpose and Use of the Information Collection

The initial survey, occurred while the NIH experienced an historic doubling of its budget (FY 1998- FY 2003) and all extramural programs, including the NIH SBIR program flourished. The primary purpose of the proposed evaluation is to assess whether NIH SBIR program goals are continuing to be met, particularly in terms of awardees demonstrating outcomes that further the NIH mission to uncover new knowledge that will lead to better health for everyone. The proposed evaluation will also (1) enable the NIH to accurately assess the results of its large financial investment in funding innovative research conducted by small business concerns; (2) enable NIH to measure, with hard data from the FY 2002 to FY 2006 cohort, whether SBIR program goals are continuing to be met—those goals that promulgate NIH’s mission and those that commercialize research products; and (3) promote compliance with Small Business Reauthorization Act of 2000 and Commerce Regulation 37 C.F.R. 401.14 requiring assessments that apply to the SBIR program.

The proposed information collection is a survey of a random sample of companies that received SBIR Phase II awards during FY 2002 through FY 2006. Whereas Phase I awards support the conduct of feasibility studies, theoretical research, and research and development, Phase II awards support the principal research or research and development that result in technological innovations and commercial applications.

As with the initial survey, this proposed second national survey will collect data about products, processes, and services that resulted from SBIR funding. It will also collect data about inventions, patents, copyrights, technical papers, and presentations. It will assess the commercialization and economic impact of research products, as well as assess the various other research outcomes that improve medical care, increase intellectual property and the knowledge base, and foster research and research tools.

Trans NIH Planning Activities

The principal stakeholders in this evaluation are the NIH Office of Extramural Research (OER) and the 23 Institutes/Centers (ICs) that participate in the program. The proposed second national survey is part of a multi-phase outcome evaluation project. The second national survey mirrors, with minor changes, the initial OMB-approved survey, which was designed with input from a broad cross-section of the NIH ICs and OER.

Phase I. Phase I of the original evaluation activity was initiated in April 2000, when the NIH project sponsors convened several meetings with NIH SBIR program managers to identify metrics for assessing whether the goals of the SBIR program were being met, potential data sources, and methodologies for data collection. A number of methodologies were considered, including questionnaires, case studies, and “industrial forensics.” This group identified seven desired program outcomes, or metrics, for assessing success:

Enhance status and demographics of women, minority, and disadvantaged persons
Produce medical/societal benefits
Use SBIR awards to increase private sector commercialization
Stimulate small business technological innovation
Generate positive metrics of economic impact
Generate other sources of funding
Contribute intellectual property and other contributions to the knowledge base

Using these to guide inquiry into program assessment, the Phase I work group developed a questionnaire to collect data from participants in the NIH SBIR program. In May 2000, the NIH project leaders convened a group of experts representing key areas related to the NIH SBIR program to review the questionnaire and its relevance to the seven outcome metrics. The participants were chosen for their expertise in NIH SBIR program management, questionnaire methodology, technology development, marketing, commercialization pathways/hurdles, ability to search patent databases, and experience as NIH SBIR awardees. As a result of this input, the questionnaire was further refined.

Phase II. The second phase of the original evaluation project began in September 2000, when funding from the 1 percent set aside funds for program evaluation was obtained to support contract task order N01-OD-7-2116, “Pilot Study for the Evaluation of the NIH SBIR Program.” The purposes of this stage of the project were to define the evaluation framework, refine and pretest the NIH SBIR questionnaire, and develop the OMB clearance package for implementation of a survey of program participants.

To develop the original evaluation framework, the project team posited NIH-specific program goals, along with related standards, indices, and measures. It mapped these to specific questions on the survey questionnaire developed during Phase I. To refine the questionnaire, the contractor convened a focus group of nine NHLBI SBIR awardees to “talk through” the questionnaire—that is, to determine which questions needed clarification, differences in how respondents interpret the questions, difficulty/ease in providing the requested information, and preferred survey administration method, i.e., a paper or electronic (on-line) questionnaire. Focus group participants included owners and principal investigators from small businesses representing the development of different types of products (both single and multiple award winners). Based on their feedback, the questionnaire was further modified to assure the best chance for producing a high rate of accurate responses, and it was formatted for on-line delivery to respondents.

Subsequently, the revised questionnaire was assessed in a pretest of nine randomly selected principal investigators whose small businesses had won Phase II SBIR awards. The goals of the pretest were to gauge the ability of the questionnaire to collect desired information and to appraise the implementation of the survey online. Nine investigators completed the online pretest and then participated in a telephone debriefing about the survey content and implementation. Study results were presented in a pretest report. According to the report, pretest respondents generally found the survey comprehensive and relevant, and they were able to complete the online version within 15-20 minutes with no technical problems. The final survey incorporated changes and clarifications suggested by both the focus group and the online pretest results.

This survey received OMB clearance (OMB Control No. 0925-0499, expiration April 30, 2003) and was fielded in FY 2002. The survey was a census of all 1,052 recipients of NIH SBIR Phase II awards from FY 1992 through FY 2001. The survey implemented extensive outreach and follow-up procedures; these efforts resulted in an outstanding response rate of 85 percent.

The survey findings supported the conclusion that through the SBIR program, small businesses have contributed to the NIH mission while enhancing the commercial potential and societal impact of their technological innovations. Following the successful fielding of the 2002 survey and the publication of the final report in 2003, OER received additional funding to create a searchable database –NIH SBIR Performance Outcome Data System (PODS). PODS houses the FY 1992-FY 2001 survey database and affords authorized NIH SBIR program staff access via the NIH Intranet to SBIR awardee information, recent data measuring award outcomes, and user-defined lists and tables of awardee information and outcome measures.

With PODS in place and serving as a baseline of performance data, OER and NIH IC program staff submitted and received additional funding from the NIH Evaluation Set Aside in FY 2006 to continue enhancements to PODS and to field a second survey to capture data from the cohort of Phase II awardees from FY 2002 through FY 2006. Incorporating data from this second survey into PODS will allow NIH SBIR program managers to assess SBIR Phase II outcomes in a systematic manner.

Information to Be Collected

The survey instrument proposed in this second national study will use the same questionnaire that was used for the FY 1992-FY 2001 survey, with some minor revisions. The revisions include minor enhancements in the wording of questions and response categories. These revisions were suggested by the prior survey respondents who supplied text responses under the “Other (please specify)” response categories or asked questions about some of the survey items. The enhancements are primarily clarifications or additional response categories. The number of survey questions remains the same.

This survey will gather information in the following areas.

Company Data: Founding year; major field of business; characteristics of product, process, or service developed under the funded project; other SBIR awards won; research and fiscal activities impacted by SBIR funding
Medical and Societal Benefits, Technological Innovations: Contributions to medical care, knowledge, and research; populations using or likely to use products, processes, and services developed under the funded project; sizes of these populations
Commercialization of Products, Processes, and Services: Types of products commercialized; requirements and status of FDA approval; trade, commercial, generic, and model names and numbers; current status of funded project in terms of commercialization and marketing
Economic Impact of Products, Processes, and Services: Expectation and realization of sales; cumulative sales related to products, process, or services developed under funded project; current number of total employees at company
Additional Funding for SBIR-Supported Project: Receipt of any additional non-SBIR funding; sources of this funding and of other additional capital; financial outcomes experienced by company indicative of success, such as joint ventures, mergers, and public stock offerings
Contributions to Intellectual Property and Knowledge Base: Items associated with technology development received by company, such as patents, copyrights, publications, presentations, and awards; numbers of each received
Experiences of Companies with NIH SBIR Award Process: Ratings of steps in the SBIR application, review, and award process; awareness of assistance available from NIH; suggestions, comments, or criticisms about the strengths and weaknesses of the NIH SBIR program; awardee demographics related to roles in SBIR-supported research and in awardee company

The data from this second national survey, in conjunction with existing NIH SBIR databases and PODS, will answer the following study questions, developed as part of the evaluation framework constructed during the initial survey development process.

Do NIH SBIR awardees (that is, the companies) produce products, processes, usages, and services, as evidence that the NIH SBIR program stimulates technological innovation and supports the NIH mission?
Do NIH SBIR awardees make contributions to knowledge, increase the dissemination of information, and express satisfaction with the usefulness of the SBIR program, as evidence that the NIH SBIR program increases the use of small businesses to meet Federal research and development needs?
Do NIH SBIR awardees increase the participation of women, minorities, and disadvantaged persons in technological innovation in health-related fields, as evidence that the NIH SBIR program fosters and encourages participation by women, minority, and disadvantaged persons in technological innovation?
Do NIH SBIR awardees increase the commercialization of health-related products, processes, services, and usages resulting from Federal support for research and development, as evidence that the NIH SBIR program increases the commercialization of innovations?

With answers to these questions, NIH will be able to continue to systematically evaluate the success of the NIH SBIR program, both in terms of the success of SBIR awardees in commercializing research products and their success in fulfilling the NIH mission of uncovering new knowledge that leads to better health.

Use of Information

The Office of Extramural Research and 23 ICs will use the second survey data to assess the results of the large financial investment in funding innovative research conducted by small business concerns. Incorporation of the FY 2002-FY 2006 cohort outcomes into the PODS database enables the NIH to (1) develop a system to evaluate the performance of the NIH SBIR Program that includes measuring the success of award recipients in commercializing products, processes, or services resulting from their research projects; and (2) learn if inventions resulting from the NIH SBIR Program are being reported. The results will also be used to determine if other measures of success defined within the NIH mission are being achieved and to enhance NIH’s administration of the SBIR Program and the support it provides to small business concerns. Study results should also provide interesting information about program successes for a host of public information purposes.

The proposed survey will collect a standard data set about products, processes, and services that resulted from SBIR funding. It will also collect data about inventions, patents, copyrights, technical papers, and presentations. It will assess the commercialization and economic impact of research products, as well as assess the various other research outcomes that improve medical care, increase intellectual property and the knowledge base, and foster research and research tools. Analysis of survey results will assist SBIR managers in assessing program status in a systematic and inclusive manner. The information gathered by the survey will be useful in highlighting areas of relative success in achieving desired outcomes—areas where retaining current program management practices appear to be warranted. Study information will also be useful in highlighting areas of relative weakness—areas where fresh approaches to improving overall program performance are most warranted. At the trans-NIH level, it will be useful to learn which ICs are most and least successful and to then build strategies to improve performance. At the individual IC level, program managers will be able to assess their own program segment’s relative success in achieving the IC’s goals and then take action to improve results.

A.3 Use of Information Technology and Burden Reduction

The same information technology will be used in this second survey as was used in the initial survey. Like the initial survey, NIH will make maximal use of technology in fielding this survey. The survey will be implemented online. All of the cover, thank you, and reminder letters will be sent via email.

Using information technology both reduces respondent burden and provides enhanced quality control of the survey data. Online surveys are convenient, require less effort to complete, elicit quicker response, and minimize nonresponse. They reduce data entry errors and costs associated with key-entering data.

Security and confidentiality safeguards are built into the automated survey process. Online surveys also reduce the reliance on paper—questionnaires and other hardcopy produced during data processing.

A.4 Efforts to Identify Duplication and Use of Similar Information

Efforts to avoid duplication of prior research, evaluations, and information collections, included searches of online government and other databases. Searching identified the following research reports with dates of publication of 2002 or later. The data sources used in the research, a brief statement of the main findings of each research activity, and limitations of the data follow each report name and identification number.

Federal Research: Observations on the Small Business Innovation Research Program (GAO-05-861T). Published June, 2005

Data Sources: Summary of GAO and DOD studies on the SBIR program from July 1985 to June 1999.

Main Findings: This is a statement for the record that summarizes SBIR program successes and improvements over time, as well as the continuing challenge of assessing the long term results of the program.

Limitations of Data: No outcomes data. The report provides a summary of observations for testimony before the Subcommittee on Environment, Technology, and Standards, Committee on Science, House of Representatives.

Federal Research: Small Business Innovation Research. Agencies Need to Strengthen Efforts to Improve the Completeness, Consistency, and Accuracy of Awards Data. (GAO-07-38).

Data Sources: Review of the SBA and the SBIR-related activities of 8 of the 11 SBIR participating agencies – DOD, DOE, EPA, NASA, NIH, NIST, NSF and USDA since the SBIR Program Reauthorization Act of 2000. Interviewed SBIR program officials at each agency and officials responsible for submitting program data to SBA. Used a protocol guide to obtain information on program operations, data reporting, data quality, and the SBA Tech-Net database. Compared data provided to SBA by the eight participating agencies with data in SBA’s Tech-Net database for fiscal years 2004 and 2005.

Main Findings: The findings were that the SBA was unable to meet the congressional directive to develop a government-use database that would provide better information on the SBIR program and allow for program evaluation. The steps on which SBA relies to ensure that data are complete and accurate are inadequate. The GAO recommends that the Administrator, SBA and the SBIR participating agencies work together to strengthen efforts to ensure that the data collected for SBA’s Tech-Net database are complete, consistent, and accurate.

Limitations of Data: The study has nothing to do with determining if the NIH SBIR Program is meeting its legislated goals.

National Research Council of the National Academies (NRC): SBIR Program Diversity and Assessment Challenges, Report of a Symposium, 2004.

Data Sources: This report is from the Committee for Capitalizing on Science, Technology, and Innovation. It is the outcome of the first phase of the National Research Council’s charge from the U.S. Congress to “conduct a comprehensive study of how the SBIR program has stimulated technological innovation and used small business to meet federal R&D needs” and make recommendations on improvements to the program.

Main Findings: This volume provides a summary of the program’s history leading up to the current assessment, a description of SBIR’s role in the nation’s innovation system, an overview of SBIR’s operations at different agencies, and the methodological issues and challenges facing the current NRC assessment.

Data Limitations: The information provides a summary of SBIR program history across Federal SBIR programs. It is not specific to NIH. The broader, comprehensive study has not yet been published. It is based on a case-study methodology, and therefore will only cover a small subset of NIH SBIR Phase II awardees.

In summation, the data resulting from these research activities have numerous limitations and are not adequate to address the study questions outlined in Section A.2 for the following reasons.

The data are not timely.
The data do not relate specifically to NIH-supported SBIR awards.
The data do not measure attainment of all important NIH SBIR award outcomes.
The major data source is not NIH SBIR awardees.

The proposed information collection will collect data to address these deficits. It will allow NIH to determine the effectiveness of its SBIR Program—what NIH gets for its significant funding support, both in terms of product commercialization and in fulfilling objectives of the NIH SBIR Program. The proposed survey and its analysis will also allow NIH to establish an accountability system to evaluate the success of its SBIR Program and to conform to current statutes and regulations mandating program assessments. Additionally, the proposed survey will enhance ongoing efforts to establish a dynamic, proactive project monitoring system.

A.5 Impact on Small Businesses or Other Small Entities

As the name of the program indicates, small businesses are the sole and intended beneficiaries of these awards. A small business concern is one that at the time of award of Phase I and Phase II grant meets all of the following criteria:

Is independently owned and operated, is not dominant in the field of operation in which it is proposing, and has a place of business in the United States and operates primarily within the United States or makes a significant contribution to the U.S. economy, and is organized for profit.
Is (a) at least 51% owned and controlled by one or more individuals who are citizens of, or permanent resident aliens in, the United States or (b) it must be a for-profit business concern that is at least 51% owned and controlled by another for-profit business concern that is at least 51% owned and controlled by one or more individuals who are citizens of, or permanent resident aliens in, the United States.
Has, including its affiliates, an average number of employees for the preceding 12 months of less than 500, and meets the other regulatory requirements found in 13 CFR Part 121. Business concerns are generally considered to be affiliates of one another when either directly or indirectly, (a) one concern controls or has the power to control the other; or (b) a third party/parties controls or has the power to control both.

A business concern may be in the form of an individual proprietorship, partnership, limited liability company, corporation, joint venture, association, trust, or cooperative. Further information may be obtained at http://www.sba.gov/size, or by contacting the Small Business Administration’s Government Contracting Area Office or Office of Size Standards.

The proposed information collection is a survey of businesses that qualified as small businesses when they received their SBIR awards. It is likely that the majority are still small businesses. Thus, NIH has placed much emphasis on minimizing the burden that will be placed on the small business respondents.

For each business with more than one Phase II SBIR award, the proposed survey randomly selects a single Phase II SBIR award from among all those won. This method minimizes the response burden. Respondents need to find information and answer questions about just a single award, not all of their SBIR awards.

Additionally, NIH has tailored the proposed survey so that it includes just questions essential for program management. This keeps the time required for completing the survey to 15 minutes or less. Up to an additional 15 minutes may be necessary to locate and retrieve any necessary information. Thus, the total time burden is 30 minutes maximum. On average, the pretest respondents for the original survey took 20 minutes to retrieve information and complete the survey. Nearly all of the survey questions are “close-ended” items—respondents need merely to select their responses from among pre-coded response categories. This too helps minimize respondent burden.

A.6 Consequences of Collecting the Information Less Frequently

This second national survey, like the initial survey will involve a one-time collection of information that will be coordinated with the further development of the NIH SBIR Performance Outcomes Data System (PODS). Evaluation data from the first OMB-approved survey covering a cohort of awardees from FY 1992-FY 2001 is now housed in PODS, which is searchable by NIH Program staff. Data from this proposed second survey covering FY 2002-FY 2006 will also be deposited in PODS. The fielding of the second survey to cover these years and the placement of the data into PODS are essential components of NIH’s goal to develop an ongoing robust electronic system for capturing and assessing SBIR awardee performance outcomes on a regular basis. Once PODS incorporates data from this period, it will contain baseline outcome measures for Phase II SBIR awardees from FY 2002 through the present.

PODS serves as a searchable and interactive repository of SBIR outcomes data. This proposed evaluation and the ongoing enhancements to PODS follow the recommendations of the OIG that, “NIH develop an accountability system to evaluate performance of the SBIR and measure the success of SBIR award recipients…” Additionally, they comply with the Final Report on the National Survey to Evaluate the NIH SBIR Program that was published by OER in July 2003—that is, to capture and assess SBIR awardee performance outcomes on a regular, continuing basis.

If information about the success of the NIH SBIR Program is not collected, then NIH will not be able to accurately assess the results of its large financial investment in funding innovative research among small businesses. It will not be able to learn if the projects supported through the SBIR Program are being commercialized, as required by the Small Business Innovation Act of 1982 and its most recent reauthorization (with increased emphasis on commercialization, outputs, and outcome data) in 2000. It will not be able to determine if other measures of success defined within the NIH mission, such as research outcomes that improve medical care, increase knowledge, and foster research, are being achieved.

Without this survey information, NIH will not be able to comply with statutes and regulations mandating program assessment. Nor will it be able to address questions raised in recent evaluations of the SBIR program or act on recommendations suggested as part of the findings of these studies. (Please see Section A.1, Circumstances Making the Collection of Information Necessary: Reasons for this Evaluation.)

NIH needs current data about SBIR award outcomes. It needs to assess these outcomes using measures that are meaningful to NIH SBIR Program administrators. It needs to learn if SBIR goals are being met by NIH awardees. It needs to be able to link award outcomes with other data in its award databases, so that long-term monitoring is both possible and efficient.

A.7 Special Circumstances Relating to the Guidelines in 5 CFR 1320.5

The proposed information collection complies with the guidelines of 5 CFR 1320.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies

Federal Register Notice

The initial (60-day) notice in the Federal Register was published on February 15, 2007, on page 7442. No public comments were received.

Outside Sources Consulted

In the development of the initial survey instrument that was fielded in 2000, NIH consulted with all 23 of its Institutes/Centers (ICs) that participate in the SBIR Program, with consultants with expertise in survey instruments and evaluation methodology, with owners and principal investigators who have received both Phase I and Phase II SBIR awards (focus group participants), and with principal investigators who have received Phase II awards (surveys and in-depth debriefing interviews). All the ICs expressed support for conducting a trans-NIH evaluation of the Program that could serve as a baseline for the ongoing, prospective evaluation of the NIH SBIR Program.

For the fielding of this second survey, the opinions of an expert and interested group were sought. This group, the NIH SBIR Evaluation Working Group, was established in 2005, and held several meetings to discuss indicators of success for NIH’s SBIR Program. IC Program staff and outside consultants with expertise in survey instruments and evaluation methodology participated in the Working Group and in the minor refinements to the questions in this second survey. The Working Group for the second survey included the following:

NIH SBIR Evaluation Working Group

JoAnne Goodnight OD/NIH [email protected]

Kathleen Shino NIDDK [email protected]

Michael Weingarten NCI [email protected]

Sheri Schully NCI [email protected]

Lawrence Solomon, Ph.D. NCI [email protected]

Susan Pucie NHLBI [email protected]

Matthew Portnoy NIGMS [email protected]

Carlos Caban OD/NIH [email protected]

Stephanie Karsten Humanitas [email protected]

Lynn Firester Consultant [email protected]

Focus Group and Pretest

The initial data collection instrument was evaluated in a focus group of nine principal investigators who had won Phase I and/or Phase II SBIR awards for their small businesses. A moderator led discussion on salient topics—the most appropriate respondent, the burden of answering the questions, the best metrics for measuring success, and the motivations for participation in the survey. Subsequently, the revised instrument was evaluated in a pretest of nine principal investigators whose small businesses had won Phase II SBIR awards. Each of the nine investigators completed the online pretest survey and then participated in a telephone debriefing interview about the content and implementation of the survey. The final survey incorporated changes and clarifications suggested by the focus group, the online pretest results, and the opinions of the nine investigators who participated in the focus group and the nine who participated in the pretest and debriefing interview. Similarly, for the second survey, the input of external consultants was sought, including SBIR awardees and additional NIH program staff.

A.9 Explanation of Any Payment or Gift to Respondents

There will not be any incentive payments or gifts given to respondents who participate in this second national survey. We believe that the survey topic continues to be salient and has high interest for companies that participate in the SBIR program. The first survey had a response rate of 85 percent without using any incentives.

A.10 Assurance of Confidentiality Provided to Respondents

The second proposed information collection is not subject to the Privacy Act because it is not collecting personal information about individuals. The survey collects information about small business concerns, specifically about outcomes that have resulted from the receipt of an NIH Phase II SBIR award and the company’s experience with the SBIR Program. (Please see Attachment C.4, Applicability of Privacy Act, for a copy of the letter certifying the non-applicability of the Privacy Act.) To the extent permitted by law, NIH will not release identifiable information about specific respondent organizations, principal investigators, or company officials.

45 CFR 46, “Regulation for Protection of Human Subjects,” does not apply to this survey. The proposed second national survey does not involve human subject research since the survey will collect information about small business concerns. Thus, the survey will not collect information about a living individual. (Please see Attachment C.3, Protection of Human Subjects, for a copy of the letter certifying the non-applicability of the regulations for Protection of Human Subjects.)

NIH plans to institute data security in the implementation of the online survey and to provide confidentiality to the full extent permitted by law.

Security and confidentiality practices include:

Confidentiality pledges signed by all contractor and subcontractor personnel
Specific materials procedures used by all contractor and subcontractor personnel for the storage, processing, and transmission of both hard copy (paper files and other output) and soft copy (computer files and faxes), and for the destruction of unneeded materials
Specific computer procedures used by all contractor and subcontractor personnel for accessing, backing up, and checking for viruses in computer data files
Secure Socket Layer (SSL) encryption (the highest level of data encryption technology available for online data transmission) of the online survey implementation
Unique and confidential password access to the online survey for each respondent

Security and confidentiality assurances include:

Advance email letters from the NIH SBIR/STTR Program Coordinator that (1) advise awardees of the upcoming survey, (2) promise confidentiality to the full extent permitted by law, (3) provide assurances that NIH will not release or distribute identifiable data about specific respondent organizations, (4) give the names and telephone numbers of NIH and survey contractor key people, and (5) state the required OMB information (responses to the information collection are voluntary; the estimated public reporting burden)
Initial email letters that (1) advise awardees of the location of the survey, (2) supply unique access passwords, and (3) restate the required OMB information
Statements on the online questionnaire that (1) confidentiality will be protected to the full extent permitted by law, (2) give the OMB control number and expiration date, and (3) provide additional required OMB information (no person is required to respond to an information collection unless it displays a currently valid OMB control number and expiration date)

A.11 Justification for Sensitive Questions

The general topic of this information collection is not construed as sensitive. Nonetheless, NIH management and research personnel have constructed the survey questions to use appropriate terminology and phrasing. Their decisions about questionnaire terminology and phrasing are based on both their expertise and suggestions from investigators who conducted NIH SBIR supported research. For the initial survey investigators’ suggestions were collected during a focus group session about survey design and a pretest of the survey instrument. Given that only minor revisions were made in the second survey, it was felt that another focus group was not needed. (Please see Section A.8, Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies.)

A.12 Estimates of Hour Burden Including Annualized Hourly Cost

The individual respondent burden is estimated as the total time it takes a respondent to complete the survey plus any additional time required to locate any necessary information and to read the cover letter and instructions. Based on the feedback from principal investigators who completed the pretest for the initial survey, individual respondent burden for the second survey is estimated to stay at a maximum of 30 minutes or one-half an hour (range of approximately 15 to 30 minutes).

The following table shows the estimate of the hour burden, given 704 respondents. [The assumption is that, although the sample is about 1,037 to start, only about 704 respondents participate in the survey. Using the rates observed in the first survey, the other sample members are unusable (14.8%), ineligible (.04%), or nonresponding (20%)] The total maximum respondent hour burden is estimated as the number of survey respondents multiplied by the time it takes an individual respondent to complete the survey.

A.12-1 Estimate of Hour Burden
Type of Respondents	Number of Respondents	Frequency of Response	Average Time Per Response	Annual Hour Burden
Awardees	704	1	.5	352
Totals	704			352

The next table shows the annual costs of the information collection to potential respondents. The calculation in the table uses $75 per hour as an approximate average hourly wage for scientific investigators.

A.12-2 Annualized Cost To the Respondents

Reading instructions, locating information, and completing the survey:

704 respondents x .5 hours = 352 hours x $75/hour

$26,400

Total Annualized Cost

$26,400

The response rate goal for the information collection is 80 percent. The anticipated maximum number of small businesses that will respond to the survey is approximately 704.

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no additional other annual cost burdens estimated for either respondents or record keepers. There are no capital, operating, or maintenance costs to report.

A.14 Annualized Cost to the Federal Government

The estimated annualized cost of the information collection to the Federal Government is $178,121. The following table includes more details regarding contract support to conduct the national survey, prepare data analyses and write reports, as well as NIH staff participation to monitor the contract.

A.14-1 Annualized Cost To the Federal Government

A.15 Explanation for Program Changes or Adjustments

Two adjustments have been made since the initial survey: (1) There are minor changes to the wording in several questions in the second survey that are based on clarifications suggested by respondents to the initial survey; and (2) The anticipated maximum number of respondents is smaller than that in the initial survey thus decreasing the annual hour burden and the annualized cost to the respondents.

A.16 Plans for Tabulation and Publication and Project Time Schedule

The data from this information collection will be evaluated, displayed, and analyzed using cross tabulations and other univariate, bivariate, and multivariate analysis procedures.

Evaluation procedures include univariate frequency distributions to assess the shape of the data distributions, their centers, and their variations. The contracted statistician will examine the statistics summarizing the distributions—means, medians, modes, variances, standard deviations, skewness, and kurtosis—to check for outliers and other anomalies.
Data displays may include ordered lists, histograms, bar charts, graphs, scatter plots, box plots, and tables, depending upon whether the variables of interest are discrete or continuous and the number of values for the variables. The displays may be grouped by variables of interest, such as explanatory or background variables (such as funding institute, funding amount, or geographical area, for example).
Analysis procedures will include bivariate and multivariate procedures to examine differences between subgroups (such as between awardees funded by different institutes), to look for correlations and associations between variables, to locate explanatory variables, to look for possible trends, and to evaluate the statistical significance of findings.

The following table shows the time schedule for the project. (Please also see Section B.2, Procedures for the Collection of Information, for complete descriptions of the major survey activities.)

A.16-1 Project Time Schedule
Activity	Time Schedule
Send email letters to respondents	Within 1 month after OMB approval
Field questionnaire	Within 1 – 3 months after OMB approval
Complete field work	Within 3 – 5 months after OMB approval
Conduct follow-up, quality control, validation	Within 3 – 5 months after OMB approval
Perform analyses	Within 6 - 8 months after OMB approval
Publish results	Within 9 – 12 months after OMB approval

The survey contractor will present the findings from this information collection to NIH. The presentation will include at least these components:

Strengths and limitations of the data
Characteristics describing the awardee businesses
Findings about medical and societal benefits, and technological innovations
Findings about the commercialization, economic impact, and additional funding of products, processes, and services resulting from funded projects
Findings about contributions to intellectual property and knowledge base
Ratings and awareness measures about the SBIR award process and resources
Suggestions for improvements and modifications to the SBIR program

The presentation will use PowerPoint color slides of graphics and tables from the Final Report submitted by the survey contractor to NIH.

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

The OMB number and expiration date are appropriate and will be displayed at the beginning of the online survey. Potential respondents will be informed that they need not comply with a collection of information that does not display a currently valid OMB control number and expiration date, per 5 CFR 1320.8 (b)(1) (Please see Authorizing Statute, ICR Part I, Item 8.)

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

There is no exception to the Certification for the Paperwork Reduction Act Submissions of OMB Form 83-I.

1 Small Business Research and Development Enhancement Act, P.L. 99-443 and P.L. 102-564, Web sites (10/24/00): http://grants.nih.gov/grants/funding/sbirsttr1/1description.htm, http://www.tc.faa.gov/aar201/SBIR?SBIR_Index.html.

File Type	application/msword
Author	Lynne Firester
Last Modified By	Authorized User
File Modified	2007-11-14
File Created	2007-11-14