2700-0095 Supporting Statement

SUPPORTING STATEMENT

OMB Control Number: 2700-0095

A. Justification.

1. Explain the circumstances that make the collection of information necessary.

Identify any legal or administrative requirements that necessitate the collection.

In accordance with the Government Performance and Results Act (GPRA) of 1993, NASA is required to demonstrate, among other attributes, the contribution of its programs to the Nation’s economic well being. On January 14, 2004, President George W. Bush announced A Renewed Spirit of Discovery: The President’s Vision for U.S. Space Exploration, a new directive for the Nation’s space program. The fundamental goal of this directive is “to advance U.S. scientific, security, and economic interests through a robust space explora­tion program.” In 2004, the President charged NASA with the responsibility for planning and implementing an integrated, long-term robotic and human exploration program structured with measurable milestones and executed on the basis of available resources, accumulated experiences, and technology readiness. Congress endorsed this directive with two appropriations and the NASA Authorization Act. NASA will focus on six major strategic goals, as stated in the NASA 2006 Strategic Plan, over the next 10 years to achieve this Vision of extending humankind’s presence across the solar system, developing innovative technologies and promoting international and commercial participation in exploration to further U.S. scientific, security, and economic interests. National Aeronautics and Space Act of 1958 inspired these partnerships, and the Vision for Space Exploration is no less explicit in requiring innovative partnerships and collaborative commercial expansion. Within this context, NASA recognizes its obligation to “diffuse” NASA technology to U.S. industry and “infuse” commercial technology into NASA, thereby meeting its strategic goals and commercial objectives through the Small Business Innovation Research Program, as well as its need to utilize metrics and long-term trends to assess how well its programs perform in meeting their goals. The SBIR inventory of technologies and SBIR firms represent targets of opportunity for technology infusion.

Beginning with the SBIR Program Reauthorization Act of 1992 (Public Law 102-564), Federal agencies are obliged to evaluate SBIR proposals in terms of: “(i) the small business concern’s record of successfully utilizing and commercializing SBIR … research; (ii) the existence of second phase funding commitments from private sector or non-SBIR funding sources; and, (iii) the existence of third phase, follow-on commitments for the subject of the research.” The subject information collection effort provides this information. In addition, the Small Business Reauthorization Act of 2000 (Public Law 106-554) cites the importance of “output and outcome data...and any other data collected by or available to any Federal agency that such agency considers may be useful for SBIR program evaluation.” Therefore, NASA has maintained its implementation of a metrics system which it designed to measure commercial activity associated with its Small Business Innovative Research (SBIR) program as well as SBIR technology’s utility to NASA. The information collection effort that is the subject of this request provides data critical to NASA’s SBIR metrics system.

2. Indicate how, by whom, how frequently, and for what purpose the information will be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The data is compiled and aggregated on an annual basis to depict the extent to which commercial applications and related commercial activity have resulted from NASA funded SBIR technology. These results have been utilized by NASA for: 1) demonstrating to the Administration, the Congress, and to the public the commercial impact of the NASA SBIR program and the extent to which NASA is meeting its obligations under the law and consistent with the Agency’s strategic plan.; and 2) designing revisions to NASA’s SBIR program implementation for the purposes of placing greater program emphasis on commercialization of NASA funded SBIR technology and NASA’s utility of the technology in the selection of future SBIR awards. The results are made publicly available in an annual report in bound, hardcopy format as well as on NASA’s SBIR website at www.sbir.nasa.gov/SBIR/survey.html. All such information is displayed in an aggregated form to protect company-specific and therefore commercially sensitive information. Without the subject information collection effort, NASA would be unable to determine the commercial impact of its SBIR program with sufficient accuracy and credibility. Currently, the subject information collection effort provides the only reliable and sufficiently comprehensive database of NASA SBIR economic outcome results.

3. Describe whether, and to what extent the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology. Also describe any consideration of using information technology to reduce burden.

In addition to mailing out the data collection instrument, the instrument is also available on NASA’s SBIR website at www.sbir.nasa.gov/SBIR/survey.html. The survey cover letter refers SBIR companies to this site. Therefore, electronic submission of the subject information is available to 100% of all surveyed firms. NASA’s processing, compilation, and reporting of results is computerized.

4. Describe efforts to identify duplication.

There is no other currently available information on a NASA-specific basis that is sufficient in terms of its comprehensiveness and credibility to meet NASA’s SBIR program evaluation needs and obligations under the law and consistent with NASA’s strategic plan. The information collected under this survey effort over the past several years has enabled NASA to respond with factual, auditable, comprehensive, and specific answers to specific questions by Congressional staff, the General Accounting Office, as well as by the OMB.

5. If the collection of information impacts small businesses or other small entities (Item 5 of form OMB 83-I, the Paperwork Reduction Act Submission form), describe any methods used to minimize burden.

Not Applicable

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If such restrictions were placed on the collection of this information, NASA would be unable to determine and credibly report the extent to which its SBIR program is meeting the objectives of GPRA, SBIR re-authorization legislation, and NASA’s strategic plan. Also, NASA would be unable to provide credible, substantive, and specific responses to specific questions from the Congress and the GAO, as well as to address internal queries from NASA top management for information regarding NASA SBIR program outcomes. The necessary, NASA-specific information does not currently exist elsewhere. Less frequently collected data would significantly impact the accuracy and current relevancy of the annual reports generated from the corresponding NASA SBIR commercial database, particularly given that the universe of firms having won NASA Phase II awards grows each year. Approximately 40% of the firms winning NASA SBIR Phase II awards each year are firms that have not previously won a NASA SBIR Phase II award. In addition, NASA’s ability to appropriately consider a firm’s history of commercially applying NASA SBIR technology would be significantly weakened regarding NASA’s evaluation of future SBIR proposals with less frequent or no data collection.

7. Explain any special circumstances that would cause an information collection to be conducted in certain manners (as listed).

Not Applicable

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR § 1320.8 (d), soliciting comments on the information collection before submission to OMB.

See attached Federal Register notices. No comments on the collection were received.

9. Explain any decision to provide any payment or gift to respondents, other than renumeration of contractors or grantees.

No payments, gifts, or other consideration will be, or have been, provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The data collection instrument contains the following statement: “The information provided below will be considered by NASA as confidential commercial data and accordingly will not be disclosed for any purpose by NASA to non-NASA private or public parties, other than as part of data aggregated with that of other respondents, except for the compiling of this information by NASA subcontractor personnel for NASA program evaluation purposes.” Accordingly, the information has not and will not be made publicly or otherwise available on an individual firm basis except as might be required by statute. In particular, commercial data of a private entity is exempt from Freedom of Information Act requests. Therefore, this information will not be made publicly available, under law, on an individual company basis. The data is made publicly available only on an aggregated basis over all of the companies.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

Not Applicable.

12. Provide estimates of the hour burden of the collection of information.

Approximately 200 firms are surveyed each fiscal year. No firm is surveyed more frequently than once every three fiscal years. Not all firms are re-surveyed. Given that the two page data collection instrument requests “best estimate” data, respondent firms are not required to compile detailed data from financial accounting records. Summary results typically provided by even very basic financial accounting systems yield most if not all of the requested data. Also, NASA’s experience in compiling responses from more than 800 firms over the past six years confirms that the management of a small firm is sufficiently familiar with the success of the firm’s products and services to be able to provide much of this “best estimate” information almost totally from memory. Accordingly, it is estimated that the data collection instrument requires an average of one hour or less to complete.

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information.

With this information collection revision we are reporting an $11,000 total cost burden for the approximately 200 firms that respond each year, an average of $61.00 per firm. Although response to the collection is not mandatory, we want to provide an accurate accounting of the costs to the companies that do respond. Over the past three years it has been NASA’s experience that data collection instrument is typically completed by a staff member at the request of the company president and signed off on by the company president. Under an assumption that the designated staff member has a salary of not more than about $125K/year, the total resulting annual cost to the private sector amounts to about $11K, given a response rate of about 92%. (i.e. upper bound total costs for the totality of about 200 firms surveyed in a given year = firms X 1hr per firm X $125K/2080 hourly rate X response rate about 92% = maximum total cost to private sector of about $11K per year). Approximately 40% of the 200 firms surveyed per year have absolutely nothing to report. In those cases, the time required to complete and return the survey form is almost negligible. Also, it is noteworthy that the information requested constitutes readily available data that firms typically would collect within their financial accounting systems or otherwise consists of product related, well known facts by company management.

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses, and any other expense that would not have been incurred without this collection of information.

Regarding Federal Government costs, it is estimated that the collection, compilation, and reporting of this information along with corresponding development of computer software algorithms requires 3/4 of a man year, or approximately $90K per year.

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83‑I.

There is no difference between the previous submission request and this submission request in item 13 of OMB Form 83-I regarding costs to the private sector. Regarding item 14 of OMB Form 83-I, the only difference between this submission and the previous submission is cost of living salary adjustments for the same 3/4ths of a man year.

16. For collections of information intended for publication, outline plans for tabulation and publication.

The information collected is compiled, analyzed, and utilized for internal NASA decision making purposes as well as for responding to inquiries from Congressional staff, the GAO, and the OMB. The information is also made available in an annual publication, NASA SBIR Program Commercial Metrics. The latest such annual report was completed and made publicly available in October 2002. The report is available to the public in hardcopy as well as electronically at www.sbir.nasa.gov/SBIR/survey.html.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display may be inappropriate.

Not Applicable.

18. Explain each exception to the certification statement identified in item 19, “Certification for Paperwork Reduction Act Submissions” of OMB Form 83-1.

Not Applicable.

B. Collections of Information Employing Statistical Methods.

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The universe of NASA SBIR Phase II awardee firms currently totals approximately 1000 companies. Based on NASA’s experience under this effort over the past six fiscal years, NASA believes that about 135 of these companies no longer exist. Therefore, about 865 firms in the current universe of Phase II awardee firms have been or will eventually be surveyed. A firm is not surveyed until it has a NASA SBIR Phase II that was awarded not sooner than five years prior. Firms are surveyed not more frequently than once every three years. If a firm reports zero results in two successive responses, three years apart, and it has received no additional Phase II awards from NASA, it is generally not re-surveyed a third time. The sample of approximately 200 firms selected each year for survey is not randomly selected either from the entire universe of NASA SBIR firms or from some stratification of that universe. Rather, the sample firms are selected based on factors such as when they were surveyed last, when they entered the NASA universe of Phase II firms, and to the extent that results previously reported by them suggests a likelihood of there being something positive to report if surveyed again. The objective is to eventually survey every NASA SBIR Phase II award winning firm over the entire 1983-present history that NASA can locate. According to the latest results, NASA has realized about a 92% response rate each year, which accounts for more than 84% of all Phase II’s awarded by NASA over the history of the SBIR program. It is NASA’s expectation that similarly high response rates will continue to be achieved.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The statistical methodology is explained in item 1. immediately above. Regarding estimation approach, the commercial metrics collected in the survey instrument are characterized in terms of “cumulative results to date” (e.g. “cumulative” sales to date resulting from certain commercial applications of NASA SBIR funded technology, “cumulative” private capital investment to date, “cumulative” number of resulting commercial products in non-USG markets to date, and other “cumulative” measures to date). Therefore, it is possible to meaningfully add results, for different survey years, over the universe of all respondents to obtain conservative, aggregated “minimums”. This “minimum” data has the attribute of being totally auditable back to company provided source documents. This approach is achievable because the available universe is currently only about 1000 firms. It is NASA’s experience that the universe of NASA Phase II awardee firms constitute a manageable number of firms to survey over a period of about three fiscal years, given that not all firms are surveyed as frequently as every three years, as previously explained. This approach circumvents the significant error (i.e. too large a confidence interval) introduced in statistical estimation as a consequence of typically low response rates and limited success in sub-sampling the population of originally non-responding firms. Also, the basis for statistical random sampling is “representativeness” of the universe by the selected sample units and an acceptably low “variability” across sampled units for each statistic to be estimated. However, results compiled by NASA over the past six fiscal years demonstrate: a) great variability across firms regarding the commercial statistics queried in the data collection instrument; and, b) insufficient correlation of magnitude of commercial results with demographic variables that might allow meaningful stratification for the purposes of designing a statistical random sampling. Consequently, if – as NASA’s experience shows - only a few of the relatively small number of firms that realize very large returns from commercial application of NASA SBIR technology are selected in a random sample, these few firms will significantly bias the sampling results upward. If these relatively few firms are missed in selecting the random sample, the converse will be true. Circumventing the danger of introducing significant upward or downward bias, and achieving credibility of the results, are precisely the reasons NASA has defined commercial metrics in terms of minimum, cumulative, commercial activity actually realized to date by each company. They also explain NASA’s designing its methodology so that the results compiled are auditable back to company provided hard copy (mail or electronic) responses.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Since the data is requested on a “cumulative to date” basis to obtain conservative “minimum” values, the effect of non-response rate on the quality of survey results is not as significant as it might be if statistical estimates were extrapolated or generalized from random sampling results. The data yielded according to NASA’s methodology is reliable since it has an audit trail back to the company-provided data collection instrument. The data collected is generalized to the universe being studied only to the extent of representing minimum results for that universe. Under NASA’s methodology, non-responding firms are considered to have reported zero results for every variable in the survey instrument. Non-responding firms can subsequently elect to respond at any time. NASA’s compilation of “minimum”, “cumulative” data for each commercial statistic can accommodate such unexpected submissions by previously non-responding firms as well as updated information volunteered by previously responding firms. NASA accomodates and encourages non-responding firms to elect to respond at any subsequent time as well as previously responding firms that wish to volunteer updated results, particularly in cases where the updated commercial results are significantly greater than originally reported. According to its latest data, NASA has realized an average annual response rate of about 92% over the six years of this survey effort.

4. Describe any tests or procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Given the more than 1100 responses received to date (some firms having been surveyed and responding more than once over the years), NASA believes that its data collection instrument, procedures, and methodology have been tested and proven to be very effective. Responses received to date demonstrate that the firms generally: a) have no difficulty in interpreting the data collection instrument; b) do not consider the data collection instrument to be unreasonably burdensome or of doubtful use; and, c) recognize the importance of the information collection effort. Also, with the exception of only a few cases, none of the firms submitted suggestions for improving the data collection instrument.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

John R. Yadvish (202-358-1981) of NASA will continue to lead the effort to survey, compile, analyze, depict, and report on the information collected. For the period regarding this request, outside contractor assistance is not anticipated.

8