Federal Research: Observations on the Small Business Innovation
Research Program (28-JUN-05, GAO-05-861T).
Since it was established in 1982, GAO has consistently reported
on the success of the Small Business Innovation Research (SBIR)
program in benefiting small, innovative companies, strengthening
their role in federal research and development (R&D), and helping
federal agencies achieve their R&D goals. However, through these
reviews GAO has also identified areas where action by
participating agencies or the Congress could build on the
program's successes and improve its operations. This statement
for the record summarizes the program's successes and
improvements over time, as well as the continuing challenge of
assessing the long term results of the program.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-861T
ACCNO: A28238
TITLE: Federal Research: Observations on the Small Business
Innovation Research Program
DATE: 06/28/2005
SUBJECT: Program evaluation
Program management
Research and development
Research program management
Research programs
Small business
Performance measures
Strategic planning
Small Business Innovation Research
Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-861T
United States Government Accountability Office
GAO Testimony
Before the Subcommittee on Environment, Technology, and Standards,
Committee on Science, House of Representatives
For Release on Delivery
Expected at 3:00 p.m. EDT FEDERAL RESEARCH
Tuesday, June 28, 2005
Observations on the Small Business Innovation Research Program
Statement for the Record of Anu K. Mittal, Director Natural Resources and
Environment Team
GAO-05-861T
[IMG]
June 28, 2005
FEDERAL RESEARCH
Observations on the Small Business Innovation Research Program
What GAO Found
Between July 1985 and June 1999, GAO reviewed, reported, and testified on
the SBIR program many times at the request of the Congress. While GAO's
work focused on many different aspects of the program, it generally found
that SBIR is achieving its goals to enhance the role of small businesses
in federal R&D, stimulate commercialization of research results, and
support the participation of small businesses owned by women and/or
disadvantaged persons. Participating agencies and companies that GAO
surveyed during the course of its reviews generally rated the program
highly.
GAO also identified areas of weaknesses and made recommendations that, if
addressed, could strengthen the program further. Some of these concerns
related to (1) duplicate funding for similar, or even identical, research
projects by more than one agency, (2) inconsistent interpretations of
extramural research budgets by participating agencies, (3) geographical
concentration of awards in a small number of states, and (4) lack of
clarification on the emphasis that agencies should give to a company's
commercialization record when assessing its proposals. Most of GAO's
recommendations for program improvement have been either fully or
partially addressed by the Congress in various reauthorizations of the
program or by the agencies themselves.
One issue that continues to remain somewhat unresolved after almost two
decades of program implementation is how to assess the performance of the
SBIR program. As the program has matured, the Congress has emphasized the
potential for commercialization as an important criterion in awarding
funds and the commercialization of a product as a measure of success for
the program. However, in 1999, GAO reported that the program's other goals
also remain important to the agencies. By itself, according to some
program managers, limited commercialization may not signal "failure"
because a company may have achieved other goals, such as innovation or
responsiveness to an agency's research needs. GAO identified a variety of
reasons why assessing the performance of the SBIR program has remained a
challenge. First, because the authorizing legislation and the Small
Business Administration's (SBA) policy directives do not define the role
of the company's commercialization record in determining commercial
potential and the relative importance of the program's goals, different
approaches have emerged in agencies' evaluations of proposals. Second, GAO
found that it has been difficult to find practical ways to define and
measure the SBIR program's goals in order to evaluate proposals. For
example, the authorizing legislation lacks a clear definition of
"commercialization," and agencies sometimes differed on its meaning.
Finally, GAO reported that as the emphasis on commercialization had grown,
so had concerns that noncommercial successes may not be adequately
recognized. For example, program managers identified various projects that
met special military or medical equipment needs but that had limited sales
potential.
United States Government Accountability Office
Mr. Chairman and Members of the Subcommittee:
We are pleased to have the opportunity to comment on the Small Business
Innovation Research (SBIR) program. Since the program's inception, we have
consistently reported on its success in benefiting small, innovative
companies, strengthening their role in federal research and development
(R&D), and helping federal agencies achieve their R&D goals. However,
through these reviews we have also identified areas where action by
participating agencies or the Congress could build on the program's
successes and improve its operations. Over the life of the program these
recommendations have largely been implemented. This statement will discuss
the program's successes as well as the continuing challenge of assessing
the long term results of the SBIR program.
As a competitor in the global economy, the United States relies heavily on
innovation through research and development. The potential of small
businesses to be sources of significant innovation led the Congress to
increase government funding for R&D projects with commercial potential
that are conducted by small high-technology companies. In this context,
the Small Business Innovation Development Act of 1982 established the SBIR
program to stimulate mission-related technological innovation, use small
businesses to meet federal R&D needs, foster participation by minority and
disadvantaged persons in technological innovation, and increase private
sector commercialization of innovations derived from federal R&D.1 The act
provided for a three-phased program: phase I to determine the feasibility
and scientific and technical merit of a proposed research idea; phase II
to further develop the idea, taking into account its commercial potential;
and phase III to commercialize the resulting product or process with no
further SBIR funding.
The original program was reauthorized in 1986, extending the program's
expiration date from 1988 to 1993.2 In 1992, it was reauthorized by the
Small Business Research and Development Enhancement Act to expand and
improve the program, to emphasize its goal of increasing private sector
commercialization, to increase participation by small businesses, and to
improve the government's dissemination of program-related information.3 In
addition, the act increased funding for phase-I and phase
1Pub.L. No. 97-219 (1982). 2Pub.L. No. 99-443 (1986). 3Pub.L. No. 102-564
(1992).
II proposals to $100,000 and $750,000, respectively, with adjustments once
every 5 years for inflation and changes in the program. The program was
again reauthorized in 2000 by the Consolidated Appropriations Act of
2001,4 which directed the Small Business Administration (SBA) and
participating agencies to, among other things, expand the scope of
publicly available information on specific grants and to annually report
on their SBIR programs. In addition, the act requires award recipients to
provide information to help SBA evaluate the program. The SBIR program is
currently scheduled to expire on September 30, 2008.
Current law requires every federal department with an R&D budget of $100
million or more to establish and operate a SBIR program funded by a set
percentage of that agency's extramural R&D budget-originally 1.25 percent
and now 2.5 percent. In addition, agencies with R&D spending above $20
million are directed to establish goals for financing small business R&D
projects at levels higher than the previous year. As of fiscal year 2004,
12 federal agencies participated in the SBIR program, including the
departments of Agriculture, Commerce, Defense (DOD), Education, Energy,
Health and Human Services (HHS), Homeland Security, Housing and Urban
Development, and Transportation; the Environmental Protection Agency; the
National Aeronautics and Space Administration (NASA); and the National
Science Foundation (NSF). Each agency manages its own program, while SBA
plays a central administrative role, such as issuing policy directives and
annual reports for the program. Awards from three agencies-DOD, National
Institutes of Health, and NASA-account for the majority of SBIR funds.
From its inception in fiscal year 1983 through fiscal year 2003, federal
agencies have awarded over $15 billion for more than 76,000 projects.
Between July 1985 and June 1999, we reviewed, reported, and testified on
the SBIR program many times at the request of the Congress. While our work
focused on many different aspects of the program, we generally found that
SBIR is achieving its goals to enhance the role of small businesses in
federal R&D, stimulate commercialization of research results, and support
the participation of small businesses owned by women and/or disadvantaged
persons. Participating agencies and companies that we surveyed during the
course of our reviews generally
SBIR Program Has Generally Met Its Goals
4Pub. L. No. 106-554 (2000).
rated the program highly. Specific examples of program success that we
identified include the following:
o High-quality research. Throughout the life of the program, awards have
been based on technical merit and are generally of good quality. For
example, in 1989 we reported that according to agency officials, more than
three-quarters of the research conducted with SBIR funding was as good as
or better than other agency-funded research. Agency officials also rated
the research as more likely than other research they oversaw to result in
the invention and commercialization of new products. When we again looked
at the quality of research proposals in 1995, we found that while it was
too early to make a conclusive judgment about the long-term quality of the
research, the quality of proposals remained good, according to agency
officials.
o Widespread competition. The SBIR program successfully attracts many
qualified companies, has had a high level of competition, and consistently
has had a high number of first-time participants. Specifically, we
reported that the number of proposals that agencies received each year had
been increasing. In addition, as we reported in 1998, agencies rarely
received only a single proposal in response to a solicitation, indicating
a sustained level of competition for the awards. We also found that the
agencies deemed many more proposals worthy of awards than they were able
to fund. For example, the Air Force deemed 1,174 proposals worthy of
awards in fiscal year 1993 but funded only 470. Moreover, from fiscal
years 1993 through 1997, one third of the companies that received awards
were first-time participants. This suggests that the program attracts
hundreds of new companies annually.
o Effective outreach. SBIR agencies consistently reach out to foster
participation by women-owned or socially and economically disadvantaged
small businesses. For example, we found that DOD's SBIR managers
participated in a number of regional small business conferences and
workshops that are specifically designed to foster increased participation
by women-owned and socially and economically disadvantaged small
businesses.
o Successful commercialization. SBIR successfully fosters
commercialization of research results. At various points in the life of
the program we have reported that SBIR has been successful in increasing
private sector commercialization of innovations. For example, past GAO and
DOD surveys of companies that received SBIR Phase II funding have
determined that approximately 35 percent of the projects resulted in the
sales of products or services, and approximately 45 percent of the
projects
received additional developmental funding. We have also reported that
agencies were using various techniques to foster commercialization. For
example, in an attempt to get those companies with the greatest potential
for commercial success to the marketplace sooner, DOD instituted a Fast
Track Program, whereby companies that are able to attract outside
commitments/capital for their research during phase I are given higher
priority in receiving a phase II award.
o Helping to serve mission needs. SBIR has helped serve agencies'
missions and R&D needs. Agencies differ in the emphasis they place on
funding research to support their mission and to support more generalized
research. Specifically, we found that DOD links its projects more closely
to its mission. In comparison, other agencies emphasize research that will
be commercialized by the private sector. Many of the projects DOD funded
have specialized military applications while NIH projects have access to
the biomedical market in the private sector. Moreover, we found that SBIR
promotes research on the critical technologies identified in lists
developed by DOD and/or the National Critical Technologies Panel.
Generally agencies reviewed these listings of critical technologies to
develop research topics or conducted research that fell within one of the
two lists.
We have also identified areas of weaknesses and made recommendations that,
if addressed, could strengthen the program further. Many of our
recommendations for program improvement have been either fully or
partially addressed by the Congress in various reauthorizations of the
program or by the agencies themselves. For example,
o Duplicate funding. In 1995, we identified duplicate funding for
similar, or even identical, research projects by more than one agency. A
few companies received funding for the same proposals two, three, and even
five times before agencies became aware of the duplication. Contributing
factors included the fraudulent evasion of disclosure by companies
applying for awards, the lack of a consistent definition for key terms
such as "similar research," and the lack of interagency sharing of data on
awards. In response to our recommendations, SBA strengthened the language
agencies use in their application packages to clearly warn applicants
about the illegality of entering into multiple agreements for essentially
the same effort and developed Internet capabilities to access SBIR data
for all of the agencies. In SBA's view, the stronger language regarding
the illegality of seeking funding for similar or identical projects
addresses the need to develop consistent definitions to help agencies
determine when projects are "similar."
Improvements Made to the SBIR Program Over Time
o Inconsistent interpretations of extramural research budgets. In 1998,
we found that while agency officials adhered to SBIR's program and
statutory funding requirements, they used differing interpretations of how
to calculate their "extramural research budgets." As a result some
agencies were inappropriately including or excluding some types of
expenses. To address our recommendation that SBA provide additional
guidance on how participating agencies were to calculate their extramural
research budgets, the Congress in 2000 required that the agencies report
annually to SBA on the methods used to calculate their extramural research
budgets.
o Geographical concentration of awards. In 1999, in response to
congressional concerns about the geographical concentration of SBIR
awards, we reported that companies in a small number of states, especially
California and Massachusetts, have submitted the most proposals and won
the majority of awards. The distribution of awards generally followed the
pattern of distribution of non-SBIR expenditures for R&D, venture capital
investments, and academic research funds. We reported that some agencies
had undertaken efforts to broaden the geographic distribution of awards
and that the program implemented by the National Science Foundation had
been particularly effective. Although we did not make any recommendations
on how to improve the program's outreach to states receiving fewer awards,
in the 2000 reauthorization of the program, Congress established the
Federal and State Technology Partnership Program to help strengthen the
technological competitiveness of small businesses, especially in those
states that receive fewer SBIR grants.
o Clarification on commercialization and other SBIR goals. Finally, in
response to our continuing concern that clarification was needed on the
relative emphasis that agencies should give to a company's
commercialization record and SBIR's other goals when evaluating proposals,
in 2000 the Congress required companies applying for a second phase award
to include a commercialization plan with their SBIR proposals. This
requirement partially addressed our concern. Moreover, in the spring of
2001, SBA initiated efforts to respond to our recommendation to develop
standard criteria for measuring commercial and other outcomes of the SBIR
program, such as uniform measures of sales and developmental funding, and
incorporate these criteria into its Tech-Net database. Specifically, SBA
began implementing a reporting system to measure the program's
commercialization success. In fiscal year 2002, SBA further enhanced the
reporting system to include commercialization results that would help
establish an initial baseline rate of commercialization. In addition,
small business firms participating in the
SBIR program are required to provide information annually on sales and
investments associated with their SBIR projects.
Assessing the Performance of the SBIR Program Remains a Challenge
One issue that continues to remain somewhat unresolved after almost two
decades of program implementation is how to assess the performance of the
SBIR program. As the program has matured, the Congress has emphasized the
potential for commercialization as an important criterion in awarding
funds and the commercialization of a product as a measure of success for
the program. However, in 1999, we reported that the program's other goals
also remain important to the agencies. By itself, according to some
program managers, limited commercialization may not signal "failure"
because a company may have achieved other goals, such as innovation or
responsiveness to an agency's research needs. We identified a variety of
reasons why assessing the performance of the SBIR program has remained a
challenge.
o First, because the authorizing legislation and SBA's policy directives
do not define the role of the company's commercialization record in
determining commercial potential and the relative importance of the
program's goals, different approaches have emerged in agencies'
evaluations of proposals. As a result, the relative weight that should be
given to the program's goals when evaluating proposals remains unclear.
Innovation and responsiveness to an agency's needs, for example, may
compete with the achievement of commercialization. In the view of many
program managers, innovation involves a willingness to undertake R&D with
a higher element of risk and a greater chance that it may not lead to a
commercial product; responsiveness to an agency's needs involves R&D that
may be aimed at special niches with limited commercial potential. Striking
the right balance between achieving commercial sales and encouraging new,
unproven technologies is, according to the program managers, one of the
key ingredients in the program's overall success.
o Second, we found that it has been difficult to find practical ways to
define and measure the SBIR program's goals in order to evaluate
proposals. For example, the authorizing legislation lacks a clear
definition of "commercialization," and agencies sometimes differed on its
meaning. This absence of a definition makes it more difficult to determine
when a frequent winner is "failing" to achieve a sufficient level of
commercialization and how to include this information in an agency's
review of the company's proposal. Similarly, efforts to define and measure
technological innovation, which was one of the program's original goals,
have posed a challenge. Although definitions vary, there is widespread
agreement that technological innovation is a complex process, particularly
in the development of sophisticated modern technologies.
o Finally, we reported that as the emphasis on commercialization had
grown, so had concerns that noncommercial successes may not be adequately
recognized. For example, program managers identified various projects that
met special military or medical equipment needs but that had limited sales
potential. These projects would be helpful in reducing the agency's
expenditures and meeting the mission of the agency but may not be
appropriately captured in typical measurements of commercialization. In
general, we found that program managers valued both noncommercial and
commercial successes and feared that the former might be ignored in
emphasizing the latter.
To help evaluate the performance of the program, in the 2000
reauthorization of SBIR, Congress required SBA to develop a database that
would help the agency collect and maintain in common format necessary
program output and outcome information. The database is to include the
following information on all phase II awards: (1) revenue from the sale of
new products or services resulting from the SBIR funded research, (2)
additional investment from any non-SBIR source for further research and
development, and (3) any other description of outputs and outcomes of the
awards. In addition, the database is to include general information for
all applicants not receiving an award including an abstract of the
project.
In conclusion, Mr. Chairman, our work has shown that, overall, the SBIR
program has been successful in meeting its goals and that the Congress and
the agencies have implemented actions to strengthen the program over time.
However, an assessment of the program's results remains a challenge
because of the lack of clarity on how much emphasis the program should
place on commercialization versus other goals.
For further information, please contact Anu Mittal at (202) 512-3841 or
mittala@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON RECYCLED PAPER
*** End of document. ***