Federal Research: Interim Assessment of the Small Business Innovation
Research and Technology Transfer Programs (Testimony, 03/06/96,
GAO/T-RCED-96-93).
GAO discussed the Small Business Innovation Research (SBIR) Program and
the Small Business Technology Transfer (STTR) Pilot Program. GAO noted
that: (1) the quality of winning SBIR and STTR proposals appears to be
good, but it is too early to assess the programs' actual results; (2)
the increase in available SBIR funding did not appear to affect the
program, since the competition for SBIR funding was high and the
agencies' ratios of awards to proposals remained essentially constant;
(3) despite the increased funding, many SBIR proposals that were deemed
worthy did not receive funding; (4) the Small Business Administration is
taking steps to implement recommendations to reduce duplicate funding of
similar research; (5) the five agencies responsible for STTR programs
are taking steps to prevent entities that submit proposals from
evaluating their own and other proposals; (6) there is no consensus on
the need for and effect of the STTR program on agency research and
development and it will take years before its effectiveness in
transferring technology from research institutions to the marketplace
can be assessed; and (7) assessments of STTR programs must determine
whether innovative ideas are originating with the research institutions
more than the small businesses, their collaboration is effective in
moving technology to the marketplace, and the SBIR program could
accomplish the technology transfers without mandatory collaboration.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-RCED-96-93
TITLE: Federal Research: Interim Assessment of the Small Business
Innovation Research and Technology Transfer
Programs
DATE: 03/06/96
SUBJECT: Research programs
Research and development contracts
Technology transfer
Conflict of interest
Small business assistance
Cost effectiveness analysis
Research grants
IDENTIFIER: Small Business Innovation Research Program
Small Business Technology Transfer Pilot Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Before the Committee on Small Business
House of Representatives
For Release on Delivery
Expected at
10:00 a.m. EST
March 6, 1996
FEDERAL RESEARCH - INTERIM
ASSESSMENT OF THE
SMALL BUSINESS INNOVATION
RESEARCH AND TECHNOLOGY
TRANSFER PROGRAMS
Statement by Victor S. Rezendes
Director, Energy, Resources, and Science Issues
Resources, Community, and Economic
Development Division
GAO/T-RCED-96-93
GAO/RCED-96-93T
(307739)
Abbreviations
=============================================================== ABBREV
DOD -
DOE -
GAO -
NASA -
NIH -
NSF -
SBIR -
SBA -
STTR -
============================================================ Chapter 0
Madam Chair and Members of the Committee:
We are pleased to be here today to discuss the results of our reviews
of the Small Business Innovation Research (SBIR) Program\1
and the Small Business Technology Transfer (STTR) Pilot Program.\2
The Small Business Innovation Development Act of 1982, which
authorized the SBIR Program, emphasized the benefits of technological
innovation and the ability of small businesses to transform the
results of research and development into new products. Reflecting
its view of the program's success, the Congress reauthorized the
program in 1992. In the same 1992 legislation, the Congress also
established the STTR Program, which is closely modeled on the SBIR
Program. The STTR Program differs from SBIR primarily in requiring a
company to form a partnership with a non-profit research institution.
We have issued two reports on these programs in which we discussed
the (1) quality of research proposals in the SBIR and STTR Programs,
(2) duplication of funding for SBIR projects, (3) steps being taken
to avoid conflict of interest that would arise if a party both
submitted and evaluated STTR proposals, and (4) effect of and need
for the STTR Program. In addition, as directed by the 1992
legislation, we will provide a detailed study covering all of the
major issues affecting the SBIR Program in 1997.
Our discussion today highlights the message of our two most recent
reports. In summary, Madam Chair:
The quality of SBIR and STTR research proposals appeared favorable
when we issued our reports. For the SBIR Program, our view was
based on the (1) high level of competition, (2) large numbers of
proposals that agencies deemed worthy of funding but that
received no award, and (3) views expressed by program officials
that quality was being maintained. Nevertheless, it was too
early to make a conclusive judgment about the long-term quality
of SBIR research proposals because the major increases in
program funding had not yet occurred. For STTR, technical
experts generally concluded that the proposals called for high
quality research. As one example, the Department of Energy
(DOE) rated the quality of proposed research in all of its
winning proposals as being among the top 10 percent of research
in the Department. At this time, however, the actual results of
these awards cannot be assessed because of the newness of the
program.
In our SBIR report, we noted that there was duplicate funding of
similar research, especially with the increasing numbers of
research proposals submitted to the SBIR Program. We made
several recommendations to reduce the possibility of duplicate
funding in the future. The Small Business Administration (SBA)
is preparing to implement our recommendations in this area.
In our STTR report, we addressed congressional concerns about
potential conflicts that might arise if a federally funded
research and development center formed a partnership with a
company submitting an STTR proposal and then helped a federal
agency judge the merits of its own and other proposals. We
found that the five agencies in the program have taken steps to
avoid such problems.
Views differed on the effect of and need for the STTR Program. The
agencies provided no evidence at this early point in the program
to suggest that the STTR Program was competing for quality
proposals with the SBIR Program or reducing the quality of
agency R&D in general. Some officials noted potentially
beneficial effects such as greater collaboration between small
businesses and research institutions in the SBIR Program. The
similarity of the two programs, however, raises an issue about
the need for the pilot program. One way to assess the need for
the program is to determine its effectiveness in transferring
technology from research institutions to the marketplace. Such
information will not be ascertainable for several years because
of the time needed to turn an initial concept into a marketable
technology.
--------------------
\1 Federal Research: Interim Report on the Small Business Innovation
Research Program (GAO/RCED-95-59, Mar. 8, 1995)
\2 Federal Research: Preliminary Information on the Small Business
Technology Transfer Program (GAO/RCED-96-19, Jan. 24, 1996)
BACKGROUND
---------------------------------------------------------- Chapter 0:1
The Small Business Innovation Development Act of 1982, which
authorized the SBIR Program, designated 4 major goals for the
program: to stimulate technological innovation, to use small
business to meet federal R&D needs, to foster and encourage
participation by minority and disadvantaged persons in technological
innovation, and to increase private sector commercialization of
innovations derived from federal R&D. The Small Business Research
and Development Enhancement Act of 1992 reauthorized the SBIR Program
and established the STTR Program, closely modeled on the SBIR
Program.
Eleven federal agencies participate in the SBIR Program. Five major
agencies--the Department of Defense (DOD); National Aeronautics and
Space Administration (NASA); Department of Health and Human Services
and particularly its National Institutes of Health (NIH); DOE; and
National Science Foundation (NSF)--also participate in the STTR
Program.\3 Each agency manages its own program while the SBA plays a
central administrative role and issues policy directives and annual
reports for each program.
The legislation establishing the SBIR Program required each agency
with an external R&D budget in excess of $100 million to set aside a
certain percentage of this amount for the program. The percentage
was increased incrementally until it reached 1.25 percent in 1986.
The 1992 reauthorization legislation increased program funding to not
less than 1.5 percent for fiscal years 1993 and 1994, not less than 2
percent for fiscal years 1995 and 1996, and not less than 2.5 percent
for fiscal year 1997 and thereafter. This increase will effectively
double the funding for the program to nearly $1 billion in fiscal
year 1997.
In establishing the STTR Program, the legislation required each
agency with an external R&D budget in excess of $1 billion to set
aside not less than 0.05 percent of that budget in fiscal year 1994,
not less than 0.1 percent in fiscal year 1995, and not less than 0.15
percent in fiscal year 1996 for the STTR Program. In the first year
of the program, the agencies expended about $20 million; they
estimate that funding will triple to $60 million in the third and
last year of the pilot program.
SBIR and STTR funding is provided in two phases. Phase I is intended
to determine the scientific and technical merit and feasibility of
ideas; it generally lasts about 6 months for SBIR and 1 year for
STTR. Phase II further develops the proposed ideas and generally
lasts about 2 years. The 1992 reauthorization directed SBA to set
the general limits on the size of SBIR phase I and II awards at
$100,000 and $750,000, respectively, although awards may be for less
than these amounts. It also set the general limits for STTR awards
at $100,000 and $500,000, respectively. A third phase for SBIR and
STTR projects, where appropriate, involves the continuation or
commercial application of the R&D.
Although the two programs have many points in common, they differ in
one important respect. To be eligible for an STTR award, a small
business must collaborate with a nonprofit research institution such
as a university, a federally funded research and development center,
or other entity. This collaboration is permitted under the SBIR
Program but is not mandatory. This special STTR requirement,
according to a 1992 report,\4 was to provide a more effective
mechanism for transferring new knowledge from research institutions
to industry.
In addition to the two reports we have already provided, the
legislation directed GAO to report on SBIR in 1997; the upcoming
report will be a detailed study covering all of the major issues
affecting the program.
--------------------
\3 The other 6 SBIR agencies include the United States Department of
Agriculture, Department of Commerce, Department of Education,
Department of Transportation, Environmental Protection Agency, and
Nuclear Regulatory Commission.
\4 H.R. Rep. No. 554, 102d Cong., 2d Sess., pt. 1 (1992). The
report accompanied H.R. 4400, a predecessor to the bill (S. 2941)
that was enacted.
QUALITY RESEARCH PROPOSALS
CHARACTERIZED BOTH PROGRAMS
---------------------------------------------------------- Chapter 0:2
The quality of the proposed research in both programs was one of the
principal issues discussed in our reports. In general, we believe
the quality of the winning SBIR and STTR proposals is favorable.
For the SBIR Program, the quality of research proposals appeared to
have kept pace with the program's initial expansion. However, at the
time of our March 1995 report, it was too early to make a conclusive
judgment about the effect of the funding increases on the quality of
SBIR research proposals receiving awards because only the first (and
smallest) of the three slated increases had occurred at the time of
our report.
In general, the level of competition for SBIR awards remained high
following the initial increase in funding in fiscal year 1993. In
all five major agencies during fiscal year 1993, the number of
proposals rose between 9 and 30 percent. The increased numbers of
proposals were important in maintaining the competitiveness of the
program during the first year that the program's funding percentage
grew to 1.5 percent. In addition, the ratio of awards to proposals
within each agency remained fairly constant, ranging from 8 percent
(for DOE) to 28 percent (for NIH). Among all five agencies, the data
for fiscal year 1993 showed virtually no change in the ratio from the
previous 2 years, suggesting that the funding increase exerted no
adverse effect on the competitiveness of the program.
In addition, agencies deemed many more SBIR proposals worthy of award
than they were able to fund. In some agencies, the large number of
worthy but unfunded projects greatly exceeded the number of projects
receiving awards; for example, the Air Force deemed 1,174 proposals
worthy of award in fiscal year 1993 but funded only 470. In general,
the data showed substantial reserves of projects deemed worthy of
funding but receiving no award. In addition, SBIR program officials
in the five major agencies stated that, in their view, the quality of
research proposals was being maintained or even improved. They cited
the level of competitiveness and the large reserves of unfunded but
worthy projects as the principal reasons for their view.
Technical evaluations of STTR proposals, which served as the basis
for the selection of winning proposals, also showed favorable views
of the quality of proposed research. Nevertheless, it was too early
for us to make a conclusive judgment about the quality because of the
newness of the program. We reviewed all of the evaluations for each
of the 206 winning STTR proposals in fiscal year 1994, the first year
in which awards were made. The evaluations (1) rated proposals among
the top 10 percent of research in certain agencies, (2) awarded
perfect scores to many proposals, (3) described some proposals as
"cutting edge," and (4) generally found the quality of proposed
research to be excellent. For example, DOE rated the quality of
research in all of its winning proposals as being among the top 10
percent of all research in the agency. Of the 48 winning proposals
in NIH, 14 were judged outstanding, 31 excellent, 2 very good, and
only 1 good. There were none in NIH's "acceptable" (or lowest
fundable) category. In general, DOD rated its 105 winning proposals
highly. Of NASA's 21 winning proposals, 11 were considered above
average, and 8 were judged as being among the top 10 percent of all
NASA proposals for comparable R&D. NSF regarded the quality of
research for its winning proposals as excellent.
As part of our review of the quality of STTR research proposals, we
also examined the technical evaluations of their commercial
potential. These evaluations were generally favorable but somewhat
more cautious. For example, in some cases there were concerns about
the cost of the product that might result or the limited size of its
potential market. Such reservations were understandable in view of
the newness of the program and the innovation or risk associated with
many of the proposed projects.
SBA HAS TAKEN STEPS TO ADDRESS
DUPLICATE FUNDING OF SBIR
RESEARCH PROPOSALS
---------------------------------------------------------- Chapter 0:3
One of the issues we discussed in our SBIR report was the duplicate
funding of research proposals. According to agency officials, a few
SBIR companies received funding for the same proposals twice, three
times, and even five times before agencies became aware of the
duplication. Several factors were contributing to this problem,
including (1) the evasion of certification procedures whereby
companies fail to identify similar proposals to other agencies, (2)
the lack of a consensus on what constitutes a duplicate proposal, and
(3) the general lack of interagency access to and exchange of current
information about recent awards by other agencies.
Officials from several agencies told us that the duplicate funding
problem should be viewed in the context of the 20,000 or more
proposals being submitted annually. They agreed, however, that the
problem should be addressed. Accordingly, we made several
recommendations to the Administrator of SBA, who has taken steps to
implement them. One important effort has involved the development of
software to provide interagency access to current information
regarding recent SBIR awards. SBA officials have recently told us
that they expect to make the system operational in the near future.
AGENCIES HAVE TAKEN STEPS TO
AVOID POTENTIAL CONFLICTS OF
INTEREST IN THE STTR PROGRAM
---------------------------------------------------------- Chapter 0:4
In our STTR report, we found that the five federal agencies with STTR
Programs have taken steps to avoid potential problems relating to
conflict of interest with federally funded research and development
centers. Such conflicts could occur if a center formed a partnership
with a company submitting an STTR proposal and then helped a federal
agency judge the merits of its own and other proposals. DOD, DOE,
and NIH have specific policies intended to prevent such conflicts
while NASA and NSF have more general procedures to avoid them. Under
DOD's policy, for example, only two R&D centers are currently
approved research partners for its STTR awardees. In fact, the Air
Force had to rescind some awards because the proposed research
partners (certain DOD laboratories) were ineligible to participate.
According to DOD's STTR Program director, future proposals will be
evaluated on a case-by-case basis to ensure that conflicts of
interest do not occur.
DOD and DOE, which accounted for 29 of the 32 awards involving
centers during the first year of the program, have also taken steps
to prevent centers from using privileged information in preparing
STTR proposals. For example, DOE's policy prohibits agency staff
members from requesting or receiving assistance from personnel in
research institutions (that are eligible to participate in the STTR
Program) in preparing technical topics for the STTR solicitation.
This policy is intended to prevent research institutions from using
their expertise to influence DOE's choice of STTR research topics.
Otherwise, research institutions could acquire a significant
advantage by designing topics to match their expertise and then
preparing a proposal in the same area.
VIEWS DIFFERED ON THE EFFECT OF
AND NEED FOR THE STTR PROGRAM
---------------------------------------------------------- Chapter 0:5
Agency officials expressed differing views regarding the effect of
STTR on SBIR and other agency R&D. For example, SBA officials
contended that STTR was too small and too new a program to have any
real effect on SBIR or on the broader range of agency research at the
present time. The officials pointed out that the program represented
only 0.05 percent of each agency's external R&D budget during its
first year and that it was only 1 year old.
In contrast to the view that STTR's effect was very limited, the
Army's STTR Program manager said that STTR was influencing SBIR in a
beneficial way. In his opinion, STTR is becoming known through
national conferences and other means. Furthermore, small businesses
are realizing that they have more credibility and chance of winning
an award by collaborating with a university or other research
institution. He believes that the STTR Program has also led to more
collaboration in SBIR. In general, according to the program manager,
STTR is a promising program that may be as successful as the SBIR
Program.
The similarity of the two programs, however, raises a broader issue
about the need for the STTR Program. In the 1992 House report, the
Committee on Small Business provided two basic arguments in favor of
the program. First, the report stated that the program addresses a
core problem in U.S. economic competitiveness, the inability to
translate its worldwide leadership [in science and engineering] into
technology and commercial applications that benefit the economy.
Second, the report stated that, although SBIR has turned out to be
remarkably effective at commercializing ideas in the small business
community, it is less effective at fostering commercialization of
ideas that originate in universities, federal laboratories, and
nonprofit research institutions.
The rationale for the program, which points to certain weaknesses in
SBIR and potential strengths in STTR, suggests three questions that
are relevant in evaluating the need for STTR.
First, is the technology originating primarily in the research
institution as envisioned in the rationale for the program or is it
originating in the small business? The technology may originate with
the research institution, the small business, or a combination of the
two. In the STTR Program, the assumption is that the research
institution will be the primary originator of the new concept.
However, data to determine the extent to which research institutions
are providing the technologies are not currently available. Neither
SBA nor the agencies have collected this information. The relative
roles of the research institution and the small business as the
source of the technology bear directly on the need for the STTR
Program. If a high percentage of the ideas are originating with
small businesses rather than with research institutions, this finding
would raise questions about the need for the program. On the other
hand, if a high percentage of ideas are originating with research
institutions, this finding would suggest that the program was
achieving the first step in moving ideas from research institutions
to small businesses.
Second, if the program is effective in moving ideas from research
institutions to small businesses, then the next logical question
concerns whether their collaboration is effective in moving them to
the marketplace. This question can be approached from two
directions: (1) Short-term views of how well the collaboration is
working in general and (2) long-term data on actual
commercialization. Information on how well the collaboration is
working can be obtained in the near future. Information on actual
commercial outcomes will require a greater amount of time before it
can be obtained. Generally, 5 to 9 years are needed to turn an
initial concept into a marketable product.
Third, because one important difference between the two programs is
that STTR makes a small business/research institution collaboration
mandatory, the question arises whether the SBIR Program could
accomplish the objective of transferring technology from research
institutions to the private sector without mandatory collaboration.
The rationale for the STTR Program tends to assume that such
collaborations were relatively rare in the SBIR Program. However,
NIH's Program manager told us that, in an SBIR survey undertaken by
NIH several years ago, collaboration between small businesses and
universities was already evident in well over half of NIH's SBIR
projects. By contrast, the Army's program manager believed that
STTR's impact will be greater in the Army than in agencies such as
NIH because the Army has had a lesser degree of involvement with
universities and other research institutions in the past. Given the
apparent variation from one agency to another and the lack of current
data, no definite conclusion can be drawn at present concerning the
need for STTR in forging new collaborations.
-------------------------------------------------------- Chapter 0:5.1
In summary, the quality of both the SBIR and STTR Programs appeared
favorable at the time of our reports, although it was too early in
each case to make a conclusive judgment about the long-term quality
of research. In addition, the agencies have taken steps to address
other concerns such as duplicate funding of SBIR projects and
potential conflicts of interest in the STTR Program. Overall, the
indicators relating to STTR in its first year provide evidence of a
potentially promising program. More time will be needed, however, to
determine whether the program is meeting a unique need or duplicating
the accomplishments of the SBIR Program. Several key questions
relating to the transfer of technology from research institutions to
the marketplace are relevant in determining the need for the STTR
Program.
This concludes my statement. I would be happy to respond to any
questions you or the members of the Committee may have.
*** End of document. ***