Federal Research: Interim Report on the Small Business Innovation
Research Program (Chapter Report, 03/08/95, GAO/RCED-95-59).
Pursuant to a legislative requirement, GAO reviewed the Small Business
Innovation Research (SBIR) Program, focusing on: (1) whether the quality
of research proposals has kept pace with the program's expansion; (2)
the implementation of a provision for technical assistance to agencies;
and (3) the duplicate funding of similar research projects.
GAO found that: (1) the quality of research proposals has kept pace with
the program's expansion; (2) it is too early to make a conclusive
judgment about the long-term quality of research proposals because major
increases in program funding have not yet occurred; (3) none of the five
major agencies taking part in the SBIR program have implemented the
discretionary provision for technical assistance to agencies and future
implementation remains uncertain; (4) these agencies have taken steps to
provide assistance with the commercialization of research results; (5)
duplicate funding of similar research projects has become a problem due
to the increasing numbers of research proposals submitted to the SBIR
program; (6) a few companies received funding for the same proposals on
more than one occasion before the agencies became aware of the
duplication; and (7) duplicate funding occurs due to the evasion of
certification procedures, lack of consensus on what constitutes a
duplicate proposal, and the general lack of current information on
recent SBIR awards.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-95-59
TITLE: Federal Research: Interim Report on the Small Business
Innovation Research Program
DATE: 03/08/95
SUBJECT: Small business assistance
Technical assistance
Research and development
Research programs
Research program management
Research grants
Federal agencies
Interagency relations
Research and development costs
IDENTIFIER: Small Business Innovation Research Program
Discretionary Technical Assistance Program
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Committees
March 1995
FEDERAL RESEARCH - INTERIM REPORT
ON THE SMALL BUSINESS INNOVATION
RESEARCH PROGRAM
GAO/RCED-95-59
Small Business Innovation Research Program
Abbreviations
=============================================================== ABBREV
DOD - Department of Defense
DOE - Department of Energy
GAO - General Accounting Office
HHS - Department of Health and Human Services
IG - Inspector General
NASA - National Aeronautics and Space Administration
NCI - National Cancer Institute
NIH - National Institutes of Health
NSF - National Science Foundation
R&D - research and development
SBA - Small Business Administration
SBIR - Small Business Innovation Research (program)
Letter
=============================================================== LETTER
B-259333
March 8, 1995
The Honorable Christopher S. Bond
Chairman
The Honorable Dale Bumpers
Ranking Minority Member
Committee on Small Business
United States Senate
The Honorable Jan Meyers
Chairman
The Honorable John J. LaFalce
Ranking Minority Member
Committee on Small Business
House of Representatives
This report was authorized by the Small Business Research and
Development Enhancement Act of 1992. It focuses on the quality of
small businesses' research proposals for technological innovation,
the implementation of a provision for technical assistance by
agencies participating in the program, and duplicate funding of
similar research by more than one agency. The report contains three
recommendations to the Small Business Administration to reduce the
risk of duplicate funding of similar research by small businesses.
Please call me at (202) 512-3841 if you or your staff have any
questions. Major contributors to this report are listed in appendix
I.
Victor S. Rezendes
Director, Energy and
Science Issues
EXECUTIVE SUMMARY
============================================================ Chapter 0
PURPOSE
---------------------------------------------------------- Chapter 0:1
As a nation competing in a global economy, the United States depends
heavily on technological innovation through research and development
(R&D). Because small business has been identified as a principal
source of significant innovation, the Small Business Innovation
Research (SBIR) program was established in 1982 to strengthen the R&D
role of small, innovative companies. Reflecting its view of the
program's success, the Congress reauthorized the program in 1992 and
provided for doubling the program's funding by fiscal year 1997.
Following the program's reauthorization, GAO conducted an interim
review that (1) determined whether the quality of research proposals
has kept pace with the program's expansion, (2) assessed the
implementation of a provision for technical assistance to agencies,
and (3) assessed the duplicate funding of similar research.
BACKGROUND
---------------------------------------------------------- Chapter 0:2
The Congress established the SBIR program in 1982 to stimulate
technological innovation, use small business to meet federal R&D
needs, foster and encourage minority and disadvantaged persons'
participation in technological innovation, and increase the private
sector's commercialization of innovations derived from federal R&D.
Eleven federal agencies participate in the SBIR program. Five of
them--the Department of Defense (DOD), the National Aeronautics and
Space Administration (NASA), the Department of Health and Human
Services and particularly its National Institutes of Health (NIH),
the Department of Energy (DOE), and the National Science Foundation
(NSF)--provide over 90 percent of SBIR's funds.\1 SBIR funding
agreements include any contract, grant, or cooperative agreement
entered into between a federal agency and any small business for the
performance of experimental, developmental, or research work funded
in whole or in part by the federal government. Each agency manages
its own program, while the Small Business Administration (SBA) plays
a central administrative role, including the issuing of policy
directives and annual reports for the program.
The legislation establishing the program required each agency with a
budget for extramural (external) R&D in excess of $100 million to set
aside a certain percentage of this amount for the program. The
percentage was increased incrementally until it reached 1.25 percent
for each participating agency in 1986. The reauthorization
legislation (P.L. 102-564, Oct. 28, 1992) increased the program's
funding percentage to not less than 1.5 percent for fiscal years 1993
and 1994, not less than 2 percent for fiscal years 1995 and 1996, and
not less than 2.5 percent for fiscal year 1997 and thereafter. The
program's total funding increased from $508.4 million in fiscal year
1992 to $698 million in fiscal year 1993 and is projected to increase
to about $900 million when the funding percentage rises to 2 percent
in fiscal year 1995.
The SBIR program provides funding for phase I and phase II awards.
(Work in phase I is intended to determine the scientific and
technical merit and feasibility of ideas; it generally lasts about 6
months. Work in phase II further develops the proposed ideas and
generally lasts about 2 years.) The size of awards in phases I and II
was generally limited under an SBA directive to $50,000 and $500,000,
respectively. However, the 1992 reauthorization directed SBA to
raise the general limits on the size of phase I and II awards to
$100,000 and $750,000, respectively, although awards may be for less
than these amounts. The 1992 reauthorization also included a
discretionary provision for technical assistance that authorized the
use of SBIR's money to assist award recipients in achieving the
technical and commercial goals of SBIR projects.
--------------------
\1 The six other agencies are the U.S. Department of Agriculture,
the Department of Commerce, the Department of Education, the
Department of Transportation, the Environmental Protection Agency,
and the Nuclear Regulatory Commission.
RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3
To date, it appears that the quality of research proposals has kept
pace with the program's expansion. GAO's view is based on the (1)
high level of competition, (2) large numbers of proposals that
agencies deemed worthy of funding but that received no award, and (3)
views expressed by SBIR officials that quality is being maintained.
However, it is too early to make a conclusive judgment about the
long-term quality of research proposals because the major increases
in the program's funding have not yet occurred.
None of the five major agencies have taken steps to implement the
discretionary provision for technical assistance to agencies, and
future implementation remains uncertain. SBIR officials saw no need
for technical assistance because projects are selected primarily for
their technical merit. However, they have taken steps, independent
of the provision, to provide assistance with the commercialization of
research results.
The duplicate funding of similar research has become a problem,
especially with the increasing numbers of research proposals
submitted to the SBIR program. According to agency officials, a few
companies received funding for the same proposals twice, three times,
and even five times before agencies became aware of the duplication.
Several factors contribute to this problem, including (1) the evasion
of certification procedures, so that companies fail to identify
similar proposals submitted to other agencies, (2) the lack of a
consensus on what constitutes a duplicate proposal, and (3) the
general lack of interagency access to and exchange of current
information about recent awards by other agencies.
PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4
QUALITY RESEARCH PROPOSALS
KEPT PACE WITH PROGRAM'S
INITIAL EXPANSION
-------------------------------------------------------- Chapter 0:4.1
Although it is too early to make a conclusive judgment about the
effect of funding increases on the quality of the SBIR research
proposals that receive awards, several factors suggest that quality
research proposals have kept pace with the program's initial
expansion. First, the level of competition for awards remained high
following the initial increase in funding in fiscal year 1993.
During fiscal year 1993, the number of proposals rose from 9 to 30
percent for all five major agencies, and the ratio of awards to
proposals within each agency remained fairly constant, ranging from 8
percent (for DOE) to 28 percent (for NIH). Second, agencies deemed
many more proposals worthy of awards than they were able to fund. At
some agencies, the large number of worthy but unfunded projects
greatly exceeded the number of projects receiving awards; for
example, the Air Force deemed 1,174 proposals worthy of awards in
fiscal year 1993 but funded only 470. Third, SBIR officials at the
five major agencies stated that in their view, the quality of
research proposals is being maintained.
THE TECHNICAL ASSISTANCE
OPTION IS NOT BEING
IMPLEMENTED
-------------------------------------------------------- Chapter 0:4.2
No agency has implemented the technical assistance provision, and
future implementation remains uncertain. SBIR officials were
critical of the provision because (1) it calls for the use of SBIR
funds, thereby reducing the number of awards the officials can make,
(2) they see little need for technical assistance when projects are
selected primarily for their technical merit, (3) they believe that
implementation would impose a significant administrative burden
arising from case-by-case considerations of companies' requests for
support, and (4) they feel that specific proposed technical
assistance requirements, such as a single vendor for technical
assistance for each agency, are unrealistic because one vendor could
not respond adequately to potentially hundreds of requests. SBIR
officials, however, have taken steps, independent of the provision,
to provide assistance with commercialization. For example, DOE has
provided special training sessions and conferences on
commercialization for its awardees.
DUPLICATE FUNDING HAS BECOME
A PROBLEM
-------------------------------------------------------- Chapter 0:4.3
Duplicate funding of similar or even identical proposals submitted to
more than one agency has occurred at NSF, NASA, and DOD. Several
factors are contributing to this problem. First, companies proposing
projects have failed to identify identical proposals that they have
made to other agencies, thereby fraudulently evading the
certification procedure that requires them to provide such
information. In response to this problem, officials in NSF's Office
of Inspector General recommended that NSF's certification form be
revised and strengthened. NSF has implemented this recommendation.
Second, the lack of definitions and guidelines regarding key terms
such as "similar" research has resulted in disagreement about what
constitutes duplicate research. Third, agencies have lacked
interagency access to and exchange of current information about
recent awards, which might help to prevent or detect duplicate
awards. Basic changes in the SBIR program, particularly the
increasing number of proposals and awards, may make the program more
vulnerable to this kind of abuse. The number of proposals to the
five major agencies rose from 17,562 in fiscal year 1992 to 20,014 in
fiscal year 1993--an increase of 2,452 during the first year that the
new funding percentage was in effect.
RECOMMENDATIONS TO THE
ADMINISTRATOR, SBA
---------------------------------------------------------- Chapter 0:5
To improve interagency coordination and to reduce the risk of funding
duplicate research, GAO recommends that the Administrator, SBA, take
steps to (1) determine whether the certification form that
accompanies SBIR proposals needs to be improved and, if so, take the
necessary steps to revise it; (2) develop substantive definitions and
guidelines for agencies and companies regarding "duplicate" research;
and (3) provide interagency access to current information regarding
recent SBIR awards.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6
Officials from DOD, NASA, NIH, DOE, NSF, and SBA reviewed the report
and provided oral comments. In general, these officials regarded the
report as accurate and balanced. Officials from several agencies
stated that the duplicate funding problem should be viewed in the
context of the 20,000 or more proposals now being submitted annually.
In this context, officials believe that the problem is limited to
relatively few cases of blatant fraud, whereas the instances of
genuine confusion about what constitutes duplication may be somewhat
more frequent. They agreed that the problem should be addressed and
that the recommendations to SBA would be helpful. Agency officials
also provided minor technical corrections that have been incorporated
in the report. Brief, additional comments are presented at the end
of chapters 3 and 4. As requested by staff on the House and Senate
Small Business Committees, GAO did not obtain written comments on a
draft of this report.
INTRODUCTION
============================================================ Chapter 1
The Small Business Innovation Development Act of 1982, which
authorized the Small Business Innovation Research (SBIR) program,
emphasizes the benefits of technological innovation and the ability
of small businesses to transform the results of research and
development (R&D) into new products. The act designated four major
goals for the program:
Stimulating technological innovation.
Using small business to meet federal R&D needs.
Fostering and encouraging participation by minorities and
disadvantaged persons in technological innovation.
Increasing private-sector commercialization of innovations derived
from federal R&D.
Under the Small Business Research and Development Enhancement Act of
1992 (P.L. 102-564, Oct. 28, 1992), important changes were made to
the program, including greater funding because of its success and
greater emphasis on the goal of commercializing technical innovations
developed under the program. The program also added a provision for
discretionary technical assistance, which authorized agencies to help
companies with the technical and commercial aspects of their SBIR
projects.
THE ADMINISTRATION OF THE SBIR
PROGRAM
---------------------------------------------------------- Chapter 1:1
In addition to establishing goals, the original legislation
determined agencies' participation in and funding for the program.
Agencies spending more than $100 million annually for external R&D
were required to set aside not less than 1.25 percent of their total
R&D funds for the SBIR program. The 1992 reauthorization directed
agencies to increase the set-aside to not less than 1.5 percent in
fiscal years 1993 and 1994, not less than 2 percent in fiscal years
1995 and 1996, and not less than 2.5 percent in fiscal year 1997 and
thereafter. This requirement will effectively double the annual
funding for the program to about $1 billion. At present, 11 agencies
participate in the program. The five largest, accounting for over 90
percent of all SBIR awards, include the Department of Defense (DOD);
the Department of Energy (DOE); the Department of Health and Human
Services (HHS) and its National Institutes of Health (NIH), in
particular; the National Aeronautics and Space Administration (NASA),
and the National Science Foundation (NSF).
The original legislation required agencies to issue a solicitation
for proposals that sets the SBIR process in motion. The
solicitation, a formal document issued by each agency, lists and
describes the topics to be addressed by each company's proposals and
invites companies to submit proposals for consideration. Each agency
with an SBIR program is responsible for targeting research areas and
administering its own SBIR funding agreements. Such agreements
include any contract, grant, or cooperative agreement entered into
between a federal agency and any small business for the performance
of experimental, developmental, or research work funded in whole or
in part by the federal government.
The original legislation also required the Small Business
Administration (SBA) to issue policy directives for the general
conduct of the SBIR programs within the federal government. The
directives were to include such features as simplified, standardized,
and timely SBIR solicitations; a simplified, standardized funding
process; and minimization of the regulatory burden for small
businesses participating in the program. Issued in January 1993, the
current policy directive incorporated changes made by the 1992
legislation. Federal agencies were also required to report key data
to SBA, which in turn has published annual reports on the progress of
the program.
SBA's SBIR policy directive states that to be eligible for an SBIR
award, small businesses must be
independently owned and operated,
other than the dominant firms in the field in which they are
proposing to carry out SBIR projects,
organized and operated for profit,
the employer of 500 or fewer employees (including employees of
subsidiaries and affiliates),
the primary source of employment for the project's principal
investigator at the time of award and during the period when the
research is conducted, and
at least 51-percent owned by U.S. citizens or lawfully admitted
permanent resident aliens.
The law established a three-phase structure for the program. The
first phase, not to exceed 6 months, is designed to determine the
scientific and technical merit and feasibility of a proposed idea.
The second phase, not to exceed 2 years, is designed to further
develop the idea. The SBA policy directives established $50,000 and
$500,000 as the general limits for phase I and II awards,
respectively. The 1992 reauthorization directed SBA to raise these
figures to $100,000 and $750,000, respectively, with an adjustment
every 5 years to reflect economic and programmatic considerations.
The third phase is somewhat more flexible and difficult to define.
In general, it is expected to result in commercialization or further
continuation of R&D. Unlike phases I and II, phase III has no
general limits in time or dollar amounts. In addition, it can
include not only federal, non-SBIR funds but private-sector funds.
Regarding the selection of phase I proposals for SBIR awards, SBA's
1993 policy directive states that an agency's criteria shall give
primary consideration to scientific and technical merit along with
the potential for commercialization. According to the directive,
funding for phase II shall be based upon the results of phase I and
the scientific and technical merit and commercial potential of the
phase II proposal.
In reauthorizing the SBIR program, the Congress added a new feature
to the program. Section 301 of the Small Business Research and
Development Enhancement Act of 1992 provides for supplying
discretionary technical assistance to SBIR awardees. The provision
authorizes SBIR agencies to enter into an agreement with a vendor to
provide small business concerns engaged in SBIR projects with
technical assistance services, such as access to a network of
scientists and engineers engaged in a wide range of technologies, or
access to technical and business literature available through
databases. The purpose of this agreement is to assist SBIR awardees
in (1) making better technical decisions regarding such projects, (2)
solving technical problems that arise during the conduct of such
projects, (3) minimizing technical risks associated with such
projects, and (4) developing and commercializing new products and
processes that might result from such projects.
In funding this assistance, the provision authorizes not more than
$4,000 for phase I award recipients and $4,000 annually for phase II
award recipients. The source of the funding, however, is somewhat
different for each of the phases. For phase I, the legislation
specifies that the money shall be given in addition to the amount of
the recipient's award. For phase II, a recipient may purchase, with
funds available from SBIR awards, services in an amount not more than
$4,000 per year.
We have issued several reports on the SBIR program.\2 The 1992 report
dealt with the program's accomplishments in phase III.
--------------------
\2 Federal Research: Assessment of Small Business Innovation
Research Programs (GAO/RCED-89-39, Jan. 23, 1989), Small Business:
Proposed Amendments to the Small Business Innovation Research Program
(GAO/RCED-89-173, June 30, 1989), and Federal Research: Small
Business Innovation Research Shows Success but Can Be Strengthened
(GAO/RCED-92-37, Mar. 30, 1992).
OBJECTIVES, SCOPE, AND
METHODOLOGY
---------------------------------------------------------- Chapter 1:2
The Small Business Research and Development Enhancement Act of 1992
directed us to submit an interim report that would include an
assessment of the quality of the research performed under the SBIR
program's funding agreements entered into by each agency that has
participated in the SBIR program beginning in fiscal year 1993. More
specifically, the legislation required us to address, with respect to
each agency, whether a demonstrable reduction occurred in the quality
of research and, in the case of such reduction, whether an increase
in the agency's SBIR participation would adversely affect the
performance of the agency's research programs. The legislation also
required us to address the implementation of the Discretionary
Technical Assistance program. In contrast to the first two
objectives, which provided specific guidance for our work, the third
objective gave us greater latitude in selecting and investigating
additional issues that we deemed appropriate. We focused on the
issue of duplicate funding of similar proposals because of several
recent occurrences.
At the start of our review, the House Committee on Small Business
directed us to focus our work on SBA as the main administrative
agency for the program and on the five major funding agencies (DOD,
HHS, NASA, NSF, and DOE) that accounted for more than 90 percent of
the total expenditures for SBIR projects. Within HHS, our review
focused on NIH, which accounted for almost all of the SBIR program at
HHS.
In responding to the first objective, we began by reviewing our
earlier report that also addressed the issue of research quality in
the SBIR program.\3 In that report, we identified two ways of
approaching the concern about quality. One involved an extensive
questionnaire survey of project managers responsible for overseeing
both SBIR and non-SBIR research; the other involved the use of a key
indicator, the SBIR proposal selection rate, to show the level of
competitiveness in the program.
We decided that a questionnaire survey would be inappropriate for
this interim report because very little time has elapsed since the
reauthorization of the program. In fact, only the first (and
smallest) of the three scheduled increases in funding had occurred at
the time of our review. Consequently, we made use of the SBIR
proposal selection rate as a key indicator of the quality of research
proposals. We obtained data from the five major agencies for fiscal
years 1991-93 regarding the number of proposals and awards. These
years were selected because they provided an overview of the program
just before and just after the reauthorization. These data enabled
us to evaluate the effects of increased funding in fiscal year 1993
and compare them with the ratios in the previous 2 years. Where
available, we also obtained the number of proposals that agencies
deemed worthy of funding but that received no award. These data
provided additional evidence of the quality of research proposals and
competitiveness of the program. We further interviewed SBIR
officials, mainly SBIR directors and managers, in the major agencies
to obtain their views on factors affecting quality.
In responding to the second objective, we interviewed SBA officials
and SBIR officials in the major agencies regarding discretionary
technical assistance and any further assistance that they may have
provided. We also attended a meeting of SBIR agency officials,
convened by staff of the House Committee on Small Business, that
addressed agencies' concerns about implementing the technical
assistance provision.
In responding to the third objective, we focused on the problem of
duplicate funding. We met with officials in NSF's Office of
Inspector General (IG), who have investigated this matter. We
obtained documents from NSF, NASA, and Department of the Army
officials regarding the problem. We also met with computer
specialists from NSF, NASA, and SBA regarding their efforts to
resolve this problem.
We conducted our audit work between February 1994 and January 1995 in
accordance with generally accepted government auditing standards.
Our work was focused on the Washington, D.C., area.
--------------------
\3 GAO/RCED-89-39.
QUALITY RESEARCH PROPOSALS KEPT
PACE WITH INITIAL PROGRAM
EXPANSION
============================================================ Chapter 2
Although it is too early to make a conclusive judgment about the
effect of funding increases on the quality of SBIR research proposals
that received awards, the high level of competition and the large
numbers of worthy but unfunded projects suggest that quality research
proposals kept pace with the program's initial expansion. In
addition, SBIR officials in the five major agencies stated that in
their view, the quality of research proposals is being maintained.
COMPETITIVENESS REMAINED HIGH
---------------------------------------------------------- Chapter 2:1
Preliminary information suggests that the level of competitiveness
remained high following the initial increase in funding (from 1.25 to
1.5 percent) in fiscal year 1993. At all five major agencies during
fiscal year 1993, the number of proposals rose from 9 to 30 percent,
and the ratio of awards to proposals remained fairly constant,
ranging from 8 to 28 percent.
One of the best indicators of the continuing quality of research
proposals is the ratio of funded to unfunded proposals in the SBIR
program. We used this ratio as a key indicator in our January 1989
report on SBIR.\4 The report noted that the SBIR proposal selection
process was highly competitive because a large "pool" of proposals
was available for agencies to consider in selecting proposals that
meet standards of technical quality. During fiscal years 1983-87,
the percentage of phase I proposals winning phase I awards ranged
from 8 to 16 percent.
As a continuation of the results of this earlier study, we found a
generally large pool of unfunded proposals evident for each of the
major agencies during the last 3 fiscal years (fiscal years 1991-93).
The data show that the ratio of awards to proposals by four of the
five major agencies fits within the 8-to-16-percent range that we
found in our previous study of the program. (The one exception, NIH,
is discussed in detail later in this chapter.) In addition, among all
five agencies, the data for fiscal year 1993 show virtually no change
in the ratio from the previous 2 years, suggesting that the funding
increase in fiscal year 1993 exerted no adverse effect on the
competitiveness of the program.
Table 2.1 shows the number of phase I proposals submitted in fiscal
years 1991-93 to the five major SBIR agencies, the number of
proposals funded, and the percentage of proposals funded.
Table 2.1
Percentage of Phase I SBIR Proposals
Funded, Fiscal Years 1991-93
Percentage of
Fiscal Proposals Proposal proposals
Agency year submitted s funded funded
---------- -------- ------------ -------- --------------
DOD 1991 11,681 1,364 12
1992 9,911 1,176 12
1993 10,802 1,226 11
NIH 1991 1,788 468 26
1992 1,842 541 29
1993 2,132 588 28
NASA 1991 2,583 280 11
1992 2,535 301 12
1993 2,880 346 12
NSF 1991 1,498 177 12
1992 1,740 231 13
1993 2,201 284 13
DOE 1991 1,401 173 12
1992 1,534 198 13
1993 1,999 168 8
------------------------------------------------------------
The number of proposals submitted to all five agencies increased from
fiscal year 1992 to 1993. The increases ranged from a low of 9
percent at DOD to a high of more than 30 percent at DOE from fiscal
year 1992 to fiscal year 1993. The increases in fiscal year 1993
were important in maintaining the competitiveness of the program
during the first year that the program's funding percentage increased
to 1.5 percent. SBIR officials believe that there is further room
for growth in the number of proposals.
Among the five major agencies, DOD was the only one that experienced
a slight decrease in proposals submitted over the 3-year interval.
DOD's program director told us that the number of proposals received
annually has leveled off for about the last 3 or 4 years. He did not
know why the number of proposals had "plateaued." He noted, however,
that at national SBIR conferences sponsored by DOD and NSF, it was
typical for 30 to 60 percent of the attending companies never to have
participated in the program. DOD feels that this indicates that the
program has not yet run out of potential sources of proposals. DOD's
9-percent increase in proposals from fiscal year 1992 to 1993 may
indicate some movement beyond the plateau that the program director
noted.
--------------------
\4 GAO/RCED-89-39.
AGENCIES DEEMED MORE PROPOSALS
WORTHY OF AWARD THAN THEY WERE
ABLE TO FUND
---------------------------------------------------------- Chapter 2:2
SBIR officials told us that the large numbers of worthy but unfunded
projects indicated a continuing pool of quality proposals. DOD,
NASA, NSF, and NIH supplied specific information on this point.\5
In general, the data showed substantial reserves of projects deemed
worthy of funding but receiving no awards.
For example, the Air Force funded 470 projects in fiscal year 1993,
but it deemed 1,174 worthy of award. NASA funded only 346 of at
least 789 projects that were considered competitive. NSF made 284
awards among 642 that were recommended for funding. The number of
worthy but unfunded projects was smaller at NIH. SBIR officials
cited the projects in this category in support of their view that the
quality of research proposals has remained high. Additional details
are provided in the next section.
--------------------
\5 Two agencies within DOD (the Department of the Navy and the
Ballistic Missile Defense Organization) do not have a separate
category for projects deemed worthy of funding. Also, DOE does not
have a separate category for projects deemed worthy of funding.
Because of these limitations in DOD's and DOE's data, we did not
develop a more detailed table on projects "deemed worthy," but we do
discuss the available information for individual agencies below.
INDIVIDUAL AGENCIES STATED THAT
QUALITY WAS BEING MAINTAINED
---------------------------------------------------------- Chapter 2:3
Officials at individual agencies stated that the quality of research
proposals was being maintained or even improved. The one exception
involved NIH's National Cancer Institute (NCI), which we explored in
greater detail because of a potential shortage of qualified proposals
for funding in fiscal year 1993.
DOD
-------------------------------------------------------- Chapter 2:3.1
DOD's SBIR director does not believe that a decline in the quality of
research proposals has occurred as a result of SBIR's expansion. He
said that DOD aims for about a 1-to-10 ratio of awards to proposals
and that the program has maintained this ratio despite the changes in
funding.
The Air Force's SBIR program gave us a special opportunity to examine
the effects of a sharp increase in funding on the quality of
proposals selected for awards. The Air Force's SBIR manager noted
that the program experienced a 1-year "spike" in funding from $70
million in fiscal year 1992 to $134 million in fiscal year 1993.
According to the Air Force's SBIR manager, the data provided by the
four Air Force laboratories and the Air Force Office of Scientific
Research, which make the awards, indicated that the Air Force was
able to absorb this increase without lessening the competitiveness of
the program. The Air Force received 3,010 proposals in fiscal year
1993, deemed 1,174 worthy of award, and funded 470 projects (or 16
percent). A very large increase in the number of proposals helped
maintain quality and create a sizable reserve of projects deemed
worthy but unfunded. The number of proposals grew from 2,119 in
fiscal year 1992 to the 3,010 previously mentioned, an increase of
about 42 percent.
Data provided by the individual agencies within DOD showed a
consistently large reserve of projects deemed worthy of funding but
receiving no award. This pool of projects was evident, for example,
not only in the Air Force, but in the Army and the Advanced Research
Projects Agency. Among the Army's 2,840 proposals received in fiscal
year 1993, the agency found 750 worthy of funding and made 162 awards
(or 6 percent of submissions). In a similar vein, the Advanced
Research Projects Agency received 1,224 proposals in fiscal year
1993, deemed 563 worthy, and funded 151 (or 12 percent of
submissions).
NIH
-------------------------------------------------------- Chapter 2:3.2
Among the five major agencies, NIH was the only one in which we found
some cause for concern about the expansion of the program in fiscal
year 1993. The problem occurred primarily in NCI, the largest of the
20 institutes. NCI accounted for about 18 percent of the program at
NIH. Further information obtained for fiscal year 1994 suggests that
the difficulty experienced in fiscal year 1993 is being overcome.
Concerns about NIH's ability to absorb a funding increase have
surfaced before. During our work for our June 1989 report on SBIR,\6
which dealt in part with the advisability of a proposed increase in
the funding percentage, NIH's SBIR manager analyzed the program's
ability to expand. At that time, the manager concluded that a
decline in quality might result from the funding of any additional
projects because the number of unequivocally meritorious SBIR
applications was not large enough to absorb such an increase.
During fiscal years 1991-93, NIH's ratio of awards to proposals, at
about 28 percent, was more than twice as high as that of the other
agencies. Within NIH, NCI is the largest single institute, and its
SBIR funding ratio in fiscal year 1993 reached almost 50 percent. In
addition, NCI nearly exhausted all of the projects deemed worthy of
funding during fiscal year 1993. Because of this recurrence of a
concern about the quality of SBIR research proposals within NIH, we
looked more closely at NIH and at NCI, in particular.
In evaluating proposals, NIH officials group them into three basic
categories: not recommended for further consideration, eligible but
not funded, and actually funded. Each proposal is reviewed by a peer
review panel assembled by the Division of Research Grants. The panel
identifies the proposals that require no further consideration and
then rates the remainder on a scale from 100 to 500, with 100 being
the best score and 500 the worst. All of the projects on this scale
are considered worthy of funding, even if they have received one of
the less favorable ratings. The level of quality is defined in terms
of the following scores:
100 to 150: Outstanding.
150 to 200: Excellent.
200 to 250: Very good.
250 to 350: Good.
350 to 500: Acceptable.
The funding of projects with less favorable evaluations was
especially evident at NCI, although all of the projects fell into the
broad general category "considered worthy of funding." For phase I
awards by NCI in fiscal year 1993, the worst score receiving an award
was 399; the worst for phase II was 289. In contrast, the worst
score for phase I at the National Institute on Deafness and Other
Communication Disorders was 186; the worst for phase II was 167. In
other words, NCI funded projects in all five categories of quality,
whereas awards made by the National Institute on Deafness and Other
Communication Disorders were concentrated in the "outstanding" and
"excellent" categories. In addition, as shown by NIH's data, NCI
left only 29 of the 182 projects recommended for funding without an
award. NCI funded 40 of 42 proposals in the 301-to-350 range and 7
of 18 proposals in the 351-400 range. Fifteen proposals with scores
between 401 and 500 were left unfunded.
The Chief of NCI's Extramural Financial Data Branch discussed this
problem with us. He said that top NCI officials had expressed
concern about the quality of research proposals in relation to the
funds available in fiscal year 1993. Therefore, NCI staff conducted
an assessment to assure themselves that all of the projects selected
for awards were in fact worthy of funding. As a result of the
review, they concluded that all of the projects selected should have
been funded.
Further information provided by the Chief indicated that the fiscal
year 1993 problem was not recurring in fiscal year 1994. Of a total
of 568 proposals in fiscal year 1994, 308 were not recommended for
funding, 147 were considered worthy but not selected for an award,
and 112 received awards. Thus, according to these data, NCI selected
only about 20 percent of its proposals for awards and funded
considerably less than 50 percent of the projects considered worthy
of support. In addition, the data showed that the NCI phase I award
with the least favorable rating (298) in fiscal year 1994 represented
an improvement of 101 points over the least favorable rating for a
phase I award in fiscal year 1993.
NIH's director told us that in his view, the quality of R&D is being
maintained because of the increased number of proposals. A nearly
40-percent boost has occurred in the number of proposals received by
NIH as a result of the program's increased funding in the last year
alone.
The SBIR director believes that the increase in the size of the
individual awards will be especially important at NIH because it will
attract more proposals from somewhat larger small businesses. He
noted that NIH has three proposal receipt dates each calendar year,
which result in awards in the following fiscal year. Proposals
received during calendar year 1992 were awarded in fiscal year 1993
at a maximum level of $50,000 for phase I and $500,000 for phase II.
Therefore, although the program's expansion occurred in fiscal year
1993, proposals received for awards in that fiscal year were
restricted to the previous levels of $50,000 and $500,000,
respectively. The award level for phase I was increased to $75,000
(a 50-percent increase) for proposals received in calendar year 1993
and awarded in fiscal year 1994, while the size of phase II
awards--at $500,000--remained the same (except for unique and costly
projects involving special factors such as clinical trials).
However, proposals received during calendar year 1994 are being
solicited at the current levels of $100,000 for phase I and $750,000
for phase II. Thus, in fiscal year 1995, NIH will be making more
awards at substantially higher dollar values per award, a combination
that the SBIR director believes will encourage even more companies to
compete, including larger small business concerns.
--------------------
\6 GAO/RCED-89-173.
NASA
-------------------------------------------------------- Chapter 2:3.3
The SBIR director at NASA was optimistic that the expansion of the
program was not jeopardizing the quality of research proposals.
Several factors accounted for his optimism. He noted that the number
of applications for phase I awards has increased and that the ratio
of phase I awards made to proposals received has remained about the
same as in the past. He attributed the increased number of proposals
to various factors, including a more general awareness of the program
after 10 years and more money for the program.
The SBIR director also pointed out that a large pool of phase I
proposals was evaluated as being in the "competitive range"\7 during
fiscal years 1991-93 but received no award. In fiscal year 1993, for
example, of the 2,880 proposals submitted, at least 789 were
considered to be in the "competitive range," but the number may have
been as high as 952.\8 However, only 346 (or 12 percent) received an
award. Using the minimum figure of 789, slightly less than half (49
percent) of the proposals in the competitive range were actually
selected for awards. Thus, although the number of awards has grown,
the SBIR director believed that the level of competitiveness has
remained high and the quality of research proposals may well have
improved.
--------------------
\7 NASA interprets "competitive range" to mean those proposals that
have received the minimum acceptable or better scores. Typically,
according to SBIR officials, the proposals recommended for funding
are a subset of those in the competitive range and amount to about 30
percent more than the proposals that are actually funded.
\8 According to the SBIR manager, incomplete records accounted for
NASA's uncertainty about the exact number of proposals in the
"competitive range." He added that steps to correct this problem were
taken in mid-1994.
NSF
-------------------------------------------------------- Chapter 2:3.4
SBIR managers at NSF told us that NSF divides SBIR proposals into
three categories on the basis of quality--declined or not recommended
for funding, recommended if funding is available, and highly
recommended. In general, they said that about twice as many projects
are recommended or highly recommended as are actually funded. They
noted, however, that they were not even able to fund all of those
that were highly recommended.
They pointed out that NSF's SBIR program is seeing a continuing
increase in the number of proposals; hence, the competitive ratio of
awards to proposals is being maintained. In addition, data for
fiscal years 1991-93 indicated that a consistently large pool of
proposals was received and recommended for funding. In fiscal year
1993, 2,201 proposals were received, 642 were recommended for
funding, and 284 (or 13 percent) received awards. Less than half (44
percent) of the recommended proposals went on to win awards. In
general, NSF's SBIR managers indicated that even with a substantial
increase in the number of awards, they foresee no shortage of
projects worthy of funding.
DOE
-------------------------------------------------------- Chapter 2:3.5
The SBIR manager for DOE noted several factors that contributed to
the continuing quality of SBIR research proposals. He pointed out
that in fiscal year 1993, a temporary reduction in the number of
awards occurred because the additional funds went into making larger
individual awards rather than more awards in the aggregate. Thus,
DOE made 198 phase I awards in fiscal year 1992 and only 168 phase I
awards in fiscal year 1993. As shown in table 2.1, the percentage of
proposals receiving awards declined from 13 percent in fiscal year
1992 to only 8 percent in fiscal year 1993, thus making the program
even more competitive.
The SBIR manager also stated that when the major funding increase
occurs in fiscal year 1995, he anticipates that DOE would make about
the same number of awards (198) as in fiscal year 1992. In fiscal
year 1994, the size of DOE's phase II awards increased to $600,000,
and in 1995 it is expected to rise to $750,000. Once again, the
growth in the size of the individual awards will consume most, if not
all, of the additional funds.
At the same time, according to DOE's SBIR manager, the increase in
funding will probably lead to a rise in quality because more
proposals, drawn by the increased dollar amounts per award, would be
competing for the same number of awards. He noted that somewhat
larger small businesses would be more likely to participate as a
result of the larger awards; they would be more apt to add their own
funds to supplement the phase I awards in the hope of receiving phase
II awards. In general, he said that DOE is already rejecting very
good proposals and that the competitiveness of the program is
expected to continue to increase in the next several years.
CONCLUSIONS
---------------------------------------------------------- Chapter 2:4
Although it is somewhat early to draw conclusions because the major
funding increases will not occur until fiscal years 1995 and 1997,
several factors suggest that quality research proposals have kept
pace with the SBIR program's initial expansion. These factors
included the general increase in the number of proposals, the large
number of worthy but unfunded proposals, and the view of SBIR
officials that the quality of SBIR proposals is being maintained.
AGENCIES HAVE NOT IMPLEMENTED THE
PROVISION FOR DISCRETIONARY
TECHNICAL ASSISTANCE BUT HAVE
TAKEN OTHER STEPS TO FOSTER
COMMERCIALIZATION
============================================================ Chapter 3
None of the five major agencies have implemented the 1992
discretionary technical assistance provision, and future
implementation remains uncertain. SBA and SBIR officials were
critical of the provision because (1) it calls for the use of SBIR
funds, thereby reducing the number of awards the officials can make,
(2) there is little need for technical assistance when projects are
selected primarily for their technical merit, (3) implementation
would impose a significant administrative burden arising from
case-by-case considerations of companies' requests for support, and
(4) specific requirements, such as selecting a single vendor of
technical assistance for each agency, are unrealistic. However, some
SBIR officials have taken steps, independent of the provision, to
provide assistance with commercialization of research conducted by
SBIR companies.
SBIR OFFICIALS WERE CRITICAL OF
THE PROVISION
---------------------------------------------------------- Chapter 3:1
SBIR officials at all five of the major SBIR agencies as well as SBA
officials involved in the program were critical of the technical
assistance provision. Officials at several agencies, including SBA,
NASA, NIH, and NSF, questioned the use of SBIR funds for technical
assistance. SBA's Assistant Administrator of the Office of
Innovation, Research, and Technology pointed out that money for phase
I awardees would be drawn from the program, which would place the
need for technical assistance in competition with the need for SBIR
funds for other potential awardees. NASA's SBIR director told us
that NASA was reluctant to use money from the program itself because
it would reduce the funds available for additional awards. NSF's
SBIR managers shared this concern. NIH's SBIR manager noted that
certain other agencies have been able to provide assistance from
non-SBIR funds, which he considered preferable to using SBIR funds.
SBIR officials also expressed reservations about the appropriateness
of the technical (as contrasted with commercial) assistance for SBIR
awardees and emphasized the need among SBIR awardees for assistance
with commercialization. NASA's SBIR director told us that SBIR firms
in general do not need technical assistance. He pointed out that
among all SBIR agencies, the largest single portion (40 percent) of a
company's score in the selection process is based on technical merit
at the outset. A company would be eliminated from the competition if
any indications of technical inadequacies appeared. He added that if
a specific need for such assistance did arise after an award was
granted, an SBIR company would be able to obtain assistance on its
own without the need for special funding to seek such help. He also
indicated that in his view, commercial assistance was more
appropriate to the needs of awardees, who frequently require help in
finding ways to commercialize their research.
SBIR officials further indicated that implementation of the provision
for either technical or commercial purposes would impose a
significant administrative burden. SBA's Assistant Administrator
told us that implementation would lead to difficulties because there
would be a need for review and approval of each small award ($4,000
or less) to each awardee that requested funding under the provision.
Officials at DOD and NSF expressed similar concerns about the
potentially large number of requests and approvals that would be
involved.
Among other reasons preventing implementation, SBIR officials pointed
to unrealistic requirements in the provision. One example that they
cited is the clause limiting each agency to the selection of only one
vendor for meeting all technical and commercial needs. SBA's
Assistant Administrator described this arrangement as a virtual
impossibility. DOD's SBIR director told us that the only way one
vendor could succeed would be as a broker for a host of other
companies under subcontract. According to the DOD director, the
concept of one small company providing all of the services for
potentially hundreds of SBIR winners was otherwise impossible.
A second problem is the clause that requires an annual selection of
the vendor using competitive criteria. Several SBIR officials noted
this requirement and told us that this time frame was unrealistically
short. In their view, the process of issuing a request for proposal
and selecting a vendor would consume much of the time if it was
required annually.
EFFORTS TO OVERCOME PROBLEMS
MAY NOT BE SUCCESSFUL
---------------------------------------------------------- Chapter 3:2
A staff person with the House Committee on Small Business convened a
meeting in March 1994 to discuss these concerns and potential
solutions. Officials from two of the five major SBIR agencies--DOD
and NIH--attended the meeting along with officials from three of the
smaller SBIR agencies. The meeting explored various ways to
implement the provision, including a "team approach" that would allow
agencies to combine their efforts as well as specific technical
amendments, such as elimination of the requirement for selecting a
vendor annually. However, no firm plans or conclusions resulted from
the meeting.
In follow-up discussions, DOD's SBIR director and NSF officials told
us that a workable approach for implementing the provision would be
to permit companies to include the need for technical assistance
directly in their proposals as an allowable cost for phases I and II.
According to DOD's SBIR director, this approach could eliminate the
administrative problems that would result from attempting to make a
large number of small awards on a case-by-case basis. NSF's SBIR
managers agreed with this suggestion. They said that technical
assistance should be an allowable cost identified by a company in its
proposal and approved "up front" as part of the award. As of October
1994, no agency had explored the feasibility of this approach.
In October 1994, the Small Business Administration Reauthorization
and Amendments Act of 1994 (P.L. 103-403, sec. 607, Oct. 1994)
amended the discretionary technical assistance provision. The
amendment allowed the selection of a vendor to provide services for 3
years at a time. In our view, this technical amendment addressed one
specific problem but did not deal with the broad range of
difficulties that have made SBIR officials reluctant to implement the
provision.
AGENCIES HAVE TAKEN OTHER STEPS
TO FOSTER COMMERCIALIZATION
---------------------------------------------------------- Chapter 3:3
Independent of the technical assistance provision, agencies have
taken other steps to foster commercialization of research results.
Some of these efforts, such as a special training course conducted by
DOE for its phase II awardees and supported by non-SBIR funds, were
underway before the technical assistance provision was enacted.
Other efforts such as a new Navy initiative that makes the final 20
percent of phase II funding contingent on a company's development of
a commercialization plan were undertaken after the provision's
enactment.
For example, as discussed in our March 1992 report,\9 DOE has
sponsored a Commercialization Assistance Project for its phase II
awardees to enhance commercialization of research results by the
private sector. DOE began the project in 1989. As of September
1994, DOE had supported four special training sessions intended to
help its awardees market their SBIR-developed products. All sessions
were conducted by Dawnbreaker, a private firm from Rochester, New
York, that assisted individual phase II awardees in preparing a
business plan for potential sponsors and presenting it at a
conference to several dozen decisionmakers from large corporations
and venture capital firms.
DOE's SBIR manager has tracked the results of the project. He told
us that the second session, conducted in 1991, has proven very
successful. He noted that 43 percent of the companies participating
in the session have received phase III funding, which has totaled $14
million as of July 1994 and that an extra $24 million is expected
over the next 3 to 5 years. He also believes that the sessions in
1993 and 1994 will eventually prove successful but indicated that
more time is needed for results to emerge.
In DOD's commercialization efforts, we found several new initiatives.
The most striking was the special strategy adopted by the Navy's SBIR
manager. The Navy is emphasizing the importance of commercialization
by making a plan for such commercialization (both within DOD and in
the private sector) a requirement for receiving the last 20 percent
of each phase II award. This requirement went into effect in 1994.
Since funding for the Navy's SBIR phase II awards is set at about
$750,000, this figure amounts to $150,000 and is having a significant
effect on motivating companies to take commercialization seriously.
The Navy's SBIR manager said that the Navy retains the money until
the company develops a testing and evaluation plan or an R&D plan as
it relates to commercialization.
--------------------
\9 GAO/RCED-92-37.
CONCLUSIONS
---------------------------------------------------------- Chapter 3:4
SBA and agencies' SBIR officials cited a wide variety of reasons for
not implementing the technical assistance provision. These reasons,
in conjunction with the optional nature of the provision, make its
future implementation uncertain. Independent of the technical
assistance provision, however, agencies have taken other steps to
foster commercialization of research results.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 3:5
SBIR officials at DOD, NSF, and NASA told us that they are still
considering steps to implement the technical assistance option.
However, they noted the difficulties involved in implementing it and
remain uncertain whether the problems can be overcome.
Officials at DOD and NSF stated that among other steps taken by
agencies to foster commercialization, the joint DOD-NSF national
commercialization conferences have played an important role in
helping SBIR companies market their products. DOD and NSF sponsored
two of these conferences in 1993-94 and are planning another major
conference to be held in Anaheim, California, during March 1995.
DUPLICATE FUNDING OF RESEARCH HAS
BECOME A PROBLEM
============================================================ Chapter 4
Duplicate funding of similar proposals submitted to more than one
agency has become a problem. Several factors have contributed to
this concern: (1) the fraudulent evasion of the proposal
certification procedure, whereby companies fail to identify identical
proposals to other agencies; (2) the vagueness of key terms such as
"similar research" and the difficulty of defining them more
effectively, the result of which is that there can be disagreement
about what constitutes duplicate research; and (3) the lack of
interagency access to and exchange of current information about
recent awards. Basic changes in the SBIR program, particularly the
increasing number of proposals (about 20,000 in fiscal year 1993) and
awards, may make the program more vulnerable to this problem.
DUPLICATE FUNDING HAS OCCURRED
---------------------------------------------------------- Chapter 4:1
Agency officials informed us that they have investigations of SBIR
companies underway involving the duplicate funding of substantially
identical proposals by multiple federal agencies. A few cases are
under review by the Department of Justice for possible criminal and
civil prosecution. In one case, the Department of Justice filed an
action for trebled damages of $4.2 million under the False Claims
Act. According to that complaint, the SBIR company had fraudulently
obtained approximately $1.4 million in duplicate funding from NSF,
NASA, and various DOD agencies. The complaint alleges that the
company "recycled" 11 research ideas 40 times in duplicate
submissions.
During 1994, further work on this problem by agency officials found
evidence that other companies received duplicate funding. According
to agency officials, a few companies received funding for the same
proposals twice, three times, and even five times before agencies
became aware of the duplication. In these cases, the companies also
submitted equivalent reports at the end of their Phase I work without
informing agencies of the duplicative research.
EVASION OF THE CERTIFICATION
PROCESS CONTRIBUTES TO THE
PROBLEM
---------------------------------------------------------- Chapter 4:2
According to SBA's 1993 policy directive, it can be anticipated that
SBIR participants will submit duplicate or similar proposals to more
than one soliciting agency when the work projects appear to involve
similar topics or requirements. The guidelines in solicitations
issued by individual agencies for SBIR proposals also recognize the
potential for duplicate funding.
SBA's 1993 policy directive further states that the standardized SBIR
solicitation will require the proposers to indicate the name and
address of the agencies to which duplicate or similar proposals were
made and to identify by subject the projects for which the proposal
was submitted and the dates submitted. Agencies' individual
solicitations also include this requirement, although the language
varies somewhat from agency to agency. For example, NSF's
solicitation includes a brief section entitled "Equivalent Proposals
to Other Federal Agencies." NSF states that a firm may elect to
submit proposals to any other federal agency that are similar or
overlapping in technical content. It requires the firm to submit a
statement that includes the name and address of the agencies and
several other items of information. NASA's solicitation discusses
the issue under "Related Proposals to and Awards From Other
Agencies." NASA requires much the same information as NSF does.
DOD's SBIR director noted that DOD's solicitation refers to
"substantially similar" proposals and requires that the company
identify them in its response to the DOD solicitation.
In spite of these requirements, individual companies have submitted
similar proposals to more than one agency and then certified that
they had no similar proposals under consideration. In response to
this evasion of the certification requirement, NSF's IG officials
told us that they were concerned about the need for more complete
certification procedures. Such procedures would compel the
applicants in their proposals to certify, under criminal penalties
for perjury, exactly what, if any, applications for similar research
were pending at other agencies.
The view of NSF's IG was that the physical format of the
certification form may interfere with achieving this objective. The
signature of the applicant on the cover page of the NSF proposal was
placed in such a way that it was not clear whether the applicant was
actually certifying the statement about other proposals or was merely
providing further information. As one result of this confusion,
NSF's IG officials were concerned about the IG office's ability to
develop criminal prosecutions against companies that received
duplicate funding. The IG recommended that NSF's SBIR forms be
revised to clarify the applicability of the certification. NSF
agreed and implemented this recommendation. SBA's Assistant
Administrator told us that the forms in use in other agencies could
also be reviewed and, if necessary, revised and strengthened to
address potential problems with certification.
VAGUENESS OF KEY TERMS AND
DIFFICULTY OF DEFINING THEM
CONTRIBUTE TO THE PROBLEM
---------------------------------------------------------- Chapter 4:3
SBA's policy directive and individual agency solicitations do not
define key terms and thus provide no guidance in avoiding the risk of
duplicate funding. According to an SBA official, certain key
terms--such as "duplicate," "similar," "equivalent," "overlapping,"
"substantially similar," and "proposals of similar content"--occur in
the solicitations. However, little effort has been made to bring
them into the context of scientific research and give them a more
specific meaning.
One exception was the recommendation by NSF's IG that NSF include a
definition of the term "overlapping work" in its SBIR solicitation.
NSF's IG suggested that the term means "any steps in the performance
of work on one proposal which would not need to be repeated to
perform the work on the second proposal." The agency agreed to
implement this recommendation. NSF's SBIR director noted that the
scope and funding of some SBIR awards in 1994 were reduced to
eliminate overlapping work.
NASA's program manager discussed with us the problem of duplicate
research and the difficulty of defining key terms. He said that the
problem had come up often enough to be a concern even though, in his
estimate, it may have involved only 1 to 2 percent of the proposals.
He said that, in those cases, he has frequently heard a reviewer
comment that in essence, the proposal was only an extension of work
done for other agencies and represents nothing innovative. In some
cases, however, he said that the conclusion may have been a
subjective judgment; what one reviewer called duplicative or
noninnovative might impress another reviewer differently.
In fact, the vagueness of key terms can lead to differences of
opinion in their interpretation by federal and company officials. In
one case, SBIR officials became concerned about potential
duplication, while a company contended that no duplication had
occurred. This example was furnished by a company that had won phase
I awards from NASA and the Army. The same company's proposals were
also selected for phase II awards by both agencies, but NASA
rescinded the selection in April 1994 after becoming aware of the
company's phase I and proposed phase II work for the Army. The Army,
however, provided a phase II award in June 1994.
The potential similarity is evident from the title of the respective
phase II proposals. The proposal to NASA involved "An Optical
Instrument to Measure Cloud Liquid Water Content and Droplet
Spectra." The proposal to the Army involved "A New Instrument for
Automatic Measurement of Cloud Liquid Water Content and Droplet
Size." According to an April 1994 letter from NASA's SBIR director to
the company, each agency awarded phase I contracts and selected phase
II proposals for contract negotiations without prior knowledge of the
work proposed to or funded by the other agency.
In the same letter, NASA's SBIR director stated to the company that
"The proposed [Army] project is a terrestrial instrument while the
proposed NASA project is an airborne instrument . . . [B]oth
instruments are based on the same technological innovation and should
have substantially the same characteristics and capabilities."
The SBIR director also stated that
"Self-certification by SBIR offerors on specific requirements is
basic to the SBIR process, and accuracy and completeness are
essential to program integrity. The omissions and inaccurate
certifications . . . resulted in our not knowing the
potential for duplication of similar work funded by another
agency. The existence of the [Army] contract was never
disclosed by your company to NASA; it was discovered
accidentally by the NASA project manager after phase I had been
completed."
The letter concluded that "Based on our findings and concerns, NASA
will not fund the phase II project."
The company responded with a detailed letter that explained the
technical differences between the two proposals and commented on the
problem of definition. In its letter, the company said, "We are
having difficulty understanding your definition of 'innovative,'
particularly as it is interpreted by different agencies and different
centers/laboratories within the agencies." The company added, "Our
opinion is that the airborne and terrestrial instrument each requires
separate innovative technology and that the instruments are not the
same technology in two different embodiments."
In response to the program director's concern about "inaccurate
certifications," the company stated, "We did not interpret the NASA
and [Army] phase II proposals as being similar proposals . . . or
that we had received funds for substantially similar work. The focus
and thrust of the two phase I projects were substantially different."
The company concluded that "There were no omissions or inaccurate
certifications."
An Army official--a physical research scientist who had served as the
technical manager for the Army's phase II award--agreed with NASA's
views about the certification problems. She said that the company
should have identified a similar proposal for NASA. (She noted that
the company cited other SBIR work but did not refer to the concurrent
SBIR work for NASA.) Although she criticized the company in this
regard, she concluded that its work was of value to the Army and that
the certification problems were not a sufficient reason to reject the
phase II proposal.
In general, the absence of substantive definitions for terms such as
"innovative" or "similar" research places the burden of judgment on
the company. The company must then certify its proposals as original
or duplicative without guidance. In cases such as the one described
involving two different instruments with a possibly similar
technology, the appropriate certification may be difficult to
determine and can lead to conflicts of opinion that may harm the SBIR
program as well as the individual company.\10
--------------------
\10 During our meeting with NSF officials in January 1995 to discuss
our draft report, NSF's SBIR director noted that NSF had also made a
phase II award to the same company for an airborne instrument
designed to measure the sizes of cloud droplets. The SBIR director
has referred the matter to NSF's IG for review.
LACK OF CURRENT INFORMATION
CONTRIBUTES TO THE PROBLEM
---------------------------------------------------------- Chapter 4:4
The lack of interagency access to and exchange of current information
about SBIR awards contributes to the problem of duplicate funding.
At present, SBA maintains a database for the SBIR program that it
uses primarily to produce its annual report to the Congress regarding
the program. The information has a "time lag" of about 9 months
because it is first processed by each agency and then forwarded to
SBA. SBA requires further time to review and "clean up" the data for
its own use. Individual agencies maintain records of recent awards,
but this information is generally not available to other agencies.
If an official in one agency wants to obtain information from another
agency about a specific proposal or company, such information is
available only through personal contacts and conversations.
DOD's SBIR director discussed with us the existing methods of
coordination within DOD and among other agencies to prevent duplicate
funding problems. He said that where there is a potential overlap,
such as between DOD and NASA, SBIR officials coordinate their reviews
and thus attempt to avoid duplicate funding. As another example, the
Air Force's SBIR manager regularly sends information on
environmentally oriented awards to the Environmental Protection
Agency to identify potential overlaps.
Some SBIR officials believe that the present methods may not be
adequate for detecting duplication when dealing with 20,000 proposals
annually. Officials at NASA and SBA have led the initial efforts to
improve interagency access to and exchange of current information. A
Senior Research Computer Scientist at NASA helped to develop software
for NASA's SBIR program that would facilitate the handling and review
of proposals, but other SBIR officials expressed the view that the
NASA software may not be easily adaptable to other agencies'
programs. As a result, SBA's Technology Transfer and
Commercialization Specialist, who manages SBA's SBIR database, met in
March 1994 with technical officials in the larger SBIR agencies to
discuss an alternative approach that would meet the needs of SBIR
agencies in general. He also convened a second meeting in September
1994 to assemble what he described as a definitive list of features
in the new approach.
SBA's specialist emphasized that the effort now underway will not
change SBA's existing SBIR database but will supplement it with a
list of all current awards. He said that the data may be made
available on Internet--each agency would maintain its own database
and provide access to other agencies--rather than on a centralized
database at SBA. However, he has not yet finalized this plan. SBA's
specialist told us that the new approach will overcome the time lag
regarding information on current awards and help in avoiding
duplicate funding.
The effort to provide interagency access to this information,
however, has raised a question about the protection of proprietary
information. DOD's SBIR director was especially concerned about this
problem. SBA's specialist said that he was aware of DOD's concern
but that steps to ensure the protection of proprietary information
would be taken.
In general, efforts to provide interagency access to current
information are at an early stage of planning. Currently, minimal
documentation exists to describe the proposed approach.
EXPANSION MAY MAKE THE SBIR
PROGRAM MORE VULNERABLE TO THE
PROBLEM
---------------------------------------------------------- Chapter 4:5
The increasing number of proposals and awards is likely to enhance
SBIR's vulnerability to duplicate funding. The five major SBIR
agencies experienced an increase from 17,562 phase I proposals in
fiscal year 1992 to 20,014 in fiscal year 1993. SBIR officials also
believe that there will be further growth in the number of proposals.
While this increase would help maintain the competitiveness of the
program, it would also heighten agencies' likelihood of receiving
duplicate proposals and their difficulty in identifying duplicate
proposals if companies do not report the proposals.
CONCLUSIONS
---------------------------------------------------------- Chapter 4:6
At present, the SBIR program is at a growing risk of willful or
accidental financial abuse. The certification process can be evaded
and, as noted by NSF's IG, the certification form may not be
sufficiently well designed to support a criminal prosecution. The
absence of definitions of key terms such as "innovative" or "similar"
research places the burden of judgment on companies at the same time
that it leaves them without effective guidance in determining
duplicate research proposals. In addition, the lack of interagency
access to and exchange of current information about awards leaves
SBIR officials with only an informal means of coordinating their work
and identifying potential duplication. Further increases in funding
and the number of proposals put the program at even greater risk in
the future.
RECOMMENDATIONS TO THE
ADMINISTRATOR, SBA
---------------------------------------------------------- Chapter 4:7
To improve interagency coordination and to reduce the risk of
duplicate funding of similar research, we recommend that the
Administrator, SBA, take steps to (1) determine whether the
certification form that accompanies SBIR proposals needs to be
improved and, if so, take the necessary steps to revise it; (2)
develop substantive definitions and guidelines for agencies and
companies regarding "duplicate" research; and (3) provide interagency
access to current information regarding SBIR awards.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 4:8
The concern of officials in several agencies, including SBA, NASA,
NSF, and DOD, was that the duplicate funding problem be kept in
perspective. They believe that the problem is limited to relatively
few cases of blatant fraud, while the instances of genuine confusion
about what constitutes duplication may be somewhat more frequent.
DOD's SBIR director noted that during the final week before a DOD
SBIR solicitation closed in January 1995, he received calls from
about a dozen companies. Each company requested guidance in
responding to DOD's certification requirement because the company was
uncertain of whether its proposals were duplicative. DOD's SBIR
director told the companies that if they thought there was a chance
of duplication, they should indicate it on the certification form.
In general, agency officials agreed that the problem of duplicate
funding should be addressed and that the recommendations to SBA would
be helpful.
MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix I
RESOURCES, COMMUNITY, AND ECONOMIC DEVELOPMENT DIVISION,
WASHINGTON, D.C.
Jim Wells, Associate Director (currently with Housing and Community
Development Issues Group)
Robin M. Nazzaro, Assistant Director
Dennis Carroll, Evaluator-in-Charge