TITLE: Noble Solutions, B-294393, September 10, 2004
BNUMBER: B-294393
DATE: September 10, 2004
**********************************************************************
Decision
Matter of: Noble Solutions
File: B-294393
Date: September 10, 2004
Henry Noble for the protester.
Craig S. McCaa, Esq., Department of the Air Force, for the agency.
Linda S. Lebowitz, Esq., and Michael R. Golden, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Agency reasonably determined not to select protester's proposal for a
Phase I SmallA Business Innovation Research contract where the agency
reasonably evaluated the protester's proposal as technically unacceptable
and the protester has not meaningfully challenged the agency's evaluation.
DECISION
Noble Solutions protests the evaluation of its proposal under solicitation
No. FY04.1, issued by the Department of Defense (DOD) for the Small
Business Innovation Research (SBIR) program. Noble, whose proposal was not
selected for award, challenges the agency's evaluation.
We deny the protest.
The SBIR program is conducted pursuant to the Small Business Innovation
Development Act, 15 U.S.C. S 638 (2000), which requires certain federal
agencies to reserve a portion of their research and development funds for
awards to small businesses. In addition to advancing the role of small
businesses and the participation of minority and disadvantaged persons in
research and development, the objectives of DOD's SBIR program include
stimulating technological innovation in DOD's critical technology area and
increasing the commercial application of DODa**supported research and
development results. The program has the following three phases: Phase I
is to determine the scientific, technical, and commercial merit of ideas;
Phase II is the principal research and development effort resulting in a
well-defined, deliverable prototype; and in Phase III, the small business
seeks to obtain private and public funding to develop the prototype into a
viable commercial product for sale to military and/or private sector
markets. Solicitation PP 1.1, 1.2.
This solicitation sought proposals for Phase I awards to be made by a
number of participating DOD component agencies and offices, including the
Department of the Air Force. Each component agency identified technical
topics. Phase I awards under Air Force topic No. AF04-087, entitled
*Expert Intelligent Match of Requirements and Solutions,* are at issue
here. As explained in the solicitation, the objective under this topic is
to develop the ability to match descriptions of warfighter requirements to
relevant descriptions of proposed solutions. More specifically, the
solicitation explained that the government is developing a database that
will include both information warfare (IW) warfighter requirements and IW
solutions. The referenced database will use a set of multifaceted IW
requirements that cut across 11 distinct areas, as follows:
counterintelligence; computer network attack; computer network defense;
electronic warfare; information assurance; integration; military
deception; operations security; physical attack; public affairs
operations; and psychological operations. The solicitation noted that the
total number of requirements at any one time is usually between 100A and
150, and that complex IWA solutions that number in the hundreds also would
be included in the database. The solicitation explained that proposed IW
solutions could be relevant to multiple requirements in multiple IW
areas. Noting that matching warfighter requirements to solutions could be
a daunting and timea**consuming effort when done by individuals, the
solicitation stated that what was needed was a computer-based means of
comparing IW requirements to the database collection of solutions to find
those that were relevant to the requirements. The solicitation pointed
out that simple, literal, worda**based searches would not be adequate for
this effort because the keywords used might not be found in every IW
solution document. Solicitation, Topic Document AF04-087.
The solicitation outlined the following four stated purposes for Phase I:
(1) to investigate emerging and existing relevant methods of intelligent
search; (2) to define the approach that would be used to intelligently
search for matches between requirements and solutions; (3)A to define a
method of measuring performance of the search; and (4) to develop an
interface prototype. Id.
The solicitation contemplated multiple awards of fixed-price Phase I
contracts (generally not to exceed $100,000 each) to those offerors whose
proposals represented the best values to the government. The solicitation
provided that proposals would be evaluated by scientists or engineers
knowledgeable in the topic area. The solicitation listed the following
three technical evaluation factors: (1) the soundness, technical merit,
and innovation of the proposed approach and its incremental progress
toward topic or subtopic solution; (2) the qualifications of the proposed
principal/key investigators, supporting staff, and consultants, including
not only the ability to perform the research and development, but also the
ability to commercialize the results; and (3)A the potential for
commercial (government or private sector) application and the benefits
expected to accrue from this commercialization. The solicitation also
stated that where proposals were essentially equal in technical merit,
cost to the government would be considered in determining the successful
offerors. The solicitation advised offerors that the technical
evaluations would be based only on the information contained in the
proposals. Solicitation PPA 4.1, 4.2.
In response to the solicitation, the Air Force received 26 proposals,
including proposals from Stottler Henke Associates, Inc., 21st Century
Technologies, Inc., and Noble. The proposals were evaluated by the Air
Force Research Laboratory's Information Directorate, which assigned point
scores for each of the technical evaluation factors; these point scores
were supported by narratives of the strengths and weaknesses in each
proposal. (The evaluation document for each proposal was prepared by the
lead technical evaluator who was also the author of Air Force topic No.
AF04-087.) Noble's proposal, which was based in part on developing IW
warfighter search scenarios, was ranked fifteenth overall, with a total
score of 77A out of a possible 100 points, and was determined to be
technically unacceptable essentially because Noble failed to demonstrate
in its proposal a clear understanding of the referenced Air Force topic.
The proposals of Stottler (ranked first overall) and 21stA Century (ranked
second overall) received 100A points and 95A points, respectively, and
both of these proposals were determined to be technically acceptable.
(The price differences between these three proposals were de minimis. In
this respect, Noble's price was $9 higher than Stottler's price and $282
lower than 21st Century's price.) Legal Memorandum at 3; Contracting
Officer's Statement at 2.
After being notified that its proposal was not selected for a Phase I
award, Noble submitted an agencya**level protest in which it requested
that the agency address four issues related to the evaluation of its
technical proposal. In preparing its response to Noble's agency-level
protest, the agency gave Noble's proposal to another member of its
technical staff, an individual who did not participate in the initial
evaluation of Noble's proposal, for the purpose of independently
reevaluating Noble's proposal. This individual, who has a doctorate
degree in computer science, a masters degree in electrical engineering,
and a bachelors degree in mathematics/physics, reevaluated Noble's
proposal, assigning the proposal a score of 30 points. In describing why
he believed that Noble's proposal was technically unacceptable, this
individual concluded that *[o]verall[,] the proposal lacks sufficient
details to understand the proposed approach and whether the approach is
viable.* Agency Report (AR), TabA 13, Independent Reevaluation Report for
Noble's Proposal, at 3. As relevant to this protest, this individual
basically commented that Noble failed to clearly describe those aspects of
its proposal involving the development of IW warfighter search scenarios.
Id. at 2-3.
The agency subsequently denied Noble's agency-level protest. This protest
followed. In responding to this protest, the agency included in its
administrative report a copy of Noble's proposal, as well as the narrative
evaluation reports for the initial evaluation and the independent
reevaluation of that proposal. In addition, in response to a request from
our Office, the agency furnished for our inA camera review the proposals
of Stottler and 21st Century and the narrative evaluation documentation
for these two proposals.
Where an agency is conducting an SBIR procurement, the agency has the
discretion to determine which proposals it will fund. The agency must act
reasonably in deciding which proposals to fund. Glatz Aeronautical Corp.,
B-293968.2,
Aug. 10, 2004, 2004 CPD P ___ at 3; R&D Dynamics Corp., B-285979.3, Dec.
11, 2000, 2000 CPD P 201 at 4. Here, we have no basis to question the
agency's decision not to select Noble's proposal for a Phase I award.
As a threshold matter, we point out that in its comments on the agency's
administrative report, Noble failed to meaningfully respond to any
position taken by the agency regarding the technical merit of its
proposal. Rather, Noble merely expressed disagreement with the agency's
technical evaluation, thereby failing to show that the agency's evaluation
was unreasonable or otherwise not in accordance with the terms of the
solicitation. Mere disagreement is not sufficient to render an evaluation
unreasonable. See Bevilacqua Research Corp., B-293051, Jan.A 12, 2004,
2004 CPD P 15 at 8 n.8.
In its comments, Noble *challenge[s] the Air Force to explain the
reasonableness of failing to mention even once a most critical component
of [its] proposal for AF04-87, which is not only shown pictorially (Figure
4-2b of AF04-087 Proposal), but mentioned more frequently than the lesser
component (Focused Crawling), which the evaluation elected to address.*
Protester's Comments at 1. The record shows that the referenced graphic
is titled *IW War-Fighter Search Scenarios (IWSS) The Power and Secret*
and contains various technical terms, for example, *Focused Crawling,* *IW
War-Fighter Scenarios,* and *Search Relevance,* and a number of
directional arrows. Noble's Proposal at 7. However, it was the agency's
view, and we concur based on our review of Noble's proposal, that Noble
failed to meaningfully explain in its proposal its proposed technical
approach. While Noble is correct that the documentation for the initial
evaluation did not expressly refer to *IW War-Fighter Search Scenarios,*
we do not view this omission as negating the agency's evaluation that
Noble failed to demonstrate in its proposal a clear understanding of the
Air Force topic since, for example, Noble did not meaningfully explain the
referenced graphic. Noble also ignores the fact that the documentation
for the independent reevaluation of its proposal specifically addressed
the search scenarios aspect of its proposal. In this regard, the
independent evaluator[1] recognized that Noble proposed to develop search
scenarios, but, in the absence of any meaningful explanation, he concluded
that it was not clear from Noble's proposal what these scenarios even
were. Noble does not dispute the conclusion contained in the
documentation of the independent reevaluation that *[o]verall[,] the
proposal lacks sufficient details to understand the proposed approach and
whether the approach is viable.* AR, Tab 13, Independent Reevaluation
Report for Noble's Proposal, at 3. In sum, an offeror, like Noble, is
responsible for providing a full discussion of its technical approach and
methodology within the four corners of its proposal and it is not
unreasonable for an agency to downgrade a proposal because the proposal
lacks a detailed discussion of an offeror's proposed approach. See, e.g.,
Wyle Labs., Inc., Ba**260815.2, Sept. 11, 1995, 95-2 CPD PA 187 at 5.
In contrast to Noble's proposal, as discussed above, the record shows that
the proposals of the two firms awarded Phase I contracts contained
detailed descriptions of their proposed approaches for identifying
potential solutions for warfighter requirements. In their respective
proposals, which we reviewed inA camera, Stottler and 21st Century
provided detailed descriptions of their Phase I work plans and
comprehensively discussed, for example, their proposed methodologies,
tasks, and milestones. In assigning the maximum of 100A points to
Stottler's proposal, the agency concluded that Stottler's proposal was
*innovative yet pragmatic,* and contained an *[e]xcellent technology
review, corporate technical history, technical approach, and presentation
of expected product performance.* AR, Supplemental Documentation,
Evaluation Report for Stottler's Proposal, at 2. In assigning 95 points
to 21st Century's proposal, the agency concluded that while there were
some weaknesses and risks associated with this firm's proposal, 21st
Century nevertheless proposed *very good technology.* AR, Supplemental
Documentation, Evaluation Report for 21st Century's Proposal, at 1.
On this record, we have no basis to question the reasonableness of the
agency's evaluation of Noble's proposal as technically unacceptable and
its decision not to select Noble's proposal for the award of a Phase I
contract.
The protest is denied.
Anthony H. Gamboa
General Counsel
------------------------
[1] Noble alleges in its comments that, in responding to its agency-level
protest, the agency could not have performed an independent reevaluation
of its proposal because the individual who performed the reevaluation
worked in the same research laboratory as the lead technical evaluator for
the initial evaluation.
To the extent Noble's allegation can be read as one challenging the
selection of individuals to serve as proposal evaluators, this is a matter
within the discretion of the agency; accordingly, we will not review
allegations concerning the qualifications of evaluators or the composition
of evaluation panels absent a showing of possible fraud, conflict of
interest, or actual bias on the part of evaluation officials. Glatz
Aeronautical Corp., supra, at 3 n.1; CAE USA, Inc., B-293002,
Ba**293002.2, Jan. 12, 2004, 2004 CPD P 25 at 10-11 n.8; Solid Waste
Integrated Sys. Corp., B-258544, Jan.A 17, 1995, 95-1 CPD P 23 at 6.
Here, Noble has not made the required showing. In this respect, the
solicitation stated that proposals would be evaluated by scientists or
engineers knowledgeable in the topic area. Noble does not challenge the
technical qualifications or expertise of the evaluators. Moreover, the
fact that the individual selected by the agency to independently
reevaluate Noble's proposal was located at the same research laboratory as
the lead technical evaluator for the initial evaluation does not
constitute evidence of fraud, conflict of interest, or actual bias.