TITLE: Kolaka Nòeau, Inc., B-291818, April 2, 2003
BNUMBER: B-291818
DATE: April 2, 2003
**********************************************************************
Kolaka Nòeau, Inc., B-291818, April 2, 2003
Decision
Matter of: Kolaka Nòeau, Inc.
File: B-291818
Date: April 2, 2003
Eugene L. Duke for the protester.
Vincent A. Salgado, Esq., National Aeronautics & Space Administration, for
the agency.
Guy R. Pietrovito, Esq., and James A. Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest that an agency did not select protester's proposal for a phase I
Small Business Innovation Research contract is denied, where the agency
reasonably evaluated the protester's proposal and where, although the
proposal was recommended for award, the proposal was reasonably not as
highly ranked as other proposals for which the agency had sufficient
funding to make awards.
DECISION
Kolaka Nòeau, Inc. protests the rejection of its proposal by the
National Aeronautics & Space Administration (NASA) under the agency's 2002
Small Business Innovation Research (SBIR)/Small Business Technology
Transfer (STTR) program solicitation No. 2002-1.
We deny the protest.
The SBIR program is designed to increase the participation of small
business concerns in federally funded research or research and development
(R&D). See Small Business Innovation Research Program Act of 1982, 15
U.S.C. S: 638 (2000). Pursuant to this authority, federal agencies (such
as NASA) with R&D *extramural* budgets in excess of $100 million are
required to provide a program under which a portion of the agency's
research or R&D effort is reserved for award to small business concerns
through a three-phased process. See 15 U.S.C. S: 638 (e)(4), (f). Under
phase I, small businesses are invited to submit proposals to conduct
research on one or more topics specified in the annual SBIR program
solicitation. Under phase II, firms that received phase I awards may, on
their own initiative, submit proposals for further development work on the
topic. Phase III contemplates that, unlike phases I and II, non-SBIR
funds will be used to pursue commercial applications of the R&D.
NASA issued a program solicitation inviting small business concerns to
submit proposals for fixed-price SBIR contracts under the following
research areas: aerospace technology, biological and physical research,
earth science, human exploration and development of space, and space
science. Topics and subtopics were identified for each area. There were
a total of 28 topics and 120 subtopics.
Under the aerospace technology area, the solicitation identified the
*engineering for complex systems* topic, and, under that, the *modeling
and simulation of aerospace vehicles in a flight test environment*
subtopic. Program Solicitation S: 9.1.1, Aerospace Technology, at 50,
70. Detailed information was provided for each area, topic, and
subtopic. With respect to the modeling and simulation of aerospace
vehicles in a flight test environment subtopic, on which the protester
submitted its proposal, offerors were informed that the agency sought more
efficient software tools for predicting and understanding airframe
response *under the simultaneous influence of aerodynamics and the control
system, in addition to pilot commands.* Id. at 70.
Each subtopic was assigned to one of NASA's 10 field centers for
evaluation. NASA's Dryden Flight Research Field Center was identified as
the lead for four subtopics and a participating center for another
subtopic. Dryden was the lead center for the *modeling and simulation of
aerospace vehicles in a flight test environment* subtopic.
Offerors were informed that the phase I contracts could be for no more
than $70,000 and 6 months. Detailed instructions for the preparation of
proposals were provided. The solicitation provided that proposals for
phase I contracts must be based on a *unique innovation, . . . be limited
in scope to just one subtopic, and . . . be submitted only under that
subtopic.* Id. S: 3, at 5. Format and page limitations were stated for
proposals, which were required to be submitted via the Internet. In their
technical proposals, offerors were required to, among other things,
identify the significance of the innovation proposed, state technical
objectives, detail the proposed work plan, and identify key personnel.
Offerors were also informed that NASA would select for award those
proposals offering *the best value to Government and the Nation* and that
*primary consideration [would be given to] scientific and technical merit,
and feasibility of the proposal, and its benefit to NASA.* Program
Solicitation S: 4.1.2, Phase I Evaluation Criteria, at 14. In this
regard, offerors were informed that evaluations would be performed *by
NASA scientists and engineers and by qualified experts outside of NASA.*
Id. S: 4.1.1, at 13.
The following technical evaluation factors were identified: (1)
scientific/technical merit and feasibility; (2) experience, qualifications
and facilities; (3) effectiveness of the proposed work plan; and (4)
commercial merit and feasibility. The RFP stated that factors (1) through
(3) would be point scored and that factor (4) would be adjectivally
rated. Factor (1) was stated to be worth 50 percent of the total
evaluation weight, and factors (2) and (3) were worth 25 percent each; the
sum of a proposal's point scores for these factors comprised the
proposal's technical merit score. The technical merit score was stated to
be more important than commercial merit. Offerors were also informed that
non-cost factors were substantially more important than cost. Id. S:
4.12, at 14.
The RFP provided that after evaluation of the proposals under the
evaluation factors, proposals would be ranked relative to all other
proposals and that *[s]election decisions will consider the
recommendations from all Centers, Strategic Enterprises, overall NASA
priorities, and program balance.* Program Solicitation S: 4.1.3, at 14.
Offerors were also cautioned that awards would be subject to availability
of funds and that there was no obligation to make a specific number of
awards under the solicitation.
In response to the RFP, 2,238 proposals were received. Dryden received
108 phase I proposals to evaluate, spread over the five subtopics for
which Dryden had responsibility. Under the *modeling and simulation of
aerospace vehicles in a flight test environment* subtopic, Dryden received
three proposals, including the protester's. Agency Report, Tab 6, Dryden
Phase I SBIR/STTR Proposal Ranking, Oct. 22, 2002, at 2.
Each proposal was reviewed by at least two evaluators with knowledge of
the subtopic area. Kolaka's proposal received the following evaluation
scores:
+------------------------------------------------------------------------+
| |Evaluator A |Evaluator B |
|----------------------------------------------+------------+------------|
|(1) Scientific/technical merit and feasibility|42 |44 |
|----------------------------------------------+------------+------------|
|(2) Experience, qualifications and facilities |22 |23 |
|----------------------------------------------+------------+------------|
|(3) Effectiveness of the proposed work plan |18 |23 |
|----------------------------------------------+------------+------------|
|TECHNICAL MERIT POINTS |82 |90 |
|----------------------------------------------+------------+------------|
|(4) Commercial merit | | |
|----------------------------------------------+------------+------------|
| |NASA |Excellent |Very Good |
| |--------------------------+------------+------------|
| |Non-NASA |Very Good |Very Good |
+------------------------------------------------------------------------+
Agency Report, Tabs 5a, 5b, SBIR Phase I Technical Evaluation Forms.
Kolaka's proposal (priced at $69,998) received an average evaluation score
of 86 technical merit points and *very good* commercial merit scores, and
was the second highest rated of the three proposals received under this
subtopic. The highest-rated proposal (priced at $69,995) under this
subtopic received an average technical merit score of 97.5 points and
*very good* commercial merit scores, and the lowest‑rated proposal
(priced at $69,972) received a technical merit score of 76.5 points and
commercial merit scores of *below average* (NASA) and *average*
(non-NASA). Kolaka's and the highest-rated offeror under this subtopic
were recommended for award by the evaluators who rated these proposals;
the lowest-rated proposal was not recommended for award.
The evaluation results were presented to subtopic managers to rank the 25
proposals that were recommended for award.[1] The subtopic managers (or,
in some cases, another technical expert) presented the proposals
recommended for award (including Kolaka's) to a center ranking committee
at Dryden. The presentations were comprised of brief summaries of each
proposal and the evaluation results. In ranking proposals, the Dryden
center ranking committee established two sets of proposals. It first
ranked (from first through fifth) the highest-rated proposal from each of
the five subtopics for which Dryden was responsible; the highest-rated
proposal in the subtopic for which Kolaka submitted its proposal was
ranked third by the committee. The remaining twenty proposals, including
Kolaka's, were then ranked between sixth and twenty-fifth. Kolaka's
proposal was ranked twentieth by the committee. Contracting Officer's
Statement at 5; Agency Report, Tab 10, Final Dryden Center Proposal
Ranking.
Dryden and the other NASA centers' rankings and recommendations were then
presented to NASA's program management office, which reviewed the
selection recommendations and presented the evaluation results to the
source selection authority (SSA). Contracting Officer's Statement at 6.
The NASA centers recommended a total of 919 proposals for award. Based
upon the estimated available budget, the program office ultimately
recommended that 270 proposals be selected for award.[2] Contracting
Officer's Statement at 6; Agency Report, Tab 11, Presentation to SSA,
at 6. In addition to providing the evaluation results, proposed prices,
and disadvantaged or women-owned business status for recommended
proposals, the program management office presented to the SSA the number
and percentage of recommendations made by each center, the number of
subtopics led by each center, and the number and percentage of *success
stories* at each center. See Agency Report, Tab 11, Presentation to SSA,
8-11.
The SSA selected 270 proposals (not including Kolaka's) for award. See
Agency Report, Tab 12, Source Selection Decision (Nov. 18, 2002). Of the
25 phase I proposals recommended for award at Dryden, only 12 were
selected by the SSA to receive awards. These 12 proposals had higher
technical merit scores and equal or higher commercial merit scores than
Kolaka's proposal.[3] Following notification of the proposals selected
for award and receipt of a debriefing letter, this protest followed.
Kolaka complains that the evaluation of its proposal was unreasonable.[4]
Specifically, the protester argues that evaluator A's assessment of
Kolaka's proposal under the effectiveness of the proposed work plan factor
was unfair. Evaluator A assigned Kolaka's proposal 18 of the 25 available
points (that is, 72 percent) for this factor, because the evaluator found
that although the *work plan described is adequate for the proposed
effort,* the *actual models planned to be used were not detailed, and no
work breakdown structure was provided for the effort.* Agency Report, Tab
5a, Evaluation Form for Evaluator A, at 1. Kolaka complains that the
solicitation did not require a work breakdown schedule and that it
provided sufficient information to warrant a higher technical score for
this factor. In this regard, Kolaka argues that evaluator B assigned
Kolaka's proposal 23 points and found that the protester's *proposed work
plan [was] detailed out in a comprehensive fashion, . . . is adequate and
will prove to be effective.* Agency Report, Tab 5b, Evaluation Form for
Evaluator B, at 1.
In reviewing protests against allegedly improper evaluations, it is not
our role to reevaluate proposals. Rather, our Office examines the record
to determine whether the agency's judgment was reasonable and in accord
with the evaluation criteria. Abt Assocs., Inc., B-237060.2, Feb. 26,
1990, 90-1 CPD P: 223 at 4. Such judgments are by their nature often
subjective; nevertheless, the exercise of these judgments in the
evaluation of proposals must be reasonable and bear a rational
relationship to their announced criteria upon which competing offers are
to be selected. Southwest Marine, Inc.; Am. Sys. Eng'g Corp., B-265865.3,
B-265865.4, Jan. 23, 1996, 96-1 CPD P: 56 at 10. The protester's mere
disagreement with the agency's judgment does not establish that an
evaluation was unreasonable. UNICCO Gov't Servs., Inc., B-277658, Nov. 7,
1997, 97-2 CPD P: 134 at 7. Moreover, the agency is accorded considerable
discretion to determine which proposals will be funded under the SBIR
program. Virginia Accelerators Corp., B-271066, May 20, 1996, 97-2 CPD P:
13 at 2.
As explained below, we find no basis to conclude that the agency's
evaluation of Kolaka's proposal was unreasonable. As noted above, NASA
considered Kolaka's proposal to be very good, assigning the proposal an
average technical merit score of 86 of 100 available points. Although
Kolaka complains that one evaluator assigned its proposal fewer points
under the effectiveness of the proposed work plan factor than another
evaluator, and found problems with its plan that the other evaluator did
not, this in and of itself does not demonstrate that the evaluation was
unreasonable, inasmuch it is not unusual for evaluators to reach different
conclusions and assign different scores when evaluating proposals, since
both subjective and objective judgments are involved.[5] See Medical
Info. Servs., B-287824, July 10, 2001, 2001 CPD P: 122 at 5; Microeconomic
Applications, Inc., B-258633.2, Feb. 14, 1995, 95-1 CPD P: 82 at 9 n.6.
With respect to the effectiveness of the proposed work plan factor, the
solicitation explained that an offeror's proposed:
work plan will be reviewed for its comprehensiveness, effective use of
available resources, cost management and proposed schedule for meeting the
Phase I objectives. The methods planned to achieve each objective or task
should be discussed in detail.
Program Solicitation S: 4.1.2, at 14. The solicitation instructed
offerors that their:
work plan should indicate how, what and where it will be done. . . . The
methods planned to achieve each objective or task should be discussed in
detail. Schedules, task descriptions and assignments, resource
allocations, estimated task hours for each key personnel, and planned
accomplishments including project milestones shall be included.
Id. S: 3.2.4, at 7.
Here, evaluator A found that although Kolaka's proposal was *adequate*
under this factor, the proposal *did not detail how much work was to be
done at which points in the schedule and by which participants.*
Statement of Evaluator A, Feb. 26, 2003, at 2-3. From our review of
Kolaka's proposal, we find no basis to object to the evaluator's
judgment.[6] See Agency Report, Tab 4d, Kolaka Technical Proposal,
at 16‑20. We also note that evaluator B, who assigned Kolaka's
proposal 23 of 25 points for this factor, noted that although he judged
Kolaka's proposal to be *very good,* he recognized that the higher-rated
proposal in this subtopic provided a *more comprehensive and complete*
work plan that *was more effectively detailed than* Kolaka's. Statement
of Evaluator B, Feb. 26, 2003, at 2. In sum, we do not find that Kolaka's
technical merit score reflected an unreasonable evaluation.[7]
Kolaka also complains that NASA's ranking and selection of proposals were
unreasonable. First, Kolaka objects that the phase I proposals were
evaluated by various pairs of evaluators and that no single person
evaluated all of the proposals to ensure that there was a *consistent*
evaluation to allow for a fair ranking of proposals. However, the use of
evaluation teams across the various NASA field centers (such as the Dryden
Center) to evaluate the more than 2,000 phase I proposals received by NASA
was consistent with the peer review nature of the evaluation of phase I
proposals contemplated by the solicitation and the SBIR program. In any
event, Kolaka has not shown that NASA's use of teams of evaluators to
assess proposals resulted in an inconsistent evaluation that affected the
ranking of proposals.
Kolaka also complains that its proposal was not fairly presented by the
applicable subtopic manager to Dryden's ranking committee.[8]
Specifically, Kolaka argues that the presentation slides prepared by the
subtopic manager to explain Kolaka's proposal were not as numerous as
slides prepared for other proposals and that Kolaka's presentation slides
did not accurately represent Kolaka's proposal. Kolaka, however, does not
demonstrate how the presentation slides presented to the ranking committee
were inaccurate or failed to accurately portray Kolaka's proposal.
Furthermore, NASA presented evidence showing that the number of
presentation slides provided for an offeror did not affect the selection
of proposals for funding. See Statement of Dryden SBIR Field Center
Program Manager, Feb. 26, 2003, at 7. Moreover, the record provides no
support for Kolaka's speculation of bad faith on the part of NASA
officials in presenting its proposal to the ranking committee. Government
officials are presumed to act in good faith and, where a protester
contends that contracting officials are motivated by bias or bad faith, it
must provide convincing proof, since this Office will not attribute unfair
or prejudicial motives to procurement officials on the basis of inference
or suppositions. ACC Constr. Co., Inc., B-289167, Jan. 15, 2002, 2002 CPD
P: 21 at 4.
Kolaka also complains that the methodology for ranking phase I proposals
was not consistent across NASA's field centers. That is, the Dryden
Center reserved the highest ranking for the highest-rated proposal in each
of the five subtopics for which the Dryden Center was responsible, while
the remainder of the proposals (that were recommended for award) were then
ranked from sixth through twenty-fifth. Kolaka complains that not only
was this ranking method not identified in the solicitation, but that other
NASA field centers did not reserve the highest rankings for top-rated
proposals from each subtopic.
While it is true that not all of NASA's field centers used a ranking
scheme that ensured priority in ranking for the highest-rated proposal in
each subtopic, we see nothing unfair about this methodology as it was used
in this case. Indeed, this methodology provides one means of ensuring
program balance to the agency, which the solicitation informed offerors
would be considered in the selection decisions. See Program Solicitation
S: 4.1.3, at 14. In any event, we fail to see how the use of such a
ranking methodology at the Dryden Center prejudiced the protester. The
top‑rated proposals in the five subtopics each received
significantly higher technical merit scores than Kolaka's proposal, and
none received lower commercial merit ratings. Competitive prejudice is an
essential element of every viable protest. Lithos Restoration, Ltd.,
B-247003.2, Apr. 22, 1992, 92-1 CPD P: 379 at 5. Where the record does
not demonstrate that, but for the agency's actions, the protester would
have had a reasonable chance of receiving the award, our Office will not
sustain a protest, even if a deficiency in the procurement is found.
McDonald-Bradley, B‑270126, Feb. 8, 1996, 96-1 CPD P: 54 at 3; see
Statistica, Inc. v. Christopher, 102 F.3d 1577, 1581 (Fed. Cir. 1996).
Kolaka raises numerous other objections to NASA's failure to select the
protester's proposal for funding. From our review of the record, we find
that that none of Kolaka's arguments provide us with a basis to object to
NASA's evaluation, ranking, and selection of proposals.[9] The fact is
that Kolaka's proposal received very good ratings from NASA and was
recommended for award. The record reflects that Kolaka's proposal was not
selected for award, not because of a misevaluation or mis‑ranking of
proposals, but because NASA did not have sufficient funding to select all
the recommended proposals. Kolaka's proposal was simply not as highly
regarded as those that NASA was able to fund.
The protest is denied.
Anthony H. Gamboa
General Counsel
------------------------
[1] Proposals that were not recommended for award were not ranked.
[2] Initially, the program office recommended 310 proposals be funded, but
reduced that number to 270 based upon the amount of funds NASA would have
available.
[3] The proposed prices of Kolaka's proposal and the 12 proposals selected
for award were essentially equal (within several dollars of each other),
other than a few proposals that were as much as $10,000 lower than
Kolaka's proposed price.
[4] Kolaka also complains that NASA used both NASA employees and
contractor employees to evaluate proposals. The solicitation specifically
informed offerors that NASA may use qualified individuals from outside the
government in the proposal review process. Program Solicitation S:S:
4.1.1, 5.5, at 13, 18. NASA states that all non‑NASA evaluators who
performed such reviews had executed nondisclosure agreements. In any
event, both of the evaluators that reviewed Kolaka's proposal were NASA
employees.
[5] Kolaka complains that NASA did not comply with its Evaluation
Guidelines, which require that, where the evaluators' scores are *widely
differing,* a third evaluator will review the proposal. NASA disagrees
that the evaluation ratings Kolaka's proposal received from the two
evaluators were widely differing, and we cannot say that this position is
unreasonable. In any event, the Evaluation Guidelines, which Kolaka
admits were not readily available to offerors (see Comments P: 2.3.1), are
merely internal guidance to the agency's evaluators and do not themselves
establish legal rights and responsibilities so as to render inconsistent
agency action illegal. Pike Creek Computer Co., Inc., B-290329, June 21,
2002, 2002 CPD P: 106 at 3 n.2. In a similar vein, Kolaka complains that
the protest record reflects a number of different evaluation *processes,*
such as, for example, the process described in the *Dryden ISO 9000
flowchart* and the Evaluation Guidelines, which, as noted, do not provide
a basis to object to the agency's evaluation. Moreover, it is the
evaluation scheme in the solicitation, not internal agency documents, to
which an agency is required to adhere in evaluating proposals and in
making the source selection decision.
[6] We disagree with Kolaka's suggestion that evaluator A's comment
regarding Kolaka's failure to provide a work breakdown structure indicated
that the agency had used an unstated evaluation factor. A *work breakdown
structure* is merely a means of dividing work to be performed into logical
segments to track performance. See *The Government Contracts Reference
Book* (2nd ed. 1998) at 546. The solicitation's request for detailed work
plan information encompassed this information.
[7] While Kolaka's now suggests that these evaluators may have acted
arbitrarily or in a biased manner in their evaluation, the fact is they
both recommended Kolaka's proposal for award.
[8] The manager for the subtopic under which Kolaka proposed was one of
the evaluators that assessed Kolaka's proposal. See Statement of
Evaluator B, Feb. 26, 2003, at 1.
[9] For example, Kolaka also complains that the record does not establish
that NASA fairly used the commercial merit ratings in ranking proposals
for award. The record shows, however, that the proposals that were
selected for award all had commercial merit ratings equal to or higher
than Kolaka's and all of these proposals had significantly higher
technical merit scores.