BNUMBER: B-258322.5; B-258322.6
DATE: March 9, 1995
TITLE: TRESP Associates, Inc.; Advanced Data Concepts, Inc.
**********************************************************************
REDACTED DECISION
A protected decision was issued on the date below and was subject to a
GAO Protective Order. This version has been redacted or approved by
the parties involved for public release.
Matter of:TRESP Associates, Inc.; Advanced Data Concepts, Inc.
File: B-258322.5; B-258322.6
Date: March 9, 1995
Mary Beth Bosco, Esq., Timothy B. Mills, Esq., and Christy L.
Gherlein, Esq., Patton Boggs, L.L.P., for TRESP Associates, Inc.; and
Cyrus E. Phillips IV, Esq., Keck, Mahin & Cate, for Advanced Data
Concepts, Inc., the protesters.
Joel R. Feidelman, Esq., Deneen J. Melander, Esq., and Brian "D"
Henretty, Esq., Fried, Frank, Harris, Shriver & Jacobson, for M.H.
Chew & Associates, Inc., an interested party.
Ronald E. Cone, Esq., and Richard S. Blakely, Esq., Department of
Energy, for the agency.
Sylvia Schatz, Esq., David A. Ashen, Esq., and John M. Melody, Esq.,
Office of the General Counsel, GAO, participated in the preparation of
the decision.
DIGEST
1. Evaluation of protester's proposal was unreasonable where agency
identified weaknesses during discussions, protester addressed the
weaknesses, and record indicates that many of the weaknesses were
thereby eliminated, but proposal rating was not increased at all and
record does not explain why increase was not warranted.
2. Cost/technical tradeoff was unreasonable where agency selected
higher-cost proposal for award over lower-cost, similarly rated
proposal based primarily on awardee's ability to respond more quickly
on short notice, and the fact that awardee's initial proposal
contained fewer weaknesses, purportedly indicating better
understanding of requirement, and selection is not supported by the
record. Finding that awardee had the ability to respond more quickly
was based largely on source selection official's general recollection
of the two firms' prior performance, which is not documented in either
the contemporaneous or protest records; and record contains no
explanation, and it is not apparent, why the types of weaknesses
identified bear on understanding and, in any case, awardee's initial
proposal was evaluated similarly to protester's.
DECISION
TRESP Associates, Inc. and Advanced Data Concepts, Inc. (ADC) protest
the Department of Energy's (DOE) award of a contract to M.H. Chew &
Associates, Inc. (CAI), under request for proposals (RFP) No.
DE-RP03-93SF19686, for technical and management services. TRESP
primarily challenges the evaluation of its proposal as improper, and
ADC primarily argues that the cost/technical tradeoff was improper.
We sustain the protests.
The RFP, issued on May 14, 1993, as a small business set-aside,
contemplated the award of a 5-year--2 base years plus 3 option
years--cost-plus-fixed-fee contract to support DOE programs in San
Francisco, California, by furnishing technical and management services
in several areas (e.g., technical studies, environment, safety, health
and facility oversight, and safeguards and security technical
services). The solicitation included an estimate of the agency's
total requirement--including 68,100 (approximately 33 persons) direct
productive man-hours for the first year and 79,100 man-hours
(approximately 38 persons) for the second year--and an estimate of the
total man-hours for each of the 27 required key and non-key labor
categories. Offerors were required to propose certain key and non-key
personnel with "L" or "Q" security clearances, and to show that they
could obtain a DOE-approved classified facility.
Award was to be made to the responsible offeror whose offer conformed
to the RFP and was most advantageous to the government under three
broad criteria: (1) technical, which was more important than (2)
business/management, which was more important than (3) cost. There
were four technical evaluation factors (in descending order of
importance): (1) qualification of proposed personnel and personnel
management, including subfactors for technical experience and
offeror's approach to staffing the required work; (2) technical
experience and past performance; (3) technical approach; and (4) task
management. There were five business/management factors, all of equal
importance: (1) business/management organization; (2)
business/management experience on similar contracts; (3) cost
management; (4) subcontracting and subcontract management; and (5)
labor management relations.
Seven proposals were received by the closing date. Three--TRESP's,
CAI's, and ADC's--were included in the competitive range. Following
site visits and oral and written discussions with the offerors, the
agency requested best and final offers (BAFO). Based upon its
evaluation of the BAFOs, the source evaluation board (SEB) concluded
that only the responses of ADC and CAI warranted increases to their
original scores. The evaluation scores (received/available) were as
follows:[1]
TECHNICAL ADC CAI TRESP
Personnel Qualifications
Experience 16/22.5 14 15
Staffing 18/22.5 17 15
Technical Experience31/35 28 25
Technical Approach 19/25 20 16
Task Management 9/15 12 10
SUBTOTAL 95/120 90 82
BUSINESS/MANAGEMENT
Organization 15/20 16 16
Experience16/201516
Cost Management 14/20 17 12
Subcontracting 14/20 15 13
Labor Mgmt. Relations13/20 15 15
SUBTOTAL 72/100 77 70
TOTAL 167/220 167 152
PROPOSED COST $21,262,193$23,380,018$23,771,519
EVALUATED COST $20,669,538$21,917,802$23,600,000[2]
In its report to the source selection official (SSO), the SEB
summarized its evaluation of BAFOs without recommending any firm for
award.
The SSO conducted a cost/technical tradeoff based on his own knowledge
and experience, the SEB report, the most probable cost analysis, and
information presented to the SSO by the SEB chairman. In comparing
ADC's and CAI's BAFOs, the SSO determined that although both BAFOs
received almost the same technical ratings, the technical strengths of
CAI's BAFO outweighed its evaluated cost premium ($1,248,264). CAI's
BAFO also was found more advantageous than TRESP's on the basis of its
higher scores under the technical and business/management factors and
its lower probable cost.
TRESP's PROTEST
TRESP challenges DOE's evaluation of its BAFO on the basis that the
agency improperly failed to recognize the improvements in its BAFO.
Specifically, the protester notes that although the final SEB report
stated that all of the 12 "weaknesses" that it found in, and
"observations" that it made with respect to, TRESP's initial technical
and business/management proposals were "adequately addressed in the
BAFO" or "remedied," TRESP's final evaluation score was not increased
under any factor or subfactor.
In reviewing protests against allegedly improper evaluations, our
Office will examine the record to determine whether the agency's
determination was reasonable and consistent with the evaluation
criteria listed in the solicitation. Amtec Corp., B-240647, Dec. 12,
1990, 90-2 CPD 482, recon. denied, Dept. of the Army--Recon.,
B-240647.2, Feb. 26, 1991, 91-1 CPD 211. Implicit in the foregoing
is that the agency must document its judgments in sufficient detail to
show that they are not arbitrary. U.S. Defense Sys., Inc., B-245563,
Jan. 17, 1992, 92-1 CPD 89; Beckman Instruments, Inc., B-246195.3,
Apr. 14, 1992, 92-1 CPD 365. In this regard, Federal Acquisition
Regulation (FAR) 15.608 requires the contracting agency to prepare
specific documentation showing the basis for evaluation, an analysis
of the technically acceptable proposals, including an assessment of
each offeror's ability to accomplish the technical requirement, and a
summary of findings. DOE did not meet this standard here.
The record shows that TRESP submitted a BAFO which responded to the
concerns raised by the agency during discussions and which led the SEB
to state in its final report that the concerns had been "adequately
addressed in the BAFO" (or, with respect to one area, had been
"remedied"). Further, the agency concedes in the agency report that
the SEB members in fact believed that most of the weaknesses
identified in TRESP's initial proposal had been corrected. In this
regard, the agency has submitted declarations from four of five SEB
members stating that, except for the evaluated weakness of TRESP's
proposed non-key Health Physicist and Environmental Engineer, "most of
the weaknesses I identified in TRESP's initial proposal were
corrected," or simply "the weaknesses I identified in TRESP's initial
proposal were corrected."
Although the agency generally maintains, and these declarations
further state, that the "clarifications and improvements were not
significant enough . . . to change [the] scoring for TRESP's BAFO,"
there is nothing in the record that explains the specific reasons
underlying the determination not to increase TRESP's BAFO score under
any factor or subfactor.[3] While TRESP may not have been entitled to
an increase in points under each of the factors and subfactors where
the SEB noted that its concerns were adequately addressed, we can find
no reasonable basis from our review of the record for concluding that
TRESP was entitled to no additional points under any evaluation
subfactor or criteria as a result of the changes it made in its BAFO
in response to the concerns raised by the agency during discussions.
For example, under the technical experience and past performance
factor of the technical criterion, where TRESP received 25 of 35
points, and the business management experience on similar contracts
factor of the business/management criterion, where TRESP received 16
of 20 points, the agency was concerned that TRESP's initial proposal
failed to show that TRESP, as opposed to its proposed subcontractors,
had prior experience in performing contracts as wide and diverse in
scope and of a similar level of effort--approximately 33-38 persons
during the base period--as contemplated under the current statement of
work. To alleviate the concern raised during discussions in this
regard, TRESP submitted in its BAFO a detailed description of numerous
tasks and services performed by TRESP itself under current and prior
government contracts, including contracts involving levels of effort
of up to 25, 27, and 80 employees. The record shows that the SEB not
only was satisfied that TRESP's response alleviated its concerns but,
indeed, considered TRESP's experience a strength. In this regard, the
SEB's final report noted that TRESP's BAFO "adequately addressed" its
concerns and described as a strength TRESP's "good" and "similar"
contract experience with DOE. Nevertheless, TRESP's BAFO scores did
not increase. DOE does not explain, nor is it otherwise evident from
the record, why an increase in TRESP's scores was not warranted by the
more detailed description in its BAFO of its experience.
As another example, under the cost management factor of the
business/management criterion, where TRESP received only 12 of 20
points, the SEB determined that TRESP's initial proposal did not
demonstrate an ability to track and control costs and achieve cost
targets. In response to the agency's request during discussions that
TRESP provide examples of a proven ability to track and control costs
with its proposed automated accounting system, TRESP described in
detail its system for tracking and analyzing costs. Although the SEB
report stated that this concern was "adequately addressed" in TRESP's
BAFO, there was no increase in TRESP's BAFO score under this factor.
Again, DOE does not specifically explain, and it is not evident, why
an increase in TRESP's score was not warranted.
As a further example, under the subcontracting and subcontract
management factor of the business/management criterion, where TRESP
received 13 of 20 points, the SEB determined that TRESP's initial
proposal had not clearly defined how it would, nor demonstrated the
ability to, manage subcontractor performance through contract
completion. In response to the agency's request during discussions to
explain how it would manage subcontractor performance, TRESP explained
in its BAFO that it would monitor subcontractor performance in three
different ways: (1) by analyzing milestones accomplished against the
subcontractor task plan; (2) by comparing forecasted expenditures with
the costs incurred; and (3) by making frequent quality control checks
throughout subcontractor task performance to discover potential
problems, and then redirecting resources from noncritical tasks,
revising the subcontractor task plan or expected milestones, and
developing and implementing a recovery plan. In addition, TRESP
explained that subcontractor management would be facilitated by
co-locating subcontractor personnel with TRESP personnel at DOE or
TRESP sites. Again, although the SEB report stated that this concern
was "adequately addressed" in TRESP's BAFO, there was no increase in
TRESP's BAFO score under this factor. And again, nothing in the
record shows why TRESP's score was not increased.
Given the evaluators' own statements that TRESP's BAFO resolved the
deficiencies raised during discussions, our own review which supports
these statements, and the absence of documentation showing otherwise,
it appears that TRESP's BAFO was not properly evaluated; although the
SEB found that TRESP had largely remedied the initial proposal
weaknesses, TRESP's score was not increased at all. At minimum, the
evaluation record is inadequate to establish that the TRESP's BAFO was
properly scored. As it also is not possible to determine the precise
impact of any evaluation improprieties on TRESP's score, we can only
conclude that a proper evaluation could result in TRESP's score
increasing sufficiently to displace both CAI and ADC. We therefore
sustain the protest on this basis.
ADC's PROTEST
ADC's protest turns on the same essential issue as TRESP's--whether
the record supports the SSO's cost/tradeoff decision. The SSO
selected CAI for award based primarily on three factors: (1) CAI's
superior past performance; (2) CAI's superior depth of staffing; and
(3) CAI's submission of a superior initial proposal. ADC maintains
that the tradeoff--resulting in selection of CAI's BAFO, which
received the same evaluation score as ADC's but was evaluated as
$1,248,264 (or 6 percent) higher in cost--was not adequately supported
by the record. We agree.
Past Performance
The SSO's determination that CAI's past performance was superior to
ADC's was based on his personal knowledge and belief that ADC's
proposed subcontractor (PAI) had failed to timely accomplish several
task assignments under a current DOE Oakland contract (and thus could
not be relied upon to timely support the contemplated contract effort
here). Such personal knowledge of course may be considered in an
evaluation. See NDI Eng'g Co., B-245796, Jan. 27, 1992, 92-1 CPD
113. However, reliance on personal knowledge of the offerors does not
eliminate the fundamental requirement that evaluation judgments be
documented in sufficient detail to show that they are not arbitrary.
U.S. Defense Sys., Inc., supra. DOE did not meet this standard in its
cost/technical tradeoff decision.
The record contains no factual information supporting the SSO's
conclusion that CAI's prior experience was superior to ADC's.
Specifically, the SSO does not cite--and the record does not
reference--any specific CAI contracts, or examples of CAI's
performance that was deemed superior, or any similar details of ADC's
or PAI's performance that would support a conclusion that their
performance was inferior to CAI's. Rather, the SSO merely references
his "years of personal experience dealing with CAI and their proposed
subcontractors and ADC and their proposed subcontractors," and states
that CAI has demonstrated superior performance in the areas of
environment, safety and health, and management support, while ADC's
performance in the area of safety and security "in my opinion was not
superior." These conclusionary statements, without some specific
explanation, are an inadequate basis for the SSO's evaluation of CAI
and ADC, since it is not possible to determine whether the SSO's
judgment was reasonable. In fact, regarding PAI, the only detailed
information in the record is inconsistent with the SSO's conclusion;
ADC has furnished three letters from DOE to PAI commending PAI for its
performance on prior DOE contracts. Indeed, one of the letters from
DOE commends PAI for its "quick turnaround on request changes" on the
DOE Oakland contract, stating that PAI's "efforts were instrumental in
completing this [facility representative training and certificate]
plan on time to allow me to submit it to management."
In light of the FAR 15.608 requirement for adequately documented
source selection decisions, we do not think the SSO's nonspecific,
undetailed references provided a reasonable basis for distinguishing
between two similarly evaluated proposals in a cost/technical
tradeoff.
Depth of Staffing
The SSO states that "the most important factor that caused me to
choose CAI over ADC was the breadth and depth of immediately available
support staff. This capability to expeditiously respond, and the
depth of varied and experienced talent was very persuasive to my
decision." However, while CAI did have 400 individuals within 100
miles of the Livermore area and 750 "Q" cleared personnel nationwide,
ADC's proposal showed 277 "Q" or "L" cleared personnel. Given that
the required staffing under the contract was only 33-38 individuals,
and that the contractor would be obligated to provide no more than 110
percent of the estimated level of effort, we do not think there was a
reasonable basis for assigning great significance to CAI's larger
staff in the tradeoff decision. While redundancy in staffing may be a
legitimate consideration for a requirement such as the one here, it is
not reasonable to evaluate a proposal as superior to another where the
larger staff will not benefit the government. Both CAI and ADC have
what appears to be significant redundancy for the level of effort
required here. CAI's staff certainly is much larger than ADC's, but
the record does not explain why CAI's large number of additional
personnel was considered beneficial in any significant way. DOE does
not assert, for example, that such a large number of employees (in
relation to the required staffing) has been necessary in the past to
properly perform similar requirements.
The SSO's conclusion regarding the superiority of CAI's available
staffing was based in part on his "concern about ADC's ability to
provide timely support staff." This concern stemmed from (1) his
personal knowledge of PAI's performance on the DOE Oakland contract,
and (2) the SSO's opinion that while ADC's BAFO provided as an example
of expeditious performance the fact that it pulled together a 35
member "Q" cleared team in 2 weeks, CAI could have assembled a similar
team in a much shorter period of time. As stated previously, the
SSO's personal knowledge of PAI's past performance consists entirely
of conclusionary statements. His opinion is also questionable because
CAI's proposal does not reference similar accomplishments (and the SSO
did not state that he was aware of any), and the SEB actually cited
this mobilization example as a strength in ADC's evaluation because it
shows a "demonstrated ability to respond to short notice/high volume
task assignments." The SSO properly may reject the SEB's evaluation
conclusions where he disagrees with them, see Wyle Labs., Inc.;
Latacoere Int'l, Inc., 69 Comp. Gen. 648 (1990), 90-2 CPD 107, but
his different judgment must be reasonable. Id. The SSO's conclusion
regarding CAI's capability, and his essential disagreement with the
SEB that ADC actually was strong in this area, are unsupported by any
references to CAI's proposal or specific information regarding the
firms' past performance.
Superior Initial Proposal
The SSO stated in the source selection statement, under the technical
experience of proposed personnel subfactor, that:
"CAI's proposed personnel are characterized as exceeding minimum
requirements with no weaknesses identified. This
characterization is based upon CAI's initial offering as opposed
to ADC's initial offering which resulted in a number of
weaknesses related to the qualifications of proposed personnel.
This ability to understand and anticipate the needs of [DOE] and
respond timely is crucial to meeting the future responsibilities
of this office. ADC's weaknesses were resolved during the BAFOs,
but demonstrate an initial lack of understanding of the personnel
requirements."
Likewise, in responding to ADC's protest, the SSO explained that his
determination that CAI's proposal was superior to ADC's, even though
ADC received five more points under the technical factor, was based in
part upon the fact that in several performance areas ADC had proposed
personnel who were not qualified, and that this needed to be discussed
and resolved during discussions. According to the SSO, if "ADC truly
understood the needs of this Operations Office, this should not have
been necessary." The SSO noted that, in contrast, CAI had "submitted
a superior original proposal which demonstrated a better understanding
of the needs of this Operations Office."
In judging an offeror's understanding of the contract requirements, an
agency properly may take into consideration a lack of understanding
demonstrated in the initial proposal, even where it subsequently is
remedied in the BAFO. See Communications Int'l Inc., B-246076, Feb.
18, 1992, 92-1 CPD 194. However, the record here does not support
the SSO's conclusion that ADC's initial understanding was inferior to
CAI's. First, CAI's initial proposal received only 14 of 22.5
available points under the technical experience of proposed personnel
subfactor, while ADC's proposal received 15 points. In fact, CAI's
BAFO still received only 14 points under this subfactor, while ADC's
received 16 points.
Second, contrary to the SSO's statement, CAI's initial proposal did
have a cited weakness with respect to the technical experience of the
proposed personnel.[4] The SSO's conclusion that ADC's initial
proposal reflected a weaker understanding than CAI's is inconsistent
with the SEB's scoring of ADC's initial proposal as slightly superior
to CAI's in this area. Again, although the SSO may disagree with the
SEB, Wyle Labs., Inc.; Latacoere Int'l, Inc., supra, the reasons for
his disagreement must be set forth in the record. The record does not
indicate why the SSO came to different conclusions regarding the two
initial proposals.
We conclude that the cost/technical tradeoff is not supported in the
record. As a result, there is no basis for concluding that DOE
reasonably determined that CAI's proposal provided technical
advantages which offset ADC's $1.2 million cost advantage.
CONFLICT OF INTEREST
TRESP and ADC argue that DOE's award to CAI gives rise to an actual or
apparent conflict of interest, due to the actions of three DOE
officials who were involved in the procurement--Messrs. Keheley,
Warner, and Taylor. The protesters allege that the three engaged in,
or were influenced by others', employment discussions with Mr. Chew,
president and chief executive officer of CAI.
There is no evidence of any impropriety here. First, the record
indicates that Mr. Warner was not involved with any aspect of the
procurement. Mr. Warner states in his affidavit that he did not
participate in, discuss, or have any knowledge of the procurement.
Although Mr. Warner was appointed an ex-officio, nonvoting member of
the SEB on the basis of his position as a division director affected
by the solicitation, the record indicates that he did not know that he
had been designated an ex-officio member of the SEB until he was told
for the first time on October 26, more than 2 months after contract
award. Second, Mr. Keheley's affidavit states that he met with Mr.
Chew for the first time on August 24, that is, about 3 weeks after his
August 2 selection of CAI for award and that, in any case, he never
discussed employment at CAI or received an employment offer. Further,
there is nothing to suggest that Mr. Keheley, Mr. Warner's supervisor,
may have improperly based his source selection on any knowledge or
belief that Mr. Warner was having employment discussions with Mr.
Chew. Finally, with respect to Mr. Taylor, there is no evidence that
his evaluation may have been improperly influenced by any knowledge or
belief that Mr. Warner had employment discussions with Mr. Chew.
Further, although Mr. Warner was Mr. Taylor's usual supervisor, during
the course of the procurement Mr. Taylor was supervised solely by the
SEB chairman.[5] We therefore deny this aspect of the protests.
RECOMMENDATION
We recommend that DOE reevaluate proposals in accordance with this
decision and fully document the evaluation and the basis for the
resulting source selection decision. In the event an offeror other
than CAI is found to be entitled to award, DOE should terminate CAI's
contract for the convenience of the government and make award to that
offeror. We also find that ADC and TRESP are entitled to
reimbursement of the reasonable costs of pursuing their protests,
including attorneys' fees. 4 C.F.R. 21.6(d)(1) (1994). The
protesters should submit their certified claims for their protest
costs directly to the agency within 60 working days after receipt of
this decision.
The protests are sustained in part and denied in part.
Comptroller General
of the United States
1. The offerors' overall technical and business/management subtotals
were not the sum of the offerors' individual subfactor and criteria
scores. Rather, according to the agency, the subtotals were obtained
by adding the five individual SEB evaluators' total scores for all of
the technical and all of the business/management subfactors and
criteria, and then dividing the resulting sums by five. The total
scores for the technical and business/management proposals were added
together to obtain the offerors' overall total scores.
2. The contracting officer reduced proposed costs based on the
recommendations of the Defense Contract Audit Agency.
3. Although one of the SEB members stated that he did not change "most
of my scores for TRESP's BAFO," in fact, the record shows that no SEB
member changed any score for TRESP's BAFO.
4. While not reflected in the SEB report, we note that the technical
evaluation committee (first line evaluators) found two additional
weaknesses under the technical experience of proposed personnel
subfactor, and the SEB chairman expressed several concerns. For
example, he noted that CAI's proposal failed to show that its proposed
project manager and physical scientist--satisfied the "PC [personal
computer] software experience requirement," and lacked specifics
regarding its physical scientist/classification specialist's skills
with respect to communicating "class[ification] guidance concepts," as
required in the RFP. The chairman also expressed concern that CAI's
proposal: (1) failed to show that CAI's proposed environmental
engineer had the required "specific experience re[garding] geologic,
hydrologic, chemical, and environmental investigations," and
experience in conducting site appraisals; (2) did not specifically
demonstrate the "technical/project leadership" skills of CAI's
proposed general engineer; and (3) did not explain the extent and type
of experience of CAI's proposed nuclear engineer with respect to
audits, appraisals, and conducting support facility tour activities.
5. DOE conducted an internal investigation in response to the
protesters' allegations and found, as we have, that there is no
evidence of a conflict of interest.