TITLE:  Future-Tec Management Systems, Inc.; Computer & Hi-Tech, B-283793.5; B-283793.6, March 20, 2000
BNUMBER:  B-283793.5; B-283793.6
DATE:  March 20, 2000
**********************************************************************
Future-Tec Management Systems, Inc.; Computer & Hi-Tech, B-283793.5;
B-283793.6, March 20, 2000

Decision

Matter of: Future-Tec Management Systems, Inc.; Computer & Hi-Tech
Management, Inc.

File: B-283793.5; B-283793.6

Date: March 20, 2000

Dean M. Dilley, Esq., Douglas C. Proxmire, Esq., and Robert K. Tompkins,
Esq., Patton Boggs, for Future-Tec Management Systems, Inc., and Edward J.
Tolchin, Esq., Fettmann, Tolchin & Majors, for Computer & Hi-Tech
Management, Inc., the protesters.

Andrew P. Hallowell, Esq., Piliero, Mazza & Pargament, for Systems
Engineering & Security, Inc., an intervenor.

David H. Turner, Esq., Naval Supply Systems Command, for the agency.

Christina Sklarew, Esq., and Paul I. Lieberman, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.

DIGEST

1. Agency's evaluation of technical proposals and the subsequent award
decision cannot be found reasonable where the evaluation record includes
only minimal information and conclusory statements and the agency fails to
provide documentation or other explanations that reasonably support the
overall assessments of the two protesters' and the awardee's proposals.

2. Agency's adjustment of proposed costs is not reasonable where it is
essentially based on agency's misunderstanding of the proposal.

DECISION

Future-Tec Management Systems, Inc. and Computer & Hi-Tech Management, Inc.
(CHM) protest the Department of the Navy's award of a contract to Systems
Engineering and Security, Inc. (SES) under request for proposals (RFP) No.
00140-99-R-G413, issued by the Navy Fleet and Industrial Supply Center
(FISC)-Norfolk Detachment Philadelphia. Both Future-Tec and CHM protest that
the Navy's evaluation of technical proposals and the resulting source
selection were improperly performed, and allege that SES proposed key
personnel that it knew or should have known would not work full time on the
project, as required by the RFP. In addition, CHM protests that the Navy's
cost realism analysis was improper, and resulted in an unreasonable and
arbitrary increase in CHM's evaluated proposed costs.

We sustain the protests.

The RFP was issued on June 25, 1999, by the FISC office on behalf of the
Navy Reserve Information Systems Office and the Systems Executive Office for
Manpower and Personnel, both located in New Orleans, Louisiana. The RFP
explained that these offices have been tasked to develop world class
software products and to market their capability to new customers, and
sought offers for a broad range of automated information system support and
infrastructure services for these offices.

The RFP, issued as a competitive procurement set aside for small
disadvantaged businesses under the Small Business Administration's 8(a)
program, contemplated the award of a cost-plus-fixed-fee (CPFF),
indefinite-delivery/indefinite-quantity contract, under which services would
be ordered by delivery or task orders. The RFP provided that the government
intended to make an award to the offeror whose proposal presented the best
value to the government, and stated that a proposal's technical merit would
be considered more important than cost. The solicitation listed the
following technical evaluation factors in descending order of importance,
stating that the first two would be considered equal in weight: corporate
experience, past performance, technical approach, management plan, and
personnel resources. RFP sect. M.

Offerors were instructed to prepare their technical proposals in two
volumes, with the first volume to include written descriptions of the
offeror's past performance and its personnel resources, and the second to
include transparencies and slides that would be used during oral
presentations, covering technical approach, management approach, and
corporate experience. RFP sect. L.I.

With respect to past performance, the RFP instructed offerors as follows (in
relevant part):

[D]escribe . . . past performance on directly related or similar contracts
. . . held within the last five (5) years which are of similar scope,
magnitude and complexity to that which is detailed in the RFP. Offerors who
describe similar contracts shall provide a detailed explanation
demonstrating the similarity of the contracts to the requirements of the
RFP. In determining the rating for the past performance evaluation factor,
the Government will give greater consideration to the contracts which are
most relevant to the RFP.

RFP sect. L.III.A.1, at 51.

The personnel resources factor listed key personnel resumes and staffing as
subfactors and stated that they were of equal importance. The RFP listed the
11 key personnel full-time equivalent positions for which the offerors were
required to submit resumes. RFP sect. L.III.A.2, at 52. It also provided that
resumes for key personnel must indicate whether the proposed employee was
currently employed by the offeror, and required offerors to certify that
proposed key personnel had confirmed their availability for contract
performance. Id.

The second volume was to consist of slides that the offeror would use in an
oral presentation addressing its technical approach, management plan, and
corporate experience. The RFP offered some guidance for preparing proposals
in each of these areas and cautioned offerors to provide sufficient detail
in their proposals. Regarding corporate experience, offerors were instructed
to "demonstrate either similar or directly related work experience of
similar scope, magnitude and complexity to the [statement of work]. The
offeror should address the following: history, organization, qualifications
and work experience within the last five (5) years as they relate to the
requirements of the [statement of work]." RFP sect. L.III.B.3, at 52.

Offerors were required to provide a detailed price/cost breakdown as the
third volume to their proposals. The RFP stated that costs would be
evaluated for cost realism. RFP sect. M.3, at 56.

Five offerors submitted proposals by the August 12, 1999 closing date. A
five-member technical evaluation committee (TEC) reviewed the initial
proposals and prepared a written technical evaluation. The TEC scored the
technical proposals under the following adjectival rating scale: highly
acceptable (HA); acceptable (A); unacceptable, but with the capacity of
being made acceptable as a result of discussions (U(a)); and unacceptable
with no reasonable chance of being selected for award (U(b)). The TEC's
rating summary for the SES, CHM and Future-Tec proposals was as follows:

 Offeror      Past.        Corp. Exp.   Tech. App.   Mgt. Plan    Pers.
              Perf.                                               Res.

 SES          HA           HA           HA           A            U(a)

 CHM          HA           A            A            A            U(a)

 Future-Tec   HA           A            A            A            U(a)

Agency Report, Tab 16, Technical Evaluation of Proposals, at 15.

The technical evaluation for each of these offerors began with the following
summary:

[The offeror's] overall rating is Unacceptable (a) due to a rating of
Unacceptable (a) in the Personnel Resources factor.

Id. at 1, 4, 9.

Although the evaluation report provides a separate evaluation for each
offeror's proposal, the comments for the three proposals are often identical
for CHM and Future-Tec and provide little or no detail or explanation of any
differences between the different proposals. For example, the following
comments are all that appears in the report for the following evaluation
factors for each of these two proposals:

Corporate Experience: [The offeror] demonstrated corporate experience on
contracts of similar scope, magnitude and complexity with the past 5 years
to that detailed in the RFP. This factor is rated acceptable.

Past Performance: [The offeror] described its past performance on contracts
of similar scope, magnitude and complexity to that detailed in the RFP. The
[contracting officer or contract performance report on a cited contract]
rated [the offeror] exceptional in all evaluation areas. This history of
quality past performance mitigates the risk to the Government on this effort
in the areas of cost, schedule and performance in meeting the requirements
of this RFP. This factor is highly acceptable.

Technical Approach: [The offeror] has submitted a technical approach which
demonstrated an understanding of the [statement of work]. This factor is
rated acceptable.

Management Plan: [The offeror] has submitted a management approach which
demonstrated an understanding of the [statement of work]. This factor is
rated acceptable.

Personnel Resources: This factor is rated as unacceptable (a).

Staffing: [The offeror's] staffing plan meets the requirements of the RFP.
This subfactor is rated acceptable.

Id.

For key personnel resumes, the report listed the resumes each offeror had
submitted, noting which ones failed to demonstrate compliance with
requirements in the RFP. Id. at 2-3, 5-6, 10-11.

The evaluation for SES was identical to the others for the factors of past
performance, management plan, personnel resources, and for the key personnel
resumes subfactor. With respect to corporate experience, the TEC made the
same initial statement--i.e., that SES demonstrated experience on contracts
of similar scope, magnitude and complexity--but then continued as follows:

Further, SES's education profile is noteworthy, demonstrating that
[deleted]% of employees have [deleted] advanced degrees. SES also identified
work experience of an exceptional magnitude and diversity, for example,
contract number [deleted] (a $73 million dollar contract for
multi-disciplinary IT services with a staffing level of [deleted] FTE). This
varied experience reduces risk to the Government in the execution of this
RFP. This factor is rated highly acceptable.

Id. at 9.

With respect to staffing, the report stated as follows:

This proposal outlines a detailed and thorough transition plan that amounts
to a blueprint for the whole transition process. This plan addressed a
comprehensive approach to staff development, recruiting, retention, and
transition of incumbent personnel. This staffing plan ensures a low-risk,
seamless transfer of control. SES staffing plan is rated highly acceptable.

Id. at 10.

Based on the TEC's evaluation and her own review of the proposals, the
contracting officer determined that only the Future-Tec, CHM and SES
proposals would be included in the competitive range. The competitive range
offerors were permitted to make oral presentations and to answer questions
posed by the TEC. At a hearing held by our Office to complete the record in
this protest, the contracting officer stated that no written record was made
of the oral presentations or the ensuing discussions. Hearing Transcript
(Tr.) at 84. The contracting officer conducted written technical
discussions, notifying the three competitive range offerors of the
deficiencies found in each of their initial proposals in the area of key
personnel and requesting revised technical proposals.

The TEC reviewed the revised proposals and prepared a technical evaluation.
The new report differed from the initial report only in the evaluation of
key personnel resumes, for which each proposal was now rated acceptable.
Overall, SES was rated highly acceptable, while CHM and Future-Tec were
rated acceptable. SES was ranked first, and CHM and Future-Tec were rated
technically equal in second place. Agency Report, Tab 24, Revised Technical
Evaluation, at 8. The three offerors were informed that their proposals were
now considered technically acceptable, and offerors were apprised of the
areas in their cost proposals that, based on a cost realism analysis
performed by the contracting officer and contract negotiator, were
considered to be understated. Offerors were invited to submit best and final
offers (BAFO). The contracting officer reminded offerors that revised
proposals had to include a certification that proposed employees had been
contacted after the date of the letter requesting final proposal revisions,
and that all proposed employees had confirmed that they were available for
contract performance. Agency Report, Tabs 25, 26-27, Requests for Final
Proposal Revisions, at 2.

SES and CHM submitted revised cost proposals, while Future-Tec resubmitted
its costs as initially proposed. In its BAFO, CHM slightly [deleted] its
overhead rate and its fee. It also proposed a [deleted] in order to reduce
the risk of [deleted]. Agency Report, Tab 29, CHM BAFO, at 2.

In its BAFO, CHM proposed a total CPFF of $[deleted]. For reasons that will
be discussed below, the Navy adjusted this figure to $[deleted] based on its
cost realism analysis. SES's costs were also adjusted, and were evaluated at
$83,666,971.37. [1] Future-Tec's costs remained unadjusted at
$102,293,791.65. Agency Report, Tab 33, Post-Negotiation Business Clearance
Memorandum, at 23-24.

The contracting officer, acting as the source selection authority, analyzed
the technical and cost ratings of the three proposals and selected SES for
award. In her source selection decision memorandum, the contracting officer
summarized the procurement process (as described above) and essentially
repeated the comments in the TEC's evaluation report in a narrative
paragraph discussing each technical proposal. [2] Regarding the source
selection, the contracting officer noted that Future-Tec's proposal was
considered in third place, based on its combined technical and cost rating.
Comparing the remaining two proposals, the memorandum notes that SES's
proposal was rated more highly on technical merit and was priced [deleted]
percent higher than CHM's, and refers to a cost/technical tradeoff analysis
for which the only documentation is the contracting officer's conclusory
statement in her source selection memorandum that "the substantial technical
superiority of the SES proposal outweighs any benefit that would be gained
from CHM's lower cost proposal." Agency Report, Tab 31, Source Selection
Decision Memorandum, at 6. Comparing the two proposals, the contracting
officer notes that both offerors were rated "highly acceptable" under past
performance, based on their performance under prior contracts, but that
under corporate experience, SES was rated highly acceptable while CHM was
rated only acceptable. The contracting officer repeated the TEC report's
reference to SES's "work experience of exceptional magnitude and diversity,"
adding that "[n]o other proposal demonstrated any experience of comparable
magnitude, and the Navy places great value on the magnitude as well as the
nature of this experience." She again noted that SES's proposal was rated
highly acceptable in technical approach, repeating the TEC's comment that
SES "demonstrated in more than sufficient detail a technical approach that
will successfully accomplish the [statement of work]," and offering as the
sole example to support this rating "SES's exceptional demonstration of how
their risk mitigation techniques and methods had been successfully used on
prior contracts and how those techniques would be applied in the instant
requirement." Id.

The contracting officer awarded the contract to SES on September 23.
Future-Tec requested a debriefing the following day. The debriefing was held
on September 28, and this protest followed on September 29. CHM was
debriefed on September 29 and filed its protest in our Office on October 1.
On October 1, the head of the contracting activity determined to override
the stay of performance that was required by the Competition in Contracting
Act of 1984, finding that continued performance of the contract was in the
best interests of the government. See 31 U.S.C. sect. 3553(d)(3)(C)(i)(I)
(1994).

THE TECHNICAL EVALUATION

Both Future-Tec and CHM protest that its own proposal merited a higher
technical rating than it received under the Navy's evaluation, and that
SES's proposal, in contrast, was rated too highly.

In reviewing an agency's evaluation of proposals and source selection
decision, our review is confined to a determination of whether the agency
acted reasonably and consistent with the stated evaluation factors and
applicable procurement statutes and regulations. Main Bldg. Maintenance,
Inc., B-260945.4, Sept. 29, 1995, 95-2 CPD para. 214 at 4. An agency's
evaluation of proposals and source selection decision must be documented in
sufficient detail to allow for the review of the merits of a protest.
Southwest Marine, Inc.; American Sys. Eng'g Corp., B-265865.3, B-265865.4,
Jan. 23, 1996, 96-1 CPD para. 56 at 10. An agency that fails to adequately
document its evaluation of proposals and source selection decision in
sufficient detail to show that it is not arbitrary bears the risk that our
Office will not conclude that the agency had a reasonable basis for its
determinations. Engineering and Computation, Inc., B-261658, Oct. 16, 1995,
95-2 CPD para. 176 at 3; U.S. Defense Sys., Inc., B-245563,

Jan. 17, 1992, 92-1 CPD para. 89 at 3. That is not to say that our Office, in
determining the reasonableness of an agency's evaluation and award decision,
limits its review to the contemporaneous evaluation and source selection
documentation. Rather, we will consider, in addition to the contemporaneous
documentation, information provided to our Office for consideration during
the protest, including the parties' arguments and explanations and testimony
elicited at a hearing. Southwest Marine, Inc.; American Sys. Eng'g Corp.,
supra. However, in considering the entire record, we accord greater weight
to contemporaneous evaluation and source selection material rather than
judgments made in response to protest contentions. Boeing Sikorsky Aircraft
Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD para. 91 at 15. Where
post-protest explanations simply fill in previously unrecorded details of
contemporaneous conclusions, we will generally consider them in our review
of the rationality of selection decisions, so long as those explanations are
credible and consistent with the contemporaneous record. NWT Inc.;
PharmaChem Labs., Inc., B-280988, B-280988.2, Dec. 17, 1998, 98-2 CPD para. 158
at 16.

Documentation of the Evaluation

Our review of the record confirms that the Navy did not adequately document
its evaluation of proposals and that the documentation and further
explanation offered during the course of the protest, including the two-part
hearing that was conducted by our Office, fail to demonstrate that the
evaluation and source selection were reasonable and supported by the facts.

All of the parties agree that the procurement at issue is for a complex and
expensive project. The Navy acknowledges in its written post-hearing
comments the complexity and magnitude of this procurement, asserting that
the agency "dedicated significant resources to the evaluation process." [3]
Agency Post-Hearing Comments at 18. However, in spite of this complexity,
the Navy's entire technical evaluation record consists of only three
documents: the TEC's extremely brief and conclusory evaluation of initial
proposals, the TEC's revised technical evaluation (which is unchanged from
the initial evaluation except in the area of key personnel resumes), and the
contracting officer's source selection decision memorandum, which generally
adopts the TEC's findings and ratings with little further explanation or
amplification, including only a single paragraph narrative covering the
evaluation of each of the competitive range offerors.

When questioned at the hearing about certain aspects of the technical
evaluation, the contracting officer repeatedly referred to an "overall
analysis" in which she "looked at each company, I looked at them in total,"
Tr. at 116-17; "I looked at . . . the big picture, and from that I developed
an overall rating plan." Tr. at 117. These statements, coupled with the
absence of any specific or detailed documentation of the evaluation process,
suggest the application of an imprecise methodology, and leave us to
speculate about what considerations actually went into the evaluation and
source selection. There simply is no reasoned discussion of the relative
strengths and weaknesses of the three proposals under each of the evaluation
factors, no evidence that the different factors were weighted differently
under the evaluation, and no evidence to show how information provided in
the oral presentations was considered. Moreover, the contracting officer's
recollections during the hearing about specific aspects of the evaluation
were at times inconsistent, as could be expected in so complex a
procurement, and exacerbate the problems arising from the lack of adequate
contemporaneous documentation. [4] In addition, she frequently referred to
information that was allegedly considered under the evaluation but was never
documented. For example, when asked why SES's proposal was rated highly
acceptable under the technical approach evaluation factor, while the other
two proposals were rated only acceptable, the contracting officer stated
that SES "went very much into detail" to explain their technical approach
during their oral presentation and that "[i]t was very clear that they
offered a technical approach that was far superior to what would be called
an acceptable approach . . . ." However, she verified that although the TEC
members discussed their ratings after the oral presentations, no notes were
made of the oral discussions, nor was the evaluation of the oral
presentations documented in any way. Tr. at 190.

In addition, the contracting officer frequently relied on the expertise of
the TEC members without necessarily understanding their conclusions, and--in
the absence of any documentation of their evaluation--there is no way to
ascertain the reasonableness of those conclusions. For example, after
testifying that SES's proposal demonstrated its ability to perform the
statement of work in a manner that was "[v]ery clear, very easy to read,
very easy to understand," Tr. at 191, and after being questioned about the
flowcharts that were cited in the source selection decision memorandum as
demonstrating SES's proven technical approach, the contracting officer
basically conceded that she did not personally understand the flowcharts and
could not explain their significance. Tr. at 193-96. Rather, she stated: "I
think if you're technical, you do understand what it means . . . if I can
remember, one of the comments that was made to me from the technical people
was, ‘They mapped it out so well anybody could pick this up and know
how to do the job.' That was their concept." Tr. at 194. After being
questioned repeatedly about what the flowcharts specifically demonstrated,
or how, the contracting officer testified that she did not have an answer.
Tr. at 198. Instead of providing any rationale for the agency's technical
evaluation, the contracting officer repeated the conclusions the TEC had
reached:

They presented a technical approach that was far beyond just demonstrating
an ability to perform the work. They flowcharted it, they showed the risks,
they showed the mitigation to the risk. They presented to me and to the team
a technical approach far superior than what was expected as an acceptable.
It was in excess of an acceptable. It showed that the risk would be reduced
to practically nothing, and that's what highly acceptable is.

Tr. at 199.

The contracting officer recognized at the hearing that the source selection
decision was poorly documented, and attempted to provide additional support.
Where post-protest explanations provide sufficient, consistent detail by
which the rationality of an evaluation decision can be judged, it is
possible to conclude that the agency had a reasonable basis for the
decision. See Quality Elevator Co., Inc., B-276750, July 23, 1997, 97-2 CPD
para. 28 at 3-4. Here, however, the explanations provide no such reasonable
basis.

EVALUATION OF CORPORATE EXPERIENCE

Future-Tec and CHM each allege that their own proposals should have received
a higher score under corporate experience than they received, and that SES's
rating for this factor was unreasonably inflated. We agree.

Future-Tec's Corporate Experience

To demonstrate its experience, Future-Tec listed seven contracts in its
proposal, six of which were for the same Navy offices as the current
requirement, five of which listed the same contracting officer's
representative as the one overseeing the current effort. Agency Report, Tab
14, Future-Tec's proposal, at 7-11. Future-Tec also referred to similar
technical support services it had provided to four other federal agencies.
Id. at 6. Future-Tec discussed all 20 of the tasks required under the RFP's
statement of work, noting areas where the firm had designed, developed and
implemented processes that are currently being used.

One of the factual premises underlying Future-Tec's allegations is that, as
the incumbent contractor under the predecessor contracts for these services,
Future-Tec's own experience (for which it received high performance ratings)
was most relevant and should have been rated more highly than any other
offeror's under the corporate experience factor. The Navy disputes
Future-Tec's characterization of itself as the "incumbent," however,
insisting that Future Tec's prior contracts involved only "[a] portion of
the effort anticipated by [this] RFP" and that "significantly more than half
of the anticipated effort had not been included" in Future-Tec's earlier
contracts. Agency Report at 3. In contrast, the Navy contends that a
contract that the [deleted] awarded to SES was "the only contract cited by
any offeror that was actually comparable to the instant requirement in
scope, magnitude and complexity." Agency Report at 8.

The agency's position in this regard is not supported by the record.
Regarding the size of the prior contract and the award at issue here, which
is currently being performed as a result of the override of the stay of
performance ("the current contract"), the contracting officer acknowledged
at the hearing that Future-Tec and its subcontractor, TAI, had performed
work valued at approximately $10 million during the last year of the
previous contracts. [5] Tr. at 77. At the hearing, the contracting officer
was unable to support the Navy's position that the current contract is
substantially larger than Future-Tec's earlier contracts. Indeed, she
acknowledged that, based on orders that have actually been placed, the
current contract is "running very roughly around the same size" as the prior
contracts. Tr. at 110. She stated that while additional work is anticipated,
it has not been funded yet; there had not been a "drastic change from where
Future-Tec and [its subcontractor] left off on September 30 to where SES
picked up;" the requirement was only expected to increase in size over the
5-year course of performance. Id. Regarding the scope of the work under the
prior and current contracts, she acknowledged that under the prior
contracts, Future-Tec and TAI had performed every one of the 20 tasks called
for in the current statement of work. Tr. at 154. Regarding staffing levels,
she acknowledged that the current contract is being performed essentially by
the employees who were performing Future Tec's prior contract (including its
subcontractor's staff), whom she estimated to number around 170. Tr. at
162-63. In addition, while she testified that she gave less credit to a firm
that had performed a number of separate contracts than to a firm that had
performed one larger-volume contract because under smaller contracts "[y]ou
don't have one person managing the full contract," Tr. at 92, she
acknowledged that the separate Future-Tec contracts that were equivalent to
the work being consolidated under the current contract had all been managed
by a single project manager. Tr. at 176.

In spite of the evidence in the record showing that Future-Tec and TAI were
performing essentially the same services as would be required under the
current contract, the contracting officer insisted that Future-Tec "never
ha[s] managed a contract anywhere near the size of this," Tr. at 92, and,
based on this conclusion, rated Future-Tec's proposal as merely acceptable
under corporate experience.

CHM's Corporate Experience

CHM listed 13 contracts in its proposal, several of which were multi-year,
multimillion dollar contracts with values such as $[deleted] million,
$[deleted] million, $[deleted] million, and $[deleted] million, and listed
revenue of $[deleted] million for 1998 and a projected revenue of $[deleted]
million for 1999. It provided a chart showing the 20 tasks that would be
required under the RFP and indicating which of them had been performed under
the previous contracts listed in the proposal. Agency Report, Tab 11, CHM
Technical Proposal, at 7, 9-10. CHM provided a description of each listed
contract, including a separate chart showing which tasks had been performed,
and standard information such as contract name, number, agency, period of
performance and dollar value, as well as a narrative description of the work
performed. Although the contracting officer stated at the hearing that "CHM
and its subcontractors have many more contracts than Future-Tec," Tr. at 98,
and that CHM "did show that they've done all the tasks in the statement of
work" and have "had many contracts" and were "certainly capable of doing the
work, very, very well-liked by the people they had worked for," CHM's
proposal was also rated only acceptable under corporate experience. The only
explanations the contracting officer offered for this rating were that not
all tasks had been performed under a single contract (an allegation that
conflicts with the record); that certain tasks were not discussed in the
narrative of CHM's proposal (although they were shown in the charts); and
that CHM had not shown "risk mitigation that really took the risk away from
me as the government" (although the evaluation identified no particular risk
presented by CHM and noted that CHM's "history of quality past performance
mitigates the risk to the Government on this effort in the areas of cost,
schedule and performance in meeting the requirements of this RFP." Agency
Report, Tab 16, Technical Evaluation, at 1.) Tr. at 89-91.

SES's Corporate Experience

The Navy's evaluation of SES's proposal for this factor provides a telling
contrast. SES's proposal lists [deleted] contracts performed over
approximately the past

10 years, among which only [deleted] 5-year contract with the [deleted]
("the [deleted] contract"), valued at a total of $[deleted] million,
exceeded an average yearly value of $3 million. In connection with the
evaluation of Future-Tec's corporate experience, the contracting officer
stated that contracts "valued at less than $3,000,000 . . . were not
considered to be large enough to be relevant based on the requirement for
contracts to be of similar scope, magnitude and complexity." Agency Report
at 18; Tr. at 141-43. Notwithstanding that statement, the contracting
officer stated that she did consider contracts valued below $3 million per
year when she evaluated SES's corporate experience. [6] Tr. at 135-41. SES's
proposal included a separate description of each of the listed contracts,
and a chart matching the 20 required tasks with the previous contracts'
requirements. The [deleted] contract was clearly the largest one listed,
showing a staffing level of [deleted] full-time equivalents (FTEs), while
the others ranged from [deleted] to [deleted] FTEs. The average staffing
level among all of the contracts that SES listed was [deleted] FTEs. The
evaluation record repeatedly reflects the Navy's conclusion that the
[deleted] contract "was the only contract cited by any offeror that was
actually comparable to the instant requirement in scope, magnitude and
complexity." [7] Agency Report at 8. This contract, and the TEC's reference
to SES's education profile, are the only examples given in the evaluation
record to explain why SES was rated highly acceptable for this factor.

In our view, these examples do not support the rating assigned. While the
description of the [deleted] contract listed a 5-year period of performance,
the contracting officer acknowledged that SES had only been performing under
this contract for 14 months when proposals were evaluated. Tr. at 123-24.
The contracting officer recognized that the [deleted] contract was an
indefinite-delivery contract--i.e., that its volume was determined by the
number and type of orders placed under it for specific tasks, Tr. at 132,
and that the level of performance could vary wildly from year to year. Tr.
at 156. She acknowledged that SES's proposal listed no task orders as having
been placed under the contract, but stated that she based her evaluation on
the total 5-year maximum value of the contract as listed in SES's proposal,
Tr. at 179-80; she did not distinguish between tasks that SES had actually
performed and tasks that might be ordered under future contract years.
Tr. at 126-27. To demonstrate the actual size of the contract thus far, CHM
provided records valuing the total orders placed under the [deleted]
contract in 1998 at $711,000 and in 1999 at $1,488,000. [8] CHM's Comments,
Nov. 19, 199, exh. 1. The evaluation record also refers to the [deleted]-FTE
staffing level for the GSA contract, demonstrating that the Navy was aware
that the staffing level under that contract of "exceptional magnitude and
diversity" represented only a fraction of the staffing level of
approximately 170 FTEs that would be required under the current contract
even at its inception, which proportional relationship it apparently chose
to ignore. [9] The chart provided in SES's proposal to match up the tasks
required under the current contract with previous contracts under which they
had been performed references the [deleted] contract for only 4 of the 20
tasks. Agency Report, Tab 10, SES's Proposal, at 53-54. We find no
reasonable basis to support the agency's assessment that this contract
demonstrated a higher level of corporate experience than the level
demonstrated by any of the other offerors.

The Navy contends that the RFP was "set up so that ‘dollar value'
would be the only available statistic to measure the magnitude of a
contract." Agency's Post-Hearing Comments at 16. However, the RFP nowhere
states that an offeror's experience under prior contracts would be valued
according to the face value of contracts it had been awarded, irrespective
of the extent of performance that actually had taken place. Regarding past
performance, the RFP instructed offerors to identify for each contract held
within the last 5 five years the dollar value of the contract and a detailed
description of the work performed (among other things), and cautioned them
to provide a detailed explanation demonstrating the similarity of the
contracts to the requirements in the RFP. RFP sect. L.III.A.1, at 51. Regarding
corporate experience, offerors were instructed to demonstrate either similar
or directly related experience of similar scope, magnitude and complexity to
the statement of work, and to address the history, organization,
qualifications and work experience within the last 5 years as they relate to
the requirements of the statement of work. RFP sect. L.III.B.3, at 52.

In our view, the methodology actually used by the agency contributed to the
unreasonableness of the evaluation in several ways. It illogically evaluated
past experience on the basis of anticipated future performance. It distorted
the value of some contracts, over-valuing a multi-year contract under which
performance had only begun, while undervaluing multiple single-year
contracts which, when viewed together (where several were performed at the
same time or represented one year in a series of awards), would equal or
exceed the value of multi-year or consolidated contracts. In addition, the
methodology seems to have been unevenly applied, since CHM's proposal, which
listed more multimillion dollar contracts than SES in its proposal, was
nonetheless rated lower in this area.

The only other basis offered by the Navy for rating SES's proposal highly
acceptable under corporate experience is that "SES's education profile was
noteworthy, demonstrating that [deleted]% of the employees have advanced
degrees." Agency Report, Tab 16, Technical Evaluation, at 9, and Tab 31,
Source Selection Memorandum, at 3. We do not see the relevance of advanced
degrees to corporate experience. Moreover, as the contracting officer
acknowledged, the referenced level of education relates only to SES staff
generally--not to the proposed key personnel, whose resumes would be
separately evaluated--and thus is even less relevant as a measure of
corporate experience for an offeror proposing to perform the contract with
the incumbent staff. Tr. at 220-22.

OTHER TECHNICAL EVALUATION ALLEGATIONS

As another example of evaluation improprieties, the protesters challenge the
ratings that the Navy assigned the three proposals under staffing. The Navy
rated SES's proposal highly acceptable under this factor, and rated the
protesters' proposals acceptable. When questioned at the hearing about how
an incoming contractor whose staffing plan was based on capturing the
incumbent workforce (and therefore included elements of uncertainty or risk)
could merit a higher rating than the incumbent, whose staffing plan would
require no transition at all, she emphasized that SES had explained in its
proposal how it would "ramp up through the full amount of the contract" as
the requirements increased. Tr. at 239. We do not find this analysis
persuasive. First, while SES's proposal does discuss supplementing the
incumbent hires "[deleted] as necessary," Agency Report, Tab 9, SES's
Technical Proposal, at 61, it describes its [deleted] methods in very
general terms, such as [deleted]." [10] Id. at 63. In comparison, CHM also
proposed to hire the incumbent staff, and presented a more detailed
methodology for recruiting, hiring and transitioning, and included
discussions of its approach for handling workload fluctuations and surges
and its employee retention rate. Future-Tec, as the incumbent whose staff
the other offerors were planning to capture, did not discuss the
transition--because it would not have any transition--but based its staffing
plan on its first-hand knowledge of the previous requirements. In these
circumstances, where the incumbent has staffed the requirement to the
agency's satisfaction (as shown by Future-Tec's exceptional performance
ratings under its Navy contracts) and proposes to continue doing so, and
where the other two offerors present very similar staffing approaches, we
see no reasonable basis for the SES proposal having been rated more
favorably than the others. Neither the evaluation record nor the contracting
officer's explanations provide any reasonable support for the ratings in
this area.

In sum, the record, including the hearing testimony in this case, offers no
convincing explanation as to why the Navy concluded that SES's proposal was
entitled to a higher overall technical score than the protesters' proposals,
as these examples demonstrate. [11] Since we cannot find reasonable support
for the technical evaluation of the three proposals, as discussed above, we
conclude that the source selection lacks a reasonable basis. We sustain both
Future-Tec's and CHM's protests on this basis.

THE COST REALISM ANALYSIS

CHM alleges that the Navy's adjustment of its proposed BAFO price, which
resulted in an increase of approximately $[deleted] million, was based
solely on a mathematical error by the agency and therefore had no reasonable
basis.

When an agency evaluates proposals for the award of a cost-reimbursement
contract, an offeror's proposed estimated costs are not dispositive, because
the offeror's estimated costs may not provide valid indications of the final
actual costs that the government is required, within certain limits, to pay.
See ManTech Envtl. Tech. Inc., B-271002 et al., June 3, 1996, 96-1 CPD para. 272
at 8. Accordingly, a cost realism analysis must be performed by the agency
whenever a cost-reimbursement contract is contemplated. Federal Acquisition
Regulation (FAR) sect. 15.404-1(d)(2). A cost realism analysis is the process of
independently reviewing and evaluating specific elements of each offeror's
proposed cost estimate to determine whether the estimated proposed cost
elements are realistic for the work to be performed, reflect a clear
understanding of the requirements, and are consistent with the unique
methods of performance and materials described in the offeror's technical
proposal. FAR sect. 15.404-1(d)(1). The requirement to conduct a cost realism
analysis of proposals for a cost-reimbursement contract does not require the
agency to conduct an in-depth cost analysis, see FAR sect. 15.404-1(c), or to
verify each and every item in the proposals. Rather, the analysis of cost
realism calls for the exercise of informed judgment by the contracting
agency involved, since it is in the best position to assess the realism of
proposed costs and it must bear the difficulties or additional expenses
resulting from a defective cost realism analysis. Our review is limited to
determining whether the agency's cost realism analysis was reasonable. The
Warner/Osborn/G&T Joint Venture, B-256641.2, Aug. 23, 1994, 94-2 CPD para. 76 at
5. Here, we conclude that the agency's cost realism analysis, and therefore
its cost realism upward adjustment of approximately $[deleted] million to
CHM's proposed cost, was unreasonable.

CHM's BAFO showed an overhead rate of [deleted]% and a G&A rate of
[deleted]%. Agency Report, Tab 29, CHM BAFO, schedules A, B-1 through B-5.
In the BAFO, CHM had offered to reduce the risk of [deleted] escalation by
[deleted] composite rate (overhead and G&A)" at [deleted]%. [12] Agency
Report, Tab 29, CHM BAFO, at 2. It is these numbers that led to the cost
realism adjustment at issue here.

At the hearing, the contracting officer stated that she had to "try and
figure out what [a composite rate] meant, and I went so far as to look in
the dictionary," Tr. at 16, and that she concluded that the "composite" rate
must mean the sum of the overhead rate and the G&A rate. She therefore added
[deleted]% and [deleted]% to conclude that CHM was proposing a rate of
[deleted]%, which was lower than the [deleted]% [deleted]. [13] In its cost
realism analysis, the Navy indicated that the capped rate CHM had offered
was inconsistent with, and higher than, the individual rates the firm had
offered for overhead and G&A. Agency Report, Tab 33, Post-Negotiation
Business Clearance Memorandum, at 14. Because the agency had concern that
rates might escalate to the [deleted], which was higher than the [deleted]%
rate that the agency believed CHM had proposed, the agency tried to
compensate by increasing CHM's G&A rate. The cost realism adjustment at
issue here resulted when the agency increased CHM's G&A rate by 2.782%,
believing that this adjustment was "the best way to account for the capped
rates . . . ." [14] Id. The result was the cost realism increase of
approximately $[deleted] million.

We find the agency's concern and the resulting adjustment unreasonable. The
schedules in CHM's BAFO show the hourly direct labor rate multiplied by the
[deleted]% overhead rate, with the resulting rate then multiplied by the
[deleted]% G&A rate--that is, the [deleted]% rate is multiplied against a
base that includes the [deleted]% overhead as well as direct costs. The BAFO
also stated, in its summary of its proposed indirect rates, that the
allocation base for overhead was direct labor dollars, and the allocation
base for G&A was total cost input (i.e., direct labor and overhead). Agency
Report, Tab 29, CHM BAFO, schedule A. In our view, it should have been clear
to the agency that CHM's G&A rate of [deleted]% was multiplied times a base
that included both direct costs and overhead. Accordingly, it should also
have been clear that the [deleted]% composite rate simply included both
component rates (that is, overhead and G&A). [15] The [deleted] was thus not
in excess of the proposed rates, and in fact reasonable review of the
schedules in CHM's proposal would have readily shown the agency that the
proposal already incorporated the [deleted]% composite rate. [16]

Because the agency's analysis was based on the mistaken assumption that the
cap was above the proposed rate, there was no reasonable basis for the
agency's cost realism concern, and the resulting adjustment to CHM's
proposed cost was unjustified. The record does not demonstrate that the
agency considered CHM's costs to be unrealistic based on any independent
review of the reasonableness of the proposed rates themselves, or that its
conclusions were substantiated through market surveys or historical cost
data from similar contracts.

As a result, if CHM's proposed total cost of $[deleted] is compared to SES's
adjusted cost of $[deleted], SES's cost is actually approximately [deleted]
percent higher than CHM's price, rather than the [deleted] percent used by
the agency in its evaluation. Accordingly, we conclude that the source
selection, based on both a flawed technical evaluation and an improper cost
realism adjustment for CHM, had no reasonable basis.

THE KEY PERSONNEL ALLEGATION

The protesters also allege that only 1 of the 11 key personnel identified in
SES's proposal has been performing on a full-time basis under the contract
and that SES essentially engaged in "bait and switch" by failing to provide
all of the proposed key personnel as full-time staff.

An offeror may not propose to use specific personnel that it does not expect
to use during contract performance, since to do so would have an adverse
effect on the integrity of the competitive procurement system and generally
provide a basis for proposal rejection. CBIS Fed. Inc., B-245844.2, Mar. 27,
1992, 92-1 CPD para. 308 at 5. The elements of "bait and switch," rendering a
contract award improper, are as follows: (1) the awardee represented in its
proposal that it would rely on certain specified personnel in performing the
services; (2) the agency relied on this representation in evaluating the
proposal; and (3) it was foreseeable that the individuals named in the
proposal would not be available to perform the contract work. Ann Riley &
Assocs., Ltd.--Recon., B-271741.3, Mar. 10, 1997, 97-1 CPD para. 122 at 2-3.

Here, the RFP specifically required offerors to certify the availability of
the key personnel being proposed, even when they were currently employed by
the offeror, and required offerors to certify in their BAFOs that personnel
they currently employed had confirmed their availability after the date of
the request for BAFOs. RFP sect. L.III.A.2.a, at 52. SES proposed to provide all
11 of the required key personnel from its current staff, certifying that
they had confirmed their availability as required. As illustrated by the
initial proposal evaluation, any deficiency in any of the 11 key personnel
resumes was considered by the agency as rendering the proposal unacceptable.
Agency Report at 7.

The protesters essentially allege that SES knew at the time it submitted its
proposal that the proposed key personnel were committed to other projects
and would not be available to work full time under this contract. As
evidence, they point to timesheets submitted by these employees for the
first 2 months of contract performance, showing that only the project
manager has worked full-time for any of the pay periods since the inception
of this contract. The remaining key personnel have been working and continue
to work primarily on the contracts to which SES assigned them prior to the
current contract.

In response, SES states that it has not relocated its personnel for valid
reasons that arose after the award, and not in furtherance of a bait and
switch plan. It cites the following reasons: a directive from the Navy
installation, issued after award, requiring SES to hire all incumbent staff,
creating an overstaffing problem that, in turn, would exhaust available
funding unless SES cut back on certain personnel hours; a decision not to
relocate personnel pending the outcome of this protest; and an unresolved
security issue, which resulted in two proposed employees being denied access
to the facility. The Navy further argues that because SES had obtained
signed statements from its proposed key personnel, their later
"unavailability" was unforeseeable. Agency's Post-Hearing Comments at 60.

The record shows that only one of the 11 proposed personnel has been working
full-time on the contract awarded to SES. Two employees, foreign nationals,
have been ineligible to work (through no fault of SES's) based on unresolved
security concerns at the Navy installation. Two others left SES during the
first week of performance and have been replaced with substitute personnel.
The remaining six persons have not been relocated to New Orleans, instead
commuting to work only part-time on this contract, working an average of 2
to 4 days per week on this contract. Tr. 477-78; SES Post-Hearing Comments
at 14. While we recognize that this represents fewer full-time employees
working on the contract than SES proposed, we see no evidence that SES
misrepresented the availability of key personnel in its proposal. Absent
such evidence, whether proposed employees are, in fact, performing on a
full-time basis under the contract is a matter of contract administration
not subject to our review. 4 C.F.R. sect. 21.5(a) (1999). We therefore deny this
basis of protest.

We recommend that the agency reevaluate all proposals consistent with the
RFP criteria, and reasonably document support for the evaluation and source
selection. We also recommend that the Navy correct the error in its cost
realism adjustment of CHM's proposal, and, if necessary, reevaluate the
proposals for cost realism. Upon completion of its review, we recommend that
the Navy make a new source selection based on those evaluations. If an
offeror other than SES is selected for award as a result of the agency's
reevaluation, the agency should terminate SES's contract for the convenience
of the government. We also recommend that Future-Tec and CHM be reimbursed
their costs of filing and pursuing their protests, including reasonable
attorneys' fees. 4 C.F.R. sect. 21.8(d)(1). The protesters should submit their
certified claims for such costs, detailing the time expended and cost
incurred, directly to the contracting agency within 60 days of receiving
this decision. 4 C.F.R. sect. 21.8(f)(1).

The protests are sustained.

Comptroller General
of the United States

Notes

1. Although this figure is listed as the "realistic" CPFF in the business
clearance memorandum, the contract award value is listed on the first page
of this same document as $81,290,286.00, which is the (unadjusted) cost that
SES proposed.

2. The narratives for Future Tec's and CHM's proposals are identical, except
for the insertion of a different contract number to reference a performance
assessment report for each offeror. Agency Report, Tab 31, Source Selection
Memorandum, at 4.

3. Offerors were required to submit two-volume technical proposals with
hundreds of pages of detailed information and to make oral presentations to
the Navy. The agency assembled its TEC of five individuals from the two
requiring activities, selected for their varied areas of expertise, who
traveled to Philadelphia and spent a full week evaluating the written
portion of the proposals and later returned for an additional 3 days to
evaluate the offerors' oral presentations. The contracting officer provided
guidance to the TEC throughout the process, performing an independent
concurrent evaluation of the proposals, with the assistance of four other
members of her staff. Tr. at 81-84. The complexity of the material itself is
illustrated by the contracting officer's response, when asked at the hearing
to explain a flowchart in SES's proposal that the evaluators had described
as easy to understand, that she could not explain it, stating that "it would
probably take me hours to put all my thoughts together." Tr. at 187.

4. For example, while she initially stated that no credit was given for past
performance of contracts valued at less than $3 million, Tr. at 143, she
later said, "I guess I actually gave them credit for it, because if I didn't
include those [contracts], they would have been unacceptable," Tr. at 146,
demonstrating both confusion about what credit was given and a willingness
to extrapolate from a sparse evaluation record what "must have been
considered."

5. The RFP itself lists Future-Tec and its subcontractor as incumbents and
values their predecessor contract efforts at $13,245,170. Agency Report, Tab
37, RFP amend. 002, Part II.

6. Although the contracting officer implied in testimony that she saw little
value in analyzing the value of multi-year contracts based on an average
yearly value, Tr. at 136-38, she in fact reduced the value Future-Tec listed
for a 2-year contract by half to evaluate it on a per-year basis. Agency
Report, Tab 44, Contracting Officer's Declaration, at 1.

7. Although the contracting officer asserted that the [deleted] contract was
just "an example of a contract," Tr. at 206, 214, and that the Navy "looked
at total corporate experience," Tr. at 214, the fact remains that the
evaluation record cites only this contract, and cites it repeatedly.
Moreover, the remaining contracts listed in SES's proposal are all
significantly smaller, falling as they do below the $3 million evaluation
threshold.

8. Neither SES nor the Navy has disputed these figures. In addition to this
example, the disparity between the magnitude of SES's total corporate
experience as listed in its proposal and as perceived by the contracting
officer is illustrated by her statement at the hearing that "I didn't do an
evaluation of how many dollars they did a year. If I were to do that and I
were to add up all of SES's, I mean, they would be up in the hundreds of
millions . . . ." Tr. at 214-15.

9. The contracting officer indicated that she did not consider staffing
levels to be particularly relevant in determining the magnitude of a
contract, even a contract for services, Tr. at 204-08; she primarily looked
to the contract's total dollar value rather than other indicators of size
such as staffing levels or work actually performed. Tr. at 153; Agency's
Post-Hearing Comments at 16.

10. In light of the emphasis that the contracting officer placed on SES's
ability to "ramp up" to full performance level (and her assertions
concerning the eventual magnitude of the contract), it is noteworthy that
SES expresses the expectation in its proposal "to hire no more than
[deleted] % of the required staff (non-incumbent new hires)" using the
hiring/recruiting process it describes. Agency Report, Tab 9, SES Technical
Proposal, at 63.

11. While both protesters have challenged the technical evaluation in
various other areas, we think the examples discussed are sufficiently
representative of the flaws in the evaluation and will not discuss the
others.

12. CHM actually listed separate composite rates for work that would be
performed on-site and off-site, as required by the RFP; in the interest of
brevity, we will discuss only the rates proposed for government-site work,
since the analysis for the two rates is the same.

13. Although she stated at the hearing that she understood that she could
have asked clarification questions without opening discussions "[I]f I
needed them," Tr. at 34, she did not request any clarification from CHM on
this matter because she concluded that it would not make any difference in
the source selection decision whether she evaluated CHM's proposal based on
the submitted price or the adjusted price. Tr. at 37-38.

14. The Navy's precise calculations that produced this adjustment figure
involved rates for both on-site and off-site rates and are not relevant
here.

15. The use of the composite rate provided a way to take two cumulative
adjustments into account in one step. The composite rate can be calculated
as follows: [deleted], which equals [deleted], or, rounded, [deleted]%.

16. For example, schedule B-1 showed a $[deleted] direct labor rate
escalated by the [deleted]% overhead rate to $[deleted], which was then
multiplied by the [deleted]% G&A rate to $[deleted]. The same result is
reached by multiplying the $[deleted] hourly rate by [deleted]%. If, as the
agency apparently believed, the proposed cost reflected a combined overhead
and G&A rate of only [deleted]%, the loaded rate would have been $[deleted].