BNUMBER: B-257607.3
DATE: September 12, 1995
TITLE: United International Engineering, Inc.
**********************************************************************
Matter of: United International Engineering, Inc.
File: B-257607.3
Date: September 12, 1995
J.
Patrick McMahon, Esq., McMahon & David, for the protester.
Paralee White, Esq., Laurel Hockey, Esq., and G. Brent Connor, Esq.,
Cohen & White, for EER Systems Corporation, an interested party.
Lt. Col. Ronald K. Heuer, and William L. Strong, Esq., Department of
the Army, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Protest of high performance risk rating in procurement for
software engineering support of battlefield automated systems is
denied where, although protester proposed to subcontract 37 percent of
effort to subcontractors with good performance records, the agency
received reports of past performance problems and demonstrated
inadequate software engineering capabilities on the part of the
protester, and reasonably determined that there existed a significant
risk that the protester lacked the technical expertise necessary in
order to effectively manage the performance of its subcontractors in
the software area.
2. Protest that agency improperly considered as an advantage
incumbent's superior experience with, and resulting knowledge of,
software in weapons systems to be supported under contemplated
contract, and as a disadvantage protester's relative lack of
experience in this regard, is denied where solicitation requested
information with respect to contracts for similar work and expressly
provided for consideration of offerors' relevant and recent experience
under a number of evaluation factors.
DECISION
United International Engineering, Inc. (UIE) protests the award of a
contract to EER Systems Corporation under request for proposals (RFP)
No. DAAH01-94-R-R008, issued as a small disadvantaged business (SDB)
set-aside by the U.S. Army Materiel Command, Army Missile Command
(MICOM), for battlefield automated systems engineering support
(BASES). UIE challenges the evaluation of past performance and
experience and the overall cost/technical trade-off.
We deny the protest.
The solicitation contemplated award of a cost-plus-award-fee contract
for 1 base year with 6 option years for engineering support of the
software engineering directorate (SED), research, development and
engineering center, MICOM, in providing total life cycle support of
battlefield automated systems, including: (1) acquisition and
development of mission critical computer resources; (2) software
verification and validation; (3) command, control and communications
engineering and interoperability testing; (4) requirements definition
for, and development of, software tools; (5) software quality
assurance; and (6) configuration and data management. The
solicitation established 16 required labor categories, setting forth
minimum educational and experience requirements and anticipated levels
of effort for each.
The solicitation generally provided for award to be made to the
responsible offeror whose proposal offered the best value to the
government. The RFP listed four specific evaluation factors: (1)
technical, including technical approach and resumes, which was
described as "most important, and significantly more important than"
(2) performance risk, (3) management and (4) "price," all of which
were of equal importance.
With respect to performance risk, the evaluation of which is at issue
here, the solicitation provided that the agency would
"make a performance risk assessment based upon each offeror's and
his subcontractor's current and past records of performance as
they relate to the probability of successful completion of the
required effort. This assessment will focus on relevant and
recent . . . contractual activity performed since 1 Jan 91. . .
."
The solicitation required offerors to describe government contracts
"received or on-going since 1 Jan 1991 . . . which are similar to the
effort required by this solicitation," including furnishing: (1) the
names and telephone numbers of the procuring and administering
contracting officers and the contracting activity's technical and
other representatives; (2) "a narrative explanation for the Statement
of Work, similarities of that work with the work required by this
solicitation, objectives achieved, and cost growths or schedule delays
encountered"; and (3) an explanation of the reasons for any failure to
"meet original requirements with regard to either cost, schedule, or
technical performance," and of "any corrective actions taken to avoid
recurrence." In addition, the solicitation provided that the
technical, management, and price evaluations would include an
evaluation of "[p]roposal risk" that would "focus on those aspects of
the offeror's proposal which may tend to decrease confidence in the
ability to satisfy the contractual requirement in a timely and
qualitative manner consistent with the proposed approach."
Seven proposals were received by the closing time. Six--including
EER's and UIE's--were included in the competitive range. Following
discussions with the offerors, the agency requested best and final
offers (BAFO).
Based upon its evaluation of BAFOs, MICOM determined that EER had
submitted the proposal offering the best value to the government.
Both UIE's and EER's proposals received above average ratings under
the technical factor, and UIE's proposal received a higher management
rating (above average) than EER's (average) and offered lower proposed
($141,685,273) and evaluated ($146,335,538) prices than did EER's
($162,555,326 proposed and $164,329,203 evaluated). However, MICOM
concluded that UIE's apparent advantage under the management and price
factors was offset by (1) the fact that EER's proposal was evaluated
as superior to UIE's under the technical evaluation elements for
command, control, communications and intelligence engineering and
interoperability testing, software quality assurance, and special
level software engineering, while UIE's proposal was not found to be
superior to EER's under any of the technical approach evaluation
elements; (2) UIE's proposal of lower indirect rates than it
historically had experienced, which MICOM determined "raises some
doubt that UIE can effect its proposed technical approach and staffing
within its proposed cost" and thereby increased overall proposal risk;
and (3) UIE's high performance risk rating, resulting from its
past performance problems in the area of software engineering. MICOM
concluded that EER's proposal represented the best value to the
government, and therefore selected that firm for award.
Upon learning of the selection of EER, UIE and two other offerors
protested EER's SDB status. After the Small Business Administration
(SBA) initially determined that EER was other than an SDB, MICOM
selected Camber Corporation for award. Thereafter, however, SBA
determined that EER was in fact an SDB; MICOM thus rescinded the
selection of Camber and again selected EER. Upon learning of the
resulting award to EER, UIE protested the selection, first to the
General Services Administration Board of Contract Appeals (GSBCA), and
then, after GSBCA declined to accept jurisdiction, to our Office.
PERFORMANCE RISK
UIE challenges MICOM's evaluation of its proposal as characterized by
high performance risk as a result of perceived past performance
problems in the area of software engineering. According to the
protester, MICOM unduly emphasized a single report of inadequate
performance and failed to take into account UIE's overall performance.
In this regard, UIE questions why its proposal received an overall
high performance risk rating and EER's received an overall low
performance risk rating when, in response to the agency's detailed
performance questionnaire, MICOM received more ratings of inadequate
performance for EER (10 of 679 responses) than for UIE (1 of 437
responses). Further, argues UIE, even if its prior performance with
respect to software engineering was reasonably found to be
unsatisfactory, this did not support MICOM's high performance risk
appraisal since UIE had proposed to subcontract software engineering
to firms with past performance records considered satisfactory by the
agency.
In reviewing an agency's evaluation of proposals, we consider whether
it was in accord with the evaluation criteria listed in the
solicitation and whether the actual evaluation was reasonable. John
Brown U.S. Servs., Inc., B-258158 et al., Dec. 21, 1994, 95-1 CPD
35; Information Sys. & Networks Corp., 69 Comp. Gen. 284 (1990), 90-1
CPD 203.
The evaluation of UIE's past performance and the resulting high
performance risk rating were reasonable. Notwithstanding UIE's claim
to the contrary, the record establishes that MICOM based its risk
assessment on a number of reports concerning UIE's performance under
several contracts, which called into question the probability of UIE's
successfully completing the required effort. UIE identified 10
contracts, of which 9 were considered relevant to this contract effort
and were reviewed by the agency.
MICOM received reports of performance problems under at least four of
these contracts; the criticism of UIE's performance on three of these
was considered sufficiently serious by MICOM as to raise "serious
doubts that the UIE team could successfully perform this effort if
managed by UIE and with two-thirds of the technical work proposed to
be done by UIE."
On the largest contract referenced by UIE (DAAH07-90-C-0008), and the
one involving the most similar work--software engineering support--the
contracting officer's technical representative (COTR) reported that:
"[i]n the area of software development, UIE has demonstrated an
inability to plan, develop, and implement procedures which result
in the delivery of high quality products and services on schedule
and within budget. Despite encountering problems in the software
development area, UIE has provided virtually no training for
their on-site personnel in this area, nor does it appear that
resources exist at the corporate level to provide assistance."
The COTR characterized UIE's software as "band aid, glitchy code"; he
stated that he knew of "no [software] product they can market."
Problems with UIE's performance led the contracting agency to issue a
cure notice to UIE in March 1994 requiring correction of unacceptable
software, and even then, according to the COTR, it took several
attempts by UIE and "lots of help from [a] special government team" to
get an acceptable product. Likewise, according to the contracting
agency's technical inspector, one of the sources identified by UIE in
its response, and described by the protester as being "in the best
position to judge the technical competence and performance of UIE,"
UIE "requires too much government direction, i.e., government has to
show contractor how to do job, quality control."
Further, it does not appear that UIE's performance difficulties under
contract -0008 were limited to the failure to perform which led to
issuance of the cure notice. The contracting officer stated in the
cure notice that "the unacceptable product initially produced by UIE .
. . is not an isolated incident"; "[i]t appears that this is
symptomatic of UIE's inability to plan, develop, and implement
procedures which result in the delivery of high quality products on
schedule and within budget." Although UIE responded during
negotiations to MICOM's expression of concern regarding UIE's
performance under this contract by noting that the contract had been
extended, both the COTR and the contracting officer advised MICOM that
the extension was issued only because all of the contracting agency's
similar contracts were being consolidated and the extension of UIE's
contract was necessary so that the contracts would end on the same
date. Likewise, while UIE enclosed with its BAFO two letters from the
contracting agency commending a number of contractor personnel for
their efforts, UIE failed to furnish the cover letter to the letters
of commendation, which specifically noted that the letters:
"recognize individual contributions. This, however, is not an
indication that as a company, United Engineering's overall performance
was up to the desired level." In summary, reported the COTR, the
contracting agency had "not been able to confidently assign support
requirements to UIE with the expectation that the requirements will be
understood and executed in an effective manner."
While issuance of the cure notice led to replacement of UIE's program
manager (PM) under contract -0008, and the new PM's efforts to cure
the problems resulted in reports that UIE was "doing better"
(according to the COTR) and that there had been no problems in the
last 6 months (contracting officer), the COTR reported that UIE "still
[has a] substantial learning curve. SW [software] is still weak."
MICOM reports that the contracting agency's technical
inspector--again, the agency official described by the protester as
being "in the best position to judge the technical competence and
performance of UIE"--similarly advised that "[o]n software engineering
efforts, UIE doesn't currently have anyone to do the job."
MICOM also relied upon reports that UIE had encountered performance
problems under other contracts. For example, under contract No.
DAAH01-89D-0139, for technical support, including software modeling,
the COTR reported that UIE was "not up on state-of-art" in the
software field, resulting in UIE being "[v]ery slow getting started"
and having "[s]everal false starts," which led to revision of the
delivery schedule. According to the COTR:
"[w]hen [we] finally got [an] acceptable product, it was due to
lots of government work. They were supposed to be experts and
government was more expert than they were.
"[UIE] [w]anted to do good job but didn't have what it took to do
good job."
Although MICOM received a generally favorable report when, at UIE's
suggestion during negotiations, it contacted the technical point of
contact for one of the task orders under contract -0139, that task did
not involve software engineering. Moreover, even that contact
cautioned that because UIE was "very penny wise" and lacked
initiative, UIE "only gives what [you] ask for." He added that since
they are a small company and lack resources, the agency can only "give
them things [they are] good at." According to the agency, the contact
questioned whether UIE could perform as well on the sizable contract
effort contemplated here as it could on a smaller contract effort.
Under another contract (No. DAAH01-92-C-0312), for system engineering
and integration support, the engineering supervisor reported that
UIE's weakness was in "[i]n-depth technical expertise." According to
the engineering supervisor, UIE "[did not] have staff"; as a result,
the "[p]rime doesn't do engineering. Sub does it." Although UIE
claimed during negotiations that it had provided one-third of the
engineers supporting the project office, the supervisor of the
engineering supervisor confirmed that the project office looks to the
subcontractor for all technical support. The overall supervisor
stated that:
"the project office does not look to UIE for technical support
and doubts [it] could get it from them. [The subcontractor]
provides all leadership and direction to the UIE employees, and
[the subcontractor] provides all the technical support to the
project office."
MICOM reasonably concluded that these reports of past performance
problems and demonstrated inadequate software engineering capabilities
raised "serious doubts that the UIE team could successfully perform
this effort if managed by UIE and with two-thirds of the technical
work proposed to be done by UIE."
The fact that UIE proposed to subcontract approximately 37 percent
of the contract effort to subcontractors that received good to
superior software engineering experience ratings does not change our
conclusion. This is because, while the subcontractors in question
were to perform approximately 37 percent of the overall contract
effort, UIE's proposal indicated that UIE would take the lead in
significant areas of contract performance, including contract
technical management, program management, task requirements,
acquisition/development engineering support, systems and management
analysis and assistance, maintaining state-of-the-art practice,
operational and acceptance testing, data collection, reduction and
analysis, and software verification and validation. MICOM reports
that UIE's prior performance problems are related to these areas. UIE
does not adequately explain, nor is it otherwise evident, why its
performance problems and demonstrated weaknesses with respect to
software engineering capability are not relevant to the significantly
larger portion of the contract that UIE would perform. In addition,
given the reported weaknesses in UIE's software engineering
capabilities, we think the agency reasonably was concerned that there
was a significant risk that UIE lacked the technical expertise
necessary to effectively manage the performance of its subcontractors
in the software area.
The fact that MICOM received more ratings of inadequate performance
for EER (10 of 679 responses) than for UIE (1 of 437 responses) does
not demonstrate that the performance risk ratings for UIE (high) and
EER (low) were unreasonable. The difference in risk ratings
essentially reflected the fact that MICOM had received reports of
significant problems on the part of UIE, but not EER, in meeting
performance and delivery requirements related to software engineering.
As noted by MICOM, while UIE received an inadequate performance rating
in the technical area, and a total of five adequate/inadequate ratings
in the technical and delivery areas, EER received no rating lower than
good in the technical area or lower than adequate in the delivery
area. Rather, seven of EER's inadequate ratings were in the area of
cost (with one in the personnel area and two in the subcontractor
area). MICOM found that EER's prior cost difficulties resulted from a
number of factors, including the failure to anticipate the complexity
of the required work and delays in obtaining clearances for proposed
staff. Based on EER's proposed approach to cost control under the
contemplated contract, and the fact that EER was the incumbent
contractor and possessed an already cleared staff (and, presumably,
knowledge of the contract requirements), MICOM determined that EER's
prior cost difficulties were unlikely to reoccur here. Having
evaluated EER as demonstrating outstanding past performance with
respect to schedule, management and quality, MICOM assigned EER a low
performance risk rating; there is no basis for finding this
determination unreasonable. Nor, as discussed above, has the
protester furnished any basis for questioning the agency's
determination that the reports of UIE's past performance problems in
the technical and delivery areas and demonstrated inadequate software
engineering capabilities raised serious doubts as to whether UIE could
successfully perform the contemplated contract effort. We conclude
that the agency reasonably determined that UIE's past technical and
delivery problems in the software engineering area were more
indicative of high risk in performance of a software engineering
contract than were EER's past cost problems. UIE's proposal thus
warranted a higher performance risk rating.
RELEVANT EXPERIENCE
UIE claims that the source selection authority (SSA) improperly took
into consideration EER's experience as the incumbent contractor and
the fact that, as a result of that experience, EER possessed more SED
and MICOM-related experience. For example, the SSA stated in this
regard that:
"EER is the incumbent contractor for the current BASES effort.
Their response to the RFP SOW was thorough. . . . They have a
good knowledge of the software and computer resources in the
tactical weapons systems at SED, and current working knowledge of
the tools and procedures needed to perform the RFP SOW [statement
of work] requirements. The proposal presented a very good
description of their experience with the ADA programming language
and with other languages used in tactical weapons systems. EER
also presented their in-depth knowledge and experience of
software development in accordance with [Department of Defense
Standard No. 2167A (DOD-STD-2167A)]. This extensive
background experience and capability made the proposal risk low.
The claims made by EER were substantiated by numerous examples of
support to SED projects."
In addition, UIE argues that MICOM improperly took into consideration
UIE's relative lack of experience with MICOM systems and the fact that
UIE's inadequate rating was received for the contract (-0008)
involving "the most MICOM experience."
We find no improper consideration of offerors' experience. The
solicitation expressly provided for consideration of relevant
experience under a number of evaluation factors. The RFP stated that
the agency would make a performance risk assessment based upon each
offeror's "relevant and recent" experience as it related "to the
probability of successful completion of the required effort." For the
cited contracts, offerors were required to explain the "similarities
of that work with the work required by this solicitation." Further,
the solicitation provided that the required proposal risk evaluation
would "focus on those aspects of the offeror's proposal which may tend
to increase confidence in the ability to satisfy the contractual
requirements in a timely and qualitative manner," including such
factors as an offeror's "lack of corporate experience directly related
to this requirement." In addition, under the technical evaluation
factor, the solicitation provided for consideration of (1) "the
offeror's experience with the development and evaluation of software
in Ada, higher order languages, in assembly languages and the
languages of [weapons] systems shown in Appendix B of the SOW"; and
(2) "the offeror's experience and knowledge with DOD-STD-2167A." In
view of these solicitation provisions providing for consideration of
relevant experience, and given the fact that the contemplated contract
called for furnishing software engineering support to SED/MICOM in
performance of its mission to support battlefield automated systems,
MICOM properly considered as an advantage EER's superior experience
with, and resulting knowledge of, SED/MICOM-supported systems, and as
a disadvantage UIE's relative lack of experience in this regard.
Likewise, the agency could properly take into account, in determining
"the probability of successful completion of the required effort," the
fact that UIE encountered performance problems on a contract for
related services.
PROPOSAL RISK
UIE challenges MICOM's assessment of an increased proposal risk as a
result of UIE's proposal of inadequate indirect rates. In this
regard, as noted above, the solicitation generally provided that an
evaluation of proposal risk would be undertaken as part of the
technical, management and price evaluations, and that this evaluation
would "focus on those aspects of the offeror's proposal which may tend
to decrease confidence in the ability to satisfy the contractual
requirement in a timely and qualitative manner consistent with the
proposed approach."
The Defense Contract Audit Agency (DCAA), in examining the realism of
UIE's initially proposed costs, noted that UIE had proposed indirect
rates which in many instances were below its actual, incurred rates
for fiscal year 1993; this discrepancy contributed to DCAA's overall
finding of a potential underestimation of costs by UIE. UIE had
proposed a newly created division for performance of this contract,
and DCAA reportedly believed that it lacked the data to recommend
specific indirect rates; instead, DCAA generally recommended that
MICOM consider negotiating indirect rate ceilings with UIE. Although,
after being advised during negotiations of the agency's concern in
this regard, UIE slightly increased most of the proposed general and
administrative rates in its BAFO, the protester lowered its proposed
labor overhead rates still further, resulting in an overall net
decrease in its proposed indirect rates. MICOM reports that, as a
result, UIE's "Most Probable Price [--i.e., evaluated cost--] . .
. incorporated the initially proposed rates," that is, was adjusted
upward by approximately $2.4 million.
Given this upward adjustment in evaluated cost, UIE challenges MICOM's
concurrent determination that UIE's proposal of lower indirect rates
than it historically incurred "raises some doubt that UIE can effect
its proposed technical approach and staffing within its proposed cost"
and thereby increased overall proposal risk. UIE argues that it was
improper for the agency to assign proposal risk as a result of the
inadequate indirect rates when it was also adjusting UIE's evaluated
cost upward to account for them.
UIE's position incorrectly assumes that the upward adjustment in its
evaluated cost accounted for and corrected its inadequate indirect
rates. In fact, reports MICOM, "the adjustments did not eliminate the
concern that UIE could well incur higher indirect rates than those
proposed." In other words, adjusting the BAFO indirect rates upward
to the levels proposed in the initial proposal did not, in the
agency's view, fully account for DCAA's concern that the initial rates
themselves were lower than what UIE historically had incurred and were
inadequate. The overall proposal risk therefore was increased to
reflect the agency's concern that the actual cost of performance could
be even higher than the evaluated cost used in the cost/technical
tradeoff. We find no basis for questioning the assessment of an
increase in proposal risk on this basis.
COST/TECHNICAL TRADEOFF
UIE challenges the overall cost/technical tradeoff on the basis that
MICOM has failed to identify and quantify the value of any technical
benefits that would warrant award to the higher-cost offeror.
In a negotiated procurement, there is no requirement that award be
made on the basis of lowest cost unless the RFP so specifies. Henry
H. Hackett & Sons, B-237181, Feb. 1, 1990, 90-1 CPD 136.
Cost/technical tradeoffs may be made in deciding between competing
proposals; the propriety of such a tradeoff turns not on the
difference in technical scores or ratings per se, but on whether the
agency's judgment concerning the significance of that difference was
reasonable and adequately justified in light of the RFP evaluation
scheme. Brunswick Defense, B-255764, Mar. 30, 1994, 94-1 CPD 225.
Federal Acquisition Regulation 15.612(d)(2) requires that
documentation supporting the selection decision show the relative
differences among proposals; their strengths, weaknesses and risks;
and the basis and reasons for the decision. There is no requirement,
however, that selection of a higher-cost proposal be justified through
an exact quantification of the dollar value to the agency of the
proposal's technical superiority. Picker Int'l, Inc., B-249699.3,
Mar. 30, 1993, 93-1 CPD 275. Further, even where a selection
official does not specifically discuss the cost/technical tradeoff in
the selection decision document, we will not object to the tradeoff if
it is clearly supported by the record. Maytag Aircraft Corp.,
B-237068.3, Apr. 26, 1990, 90-1 CPD 430.
The record supports the agency's cost/technical tradeoff. Although
EER's proposal received the same overall rating as UIE's under the
technical evaluation factor, which was significantly more important
than the remaining evaluation factors, the SSA determined that as a
result of superiority with respect to command, control, communications
and intelligence engineering and interoperability testing, software
quality assurance, and special level software engineering, EER's
proposal in fact was superior to UIE's under the overall technical
factor. At the same time, UIE's past performance record was viewed as
casting serious doubt on whether it could successfully perform the
required contract effort. Further, although the evaluated cost of
UIE's proposal was somewhat lower than EER's, the agency determined
that there was some risk that UIE's actual cost of performance would
exceed its evaluated cost, thereby diminishing UIE's advantage in this
regard. In light of these considerations, we think the agency
reasonably determined that, notwithstanding the higher management
rating and lower evaluated cost of UIE's proposal, EER's proposal was
most advantageous under the stated evaluation criteria.
The protest is denied.
/s/ Ronald Berger
for Robert P. Murphy
General Counsel