BNUMBER: B-277422, B-277422.2
DATE: October 14, 1997
TITLE: ECC International Corporation, B-277422, B-277422.2, October
14, 1997
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective
Order. This redacted version has been approved for public release.
Matter of:ECC International Corporation
File: B-277422, B-277422.2
Date:October 14, 1997
Melvin Rishe, Esq., Howard J. Stanislawski, Esq., and Richard L.
Larach, Esq., Sidley & Austin, for the protester.
Thomas J. Madden, Esq., Jerome S. Gabig, Esq., and Johana A. Reed,
Esq., Venable Baetjer, Howard & Civiletti, LLP, for AAI Corporation,
an intervenor.
Marian E. Sullivan, Esq., and Gregory D. Whitt, Esq., Department of
the Air Force, for the Agency.
Scott H. Riback, Esq., and John M. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest challenging agency's assignment of risk assessment ratings is
denied where record shows that there was a reasonable basis for the
agency's conclusions that protester's proposal presented a high risk
of unsuccessful performance in one technical area and that awardee's
proposal presented only a moderate cost risk.
DECISION
ECC International Corporation protests the award of a contract to AAI
Corporation under request for proposals (RFP) No. F33657-97-R-2009,
issued by the Department of the Air Force for concurrency upgrades and
logistics support of the agency's C-17 aircraft maintenance
training devices. ECC maintains that the Air Force misevaluated
proposals in the area of past performance and made an irrational
cost/technical tradeoff in awarding the contract to AAI. ECC also
maintains that the evaluation reflects bias on the part of the agency
against ECC.
We deny the protest.
The RFP contemplated the award of a combination cost
reimbursement/firm, fixed-price contract on a best value basis to
perform modifications to two suites of C-17 aircraft maintenance
training devices (MTD), as well as ongoing logistics support and the
conduct of a maintenance training analysis study for the C-17 MTD
program. The modifications portion of the contract, the largest
dollar item being procured, is to be performed on a cost reimbursement
basis, while the logistics support and maintenance training analysis
study are to be performed on a firm, fixed-price basis. The MTDs are
used to train agency personnel in various aspects of C-17 aircraft
maintenance. The modifications are necessary in order to make the
configuration of the C-17 MTDs "concurrent" with, or identical to, the
configuration of presently fielded aircraft; design and engineering
changes to the aircraft since the time the C-17 MTDs were originally
built have necessitated the current requirement.
The RFP advised offerors that proposals would be evaluated for cost
realism, reasonableness, and completeness, as well as for technical
adequacy under three primary technical criteria (none of which is at
issue in this protest), and that a proposal risk rating would also be
assigned in each of the technical evaluation areas. In addition, the
RFP advised that firms would be evaluated by a performance risk
assessment group (PRAG) for past performance based on information
supplied with the proposals, as well as information obtained
independently by the agency.
In response to the solicitation, the Air Force received five initial
offers. After evaluating the offers and conducting a performance risk
assessment, the agency determined that it was in the best interest of
the government to make award based on initial offers. ECC's and AAI's
proposals received comparable technical and proposal risk scores, and
ECC's evaluated cost was low. The Air Force nonetheless made award to
AAI, the firm offering the second lowest evaluated cost, based largely
on the conclusion of the PRAG that AAI presented a lower performance
risk than ECC. The agency's performance risk evaluation is the focus
of ECC's protest.
The PRAG conducted its evaluation by reviewing each offeror's
performance under the three technical evaluation areas: Concurrency
Approach/Implementation, Maintenance Training Analysis, and Contractor
Logistics Support. Risk ratings also were assigned for Cost/Price.
Ratings of low, moderate, or high risk were assigned under each of the
four areas unless the agency concluded that there was insufficient
information to evaluate a firm in a given area, in which case the PRAG
assigned a rating of not applicable. In conducting its assessment,
the PRAG reviewed the information submitted in the past performance
volumes of the proposals, Contractor Performance Assessment Reports
(CPAR),[1] and responses to questionnaires prepared by contracting
activities that had administered contracts performed by the firm being
reviewed. The ratings assigned to ECC and AAI for performance risk
were as follows:
CRITERIA AAI ECC
Concurrency/ImplementationModerate High
Maintenance Training AnalysisNot ApplicableNot Applicable
Contractor Logistics SupportLow Low
Cost Moderate Not Applicable
CONCURRENCY APPROACH/IMPLEMENTATION
ECC takes issue for several reasons with the agency's assignment of a
high risk rating to its proposal under the Concurrency
Approach/Implementation[2] criterion, as compared with the assignment
of a moderate risk rating to AAI's proposal. ECC maintains that the
risk evaluation improperly failed to consider its positive performance
under several contracts and at the same time ignored AAI's allegedly
negative performance under several of that firm's contracts. With
respect to those CPARs that were considered, ECC maintains that the
agency essentially ignored criticisms of AAI's performance and also
ignored ECC's comments submitted in response to the CPARs reviewed.
In this latter regard, ECC maintains that the evaluation record fails
to show that the PRAG considered ECC's comments in response to the
CPARs.[3]
In reviewing an agency's performance risk evaluation, our Office
considers only whether the evaluation was reasonable and consistent
with the solicitation's stated evaluation scheme. H.F. Henderson
Indus., B-275017, Jan. 17, 1997, 97-1 CPD para. 27 at 3. In performing
these evaluations, agencies properly may discriminate between past
performance references based on the relevance of the prior contract to
the requirement being solicited, and may attach greater weight to
those contracts reasonably found to be more relevant. Israel Aircraft
Indus., Ltd., MATA Helicopters Div., B-274389 et al., Dec. 6, 1996,
97-1 CPD para. 41 at 9.
ECC's Performance Risk Rating for Concurrency Approach/Implementation
This aspect of the performance risk assessment was reasonable. In
evaluating ECC's past performance, the agency relied heavily on the
CPARs prepared in connection with the firm's performance of the
predecessor production contract for the C-17 MTDs. (ECC was the
contractor for the fabrication of the C-17 MTDs, and the current
contract is essentially a follow-on upgrade contract to that effort.)
The record shows that ECC experienced significant performance
difficulties throughout the course of the predecessor C-17 MTD
contract. In particular, [DELETED]. We find nothing unreasonable in
the agency's assigning ECC a high performance risk rating based on
this information.
There also is no basis for finding that the agency unfairly ignored
ECC's comments submitted during the preparation of the CPARs. During
preparation of the CPARs for the predecessor contract, ECC commented
extensively regarding the negative performance evaluations, and the
record shows that these comments were considered, but ultimately
discounted, by the agency's reviewing official. In responding to
ECC's comments on the final CPAR, the reviewing official explains
that, after examining the criticisms of ECC's performance, discussing
the matter with the cognizant program officials, and reviewing the
firm's comments in response thereto, he found that the CPAR amounted
to a fair but critical evaluation of ECC's performance. The reviewing
official goes on to cite numerous examples where ECC's comments were
not borne out by the facts and concludes:
In addition to the above discrepancies in the contractor's
response, the program team answered all the contractor's points
of dispute in telephone conversations, and during a video
teleconference held solely to confirm the contractor's
shortcomings in meeting contractual requirements. I am confident
that the contractor's performance has been fairly reported.
The record thus shows that the protester's comments were reviewed in
detail at the time they were submitted, and that the agency's
conclusion at the time--that they were at variance with the facts--was
reflected in the CPARs. The PRAG could reasonably have relied on that
conclusion in essentially discounting ECC's comments, and the fact
that they did not specifically state this in their evaluation report
is immaterial in light of the agency's earlier detailed review.
As for the remainder of material relating to ECC's performance risk,
the record shows that there were questionnaire responses for ECC
relating to three other prior contracts. One of these was a "build to
print" requirement, which the agency did not view as especially
relevant to the current requirement, since it did not involve any
design or engineering on the part of the protester. In addition, the
record shows that this questionnaire included negative performance
information about ECC. The agency explains that, as with all
offerors, it did not consider any negative performance information on
which the firm had not had a previous opportunity to comment, since
award was to be made without discussions. As for the remaining two
contracts, the record shows that, while they were relevant and
contained favorable information relating to the protester's
performance, the agency concluded that the information was
insufficient to overcome the negative information included in the
CPARs for ECC's prior C-17 MTD contract. Given the high degree of
relevance of ECC's performance on the C-17 MTD contract ([DELETED]),
we think the agency reasonably accorded substantially less weight to
the more favorable information in these two questionnaires, and
reasonably assigned ECC a high risk rating. See Israel Aircraft
Indus., Ltd., MATA Helicopters Div., supra.[4]
AAI's Performance Risk Rating for Concurrency Approach/Implementation
ECC maintains that the agency essentially ignored negative criticism
of AAI's past performance in assigning AAI a moderate risk rating in
the Concurrency Approach/Implementation area. The record shows that
the agency assigned a moderate risk rating in this area based on
[DELETED] AAI experienced on one of its predecessor contracts; ECC
maintains that AAI also experienced [DELETED]. According to the
protester, AAI should have received a high risk rating based on these
facts.
The record shows that AAI's [DELETED] problems were less severe than
ECC's. In this respect, [DELETED]. ECC also maintains that the
agency ignored deficiencies in AAI's performance [DELETED] under the
same predecessor contract. The record shows, [DELETED]. We conclude
that the record reflects a qualitative difference between the
difficulties experienced by ECC and AAI, and that this difference
provides a reasonable basis for the agency's different risk ratings
for the two firms.
Finally, ECC maintains that the Air Force improperly ignored several
questionnaire responses that contained criticism of AAI's performance
under a number of other contracts. This argument is without merit.
For one of these contracts ([DELETED]. Since the record shows,
however, that the agency was not requiring AAI to make concurrency
upgrades in connection with that contract, we think the agency could
properly view the rating as having only marginal relevance for
purposes of assessing AAI's potential for success under the present
RFP. As for the remaining contract questionnaire (which the agency
concedes it did not consider), the record shows that, although AAI
received some mild criticism in connection with its performance, it
nonetheless received an "adequate" rating in the area identified by
ECC in its protest.
In sum, the record shows that there was a reasonable basis for
distinguishing between AAI's and ECC's past performance records.
There is nothing in the record to show that the agency engaged in
disparate treatment in evaluating performance risk in the Concurrency
Approach/Implementation area for the two firms.
COST RISK ASSESSMENT
ECC maintains that the agency misevaluated cost in connection with the
performance risk assessment. In this respect, the RFP called for the
PRAG to assess risk on the basis of the offerors' performance under
past contracts. The protester maintains that the agency improperly
ignored its positive performance under a prior cost reimbursement
contract, failed to consider information submitted by the offerors
relating to their performance of fixed-price contracts, and improperly
assigned a moderate, rather than a high, cost risk rating to AAI.
[DELETED].
The record shows that ECC submitted information relating to only one
cost reimbursement contract. The agency found that this contract was
of limited value for purposes of predicting the potential cost risk
associated with ECC's performance because the contract had not been
competitively awarded and was for a substantially lower dollar value
than the solicited requirement ([DELETED]). Accordingly, the PRAG
assigned ECC a neutral ("not applicable") cost risk rating. Since, in
our view, the agency acted reasonably in distinguishing this prior
contract from the present requirement on the basis of its much smaller
value, we have no basis to question this aspect of the evaluation.
Similarly, we see nothing unreasonable in the agency's conclusion that
fixed-price contracts would be of limited predictive value, since they
would not provide insight into the offerors' performance in a cost
reimbursement setting.[5]
Finally, with respect to ECC's assertion that the agency should have
assigned a high, rather than a moderate, risk rating to AAI in the
cost risk area, the protester's position amounts to little more than
disagreement with the agency's evaluation conclusions. The record
shows that, [DELETED]. We conclude that the PRAG reasonably assigned
AAI's proposal a moderate risk rating based on AAI's improved
performance over the life of the contract.[6]
MAINTENANCE TRAINING ANALYSIS RISK ASSESSMENT
ECC contends that the agency erred in assigning it a neutral ("not
applicable") risk rating in the Maintenance Training Analysis area.
According to the protester, it should have received a positive rating
in this area because it proposed a subcontractor with an outstanding
performance history. The agency explains that it did not evaluate
ECC's subcontractor's performance risk because the subcontractor was
proposed to perform less than 10 percent of the contract effort; the
agency states that it treated all offerors alike in this regard. The
record further shows that ECC did not include any information in the
performance risk assessment portion of its proposal relating to prior
contracts that had been performed by the subcontractor. We think the
agency's actions in not assigning a maintenance and training analysis
risk rating to ECC was reasonable, given the limited participation of
its subcontractor, the lack of past performance information for the
firm in ECC's proposal, and the fact that all offerors were treated
alike in assessing the performance risk of their subcontractors.
COST/TECHNICAL TRADEOFF & SOURCE SELECTION
ECC contends that the agency failed to conduct a rational
cost/technical tradeoff. To the extent that this argument depends on
the challenges to the Air Force's evaluation of performance risk, it
fails, since, as discussed above, we find no basis to question the
reasonableness of the agency's evaluation. To the extent that the
protester believes that the agency was required to quantify the dollar
value of AAI's superior performance risk rating, the protest is
without merit. There is no requirement that an agency quantify the
value of technical superiority vis-�-vis low cost/price to determine
the best value to the government. Picker Int'l, Inc., B-249699.3,
Mar. 30, 1993, 93-1 CPD para. 275 at 12. On the record here, we find that
the agency has adequately justified its cost/technical tradeoff and
award decision.
The protest is denied.
Comptroller General
of the United States
1. The CPARs are part of the Air Force's ongoing performance review
process. These annual reports review a firm's performance for the
preceding year and, after being prepared, are submitted to the
contractor for comment. After the contractor's comments are
submitted, the CPARs are reviewed by a senior-level agency official
who evaluates the relative merits of the CPARs and contractor comments
and prepares a written opinion regarding the entire package. This
package then is entered into a central Air Force data base which is
available to contracting activities for purposes of conducting past
performance reviews on subsequent procurements.
2. Under this criterion, technical proposals were evaluated under
several subfactors, including the firm's ability to systematically and
timely evaluate data to ensure concurrence between the training
services and actual C-17 aircraft configuration, the firm's ability to
integrate testing and evaluation requirements throughout the life of
the contract so as to avoid resource-intensive, end-of-contract
verification of requirements, and the offeror's schedule and
management approach to ensure initial and continued concurrence
between the training devices and the actual aircraft. In evaluating
performance risk, the agency assigned ratings only to the "area" level
(i.e., the agency did not assign risk ratings for each subfactor), but
it is clear from the record that the same considerations were at
issue. In essence, the offerors' past performance was reviewed to
predict how well the firm might perform the central objectives of the
contract as embodied in the evaluation criteria.
3. ECC also maintains that the contemporaneous record from the
agency's source selection activities shows that important information
was not considered. While it is true that there are several matters
not explicitly addressed in the contemporaneous record, the record as
a whole, including sworn statements submitted by the agency's
cognizant source selection personnel, indicates that the matters were
discussed, even if no contemporaneous record of that discussion was
kept; we therefore conclude that the protester's contention in this
respect is without merit.
4. In a related contention, ECC maintains that the agency's evaluation
and source selection officials were biased against it because of the
firm's performance on the predecessor C-17 MTD contract. As
discussed, we find that the agency's performance risk assessment of
ECC was reasonable. Thus, even if we were to find evidence of bias
(and we have found none here), it would provide no basis for our
Office to sustain ECC's protest since, in order to demonstrate bias, a
protester must also show that the bias translated into action; where
the evaluation is otherwise reasonable, there is no basis for making
this latter finding. Hagler Bailey Consulting, Inc., B-265708, Dec.
19, 1995, 95-2 CPD para. 276 at 3.
5. The assessment of the relevance of performance under fixed-price
contracts is an area where the contemporaneous record does not reflect
the agency's assertion that this matter was discussed at the time and
that it was decided that those contracts were of limited predictive
value. As noted above, we have accepted the agency's assertion that
this matter was discussed, because of the sworn statement to that
effect submitted in response to the protest, the lack of
countervailing argument or evidence, and the reasonableness of the
position itself. We note as well that ECC has failed to demonstrate
prejudice in connection with the agency's alleged failure to consider
fixed-price contract information. [DELETED].
6. ECC also maintains that the agency improperly did not consider
AAI's performance on two other contracts where, according to the
protester, AAI's performance was unsatisfactory. The record shows
that, as to one of these contracts, the requirement was developmental
in nature, that the rating official stated with respect to cost that
there were many "unknowns" at the beginning of performance and that,
as a consequence, there was not a firm target cost against which to
measure performance. The rating official ultimately assigned AAI an
"acceptable" rating for "effective cost performance." The record
shows that the other contract was only in the first year of contract
performance and that costs still were within the government's
evaluated cost estimate used for source selection purposes. This
rating official also ultimately rated AAI's effective cost performance
as "acceptable."