BNUMBER:  B-277422, B-277422.2 
DATE:  October 14, 1997
TITLE: ECC International Corporation, B-277422, B-277422.2, October
14, 1997
**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective 
Order.  This redacted version has been approved for public release.
Matter of:ECC International Corporation

File:     B-277422, B-277422.2

Date:October 14, 1997

Melvin Rishe, Esq., Howard J. Stanislawski, Esq., and Richard L. 
Larach, Esq., Sidley & Austin, for the protester.
Thomas J. Madden, Esq., Jerome S. Gabig, Esq., and Johana A. Reed, 
Esq., Venable Baetjer, Howard & Civiletti, LLP, for AAI Corporation, 
an intervenor.
Marian E. Sullivan, Esq., and Gregory D. Whitt, Esq., Department of 
the Air Force, for the Agency.
Scott H. Riback, Esq., and John M. Melody, Esq., Office of the General 
Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest challenging agency's assignment of risk assessment ratings is 
denied where record shows that there was a reasonable basis for the 
agency's conclusions that protester's proposal presented a high risk 
of unsuccessful performance in one technical area and that awardee's 
proposal presented only a moderate cost risk. 

DECISION

ECC International Corporation protests the award of a contract to AAI 
Corporation under request for proposals (RFP) No. F33657-97-R-2009, 
issued by the Department of the Air Force for concurrency upgrades and 
logistics support of the agency's     C-17 aircraft maintenance 
training devices.  ECC maintains that the Air Force misevaluated 
proposals in the area of past performance and made an irrational 
cost/technical tradeoff in awarding the contract to AAI.  ECC also 
maintains that the evaluation reflects bias on the part of the agency 
against ECC.  

We deny the protest.

The RFP contemplated the award of a combination cost 
reimbursement/firm, fixed-price contract on a best value basis to 
perform modifications to two suites of C-17 aircraft maintenance 
training devices (MTD), as well as ongoing logistics support and the 
conduct of a maintenance training analysis study for the C-17 MTD 
program.  The modifications portion of the contract, the largest 
dollar item being procured, is to be performed on a cost reimbursement 
basis, while the logistics support and maintenance training analysis 
study are to be performed on a firm, fixed-price basis.  The MTDs are 
used to train agency personnel in various aspects of C-17 aircraft 
maintenance.  The modifications are necessary in order to make the 
configuration of the C-17 MTDs "concurrent" with, or identical to, the 
configuration of presently fielded aircraft; design and engineering 
changes to the aircraft since the time the C-17 MTDs were originally 
built have necessitated the current requirement.

The RFP advised offerors that proposals would be evaluated for cost 
realism, reasonableness, and completeness, as well as for technical 
adequacy under three primary technical criteria (none of which is at 
issue in this protest), and that a proposal risk rating would also be 
assigned in each of the technical evaluation areas.  In addition, the 
RFP advised that firms would be evaluated by a performance risk 
assessment group (PRAG) for past performance based on information 
supplied with the proposals, as well as information obtained 
independently by the agency. 

In response to the solicitation, the Air Force received five initial 
offers.  After evaluating the offers and conducting a performance risk 
assessment, the agency determined that it was in the best interest of 
the government to make award based on initial offers.  ECC's and AAI's 
proposals received comparable technical and proposal risk scores, and 
ECC's evaluated cost was low.  The Air Force nonetheless made award to 
AAI, the firm offering the second lowest evaluated cost, based largely 
on the conclusion of the PRAG that AAI presented a lower performance 
risk than ECC.  The agency's performance risk evaluation is the focus 
of ECC's protest.

The PRAG conducted its evaluation by reviewing each offeror's 
performance under the three technical evaluation areas:  Concurrency 
Approach/Implementation, Maintenance Training Analysis, and Contractor 
Logistics Support.  Risk ratings also were assigned for Cost/Price.  
Ratings of low, moderate, or high risk were assigned under each of the 
four areas unless the agency concluded that there was insufficient 
information to evaluate a firm in a given area, in which case the PRAG 
assigned a rating of not applicable.  In conducting its assessment, 
the PRAG reviewed the information submitted in the past performance 
volumes of the proposals, Contractor Performance Assessment Reports 
(CPAR),[1] and responses to questionnaires prepared by contracting 
activities that had administered contracts performed by the firm being 
reviewed.  The ratings assigned to ECC and AAI for performance risk 
were as follows:

   CRITERIA                 AAI          ECC

   Concurrency/ImplementationModerate    High

   Maintenance Training AnalysisNot ApplicableNot Applicable

   Contractor Logistics SupportLow       Low

   Cost                     Moderate     Not Applicable 
CONCURRENCY APPROACH/IMPLEMENTATION

ECC takes issue for several reasons with the agency's assignment of a 
high risk rating to its proposal under the Concurrency 
Approach/Implementation[2] criterion, as compared with the assignment 
of a moderate risk rating to AAI's proposal.  ECC maintains that the 
risk evaluation improperly failed to consider its positive performance 
under several contracts and at the same time ignored AAI's allegedly 
negative performance under several of that firm's contracts.  With 
respect to those CPARs that were considered, ECC maintains that the 
agency essentially ignored criticisms of AAI's performance and also 
ignored ECC's comments submitted in response to the CPARs reviewed.  
In this latter regard, ECC maintains that the evaluation record fails 
to show that the PRAG considered ECC's comments in response to the 
CPARs.[3]

In reviewing an agency's performance risk evaluation, our Office 
considers only whether the evaluation was reasonable and consistent 
with the solicitation's stated evaluation scheme.  H.F. Henderson 
Indus., B-275017, Jan. 17, 1997, 97-1 CPD  para.  27 at 3.  In performing 
these evaluations, agencies properly may discriminate between past 
performance references based on the relevance of the prior contract to 
the requirement being solicited, and may attach greater weight to 
those contracts reasonably found to be more relevant.  Israel Aircraft 
Indus., Ltd., MATA Helicopters Div., B-274389 et al., Dec. 6, 1996, 
97-1 CPD  para.  41 at 9.

ECC's Performance Risk Rating for Concurrency Approach/Implementation

This aspect of the performance risk assessment was reasonable.  In 
evaluating ECC's past performance, the agency relied heavily on the 
CPARs prepared in connection with the firm's performance of the 
predecessor production contract for the C-17 MTDs.  (ECC was the 
contractor for the fabrication of the C-17 MTDs, and the current 
contract is essentially a follow-on upgrade contract to that effort.)  
The record shows that ECC experienced significant performance 
difficulties throughout the course of the predecessor C-17 MTD 
contract.  In particular, [DELETED].  We find nothing unreasonable in 
the agency's assigning ECC a high performance risk rating based on 
this information.

There also is no basis for finding that the agency unfairly ignored 
ECC's comments submitted during the preparation of the CPARs.  During 
preparation of the CPARs for the predecessor contract, ECC commented 
extensively regarding the negative performance evaluations, and the 
record shows that these comments were considered, but ultimately 
discounted, by the agency's reviewing official.  In responding to 
ECC's comments on the final CPAR, the reviewing official explains 
that, after examining the criticisms of ECC's performance, discussing 
the matter with the cognizant program officials, and reviewing the 
firm's comments in response thereto, he found that the CPAR amounted 
to a fair but critical evaluation of ECC's performance.  The reviewing 
official goes on to cite numerous examples where ECC's comments were 
not borne out by the facts and concludes:

     In addition to the above discrepancies in the contractor's 
     response, the program team answered all the contractor's points 
     of dispute in telephone conversations, and during a video 
     teleconference held solely to confirm the contractor's 
     shortcomings in meeting contractual requirements.  I am confident 
     that the contractor's performance has been fairly reported.

The record thus shows that the protester's comments were reviewed in 
detail at the time they were submitted, and that the agency's 
conclusion at the time--that they were at variance with the facts--was 
reflected in the CPARs.  The PRAG could reasonably have relied on that 
conclusion in essentially discounting ECC's comments, and the fact 
that they did not specifically state this in their evaluation report 
is immaterial in light of the agency's earlier detailed review.

As for the remainder of material relating to ECC's performance risk, 
the record shows that there were questionnaire responses for ECC 
relating to three other prior contracts.  One of these was a "build to 
print" requirement, which the agency did not view as especially 
relevant to the current requirement, since it did not involve any 
design or engineering on the part of the protester.  In addition, the 
record shows that this questionnaire included negative performance 
information about ECC.  The agency explains that, as with all 
offerors, it did not consider any negative performance information on 
which the firm had not had a previous opportunity to comment, since 
award was to be made without discussions.  As for the remaining two 
contracts, the record shows that, while they were relevant and 
contained favorable information relating to the protester's 
performance, the agency concluded that the information was 
insufficient to overcome the negative information included in the 
CPARs for ECC's prior C-17 MTD contract.  Given the high degree of 
relevance of ECC's performance on the C-17 MTD contract ([DELETED]), 
we think the agency reasonably accorded substantially less weight to 
the more favorable information in these two questionnaires, and 
reasonably assigned ECC a high risk rating.  See Israel Aircraft 
Indus., Ltd., MATA Helicopters Div., supra.[4] 

AAI's Performance Risk Rating for Concurrency Approach/Implementation

ECC maintains that the agency essentially ignored negative criticism 
of AAI's past performance in assigning AAI a moderate risk rating in 
the Concurrency Approach/Implementation area.  The record shows that 
the agency assigned a moderate risk rating in this area based on 
[DELETED] AAI experienced on one of its predecessor contracts; ECC 
maintains that AAI also experienced [DELETED].  According to the 
protester, AAI should have received a high risk rating based on these 
facts.

The record shows that AAI's [DELETED] problems were less severe than 
ECC's.  In this respect, [DELETED].  ECC also maintains that the 
agency ignored deficiencies in AAI's performance [DELETED] under the 
same predecessor contract.  The record shows, [DELETED].  We conclude 
that the record reflects a qualitative difference between the 
difficulties experienced by ECC and AAI, and that this difference 
provides a reasonable basis for the agency's different risk ratings 
for the two firms.  

Finally, ECC maintains that the Air Force improperly ignored several 
questionnaire responses that contained criticism of AAI's performance 
under a number of other contracts.  This argument is without merit.  
For one of these contracts ([DELETED].  Since the record shows, 
however, that the agency was not requiring AAI to make concurrency 
upgrades in connection with that contract, we think the agency could 
properly view the rating as having only marginal relevance for 
purposes of assessing AAI's potential for success under the present 
RFP.  As for the remaining contract questionnaire (which the agency 
concedes it did not consider), the record shows that, although AAI 
received some mild criticism in connection with its performance, it 
nonetheless received an "adequate" rating in the area identified by 
ECC in its protest.

In sum, the record shows that there was a reasonable basis for 
distinguishing between AAI's and ECC's past performance records.  
There is nothing in the record to show that the agency engaged in 
disparate treatment in evaluating performance risk in the Concurrency 
Approach/Implementation area for the two firms.  

COST RISK ASSESSMENT

ECC maintains that the agency misevaluated cost in connection with the 
performance risk assessment.  In this respect, the RFP called for the 
PRAG to assess risk on the basis of the offerors' performance under 
past contracts.  The protester maintains that the agency improperly 
ignored its positive performance under a prior cost reimbursement 
contract, failed to consider information submitted by the offerors 
relating to their performance of fixed-price contracts, and improperly 
assigned a moderate, rather than a high, cost risk rating to AAI.  
[DELETED].

The record shows that ECC submitted information relating to only one 
cost reimbursement contract.  The agency found that this contract was 
of limited value for purposes of predicting the potential cost risk 
associated with ECC's performance because the contract had not been 
competitively awarded and was for a substantially lower dollar value 
than the solicited requirement ([DELETED]).  Accordingly, the PRAG 
assigned ECC a neutral ("not applicable") cost risk rating.  Since, in 
our view, the agency acted reasonably in distinguishing this prior 
contract from the present requirement on the basis of its much smaller 
value, we have no basis to question this aspect of the evaluation.  
Similarly, we see nothing unreasonable in the agency's conclusion that 
fixed-price contracts would be of limited predictive value, since they 
would not provide insight into the offerors' performance in a cost 
reimbursement setting.[5]  

Finally, with respect to ECC's assertion that the agency should have 
assigned a high, rather than a moderate, risk rating to AAI in the 
cost risk area, the protester's position amounts to little more than 
disagreement with the agency's evaluation conclusions.  The record 
shows that, [DELETED].  We conclude that the PRAG reasonably assigned 
AAI's proposal a moderate risk rating based on AAI's improved 
performance over the life of the contract.[6]

MAINTENANCE TRAINING ANALYSIS RISK ASSESSMENT

ECC contends that the agency erred in assigning it a neutral ("not 
applicable") risk rating in the Maintenance Training Analysis area.  
According to the protester, it should have received a positive rating 
in this area because it proposed a subcontractor with an outstanding 
performance history.  The agency explains that it did not evaluate 
ECC's subcontractor's performance risk because the subcontractor was 
proposed to perform less than 10 percent of the contract effort; the 
agency states that it treated all offerors alike in this regard.  The 
record further shows that ECC did not include any information in the 
performance risk assessment portion of its proposal relating to prior 
contracts that had been performed by the subcontractor.  We think the 
agency's actions in not assigning a maintenance and training analysis 
risk rating to ECC was reasonable, given the limited participation of 
its subcontractor, the lack of past performance information for the 
firm in ECC's proposal, and the fact that all offerors were treated 
alike in assessing the performance risk of their subcontractors.

COST/TECHNICAL TRADEOFF & SOURCE SELECTION

ECC contends that the agency failed to conduct a rational 
cost/technical tradeoff.  To the extent that this argument depends on 
the challenges to the Air Force's evaluation of performance risk, it 
fails, since, as discussed above, we find no basis to question the 
reasonableness of the agency's evaluation.  To the extent that the 
protester believes that the agency was required to quantify the dollar 
value of AAI's superior performance risk rating, the protest is 
without merit.  There is no requirement that an agency quantify the 
value of technical superiority vis-ï¿½-vis low cost/price to determine 
the best value to the government.  Picker Int'l, Inc., B-249699.3, 
Mar. 30, 1993, 93-1 CPD  para.  275 at 12.  On the record here, we find that 
the agency has adequately justified its cost/technical tradeoff and 
award decision.

The protest is denied.

Comptroller General
of the United States

1. The CPARs are part of the Air Force's ongoing performance review 
process.  These annual reports review a firm's performance for the 
preceding year and, after being prepared, are submitted to the 
contractor for comment.  After the contractor's comments are 
submitted, the CPARs are reviewed by a senior-level agency official 
who evaluates the relative merits of the CPARs and contractor comments 
and prepares a written opinion regarding the entire package.  This 
package then is entered into a central Air Force data base which is 
available to contracting activities for purposes of conducting past 
performance reviews on subsequent procurements. 

2. Under this criterion, technical proposals were evaluated under 
several subfactors, including the firm's ability to systematically and 
timely evaluate data to ensure concurrence between the training 
services and actual C-17 aircraft configuration, the firm's ability to 
integrate testing and evaluation requirements throughout the life of 
the contract so as to avoid resource-intensive, end-of-contract 
verification of requirements, and the offeror's schedule and 
management approach to ensure initial and continued concurrence 
between the training devices and the actual aircraft.  In evaluating 
performance risk, the agency assigned ratings only to the "area" level 
(i.e., the agency did not assign risk ratings for each subfactor), but 
it is clear from the record that the same considerations were at 
issue.  In essence, the offerors' past performance was reviewed to 
predict how well the firm might perform the central objectives of the 
contract as embodied in the evaluation criteria.

3. ECC also maintains that the contemporaneous record from the 
agency's source selection activities shows that important information 
was not considered.  While it is true that there are several matters 
not explicitly addressed in the contemporaneous record, the record as 
a whole, including sworn statements submitted by the agency's 
cognizant source selection personnel, indicates that the matters were 
discussed, even if no contemporaneous record of that discussion was 
kept; we therefore conclude that the protester's contention in this 
respect is without merit.

4. In a related contention, ECC maintains that the agency's evaluation 
and source selection officials were biased against it because of the 
firm's performance on the predecessor C-17 MTD contract.  As 
discussed, we find that the agency's performance risk assessment of 
ECC was reasonable.  Thus, even if we were to find evidence of bias 
(and we have found none here), it would provide no basis for our 
Office to sustain ECC's protest since, in order to demonstrate bias, a 
protester must also show that the bias  translated into action; where 
the evaluation is otherwise reasonable, there is no basis for making 
this latter finding.  Hagler Bailey Consulting, Inc., B-265708, Dec. 
19, 1995, 95-2 CPD  para.  276 at 3.

5. The assessment of the relevance of performance under fixed-price 
contracts is an area where the contemporaneous record does not reflect 
the agency's assertion that this matter was discussed at the time and 
that it was decided that those contracts were of limited predictive 
value.  As noted above, we have accepted the agency's assertion that 
this matter was discussed, because of the sworn statement to that 
effect submitted in response to the protest, the lack of 
countervailing argument or evidence, and the reasonableness of the 
position itself.  We note as well that ECC has failed to demonstrate 
prejudice in connection with the agency's alleged failure to consider 
fixed-price contract information.  [DELETED].

6. ECC also maintains that the agency improperly did not consider 
AAI's performance on two other contracts where, according to the 
protester, AAI's performance was unsatisfactory.  The record shows 
that, as to one of these contracts, the requirement was developmental 
in nature, that the rating official stated with respect to cost that 
there were many "unknowns" at the beginning of performance and that, 
as a consequence, there was not a firm target cost against which to 
measure performance.  The rating official ultimately assigned AAI an 
"acceptable" rating for "effective cost performance."  The record 
shows that the other contract was only in the first year of contract 
performance and that costs still were within the government's 
evaluated cost estimate used for source selection purposes.  This 
rating official also ultimately rated AAI's effective cost performance 
as "acceptable."