BNUMBER:  B-274204.6; B-274204.7
DATE:  November 27, 1996
TITLE:  Northrop Grumman Corporation; ITT Gilfillan

**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a 
GAO Protective Order.  This version has been redacted or approved by 
the parties involved for public release.
Matter of:Northrop Grumman Corporation; ITT Gilfillan

File:     B-274204; B-274204.2; B-274204.3; B-274204.4; B-274204.5; 
          B-274204.6; B-274204.7

Date:November 27, 1996

Richard P. Rector, Esq., Kevin P. Mullen, Esq., and Chandra Emery, 
Esq., Piper & Marbury, for Northrop Grumman Corporation; and Richard 
L. Moorhouse, Esq., Dorn C. McGrath III, Esq., Steven A. Diaz, Esq., 
Mary F. Withum, Esq., and Stacey E. Young, Esq., Holland & Knight, for 
ITT Gilfillan, the protesters.
Thomas J. Madden, Esq., John J. Pavlick, Jr., Esq., Carla D. Craft, 
Esq., Jerome S. Gabig, Jr., and Paul A. Debolt, Esq., Venable, 
Baetjer, Howard & Civiletti, for Raytheon Company, an intervenor. 
Gregory H. Petkoff, Esq., Marian E. Sullivan, Esq., Roger J. McAvoy, 
Esq., Edward L. Fitzmaurice, Jr., Esq., and Stephen T. Davis, Esq., 
Department of the Air Force, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General 
Counsel, GAO, participated in the preparation of the decision.

DIGEST

1.  Protest that agency improperly determined most likely mean time 
between corrective maintenance action (MTBCMA) for each offeror's 
proposed airport surveillance radar system, rather than accepting the 
proposed MTBCMAs, is denied where solicitation indicated agency's 
intent to conduct its own assessment and, moreover, accepting 
offerors' proposed figures would render this area of the evaluation 
meaningless, since offerors would be expected to employ the most 
favorable assumptions in their calculations, leaving the agency with 
no basis for comparing the proposed MTBCMAs.

2.  Protest that agency improperly failed to disclose during 
discussions the mathematical formula used to calculate mean time 
between corrective maintenance action (MTBCMA) is denied; such 
methodologies for evaluating proposals under the stated evaluation 
criteria need not be disclosed by agencies and, in any case, since 
nondevelopmental system was required and system operational testing 
already had taken place, offerors would not have been in a position to 
redesign their proposed systems so as to significantly improve their 
MTBCMAs and their competitive standing. 

DECISION

Northrop Grumman Corporation (NG) and ITT Gilfillan protest the 
Department of the Air Force's award of a contract to Raytheon Company, 
under request for proposals (RFP) No. F19628-95-R-0007, for digital 
airport surveillance radar (DASR) systems.  NG and ITT challenge the 
agency's conduct of discussions and evaluation of proposals.

We deny the protests.

BACKGROUND

The solicitation contemplated the award of a contract for site survey, 
site design, site preparation, production, installation and logistics 
support for up to 213 DASR radar systems.  The DASR radar system 
includes a primary surveillance radar (PSR) to perform two-dimensional 
detection of aircraft and weather intensity and an integrated 
monopulse secondary surveillance radar (MSSR) to perform 
three-dimensional aircraft position location and identification via 
communication with an aircraft transponder.  The DASR radar system is 
intended to replace Department of Defense (DOD) and Federal Aviation 
Administration (FAA) analog radar systems (at low-to-medium density 
air traffic control facilities) first fielded 25 years ago, which are 
nearing the end of their life cycle and are now subject to sporadic 
loss of airport surveillance radar coverage.  The DASR radar system 
will "improve aircraft detection in clutter, provide National Weather 
Service calibrated six level weather, interface to new digital 
automation systems, improve reliability, and reduce support costs."  
(System Requirements Document (SRD), paragraph 1.0.)  

Of particular importance for this protest, and as noted in the 
solicitation's executive summary, the solicitation sought the proposal 
of "Non-developmental Item (NDI) radars."  Toward this end, it 
provided that proposed radars would undergo "[a]n Operational 
Capability Demonstration (OCD) conducted to verify that the offeror 
has a functional, NDI system," and that "the Offeror's NDI product is 
representative of the DASR system proposed" and "exists as a 'system' 
consisting of all integrated subsystems and other items needed to 
operate and maintain the system per the SRD (except for development 
required by the contract)."  In this regard, the solicitation's 
statement of evaluation criteria specifically cautioned offerors that 
"[v]ariations between the design proposed in written proposals and the 
design/equipment actually demonstrated at the OCD may be considered a 
weakness and may add to the evaluated proposal risk rating."

Award was to be made to the responsible offeror whose offer was most 
advantageous to the government under three broad criteria:  (1) 
technical, which was equal in importance to (2) the offeror's 
capabilities (as shown by the results of a past and present 
performance evaluation, expanded pre-award survey, and possible 
in-plant executive review of the offeror's facilities), both of which 
were more important than (3) cost/price (for a best estimated quantity 
of 89 DASR systems).  The technical area included three factors of 
equal weight:  (1) system performance and system specification, (2) 
reliability, maintainability, availability and supportability, and (3) 
the offeror's proposed statement of work (SOW), contract data 
requirements list (CDRL) and integrated master plan (IMP).  Under both 
the technical factors for system performance/system specification and 
for reliability/maintainability/availability, the solicitation 
generally provided for proposal strengths to be given to the extent an 
offeror's proposed system specification "captures the desired (i.e., 
'should') SRD requirements in a manner beneficial to the Government"; 
"[s]pecial emphasis" was to be given to the extent that the offeror's 
proposed system offered certain specified "desired characteristics the 
Government believes add significant value to the Offeror's product."  
(Among these latter characteristics was a desired mean time between 
corrective maintenance action (MTBCMA) of 1,070 hours.)  Under the 
cost/price area, the solicitation provided for calculation of a total 
cost/price and for evaluation "of the realism of the Offeror's 
proposed cost/price(s)," including an evaluation of "the extent to 
which proposed prices and supporting cost data . . . are consistent, 
indicate a clear understanding of the solicitation requirements, and 
reflect a sound approach to satisfying those requirements."

Four initial technical proposals from three offerors--Raytheon, ITT 
and NG (which submitted two independent proposals, only one of which 
is relevant here)--were received by the closing time on December 19, 
1995.  OCDs for each offeror were conducted from February 6 to 
February 23, 1996.  The Air Force then opened written discussions with 
all offerors on April 5, with written responses required by April 22, 
and conducted oral discussions during May.  Complete proposals, 
including cost/price proposals and revised technical proposals, were 
received by June 7.  Following additional written discussions with 
offerors, the Air Force requested best and final offers (BAFO).

Based upon the evaluation of BAFOs and the offerors' capabilities 
evaluation, the source selection evaluation board (SSEB) and the 
source selection advisory council reported to the source selection 
authority (SSA) that, while the evaluation of offerors' past 
performance and the results of the expanded pre-award survey indicated 
a low performance risk for all three offerors, Raytheon's technical 
proposal was superior to that of the other offerors.  The specific 
results of the evaluation were as follows:

                Raytheon        NG              ITT

TECHNICAL                                       

  System
  Performance/
  Specification Exceptional (Blue)/
                Low Risk        Acceptable/
                                Moderate Risk   Acceptable/
                                                High Risk

  Reliability,
  Maintainability,
  Availability
  and
  SupportabilityExceptional/
                Low Risk        Acceptable/
                                Low Risk        Acceptable/
                                                Moderate Risk

  SOW/CDRL
  /IMP          Acceptable
                (Green)/
                Low Risk        Acceptable/
                                Low Risk        Acceptable/
                                                High Risk

  MTBCMA        1,134 Hours     380 Hours       481 Hours

  OVERALL
  TECHNICAL     Exceptional/
                Low Risk        Acceptable/
                                Moderate Risk   Acceptable/
                                                High Risk

CAPABILITY      Low Risk        Low Risk        Low Risk

COST/PRICE      $[DELETED]         Million$[DELETED]         
                                                Million$[DELETED]         
                                                Million
Based upon the evaluation record, the SSA concluded that Raytheon's 
offer was most advantageous.  The SSA noted that Raytheon's technical 
proposal was evaluated as exceptional/low risk and superior to those 
of the other two offerors under the system performance/system 
specification factor.  In this regard, the SSA specifically noted that 
Raytheon's proposal, which had no significant weaknesses in this area, 
offered several beneficial strengths, including superior MSSR 
operating characteristics, easy upgrade to a Mode S radar 
capability,[1] a Surveillance Data Translator design which allowed for 
easy growth and installation,[2] and a third DASR system to support 
the in-plant test program.  (In addition, Raytheon's low risk rating 
was partly based on the fact that the system demonstrated at OCD was a 
current production model pulled from the production line and destined 
for a [DELETED] customer.)  In contrast, the SSA noted that, although 
ITT's proposed Surveillance Data Translator likewise allowed for easy 
growth and installation and its technical proposal was considered 
acceptable under this factor, the proposal was evaluated as high risk 
under the system performance/system specification factor on the basis 
that ITT had significantly underestimated the manhours required to 
accomplish the proposed software modifications and had failed to 
demonstrate a thorough understanding of the throughput requirements 
with respect to interfacing with the existing automation system.  
Likewise, although NG's proposed system offered several beneficial 
operating characteristics, a weather channel design based upon 
operationally proven algorithms, and a third in-plant DASR test 
system, its proposal was evaluated as moderate risk under this factor 
on the basis that NG had significantly underestimated the manhours 
required to accomplish the proposed software modifications.

The SSA further noted that Raytheon's technical proposal was evaluated 
as exceptional/low risk and superior to those of the other two 
offerors under the reliability/maintainability/availability factor.  
In this regard, the SSA specifically noted that Raytheon's proposal, 
which had no significant weaknesses in this area, offered several 
beneficial strengths, including a system with a MTBCMA of 1,134 hours 
(that is, in excess of the desired 1,070-hour MTBCMA), a [DELETED] 
transmitter and a mature, easy-to-use human interface, and likely 
significant software maintenance cost savings from the use of 
[DELETED] software development tools.  While NG also offered a 
[DELETED] transmitter, its proposed system was evaluated as having an 
MTBCMA of only 380 hours, significantly below the desired level (and 
its proposed MTBCMA of 734 hours).  ITT likewise offered a [DELETED] 
transmitter, but its system was evaluated as having an MTBCMA of only 
481 hours, again, significantly below the desired level of 1,070 hours 
(and its proposed MTBCMA of 1,436 hours).  In addition, ITT's proposal 
was rated as moderate risk under the reliability/maintainability/avail
ability factor on the basis that the system architecture included a 
significant amount of hard-to-support [DELETED] software and was 
characterized in significant measure by a custom design--rather than 
an open architecture design--such that, should the need arise to 
replace the [DELETED] processors (which the agency feared would become 
unavailable) or associated older hardware, significant redesign would 
be required.  Further, while both Raytheon's and NG's proposals were 
evaluated as acceptable/low risk under the SOW/CDRL/IMP factor, the 
SSA noted that ITT's proposal, although considered acceptable, was 
rated high risk on the basis of its unrealistically aggressive test 
schedule, insufficient test labor hours, and limited understanding of 
the site activation process.   

The SSA concluded that the overall technical superiority and lower 
risk of Raytheon's proposal, which was rated exceptional/low risk 
while NG's and ITT's proposals were rated only acceptable/low risk and 
acceptable/high risk, respectively, was worth its somewhat higher 
cost/price.  Upon learning of the resulting August 9 award to 
Raytheon, NG and ITT filed these protests with our Office.

NG and ITT raise numerous arguments concerning the agency's conduct of 
the procurement generally and the evaluation specifically.  We discuss 
a number of their most significant arguments below.

MTBCMA

As used by the Air Force, MTBCMA generally measured the mean time 
between required immediate, unscheduled corrective maintenance visits 
to DASR sites, that is, requirements for maintenance visits other than 
the regularly scheduled preventative maintenance visits.  For purposes 
of calculating MTBCMA, the SRD stated that:

     "Corrective maintenance actions are those required to maintain 
     user confidence in the integrity of the DASR System.  Those would 
     include repair of failed redundant units considered critical to 
     meeting the DASR System mission of providing PSR, weather and 
     MSSR data to the automation system."

Although the agency accepted the reliability block diagrams and 
component failure rates furnished by each offeror in support of their 
estimated MTBCMAs--finding that the failure rates were based on 
appropriate reliability analyses and reflected the best estimates of 
the actual likely rates of component failures--the agency noted that 
each offeror's MTBCMA was based on different assumptions as to what 
failed components in their proposed system architecture would 
compromise user confidence in the integrity of the DASR system and 
when they should be replaced.  Accordingly, when discussions failed to 
eliminate the differences with respect to the assumed government 
maintenance approach, and in order to assure a common basis on which 
to measure and compare the MTBCMAs of the proposed systems, the agency 
"rationalized" each proposed MTBCMA using its intended maintenance 
approach.  Thus, in computing the evaluated MTBCMA of each proposed 
DASR system, the Air Force essentially assumed that failures of units 
in series and in one-of-two redundant configurations--that is, where 
only one of the two units is necessary for performance of the function 
and the second unit is redundant--would be repaired immediately; it 
assumed that repair of a configuration with greater redundancy would 
be delayed until the failure of the last redundant unit, such that 
failures for a subsystem consisting of three units in which only one 
was necessary for operation would not be repaired until two units 
failed.  This rationalization resulted in evaluated MTBCMAs for NG and 
ITT significantly below those estimated by the offerors--that is, an 
evaluated 380 hours for NG (734 hours proposed) and 481 hours for ITT 
(1,436 hours proposed).

As noted above, in making his source selection decision, the SSA 
expressly took into account that NG's and ITT's evaluated MTBCMAs were 
significantly below the desired level and Raytheon's  MTBCMA.  In 
addition, the SSA was briefed on the likely cost of the greater or 
lesser number of unscheduled corrective maintenance visits to DASR 
sites to be expected as a result of MTBCMAs below or above the target 
MTBCMA (which was likely to result in eight unscheduled corrective 
maintenance visits per year per system).  Specifically, the SSA was 
advised that NG's MTBCMA of 380 hours was likely to result in an 
additional 24,276 unscheduled corrective maintenance visits over the 
expected 20-year life of the DASR systems, for an additional total 
labor cost of approximately $20 million, and that ITT's MTBCMA of 481 
hours was likely to result in an additional 16,372 unscheduled visits, 
for an additional labor cost of approximately $13.5 million.  In 
contrast, the fact that Raytheon's evaluated MTBCMA of 1,134 hours was 
likely to result in 755 fewer unscheduled corrective maintenance 
visits relative to those likely at the target MTBCMA of 1,070 hours 
was expected to result in an approximate $700,000 savings relative to 
the target MTBCMA (and thus approximately $20.7 and $14.2 million 
savings relative to NG's and ITT's MTBCMAs, respectively).  (The 
numbers of additional or fewer visits to be expected at the various 
MTBCMAs which were briefed to the SSA were mistakenly based on 82 DASR 
systems and not the correct total of 89 systems.  When corrected to a 
total of 89 DASR systems, the resulting number of additional visits 
would have been 26,461 visits for NG and 17,845 for ITT, while 
Raytheon's evaluated MTBCMA was likely to result in 822 fewer 
visits.)[3]

Notice of Evaluation Approach

NG and ITT challenge several aspects of the Air Force's calculation of 
the MTBCMAs for the proposed DASR systems and its determination of the 
related impact on future maintenance costs.  As an initial matter, NG 
challenges the Air Force's rationalization of the offerors' proposed 
MTBCMAs.  NG argues that the agency failed to adequately advise 
offerors of its approach to evaluating MTBCMA and that the agency's 
action in rationalizing proposed MTBCMAs amounted to the imposition of 
an unstated evaluation criterion; the protester essentially maintains 
that the agency instead was required to accept the proposed MTBCMAs.  

We find that the RFP adequately advised offerors of the agency's 
intended approach in evaluating MTBCMA.  The solicitation provided for 
evaluation of "[t]he Offeror's approach to meeting and substantiating 
the DASR SRD RMA [reliability/maintainability/availability] and 
supportability requirements," with "[p]roposal strength [to] be given 
for the extent to which the Offeror's proposed system specification 
captures the desired (i.e., 'should') SRD requirements in a manner 
beneficial to the Government" and for "[s]pecial emphasis" to be given 
for certain listed desired SRD requirements.  One of the "special" 
requirements included in the SRD as a reliability/maintainability/avai
lability requirement was a desired DASR system MTBCMA "greater than or 
equal to 1070 hours."  The evaluation was consistent with this 
scheme--the agency considered the offerors' ability to meet the 
desired MTBCMA of 1,070 hours.  The solicitation also generally 
provided for the agency to "assess the risk associated with the 
Offeror's proposed approach as it relates to accomplishing the 
requirements of this solicitation" and specifically required that 
"[a]ll [reliability/maintainability/availability] claims . . . be 
substantiated through the use of the Reliability Model for all 
elements of the proposed DASR system."  We think this was a clear 
indication that the agency intended to assess the likelihood that the 
offeror's proposed MTBCMA would be achieved in actual operation. 

NG's position that the agency was required to accept the offerors' 
estimated MTBCMAs--even where such numbers differed from those most 
likely to be achieved in actual operation of the proposed systems--is 
simply untenable.  First, there is no language in the RFP suggesting 
that the agency intended such an approach.  More importantly, such an 
approach would render this area of the evaluation meaningless, since 
offerors would be expected to employ the most favorable assumptions in 
their calculations, leaving the agency with no basis for comparing the 
proposed MTBCMAs.       

Inadequate Discussions

NG and ITT argue that the Air Force should have furnished more 
detailed information concerning its approach to rationalizing 
offerors' proposed MTBCMAs.

The solicitation did not furnish a precise definition of when 
corrective maintenance would be required and how to calculate the 
MTBCMA of the proposed DASR systems.  The definition of MTBCMA in the 
SRD did not specify what maintenance actions were required "to 
maintain user confidence in the integrity of the DASR System" other 
than to indicate that they "would include repair of failed redundant 
units considered critical to meeting the DASR System mission of 
providing PSR, weather and MSSR data to the automation system."  
Further, as noted by the agency, MTBCMA as defined in the SRD "is a 
non-standard parameter in the sense that it is not specifically 
addressed in the usual handbooks and standards that are commonly found 
in nearly every reliability engineer's tool box."  Indeed, the SSEB's 
technical adviser testified at the hearing conducted on this protest 
that he had never seen the term MTBCMA used before.  Hearing 
Transcript (Tr.) at 646, 647.
  
However, the Air Force did advise offerors during discussions, where 
necessary, of the need for immediate corrective maintenance actions 
with respect to failed critical units and of the agency's position 
concerning the number of components to be characterized as critical.  
For example, the agency issued written clarification requests to 
Raytheon and NG advising that their assumption that all failed 
redundant units could be replaced at fixed intervals (rather than 
immediately) conflicted with their proposed system specifications, 
modeled on the SRD MTBCMA provision, which stated that:

     "Corrective maintenance actions are those required to maintain 
     user confidence in the integrity of the [DASR] System.  These 
     [include] repair of failed redundant units considered critical to 
     meeting the [DASR] system mission."

In addition, NG and Raytheon were furnished with two scenarios to 
illustrate the agency's position in this regard.  Scenario 1 concerned 
a multiple redundancy configuration in which the first failure did not 
require immediate corrective maintenance, while Scenario 2 illustrated 
a one-of-two-redundancy configuration in which immediate corrective 
maintenance was required, as set forth below:

     "Scenario 2

     --One of two redundant antenna drive motors fail.

     --Impact:  Only one drive motor operating; integrity of DASR 
     compromised.

     --Immediate corrective maintenance action IS required."

NG's consulting engineer for the DASR proposal (and development 
manager in the air traffic control area) acknowledged in testimony 
that, as a result of this clarification request and subsequent May 
20-22 oral discussions with the agency, NG was aware of "what's 
critical, what's not," and of the need for immediate, not deferred, 
maintenance of critical units.  Tr. at 137.  This understanding of the 
agency's position as to when corrective maintenance action would be 
required was evidenced by NG's subsequent reduction in its proposed 
MTBCMA from the 1,130 hours previously proposed to 734 hours in its 
BAFO.  

NG complains that the agency did not disclose that the mathematical 
formula used to calculate MTBCMA included the failure rates of both 
the on-line and the redundant unit in a one-of-two redundancy 
configuration (and not just the on-line unit), resulting in a 
significant reduction to the MTBCMA for NG's proposed system;  on the 
contrary, the agency allegedly indicated during discussions that only 
the failure rate for one of the units would be counted.  Tr. at 
146-147.  

Contracting agencies are not required to disclose their methodology 
for evaluating proposals under the stated evaluation criteria.  
Lexis-Nexis, B-260023, May 22, 1995, 95-2 CPD  para.  14; see DynCorp et 
al., B-257037.2 et al., Dec. 15, 1994, 95-1 CPD  para.  34.  The Air Force 
thus was not required to disclose to offerors the formula to be used 
in its MTBCMA calculations.  Moreover, it does not appear that 
offerors were in a position after the OCD to redesign their proposed 
systems so as to significantly improve both their MTBCMA and their 
competitive position.  Again, the DASR solicitation sought the 
proposal of NDI radars and required the proposed radars to undergo an 
OCD "to verify that the offeror has a functional, NDI system" that 
"exists as a 'system' consisting of all integrated subsystems and 
other items needed to operate and maintain the system per the SRD 
(except for development required by the contract)."  Although offerors 
were not prohibited from changing their proposed systems after the 
OCD, which occurred before discussions commenced, as indicated above, 
the solicitation specifically cautioned that variations between the 
proposed design and the design/equipment actually demonstrated at the 
OCD could be considered a weakness and add to the evaluated proposal 
risk.[4]  NG argues that it could have retained its proposed design 
and improved its evaluated MTBCMA by substituting more optimistic 
assumed component failure rates.  This argument is unpersuasive--if 
more optimistic rates legitimately could have been used, it is unclear 
why NG did not do so when it realized that its MTBCMA could be no 
higher than 734 hours, that is, well below the desired 1,070 hours.  
Moreover, NG's sudden adoption of significantly more optimistic 
component failure rates--after having submitted what presumably was 
its best, good faith estimate of likely failure rates--reasonably 
would have been considered suspect.

ITT argues that its discussions were inadequate because, unlike NG's 
and Raytheon's clarification requests, ITT's clarification request did 
not include  Scenario 2, which set forth a requirement for immediate 
corrective maintenance in a hypothetical system where one of two 
antenna drive motors fail.  However, discussions with different 
offerors need not be identical; on the contrary, discussions should be 
tailored for different offerors, since the need for revision 
ordinarily will vary with each proposal.  Delta Data Sys. Corp., 
B-260791, July 21, 1995, 95-2 CPD 42; The Pragma Corp., B-255236 et 
al., Feb. 18, 1994, 94-1 CPD 124.  The Air Force explains that it was 
not necessary to furnish Scenario 2 to ITT because its proposal 
indicated that, unlike NG and Raytheon, it understood the requirement 
for immediate corrective maintenance when critical components fail; 
the only question about ITT's proposal in this area concerned the 
firm's understanding as to which components were considered critical, 
and the discussions with ITT addressed this concern.[5]  The 
discussions therefore were adequate.

Reliability of Redundant Configurations

ITT argues that the Air Force improperly used a model for determining 
the reliability of configurations with multiple redundancies (that is, 
with more than one redundant or backup unit) which had the effect of 
understating ITT's MTBCMA; according to the protester, had the agency 
used the correct model, ITT's evaluated MTBCMA would have increased 
from 481 hours to 651 hours (still 785 fewer hours than ITT proposed).  

The Air Force (and Raytheon) maintains that ITT's calculation of the 
effect of applying a different model, although purportedly based on 
the agency's expected maintenance approach, nevertheless departs from 
that maintenance approach by failing to account for the impact of 
requiring immediate corrective maintenance when the last redundant 
element in a multiple redundant configuration fails and that ITT's 
calculation instead assumes deferring maintenance until the next 
scheduled preventative maintenance visit.  

Our review of the record confirms the Air Force's position that ITT's 
calculations fail to take certain aspects of the required maintenance 
approach into consideration.  For example, although the configuration 
of ITT's [DELETED] system is shown to have a two-of-four 
redundancy--two of the four units are redundant--in the worksheets 
supporting ITT's claimed 651 hour MTBCMA, the worksheets do not appear 
to include any allowance for the required immediate repair when the 
second unit fails.  Taking these discrepancies into account, the 
record suggests that, at best, use of ITT's suggested reliability 
model would have increased its evaluated MTBCMA to no more than 497 
hours (as calculated by Raytheon).  (Furthermore, it is not clear why 
use of ITT's suggested model would not also have resulted in some 
offsetting increase in the MTBCMAs for the other offerors, thereby 
eliminating or reducing any improvement in ITT's relative competitive 
position in this regard.)  Thus there is no basis to conclude that the 
reliability model used by the agency resulted in an understated MTBCMA 
for ITT.     

Raytheon Reliability Data

NG challenges the Air Force's acceptance of Raytheon's estimated 
component failure rates, which were based on historical factory and 
field test data for the radars proposed.  According to the protester, 
the assumed component failure rates instead should have been based on 
the predictions in Military Handbook No. 217 (MIL-HDBK-217), 
"Reliability Prediction of Electronic Equipment," as were NG's 
proposed component failure rates.  DOD's instructions for use of the 
handbook, however, state that "[i]t should be emphasized that if 
similar equipment field historical data are available in sufficient 
quantity and quality it should always be used in lieu of a prediction.  
If the operating or environmental conditions are different, then 
modifications can be performed using MIL-HDBK-217 adjustments."  
MIL-HDBK-217 Use and Application (Technical Brief, April 1990).  These 
instructions are consistent with testimony by the SSEB's technical 
adviser, who possesses extensive experience in reliability prediction, 
that data obtained from the operation of similar equipment in a 
similar environment is the most reliable predictor of the reliability 
of a radar system and is "a much more reliable predictor and gives a 
higher confidence in the prediction" than handbook reliability 
predictions.  Tr. at 610-612.

Raytheon's estimated component failure rates were based on [DELETED] 
hours of experience--more than [DELETED] years of continuous 
operation--with its proposed MSSR, and [DELETED] hours--nearly 
[DELETED] years of continuous operation--with its proposed PSR.  
Testimony by government engineers confirms that there was sufficient 
similarity between the factory and field environments in which 
Raytheon's radars were tested to justify combining the total test 
data; there was a sufficient quantity of test data to form the basis 
for a reasonable reliability prediction; and there was sufficient 
similarity between the radars tested and those proposed by Raytheon to 
render the historical data a reliable predictor of  reliability.  Tr. 
at 612-618, 648-649, 716-734.  (If anything, the Raytheon historical 
data may understate the likely reliability of its radars; normally, 
the reliability of electronic systems improves over time from when it 
is first introduced until mid-life.  Tr. at 649-650.)  Thus, we find 
no basis to question the agency's acceptance of Raytheon's estimated 
component failure rates.

Additional Corrective Maintenance Visits

In calculating the average labor cost of an unscheduled corrective 
maintenance visit to a DASR site, the Air Force made a number of 
assumptions, some of which appear to have overstated to some extent 
the likely cost of a visit.[6]  For example, one of the most 
significant determinants of the cost of a visit is the number of 
technician man-hours required.  It is undisputed that when working on 
radar systems accepted safety practice requires the presence of at 
least two workers.  A working group comprised of persons familiar with 
radar system maintenance from the FAA and DOD considered in detail 
each activity to be performed in connection with a corrective 
maintenance visit, including closing off the current activity of the 
technicians after notification of a fault, identifying the faulty unit 
(usually to within one of three possible units), obtaining replacement 
units from logistics control, travelling to the DASR site from the 
maintenance center, accessing the site, coordinating the repair with 
operations/maintenance control, repairing the fault, securing the DASR 
site, returning to the maintenance center, returning units to 
logistics control, and closing out the maintenance action.  The 
working group concluded that an average corrective maintenance visit 
by two technicians to a DASR site will take 6 hours from notification 
of fault to close out of the maintenance action, for a total of 12 
technician man-hours.

NG challenges the Air Force's estimate of the cost of the additional 
unscheduled corrective maintenance visits to DASR sites--$20 million 
for NG and $13.5 million for ITT--expected as a result of the lower 
evaluated MTBCMAs.  NG argues that the required work actually can be 
performed in less than 6 hours and with fewer than 12 technician 
man-hours.[7]  

Our review of the record, including testimony by an experienced 
military maintenance technician, suggests that the agency's estimate 
of 12 technician
man-hours may in fact be somewhat overstated.  This testimony 
indicated that (1) after allowing for the fact that not all of the 
tasks to be undertaken before departing for the DASR site and after 
returning to the maintenance center require two technicians, and (2) 
given that the potential for task sharing and concurrent task 
performance exists, it may be possible to accomplish the required work 
with no more than approximately 7-1/2 technician man-hours (and 
perhaps with only a little more than 7 hours).  Tr. at 702-710.  
However, the military maintenance technician also testified that when 
two technicians visit a radar site, which as indicated above is a 
safety requirement when working on radar systems, "[t]ypically one 
person does it all and the other person watches."  Tr. at 711.  The 
record furnishes no basis for questioning the technician's assessment 
in this regard, which was based on approximately 16 years of 
experience maintaining air traffic control radar systems.  Thus, the 
historical record suggests that the potential for task sharing and 
concurrent task performance may not in fact be realized in practice.  
While we cannot precisely quantify the hours required under this 
latter scenario, it appears from our review of the record, including 
the numbers suggested by the agency, NG and Raytheon, that after 
allowing for the fact that some of the tasks to be undertaken before 
departing for the DASR site and after returning to the maintenance 
center only require one technician, an average corrective maintenance 
visit may require as much as approximately 9-3/8 technician hours.

In addition, although the agency assumed that the technicians 
repairing the DASR system will be higher cost civilian employees 
rather than military personnel, the agency reports that the FAA will 
man no more than 67 of the 89 sites--its own as well as some of the 
DOD sites--and it appears reasonable to assume that the remaining 22 
sites will be serviced by lower cost military technicians.  Further, 
it appears unreasonable for the agency to have calculated a work-year 
of 1,800 hours by reducing the available time of 2,080 hours--40 hours 
times 52 weeks--by expected holidays and vacations, and then 
allocating the fully burdened cost of a technician, including a factor 
for leave, over 1,800 hours.  In so doing, the agency appears to have 
double-counted the effect of leave in calculating a burdened hourly 
rate for the technicians.

After allowing for military manning of some sites and correcting the 
productive man-year to 1,800 hours, and assuming a continuation of 
historic working patterns among air traffic control radar maintenance 
technicians such that the average corrective maintenance visit 
requires approximately 9-3/8 technician hours, it appears that the 
additional labor cost to the government of NG's and ITT's lower 
MTBCMAs relative to the target MTBCMA (and after giving ITT credit for 
an MTBCMA of 497 hours) could total as much as approximately $12.7 
million for NG and $8.1 million for ITT.[8]  Furthermore, whatever the 
precise labor cost of the additional maintenance visits, the very fact 
that NG's and ITT's lower evaluated MTBCMAs can reasonably be expected 
to result in a requirement for thousands more unscheduled corrective 
maintenance visits to DASR sites--26,461 additional visits for NG's 
system and 17,845 additional visits for ITT's--than desired by the 
agency or required by Raytheon's system could reasonably be evaluated 
as a major competitive disadvantage.[9] 

IN PRODUCTION STATUS

ITT and NG argue that the Air Force improperly failed to recognize 
that their proposed DASR systems, like Raytheon's, were "in 
production."  As noted by the agency, however, ITT's proposed DASR 
system was a combination of modified production subsystems and 
engineering model subsystems--e.g., its [DELETED]--integrated as a 
complete system for the first time at OCD, while NG's proposed DASR 
system, although it included many components currently in production, 
required a newly developed [DELETED] and was comprised of subsystems 
that must be integrated for the first time to meet the DASR 
requirements.  In contrast, Raytheon demonstrated at OCD a fully 
integrated production system taken directly off the production line 
(and scheduled for delivery to [DELETED]), which required only minor 
modification to meet the DASR system requirements.  The agency thus 
reasonably assigned Raytheon's system a relative advantage with 
respect to production status (and also reasonably concluded that the 
mature character of the system would result in lower risk). 

MODE-S UPGRADE/SURVEILLANCE DATA TRANSLATOR GROWTH CAPABILITY

NG and ITT challenge the Air Force's determination that the ease with 
which Raytheon's proposed DASR system could be upgraded to Mode-S 
capability was a discriminator offering a significant benefit to the 
government.  As noted by the agency, ITT's approach to upgrading to 
Mode-S was to [DELETED], while NG's approach required [DELETED].  In 
contrast, Raytheon's Mode-S upgrade approach essentially consisted of 
[DELETED].  We find nothing unreasonable in the agency's determination 
that the greater ease with which Raytheon's proposed DASR system could 
be upgraded to Mode-S capability represented a relative advantage.  
Although NG suggests that its Mode-S approach offers other unique, 
offsetting advantages based on a more advanced Mode-S configuration, 
its position essentially amounts to mere disagreement with the agency 
as to the relative desirability of various features and in no way 
demonstrates that the agency's focus on the greater simplicity of 
Raytheon's approach was unreasonable.    

NG also challenges the Air Force's assignment of a strength to 
Raytheon's proposal on the basis of the ability of its Surveillance 
Data Translator to support more [DELETED] display indicators, an 
ability the agency viewed as desirable since it will eliminate the 
necessity for the purchase of additional translators at some sites.  
NG contends that its proposal should have received similar credit 
because its proposed translator likewise can support more [DELETED] 
display indicators.  However, the contracting officer reports, and the 
agency's Proposal Analysis Report confirms, that Raytheon's strength 
in this area also was based on the greater ease with which its 
translator can support additional display indicators.  Specifically, 
the agency states that Raytheon's translator [DELETED].  There is no 
basis to object to the agency's determination that Raytheon's proposal 
was superior on this basis.

[DELETED] 

NG and ITT challenge the Air Force's determination that Raytheon's use 
of [DELETED] was a discriminator offering a significant benefit to the 
government (because the use of [DELETED] increases efficiency during 
software development and produces more maintainable code).  The 
protesters argue that they likewise deserved credit in this regard 
since they also proposed to use [DELETED] during software development.  
As noted by the agency, however, while [DELETED].  The agency 
reasonably rated Raytheon's proposal superior based on Raytheon's 
greater experience with [DELETED] and the greater proportion of its 
existing code that was developed using [DELETED].

ITT SYSTEM ARCHITECTURE

ITT challenges the Air Force's determination that its system 
architecture warranted a rating of moderate risk under the 
reliability/maintainability/availability factor.  The rating is 
unobjectionable.  ITT's designs for its [DELETED] and its [DELETED] 
use a custom architecture--as opposed to an open, modular architecture 
which facilitates substitution of functionally similar parts that 
is--designed around [DELETED] processors and uses a total of [DELETED] 
lines of [DELETED] software code.  (In contrast, ITT's remaining 
processors, and all of Raytheon's and NG's processors, are of an open 
architecture design.)  [DELETED] generally is relatively more 
difficult to maintain and support, since it typically is more 
difficult to understand and requires as much as three times more lines 
of code than higher order languages to develop the same functionality, 
with more lines of code resulting in higher maintenance costs.  In 
addition, the agency determined that should the need arise to replace 
the processors (which at least one prior manufacturer had ceased to 
produce) or associated older hardware (such as [DELETED]), or to 
otherwise upgrade the system during its expected 20-year life using 
new processors, ITT's custom design would require significant, costly 
and time-consuming hardware and software redesign.  Tr. at 505-514, 
553-563.[10]
 
ITT asserts that, had the agency raised this matter during 
discussions, it could have advised it of the existence of several 
manufacturers still producing [DELETED] processors (or their 
equivalent), at least one of which reportedly possesses a significant 
inventory of the processors.  However, the current availability of the 
processors is not a substitute--and does not establish as unreasonable 
the agency's preference--for an open system architecture that 
facilitates any processor upgrades that might be required during the 
20-year life of the system.[11]  (In practice, the service life of 
radar systems of this type usually extends to 25 to 30 years.  Tr. at 
506.)  There thus is no basis to question ITT's moderate risk rating.

COST/TECHNICAL TRADEOFF

In a negotiated procurement, the government is not required to make 
award to the lowest-cost, technically acceptable offeror unless the 
RFP specifies that cost will be determinative.  General Servs. Eng'g, 
Inc., B-245458, Jan. 9, 1992, 92-1 CPD  para.  44.  Cost/technical tradeoffs 
may be made in selecting an awardee, subject only to the tests of 
rationality and consistency with the established evaluation factors.  
Varian Assocs., Inc., B-238452.4, Dec. 11, 1990, 90-2 CPD  para.  478.

Raytheon's technical proposal was evaluated as exceptional/low risk 
and superior to those of the other two offerors.  First, Raytheon's 
technical proposal was evaluated as exceptional/low risk and superior 
to NG's and ITT's under the system performance and system 
specification factor.  Although NG and ITT challenge their moderate 
(NG) and high (ITT) risk ratings under this factor, we note that their 
proposals were considered as only acceptable, while Raytheon's was 
evaluated as exceptional under this factor on the basis of a number of 
significant, beneficial strengths, including superior Mode-S upgrade 
and Surveillance Data Translator growth capabilities.  In addition, 
the agency viewed the fact that the Raytheon  system demonstrated at 
OCD was a current production model to be a significant advantage 
indicative of the lower risk associated with its proposal.  These 
advantages would remain notwithstanding the elimination of NG's and 
ITT's higher risk ratings.  Second, Raytheon's technical proposal was 
evaluated as exceptional/low risk and superior to those of the other 
two offerors under the reliability/maintainability/availability 
factor.  Third, while both Raytheon's and NG's proposals were 
evaluated as acceptable/low risk under the SOW/CDRL/IMP factor, ITT's 
proposal was rated high risk.  Furthermore, Raytheon's proposed DASR 
system was evaluated as having a MTBCMA of 1,134 hours, significantly 
higher than NG's and ITT's, leading the agency reasonably to conclude 
that award to Raytheon would result in fewer unscheduled corrective 
maintenance visits and significant labor cost savings.  Our review of 
the arguments raised by NG and ITT furnishes no basis to question the 
agency's determination of the technical superiority of Raytheon's 
proposal.  Since cost/price was less important than the technical 
criterion and, in any case, the evaluated costs of Raytheon's proposal 
were not substantially higher than NG's and ITT's, we conclude that 
the Air Force reasonably found Raytheon's proposal to be most 
advantageous.

The protests are denied.

Comptroller General
of the United States

1. The Mode S radar is designed to (1) identify, locate, and track 
aircraft by using radar signals to obtain information from up to 700 
individual aircraft at a time and (2) provide users with a 
communications channel between aircraft and ground facilities.

2. The surveillance data translator converts the digital target and 
weather information from the DASR system and reformats the information 
into a form acceptable for use in current air traffic control 
automation computers and analog displays.

3. The agency did not calculate the parts costs associated with the 
various MTBCMAs.

4. To the extent that NG believes that the agency was required to 
furnish a more detailed statement of its rationalization approach in 
the solicitation, such argument had to be raised prior to the closing 
time for receipt of initial proposals in order to be timely.  EG&G 
Management Sys., Inc., B-270509.2, Apr. 1, 1996, 96-1 CPD  para.  195.

5. Specifically, the Air Force issued ITT a clarification request 
stating that: 

            "You have identified only the [DELETED]and the [DELETED] 
            as critical elements which need immediate fixing.  This 
            seems to be in conflict with the SRD definition stating 
            the MTBCMA critical elements include more than just those 
            two.  Please clarify and update availability calculations 
            as required."

6. Although the SSA was briefed on the likely labor cost of the 
additional or fewer unscheduled corrective maintenance visits to DASR 
sites expected as a result of the evaluated MTBCMAs and was presented 
with a briefing chart which showed the labor cost of the additional 
visits added to the cost of the NG and ITT systems, the record read as 
a whole (including the source selection decision) indicates that the 
additional labor cost essentially was viewed by the SSA as a 
quantification of the impact of the MTBCMAs for purposes of the 
technical evaluation.  According to the SSA, "the quantification of 
future potential cost savings was not used by me as a 
dollar-for-dollar offset against the evaluated cost/prices in making 
my source selection decision."  See also Tr. at 83-84, 502-503.

7. In addition, NG argues that the elapsed time of an average visit 
actually must be less than 3 hours if the operational availability 
requirements for the DASR system are to be met.  However, in view of 
the agency's detailed review of each activity to be performed in 
connection with a corrective maintenance visit, undertaken by a 
working group comprised of persons familiar with radar system 
maintenance, we do not find such a general allegation to be 
persuasive.

8. The above numbers for the additional labor costs resulting from 
NG's and ITT's lower MTBCMAs--$12.7 million for NG and $8.1 million 
for ITT--and the lower labor costs resulting from Raytheon's higher 
MTBCMA ($400,000) were derived from calculations performed by the 
parties in response to various scenarios suggested by our Office based 
on the record here. 

9. NG also argues that the Air Force improperly took into account an 
inflation factor (2 percent per annum) in calculating future labor 
costs.  The agency defends its use of an inflation factor and, in 
fact, asserts, with some justification, that an inflation factor of 3 
percent per annum would have been more accurate.  Even if NG were 
correct in its position, the consequent approximate 19-percent 
decrease in estimated labor cost it calculates would not affect the 
reasonableness of the award decision. 

10. Although agency evaluators also noted that there were [DELETED] 
Raytheon items for which there was only a single source, the agency 
did not view this as a matter of significant concern since the open 
architecture of Raytheon's system facilitated the substitution of 
functionally equivalent items if that should prove necessary.

11. Further, since redesigning this aspect of its system to move from 
a custom to an open architecture and rewriting the extensive body of 
[DELETED] code in a higher order software language would have required 
a significant effort and redesign, especially in the context of an NDI 
procurement, Tr. at 563, the agency was not required to raise the 
matter during discussions; agencies are not required to point out 
weaknesses derived from an inherent aspect of an offeror's approach 
which would require substantial revision to resolve the agency's 
concern.  Northrop Worldwide Aircraft Servs., Inc., B-262181, Oct. 27, 
1995, 95-2 CPD  para.  196; Tracor Flight Sys., Inc., B-245132, Dec. 17, 
1991, 91-2 CPD  para.  549.