BNUMBER:  B-272418; B-272418.2; 272418.3
DATE:  October 30, 1996
TITLE:  Hughes Missile Systems Company

**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a 
GAO Protective Order.  This version has been redacted or approved by 
the parties involved for public release.
Matter of:Hughes Missile Systems Company

File:     B-272418; B-272418.2; 272418.3

Date:October 30, 1996

Mark D. Colley, Esq., David P. Metzger, Esq., Laura E. Gasser, Esq., 
Craig A. Holman, Esq., Laura A. Eilers, Esq., Melinda C. Burrows, 
Esq., and Bradley D. Wine, Esq., Holland & Knight, and David L. 
Fowler, Esq., for the protester.
William A. Roberts III, Esq., Lee P. Curtis, Esq., Jerone C. Cecelic, 
Esq., J. Eric Andre, Esq., and Terry Petrie, Esq., Howrey & Simon, and 
Bucky P. Mansuy, Esq., for Lockheed Martin Integrated Systems, Inc., 
David V. Anthony, Esq., Gregory A. Smith, Esq., Mary Ita Snyder, Esq., 
Stanley R. Soya, Esq., and John E. Benedict, Esq., Piper & Marbury, 
and Stephen E. Seele, Esq., for McDonnell Douglas Aerospace 
Corporation, intervenors.
Gregory Petkoff, Esq., John A. Dodds, Esq., Marian E. Sullivan, Esq., 
Capt. Rebecca Pearson, and Gary J. Rosnick, Esq., Department of the 
Air Force, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General 
Counsel, GAO, participated in the preparation of the decision.

DIGEST

1.  Protest against evaluation of past performance is denied where 
information reasonably available to the procuring agency supported the 
report of the agency administering the protester's contract to the 
effect that protester was deficient in its cost/schedule performance 
on that contract.

2.  Protest that agency did not adequately consider the complexity of 
the products being developed and/or procured when evaluating past 
cost/schedule performance and arriving at an overall rating, and 
instead placed undue emphasis on program complexity, is denied where 
offerors were advised prior to the closing date for receipt of 
proposals that agency viewed program similarity as more important than 
product similarity.

DECISION

Hughes Missile Systems Company protests the Department of the Air 
Force's award of contracts to Lockheed Martin Integrated Systems, Inc. 
(LM) and McDonnell Douglas Aerospace Corporation (MD), under request 
for proposals (RFP) No. F08626-96-R-0002, for the definition and 
development of the Joint Air-to-Surface Standoff Missile (JASSM).  
Hughes challenges the past performance, cost and technical 
evaluations.

We deny the protest.

BACKGROUND

The JASSM missile will be an autonomous, precision strike standoff 
weapon, to be carried aboard a number of types of aircraft, which can 
be launched from beyond the range of enemy air defenses and will 
provide the capability to strike heavily defended, high value targets.  
Offerors were advised that the agency considered the need for such a 
standoff attack capability to be urgent and compelling.  The 
solicitation contemplated the award of two cost-plus-fixed-fee 
contracts for a 24-month Program Definition and Risk Reduction (PDRR) 
phase, which would include priced options for a follow-on 
cost-plus-incentive-fee Engineering, Management and Development (EMD) 
phase, to be exercised on the basis of a downselect competition 
between the two PDRR contractors.  The downselect contractor will then 
commence production, which is expected to include 10 annual production 
lots.  Offerors were required to furnish cost and pricing information 
for the PDRR and EMD phases and cost estimates for the production 
phase; during the downselect competition, the PDRR contractors will 
have the opportunity to update their EMD phase pricing and to submit 
cost and pricing information for the production quantities.

Award was to be made to the offerors whose proposals were most 
advantageous to the government under three broad criteria:  (1) past 
performance--including past technical performance (with factors for 
product performance, computer software, and aircraft integration) and 
past affordability performance (with factors for manufacturing and 
cost/schedule), which were of equal weight--which was equal in 
importance to the aggregate of (2) technical performance (key 
performance parameters, other requirements, and integrated master plan 
and schedule) and (3) affordability, which were of equal weight.  The 
affordability criterion included four factors:  (1) average unit 
procurement price (AUPP) for production lots 1-5 (the most important 
factor); (2) mission/cost effectiveness--including the expected number 
of missiles to kill the specified JASSM target set, total cost (number 
of missiles times most probable AUPP) to kill the JASSM target set, 
average cost to kill a JASSM target, and number of aircraft sorties 
(missions)--which was equal in importance to (3) total contract price 
for the PDRR and EMD phases; and (4) AUPP for production lots 6-10 
(the least important factor).

Five proposals--including Hughes's, LM's, and MD's--were received by 
the closing time on April 29, 1996.  All were included in the 
competitive range.  Following written and oral discussions with 
offerors, the Air Force requested PDRR and EMD "price" proposals.  

Based upon the evaluation of offerors' past performance and their 
written and oral submissions, the source selection advisory council 
emphasized the following considerations in its report to the source 
selection authority (SSA):  (1) while LM had the strongest past 
performance and MD's performance was "consistently good," Hughes's 
performance was characterized by a weakness with respect to ability to 
perform to cost/schedule; (2) all offerors' proposed missiles exceeded 
the minimum key performance parameters; and (3) while Hughes's offer 
had the best mission/cost effectiveness rating, with the lowest AUPP 
and average cost per target, LM's and MD's proposed and evaluated AUPP 
were below the agency's objective unit price of $[DELETED] and their 
offers had lower development (PDRR and EMD) costs.  The specific 
results of the evaluation were as follows:

                Hughes          LM              MD

Past Performance                                

   Product      Exceptional (Blue)Acceptable    Acceptable

   Software     Acceptable (Green)Acceptable    Acceptable

   Aircraft              IntegrationAcceptableExceptionalExceptional

   ManufacturingAcceptable      Exceptional     Acceptable

   Cost/ScheduleMarginal (Yellow)Acceptable     Acceptable

Technical                                       

   Key                   PerformanceExceptional
                /Moderate Risk  Exceptional
                                /Moderate Risk  Exceptional
                                                /Moderate Risk

   Other                 RequirementsExceptional
                /Moderate Risk  Exceptional
                                /Moderate Risk  Exceptional
                                                /Moderate Risk

   Plan/ScheduleAcceptable
                /Moderate Risk  Exceptional
                                /Moderate Risk  Exceptional
                                                /Moderate Risk

Affordability                                   

   Most Probable
   AUPP 
   (Lots 1-5)   $[DELETED]      $[DELETED]      $[DELETED]Mission/Cost       
Effectiveness

      Total Missiles
      Sorties
      $ per TargetThreshold/
                             Objective
                
                  [DELETED]
                  [DELETED]
                $[DELETED]/
                $[DELETED]       Threshold/
                                             Objective
                                
                                  [DELETED]
                                  [DELETED]
                                $[DELETED]/
                                $[DELETED]      Threshold/
                                                            Objective
                                                
                                                  [DELETED]
                                                  [DELETED]
                                                $[DELETED]/
                                                $[DELETED]

   Total Contract
   (PDRR and         EMD)$[DELETED]$[DELETED]   $[DELETED]

   Most Probable
   AUPP 
   (Lots 6-10)  $[DELETED]      $[DELETED]      $[DELETED]  

Based upon the evaluation record, the SSA concluded that LM's and MD's 
offers were most advantageous.  Specifically, the SSA noted that only 
LM and MD were rated as consistently good-to-excellent performers and, 
in particular, only those two were rated acceptable with respect to 
past cost/schedule performance.  The SSA considered their good 
performance in the cost/schedule area to be "a very key discriminator 
because the schedule on this program is of paramount importance to the 
user due to the critical operational shortfall this system fills."  In 
contrast, the SSA noted that while Hughes's past performance was 
generally good, it was characterized by a significant weakness with 
respect to cost/schedule performance as a result of schedule problems 
experienced on the Advanced Medium Range Air-to-Air Missile (AMRAAM) 
program and the Tomahawk (Cruise Missile) Baseline Improvement Program 
(TBIP), and significant cost overruns experienced on the TBIP program.  
The SSA found the TBIP cost and schedule problems to be "particularly 
troubling as this program is of similar scope to JASSM."  In addition, 
the SSA noted that only LM's and MD's proposals were rated as superior 
(exceptional) with moderate risk under all three technical performance 
factors.  The SSA specifically concluded that MD's proposed design was 
"the lowest risk approach due to their use of a proven off-the-shelf 
[DELETED] and existing warheads," while LM's design was "of somewhat 
higher risk, principally due to the  [DELETED]."  In contrast, the SSA 
considered Hughes's design to have "the highest technical risk of 
these three contractors, driven" by their proposal of a [DELETED] 
target seeker and new ([DELETED]) warhead.  

In the affordability area, the SSA viewed the fact that LM's and MD's 
proposals had the lowest total contract prices for the PDRR and EMD 
phases to be more significant than Hughes's "roughly [DELETED] 
percent" advantage with respect to estimated AUPP for production lots 
1-5, noting that he "did not consider this to be a significant 
difference at the beginning of the PDRR phase" and that LM's higher 
AUPP was based on a [DELETED].  As for Hughes's advantage with respect 
to average cost per target killed, the SSA noted that this was 
primarily due to their proposal to develop a new, more powerful 
warhead, which made unnecessary for many targets the greater accuracy 
afforded by a target seeker and thereby enabled Hughes to propose a 
higher proportion of less expensive non-seeker missiles; he considered 
this approach to be characterized by higher risk than MD's offer of an 
off-the-shelf, proven warhead.  The SSA concluded that LM's and MD's 
superior past performance, superior technical proposals and lowest 
total contract price overshadowed Hughes's evaluated advantage with 
respect to AUPP and average cost per target.  

Upon learning of the resulting June 17 awards to LM and MD, Hughes 
filed this protest with our Office, challenging numerous aspects of 
the evaluation.  We discuss its primary arguments below. 

PAST PERFORMANCE

Background

Hughes challenges the evaluation of past performance.  As noted above, 
past performance was the single most important evaluation area, with a 
weight of approximately 50 percent.  Past performance was evaluated 
based on a maximum of three contracts/programs for each of the five 
past performance factors--product performance, computer software, 
aircraft integration, manufacturing and cost/schedule.  The RFP 
generally stated that "[t]he Government will evaluate past performance 
using the most recent and relevant performance data" and that 
"[w]henever possible, the Government will gather past performance 
[data] on contracts/programs which are similar to the JASSM program in 
product types, program phase, complexity, scope, competitive 
environment, and contract type."  In addition, for each of the five 
past performance factors, the solicitation specifically described the 
prior contracts/programs that would be considered similar.  For 
example, with respect to cost/schedule performance, the evaluation of 
which is at issue here, the solicitation described a similar 
contract/program as one involving "a complex missile program that used 
streamlined acquisition initiatives, such as limited government 
oversight or small industry-Government teams."  Potential offerors 
were notified (prior to the issuance of the final RFP) of the 
contracts/programs selected for evaluation.

The solicitation stated that the agency would evaluate past 
performance under the selected contracts using information from such 
sources as Contractor Performance Assessment Reports (CPAR), the 
Defense Contract Management Command (DCMC), program offices, other 
service organizations and the offerors.  It required each offeror to 
identify in its offer "data sources and points of contact used to 
substantiate his strengths, what he has done to overcome any previous 
problems encountered . . . and any mitigating factors that precluded 
the offeror from meeting his contract cost, schedule or performance 
requirements."  Although the solicitation limited the past performance 
proposal to 20 pages, potential offerors were advised prior to 
issuance of the solicitation that they could furnish data to DCMC 
supporting their past performance proposals.  (Although advised by the 
Air Force on three occasions and by DCMC on a fourth that it could 
furnish additional supporting data, Hughes, unlike the other four 
offerors, declined the opportunity afforded it to submit additional 
data prior to the closing date.)  In addition, where the agency 
questioned an offeror's past performance in a particular area on a 
particular contract, it raised the matter during discussions and 
afforded the offeror an opportunity to respond in writing and orally.  
(While the agency's points for negotiation (PFN) included limits on 
the length of responses, Hughes was afforded an additional 10 pages 
for responding to the PFNs concerning its past performance.)  
TBIP

As noted above, in evaluating Hughes's past cost/schedule performance, 
the agency received reports that Hughes experienced schedule problems 
on the AMRAAM air-to-air[1] and TBIP cruise missile programs and 
significant cost overruns on the TBIP program.  These reports led the 
agency to evaluate Hughes's overall cost/schedule performance as 
marginal (yellow). 

Hughes challenges this evaluation to the extent it is based on 
Hughes's TBIP performance, primarily asserting that the agency 
undertook only a superficial investigation in this regard and, 
furthermore, unreasonably concluded that cost growth on the TBIP 
program would inevitability result in schedule delays, when, according 
to Hughes, this has not proven to be the case.

We will review an evaluation of an offeror's performance risk to 
ensure that it was reasonable and consistent with the stated 
evaluation criteria.  See Dragon Servs., Inc., B-255354, Feb. 25, 
1994, 94-1 CPD  para.  151.  An evaluation of past performance may be based 
on the agency's reasonable perception of inadequate prior performance, 
even where the contractor disputes the agency's interpretation of the 
facts.  See Cessna Aircraft Co., B-261953.5, Feb. 5, 1996, 96-1 CPD  para.  
132; Rockwell Int'l Corp., B-261953.2; B-261953.6, Nov. 22, 1995, 96-1 
CPD  para.  34; Pannesma Co. Ltd., B-251688, Apr. 19, 1993, 93-1 CPD  para.  333.  

The Air Force reasonably determined that Hughes's cost/schedule 
performance on the TBIP program had been materially deficient.  The 
record shows that the Air Force relied on information received from 
the Navy TBIP program office (including documentation generated by 
Hughes) and from Hughes, which reasonably indicated that Hughes had in 
fact experienced cost and schedule problems on the TBIP program.

Specifically, we note that in its April 29 response to the Air Force's 
request for an evaluation of Hughes's performance, the Navy TBIP 
program office rated both Hughes's cost and schedule performance as a 
"2" on a scale of 1-4, with "4" indicating an ability to perform with 
little or no government oversight and "1" indicating an inability to 
perform; the TBIP program office reported that Hughes had demonstrated 
the ability to execute program plans, including implementation of any 
changes or recovery initiatives, within cost and schedule only "with 
substantial Government oversight."[2]  The agency also obtained 
specific information and documents from the TBIP program office which 
supported the Navy's determination that Hughes had encountered 
significant cost/schedule problems in its performance.  For example, 
although awarded the TBIP cost contract in the amount of $226 million 
in September 1994, Hughes encountered sufficient cost growth in the 
first year of the contract that in July 1995 it requested Navy 
approval for an Over Target Baseline (OTB) of $294.8 million 
(ultimately $298.7 million).  Indeed, the TBIP program office 
furnished Air Force evaluators information indicating that in February 
1996 Hughes had identified the "'potential' for additional $42.2M 
[million] cost growth," and that based on Hughes's April 1996 briefing 
it appeared that the OTB was "becoming obsolete."  Hearing transcript 
(Tr.) at 1218-1219.  (In a July 1996 "TBIP Baseline Status Review," 
Hughes indicated an estimate at completion (EAC) of $381 million for 
the TBIP program.)  In this regard, the TBIP program manager testified 
at the hearing held on this protest that Navy source selection 
officials had concluded during the TBIP source selection that Hughes 
had significantly underestimated the cost of performance and that, as 
a result, the Navy had added an additional $70 million to the TBIP 
budget beyond the $226 million proposed by Hughes.  Tr. at 1019, 1094.  

Further, the information available to the Air Force at the time it was 
evaluating Hughes's TBIP performance (April/May 1996) reasonably 
indicated that Hughes was encountering performance delays that could 
ultimately delay delivery of the TBIP improvements.  For example, 
Hughes was unable to hold the preliminary design review (PDR) on the 
TBIP contract in September 1995, when originally scheduled, and 
ultimately was not ready for PDR until February 1996.  In the view of 
the TBIP program office, Hughes's delay in this regard created the 
risk that Hughes would be unable to meet the required date for initial 
operating capability (IOC).  Tr. at 1013-1015, 1125-1127.  In 
addition, by letter dated February 26, 1996, Hughes advised the TBIP 
program office that the funds allocated to the TBIP contract--which 
was incrementally funded on an annual basis and contained a 
contractual limitation on the funds available for any given 
period--would be insufficient at current rates of expenditure to 
continue performance for the last approximately 3ï¿½ months of the 
current contract period.  Further, in an April 1996 briefing of the 
TBIP program office, Hughes warned the Navy that "TBIP Program IOC 
will slip 12 months and FOC [full operating capability] 15 months."  
Although Hughes referred to only a 10-month slip in IOC in a May 1996 
electronic mail to the TBIP contracting officer, it also referred to a 
"15 month replan schedule," while the TBIP program manager testified 
that he anticipated a 15-month delay in IOC (and not merely the 10- or 
12-month delay forecast by Hughes).  Tr. At 1054-1055.  Further still, 
the TBIP program office furnished Air Force evaluators information 
indicating that Hughes was proposing to stop work on several system 
capabilities.  Tr. at 1218-1219.  The fact (as pointed out by Hughes) 
that the schedule problems had not yet caused Hughes to miss any 
contractually established performance dates did not diminish the 
seriousness of Hughes's schedule problems; the only date established 
by the TBIP contract was IOC at the conclusion of the contract in the 
year 2000 and, again, as noted in an April 1996 briefing chart 
furnished to the TBIP program office, Hughes was advising contracting 
officials that it assumed that "TBIP Program IOC will slip 12 months."  
Tr. at 1013.

When questioned by the Air Force during discussions about its 
performance on the TBIP contract, Hughes's response led the agency to 
believe that Hughes was experiencing serious cost/schedule problems 
for which it bore significant responsibility.  In this regard, the Air 
Force's PFN in this area noted that there was an "approximate 32% 
($72M [million]) [cost overrun] plus another $42M problem identified 
by contractor in latest financial review," and a "[p]otential 3ï¿½ 
months schedule delay due to lack of funding plus another 12-month 
slip presented by contractor in latest financial review."  In its 
response, Hughes appeared to concede some responsibility for the cost 
overruns, stating that:

        "[a]dmittedly, Hughes's bid reflected specific optimistic 
        assumptions in its cost estimate, in particular related to the 
        [TBIP] seeker subcontract and engineering labor.  In addition, 
        it is Hughes's position that added work comprises $41.9M of 
        the $72M cost overrun cited in the PFN."  

(Hughes's response also noted that the Navy did not agree with its 
position.)  

The Air Force reasonably viewed Hughes's statement as an admission 
that it had underestimated its costs when competing for the contract 
and that the contract was at least $30.1 million over cost 
irrespective of any work allegedly added by the Navy.  Tr. at 1211, 
1235.  The agency's conclusion in this regard is consistent with a 
Hughes summary of the TBIP cost history obtained by the agency, in 
which Hughes attributed the cost growth to an "aggressive bidding 
strategy," as well as to "requirements clarifications/evolution."  It 
is also consistent with the TBIP program manager's position that 
Hughes "was largely responsible for the cost overruns" on the TBIP 
contract.  Tr. at 1019-1020.  Further, in its PFN response, Hughes did 
not deny the existence of these potential 3ï¿½- and 12-month schedule 
slips.  In our view, the Air Force could (and did) reasonably 
attribute the potential 3ï¿½-month funding-based delay in the first 
instance to Hughes's failure to properly cost its proposed TBIP 
contract effort.  Although Hughes attributed the "potential 12 month 
slip" to the cancellation of another missile program under which its 
seeker subcontractor was developing the target seeker Hughes proposed 
for TBIP, the information available to the Air Force from the Navy 
indicated that the choice of the seeker to propose had been up to 
Hughes, and suggested that, in any case, Hughes had unduly delayed in 
selecting a replacement seeker.  Tr. at 1037, 1045-1046, 1056, 1273. 

Having reasonably determined that Hughes was experiencing serious cost 
and schedule problems on the TBIP contract for which it bore 
significant responsibility, and in view of Hughes's 9-month schedule 
delay in its performance on the AMRAAM program, the Air Force 
reasonably evaluated Hughes's past cost/schedule performance as 
marginal.  Further, given the fact that its cost/schedule problems had 
occurred on programs that, like the JASSM program, involved missiles, 
and the fact that the schedule on the JASSM program was of paramount 
importance "due to the critical operational shortfall" the system was 
to fill, the SSA, in his cost/technical tradeoff, reasonably viewed 
Hughes's deficient past cost/schedule performance as a significant 
weakness.

Product Similarity

Hughes more generally argues that the evaluation of its past 
cost/schedule performance reflects an overall problem in the 
evaluation of past performance.  Essentially, Hughes contends that the 
agency unreasonably failed to consider the complexity of the products 
being developed and/or procured when evaluating past performance and 
arriving at an overall rating.  It specifically contrasts in this 
regard the evaluation of its past cost/schedule performance on the 
TBIP program with the evaluation of LM's performance on the Joint 
Direct Attack Munition (JDAM) program.

The Air Force examined LM's cost/schedule performance on three 
programs that it had determined would furnish useful data as to LM's 
likely JASSM cost/schedule performance:  (1) JDAM, an inertial 
guidance/global positioning system tail kit added to bombs to guide 
them to their target; (2) the Hellfire II program, an upgrade to the 
Hellfire air-launched anti-tank missile, with a semiactive laser 
seeker for guidance; and (3) the Low Altitude Navigation and Targeting 
Infrared System for Night (LANTIRN) program, consisting of a 
navigation pod and a targeting pod mounted on aircraft and primarily 
used for targeting laser-guided air-to-ground missiles.  Although LM 
experienced schedule delays during the development and initial 
production of LANTIRN and Hellfire II, and cost overruns of 
approximately 60 percent during the development of Hellfire II, the 
Air Force concluded that these problems were overshadowed by the facts 
that (1) major problems no longer existed on the LANTIRN program and 
LM was performing ahead of schedule, resulting in a LANTIRN program 
manager rating of "4," the top rating, (2) there had been no schedule 
problems on the JDAM program, and (3) LM was performing or had 
performed without cost problems during the Hellfire II production 
phase and generally on the LANTIRN and JDAM programs.  In particular, 
with respect to JDAM, the agency noted that the JDAM program manager 
reported that LM had never experienced any cost/schedule problems and, 
indeed, had performed ahead of schedule in some instances 
notwithstanding a "very challenging" schedule; according to the JDAM 
program manager, LM's JDAM performance was "absolutely superb."  As a 
result, the Air Force rated LM's past cost/schedule performance 
acceptable.

Noting the importance of LM's JDAM performance in LM's acceptable 
rating, and pointing out that unlike TBIP or AMRAAM, JDAM includes 
neither an engine, power supply nor warhead, Hughes contends that the 
agency has failed to account for the fact that Hughes's marginal 
cost/schedule performance rating was earned for products that were 
considerably more complex than LM's JDAM (or even LANTIRN).  According 
to Hughes, the agency's approach resulted in an overemphasis on 
program similarity at the expense of product similarity.  

Hughes's position is without merit.  The Air Force determined that 
LM's performance on the JDAM program would furnish a reliable 
predictor of its likely cost/schedule performance on JASSM based 
primarily on similarities in program complexity, not product 
complexity.  Tr. 721-747.  This determination was consistent with the 
agency's approach to past performance as announced prior to the due 
date for receipt of initial proposals.  Specifically, worksheets 
provided to potential offerors along with the draft RFP (for purposes 
of soliciting offeror input into the selection of the contracts to be 
evaluated) advised that the Air Force would evaluate the relevance of 
contracts for consideration under the cost/schedule factor using seven 
criteria:  (1) product similarity, with the greatest weight to be 
accorded to contracts/programs involving cruise missiles, and lesser 
(but still significant) weight to be given to guided missiles and 
guided bombs; (2) program phase; (3) system complexity; (4) 
subcontractor integration; (5) contract type; (6) contract 
environment; and (7) whether the contract/program involved a 
streamlined acquisition approach.  At most, only two of these factors 
(product similarity and system complexity), accounting for only 35 of 
105 rating points, related to product complexity.  (Moreover, even for 
those factors, it was clear that the agency may consider a guided bomb 
like JDAM to be somewhat similar to a cruise missile like TBIP since, 
although a full 20 points were available under the product similarity 
factor for a cruise missile, 12 points were available for a guided 
bomb.)  In general, the worksheets indicated the agency's position 
that for purposes of evaluating past cost/schedule performance, 
program similarity would be more significant than product similarity.  
Tr. at 748, 773-775.  Given the agency's previously announced position 
in this regard, the fact that the agency's approach resulted in 
programs for products of dissimilar complexity being considered 
similarly relevant for purposes of serving as a predictor of future 
cost/schedule performance on JASSM based on similarities in program 
complexity is not objectionable.

TARGET X

Hughes challenges the evaluation of mission/cost effectiveness under 
the affordability factor.  The solicitation provided for evaluation of 
the probable mission/cost effectiveness of the proposed JASSM missile 
based on consideration of the expected number of missiles required to 
kill the specified JASSM target set, total cost (number of missiles 
times most probable AUPP) to kill the JASSM target set, average cost 
to kill a JASSM target, and number of aircraft sorties (missions) 
required to kill the target set.  In this regard, the ultimate overall 
JASSM target set adopted by the Air Force and furnished to offerors 
for use in preparing their designs included 20 target types:  a 
threshold target set of 17 target types and 686 targets, and an 
objective target set comprised of all 20 target types and a total of 
717 targets.  In other words, 3 of the target types and 31 of the 
targets were simply objective targets.  While the solicitation 
provided for the entire JASSM target set, including objective targets, 
to be taken into account in evaluating mission/cost effectiveness, 
only threshhold target types were considered in the evaluation of 
compliance with key technical performance parameters.  One of the 
objective target types, code-named Target X for purposes of this 
protest, was a relatively large target that was most vulnerable to 
warheads emphasizing blast effectiveness--e.g., through use of a 
thinner warhead casing and more explosive--over penetrating 
capability.[3]   Target X was not included in the initial draft target 
set but, rather, was added in October 1995, approximately 5 months 
before the final solicitation was issued.

The prescribed approach adopted to measure missile effectiveness 
against most of the target types, including Target X, was the 
Department of Defense's Joint Munitions Effectiveness Manual (JMEM) 
Open End Methods model.  Tr. at 35, 167-168.  For a particular warhead 
against a particular target, this program calculates a single shot 
probability of damage (Pd), that is, the percentage of the target that 
a single warhead is likely to destroy.  The solicitation provided that 
"[t]he offeror shall use only Government provided methodology for 
computing Pd."    
Prior to issuance of the final solicitation, Hughes, MD, and LM 
advised the Air Force that destroying Target X would be difficult.  
Hughes advised the agency that the Target X class of targets "drives 
the need for" high explosive, and that without its proposed new 
[DELETED] warhead, which offered additional blast effectiveness, "it 
would cost you another thousand missiles" to destroy the Target X 
class of targets.  Tr. at 517-518.  Likewise, MD cautioned the agency 
that Target X "sucks up warheads like crazy," and that if it relied on 
a penetrator type of warhead against Target X, it would require 
significantly more missiles.  Notwithstanding these warnings, the Air 
Force retained Target X in the JASSM target set.

Upon evaluating the effectiveness of offerors' proposed warheads, 
however, the Air Force noted that the JMEM methodology predicted that 
large numbers of missiles would be required to destroy the Target X 
targets.  Although this was true for the warheads proposed by all of 
the offerors, it was particularly true for LM's warhead, which 
apparently [DELETED].  Indeed, the agency's initial evaluation 
indicated that approximately [DELETED] more LM missiles would be 
required to destroy the objective target set (which included Target X) 
than Hughes or MD missiles.  However, the Air Force concluded that the 
data indicated that the outputs from the JMEM model with respect to 
the Pd and number of weapons required to destroy Target X were 
unreliable due to Target X's vast size.  Specifically, the agency 
maintains that where the Pd value--that is, the probable single shot 
percentage of destruction of the target--is sufficiently small, 
inherent uncertainty in the JMEM model renders comparison among 
warheads unreliable.  Tr. at 47-65.[4]  Given the perceived 
unreliability of the data with respect to Target X, and the fact that 
the solicitation provided for the agency to evaluate the realism of 
the primary parameters used to calculate missile effectiveness, the 
agency essentially decided to normalize the results to prevent Target 
X from becoming a discriminating factor in the source selection; the 
agency adjusted the Target X effectiveness of all warheads to that 
achieved by the most effective warhead, which was [DELETED].    
    
Hughes argues that it was unreasonable for the Air Force to abandon 
the prescribed method (the JMEM model) for calculating missile 
effectiveness against Target X, and to instead equalize the warheads' 
effectiveness against Target X.  Hughes maintains that this had the 
effect of improperly eliminating from consideration the likely 
superior performance of Hughes's warhead against Target X relative to 
the performance of LM's warhead.  

It is clear that the Air Force has determined that using the 
prescribed evaluation approach--evaluating the effectiveness of 
warheads against Target X using the JMEM model--would produce a result 
which does not represent the agency's actual minimum needs.  According 
to the agency, given the relative ineffectiveness of all of the 
warheads against Target X, warfighters would be unlikely to mount a 
broad, area-based attack with JASSM missiles; they instead likely 
would either launch a precision attack against critical elements of 
Target X--about which insufficient information was provided to 
offerors to permit them to calculate such an attack--or attack Target 
X with a different weapon system.  

Upon determining that the prescribed evaluation approach would lead to 
an overemphasis on Target X and thereby produce a result which did not 
represent the agency's actual minimum needs, the agency was faced with 
a choice between effectively eliminating Target X from the evaluation 
without reopening negotiations or reopening negotiations on the basis 
of a revised evaluation approach.  In this regard, as a general 
matter, source selection officials do not have the discretion to 
announce in the solicitation that they will use one evaluation plan 
and then follow another without informing all offerors of any 
significant changes in the evaluation scheme and affording them an 
opportunity to respond to the modified evaluation approach.  See 
DynCorp, 71 Comp. Gen. 129 (1991), 91-2 CPD  para.  575; Colonial Storage 
Co.; Paxton Van Lines, Inc., B-253501.5 et al., Oct. 19, 1993, 93-2 
CPD  para.  234, aff'd, Colonial Storage Co.--Recon., B-253501.8, May 31, 
1994, 94-1 CPD  para.  335.  Here, the Air Force decided to eliminate Target 
X from the evaluation but not to reopen negotiations. 

Hughes argues that the Air Force made a significant change in the 
stated evaluation approach such that it was required to reopen 
negotiations so as to afford offerors an opportunity to respond by 
modifying their proposals.  Specifically, Hughes claims that had it 
been advised of the agency's decision, essentially, to remove Target X 
from the target list, it would have altered its warhead approach to 
account for this, perhaps by choosing a nondevelopmental warhead (and 
thereby reducing its evaluated warhead development risk and saving on 
the cost of developing the [DELETED]), or perhaps by redesigning the 
[DELETED] to optimize its performance against the remaining targets.  
(Hughes's chief engineer for the JASSM proposal estimated that 
choosing a nondevelopmental warhead could save approximately 
$[DELETED] in development costs and reduce Hughes's AUPP by 
approximately $[DELETED], while redesigning the [DELETED] could reduce 
Hughes's total contract price by approximately $[DELETED] and its AUPP 
by approximately $[DELETED]).  Tr. 507-518, 540-548, 571-572.  (We 
note, however, that in its prior submission dated July 25, 1996, 
Hughes stated that "Hughes likely would have proposed to use a 
significantly less expensive off-the-shelf warhead were it not for 
Target X.")  

Where the record establishes a procurement deficiency, we will sustain 
a protest on that basis only where it resulted in competitive 
prejudice; where an agency improperly changes its stated evaluation 
approach after receipt of offers, such prejudice exists where the 
record shows a reasonable possibility that the protester would have 
altered its proposal to its competitive advantage had it been given 
the opportunity to respond following the change.  Federal Computer 
Corp., B-239432, Aug. 29, 1990, 90-2 CPD  para.  175; see Global Assocs. 
Ltd., B-271693; B-271693.2, Aug. 2, 1996, 96-2 CPD  para.  100; Akal Sec., 
Inc., B-261996, Nov. 16, 1995, 95-2 CPD  para.  216.

The record in this case is extensive, including contemporaneous 
internal documentation of Hughes's design and warhead selection 
process and testimony by the Hughes chief engineer for the JASSM 
project.  Based on a close review of this record, we conclude that, at 
most, had the Air Force sought revised proposals, Hughes could have 
reduced its development costs by a relatively small amount.  While it 
is possible that a change in warhead could have resulted in a less 
expensive Hughes missile, i.e., a lower AUPP, additional Hughes 
missiles would be required to destroy the JASSM target set, so that 
overall mission/cost effectiveness would not have materially improved 
and might actually have suffered.  Given the advantages both LM and MD 
had in past performance and technical performance, together worth 75 
percent of the evaluation, we share the Air Force's opinion that 
Hughes could not have altered its proposal to its competitive 
advantage.  We reach this view even assuming that given an opportunity 
to revise their proposals to reflect the elimination of Target X, LM, 
and MD would not have improved their own competitive standing. 

As an initial matter, Hughes has not specified which warhead it would 
have substituted for the [DELETED]; according to the protester, a 
reexamination of its warhead approach "would take thousands of hours 
because that's how many hours it took for our engineers to do the 
original to determine the best warhead."  Tr. at 517, 542, 567.  This 
inability to specify the particular replacement warhead Hughes would 
have selected left the Air Force and the awardees unable to address 
specifically Hughes's claim of prejudice, but it also left Hughes with 
only speculation about a possible warhead redesign to contradict the 
existing record, which leads us to conclude that Hughes would not have 
offered a different warhead.  In this regard, the JASSM target list 
included other blast-intensive targets (besides Target X) and internal 
Hughes documentation produced in response to an agency document 
request shows that Hughes's proposed [DELETED] warhead clearly would 
be the most effective warhead--i.e., it would require fewer total 
missiles--against the JASSM target set even after the elimination of 
Target X.  Tr. at 525-528, 533-539, 549-551, 574-576, 927-928.  

For example, Hughes proposed the [DELETED] as a backup in case 
developmental problems were encountered on the [DELETED] warhead.  Tr. 
at 499-500.  Approximately 23 percent more JASSM missiles with 
[DELETED] warheads than missiles with [DELETED] warheads would be 
required to destroy the JASSM target set even after the elimination of 
Target X.[5]  Further, the Hughes documentation indicates that the 
[DELETED] was viewed by Hughes as having a higher risk ("high" risk) 
and unit cost than Hughes's [DELETED] warhead design (which was viewed 
as "low" risk) from which the [DELETED] design was derived.[6]  In 
this regard, the Hughes documentation indicates that the [DELETED] was 
viewed as having the lowest risk and unit price of any of the warhead 
options discussed in the documentation and Hughes's chief engineer 
testified that the [DELETED] warhead had the same unit cost and risk 
ratings as the [DELETED].  Tr. at 550-551.[7]  (According to the 
agency, however, since the warhead on Hughes's [DELETED] design was 
smaller than the one on the [DELETED] design Hughes's missile 
effectiveness ratings would have suffered had it substituted the 
[DELETED] for the [DELETED], while the evaluated development risk 
would have remained the same.)[8]  As for the possibility that Hughes 
could have begun developing still another new warhead had it been 
advised of the removal of Target X from the target set, we note that 
this would not have eliminated the evaluated risk associated with 
Hughes's proposal to develop the [DELETED] warhead, but instead could 
very well have increased that risk given Hughes's belated start on 
designing such a warhead.  Since the record--including Hughes's 
contemporaneous documentation--indicates that Hughes's [DELETED] 
warhead remained its most effective warhead even after removal of 
Target X from the JASSM target set, and given the substantial effort 
required to change warheads, we simply do not find credible Hughes's 
claim that elimination of Target X would have led it to alter its 
proposal to its significant competitive advantage.

Furthermore, even if Hughes had selected a different warhead--for 
example, the [DELETED]--and, as a result, had been able to reduce its 
total contract price for the development phases of the JASSM program 
by up to Hughes's estimate of $[DELETED] for the [DELETED] (leaving 
its price at $[DELETED], considerably higher than the $[DELETED] and 
$[DELETED] prices offered by LM and MD), and also lowered its 
estimated AUPP for the [DELETED] production quantities, there is no 
basis for concluding that Hughes would have been in line for an award.  
Hughes offered no contradiction to agency assertions that the 
[DELETED] would have markedly reduced the Hughes mission effectiveness 
numbers.  Similarly, a $[DELETED] reduction in development cost that 
Hughes asserts would result from redesign of the [DELETED], and a 
reduction in AUPP, would not have placed Hughes in line for award.  
Here, too, agency assertions that Hughes's overall missile 
effectiveness numbers would have suffered--because a reduction in 
warhead size (which Hughes suggested) would have increased the number 
of missiles needed to destroy the JASSM target set--were effectively 
uncontradicted by the protester.  Hughes's proposal was reasonably 
evaluated as having a significant weakness with respect to past 
cost/schedule performance, and being less advantageous than LM's or 
MD's proposals with respect to overall past performance, which was 
worth 50 percent of the evaluation.  In addition, Hughes's proposal 
was also evaluated as being of higher risk and less advantageous with 
respect to technical performance, which was worth another 25 percent 
of the evaluation.[9]  Further, its total contract price for the 
development phases of the JASSM program under any scenario would have 
remained substantially higher than either LM's or MD's prices.[10]  In 
these circumstances, even assuming no improvement in LM's and MD's 
proposals had the Air Force reopened negotiations, we see no 
reasonable possibility that any improvement in Hughes's position with 
respect to the affordability criterion, which accounted for only 25 
percent of the evaluation, would be sufficient to offset its evaluated 
disadvantages under evaluation criteria worth 75 percent of the 
evaluation.

The protest is denied.

Comptroller General
of the United States

1. Information in CPARs (including yellow performance ratings) and 
from DCMC indicated that Hughes had experienced schedule delays in its 
performance on the AMRAAM program.  In response to the PFN issued in 
this regard, Hughes conceded that a 9-month schedule delay had been 
encountered on an AMRAAM contract, but pointed out that this had not 
resulted in a delay on a subsequent production contract and that its 
current CPAR performance ratings on the AMRAAM program were either 
blue or green.  The agency nevertheless considered the delay to be 
relevant since it appeared that a delay in production had been avoided 
only because the production schedule was able to accommodate the delay 
under the preceding contract.

2. Although, as pointed out by Hughes, the TBIP program manager at one 
point advised the Air Force that "[i]n general, [he] was pleased with 
the performance of both McDonnell Douglas and Hughes" on the overall 
Tomahawk program, the record indicates that the program manager 
concurred in the evaluation that Hughes had encountered cost and 
schedule problems on TBIP and, as noted below, he specifically 
testified that Hughes "was largely responsible for the cost overruns" 
on the TBIP contract.  Tr. at 1012-1024, 1112-1113.

3. Since some of the information concerning the JASSM target set is 
classified, our discussion of the Target X issue is somewhat limited.

4. We note, however, that the record indicates that the warheads in 
fact possessed different blast capabilities.  In this regard, an 
agency engineer who participated in the JASSM lethality assessment 
testified that the inherent uncertainties in the JMEM model when used 
to evaluate the effectiveness of the proposed JASSM warheads against 
large targets such as Target X were such that it was "[n]ot 
necessarily" the case that the LM warhead would be "considerably less 
effective" against Target X than the Hughes warhead.  Tr. at 70, 73, 
75, 608.  However, he also conceded that there was indeed a difference 
between Hughes's and LM's warheads; according to the engineer, the 
fact that Hughes's warhead [DELETED] will result in Hughes's warhead 
producing a bigger blast than LM's, Tr. at 213-214, 237, 605, and a 
warhead with a greater blast effectiveness was likely to be more 
effective against Target X.  Tr. at 607-608.  The agency engineer 
further conceded that using the effectiveness data for the [DELETED] 
warhead would not furnish a more reliable predictor of warhead 
effectiveness against Target X than the effectiveness data supplied by 
LM for its own warhead.  Tr. at 260-261. 

5. The [DELETED] warhead's approximate 23 percent advantage in this 
regard is based on a comparison between Hughes's effectiveness numbers 
for the [DELETED] APW and the evaluated effectiveness of the [DELETED] 
at a low circular error probable (CEP).  (The CEP denotes the radius 
around the target in which 50 percent of the missiles can be expected 
to land.)  The JASSM program manager testified that the agency 
likewise had concluded during its evaluation that the effectiveness 
numbers for Hughes's missile would have "declined markedly" and the 
unit price increased had Hughes substituted the [DELETED] warhead for 
the [DELETED] warhead.  Tr. at 915, 925-928, 979-982.    

6. The [DELETED] warhead was [DELETED] inches longer, included 
approximately [DELETED] pounds more high explosive, and was viewed as 
a better penetrator than the [DELETED] warhead.  Tr. at 506-507.

7. In this regard, Hughes's chief engineer testified as follows:

               "Question:   And does this chart show that of these 
               [warhead] options, the lowest unit cost was for the 
               [DELETED]?

               Answer:   Yes.

               Question:   And do you know if the unit cost of the 
               [DELETED] also would be the lowest unit cost in 
               comparison with these other warhead options?

               Answer:   Yes.

               Question:   If Target [X] were removed from the target 
               set, would that change the unit cost on any of these 
               warhead options?

               Answer:   These were estimates, but no. . . .

               Question:   I see the risk assessments for the 
               [DELETED] warhead candidate, and the risk assessments 
               across the row are all low.  Do you see that?

               Answer:   Correct.

               Question:   Would these risk assessments be the same 
               for the [DELETED] warhead?

               Answer:   Probably pretty close, yes.

               Question:    Would any of these risk assessments change 
               as a result of the removal of Target [X] from the 
               target set?

               Answer:   No."

Tr. at 550-551.
   
8. The Air Force notes that when during discussions it questioned 
Hughes's proposed development schedule, Hughes responded that it had 
"initiated work on analysis and design of the JASSM [DELETED] series 
warhead configurations in September of 1995," that is, prior to the 
addition of Target X to the JASSM target set.  Hughes's chief 
engineer, however, testified that this reference was only "an 
after-the-fact term we gave to the entire development series for the 
[DELETED] type warheads that we were looking at, like the [DELETED]"; 
according to the engineer, Hughes only started looking at the 
[DELETED] design "when . . . the target set changed, and that was in 
October," and "actually arrived at the design in January."  Tr. at 
508-510.  

9. Although Hughes's selection of a nondevelopmental warhead could 
have reduced its risk under the technical criterion, its evaluated 
higher risk in this area was based on additional considerations beyond 
the development risk associated with its new [DELETED] warhead and, in 
any case, its substitution of a nondevelopmental warhead such as the 
[DELETED] would have resulted in a marked decline in the effectiveness 
numbers for its missile.    

10. Hughes also challenges the cost/technical tradeoff, primarily on 
the ground that the SSA failed adequately to take into account 
Hughes's affordability advantage.  The record shows that the SSA did 
not consider Hughes's "roughly [DELETED] percent" lower AUPP for 
production lots 1-5 significant in light of the fact that the AUPP for 
LM and MD were not viewed as excessive and the speculative nature of 
the AUPP at this stage in the program (in that the program is only at 
the beginning of the PDRR phase and production is years away, while 
the offerors besides Hughes are not committed to their current 
production pricing and Hughes is only committed to its current pricing 
for the first 165 missiles of the planned 2,400 missile purchase).  
Nothing in the solicitation precluded the SSA from considering the 
significance of Hughes's AUPP advantage in light of these factors.