BNUMBER: B-261154.4
DATE: October 16, 1995
TITLE: AT&T Corporation, Advanced Technology Systems
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a
GAO Protective Order. This version has been redacted or approved by
the parties involved for public release.
Matter of:AT&T Corporation, Advanced Technology Systems
File: B-261154.4
Date:October 16, 1995
C. Stanley Dees, Esq., Daniel G. Jarcho, Esq., and Patrick K. O'Keefe,
Esq., McKenna & Cuneo, for the protester.
Paul Shnitzer, Esq., and David Z. Bodenheimer, Esq., Crowell & Moring,
for Loral Federal Systems, an interested party.
Robert D. Hogue, Esq., Debra B. Haworth, Esq., and Stephanie Buser,
Esq., Space and Naval Warfare Systems Command, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest against source selection is denied where (1) agency reasonably
determined that awardee had proposed a fundamentally less complex
approach to deploying an underseas surveillance system which was more
likely to successfully and timely deploy a survivable underwater
segment than was the protester's, and (2) the record provides no basis
for concluding that the awardee's contract cost was likely to be so
substantially higher than the protester's as to offset the awardee's
superiority under the substantially more important technical factors.
DECISION
AT&T Corporation, Advanced Technology Systems, protests the award of a
contract to Loral Federal Systems under request for proposals (RFP)
No. N00039-94-R-0020, issued by Department of the Navy, Space and
Naval Warfare Systems Command (SPAWAR), for the
demonstration/validation (DEM/VAL) phase of an underseas surveillance
system. AT&T challenges the evaluation of cost and technical
proposals.[1]
We deny the protest.
The RFP contemplated the award of a cost-plus-award-fee contract to
design, develop, document, fabricate, assemble, inspect, integrate,
and support the development of a DEM/VAL phase prototype of the
Advanced Deployable System (ADS), a passive underseas surveillance
system providing tactical warfare information to commanders in
littoral (shallow) water areas. The solicitation required offerors to
submit a technical proposal and system design documentation package
which:
"in combination shall be sufficiently specific, detailed, and
complete so as to clearly and fully demonstrate to the
Government that the prospective offeror has a thorough
understanding of the requirements for and problems inherent in
performing, as well as the capability to perform the
Demonstration and Validation Phase and delivery of ADS."
The solicitation further required offerors to submit cost proposals
which explained:
"in whatever detail is required to demonstrate cost
reasonableness and supportability, the methodology used to
estimate each element of cost (e.g., labor, material, etc.).
Enough data shall be provided so that an independent cost
analysis verification can be performed."
In this regard, offerors were required to submit their cost estimates
using a work breakdown structure (WBS) furnished with the solicitation
and to provide:
"[a]dequate information . . . to allow Government evaluation
of proposed labor hours, material costs, subcontractor costs,
other direct costs and related overhead costs by the fourth
level of the WBS. Major subcontractor estimates (i.e.,
exceeding Five Million Dollars ($5M)) shall be submitted using
the same formats and degree of rationale."
The solicitation generally provided for award to be made to the
offeror whose proposal is "considered most likely to satisfy the
requirements of the government and to be in the best interest of the
Government, cost and other factors considered." The RFP provided for
the evaluation of proposals on the basis of the following four
specific evaluation factors (in descending order of importance):
(1) overall system design/development and (2) prototype development,
implementation and demonstration, which were "substantially more
important than" (3) management and (4) cost. Cost proposals were to
be evaluated for (1) contract cost affordability/reasonableness and
(2) life-cycle cost. With respect to the proposed contract costs, the
RFP stated that:
"[t]he Government will conduct its own evaluation of these
costs and the associated ranges, and will rely on this
evaluation when determining the award. The Government's cost
evaluation will be affected by the quality of the supporting
data provided and on the basis of traceability of the proposed
costs to the technical/management proposal, as well as the
offeror's demonstrated ability to deliver large, technically
complex development programs within budget. The primary
purpose of this criterion is to determine if the cost proposal
for the prototype development, demonstration/validation is
complete, realistic and reasonable."
The RFP likewise provided for the government to conduct its own
evaluation of the proposed life-cycle costs for use in selecting a
proposal for award.
Four proposals were received by the closing time. The proposals
submitted by Loral and AT&T were based on [DELETED]; two other
proposals were based, entirely or primarily, on [DELETED]. Subsequent
to the receipt of initial proposals, SPAWAR amended the solicitation
to delete provisions for potential installation by air drop. Offerors
were requested to submit updated proposals, but were not afforded the
opportunity for discussions.
Since an autonomous architecture was not achievable in time for
prototype demonstration, resulting in both of the proposals that
emphasized an autonomous approach receiving an overall "below
adequate" rating under the technical factors, and the life-cycle costs
of the autonomous approach were expected to be substantially higher
than those for the cabled approach, the source selection decision
focused on the cable-connected proposals submitted by Loral and AT&T.
The evaluation results are shown below.
AT&T Loral
TECHNICAL
System Design Adequate Above Adequate
Prototype DemonstrationBelow
AdequateAdequate
Management Below Adequate Adequate
Overall Technical Adequate Above Adequate
COST
Proposed DEM/VAL $30.5 million $42.4 million
Evaluated DEM/VAL $30.5-62.6 million $42.4-56.2 million
Evaluated Life-Cycle$[DELETED] million $[DELETED] million
The record establishes that cost considerations were secondary in the
source selection decision. As noted above, cost and management were
"substantially" less important than the system design and prototype
demonstration evaluation factors. Further, although the evaluated
life-cycle cost of AT&T's approach, as determined by application of a
parametric cost model, was slightly higher than Loral's, the source
selection advisory council (SSAC) evaluated the life-cycle costs of
the two proposals as being "virtually the same" and concluded that
they "provide[d] no conclusive basis for selection of one over the
other."
Offerors' proposed DEM/VAL costs were not determinative because the
cost evaluation team (CET) found the task of developing an independent
estimate of the most probable cost of each offeror's DEM/VAL approach
to be "extremely difficult." The CET concluded that the proposals
lacked sufficient detail to permit it to undertake the detailed,
"bottoms-up" cost realism analysis recommended in the Navy's Cost
Realism Handbook and traditionally used at SPAWAR--that is, a detailed
build-up of contract cost from the lowest WBS levels based on a
detailed review of proposed labor and hardware costs. In this regard,
although the solicitation generally required offerors to submit cost
proposals which included "whatever detail is required to demonstrate
cost reasonableness and . . . the methodology used to estimate each
element of cost . . . so that an independent cost analysis
verification can be performed," and specifically extended the
obligation to furnish detailed WBS costs and an explanation of the
underlying cost methodology to major subcontractors, AT&T failed to
furnish a cost proposal for a subcontractor accounting for [DELETED]
percent of its proposed cost. (In addition, the source selection
evaluation board found that AT&T had proposed inadequate staffing
levels to support its technical approach.) As for Loral's proposal,
the CET expressed concern that the cost proposal of its principal
subcontractor was based only on "engineering judgment" and that no
historical data supported the subcontractor's proposed efficiency and
learning curves. Since the CET was unable to develop independent
estimates of offerors' DEM/VAL costs, the panel recommended that
SPAWAR open discussions with offerors to acquire detailed cost backup
data.
Instead of commencing discussions, however, SPAWAR determined that
DEM/VAL phase cost realism could be evaluated by using the parametric
cost model used to develop the life-cycle cost estimates, as modified
to take into account certain proposal-unique information and eliminate
costs associated with the engineering and manufacturing development
(EMD) and production phases. Although the CET had found there were
"serious weaknesses inherent in the model . . . which . . . prevent it
from calculating other than rough approximations of
contractor-specific [DEM/VAL] cost estimates," the SSAC concluded that
the parametric cost model contained sufficient
proposal-and-item-unique cost drivers to permit its use to define the
upper limit of the cost estimate, with the lower limit of the range
consisting of the offerors' proposed costs.[2]
Viewing the wide range between AT&T's proposed cost ($30.5 million)
and the cost derived from the modified parametric cost model ($62.6
million) as calling into question the reliability of AT&T's cost data,
and given AT&T's failure to furnish a cost proposal for its [DELETED]
subcontractor and its proposal of inadequate staffing to support its
proposed technical approach, the SSAC determined that AT&T's proposal
was not well supported. Noting also that a "negative 25 % cost
variance"--cost overrun--had occurred with respect to AT&T's
performance on the underwater segment of another underseas
surveillance system, the recent program most like the ADS program, the
SSAC found AT&T's cost proposal to be characterized by "substantial
cost risk."
In contrast, the SSAC concluded that while "it may have been
optimistic, Loral's [cost] proposal is not considered unrealistic, and
is more realistic than AT&T['s]." The panel based its conclusion on
the fact that: (1) the range between Loral's proposed cost ($42.4
million) and the cost derived from the modified parametric cost model
($56.2 million) was considerably narrower than the range for the AT&T
proposal, supporting the conclusion that Loral's proposed costs were
more realistic; (2) a review of proposed labor hours by WBS element
indicated that Loral's proposed staffing was adequate to support its
technical approach; (3) its subcontractor had supported its proposed
cost down to the seventh WBS level, attempted to account for known
technical risks in terms of potential additional labor hours, and
added [DELETED] percent to its own vendor cost inputs; (4) the
proposed direct and indirect rates were reasonable; and (5) only an
approximately 10 percent cost overrun had occurred with respect to
Loral's performance on the contract for the shore processing segment
of the previous underseas surveillance system contract.
Notwithstanding its ultimate determination that Loral's cost proposal
was "realistic and reasonable," and not characterized by the
substantial cost risk associated with AT&T's proposal, the SSAC
ultimately concluded that it "would not expect costs for AT&T and
Loral to differ significantly at the end of DEM/VAL."
The evaluated technical superiority of Loral's approach was
determinative in the source selection. The SSAC report noted that
"[s]urvivability is the primary risk of the [DELETED] architectures.
Deployment of the [DELETED] architectures is an aspect of the
survivability risk . . . ." Likewise, according to the SSAC
co-chairman (and program manager), "[d]eployment is everything in
ADS."
[DELETED]
SPAWAR evaluated AT&T's [DELETED] approach to [DELETED] as an overly
complex, high risk approach which conferred little or no added, real
benefit to the deployment process and would be unable to deploy at the
full range of speeds necessary to meet the ADS deployment time
requirements. Hearing Transcript (Tr.) at 53-54, 108-109, 157-158.
[DELETED]
The lead evaluator (and agency expert) in this area reported to the
SSAC that the AT&T deployment approach was "not a suitable method for
installing ADS [DELETED] systems." The SSAC determined that "[t]he
Loral approach is judged to be of lower technical risk than AT&T['s]
to successfully deploy a survivable underwater segment. This is due
to the simplicity of the Loral deployment approach compared to the
method proposed by AT&T." Further, noting that cost "estimates at
this stage of development are inherently uncertain due to technical
risk," and concluding that discussions were unlikely to result in
further useful information, the SSAC recommended against commencing
discussions. The SSAC instead recommended award to Loral, explaining
that:
"[g]iven the technical superiority of Loral, its reasonable
and affordable [life-cycle cost] evaluation (which is
virtually the same as AT&T), and its sufficiently realistic
(and more realistic than AT&T) DEM/VAL offer, discussions
would add little value to the selection.
"Despite the proposed cost difference, the SSAC would not
expect costs for AT&T and Loral to differ significantly at the
end of DEM/VAL. And even if AT&T were to cost $10 M[illion]
less than Loral, as proposed, the latter's technical
superiority, short term cost reasonableness and affordability,
and long term cost effectiveness approximately equal to AT&T,
make it the best value to the Government."
The source selection authority accepted the SSAC's recommendation and
made award to Loral without discussions. AT&T thereupon filed this
protest with our Office.
TECHNICAL EVALUATION
Although AT&T raises a number of arguments challenging the technical
evaluation, the reasonableness of SPAWAR's overall conclusion that
Loral submitted a technically superior proposal depends on the
reasonableness of the agency's fundamental technical determination
that Loral's less complex deployment approach was more likely to
successfully and timely deploy a survivable ADS underwater segment
than was AT&T's. We find the agency's determination in this regard
reasonable.
Deployment Speed
AT&T challenges SPAWAR's determination that the deployment speed of
its proposed [DELETED] would be inadequate. [DELETED]
As an initial matter, to the extent that the agency allegedly failed
to fully appreciate AT&T's intent in this area, the responsibility
must rest largely with AT&T for failing to submit an adequately,
clearly written proposal. The solicitation required offerors to
submit proposals which were:
"sufficiently specific, detailed, and complete so as to
clearly and fully demonstrate to the Government that the
prospective offeror has a thorough understanding of the
requirements for and problems inherent in performing, as well
as the capability to perform the Demonstration and Validation
Phase and delivery of ADS."
As noted by the agency, AT&T's proposal did not describe in any detail
its intended [DELETED]
Complexity
Even if AT&T's [DELETED] were to prove capable of deploying at speeds
of interest for ADS, the record supports the agency's position that
AT&T's approach would be more complex than Loral's [DELETED] approach,
without adding any appreciable compensating benefit. Further, AT&T
has furnished no basis for questioning SPAWAR's position that a less
complex approach that satisfies the agency's needs is to be preferred.
[DELETED]
It is clear from the record that the agency viewed the lesser
complexity of Loral's deployment approach as significantly enhancing
the likelihood of successful deployment of ADS.
We find that SPAWAR reasonably determined that Loral submitted a
technically superior proposal which offered a fundamentally less
complex deployment approach that was more likely to successfully and
timely deploy a survivable ADS underwater segment than AT&T's.
COST EVALUATION
AT&T challenges SPAWAR's cost evaluation primarily on the basis that
the agency failed to prepare independent estimates of the likely
actual cost of DEM/VAL performance based on the approach proposed by
each offeror.
As a general rule, agencies are required to include cost or price as a
significant factor in the evaluation of proposals. Competition in
Contracting Act (CICA), 10 U.S.C. sec. 2305(a)(2)(A)(i) (1994);
Federal Acquisition Regulation (FAR) sec. 15.605(b). An evaluation and
source selection which fails to give significant consideration to
cost, or which varies from the RFP's cost evaluation provisions, is
inconsistent with CICA and cannot serve as the basis for a reasonable
source selection decision. See Lockheed, IMS, B-248686, Sept. 15,
1992, 92-2 CPD para. 180. Further, when a cost reimbursement contract is
to be awarded, the offerors' estimated costs of contract performance
should not be considered as controlling since the estimates may not
provide valid indications of the final contract costs which the
government is required to pay. See FAR sec. 15.605(d). Consequently,
the contracting agency must perform a cost realism analysis to
determine the realism of an offeror's proposed costs and to determine
what the costs are likely to be under the offeror's technical
approach, assuming reasonable economy and efficiency. CACI,
Inc.-Fed., 64 Comp. Gen. 71 (1984), 84-2 CPD para. 542; GTE Gov't Sys.
Corp., B-260022; B-260022.2, May 16, 1995, 95-1 CPD para. 245.
SPAWAR's cost evaluation approach resulted in the calculation for
AT&T's proposal of only a broad range--from $30.5-$62.6 million--of
possible cost of DEM/VAL performance. However, SPAWAR's failure to
more accurately measure the most probable cost of AT&T's proposal does
not warrant sustaining AT&T's protest in this regard, since this
failure was largely the result of AT&T's failure to submit an
adequately supported cost proposal, in particular, [DELETED]
"[a]dequate information . . . to allow Government evaluation of
proposed labor hours, material costs, subcontractor costs, other
direct costs and related overhead costs by the fourth level of the
WBS," so that "an independent cost analysis verification can be
performed."
Further, given the agency's inability to calculate a specific most
probable DEM/VAL cost for AT&T's proposal (as a result of AT&T's
failure to submit an adequately written cost proposal), there was no
specific AT&T most probable cost with which to compare a specific
Loral most probable cost. Moreover, the SSAC generally concluded that
Loral's relatively detailed DEM/VAL cost proposal was not unrealistic.
AT&T has not shown that Loral's cost proposal omitted any probable
costs of DEM/VAL performance which were so significant as to affect
the reasonableness of the source selection decision. As discussed
above, it is clear from the record that any uncertainties with respect
to cost were not significant in the source selection decision since
the agency was primarily concerned with successful and timely
deployment of a survivable ADS underwater segment.
CONCLUSION
Since SPAWAR reasonably determined that Loral submitted a technically
superior proposal, the proposed approaches of AT&T and Loral also
entailed approximately equal life-cycle costs, and the record provides
no basis for concluding that Loral's DEM/VAL cost was likely to be so
substantially higher than AT&T's as to offset Loral's superiority
under the substantially more important technical factors, we find the
selection of Loral unobjectionable.[3]
The protest is denied.
Comptroller General
of the United States
1. Inasmuch as portions of the record are classified, the following
unclassified discussion is necessarily somewhat general with respect
to several aspects of the procurement.
2. The CET noted that the parametric cost model was based on a
database that did not separate DEM/VAL and EMD costs and assumed the
use of less commercial-off-the-shelf/nondevelopmental items (COTS/NDI)
than proposed for ADS. According to SPAWAR, however, the limitation
with respect to COTS/NDI did not prejudice AT&T, since AT&T and Loral
reportedly proposed approximately the same level of COTS/NDI when
common definitions are used.
3. AT&T contends that SPAWAR should have conducted discussions in
order to calculate a most probable cost for each proposal and
otherwise resolve uncertainties in the proposals. Again, however, the
agency had substantial cost information, and given that this
information reasonably indicated that any cost differences would not
affect the award decision--which was driven primarily by technical
concerns--discussions (in lieu of award based on revised initial
proposals) were not required.