BNUMBER: B-276487.2
DATE: June 30, 1997
TITLE: GTE Hawaiian Telephone Company, Inc., B-276487.2, June 30,
1997
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a
GAO Protective Order. This version has been redacted or approved by
the parties involved for public release.
Matter of:GTE Hawaiian Telephone Company, Inc.
File: B-276487.2
Date:June 30, 1997
Rand L. Allen, Esq., Paul F. Khoury, Esq., David A. Vogel, Esq., and
R. Paul Margie, Jr., Esq., Wiley, Rein & Fielding, and Michael W.
Clancy, Esq., for the protester.
C. Stanley Dees, Esq., Thomas C. Papson, Esq., Patrick K. O'Keefe,
Esq., and Stephen E. Ruscus, Esq., McKenna & Cuneo, and Steven W.
DeGeorge, Esq., for AT&T Corporation, an intervenor.
H. Jack Shearer, Esq., Defense Information Systems Agency, for the
agency.
Guy R. Pietrovito, Esq., and James A. Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. In a negotiated procurement for telecommunications services where
the solicitation established detailed functional performance
specifications and a page limitation for technical proposals, the
procuring agency reasonably determined the compliance of the awardee's
(and protester's) proposals based in part upon the firms' promises of
compliance with more than 1,300 performance requirements and evidence
of technical capability or feasibility.
2. Lower-level evaluators' assessment of significant risk of delay in
meeting schedule requirements did not demonstrate that the awardee
would not satisfy these requirements, where the source selection
official reasonably found that the awardee had committed to, and was
capable of, satisfying the schedule requirements.
3. Where price is stated to be more important than the technical and
management evaluation factors, the agency's judgment that the
awardee's substantial price advantage outweighed the protester's
acknowledged technical superiority, primarily in the area of evaluated
schedule risk, was reasonable and in accord with the award evaluation
criteria.
DECISION
GTE Hawaiian Telephone Company, Inc. protests the award of a contract
to AT&T Corporation under request for proposals (RFP) No.
DCA300-96-R-0001, issued by the Defense Information Technology
Contracting Organization, Defense Information Systems Agency (DISA)
for the Hawaii Information Transfer System (HITS). GTE contends that
AT&T's proposal is not compliant with mandatory solicitation
requirements and that DISA performed an unreasonable best value
analysis in selecting AT&T's lower-rated, lower-priced proposal for
award.
We deny the protest.
The RFP provided for the award of a fixed-price, indefinite delivery
services contract to provide end-to-end switched voice, switched data,
integrated services digital network (ISDN) and dedicated transmission
services to Department of Defense users throughout the State of Hawaii
for a 10-year contract period. The HITS contract will replace the
existing Oahu Telephone System, the Hawaii Area Wideband Systems, and
other dedicated transmission service systems,[1] allowing the
government to consolidate its telecommunication services under one
contract. HITS will interface and interoperate with other
communication networks, including the Defense Information System
Network, the Government Emergency Telecommunication Service, the
Defense Satellite Communication System, FTS2000, and public telephone
networks.
A Functional Requirements Specification (FRS) was provided in the RFP,
which specified the information transfer requirements for HITS,
identified the services and features that HITS must provide, and
stated performance specifications and network management system
requirements. The contractor is required to provide whatever
equipment, facilities, and network management are necessary to deliver
services in compliance with the FRS requirements. The RFP stated with
regard to the configuration of the HITS:
"The Contractor shall make maximum use of commercially available
off-the-shelf technology and services. The Contractor's system
engineering shall maximize configuring HITS from existing
off-the-shelf equipment, services, and subsystems and minimize
designing new equipment, subsystems, and services. The HITS
configuration shall represent the most cost effective and
technically efficient solution to meet Government requirements."
The RFP also identified the locations for which service was required
and the types of service that would be provided at each location
(e.g., switched voice service, dedicated transmission service, or all
required services).
The contractor is required to provide Initial Operational Capability
(IOC) within 8 months of contract award. IOC is defined as the point
at which the first HITS location and services are operational and
accepted by the government. The contractor is also required to
provide Full Operational Capability (FOC) within 18 months of contract
award. FOC is defined as the point at which all service requirements
items and features for all locations have been "cutover" and accepted
by the government. Offerors were to provide transition plans to
describe their strategy for consolidating existing services into a
single communication infrastructure and to satisfy the IOC and FOC
schedule requirements. Detailed acceptance testing procedures were
also set forth in the RFP; among other things, the contractor was
required to plan for pre-cutover and post-cutover testing to
demonstrate that all HITS requirements will be satisfied.
The RFP provided a best value basis for award and identified the
following evaluation criteria: technical quality, management quality,
and price.[2] Price was stated to be approximately equal in weight to
the combination of the technical and management quality criteria;
technical quality was stated to be of greater weight than management
quality. The RFP cautioned that the government would "not pay a
significantly higher price for slightly superior technical and
management quality."
Offerors were informed that DISA would first evaluate proposals on a
pass/fail basis to determine compliance with every requirement
identified in the statement of work and FRS. Compliant proposals
would then be qualitatively assessed under the technical and
management quality criteria for understanding, compliance with
requirements, and risk. In this regard, offerors were instructed to
discuss their design and technical approach to satisfy each RFP
requirement and to fully demonstrate their understanding of the
requirements.
DISA received proposals from only GTE and AT&T.[3] Four rounds of
discussions were conducted with the two offerors. During discussions
and based upon the offerors' responses, DISA decided that the RFP
requirements were overly restrictive in a number of regards and
amended the RFP to relax 17 solicitation requirements. At the
conclusion of technical discussions, DISA's source selection
evaluation board (SSEB) found that both GTE's and AT&T's proposals
were fully compliant with the RFP requirements, as amended; in so
concluding, the SSEB compared the offerors' proposals against a
"compliance matrix" that ensured that each proposal satisfied the
1,350 RFP performance requirements. Hearing Transcript (Tr.) at 52,
149, 211.[4] The final technical quality and management quality
criteria scores were as follows:
Factor Score[5] Risk[6] Weight Result
GTE Technical 6.13 Low to Moderate 60% 3.68
Management5.73 Low 40% 2.29
TOTAL 5.97
AT&T Technical 4.20 Moderate to High 60% 2.52
Management6.13 Low 40% 2.45
TOTAL 4.97
As indicated by the scores, the SSEB found GTE's technical proposal to
be superior to AT&T's, while AT&T's management proposal was found to
be superior to GTE's.
Regarding AT&T's technical proposal, the SSEB assessed as a strength
AT&T's proposal to build its system around the 5ESS-2000 switch,[7] a
telecommuncations switch considered established, reliable and
flexible, which AT&T proposed to interconnect with multiple SONET
rings.[8] The SSEB also noted a number of disadvantages and risks in
AT&T's proposal arising primarily from AT&T's need to provide
extensive software development to satisfy a number of critical HITS
requirements, and AT&T's failure to adequately describe its approaches
to accomplishing "grade of service" and timing and synchronization
requirements.
Regarding GTE's technical proposal, the SSEB noted, as a strength,
GTE's offer to provide the existing, "state-of-the-art" DMS-100 switch
interconnected with a SONET transmission network. Disadvantages and
risks were also noted for GTE's proposal based upon GTE's failure to
completely document some elements of its proposed system, and GTE's
need for considerable software customization and integration to
support the network management system.
"Cost only" best and final offers (BAFO) were requested and evaluated
as follows:
Actual Life Cycle CostDiscounted Life Cycle Cost[9]
AT&T $291.29M[10] $197.19M
GTE $365.33M $243.57M
IGCE[11] $284.39M $187.55M
The SSEB prepared a report, detailing the advantages, disadvantages,
and risks posed by each offeror's proposal, and briefed the source
selection advisory council (SSAC) and SSA regarding the SSEB's
evaluation findings. The SSEB's technical and management evaluation
conclusions, including its determination that both proposals were
compliant with all RFP requirements, were accepted by the SSAC, which
then focused its assessment on the impact on the government of the
SSEB's findings. Tr. at 292-294, 330-331.
A BVWG, consisting of some members of the SSAC, was convened to
identify, analyze, and quantify significant differences between GTE's
and AT&T's technical and management proposals.[12] In general, the
BVWG concluded that the evaluated point difference between the two
firms' proposals reflected a lack of detail in AT&T's proposal as well
as greater risks of delay in the AT&T approach, primarily arising from
AT&T's need to perform greater software development and customization
than GTE.
Significant discriminators between the two firms' proposals were
identified. These discriminators were grouped into the following
categories--switched services, network management, network topology
and survivability, and key personnel/experience. For the areas in
which schedule or compliance risk was identified, the BVWG quantified
the estimated costs to the government of those risks posed in
additional government resources or to acquire the required services
due to any possible delay in performance. The net calculation of this
quantification was that AT&T's proposal posed an additional $1.1M of
risk to the government. In addition, the BVWG found that AT&T offered
slightly better, experienced key personnel than GTE, but that GTE
offered slightly better critically assured service[13] than AT&T;
these discriminators were not quantified. The SSAC prepared a report
detailing the BVWG's findings and unanimously concluded that AT&T's
substantially lower-priced proposal was a better value to the
government than GTE's slightly superior, but much higher-priced,
proposal.
On the basis of the SSEB's and SSAC's reports, as well as his
attendance at the SSEB's briefing to the SSAC, the SSA selected AT&T's
proposal for award. The SSA stated in his written source selection
decision:
"In this instance, both offerors have proposed acceptable network
architecture and network management plans. Both offerors propose
proven, state of the art equipment. Both offerors have provided
good management plans, with highly qualified personnel, and
generally have good to exceptional records of past performance.
Therefore, the award decision does not involve a serious question
of whether an offeror would provide the required services--both
offerors plainly would do so. The decision instead involves a
determination of which proposal is most advantageous to the
Government, consistent with the award criteria in the RFP.
"There are some significant differences in the technical and
management proposals of the two offerors. The above point scores
provide useful guidelines in distinguishing the differences
between the two proposals. However, the point differentials do
not necessarily accurately reflect the relative differences
between the two proposals. In particular, while I agree that GTE
generally has provided a better technical proposal than AT&T, I
do not think the difference between those two proposals is as
great as might be suggested by the difference in point score,
i.e., a 6.13 compared to 4.20. I base this conclusion in large
part on my reading of the SSAC report, as well as the BVWG/SSAC
discussions which I attended."
The SSA then discussed in detail each of the technical discriminators
and attendant potential costs, as identified by the BVWG. The SSA
found that the $1.1M probable additional expense to the government
associated with AT&T's proposal was "relatively trivial" in comparison
to enormous price disparity between the proposals, i.e., $46M in
discounted life cycle costs and $74M in actual life cycle costs. The
SSA also found that the risk attributed to AT&T's proposal may be
overstated because much of the risk was attributable to lack of detail
in AT&T's proposal and not equipment or network deficiencies, and
given AT&T's record of past performance, extensive past experience,
and substantial financial and personnel resources. Accordingly, the
SSA concluded:
"that the AT&T proposal is most advantageous to the Government.
The AT&T proposal provides a enormous cost savings to the
Government relative to the GTE proposal. Although the GTE
proposal does offer some technical advantages, and a lower risk
of delays in implementation, I do not believe that these
technical advantages outweigh the substantial cost premium. When
quantified, the technical advantages of the GTE proposal are
relatively insignificant compared to the disparity in price,
i.e., a net advantage to GTE of approximately $1.1 million versus
a $46 million advantage in AT&T's [discounted life cycle cost].
Even if the quantification were vastly understated, such as by a
factor of ten, the AT&T proposal nonetheless would be most
advantageous to the Government by a wide margin."
Award was made to AT&T. This protest followed.
GTE complains that AT&T's proposal is not compliant with a number of
mandatory solicitation requirements, such that AT&T's offer should not
have been found eligible for award. Specifically, GTE asserts that
AT&T's proposal is noncompliant with the ISDN, grade of service, and
the timing and synchronization requirements specified in the FRS.[14]
GTE also argues that AT&T's proposal is noncompliant because, contrary
to the RFP requirement that offerors describe their design and
technical approaches, AT&T did not sufficiently describe how it would
satisfy the RFP requirements regarding specified MUFs,[15] the
internal Stratum 3 clock, the Defense Switched Network Integrated
Management Support System (DIMSS) interface, critical assured service,
and the network management system. Finally, GTE argues that AT&T will
not meet the mandatory FOC schedule date, given the likely delay that
DISA evaluated in AT&T's proposal due to that firm's need to perform
substantial software development and customization to satisfy RFP
requirements to provide the MUFs, [DELETED], and the network
management system.
DISA and AT&T dispute each of these allegations and contend that
AT&T's proposal is fully compliant with all of the RFP's requirements.
They state that AT&T's proposal, like GTE's, was found compliant based
upon a commitment of the offeror to perform each RFP requirement and
the SSEB's finding that the proposed solution to accomplishing the
requirement was technically feasible. The agency and intervenor also
argue that the implementation and schedule risks, which GTE points to
in the evaluation record, do not demonstrate that AT&T's proposal is
noncompliant, but rather represent the agency's best value analysis of
AT&T's acceptable approach.
In considering protests of an agency's evaluation of proposals, we
examine the record to determine whether the agency's judgment was
rational and consistent with stated evaluation criteria and applicable
statutes and regulations. Abt Assocs., Inc., B-237060.2, Feb. 26,
1990, 90-1 CPD para. 223 at 4. Such judgments are by their nature often
subjective; nevertheless, the exercise of these judgments in the
evaluation of proposals must be reasonable and bear a rational
relationship to their announced criteria upon which competing offers
are to be selected. Southwest Marine, Inc.; American Sys. Eng'g
Corp., B-265865.3; B-265865.4, Jan. 23, 1996, 96-1 CPD para. 56 at 10.
From our review of the record, including the hearing testimony and the
parties' protest arguments, we conclude that the agency's evaluation
of proposals was reasonable.
First, the record does not support GTE's arguments that AT&T's
proposal was not compliant with the ISDN, grade of service, and timing
and synchronization requirements specified in the RFP.
ISDN is an international telecommunications standard for transmitting
voice, video, and data over digital lines, allowing the transmission
of digitized voice data and network management information
simultaneously. See Tr. at 196-197. The RFP ISDN requirement
specifies that the contractor provide basic rate and primary rate ISDN
services in accordance with Bellcore National ISDN standards (NI-1 and
NI-2), "except as otherwise specified by the HITS FRS." The RFP then
identified the 28 features of the National ISDN standards which
contractors must provide and required an additional feature,
MLPP[16]--a MUF--which is not included in the National ISDN standards.
Although software to comply with the National ISDN standards is
readily available, DISA recognized that offerors would be required to
offer customized software in order to provide the required MLPP
feature.
AT&T offered [DELETED].[17] The SSEB found that AT&T's [DELETED] ISDN
[DELETED] satisfied the RFP requirements. Tr. at 198-199. Offerors
were not required to provide all the National ISDN features, as GTE
suggests; rather the RFP only required offerors to provide the 28
National ISDN features identified in the RFP. Although GTE complains
that AT&T provided HITS users with [DELETED], GTE does not explain why
this renders AT&T's proposal noncompliant, given that AT&T's [DELETED]
satisfies the RFP requirements. Based on our review, we find
reasonable the agency's determination that AT&T's proposal satisfied
the ISDN requirement.
With regard to the next protested requirement, "grade of service"
refers to the RFP requirements for a switched traffic surge capacity
sufficient to handle up to 125 percent of the normal peak load
network-wide[18] and a switching processor capacity sufficient to
handle up to 200 percent of the switching system normal processor
load. GTE contends that the SSEB found that AT&T's proposed [DELETED]
switch and AT&T's switching system configuration cannot accommodate
the required surge capacity.
The record shows that the SSEB determined that AT&T's [DELETED]
switches and switching system configuration do both have sufficient
capacity to satisfy the RFP surge requirements. Tr. at 189-196.
While the SSEB was concerned that AT&T had not provided complete
details as to how it would configure its system to meet these
requirements, AT&T committed itself to providing services satisfying
the surge requirements, and the SSEB found that sufficient information
was provided by AT&T regarding its switches and architecture to allow
the evaluators to determine that the grade of service requirements
would be met. Tr. at 189. We find no basis in this record to
question this determination.[19]
With regard to the timing and synchronization requirements--which are
necessary to permit network switches to work as a synchronized
team--the RFP required that all gateway switches be externally timed
by reference to a Stratum 1 reference signal. AT&T proposed to
[DELETED]. Although the SSEB determined that this approach was
technically feasible by referring to technical literature for the
[DELETED] that would support this requirement, AT&T had not cited this
technical reference in its proposal to explain the feasibility of its
approach, even though AT&T had provided the technical literature with
its proposal. Tr. at 181. GTE, on the other hand, proposed a HITS
synchronization plan based upon provisioning a Building Integrated
Timing supply using the global positioning satellite system as its
primary reference source at all HITS locations; this, the SSEB found,
is a method commonly used by military communication sites in the
Pacific region. The SSEB concluded, reasonably we find, that both
AT&T's and GTE's proposals satisfied this requirement, but that GTE's
proposed approach was superior to AT&T's.[20]
GTE also contends that AT&T provided insufficient information in its
proposal to reasonably demonstrate compliance with the RFP
requirements for the specified MUFs, the DIMSS interface, the critical
assured service, the internal Stratum 3 clock, and the network
management system.[21] Specifically, GTE contends that both the
solicitation proposal preparation instructions and evaluation criteria
required offerors to describe and document their approaches to
satisfying the RFP requirements. GTE argues, citing the disadvantages
and risks sections of the SSEB report, that AT&T did not
satisfactorily demonstrate compliance with the above requirements.
Because the contracting agency is responsible for evaluating the data
submitted by an offeror and ascertaining whether it is sufficient to
determine the acceptability of the proposal, we will not disturb an
agency's determination in this regard unless it is shown to be
unreasonable. SAIC Computer Sys., B-258431.2, Mar. 13, 1995, 95-1 CPD para.
156 at 8; Inframetrics, Inc., B-257400, Sept. 30, 1994, 94-2 CPD para. 138
at 3.
Here, we find that DISA had a reasonable basis for its determination
that AT&T had provided sufficient proposal information to demonstrate
compliance with the RFP requirements. We do not agree with GTE that
the RFP required that proposals be found noncompliant for any
requirement where they failed to provide what the evaluators believed
to be a complete discussion. In this regard, although there were more
than 1300 technical requirements for which offerors had to show
compliance, the RFP limited technical proposals to 800 pages. Given
the complexity of HITS, the length and detail of the FRS, and this
proposal page limitation, we do not find it unreasonable to expect
that offerors would not fully discuss and describe their compliance
with each and every requirement, and that the agency would accept
statements of compliance with each offeror concerning some of the RFP
requirements.[22] See SAIC Computer Sys., supra at 8-10. In this
case, to determine compliance with the RFP's technical requirements,
the SSEB sought a commitment to comply with the specified requirement
and evidence of technical feasibility. Tr. at 53, 170, 215.
For example, in reviewing AT&T's proposal to determine whether AT&T
had satisfied the requirement to provide the MUFs--e.g., MLPP,
DSN7,[23] and PAT[24]--the SSEB found that AT&T had committed to
providing these features in a future switch software revision.
Specifically, the SSEB found AT&T was relying on the future release of
Lucent Technologies software release [DELETED][25] to provide a number
of required HITS features, including DSN7 and MLPP, for its [DELETED]
switches, and PAT for its [DELETED] switches.[26] Although the SSEB
reported that AT&T had not documented the [DELETED] software in its
proposal, the technical evaluation team chair explained in his hearing
testimony that this meant that the unreleased software had yet to be
tested and certified as to its ability to provide these features. Tr.
at 47-48. The SSEB determined compliance in accordance with the above
standard when it found that AT&T had committed to providing these
features, which its proposal said would be provided through the future
software release; that Lucent Technologies had committed, in a letter
included in AT&T's management proposal, to providing these features in
the [DELETED] software release; and that AT&T had successfully
provided other 5ESS switches in other military networks with MLPP,
PAT, and CCS7 (the commercial channel signaling protocol), which
indicated to the evaluators that AT&T was capable of providing the
MUFs here. See Tr. at 47-48, 55, 70, 234.
We note that GTE's proposal was found to be compliant, although it
also was relying upon "undocumented," future switch software releases
to provide HITS features. While, like AT&T's proposed future software
releases, these future software releases were not documented or
completely described to the evaluators' satisfaction, the SSEB
similarly found GTE's proposal compliant (albeit representing a lower
risk than AT&T's proposal), based upon GTE's commitment to provide the
features and the evaluators' determination that GTE's solution was
feasible. See Tr. at 56, 239-47.
GTE and AT&T were also treated similarly with regard to the agency's
determination that both offerors proposed sufficient information to
demonstrate compliance with the network management system
requirements.[27] Both offerors proposed a commercial off-the-shelf
(COTS) software solution to providing the HITS network management
system features, and the SSEB found that both offerors would require
extensive customization and integration of their software before
implementation. In this regard, both offerors' network management
system solutions were assessed to be high risk. Thus, the SSEB found,
regarding AT&T's proposal, that:
"[AT&T's] proposed HITS network management solution is based on
proved network management concepts and provides the comprehensive
functionality needed to manage and operate the HITS network and
services.
"The proposed [network management system] will employ high
quality, highly capable, state-of-the-art, standards-based, COTS
hardware and software.
"Based on the disparities, inconsistencies, and uncertainties in
[AT&T's] proposal, proof that integration of the proposed
[network management] tools and COTS application software to
substantiate [AT&T's] statement of compliance has not been
demonstrated or provided."
Regarding GTE's proposed network management system solution, the SSEB
found:
"While the proposed [network management system] design appears to
be technically sound and shows a good understanding of specified
HITS network management requirements, the risk associated with
implementing a fully compliant HITS [network management system]
in time to meet specified milestones is high due to the apparent
need to extensively customize and integrate the proposed COTS
network management application software. Although stating
compliance that network management of both switching and
transmission network elements will be provided, this risk
assessment is also based on the lack of detailed information in
the proposal regarding the design, development, implementation
and operation of network management functions for transmission
network elements."
In sum, the SSEB had concerns with both offerors' proposed network
management system solutions, see Tr. at 93-94, 254-257, but
nonetheless found the offerors' proposals compliant with regard to
these requirements. The SSEB found that AT&T's specific proposal
commitment, coupled with the COTS hardware and software proposed and
the evaluators' knowledge that the proposed hardware and software
could meet the agency's needs, was sufficient to demonstrate AT&T's
compliance with this requirement.
The SSEB also found that AT&T provided sufficient information in its
proposal to demonstrate compliance with the RFP requirement that
"[a]ll switches have an internal reference or system clock with a
frequency and stability equal to a least a Stratum 3. . . ." Although
AT&T committed itself to provide the required Stratum 3 internal clock
for all switches, [DELETED]. Because AT&T did not specifically state
that it was replacing the [DELETED] with a Stratum 3 clock, the SSEB
assessed this aspect of AT&T's proposal as a risk. Nevertheless, the
agency found that the switch could be equipped with a Stratum 3 clock
as specifically promised by AT&T. In this regard, GTE does not assert
that these switches cannot be provided with a Stratum 3 clock;
[DELETED].
AT&T's proposal was also found to be compliant with the critical
assured service requirement of the RFP, despite the SSEB's concerns
with the lack of detail provided in the proposal, because AT&T
described an approach that the SSEB found would meet this requirement.
The agency also found that critical assured service is a routine
feature of military communications networks and would not be difficult
for experienced vendors, such as GTE and AT&T. We find no basis to
question this aspect of DISA's evaluation of AT&T's proposal.
In sum, as noted by GTE, the SSEB found considerable risk with various
aspects of AT&T's proposal because of AT&T's limited description of
how it would accomplish the foregoing (and other) requirements and
because AT&T required software development or customization to meet
them.[28] However, this does not mean that AT&T was not compliant
with these requirements; to the contrary, this evidences the
reasonableness of the agency evaluation--that is, DISA recognized that
AT&T had offered a compliant solution but one that entailed risk to
the government. See TEAC Am. Corp., Inc., B-259831 et al., May 3,
1995, 95-1 CPD para. 273 at 11.
GTE also complains that AT&T will not comply with the mandatory
schedule requirements. Specifically, GTE argues that DISA's
evaluation record establishes that it is probable, if not likely, that
AT&T will not be able to satisfy the RFP's FOC date, inasmuch as the
BVWG determined that AT&T would likely miss the IOC date. DISA and
AT&T respond that AT&T took no exception to the schedule requirements
in its proposal and therefore is compliant with the IOC and FOC
requirements, and that GTE has misconstrued the purpose and
conclusions of the BVWG's risk quantification analysis.
The record shows that the BVWG accepted the SSEB's assessment that
AT&T's proposal complied with the RFP requirements as well as the
SSEB's determination that AT&T's approach entailed a moderate to high
risk to the government that the IOC and FOC dates would not be met;
the BVWG quantified the potential for delay that arose from each of
the areas of AT&T's proposal for which the SSEB assessed moderate or
high risk. Tr. at 292-294, 330-331. The BVWG found that a
significant potential for delay existed for AT&T meeting the IOC date
because AT&T needed switch software revisions to satisfy various HITS
requirements, such as providing MUFs on all switches and [DELETED];
customizing and integrating the COTS network management system
software; and configuring AT&T's switches to satisfy grade of service
requirements. The BVWG found that it was probable that AT&T's
software revisions could result in a delay of [DELETED] after the IOC
date and that AT&T's customization of the network management software
could result in up to a [DELETED] delay in satisfying the IOC
requirement. The BVWG did not find that AT&T would not meet the FOC
date or calculate any risks of AT&T's proposal not meeting the FOC
date, see Tr. at 343-344, although the BVWG found it was likely that
there may be grade of service problems during contract performance
with AT&T's solution, and that, given the minimal impact on the user,
it may take between [DELETED] after the FOC date to detect and correct
any grade of service problem. Tr. at 359, 445-446. The BVWG chair
testified, and the record otherwise evidences, that the BVWG's
calculations of probable delay were prepared as a means of quantifying
the risk assessed in AT&T's proposal and were not intended to
represent "a forecast of [a] future event actually occurring or not."
Tr. at 301-302; see Tr. at 324-325, 405-406, 409.
The BVWG's risk analysis, including estimates of probable delay, was
fully presented to the SSA, who determined that AT&T would, in fact,
comply with the RFP's required schedule and that, in this regard, the
BVWG's estimates of risk and delay were too pessimistic. The SSA
testified that, based upon his more than 20 years of software
development experience, he believed that AT&T would satisfy the RFP's
schedule requirements, notwithstanding the significant software
development and customization that had to be performed by that firm.
Tr. at 548. Specifically, the SSA testified that in his opinion the
software development required would not be so difficult as to cause
AT&T to miss the FOC date. Regarding the customization of the network
management system software, he stated that "[t]his network stuff is
done every day. We run millions of lines of this code around the
country." Tr. at 548-49. Regarding the switch software, the SSA was
aware that the features to be provided, although not under the DSN7
protocol, were already available and should not be difficult to
develop for this contract. Tr. at 570, 573. The SSA also concluded
that he had two "very good companies"--AT&T and Lucent
Technologies--doing the software revisions and software customization,
and that this would mitigate any potential for delay, particularly
given AT&T's financial incentive to satisfy the schedule requirements;
that is, AT&T would not be paid until it provided service in
accordance with the HITS contract. Tr. at 570; see Tr. at 548-549.
The record also shows that the BVWG's determination of the length of
AT&T's likely delay was based upon that group's subjective judgment
and not upon any calculation of the level of effort actually required
to perform this software revision and customization. Tr. at 340, 361,
384-85. In addition, the BVWG's assessment of potential delay
associated with AT&T's need to provide the [DELETED] software revision
was based in part upon the SSEB's and BVWG's misunderstanding of when
the relevant software release would occur. During the protest
process, AT&T provided the affidavit of Lucent Technologies [DELETED]
for the 5ESS-2000 switch, who states the MUFs required under DSN7
would be supplied by software release [DELETED]. In its proposal,
AT&T had incorrectly identified the software to provide the MUF
features as software release [DELETED], rather than [DELETED].
[DELETED], DISA incorrectly believed that this represented a delay in
the anticipated release of the [DELETED] software, which was planned
for late 1996. Tr. at 231-234, 330. In fact, software release
[DELETED] (which does not include MUFs for this contract) was released
on schedule in December 1996. The SSEB chair testified that this
misunderstanding affected the evaluators' assessment of the risk
associated with AT&T's need for switch software revisions. Tr. at
232. Moreover, [DELETED].[29] Tr. at 351.
We find no basis to object to the SSA's judgment that AT&T would
satisfy the RFP schedule requirements. Contrary to GTE's argument,
DISA's lower-level evaluators did not determine that AT&T would not
satisfy the contract schedule requirements, but found that AT&T's
approach entailed risks that it would not meet the schedule. Indeed,
the record establishes that the BVWG's estimates of delay were not
intended to forecast AT&T's noncompliance with the schedule
requirements, but to quantify possible risk to the government, should
delays occur. In any event, source selection officials are not bound
by recommendations or evaluation judgments of lower-level evaluators
but may make their own judgments, which are subject to the tests of
rationality and consistency with the stated evaluation criteria. See
PRC, Inc., B-274698.2; B-274698.3, Jan. 23, 1997, 97-1 CPD para. 115 at 7.
Here, GTE has not shown that the SSA's judgment that AT&T would meet
the RFP schedule requirements was unreasonable.[30]
GTE also complains that the agency's best value analysis was
unreasonable because DISA did not assign any value to GTE's technical
superiority and because DISA underestimated the cost to the government
of the risk in AT&T's proposal. In this regard, GTE asserts that DISA
effectively converted this procurement from a best value basis for
award to one in which award was simply made to the offeror with the
lowest-priced, technically acceptable proposal.
Source selection officials have broad discretion to determine the
manner and extent to which they will make use of the technical and
cost evaluation results in a negotiated procurement. Grey
Advertising, Inc., 55 Comp. Gen. 1111, 1118-21 (1976), 76-1 CPD para. 325
at 9-12. In deciding between competing proposals, cost/technical
tradeoffs may be made, the propriety of which turns not on the
difference in technical scores or ratings per se, but on whether the
source selection officials judgment concerning the significance of the
difference was reasonable and adequately justified in light of the RFP
evaluation scheme. Southwest Marine, Inc.; American Sys. Eng'g Corp.,
supra at 17; DynCorp, B-245289.3, July 30, 1992, 93-1 CPD para. 69 at 8.
The record contains detailed documentation that establishes that the
BVWG and SSA performed a thorough and complete best value analysis of
the two firms' proposals and gave GTE's proposal appropriate credit
for its assessed technical superiority. As indicated above, the BVWG
identified the significant discriminators between the two firms'
proposals, where GTE was found to offer a superior solution or where
AT&T's proposal was found to pose greater risk than GTE's. Each of
the discriminators identified by DISA's evaluators was reviewed by the
BVWG and SSA to assess the impact on the user.[31] Tr. at 296.
In making this assessment, the BVWG and SSA recognized that, given the
nature of this contract as a fixed-price service contract with tight
performance specifications, the level and quality of service received
from a compliant vendor would not significantly vary; as stated by the
agency, the services were basically "transparent to the user." Tr. at
260-262, 336-338, 488-489. In this regard, the SSA noted in his
source selection decision [DELETED]--dedicated transmission and
transmission-dependent switched voice services--that "[t]ransmission
essentially is a commodity, which reputable vendors provide with
little or no difference in quality." The RFP's extensive
pre-acceptance testing procedure also provided confidence to the BVWG
and SSA that any potential problems in AT&T's performance that would
affect the level of service provided would be identified and corrected
before implementation of the system. Tr. at 296-297, 520. We cannot
say from our review of the record that the BVWG's and SSA's judgment
in this regard was unreasonable.
Accordingly, DISA evaluated the value of what it considered to be the
real benefits of GTE's higher-rated proposal to ascertain whether they
justified the payment of the associated price premium. In performing
this evaluation, the BVWG and SSA assessed the impact of the evaluated
discriminators by focusing upon the cost to the government of
acquiring services from other sources during any period of delay or of
providing additional government oversight and monitoring, Tr. at 414,
and the cost to the government of directing AT&T to replace a
compliant solution with a more expensive solution. Tr. at 391.
For example, the BVWG and SSA, in reviewing the discriminator based
upon AT&T's reliance upon the future [DELETED] software release,
concluded that the risk of additional cost to the government for
oversight/monitoring and acquisition of services elsewhere could be
approximately $[DELETED]. Regarding the network management system
software, AT&T was found to pose greater risk than GTE to perform
necessary customization and integration of COTS software, because AT&T
had failed to completely describe how its various software
applications would interface; the BVWG and SSA assessed the potential
cost of this to the government to be approximately $[DELETED] for
additional oversight/monitoring. Regarding timing and
synchronization, the BVWG and SSA found that GTE had proposed a more
reliable and robust method of ensuring timing and synchronization than
had AT&T, although AT&T's solution was compliant; to assess the impact
of this discriminator, the BVWG and SSA determined that approximately
$[DELETED] would be the additional cost to the government of directing
AT&T to replace its compliant solution with GTE's superior
solution.[32]
GTE's objection to DISA's quantification of the risk in AT&T's
proposal is based upon its view that there will be a diminution in the
level of service received by the government from AT&T. The protester
calculates the value of this diminution of service by reference to the
system of "outage credits" provided by the RFP where a contractor
fails to meet specified grade of service requirements.[33] This
argument, however, fails to account for the agency's assessment--which
we have found reasonable--that there will be no diminution in the
level of service received from AT&T, and for this reason provides us
with no basis to object to DISA's quantification of the discriminators
between the two firms' proposals.
The record simply does not support GTE's contention that award was
based on the lowest-priced, technically acceptable proposal, but
evidences that a best value award decision was reasonably made in
accord with the RFP evaluation scheme. In this regard, the record
shows that SSA accepted the findings of the lower level evaluators
that GTE's technical proposal was superior to AT&T's; he also accepted
that AT&T's management proposal was superior to GTE's. The record
establishes that the SSA went beyond the evaluation ratings to assess
the impact of the difference between the two firms' proposals. The
SSA recognized that the essential technical difference between AT&T's
and GTE's proposal was that AT&T's proposal posed a greater risk of
delay and of a need for additional government oversight than did
GTE's, Tr. at 563, but that the service the customer would ultimately
receive from either vendor would be essentially the same. Tr. at 487.
Thus, it was AT&T's greater risk, as quantified by the BVWG, that the
SSA weighed against GTE's substantial price premium (even though the
SSA believed that this risk had been exaggerated by the lower-level
evaluators). The SSA also gave appropriate weight to AT&T's superior
management proposal rating and good past performance record, which the
SSA found mitigated the risks in AT&T's proposal. Tr. at 557.
Furthermore, the SSA was aware that telecommunications services were
available under the FITS contract, at lower rates than offered by AT&T
or GTE, for eleven months after the FOC date; this "safety net" also
mitigated the risk of delay in AT&T's proposal. Tr. at 482, 499-501.
In sum, the SSA found that the evaluated risks in AT&T's fixed-price
proposal were worth the substantial price advantage. Contrary to
GTE's arguments, we find this is an appropriate best value analysis,
which is both reasonable and consistent with the stated evaluation
criteria.
The protest is denied.
Comptroller General
of the United States
1. Although these contracts have expired, DISA has been acquiring
telecommunication services under the Follow-on Interim Telephone
System (FITS) contract, which was awarded to GTE on a sole source
basis and can be extended through July 1999.
2. Subcriteria were also identified for the technical quality and
management quality criteria.
3. GTE also submitted an alternate proposal that offered to continue
use of the older switches that had supported service under the Oahu
Telephone System contract. This proposal was rejected because it did
not comply with several critical HITS requirements, posed significant
scheduling risk, and would provide inferior service relative to that
offered by the other proposals.
4. A hearing was conducted to elicit the testimony of the chair of the
SSEB's technical team, the chair and other members of the best value
working group (BVWG), and the source selection authority (SSA).
5. Under DISA's scoring methodology, a score of 4 reflected an
acceptable proposal--"[t]he offeror's proposal meets requirements in
an acceptable manner"; a 5 to 8 score reflected a good
proposal--"[t]he offeror's proposal exceeds minimal requirements in
some meaningful aspects and should result in successful completion of
the requirement."
6. Low risk was defined as having little potential to cause disruption
of schedule, increase in cost, or degradation of performance.
Moderate risk was defined as having some potential to cause disruption
of schedule, increase in cost, or degradation of performance, but
where special contractor emphasis and close government monitoring
could probably overcome difficulties. High risk was defined as likely
to cause significant serious disruption of schedule, increase in cost,
or degradation of performance, even with special contractor emphasis
and close government monitoring.
7. AT&T proposed 12 5ESS-2000 switches in the following
configurations: [DELETED].
8. SONET is an optical transmission network. Tr. at 90.
9. The discounted price was calculated by using a discount rate of 7.9
percent and mid-year discounting factors derived from Office of
Management and Budget Circular A-94.
10."M" equals a million.
11."IGCE" refers to the independent government cost estimate.
12. The SSAC and BVWG were not aware of GTE's and AT&T's proposed
prices until after the completion of the best value analysis. Tr. at
293.
13."Critical assured service" is a sub-element of the topology and
survivability discriminator category. It refers to the requirement
that offerors provide certain designated critical switched voice users
assured service "to guarantee these subscribers will be able to
initiate and receive calls even when the subtending central office or
access line is unable to process calls."
14. GTE originally argued that AT&T was not compliant with a number of
other RFP requirements, such as the provision of specified military
unique features (MUF) and an internal Stratum 3 clock. After review
of the agency's report, GTE contended only that AT&T had provided
insufficient information in its proposal to demonstrate compliance
with these requirements.
15. The MUFs required by the RFP include multilevel precedence and
preemption (MLPP), Defense Switched Network (DSN) 7 Common Channel
Signaling (CCS), and Precedence Access Threshold (PAT).
16. MLPP provides specified users with the capability to get assured
service. Tr. at 43.
17. Both AT&T and GTE proposed to satisfy some of the 28 required
National ISDN features with functional equivalents. DISA treated GTE
and AT&T similarly in finding that their proposals satisfied the ISDN
requirement. GTE does not challenge this aspect of DISA's evaluation.
18."Normal peak load" was defined as five call attempts per line per
hour.
19. We note that AT&T's proposal's lack of detail concerning how its
network design would satisfy this requirement was accounted for by the
SSEB in its assessment of this aspect of AT&T's proposal as a moderate
risk. GTE's proposal was treated similarly in this regard; although
GTE's proposal was found compliant with this requirement, the SSEB was
similarly concerned that GTE had not provided complete enough detail
concerning its network to allow the evaluators to verify that GTE's
network would satisfy surge requirements. Tr. at 251-253.
20. The difference in GTE's and AT&T's approaches was accounted for by
DISA in its evaluation by assessing this to be a significant
discriminator; in fact, AT&T's proposal was rated high risk for this
requirement because of AT&T's vague explanation of its timing and
synchronization plan, while GTE's proposal was assessed as low risk.
21. GTE does not contend that AT&T will be incapable of providing
these features under the HITS contract.
22. In fact, during discussions DISA informed the offerors that their
"failure to fully substantiate compliance may result in a lower
evaluation score and/or higher risk assessment," which indicated to
the offerors that any lack of detail could be addressed in the
agency's comparative evaluation of proposals.
23. DSN7 is the military version of the commercial common channel
signaling requirement. Tr. at 42-43.
24. PAT is the tool used by the switching element to allow
implementation of MLPP. Tr. at 42-43.
25. The record shows that operating system and core
applications--designated as 5E software--are the same for all the
5ESS-2000 switches regardless of configuration. Each software release
is denominated as 5E with a revision number; thus, for example,
[DELETED]. Although not explained in AT&T's proposal nor recognized
by the evaluators, the actual software release that will provide the
MUFs for the HITS is [DELETED].
26. AT&T is also relying upon future software releases--specifically,
[DELETED]--to satisfy the [DELETED] requirements.
27. The HITS network management system will provide the government
with access and control capability of all the HITS resources. Tr. at
89.
28. The risk to the government was fully identified and described in
the SSEB and SSAC reports that were provided to and reviewed by the
SSA.
29.[DELETED].
30. GTE also argues that DISA treated offerors unequally when it
rejected GTE's alternate proposal allegedly because of the risk that
GTE could not satisfy the schedule requirements, while accepting
AT&T's proposal evidencing similar schedule risk. The record
establishes, however, that DISA rejected GTE's alternate proposal
because it was based upon outdated technology, would not meet numerous
requirements even after GTE upgraded its proposed system, and was
markedly inferior to AT&T's proposal as well as to GTE's other
proposal.
31. We note that in objecting to the BVWG's analysis, GTE does not
assert that the discriminators that were identified are not
appropriate or that there were additional technical discriminators
that should have been considered; rather, GTE's complaint is that it
did not receive credit for its higher technical score and that the
cost of AT&T's potential for failure was underestimated.
32. The agency also recognized that GTE had proposed a [DELETED]
transmission backbone than did AT&T. Specifically, GTE proposed to
provide an [DELETED] ("OC" refers to an optical carrier--i.e., a fiber
optic transmission channel) to most HITS locations, while AT&T offered
to provide [DELETED], if later required. DISA determined that the
benefit of [DELETED] was minimal, however, because the agency
[DELETED]. The BVWG quantified this difference between the proposals
to be approximately $[DELETED], representing [DELETED]. GTE has not
shown this determination to be unreasonable.
33. The RFP provides for payments by the contractor to the government
during periods of severe service interruption.