BNUMBER: B-274246.10
DATE: September 17, 1998
TITLE: Symetrics Industries, Inc., B-274246.10, September 17, 1998
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective
Order. This redacted version has been approved for public release.
Matter of:Symetrics Industries, Inc.
File: B-274246.10
Date:September 17, 1998
Edward J. Kinberg, Esq., for the protester.
Sophia L. Rafatjah, Esq., Tracor, Inc., and Robert T. Ebert, Esq., and
Paul Shnitzer, Esq., Crowell & Moring, for Tracor Aerospace, Inc., the
intervenor.
Jo Ann Stringfield, Esq., and Gregory H. Petkoff, Esq., Department of
the Air Force, for the agency.
Ralph O. White, Esq., and Christine S. Melody, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Contention that agency unreasonably evaluated protester's
technical proposal, and improperly concluded that the awardee's
higher-rated, higher-priced proposal offered the best value to the
government, is denied where the record shows that the agency
evaluation was reasonable, and in accordance with the stated
evaluation criteria.
2. Protester's assertion that the agency failed to hold meaningful
discussions regarding two areas where its proposal was assessed
moderate risk is denied where the record shows that in both instances
the agency clearly advised the protester of the factual underpinning
of its concerns.
DECISION
Symetrics Industries, Inc. protests the award of a contract to Tracor
Aerospace, Inc. by the Department of the Air Force, pursuant to
request for proposals (RFP) No. F33657-96-R-0001, issued for the
purchase of Lots V through VII of the AN/ALE-47 Countermeasures
Dispenser System (CMDS). Symetrics argues that the evaluation of
proposals was unreasonable, that discussions were improper, and that
the selection decision lacked a reasonable basis.
We deny the protest.
BACKGROUND
This protest challenges the reselection of Tracor for award by the Air
Force following an earlier decision by our Office sustaining
Symetrics's protest of the initial selection of Tracor. The earlier
protest was sustained because the record showed there had been a
material change in the agency's requirements after the solicitation
was issued and before the award date, and because the change in
requirements may have significantly affected the evaluation.
Symetrics Indus., Inc., B-274246.3 et al., Aug. 20, 1997, 97-2 CPD para.
59 at 8. In response, the Air Force amended its solicitation,
requested a new round of proposals, held discussions with the three
offerors still participating in the competition, and reevaluated best
and final offers (BAFO).
As explained in our earlier decision, the AN/ALE-47 CMDS is an
electronic warfare system used by the Army, Navy and Air Force to
protect aircraft from hostile missile attacks. The system discharges
chaff cartridges and decoy flares to distract ground-launched missiles
aimed at aircraft. Five distinct line replaceable units (LRU)
comprise the system in varying numbers and configurations depending on
the aircraft involved. These LRUs are a control-display unit, a
programmer, a switch assembly, a digital sequencer, and a dispenser
assembly.
The RFP here, first issued in October 1996, contemplated the award of
a 4-year fixed-price, indefinite-delivery/indefinite-quantity (ID/IQ)
contract to the offeror whose proposal provided the best value to the
government.[1] The RFP explained that the proposal with the best
value would be "the most advantageous offer, price and other factors
considered . . . providing the best mix of utility, technical quality,
business aspects, risks, and price for a given application." RFP sec.
M.1.0.
The RFP advised that each proposal would be evaluated in four areas,
in descending order of importance: technical, schedule,
cost/price--most probable life cycle cost, and management. Under
these four factors, the technical factor included four subfactors of
equal weight: manufacturing/quality assurance, integrated logistics
support, systems engineering, and testing. In addition to the four
factors, the RFP also listed four general considerations of equal
weight, all of which were less important than the four evaluation
factors. The general considerations were pre-award survey, executive
in-plant review, plant visits, and RFP terms and conditions. RFP sec.
M.2.0.
Offerors were also advised that two of the evaluation
factors--technical (including the four subfactors) and
management--would be assigned a color/adjectival rating, a proposal
risk assessment, and a performance risk assessment.[2] They were also
advised that the color/adjectival rating and the two risk ratings
would receive equal consideration. RFP sec. M.6.0. The remaining two
evaluation factors--schedule and cost/price--were not rated but were
assigned a performance risk assessment.
After the evaluation of BAFOs in the most recent round of this
procurement, Tracor's proposal received two blue ratings (exceptional)
and two green ratings (acceptable) under the four equally-weighted
technical subfactors, a green rating under the management factor, and
low risk ratings in every assessment of proposal and performance risk.
However, Tracor's proposal had the highest price of any offeror, $19.8
million with a most probable life-cycle cost (MPLCC) of $78.2 million.
Symetrics's proposal received all green ratings under the technical
and management factors. While Symetrics received low risk ratings in
all of the assessments of performance risk, it received two moderate
risk ratings and three low risk ratings under the assessments of
proposal risk. Specifically, Symetrics received a moderate risk
rating under the technical subfactor of manufacturing/quality
assurance, and under the management evaluation factor. Symetrics's
proposal offered the lowest price of the three remaining competitive
range offerors, $18.7 million with a MPLCC of $77.4 million. A
summary of these results is shown in the table below:
TRACOR SYMETRICS
Color
Rating Prop
Risk Perf
Risk Color
Rating Prop
Risk Perf
Risk
Technical
--Mfg./QA Blue Low Low Green Mod. Low
--Int. Logistics SupportBlueLowLowGreenLowLow
--Systems Eng'gGreenLow Low Green Low Low
--Testing Green Low Low Green Low Low
Schedule ----- -----Low ----- -----Low
Price (MPLCC) $78.2M -----Low $77.4M -----Low
Management Green Low Low Green Mod. Low
In selecting Tracor over Symetrics, the source selection authority
(SSA) noted that Tracor had a higher price and MPLCC, but concluded
that the price difference was insignificant given Tracor's superior
manufacturing processes and integrated logistics support capability,
reflected in its blue ratings under the manufacturing/quality
assurance and the integrated logistics technical subfactors. In
addition, the SSA expressed concern that Symetrics might not
effectively manage this program because of a recent change in
Symetrics's ownership and corporate structure. Accordingly, the SSA
concluded that Tracor's proposal offered the best value to the
government and awarded to Tracor.[3]
DISCUSSION
Symetrics disputes each evaluation conclusion that provides a basis
for distinguishing between its proposal and Tracor's. Specifically,
Symetrics argues that there was no reasonable basis for awarding
Tracor a higher rating under the manufacturing/quality assurance and
the integrated logistics technical subfactors, and no reasonable basis
for concluding that Symetrics's proposal presented a moderate level of
risk under the manufacturing/quality assurance technical subfactor,
and under the management factor.
In considering protests against an agency's evaluation of proposals,
we will examine the record to determine whether the agency's judgment
was reasonable and consistent with the stated evaluation criteria and
applicable statutes and regulations. ESCO, Inc., B-225565, Apr. 29,
1987, 87-1 CPD para. 450 at 7. Here, we have considered each of
Symetrics's contentions, the evaluation materials, the proposals, and
the Air Force's response to Symetrics's arguments. As a result of our
review, we find no basis for concluding that the evaluation was
unreasonable, or not in accordance with the stated evaluation
criteria. To illustrate our conclusion, we will discuss in detail a
sampling of Symetrics's arguments.
With respect to the distinction drawn between Tracor and Symetrics
under the manufacturing/quality assurance subfactor, Tracor received a
rating of blue (exceptional) after the evaluators identified several
significant strengths in this area of its proposal. Specifically, the
evaluation shows that Tracor's proposal indicates "[e]xceptional work
measurement and quality assurance programs," "[a]ll known production,
schedule and [quality assurance] risks have been mitigated," and
"[s]killed, experienced and stable workforce in place." Under these
three headings were other more detailed evaluation comments. The
evaluators identified no significant weaknesses in the proposal, and
no proposal risk. Addendum to Proposal Analysis Report, May 12, 1998,
at 16.
Symetrics received a rating of green (acceptable) after the evaluators
identified one significant strength, no significant weaknesses, and
moderate proposal risk. Symetrics's significant strength was its
"[s]trong understanding of work measurement," but the company was
assessed a moderate proposal risk because the proposal's "labor hours
appear to be low for a first time producer of a system of this
complexity." Id. at 14.
The SSA made the following comments when comparing the offerors under
this technical subfactor:
Tracor's rating was based, in part, upon Tracor's exceptional
work measurement and quality assurance programs. Tracor's "6
Sigma" statistical process control minimizes manufacturing
defects as demonstrated by the [deleted] success rate for circuit
card tests currently being obtained on the AN/ALE-47 production.
Tracor and [Offeror A] received a low proposal risk rating for
this factor. Symetrics received a moderate proposal risk rating
for this factor based upon their unrealistically low labor hours
in their cost proposal for a first time producer of a system of
this complexity. The hours in the cost proposal were lower than
the hours detailed in their technical proposal which the
evaluation team believed to be more reasonable. During the
course of discussion over this difference, Symetrics revised
their technical proposal to match the lower hours in the cost
proposal; however, they did not offer an adequate justification
for the difference between the two portions of their proposal.
Source Selection Decision, May 14, 1998, at 2-3.
Symetrics argues that it was unreasonable for the agency to give
additional credit to Tracor for its Six Sigma Program, a quality
assurance process, because the program is not as highly regarded as
the Air Force believes; because the Air Force mistakenly concluded
that Tracor's performance under the Six Sigma Program was exemplary;
and because giving credit to Tracor for this expensive quality program
constructively eliminates small businesses from the competition in
violation of congressional direction that a full and open competition
be held for this system. Symetrics also argues that the Air Force
improperly overlooked favorable past performance information for
Symetrics contained in performance reports from 1994, 1995, 1996, and
1997 (among other places). In addition, Symetrics argues that, under
the guidelines in the Air Force's own evaluation plan, Symetrics
should have received a rating of blue, not green, under this factor.
As an initial matter, magazine, newspaper, and on-line articles
submitted by both the Air Force and Symetrics show that the Six Sigma
Program--for which Tracor's proposal received credit--is a quality
assurance process that has received considerable favorable review in
the mainstream and manufacturing press. While the articles overall
indicate that the Six Sigma Program can lead to significant
improvements in manufacturing quality, Symetrics offers two articles
that question whether the program is worth its cost, or is merely the
latest in a series of industry fads related to improving quality.
We need not reach a conclusion about whether the Air Force or
Symetrics is correct about the merits of the Six Sigma Program.[4]
The record amply demonstrates that the program is a mainstream effort
to improve manufacturing quality and that a number of major
corporations have implemented portions of the program. In our view,
it was reasonable for the Air Force evaluators to recognize and value
Tracor's attempt to implement this kind of quality improvement
program. In addition, we do not view the evaluation's recognition of
Tracor's Six Sigma Program as a constructive elimination of small
businesses from consideration here. While Symetrics may be correct in
its contention that the cost of implementing the program is generally
beyond the reach of small businesses, the solicitation here did not
require the program. Instead, the evaluators viewed the program as a
benefit of Tracor's proposal, and included the program in the list of
discriminators when deciding which proposal represented the best value
to the government. We see nothing improper about the evaluation in
this regard.
Symetrics also argues that the Air Force's award of a blue rating to
Tracor under this factor was unreasonable because Tracor's
achievements under its Six Sigma Program were overstated.
Specifically, Symetrics argues that the Air Force was wrong when it
stated that "Tracor is currently operating at an average of [deleted]
in the six sigma program which puts their manufacturing facility in
line with the top facilities in the world." Addendum to Proposal
Analysis Report, supra, at 17. Symetrics points to one of the
articles on the Six Sigma Program reprinted in the agency report and
urges that under the standards set forth there Tracor's [deleted]
rating puts the company "at the top end of the industry average."
Symetrics's Comments on the Agency Report, July 31, 1998 at 15. Thus,
in Symetrics's view the evaluation conclusion was unreasonable.
In our view, even if we accept Symetrics's characterization of
Tracor's [deleted] quality rating within the Six Sigma Program, the
distinction between the Air Force's comments that Tracor's facility is
"in line with the top facilities in the world" and Symetrics's
comments that Tracor is "at the top end of the industry average" does
not compel us to conclude that the evaluation was unreasonable and
should be overturned. The minor difference between these two
assessments is well within the realm of the agency's discretion to
make reasonable assessments of proposals. See ESCO, Inc., supra.
We also reject Symetrics's contention that the distinction drawn
between its proposal and the proposal submitted by Tracor was
unreasonable because past performance reports for Symetrics show very
high quality ratings in its manufacture of a related system. As shown
above, in the agency's assessment of past performance, both Symetrics
and Tracor received ratings of low performance risk. However, the
proposal ratings were not based on past performance. Instead, the RFP
at section M.3.1.1 explained that the agency would evaluate the
offeror's manufacturing management program and quality system
approaches. The evaluators concluded that Symetrics had a strong
understanding of the work measurement requirements of the
solicitation, and awarded the company a rating of green (acceptable).
Favorable reports about Symetrics's past performance do not establish
that this rating was unreasonably low.
As a final evaluation matter, Symetrics contends that the Air Force's
own evaluation plan requires that Symetrics receive a rating of blue,
not green, under the manufacturing/quality assurance subfactor.
Specifically, Symetrics points out that the Air Force defined the
rating of blue (exceptional) as "[e]xceeds specified performance or
capability in a beneficial way to the Air Force and has no significant
weakness."[5] According to Symetrics, since the definition of a blue
rating uses the word "way" in the singular, and since the evaluators
noted a strength under this subfactor (with no significant weakness),
the agency's own evaluation scheme mandates a rating of blue
(exceptional).
As a preliminary matter, we note that the standardized color ratings
used by the Air Force here are set forth in the Air Force Federal
Acquisition Regulation Supplement (AFFARS), Appendix BB, sec. BB-304.
These rating definitions reflect internal instructions to agency
personnel and do not provide outside parties with legal rights.
Rockwell Int'l Corp., B-261953.2, B-261953.6, Nov. 22, 1995, 96-1 CPD para.
34 at 13 n.16. Instead, our review focuses not on the Air Force's
internal definitions but on whether the evaluations at issue are
consistent with the RFP's evaluation scheme. Loral Aeronutronic,
B-259857.2, B-259858.2, July 5, 1995, 95-2 CPD para. 213 at 9-10;
Sabreliner Corp., B-242023, B-242023.2, Mar. 25, 1991, 91-1 CPD para. 326
at 12 n.16.
Here, section M.3.1.1 of the RFP advised offerors that the agency
would consider numerous elements of the proposal's manufacturing
management program and its quality system. We have reviewed Tracor's
evaluation under this subfactor and the record clearly shows that
several strengths are identified. In addition, the presence of these
strengths reasonably supports the distinction between Tracor's and
Symetrics's proposals. Under these circumstances, we have no basis to
conclude that the evaluation here was improper. Moreover, there is no
requirement that an agency view as exceptional a proposal for which
the evaluators have identified a strength and no significant
weaknesses, nor is this result mandated by the evaluation
guidelines.[6] The difference between the assessment that a proposal
is acceptable rather than exceptional is a matter of judgment that we
will not disturb without a showing that the agency's application of
the ratings was unreasonable or unequal in some way.
In a similar vein, Symetrics argues that the Air Force failed to hold
meaningful discussions because the agency violated its own internal
regulatory guidance about discussions. In particular, Symetrics
argues it was not given a clear indication during discussions that the
agency was considering assessing a moderate risk against the company's
proposal under the manufacturing/quality assurance subfactor, and
under the management factor.[7]
As stated above, while we will not consider whether the Air Force
violated its internal guidelines in this area, we will address the
issue of whether discussions were meaningful. Our review of the
adequacy of discussions seeks to ensure that agencies point out
weaknesses that, unless corrected, would prevent an offeror from
having a reasonable chance for award. Ann Riley & Assocs., Ltd.,
B-271741.2, Aug. 7, 1996, 97-1 CPD para. 120 at 9, recon. denied,
B-271741.3, Mar. 10, 1997, 97-1 CPD para. 122.
Here, there is no dispute that while the Air Force did not indicate
the specific action it was considering based on its concern--i.e.,
assigning a moderate risk rating to the proposal in two areas--the
factual basis for its ultimate conclusion was clearly raised with the
protester during discussions. For example, in the area of labor
hours, the Air Force asked Symetrics why the labor hours in its cost
volume did not match those in its technical proposal, and gave the
company an opportunity to explain the discrepancy. Given that the
factual issues that formed the underpinning for the evaluation
concerns were clearly raised with Symetrics, we conclude that there
was no violation of the more general requirement to point out all
weaknesses that prevent an offeror from having a reasonable chance for
award. Robbins-Gioia, Inc., B-274318 et al., Dec. 4, 1996, 96-2 CPD para.
222 at 16-17.
Finally, Symetrics argues that the source selection statement lacked a
reasonable basis because the evaluation conclusions upon which it was
based were in error, because portions of the statement were
inconsistent with the evaluation conclusions, and because the
magnitude of the cost/technical tradeoff was obscured by the
comparison of MPLCC, rather than each offeror's proposed price.
With respect to Symetrics's challenge that the source selection
statement was based on erroneous evaluation conclusions, our review of
the record, including the issues discussed above, leads us to conclude
that the evaluation was reasonable, and thus provided a reasonable
basis for the SSA's decision.
With respect to Symetrics's contention that portions of the source
selection statement were inconsistent with the evaluation conclusions,
we have reviewed each of Symetrics's arguments and we conclude that
there is no basis to find that the statement was unreasonable. For
example, the agency's evaluators concluded that Symetrics's proposal
posed a moderate risk under the manufacturing/quality assurance
subfactor because the proposal's "labor hours appear to be low for a
first time producer of a system of this complexity." Addendum to
Proposal Analysis Report, supra, at 14. In Symetrics's view, there is
an inconsistency between the conclusion above and the source selection
statement, which indicates that Symetrics received the moderate risk
rating because of the "unrealistically low labor hours in [its] cost
proposal for a first time producer of a system of this complexity."
Source Selection Statement, supra.
We disagree. In our view, there is no meaningful difference between
the observation made by the evaluators ("hours appear to be low") and
the statement by the SSA ("unrealistically low labor hours"), and
nothing about the difference in these two statements leads us to
conclude that the SSA's conclusions are unsupported by the record.
Contrary to Symetrics's contentions, the SSA's statement does not mean
there was an improper or unreasonable cost realism review here.
Instead, we interpret the SSA's choice of words as within the normal
range of variation in language. There is no doubt that the evaluators
and the SSA are describing the same phenomenon--i.e., that Symetrics's
proposed labor hours in its technical proposal raised concerns about
the company's ability to produce the system as claimed, especially
after the proposed hours were unilaterally lowered in response to a
discussion question that directed Symetrics to discrepancies in the
number of labor hours identified in the technical and cost portions of
its proposal. Id. at 3.
Finally, we turn to Symetrics's claim that the magnitude of the
cost/technical tradeoff was obscured by the SSA's comparison of
overall life-cycle costs, or MPLCC, rather than each offeror's
proposed price.
Section M.4 of the RFP sets forth the process by which the agency
intended to calculate each offeror's MPLCC. In essence, the record
here shows that the calculation of the MPLCC involved adding $58.4
million to Tracor's and $58.7 million to Symetrics's price. In
selecting Tracor, the SSA compares the MPLCC developed for Symetrics
and Tracor and concludes that "while Tracor's total MPLCC is 1.4
[percent] higher than [Symetrics's], discounting risk, I consider this
difference to be insignificant." Source Selection Statement, supra,
at 5.
Our review of the record shows that there is, in fact, a 5.4-percent
difference between the proposal price offered by Symetrics ($18.716
million) and the price offered by Tracor ($19.771), and that the
addition of virtually identical total life-cycle costs tends to
obscure, rather than illuminate, the true cost/technical tradeoff
performed here. Although Symetrics raises this issue in its protest,
the Air Force's approach to evaluating the proposed prices has been
clear since the beginning of this procurement. Not only was this
matter apparent from the face of the solicitation, but it was apparent
when we reviewed the earlier award decision in Symetrics Indus., Inc.,
supra. Accordingly, this element of Symetrics's challenge to the
cost/technical tradeoff is untimely. 4 C.F.R. sec. 21.2(a)(1) (1998).
The protest is denied.
Comptroller General
of the United States
1. The appropriate method for procuring these items--after procuring
them since 1988 on a sole-source basis from Tracor, the original
equipment manufacturer--has been in dispute for several years. A
recitation of the earlier controversies surrounding this procurement
is found in our decision in Datacom, Inc.--Protests and Request for
Costs, B-274175 et al., Nov. 25, 1996, 96-2 CPD para. 199 at 2-4. In
addition, after our decision sustaining Symetrics's challenge to the
earlier selection of Tracor for award, Symetrics and another offeror,
Varo, LLC, challenged the revised solicitation. After mediation of
the dispute by our Office, the Air Force agreed to amend the
solicitation. In response, Symetrics withdrew its protest, and Varo's
protest was dismissed as academic. Symetrics Indus., Inc.,
B-274246.6, Nov. 19, 1997; Varo LLC, B-274246.8, Nov. 19, 1997.
2. The color/adjectival ratings used in the evaluation were
blue/exceptional, green/acceptable, yellow/marginal, and
red/unacceptable. Proposal risk was rated high, moderate or low, and
was defined as the risks identified in an offeror's proposed approach.
Performance risk was rated high, moderate, low, or not applicable, and
was defined as an assessment of the offeror's present and past work
record to gauge confidence in the offeror's ability to successfully
perform as proposed.
3. The SSA also concluded that the third offeror, whose MPLCC was
higher than Tracor's, offered the government no additional benefit
commensurate with its higher long-term cost.
4. We note, however, that one of the two articles cited by Symetrics
appears under the heading "Unconventional Wisdom," which suggests that
the Air Force's view of the benefits of Tracor's Six Sigma Program is
squarely within the mainstream of conventional thought on this issue.
5. Air Force Federal Acquisition Regulation Supplement (AFFARS),
Appendix BB sec. BB-304. In contrast, a green (acceptable) rating is
defined as "[m]eets evaluation standards and any weaknesses are
readily corrected." Id.
6. In fact, even if we were to consider Symetrics's challenge to the
application of the color rating scheme, Symetrics's argument is based
on an unreasonably narrow reading of the definitions in the scheme.
7. Again, Symetrics's argument is based on a provision in Appendix BB
of the AFFARS which requires that the agency prepare significant
clarification requests (SCR) whenever a performance risk assessment of
moderate or high is made in evaluating an offeror. AFFARS sec.
BB-307(a)(1). In the event that the agency establishes a competitive
range, these SCRs are to be provided to the offeror during
discussions, on the same basis as deficiency reports. AFFARS sec.
BB-307(a), (b).