TITLE: DRS Systems, Inc., B-289928.3; B-289928.7, September 18, 2002
BNUMBER: B-289928.3; B-289928.7
DATE: September 18, 2002
**********************************************************************
DRS Systems, Inc., B-289928.3; B-289928.7, September 18, 2002
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective
Order. This redacted version has been approved for public release.
Decision
Matter of: DRS Systems, Inc.
File: B-289928.3; B-289928.7
Date: September 18, 2002
W. Jay DeVecchio, Esq., Kathleen E. Karelis, Esq., Jeffrey C. Walker,
Esq.,
Christine S. Trafford, Esq., Duncan N. Stevens, Esq., and Alexa Zevitas,
Esq.,
Miller & Chevalier, for the protester.
Mark D. Colley, Esq., Michael W. Clancy, Esq., Maria Whitehorn Votsch,
Esq.,
Stuart W. Turner, Esq., David S. Black, Esq., Kristin E. Ittig, Esq., and
Dorothy C. Slovak, Esq., Holland & Knight, for Raytheon Company, an
intervenor.
Joshua A. Kranzberg, Esq., Christine L. Kachan, Esq., Capt. Gary P.
Bilski, Susan M. Lewandowski, Esq., and William Reed, Esq., Army Materiel
Command, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
Agency reasonably considered in the evaluation the extent to which the
awardee's proposed thermal sight system (for light armored vehicle)
exceeded the stated required and desired range performance by furnishing
better resolution and discernment of finer details; where a solicitation
contains evaluation criteria that allow for qualitative or graduated
assessments of proposals, rather than pass/fail evaluations, it is proper
to give a proposal more credit for superiority under one of those
evaluation criteria.
DECISION
DRS Systems, Inc. protests the Army Materiel Command's (AMC) award of a
contract to the Raytheon Company, under request for proposals (RFP)
No. DAAE07‑01-R-M005, for development and production of an Improved
Thermal Sight System (ITSS) for a United States Marine Corps light armored
vehicle (LAV), the LAV-25. DRS asserts that the evaluation failed to
adequately account for the superiority of its proposed system and was
otherwise unreasonable.
We deny the protest.
BACKGROUND
Marine Light Armored Reconnaissance battalions are equipped with the LAV
family of vehicles, including the LAV-25, an all-terrain, all-weather
wheeled vehicle equipped with a 25mm chain gun and a machine gun. As part
of a service life extension and upgrade program for the LAV-25s, which
were first introduced in the early 1980s, the current thermal imaging
system will be replaced with an ITSS providing thermal (night vision)
imaging, day/night sight optics, and an integrated laser range finder
(LRF). The ITSS will provide at least a 55-percent increase in target
recognition range and a 35-percent increase in target identification range
relative to the current system.
The ITSS Purchase Description (PD) included in the RFP established
required minimum performance ranges for the ITSS thermal channel at 0DEG
elevation under 16 conditions, including: (1)-(4) wide field of view
detection of target in moderate weather, adverse weather, CL 2 (white
phosphorous smoke), or fog oil; (5)‑(8) narrow field of view
detection of target under the same four environmental conditions;
(9)‑(12) narrow field of view recognition of target under the same
four environmental conditions; and (13)-(16) narrow field of view
identification of target under the same four environmental conditions. In
addition, the PD provided: *It is also desired that the 0DEG elevation
range requirements for each of the sixteen given conditions be met or
exceeded over the full elevation excursion of the LAV-25 main gun (i.e.,
-8DEG to 60DEG).* PD P: 3.3.2.2.1.[1] As discussed below, thermal
channel performance for these 16 scenarios was calculated for each of six
gun elevation angles (‑8DEG, 0DEG, 15DEG, 30DEG, 45DEG, and 60DEG ),
for a total of 96 individual scenarios (as well another 96 individual
scenarios when performance with electronic boost was considered). The PD
required the LRF to display target location range to within
+/‑10 meters with a 99‑percent probability of successfully
ranging on standard North Atlantic Treaty Organization targets from 200 to
7,995 meters given a visibility of 8,000 meters with no precipitation and
to 9,995 meters given a visibility of 23,000 meters with no
precipitation. The PD also established a desired accuracy of +/‑5
meters. PD P: 3.3.2.3.
Award was to be made to the offeror whose proposal represented the *best
value.* The RFP provided for proposals to be evaluated in three
areas--technical, cost, and program management. Technical was
significantly more important than cost and program management combined,
while cost was more important than program management. The technical area
consisted of: (1) ITSS performance, including consideration of key
performance parameters, which was significantly more important than other
performance requirements and desired performance enhancements; and (2)
supportability, including consideration of provisioning/supply support,
technical manuals, training, test measurement diagnostic equipment (TMDE),
configuration management, and test support. There were five key
performance parameters: (1) thermal channel performance (as described
above), which was more important than (2) LRF performance (as described
above), which was as important as (3) reliability, (4) availability and
(5) maintainability combined. Beyond these weighted considerations, the
RFP also provided for consideration, under the technical area, of proposed
measures to reduce operation and support (O&S) costs.
Following receipt of initial proposals, discussions and receipt and
evaluation of final proposal revisions (FPR), AMC initially awarded a
contract to DRS in December 2001. Raytheon thereupon filed an
agency-level protest, and then filed a protest with our Office challenging
the award. Upon determining that it had credited DRS with a higher level
of performance and design maturity than DRS's proposal warranted, and that
it had failed to evaluate proposed operation and support cost savings, AMC
proposed to take corrective action in the form of amending the
solicitation and reopening discussions. We dismissed Raytheon's protest
as academic. B-289928, B‑289928.2, Mar. 21, 2002.
After requesting revised proposals and conducting discussions with
offerors, AMC requested second FPRs. Second FPRs were received from DRS,
Raytheon and a third offeror (not relevant here). Both Raytheon and DRS
received an overall excellent rating under the technical performance
element of the technical factor. Although DRS's proposal was evaluated as
having an advantage with respect to desired performance enhancements
(being rated excellent, in contrast to Raytheon's good rating), the source
selection authority (SSA) determined that Raytheon's proposal was superior
overall for technical performance, based on advantages with respect to the
key performance parameters, which were significantly more important than
other performance requirements and desired performance enhancements
combined. The Source Selection Decision (SSD) recognized that DRS's LRF
approach provided *slightly* better capability than Raytheon's based on
DRS's having *a tighter [laser] beam divergence ([DELETED]), which allows
DRS to mark a target with less risk of hitting the ground and creating
false returns,* and a greater range capability. SSD at 5. While DRS's
proposal, like Raytheon's, also was rated excellent for thermal channel
operational performance, the most important key performance parameter
subelement, Raytheon's approach was considered to be more advantageous.
As explained in the SSD, while both Raytheon's and DRS's thermal channel
exceeded all the range requirements at the 0DEG elevation angle, and both
met or exceeded the desired range requirements at 0DEG in 8 of the
16 mission cases without electronic boost and 10 with boost, only
Raytheon's thermal channel offered the desired range performance across
the full excursion of the elevation angles (-8DEG, 0DEG, 15DEG, 30DEG,
45DEG, and 60DEG) in all 16 mission cases (whether boosted or not). In
contrast, DRS's thermal channel offered the desired range performance
across the full excursion of the elevation angles in only 8 of 16
conditions unboosted, and 9 conditions boosted, due to a failure to offer
the desired performance in all cases at the [DELETED]DEG elevation. In
addition, the agency determined that Raytheon's thermal channel offered
superior high frequency performance such that it would provide better
resolution and discernment of finer details. Finally, while DRS's
proposal was more advantageous with respect to maintainability, AMC
determined that Raytheon's proposal was more advantageous with respect to
reliability and availability. Based on its advantages with respect to
thermal channel performance, reliability and availability, Raytheon's
overall technical performance was found to be superior to DRS's.
Raytheon's proposal also held an advantage with respect to supportability
(the other element of the technical area), for which it received an
excellent rating, while DRS's received only a good rating. In explaining
the evaluation in this regard, the SSD noted that for provisioning and
support, the most important factor under the supportability element, DRS's
final proposal received only a good rating, on the basis that it had
*proposed a level of CLS [contractor logistics support] spares that, when
considered along with their proposed Service reps approach, results in a
risk, or lack of confidence, that DRS will be able to meet the follow-on
support requirements in a timely or efficient manner.* SSD at 7. In
contrast, Raytheon's proposal received an excellent rating for
provisioning and support based upon having a robust contractor logistics
support package, with an outstanding plan to support the fielded systems
with very low risk. In addition, not only did Raytheon's proposed O&S
cost savings ($[DELETED]), based on an improved mean time between failure,
exceed DRS's proposed O&S savings ($[DELETED]), but Raytheon also was
credited in the technical area for proposing an [DELETED], which would
result in another $[DELETED] million in savings. Both Raytheon's and
DRS's proposal received an excellent rating for program management.
Raytheon's evaluated cost ($[DELETED]) was higher than DRS's ($[DELETED]),
but the SSA determined that, given Raytheon's proposal's technical
superiority, including advantages with respect to technical performance
and supportability, *Raytheon's advantages, as well as the probable
$[DELETED] in O&S cost savings justify the Government's payment of the
[DELETED] . . . cost premium over the DRS proposal.* SSD at 10.
Upon learning of the resulting award to Raytheon, DRS filed this protest
with our Office challenging the award on several grounds. Based on our
review of the record, including testimony taken at a hearing our Office
conducted in this matter, we find no basis to question the award to
Raytheon. We discuss the most significant arguments below.
THERMAL CHANNEL PERFORMANCE
DRS asserts that, in its thermal channel evaluation, AMC did not
adequately credit DRS's proposal for having superior range performance at
a majority of the individual scenarios. In addition, DRS argues that
evaluating high frequency performance amounted to a departure from the
evaluation approach set forth in the RFP and that, moreover, Raytheon's
evaluated advantage in this regard was illusory.
In reviewing an agency's evaluation of proposals and source selection
decision, our review is confined to a determination of whether the agency
acted reasonably and consistent with the stated evaluation factors and
applicable procurement statutes and regulations. United Def. LP,
B-286925.3 et al., Apr. 9, 2001, 2001 CPD P: 75 at 10‑11; Main Bldg.
Maint., Inc., B-260945.4, Sept. 29, 1995, 95-2 CPD P: 214 at 4. Based on
our review of the record, we find no basis for questioning AMC's
determination that Raytheon's proposed thermal channel offered superior
performance.
Excursion of Angles
DRS's focus on the number of individual scenarios for which its evaluated
range exceeded that of Raytheon's system ignores the RFP's focus on
overall performance across the full excursion of ranges. As noted above,
the PD provided that: *[I]t is also desired that the 0DEG elevation range
requirements for each of the sixteen given condition be met or exceeded
over the full elevation excursion of the LAV-25 main gun (i.e., -8DEG to
60DEG).* PD P: 3.3.2.2.1. In response to a concern expressed by DRS
during the procurement that the agency's intended evaluation approach
would only consider whether the desired range was met collectively for all
six elevations (-8DEG, 0DEG, 15DEG, 30DEG, 45DEG, and 60DEG), rather than
evaluate thermal channel range performance at each elevation, AMC
responded (prior to the closing time for receipt of second FPRs) that:
Optimally, the thermal channel range performance should be demonstrated
over the full range of elevations. The SSA will be made aware of the
thermal channel performance at each of the 6 discrete angles under 4
meteorological conditions and 4 fields of view.
DRS letter to AMC, May 8, 2002, at 5-8; AMC Letter to DRS, May 9, 2002, at
1. As discussed above, only Raytheon's thermal channel offered the
desired range performance across the full excursion of the elevation
angles in all 16 conditions (whether boosted or not); DRS's thermal
channel offered the desired range performance across the full excursion of
the elevation angles in only 8 of 16 conditions unboosted and 9 conditions
boosted. This shortcoming reflected an inherent limitation in DRS's
system. DRS's thermal channel included a smaller aperture than Raytheon's
and, as a result, suffered more thermal sight vignetting, that is, a
reduction in intensity of illumination near the edge of an optical
instrument's field of view caused by obstruction of light rays by the edge
of the aperture. This can cause increasing degradation in thermal channel
range performance with higher head‑mirror elevation angles. Agency
Comments, Aug. 7, 2002, at 5 n.2; Hearing Transcript (Tr.) at 74-75,
119-20, 125-26, 200.[2]
Consistent with the agency's advice to DRS that it would make the SSA
aware of the thermal channel performance at each of the individual
scenarios, thus indicating that performance in this regard would be
accorded some, lesser weight in the evaluation, the record indicates that
the agency in fact took into account DRS's performance in this regard. As
noted in the Source Selection Evaluation Board (SSEB) report, DRS's
*proposed ITSS meets the majority of the range predictions . . . by
significant margin, and this is . . . true over more than 90 percent of
the LAV-25's gun angle excursion*; noting that DRS's performance therefore
*warrants substantial credit,* the SSEB determined that DRS's thermal
channel rating, previously a *strong good,* had *edged into the rating
category of excellent for its operational performance.* SSEB Report at
2. Likewise, the SSA recognized that *DRS was considered able to maintain
range performance up to about a [DELETED]DEG elevation . . . .* SSD at
4. However, it remains that only Raytheon offered the preferred level of
performance, that is, offered the desired range performance across the
full excursion of the elevation angles in all 16 mission cases (whether
boosted or not). The agency has reasonably determined that Raytheon's
thermal channel was superior in this respect.
Moreover, even had thermal channel performance at each of the individual
scenarios been accorded greater weight, it is not clear that DRS's system
in fact was clearly superior in this respect. Although DRS's thermal
channel was evaluated as having a greater range for more scenarios (50)
than was Raytheon's (43) when unboosted performance is considered,
Raytheon's thermal channel was evaluated as having a greater range for
more scenarios (49) than was DRS's (45) when boosted performance is
considered. Agency Comments, Aug. 16, 2002, at 3-6. (In response to
DRS's complaint that the agency's intended evaluation approach would not
consider electronic, digital boost when evaluating range performance, AMC
had responded that *[f]or offerors committing to the boosted values the
[SSA] will be advised of these values, in addition to the baseline
values,* that is, the unboosted values. DRS Letter to AMC, May 8, 2002,
at 2-5; AMC Letter to DRS, May 9, 2002, at 1.) Further, Raytheon's
thermal channel was evaluated as offering superior range for 12 of
16 conditions unboosted and 9 of 16 conditions boosted at 0DEG elevation,
the elevation which is the region of the main gun elevation continuum
which represents the highest operational frequency (and which was the only
elevation for which the RFP
established performance range requirements). Tr. at 66; DRS Comments,
July 29, 2002, Declaration of Technical Consultant, at 5‑6; Agency
Comments, Aug. 7, 2002, at 10.[3]
High Frequency Performance
Based on the output of its thermal channel models, AMC determined that
Raytheon's thermal channel offered superior high frequency performance
such that it would provide better resolution and discernment of finer
details. In this regard, in evaluating the thermal channel performance of
the proposed ITSS systems, the agency first obtained detailed system
information from the offerors. After running an optical system modeling
program, system data (including, e.g., detector sensitivity and optical
characteristics) was input into the Night Vision and Electronic Sensors
Directorate's NVTHERM thermal imager performance modeling program to
generate minimum resolvable temperature (MRT) difference data. In this
regard, in general, the smaller the minimum temperature differential--that
is, the difference in temperature between a target (or elements of a
target) and the background in a viewed scene--a thermal device can sense,
the finer the resolution and the sharper the image that can be obtained
from the system.[4] The MRT difference data then was plotted to create an
MRT curve (shown below in an illustrative figure furnished by AMC) which
characterizes the imaging system's resolving power, or ability to perform
imaging tasks, based on relative spatial frequency (cycles on target) and
the differential temperature sensitivity associated with the object being
imaged. Spatial frequency increases across a plotted MRT curve moving
left to right across the horizontal axis to represent the increasing
degree to which fine details in a scene being imaged can be discerned.
The increasing frequency along the horizontal axis also is related to
range; as the MRT curve moves further to the right along the horizontal
axis, it represents an ability to perform the same imaging task at a
greater distance. The vertical axis represents an increasing temperature
delta, that is, the difference in temperature between the target and its
background; as the MRT curves here move to the right towards higher
frequencies and detail, they also tend to move upward, representing the
fact that resolving target and scene details viewable in the higher
frequencies requires greater temperature differences. In sum, AMC
maintains, and the record indicates, that an MRT curve describes the
overall range performance capabilities of the thermal imager. Tr. at
11-35; Agency Comments, Aug. 31, 2002, at 4-5; 8; Contracting Officer's
Statement at 12-13.
Here, although the MRT curves of Raytheon's and DRS's ITSS thermal channel
performance generally coincided at the low to mid frequencies (at
elevations [DELETED]), with the curves reflecting a mid-frequency DRS
advantage [DELETED], but indicating very similar overall performance at
low to mid frequencies, the MRT curves also generally indicated an
increasing Raytheon advantage--that is, a curve further to the right along
the horizontal, frequency axis, representing an increased resolving
power--generally commencing at the mid‑frequency range, increasing
with frequency, and becoming more pronounced in the wide field of view,
boosted and high elevation cases. Agency Comments, Aug. 31, 2002, at 4,
MRT Curves; Tr. at 57-58, 77, 161-62, 195, 266-67, 202.[5]
DRS, however, argues that focusing on Raytheon's high frequency
performance, and the associated evaluated greater resolving power and
resulting finer detail, as shown by the MRT curves, was inconsistent with
the RFP's focus on range performance at low-to-mid frequency. In any
case, according to the protester, Raytheon's evaluated superiority in high
frequency performance was illusory.
DRS's position is not persuasive. As an initial matter, we note that
hearing testimony supports AMC's position that the agency's NVTHERM
thermal imager performance modeling program, from which the MRT curves are
derived, is the generally accepted Army and industry standard for
calculating thermal imager performance; and that the model has resulted
from continual improvement and validation over time. Tr. at 12,
36‑40, 539-40, 589-94. Indeed, DRS's own ITSS program manager
testified that, in designing its system, DRS looked to the NVTHERM model
and ACQUIRE, describing them as Army accepted standard tools, and DRS's
consultant agreed in his testimony that NVTHERM was the standard model in
industry and government, which *[e]veryone uses.* Tr. at 392, 429,
539-40. Further, the record indicates that offerors were on notice that
the agency would use NVTHERM to perform its thermal analysis. Tr. at
254-55, 278. Although AMC instructed DRS not to include MRT curves in the
PD to be included in its proposed contract, DRS's program manager
testified that DRS understood that *MRT obviously is part of the
evaluation,* and that the agency would look at MRTs in evaluating its
proposal; he testified further that DRS in fact furnished *MRT data* to
the agency. Tr. at 421-23, 429-30. Thus, the record indicates that, not
only did NVTHERM and MRT data furnish a reasonable basis for evaluating
the thermal channel performance of the proposed ITSS systems, but it also
was the evaluation approach a reasonable offeror should have expected and,
indeed, was the approach that the protester itself understood would be
employed.
Where, as here, a solicitation contains evaluation criteria that allow for
qualitative or graduated assessments of proposals, rather than pass/fail
evaluations, it is proper to give a proposal more credit for superiority
under one of those evaluation criteria. AdvanChip Corp., B‑282571,
July 29, 1999, 99-2 CPD P: 35 at 4 n.3; F2M‑WSCI, B‑278281,
Jan. 14, 1998, 98-1 CPD P: 16 at 7-8. We agree with the agency that
considering whether one proposed ITSS system offered superior resolving
power essentially amounted to determining whether that system exceeded the
requirements in a stated evaluation area, thermal channel performance, in
a manner beneficial to the agency.
In any case, while agencies are required to identify the major evaluation
factors in a solicitation, they are not required to identify all areas of
each factor that might be taken into account, provided that the
unidentified areas are reasonably related to or encompassed by the stated
criteria. S3 LTD, B-288195, Sept. 10, 2001, 2001 CPD P: 164 at 10. Here,
a system's high frequency performance was indicated by the model (NVTHERM)
the offerors knew would be used in the evaluation, and increasing
frequency was related to range in that it indicated an ability to perform
the same task at a greater range. Tr. at 29-30, 193. At a minimum, in
these circumstances, high frequency performance was reasonably related to
the thermal channel performance specified in the RFP.
As noted above, the MRT output from the accepted NVTHERM model indicated a
decided advantage for Raytheon with respect to high frequency
performance. Further, persuasive testimony from agency and other
witnesses confirmed that this was a meaningful advantage. Tr. at 28-32,
61, 67-68, 77, 83, 112-13, 172, 179-85, 192‑93, 196-200, 261-63,
267-69, 642-47, 652, 659-63. DRS, however, points out that the ability to
resolve finer detail at higher frequency requires greater temperature
difference between the target area and its background as the detail gets
finer. Tr. at 29, 90, 174-75; Agency Comments, Aug. 31, 2002, at 8. In
this regard, while the RFP specified a standard vehicle target (2.3 meters
x 2.3 meters), frontal view, at a target‑to-background temperature
difference of 1.25DEG Celsius, the study done during the evaluation that
set forth Raytheon's high frequency performance superiority assumed a
smaller target, a hull defilade armored fighting vehicle (i.e., only the
turret is visible), with an inherent temperature differential of 5DEG
Celsius. PD P: 3.3.2.2.1; Best Value Opinion, Thermal Channel, May 20,
2002, at 3. DRS, noting that this temperature differential was higher
than the 1.25DEG Celsius assumed in the RFP, asserts that the higher
temperature differentials required to discern finer detail are unlikely to
be encountered. However, testimony at the hearing, including that of
DRS's own consultant, confirmed that assuming temperature differentials of
at least 5DEG Celsius was realistic. Tr. at 89, 179‑85, 261-63,
284, 512.[6] In any case, we note that the MRT curves for the systems
begin to diverge in Raytheon's favor at a temperature differential below
5DEG Celsius. Agency Comments, Aug. 31, 2002, at 10.
We conclude that AMC reasonably evaluated Raytheon's ITSS as offering
superior high frequency thermal channel performance, representing an
increased resolving power and ability to discern finer details, and that
the agency acted reasonably in taking this advantage into account in the
evaluation. In this regard, as noted by the agency, recognition and
identification of hostile and friendly forces is often done using thermal
cues or hot spots, including engine location and size, exhaust location
and size, structural shapes and curvatures, suspension, track details,
drive wheels, number and relative size of road wheels, gun length, and
muzzle flash suppressors. See, e.g., Combat Vehicle Identification Guide
(Thermal), vol. 2 (Eval. 1.0). AMC maintains, and the record does not
show otherwise, that thermal sight systems that provide better high
frequency capability are able to resolve finer details that are critical
in identifying targets at longer range, thereby improving stand-off
distances and survivability and increasing the quality of reconnaissance
information. Tr. at 80‑81, 87-88; Agency Comments, Aug. 31, 2002,
at 4. In these circumstances, and given Raytheon's advantage with respect
to furnishing the desired performance throughout the excursion of gun
elevations, we find that Raytheon was reasonably evaluated as offering
superior thermal channel performance.
Signal Intensity Transfer Function
DRS asserts that AMC failed to recognize the superiority and the effect on
thermal system performance of its [DELETED] Signal Intensity Transfer
Function (SITF). SITF essentially is an algorithm that assigns image
intensity levels on a display screen, which has only a limited number of
display levels available, across ranges of temperature differentials
encountered in a given observable scene.
This argument is without merit. As noted by the agency, the PD provided
that thermal channel performance would be calculated *with a clear filter,
without electronic zoom, and without frame integration.* PD P:
3.3.2.2.1. To further clarify its intent in this regard, AMC advised DRS
in writing as follows:
The Government's evaluation of range performance (PD paragraph 3.3.2.2.1)
is being conducted without including effects of such image enhancing
features as e-zoom, frame integration, LACE [local area contrast
enhancement], digital boost filtering, etc. The Government's evaluation
will acknowledge such features as advantageous; however, the Government's
thermal channel operational performance assessment will be performed
without including such features.
E-mail from AMC to DRS, May 1, 2002. DRS responded: *This part
understood.* E‑mail from DRS to AMC, May 1, 2002. AMC
determined‑‑and DRS has not shown otherwise--that DRS's
[DELETED] SITF was an image enhancement feature. Agency Comments, Aug.
31, 2002, at 17-18; Tr. at 402, 405-07. As such, consistent with the PD
and the quoted e-mail correspondence, the SITF was not considered in the
thermal channel operational performance assessment. In any case, we note
that the SITF curve does not affect MRT results, which, as discussed
above, DRS understood would be used in the evaluation. Tr. at 205, 258.
This being the case, we agree with the agency that there was no reasonable
basis for DRS to expect that its SITF would be used to modify the reported
thermal channel performance of its proposed ITSS. As for the agency's
advice that it would otherwise consider proposed image enhancement
features in the evaluation, the record indicates that the agency in fact
evaluated DRS's proposed SITF approach, in conjunction with its proposed
[DELETED], as an *advantage* which *can enhance operational effectiveness
of the ITSS.* SSEB Report at 23. Further, while DRS's proposal included
pictures reportedly depicting the effect of its [DELETED] SITF when used
on another system, it did not include supporting quantifiable data
demonstrating the likely performance when used on the proposed ITSS; thus,
there is no basis for finding that DRS's proposal should have been
accorded an even greater advantage in this regard. DRS Technical Proposal
at 4-11 to 4-13, 4‑115A, 4-115D to 4-115F.
LRF
DRS asserts that AMC did not fully consider the superiority of its
proposed LRF system. In this regard, the SSD rated DRS's LRF approach
capability *slightly* better than Raytheon's based on DRS's LRF having (1)
*a tighter [laser] beam divergence ([DELETED]), which allows DRS to mark a
target with less risk of hitting the ground and creating false returns,*
and (2) a greater range capability, reflecting an approximate [DELETED]
percent performance margin beyond the RFP range requirements, as compared
to Raytheon's [DELETED] percent margin. SSD at 5; SSEB Report at 139; LAV
ITSS Final Briefing to SSA, May 28, 2002, at 10; Tr. at 679, 692, 694.
DRS asserts that AMC failed to take into account the fact that DRS's
narrower laser beam divergence would be less likely to encounter false
returns from clutter (clutter rejection), that is, a false range reading
based on a return of the reflected laser beam from an object other than
the target.
We find no basis to question AMC's evaluation in this area. Again, DRS's
proposal was assigned a strength on account of its tighter proposed beam
divergence, which was viewed as reducing the likelihood of false range
returns, and indeed was evaluated as offering a (somewhat) more
advantageous LRF. SSD at 5; Tr. at 679, 692. Although DRS believes that
its LRF approach should have received even more credit for avoiding
clutter rejection, the record supports AMC's determination not to assign a
greater strength in this regard. As an initial matter, it is clear that,
given the numerous potential variables involved, including the size,
reflectivity, range, range distribution, and relative placement of the
clutter, among other considerations, modeling the effects of clutter would
be very complex. As a result, there is no validated quantitative model
that measures the effects of clutter--and indeed, apparently, no validated
quantitative model that measures all of the considerations that could
affect laser performance--and the evaluation did not calculate the effect
of clutter in measuring LRF performance. Tr. at 681-704, 724-25, 810-11;
Contracting Officer's Statement at 32.
In any case, the record indicates that the choice of a laser beam
divergence figure involves a tradeoff between clutter rejection and aiming
error; while a narrower, tighter beam divergence focuses more concentrated
energy on the target and reduces spilling over of the beam onto clutter,
thereby decreasing clutter rejection, the tighter laser beam is more
susceptible to aiming error. Tr. at 684, 699, 727; Agency Comments, Aug.
31, 2002, at 27. In this regard, LRF aiming accuracy can be affected by
such factors as pointing jitter caused by operator control, platform
motion, engine vibration, turret movement and vehicle movement, boresight
error, and turbulence and other atmospheric effects. Tr. at 695-700, 726;
Agency Comments, Aug. 31, 2002, at 26-27.[7] Of particular note is the
fact that the LAV‑25 is an older platform with a minimal
stabilization system, which may create a pointing jitter problem for a
tight laser beam. Tr. at 753; Agency Comments, Aug. 31, 2002, at 26.[8]
AMC concludes, and DRS's own consultant testified, that given its narrower
laser beam divergence, DRS's laser beam was more likely to miss the target
than Raytheon's beam, with its wider beam divergence. Tr. at 699-700,
873-74, 880; Agency Comments, Aug. 31, 2002, at 26.
We also note that, even where the laser beam overspills the target and is
reflected off of clutter, the record indicates that this need not result
in acceptance of a false range report. Raytheon's proposed LRF (and most
LRF systems) includes a First/Last logic response function, which allows
the LRF user to focus on either the image that appears in the foreground
of the scene or in the background of the scene. Thus, for example, if
clutter were in front of the target, the LRF user could select the last
return as the one most likely to represent the range to the target rather
than to the clutter. In addition, the LRF operator can overcome clutter
to accurately determine target range by using the First/Last logic
response function to illuminate the target more than once from different
angles; the target range is the common value between the first and last
returns that were taken at different angles. Tr. at 684-88, 779,
792‑800; Agency Comments, Aug. 31, 2002, at 26. In this regard, we
consider it significant that, according to AMC, experience with fielded
LRFs has not shown there to be a significant false ranging problem.
Agency Comments, Aug. 7, 2002, at 22. We conclude that the agency
reasonably did not assign DRS's narrower laser beam divergence advantage
greater weight in the evaluation.
CONTRACTOR LOGISTICS SUPPORT (CLS)
Noting that its proposal had previously been rated excellent for
provisioning and support, the most important evaluation factor under the
supportability element, DRS questions the final rating of its proposal for
this factor as only good. In this regard, the RFP required that the
contractor provide CLS in accordance with Attachment 009 of the RFP, which
set forth *the government's concept for Contractor Logistics Support (CLS)
services/maintenance, which can be used as guidance, in the preparation of
the contractor's proposed CLS . . . .* RFP, Attach. 009, at 2.
Attachment 009 listed 12 tasks for which the contractor will be
responsible, including Task J: *Provide intermediate level repair through
Depot repair of major ITSS system components.* Id. Further, the
attachment requires that repair parts be delivered within 48 hours in the
event of priority 1 or 2 requisitions, 2 working days for priority
3 requisitions, and 5 working days for priority 4 through 15
requisitions. Id. at 3.
As noted by the agency, DRS proposed a [DELETED] CLS approach under which,
[DELETED]. DRS Technical Proposal at 9‑66 to 9-66B. In its first
FPR, DRS proposed an $[DELETED] fixed price for the overall CLS task, and
allocated $[DELETED] to Task J. DRS First FPR Cost Proposal at 2-109F to
2-109G. DRS's Task J allocation in its first FPR in turn was divided
among three subtasks: (1) [DELETED]; (2) [DELETED]; and (3) [DELETED].
Id. In its second FPR, however, DRS reduced its overall CLS price from
$[DELETED] to $[DELETED], and reduced its Task J allocation from
$[DELETED] to $[DELETED]. Most of the Task J reduction resulted from a
reduction in the cost of repair. According to DRS's second FPR cost
proposal:
We evaluated our CLS repair costs using the [DELETED] model in response to
Amendment 0007. This reduced our projected repair costs in our 5 Year
CLS.
. . . . .
We revised the estimate for the cost of repairs from $[DELETED] to
$[DELETED] based on the results of the in-depth analysis of O&S costs we
conducted in response to Amendment 0007 to the RFP. Based on this
analysis, we estimate annual repair costs of $[DELETED] to support normal
failure modes of our ITSS equipment. In addition to these equipment
repairs, the CLS program provides for repairs of externally induced
failures, for which we have no firm data to project. We have estimated
this level of repair to be approximately equal to [DELETED]. This
provides a 5‑year repair cost estimate of $[DELETED].
DRS Second FPR, Cost Proposal, at 2-107, 2-109h.[9] In addition, DRS's
second FPR included an *ITSS Cost Detail* in which an overall entry for
*CLS Services,* previously allocated $[DELETED] and described as *Estimate
for repair activities, supply support, services,* was replaced by an
allocation of $[DELETED] as the *Estimate for supply support, and other
services,* that is, without reference to repair activities. DRS First
FPR, ITSS Cost Detail, Contract Line Item No. (CLIN) 2005AA; DRS Second
FPR, ITSS Cost Detail, CLIN 2005AA.
Although AMC was able to ascertain the basis for, and resolve its concerns
(raised by the overall extent of the CLS reduction) with respect to, most
of the overall CLS reduction in DRS's second FPR, the agency determined
that DRS had not adequately supported the $[DELETED] reduction (from
$[DELETED] to $[DELETED]) in the repair and refurbishment allocation. In
addressing the reduction in this area, the SSEB determined that DRS had
*[d]eleted approximately 50% of the spare parts support at the repair
facility.* SSEB Report at 58. According to the agency:
The quantity of repair parts on hand at the repair facility is related
proportionally to the operational availability of the equipment in the
field. . . . A reduction of the parts on hand may lead to increased
logistics delay time awaiting parts from procurement.
SSEB Report at 58; see LAV ITSS Final Briefing to SSA, May 28, 2002, at
30. Likewise, in explaining the reduction in DRS's provisioning and
support rating from excellent to good, the SSA noted that DRS had
*proposed a level of CLS spares that, when considered along with their
proposed Service reps approach, results in a risk, or lack of confidence,
that DRS will be able to meet the follow-on support requirements in a
timely or efficient manner.* SSD at 7.
DRS asserts that the reduced evaluation rating was based on the mistaken
assumption that DRS was proposing to reduce the number of spare/repair
parts in inventory by approximately 50 percent. DRS notes in this regard
that the cost detail for CLS in its second FPR specified a quantity of
[DELETED] for most ITSS assemblies, which was only a minor reduction in
quantity from the [DELETED] specified in its first FPR. DRS First FPR,
ITSS Cost Detail, CLIN 2005AA; DRS Second FPR, ITSS Cost Detail, CLIN
2005AA. In addition, DRS argues that the agency's analysis ignores the
fact that the CLS CLIN was a firm, fixed-price CLIN under which DRS was
obligated to furnish the promised services at the CLIN price.
The evaluation in this area was reasonable. As an initial matter, the
solicitation specifically warned that the agency would perform a risk
analysis that would take into account unrealistically low pricing, stating
that a *risk analysis will be conducted to evaluate the Offeror's
capability to successfully execute its proposal to deliver a product
meeting performance requirement[s] within proposed cost and schedule
requirements.* RFP P: M.2.6. The RFP warned further that *[a]ny proposal
which is unrealistic in terms of technical or schedule commitments, or
unrealistically high or low in price may be considered reflective of an
inherent lack of technical competence or a failure to comprehend the
complexity and risks of the Government's requirements, and may be
rejected.* RFP P: M.2.4; see NLX Corp., B‑288785, B‑288785.2,
Dec. 7, 2001, 2001 CPD P: 198 at 198 (agency reasonably may consider risk
associated with low proposed fixed prices where the risk appropriately
relates to the offeror's understanding); The Cube Corp., B-277353, Oct. 2,
1997, 97-2 CPD P: 92 at 4 (agency may provide for price realism analysis
in the solicitation for such purposes as measuring an offeror's
understanding of the solicitation requirements or to avoid the risk of
poor performance from a contractor who is forced to provide services at
little or no profit).[10]
The record indicates that DRS's price reduction was not adequately
explained. Although DRS stated in its second FPR, submitted on May 14,
that it had *evaluated our CLS repair costs using the [DELETED]
model*‑‑presumably a reference to the
[DELETED]‑‑and that *[t]his reduced our projected repair
costs,* DRS Second FPR, Cost Proposal, at 2‑107, DRS has not pointed
to anything in its proposal specifically describing the model, or
explaining its use of the model or all of the inputs into the model.
(Even in its protest, DRS has not explained its use of the model or all of
the inputs into the model.) In his hearing testimony, DRS's consultant
generally referred to the discussion in DRS's second FPR concerning an
assumed hardware usage rate of 84 hours and a predicted mean time between
failure of [DELETED] hours. Tr. at 980-990. However, DRS cited the very
same numbers in its first FPR, for which the allocated repair cost was
more than twice that in its second FPR. DRS First FPR, Cost Proposal, at
2-108; DRS Second FPR, Cost Proposal, at 2-108. DRS's second FPR also did
not explain the basis for its formula, which assumed $[DELETED]
[DELETED]. In summary, while DRS claimed in both its first and second
FPRs that it had *done significant bottoms-up pricing of our hardware and
cost to support this hardware under a CLS program,* id., the analysis in
its first FPR assumed a repair cost of $[DELETED], while the analysis in
its second FPR assumed a repair cost of $[DELETED], without any meaningful
explanation as to how two such thorough reviews arrived at such
dramatically different results. Given DRS's failure to adequately support
its second FPR price reduction in this area, there is no basis for
concluding that AMC misevaluated the price reduction. It is well
established that an offeror is responsible for submitting an adequately
written proposal, and runs the risk that its proposal will be evaluated
unfavorably where it fails to do so. Carlson Wagonlit Travel, B- 287016,
Mar. 6, 2001, 2001 CPD P: 49 at 3.[11]
The protest is denied.
Anthony H. Gamboa
General Counsel
------------------------
[1] The PD defined *detection* as the perception or sensing of the
presence of a target of potential military interest; *recognition* as
discrimination between targets of different classes (e.g., between a truck
and tank); and *identification* as discrimination between targets within a
class (e.g., between different models of tanks). (The PD also specified a
test for each level of knowledge.) PD P: 3.3.2.2.1.
[2] In this regard, we note that when asked with respect to the draft PD
whether *vignetting [was] allowed for extreme look-up angles,* AMC
responded that *[i]t is desired that there be no vignetting during
operation of the system.* Question and Answer No. 17, Draft RFP, June 16,
2000.
[3] The record also indicates that, unlike Raytheon's, DRS's evaluated
range performance, which (as discussed below) was based on system
information furnished by DRS, did not reflect the vignetting that would
occur at the [DELETED]DEG elevation (approximately [DELETED]), in the
system as described in DRS's proposal, from [DELETED]. Although DRS
claims to have a design fix for this vignetting, it has furnished no data
demonstrating the effects on performance of an addition to its described
system. Tr. at 259, 572-78; Agency Comments, Aug. 31, 2002, at 2 n.1, 11.
[4] The MRT difference data from the NVTHERM modeling runs was then input
into the ACQUIRE software model, which is based on field tests, and which
predicts the range performance of the thermal imaging system corresponding
to the MRT differences. Tr. at 24-26, 31, 36; Contracting Officer's
Statement at 12-13.
[5] The MRT curves reflect a mid-frequency DRS advantage at the
[DELETED]DEG elevation but, as discussed above, DRS's input data for that
elevation failed to reflect vignetting in the system (as described in
DRS's proposal) caused by [DELETED].
[6] DRS's consultant testified that a temperature differential 5DEG
Celsius between a turret and its background will happen. Tr. at 512.
[7] Although DRS notes that it proposed a [DELETED], and furnished recent
test data that the agency found indicated that its assumptions in this
regard *were reasonable, if not conservative,* SSEB Report at 3, as noted
above, the record indicates that [DELETED] error is only one of a number
of potential causes of aiming error. (We also note that the agency still
assigned DRS's proposal a minor risk in this regard, with the evaluators
noting in the briefing to the SSA that there was *[s]ome risk in LRF
design modifications and need to maintain [DELETED].* LAV ITSS Final
Briefing to SSA, May 28, 2002, at 10.)
[8] Although DRS's consultant suggested during the hearing that aiming
inaccuracy can be reduced by stopping the vehicle and turning off the
engine, Tr. at 845, 862-63, we find reasonable the agency's position that
this mitigation approach would be inconsistent with the fact, specified in
the PD, that the vehicle will be operating in a dynamic environment 70
percent of the time. According to the PD, when operating in a dynamic
environment, the LAV-25 can be expected to move 100 miles in a 2-day
mission, illuminating with the LRF 10 times every 2 hours. PD P:
3.3.1.9.1.
[9] RFP amendment No. 0007 added instructions with respect to describing
the basis for an offeror's proposed O&S cost savings and clarified the
PD's provisions with respect to thermal channel performance.
[10] The RFP also stated that *the total Acquisition Cost of each
Offeror's proposal is the sum of all priced CLINs in Section B, and the
total proposed cost for all options,* and specifically provided that
*[t]he total acquisition cost element shall be evaluated for
reasonableness and realism.* RFP P: M.4.
[11] In a May 1 submission to the agency during discussions, shortly
before the May 14 submission of its second FPR, DRS indicated that it was
reducing its Task J allocation for the cost of repairs from $[DELETED] to
$[DELETED] *based on the results of the in-depth analysis of O&S costs we
conducted in response to Amendment 0007 to the RFP.* DRS Response, No.
IFD-CAD115R1, May 1, 2002, at 4. In a July 11 supplemental protest, DRS
asserted for the first time that the agency had failed to conduct adequate
discussions in this regard. However, DRS was aware as of its June 6
debriefing that the agency had reduced the final rating of its proposal
under the provisioning and support factor on account of an alleged factual
error with respect to the number of spares it was proposing, Protest, June
11, 2002, at 16. DRS's argument with respect to discussions therefore is
untimely because it was not filed within 10 days after the debriefing. 4
C.F.R. 21.2(a)(2) (2002).