Electronic Warfare: Navy's New Radar Warning Receiver Needs More Testing
(Letter Report, 06/20/96, GAO/NSIAD-96-68).
Pursuant to a congressional request, GAO reviewed the Navy's development
and acquisition of the ALR-67(V)3 radar warning receiver (RWR).
GAO found that: (1) the Navy plans to start low-rate production of the
ALR-67(V)3 receiver early in fiscal year 1997 despite numerous
performance problems; (2) the premature procurement of earlier models
before completion of operational testing led to the deployment of an
unsatisfactory system to operational forces that refused to use it, and
the storage of newly produced units until they were modified; (3) the
ALR-67(V)3 receiver misidentifies threatening and friendly radar, fails
to detect tracking radar, and may not be compatible with F/A-18 aircraft
radar; (4) the Navy does not know if the receiver is operationally
effective and suitable for field use; (5) in response to legislative
requirements, the Navy plans to conduct operational testing in late 1996
and early 1997 on production representative receivers to determine
potential modifications; and (6) the Navy risks deploying a deficient
system which may require extensive modifications and retrofit to achieve
adequate performance.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-96-68
TITLE: Electronic Warfare: Navy's New Radar Warning Receiver Needs
More Testing
DATE: 06/20/96
SUBJECT: Navy procurement
Warning systems
Radar equipment
Concurrency
Appropriation limitations
Military cost control
Fighter aircraft
Weapons research
Electronic warfare
Advanced weapons systems
IDENTIFIER: ALR-67 Radar Warning Receiver
ALR-67(V)3 Radar Warning Receiver
ALR-67(V)2 Radar Warning Receiver
ALR-67(V)4 Radar Warning Receiver
F/A-18C/D Aircraft
F/A-18E/F Aircraft
Hornet Aircraft
F-14 Aircraft
Tomcat Aircraft
AV-8B Aircraft
Harrier Aircraft
DOD System Anomaly Report
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Honorable
William V. Roth, Jr., U.S. Senate
June 1996
ELECTRONIC WARFARE - NAVY'S NEW
RADAR WARNING RECEIVER NEEDS MORE
TESTING
GAO/NSIAD-96-68
Electronic Warfare
(707104)
Abbreviations
=============================================================== ABBREV
DOD - Department of Defense
DOT&E - Director of Operational Test and Evaluation
RWR - radar warning receiver
OSD - Office of the Secretary of Defense
Letter
=============================================================== LETTER
B-260441
June 20, 1996
The Honorable William V. Roth, Jr.
United States Senate
Dear Senator Roth:
At your request, we examined the ALR-67(V)3 radar warning receiver
(RWR) program. The ALR-67(V)3 is intended to help protect Navy
aircraft and the program is expected to cost about $1 billion. We
are issuing this report to bring to your attention certain aspects of
the Navy's acquisition plans that we believe will unnecessarily
increase the Department of Defense's (DOD) risk on the program.
BACKGROUND
------------------------------------------------------------ Letter :1
RWRs are electronic warfare devices that help protect aircraft
against radar-controlled weapons. RWRs sense the signals from
hostile radars, provide an audio warning to the pilot, and display
the warning information on a video screen in the cockpit. The
display identifies the threats, provides their location or relative
bearings, and ranks the threats in order of danger to the aircraft.
Based on the warning, the pilot chooses from various options for
defeating the threat, such as employing electronic jamming to
interfere with the radar.
The ALR-67(V)3 is intended to be the Navy's next generation RWR for
use on its future F/A-18 E/F aircraft. The ALR-67(V)3 is also
planned for use on some current F/A-18 C/D aircraft, and with minor
antenna modifications, can also be retrofitted as the ALR-67(V)4 into
the F-14 and AV-8B aircraft.
The Navy expects the ALR-67(V)3 to significantly outperform the
current RWR, the ALR-67(V)2. The ALR-67(V)3 is expected to be
capable of detecting threats at a greater range and identifying more
types of threat radars than its predecessor.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
Despite numerous performance problems that surfaced in developmental
testing of the ALR-67(V)3, the adverse consequences from the
premature procurement of the original ALR-67 and the ALR-67(V)2, and
having already produced sufficient test articles for all operational
testing, the Navy plans to commit to low-rate production of the
ALR-67(V)3 before determining through completion of operational
testing that the system is operationally effective and operationally
suitable.\1 As a result, the Navy risks procuring a deficient system
that may require expensive modifications and retrofit to achieve
adequate performance.
--------------------
\1 Operational effectiveness refers to the ability of a system to
accomplish its mission in the planned operational environment.
Suitability is the degree to which a system can be placed
satisfactorily in field use considering such factors as reliability
and maintainability.
ADVERSE IMPACT OF PREMATURE
PRODUCTION OF ALR-67 RWRS
------------------------------------------------------------ Letter :3
Despite the importance of operational testing\2 as a management
control to ensure adequate system performance, DOD started production
of the original ALR-67 and its successor, the ALR-67(V)2, before they
were operationally tested. The adverse consequences included
deploying an unsatisfactory system to the operational forces and
placing newly produced systems in storage rather than having them
enhance the Navy's combat posture.
The Navy began producing the original ALR-67 in the early 1980s
before proving it was operationally effective. Subsequently, the
system performed so poorly during operational testing that the
testing had to be curtailed. However, the Navy continued production
while the system's performance problems remained unresolved.
As a result, the Navy installed the unsatisfactory system in
operational aircraft for use by the combat forces. Subsequently,
some Navy pilots during Operation Desert Storm distrusted the system
to the extent that they stopped using it to detect threat radars and
relied instead on other means that DOD considers classified.
To solve the ALR-67's problems, the Navy acquired the ALR-67(V)2.
Once again, however, the Navy started production before correcting
performance deficiencies. After operational testing showed that the
ALR-67(V)2 did not meet the Navy requirement, some 230 production
units, costing $75 million, were placed in storage pending
modifications to achieve satisfactory performance.
--------------------
\2 Operational testing is DOD's primary means of evaluating weapon
system performance. It is also a key internal control to ensure that
decisionmakers have objective information available on a weapon
system's performance to minimize risks of procuring costly and
ineffective systems.
DEVELOPMENTAL TESTS REVEAL
PROBLEMS WITH ALR-67(V)3
PROTOTYPE
------------------------------------------------------------ Letter :4
The Navy is developing the ALR-67(V)3 to overcome the problems with
the previous versions of the ALR-67. However, developmental testing
of ALR-67(V)3 prototypes conducted so far has disclosed numerous
deficiencies with its performance. According to the System Anomaly
Reports\3 the Navy provided to us, some of the deficiencies include
-- misidentification of threat radars or identification of friendly
radar as a threat,
-- warnings of multiple threats when only one is present or
vice-versa, or
-- failure to detect radars tracking the aircraft.
Moreover, developmental tests show that it has not been proven
compatible with the F/A-18 aircraft's radar. For example, normal
operation of the F/A-18 radar can cause the ALR-67(V)3 to warn of
nonexistent threats.
The System Anomaly Reports indicate that some of the reported
deficiencies were identified as early as mid-1993, and the
contractor's recommended action for many of the reported deficiencies
was not to proceed to operational evaluation until corrected.
Without operational evaluation, however, the Navy cannot have
complete assurance that any planned fixes for those deficiencies will
work.
In addition to the developmental tests of effectiveness conducted
earlier, the Navy conducted an operational assessment of a
developmental model of the ALR-67(V)3 in September 1994. However,
because the system was not production representative, that is, it was
not of the final design planned to be produced, the Navy deferred
testing of operational suitability. Thus, the Navy has no assurance
yet that suitability considerations, such as reliability and
maintainability, will be adequate.
--------------------
\3 System Anomaly Reports are official records created to document
instances when a system fails to perform as expected.
LEGISLATIVE REQUIREMENT IMPOSED
ON ALR-67(V)3 PROGRAM
------------------------------------------------------------ Letter :5
Because of concerns that the Navy was proceeding with a high-risk
acquisition plan for the ALR-67(V)3, the Congress included, among
other requirements in the fiscal year 1995 Department of Defense
Appropriation Act, a requirement that DOD certify, based on the
results of joint developmental tests and operational test flights,
that the ALR-67(V)3 was potentially operationally effective and
potentially operationally suitable before low-rate production. This
requirement was applicable only during fiscal year 1995. In its
comments on a draft of this report, however, the Navy states that it
will adhere to this and other fiscal year 1995 requirements
pertaining to the ALR-67(V)3.
The Navy has defined an operational test program for the ALR-67(V)3
calling for a first phase of joint developmental tests with
1 month of operational test flights, which it calls OT-IIA. OT-IIA
will occur primarily in the Navy's indoor chamber facility at
Patuxent River, Maryland, and is planned to be completed in August
1996. This is to be followed by a 4-month phase of operational
flight testing called OT-IIB that is planned for the Navy's open air
range at China Lake, California. The Navy's program schedule
indicates that OT-IIB is scheduled to be completed during the spring
of 1997. According to the program office, potential fixes for the
developmental problems described above will be tested and confirmed
using 20 production representative assets procured specifically for
the operational tests.
ALR-67(V)3 SCHEDULE CONTAINS
UNNECESSARY RISK
------------------------------------------------------------ Letter :6
Notwithstanding the unresolved deficiencies in the ALR-67(V)3's
effectiveness, the unknown nature of its suitability, and the Navy's
experiences with earlier versions of the ALR-67, the Navy plans to
commit to the ALR-67(V)3's low-rate initial production before
completing the second phase of operational testing (OT-IIB) in the
spring of 1997. According to the program schedule, the Navy plans to
award a low-rate production contract in the first quarter of fiscal
year 1997 for the manufacture of ALR-67(V)3 systems for deployment on
F/A-18 aircraft after the first phase (OT-IIA) of joint developmental
and operational testing. However, while OT-IIA may demonstrate
potentialities, the Navy will not be able to conclude from this
testing phase alone that the system is operationally effective and
suitable; such a conclusion must be based on a large number of flight
hours in a realistic environment that only OT-IIB can provide.
Furthermore, because the Navy intends to conduct both operational
test phases using ALR-67(V)3 production representative systems
previously procured as test assets, low-rate production of additional
new systems is not needed to complete operational testing.
While the Navy's stated intent to adhere to an earlier requirement
that the ALR-67(V)3 be potentially effective and suitable may provide
some assurance of future success, we do not believe that this goes
far enough. Past failures with earlier versions, unresolved
deficiencies with the ALR-67(V)3, and the availability of test assets
for all operational tests, leads us to conclude completion of OT-IIB
rather than an early commitment to low-rate initial production after
OT-IIA would be a far more prudent acquisition strategy.
RECOMMENDATION
------------------------------------------------------------ Letter :7
To minimize the risk of procuring another deficient RWR, and because
the Navy does not need additional ALR-67(V)3 units to complete
operational testing, we recommend that the Secretary of Defense
require that the ALR-67(V)3 complete both phases of operational
testing (OT-IIA and OT-IIB) to determine its effectiveness and
suitability, and that the deficiencies identified during
developmental testing have been resolved before committing to
low-rate production.
MATTER FOR CONGRESSIONAL
CONSIDERATION
------------------------------------------------------------ Letter :8
Because the Navy has not modified what we believe is a high-risk
acquisition strategy, the Congress may wish to restrict DOD from
spending any funds on ALR-67(V)3 production until both phases of
operational testing (OT-IIA and OT-IIB) are complete and the system
is certified effective and suitable by DOD.
AGENCY COMMENTS
------------------------------------------------------------ Letter :9
DOD generally concurred with this report, but indicated it did not
agree with our draft report recommendation that the Secretary of
Defense require the Navy to do sufficient testing before production
to ensure that developmental deficiencies have been resolved. DOD
maintained that because of the congressional reporting requirements,
it saw no need to place any additional requirements on the Navy.
However, when the requirements were enacted, the deficiencies
identified during developmental testing were not generally known. As
we stated in this report, the Navy's intent to adhere to the earlier
requirements may provide some assurance of future success, but in
light of the unresolved deficiencies, the Navy's continued adherence
to its acquisition strategy is not reassuring. We have, therefore,
modified our report and recommendation to address the congressional
reporting requirements and restate our concern that the Navy
continues to pursue a high-risk acquisition strategy by not
completing both phases of operational testing before low-rate
production and by planning to acquire additional systems not needed
for testing. DOD's comments are presented in their entirety in
appendix I along with our evaluation of them.
SCOPE AND METHODOLOGY
----------------------------------------------------------- Letter :10
We performed our work at the Naval Air Systems Command, Washington,
D.C.; the Offices of the Director of Electronic Combat and Director
of Operational Test and Evaluation (DOT&E) and the Office of the
Secretary of Defense (OSD), Washington, D.C. In evaluating
ALR-67(V)3 performance, we reviewed developmental test results,
including System Anomaly Reports and operational assessment results.
We also discussed the test results and potential performance issues
with DOT&E, OSD, and Navy officials responsible for oversight of
electronic warfare systems' acquisition.
Our review was performed from January 1995 through April 1996 in
accordance with generally accepted government auditing standards.
--------------------------------------------------------- Letter :10.1
We are sending copies of this report to other appropriate
congressional committees; the Secretaries of Defense and the Navy;
and the Director, Office of Management and Budget. We will make
copies available to others upon request.
Please contact me at (202) 512-4841 if you or your staff have any
questions concerning this report. Major contributors to this report
were Jackie B. Guin, Charles A. Ward, and A. Delores Cohen.
Sincerely yours,
Louis J. Rodrigues
Director, Defense Acquisition Issues
(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on the Department of Defense's (DOD)
letter dated March 7, 1996.
GAO COMMENTS
----------------------------------------------------------- Letter :11
1. According to the ALR-67(V)3 program schedule provided to us by
the program office, the Navy plans to begin low-rate production of
the system after the first of two phases of testing (OT-IIA). DOD
considers OT-IIA to be joint developmental tests and operational
flight testing. However, as discussed in DOD's letter, the first
phase will be conducted primarily indoors in a chamber facility at
Patuxent River, Maryland. Thus, most of OT-IIA will not be done in a
realistic operational environment as required by DOD policy and
should therefore not be considered to constitute realistic
operational testing. The Navy's program schedule further indicates
that the second phase of testing (OT-IIB), which is planned for a
4-month period at the Navy's open air range at China Lake,
California, is to take place after low-rate production begins.
2. Requiring the Navy to comply with our recommendation would better
ensure that the Navy does not commit to production of a deficient
system, as it did with earlier versions of the ALR-67. DOD's
compliance with the congressional reporting requirements, including
mandatory certification from the Director of Operational Test and
Evaluation that the ALR-67(V)3 is potentially operationally effective
and potentially operationally suitable, may provide some assurance of
future successful performance. However, DOD can only be certain that
the ALR-67(V)3's problems have been solved by satisfactorily
completing both phases of testing (OT-IIA and OT-IIB) and determining
that it is both effective and suitable.
3. The majority of the flight testing, lasting 4 months according to
the Navy's schedule, is to be conducted after the low-rate production
decision.
4. We believe that the intent to comply with congressional reporting
requirements will ensure that the ALR-67(V)3 is better tested prior
to beginning production than its predecessor ALR-67 models. However,
assurance of operational effectiveness and suitability can only be
achieved through successful completion of both test phases (OT-IIA
and B). Despite the various congressional reporting requirements,
the Navy may still commit to the ALR-67(V)3's low-rate production
before demonstrating in a realistic operational environment that the
system's performance is satisfactory by completing both phases of
testing. Compliance with our recommendation, in addition to the
reporting requirements, could prevent premature commitment to
production.
*** End of document. ***