Mine Detection: Army Detector's Ability to Find Low Metal Mines Not
Clearly Demonstrated (Letter Report, 08/28/96, GAO/NSIAD-96-198).
Pursuant to a congressional request, GAO reviewed the Army's development
of a portable land mine detector, focusing on: (1) how the Army's
AN/PSS-12 mine detector performed in detecting low-metallic mines in
procurement tests; (2) the nature of the land mine threat in
Bosnia-Herzegovina; and (3) the mine detector's potential effectiveness
in Bosnia.
GAO found that: (1) the Army has not clearly demonstrated the ability of
its AN/PSS-12 mine detector to detect low metallic mines; (2) the
detector performed poorly during operational testing and failed to meet
the Army's 92-percent detection requirement against low metallic mines;
(3) although both candidate detectors performed equally well after the
Army removed low metallic targets from the procurement tests, the Army
selected the AN/PSS-12 because of its lower price; (4) the detector's
field accuracy is questionable, since the Army did not sufficiently
control other environmental and operating factors that can affect
detector performance; (5) the detector's usefulness in Bosnia may be
limited because about 75 percent of the buried mines have a low metallic
content; (6) although the detector's reported performance in Bosnia is
good, the Army has limited the detector's use there; (7) the Air Force
has warned its personnel in Bosnia that the detector is not sufficiently
sensitive to low metallic mines and some countries have switched to
other mine detectors; and (8) the Army has reduced its reliance on the
detector through alternative threat-reduction practices, such as
extensive personnel training in mine awareness, avoiding or carefully
selecting routes through suspected mine fields, and using heavy
equipment to clear paths.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-96-198
TITLE: Mine Detection: Army Detector's Ability to Find Low Metal
Mines Not Clearly Demonstrated
DATE: 08/28/96
SUBJECT: Warning systems
Munitions
Explosives
Army procurement
Product performance evaluation
Military operations
IDENTIFIER: Bosnia
Yugoslavia
AN/PSS-11 Mine Detector
AN/PSS-12 Handheld Metallic Mine Detector
Army Handheld Standoff Mine Detection System
M-14 Mine
M-15 Mine
M-16 Mine
PMN-6 Training Mine
Somalia
Kuwait
Canada
Sweden
Germany
United Kingdom
Netherlands
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Ranking Minority Member, House Committee on
International Relations
August 1996
MINE DETECTION - ARMY DETECTOR'S
ABILITY TO FIND LOW-METAL MINES
NOT CLEARLY DEMONSTRATED
GAO/NSIAD-96-198
Mine Detection
(703127)
Abbreviations
=============================================================== ABBREV
DOD - Department of Defense
ROSE - test
Letter
=============================================================== LETTER
B-272391
August 28, 1996
The Honorable Lee H. Hamilton
Ranking Minority Member
Committee on International Relations
House of Representatives
Dear Mr. Hamilton:
The dangers posed by over 80 million landmines emplaced worldwide are
the subject of much discussion. You expressed concern over the
threat of landmines to U.S. troops as they carry out their mission
in Bosnia-Herzegovina. Landmines, especially those of low-metallic
content, have been used extensively by all warring factions in the
former Republic of Yugoslavia, and 5 to 7 million mines are estimated
to be in the region. Between April 1992 and June 1995--prior to the
deployment of U.S. troops to the former Republic--there were 174
landmine incidents involving U.N. Peacekeeping Forces, which
included 204 casualties and 20 deaths.
In response to your request, this report addresses (1) how the Army's
AN/PSS-12 portable mine detector performed in detecting low-metallic
mines in tests conducted prior to procurement, (2) the nature of the
landmine threat in Bosnia-Herzegovina, and (3) the AN/PSS-12's
potential effectiveness there.
BACKGROUND
------------------------------------------------------------ Letter :1
The military services use portable or handheld metal detectors as one
of several devices to detect and clear hazards such as landmines. As
we reported last year, the detection and clearance of buried
explosives like landmines is very difficult, and no ideal solution
has emerged.\1 Low-metallic content landmines--generally
plastic-encased explosives with some metal parts inside--are among
the most difficult mines for a metal detector to find, especially
when buried. Mines placed on or protruding above the ground surface
do not pose the same detection problem as buried mines because it is
possible that they could be detected visually.
The typical portable metal detector uses electromagnetic induction
technologies to find metal objects at or below the ground surface.
These detectors induce a magnetic field, which in turn causes a
secondary magnetic field to form around nearby objects that have
conductive properties such as the metal in landmines. An object's
detectability is a function of the induced magnetic field's strength
and an object's conductive properties, size, shape, and position.
For example, copper, aluminum, and ordinary steel are good conductors
and relatively easy to detect. Stainless steel is harder to detect
than an identical piece of ordinary steel because it offers more
resistance to the induced magnetic field and thus produces a weaker
or smaller secondary magnetic field. Portable metal detectors
operate on either the continuous wave or pulse method of transmitting
and receiving. Continuous wave detectors induce and monitor magnetic
fields continuously to sense any disruptions caused by a conductive
object's secondary field; pulse detectors transmit and receive in
alternating cycles in search of secondary magnetic fields.
In 1962, the Army fielded the AN/PSS-11, a continuous wave portable
mine detector. The last AN/PSS-11s were purchased in 1972. In the
late 1970s, the Army began a program to improve the AN/PSS-11's
durability and maintainability by replacing its outdated electronics.
As the lead service for the Department of Defense (DOD), the Army
developed such a detector, tested it successfully, and approved its
production in 1984. However, separate attempts to produce the
detector to Army specifications--in 1985 with one manufacturer and in
1987 with another--failed due to the manufacturers' technical or
financial problems. As the AN/PSS-11 became increasingly more
difficult to support due to the unavailability of replacement parts,
the Army was faced with a shortfall. In May 1990, the Army decided
to forgo development of a new or improved detector and instead to
purchase a commercially available detector as an interim solution to
its immediate shortfall.
After screening 12 commercially available metal detectors for
sensitivity, suitability, and availability, the Army narrowed the
field to two candidates--the AN-19/2 pulse detector made by Schiebel
GmbH of Austria and the Metex 4.125 continuous wave detector made by
Foerster Instruments, Inc. In March and July 1991, the Army awarded
contracts to the respective manufacturers for test articles, with
options for future buys. In December 1991, the Army selected the
Schiebel detector to replace the AN/PSS-11 and designated it as the
AN/PSS-12 Handheld Metallic Mine Detector. By the time the contract
expired in March 1996, 18,235 AN/PSS-12s had been ordered and all but
a few hundred had been delivered. The total cost of the detectors,
including support items, came to $21.9 million. Of the total, 15,553
are for the Army, 571 for the Marine Corps, 326 for the Air Force,
323 for the Navy, and 1,462 for foreign military sales. As of March
1996, the Army had sent 261 of the detectors to Bosnia. The Air
Force has also sent AN/PSS-12s to Bosnia. Until 2001, when the Army
plans to field a new portable detector it is developing for
low-metallic and nonmetallic mines, the AN/PSS-12 will remain the
Army's primary portable mine detector.
--------------------
\1 Unexploded Ordnance: A Coordinated Approach to Detection and
Clearance Is Needed (GAO/NSIAD-95-197, Sept. 20, 1995).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
The ability of the Army's AN/PSS-12 to detect low-metallic content
mines has not been clearly demonstrated. The AN/PSS-12 performed
poorly against low-metallic targets in the operational tests leading
to its 1991 procurement. It did not perform as well as either the
aging AN/PSS-11 or the Foerster detector against low-metallic
targets, although none met the Army's 92-percent detection
requirement. As a result of its decisions to (1) remove the only
target that met the definition for low-metallic content and (2) add a
large metal washer to the next lowest metallic content target, the
Army eliminated low-metallic targets from the tests and from its
evaluation of the two detector candidates. Since both candidates
performed equally well against the higher metallic targets remaining
in the tests, the Army selected the Schiebel because of its lower
price. In addition to the poor portrayal of low-metallic mines from
test to test, the Army has not sufficiently controlled other factors
that can affect detector performance, such as soil type and operator
proficiency. Not controlling these factors can impair the tests'
usefulness as a predictor of performance in the field.
The AN/PSS-12's testing history indicates that the detector may offer
only a limited capability in Bosnia, where the majority of buried
mines are of low-metallic content. Army officials informed us that
Army units have only used the AN/PSS-12 on a limited basis in Bosnia.
They stated that (1) the AN/PSS-12 is performing well in Bosnia, and
the division engineer there expressed no interest in enhancements to
the detector; (2) several other countries chose the AN/PSS-12 and two
countries successfully used it in the former Yugoslavia, before U.S.
troops deployed there; and (3) the aluminum in the low-metallic
content mines in Bosnia enhances their detectability by the
AN/PSS-12. However, information from other sources raises questions
about the AN/PSS-12's potential performance in Bosnia. The Air Force
recently cautioned its explosive ordnance disposal technicians in
Bosnia that the AN/PSS-12 does not have the sensitivity to detect
low-metallic mines they may encounter. Some countries that
originally procured the Schiebel detector have since switched to
other detectors. An Army after-action report on U.S. operations in
Somalia states that the detector could not find low-metallic content
mines. Moreover, we did not find data to confirm that the aluminum
found in the mines buried in Bosnia is more detectable than the steel
targets used in Army tests.
We believe that the more important factor in explaining the
AN/PSS-12's performance in Bosnia to date has been the prudent steps
taken by the Army to minimize the threat posed by the landmines
there. In addition to receiving extensive mine-awareness training,
U.S. troops have been able to pick routes that either avoid
minefields or use heavy equipment, such as vehicles equipped with
rollers, to clear paths. The resultant infrequent reliance on the
AN/PSS-12 helps explain why the shortcomings it exhibited against
low-metallic targets in testing may not have been exhibited thus far
in Bosnia.
LOW-METALLIC CONTENT MINES ARE
A KNOWN THREAT
------------------------------------------------------------ Letter :3
According to defense intelligence information, low-metallic content
mines have been a recognized threat for the last 14 years and are a
prevalent threat in Bosnia. Low-metallic mines have been represented
in Army tests of portable and other detection systems since 1983 and
were included in the performance specifications used for the 1991
procurement of the AN/PSS-12. Army officials informed us that a
separate technology effort was underway before 1991 to address the
low-metallic and nonmetallic threat. The Army plans to complete this
effort by fiscal year 2001.
LOW-METALLIC MINES ARE
PREVALENT IN BOSNIA
---------------------------------------------------------- Letter :3.1
According to the National Ground Intelligence Center,\2 mines with
minimal metal content were first fielded in the early 1950s. For
years, however, no criterion or standard existed for defining a mine
as having low-metallic content. In the early 1980s, the Center
established the U.S. M-19 mine, which contains 2.46 grams of metal,
as the threshold for low-metallic mines. By this standard, only
mines containing 2.46 grams of metal or less are considered as
low-metallic threats.
According to intelligence reports, over half of the landmines in
Bosnia are buried, and about 75 percent of them are low-metallic
mines. The metal content of these mines is confined to the aluminum
casing around their chemical action fuzes. About eight different
types of Yugoslav landmines with this type of fuze have been
identified. The Center reported that some former Yugoslav mines
containing no metal were known to have been manufactured. These
mines' fuzes are wrapped in plastic and would not be detectable by
the AN/PSS-12 or any other metal detector. However, the mines
recovered so far have all contained aluminum-clad fuzes.
Fuzes used in some of these mines contain between 0.4 and 1.5 grams
of aluminum. Depending on the type, these mines may contain from one
to three fuzes, any one of which is capable of detonating the mine.
Examples include the TMA-1 and TMA-5, which contain one fuze; the
TMA-2, which contains two fuzes; and the TMA-3 and TMA-4, which
contain three fuzes. The most difficult to detect are the PMA-1,
which contains less than 0.4 grams of aluminum in its fuze, laid
horizontally in the mine, and the PMA-2, which has a vertical fuze (a
more difficult position for detection) containing 0.5 grams of
aluminum. For detection purposes, the metallic content of multiple
fuzes is not additive; according to Army officials, the fuzes are
positioned far enough apart in the mine as to generally limit
detection to one fuze at a time.
--------------------
\2 The National Ground Intelligence Center, located in
Charlottesville, Va., is the U.S. authority for assessing the threat
posed by all foreign mines. Before October 1994, it was known as the
Foreign Science and Technology Center.
LOW-METALLIC MINE TARGETS
WERE INCLUDED IN
REQUIREMENTS FOR THE 1991
PROCUREMENT
---------------------------------------------------------- Letter :3.2
According to Army officials, the purpose of the 1991 procurement was
to buy a detector with performance equal to or better than the
AN/PSS-11. The detection and other performance requirements for the
1991 procurement were contained in a modified military specification
associated with the earlier attempt to develop an improved version of
the AN/PSS-11. This specification required that the detector have a
greater than 92-percent probability of detecting metallic mines and
mines with small metallic content.\3 The specification described the
following targets to be detected in three different types of
soils--sand, loam, and magnetite (an iron-based soil):
-- a small steel pin, 4.5 millimeters long, to simulate the M-14
mine (detection of this pin was desired but not required in
magnetite);
-- a hollow aluminum tube, 44.5 millimeters long and 6.4
millimeters in diameter;
-- a steel PMN-6 striker pin, 57 millimeters long, one-third of
which was
4.8 millimeters in diameter and the remainder 9.5 millimeters in
diameter; and
-- a simulated M-16 mine.
According to the 2.46 gram standard, the M-14 pin and the aluminum
tube represented low-metallic targets. The M-16 is a metal-clad
mine. The PMN-6 striker pin falls somewhere between the M-16 and the
low-metallic targets. The designation PMN-6 refers to a British-made
training mine that is a replica of the Soviet PMN mine. Like the
Soviet mine, the PMN-6 training mine has a nonmetallic case and
contains several metal components in addition to the striker pin,
which collectively amount to over 17 grams of metal. According to
the National Ground Intelligence Center, the striker pin itself would
not qualify as a low-metallic target because it contains several
times the amount of metal as the M-19.
--------------------
\3 This requirement falls between the nearly 100-percent detection
required for mine clearance operations, like those in Bosnia, and the
80-percent detection required for rapidly cutting through minefields
under combat conditions.
ARMY PROGRAM IS AIMED AT
DETECTING LOW-METALLIC AND
NONMETALLIC MINES
---------------------------------------------------------- Letter :3.3
According to Army officials, the Army began developing other
technologies in the mid-1980s to detect low-metallic and nonmetallic
mines. Under a program now known as the Handheld Standoff Mine
Detection System, a detector is being developed that integrates
ground-penetrating radar, infrared, and metal detection technologies,
along with electronics that are intended to synthesize and interpret
the signals from the three sensors for the operator. The detector is
now in competitive prototype testing and is slated for a production
decision in fiscal year 2001. A gap in detection capability against
low-metallic and nonmetal mines may remain until then. Our September
1995 report on unexploded ordnance provides additional information on
these technologies.
To provide some additional capability for U.S. forces in Bosnia, the
Army is evaluating commercially available detectors that combine
technologies such as ground-penetrating radar with electromagnetic
induction methods. These detectors do not possess all of the
capabilities planned for the detector in prototyping. According to
Army officials, recent tests of such systems demonstrated a
70-percent detection capability against low-metallic and nonmetallic
mines. The Army does not consider this detection rate acceptable for
use by troops in the field. Further testing is planned.
AN/PSS-12 DETECTOR PERFORMED
POORLY AGAINST LOW-METALLIC
MINES IN OPERATIONAL TESTING
------------------------------------------------------------ Letter :4
The Army's Test and Experimentation Command, under the auspices of
the Operational Test and Evaluation Command, conducted two
operational tests during 1991 to assess the performance of the
candidate metal detectors in a field environment. Short of war,
operational testing is the most realistic way of assessing a system's
effectiveness and suitability for fielding. However, the operational
tests had several shortcomings that complicate the assessment of the
comparative performance of the two detector candidates and the
baseline AN/PSS-11 against low-metallic mines. In the first test,
the Schiebel detector found 3.4 percent of the low-metallic targets,
compared with 24.2 percent for the AN/PSS-11. Because the Foerster
detector was not included in the first test and the low-metallic
targets were excluded from the second test, the Foerster was not
tested against these low-metallic targets and no comparison could be
made. The Foerster detected twice as many of the lowest metal
content targets present at the beginning of the second test, but the
Army concluded the targets were not representative of the higher
metallic content Soviet mine and ruled them invalid. As performance
against higher metallic targets was equal, price became the deciding
factor in the procurement decision.
MARCH 1991 OPERATIONAL
TESTING
---------------------------------------------------------- Letter :4.1
The first operational test of the detector candidates was conducted
during March 20-28, 1991. Prior to this test, the Army had screened
12 different commercial detectors and had eliminated all but one
because of (1) technical, performance, or production shortcomings or
(2) high prices. Two Foerster candidates were among the detectors
eliminated on the basis of price. Thus, the Schiebel was the only
detector forwarded for operational testing with the baseline
AN/PSS-11 detector.
This test included four target types: metal-clad training M-15 and
M-16 mines, the M-14 pin, and the PMN-6 striker pin. Given that the
hollow aluminum tube described in the specification was not used in
the test, the M-14 pin was the only low-metallic target. The results
of the test are shown in table 1.
Table 1
Detection Results From March 1991
Operational Test
Percen
t
detect
Landmine target Detector ed
------------------------------ ------------------------------ ------
M-15 metal-clad training mine Schiebel 99.46
AN/PSS-11 99.71
M-16 metal-clad training mine Schiebel 99.43
AN/PSS-11 99.50
PMN-6 steel striker pin Schiebel 59.64
AN/PSS-11 80.83
M-14 steel pin Schiebel 3.38
AN/PSS-11 24.15
----------------------------------------------------------------------
Using the 92-percent detection requirement, the Army Test and
Experimentation Command concluded that the Schiebel was not an
effective mine detector and stated:
It is strongly recommended that the Government not purchase this
mine detector as a replacement for the AN/PSS-11 at this time.
Rather, another survey should be conducted to identify candidate
mine detectors that meet military specifications outlined in the
test and evaluation master plan. Further technical and
operational testing should result in a more suitable replacement
mine detector.
In a separate evaluation of the test, the Army Test and Evaluation
Command, under the auspices of the Army Materiel Command, found that:
against mines with small metallic content (i.e., the M-14 and
the PMN-6), the AN-19/2 [Schiebel] fell considerably short of
the PS [performance specification]
requirement. . .Indeed, its performance during test was
distinctly inferior to that of the AN/PSS-11 under the same
conditions, although the AN/PSS-11 itself did not meet the PS
requirement either.
The Test and Evaluation Command did state that the procurement
decision should not depend too heavily on the detectors' inability to
detect low-metallic mines because such mines were just a step away
from nonmetal mines, which would render a metallic mine detector
useless. Nonetheless, the Command recommended that the Army (1) not
approve the Schiebel for fielding as the AN/PSS-12 and (2) reexamine
the role of the mine detector in the Army and confirm that the
detection of mines with small metallic content remained a valid need.
Ultimately, the Army decided to eliminate the M-14 target from
further testing because it concluded that the target's metal content
was so low that it was essentially nonmetal. It was not replaced
with another low-metallic target. Army officials informed us that
the user representative at the time did not want to reject the
Schiebel on the basis of its performance against less lethal mines
such as the M-14--considered likely to injure, rather than kill--if
it could detect more lethal mines that could kill several
individuals. This was a significant decision because the M-14 pin
had been cited in the performance specification and had been used in
Army testing of portable mine detectors since 1983. Such testing
included the attempted product improvements of the AN/PSS-11 and the
original screening of the 12 commercial detector candidates.
The Army realized in 1992 that the M-14 pin contained only a
portion--
0.29 grams--of the total metal in the M-14 mine. According to
testing conducted in 1996, the actual mine is more detectable than
the target used. Had the Army known this at the time of the 1991
testing, it may have been able to substitute a more authentic
low-metallic mine target.
SEPTEMBER-OCTOBER 1991
OPERATIONAL TESTING
---------------------------------------------------------- Letter :4.2
Following the filing of a bid protest, the Army decided to readmit
one of Foerster's two original candidates to the competition and
therefore it participated in the second operational test. The second
operational test was held from September 17, 1991, to October 4,
1991. It included three examples each of the Foerster, Schiebel, and
AN/PSS-11. As in the previous test, this test used targets
designated as PMN-6s to simulate low-metallic mines. However, the
second test used PMN-6 training mine casings, which contained the
steel striker pin, a spring, and a small washer. Shortly before the
test began, representatives from the program manager's office and the
U.S. Army Engineer School (which represented the user) contended
that the PMN-6 target did not contain as much metal as a real Soviet
PMN mine. They stated that metal would have to be added to the PMN-6
test targets already buried to make them realistic.
However, the purpose of the target was not to replicate the Soviet
mine. In fact, the test report indicated that the purpose of the
PMN-6 striker pin was to simulate the M-19 mine. The Soviet mines
that the PMN-6 was modeled after are not low-metallic mines. The
National Ground Intelligence Center reports that no Soviet landmine
contains less than
8 grams of metal, which is more than the 2.46-gram threshold. While
it would have been reasonable to ensure that the target was a fair
replica of either the M-19 low-metallic mine or the striker pin
described in the specification, it was not reasonable to insist that
the target replicate the Soviet mine. The test team maintained that
adding metal to the PMN-6 target could make its detectability climb
to 100 percent; thus, there would be no way to discriminate one
detector's performance from another's. Ultimately, it was agreed
that a 2-inch metal washer would be added to each PMN-6 target. This
was done by inserting the washers beneath the surface and on top of
the buried targets, without digging them up. Because the M-14
low-metallic target had already been eliminated, adding metal to the
PMN-6 was a key decision because it effectively eliminated the only
remaining target the test team considered to have a metal content low
enough to differentiate the performance of one detector from another.
The Test and Experimentation Command had planned a 1-day pilot test
to work out procedures and firm up preparations for the operational
test. The Command decided to conduct the pilot test with the PMN-6
targets in their original condition--without the large washer. Table
2 shows the results of the pilot test.
Table 2
Detection of PMN-6 Mine Targets During
September 1991 Pilot Test
Percen
t
detect
Detector ed
-------------------------------------------------------------- ------
Foerster 66.67
Schiebel 32.22
AN/PSS-11 28.89
----------------------------------------------------------------------
The percentages shown above are the averages for the three detectors
of each type used. The best performance by a Foerster was 76.67
percent; by a Schiebel, 43.33 percent; and by an AN/PSS-11, 43.33
percent. While these results were included in the test report, they
were excluded in the analysis of operational test results for the
procurement decision on the flawed basis that the target was
unrepresentative of a Soviet mine.
After the pilot test, the second operational test was conducted with
the PMN-6 targets augmented with the large metal washers. The other
two mine targets included in the test were metal-clad mines and thus
had high metal content. These were the M-8, a training version of
the M-16 mine, and the TMN-46, a training version of a Soviet
antitank mine. Table 3 shows the results of the test.
Table 3
Detection Results From the September/
October 1991 Operational Test
Percen
t
detect
Landmine target Detector ed
------------------------------ ------------------------------ ------
M-8 metal-clad training mine Schiebel 99.91
Foerster 99.82
AN/PSS-11 99.64
TMN-46 metal-clad mine Schiebel 99.86
Foerster 100.00
AN/PSS-11 99.59
PMN-6 training mine, augmented Schiebel 99.58
with washer
Foerster 99.09
AN/PSS-11 98.51
----------------------------------------------------------------------
These results showed that against the high-metallic mine targets
remaining in the operational test, all three detectors found
virtually all the mines and passed the Army's 92-percent detection
requirement. The results also confirmed the test team's concern that
adding metal to the PMN-6 target could cause detection percentages to
climb to 100 percent for all the detectors.
ARMY DECISION TO PROCURE
SCHIEBEL DETECTOR DID NOT
WEIGH PERFORMANCE AGAINST
LOW-METALLIC MINES
---------------------------------------------------------- Letter :4.3
The Army's decision to procure the Schiebel was based on detection
performance against only the high metal content mines. In a December
13, 1991, memorandum, the Chairman of the Source Selection Board in
charge of selecting the best detector candidate concluded that the
performance difference between the detectors was not significant and
that the additional cost of the Foerster was not justified by any
significant increase in technical or operational benefit.
Army officials informed us that because none of the detectors,
including the AN/PSS-11, could meet the 92-percent requirement
against low-metallic mines, they were equally unable to
satisfactorily detect such mines. Therefore, the ability to detect
low-metallic mines was no longer a discriminating factor in selecting
a replacement for the AN/PSS-11. Nonetheless, in the only comparable
operational test, the Foerster detector demonstrated a significantly
better ability to detect the lowest metal mine target--the pilot
test's PMN-6 target--than the Schiebel detector. Whether the
Foerster's better performance in the pilot test was worth its higher
price was not assessed because low-metallic mines had already been
eliminated as a factor by the time the decision was made.
VARYING TEST CONDITIONS MAKE
COMPARISON OF TEST RESULTS
DIFFICULT
------------------------------------------------------------ Letter :5
Over the years, the Army has gathered performance data on portable
mine detectors from a number of sources, including technical tests,
operational tests, demonstrations, and from actual use in operations,
such as in Somalia. Regardless of how data is gathered, the
performance of portable mine detectors is affected by several factors
that, if not controlled, make it difficult to compare one test or
operation with another. In the numerous tests and demonstrations of
portable mine detectors conducted since 1983, these factors have not
been held to consistent, realistic, or technically sound standards.
The factors include target type, target burial depth and position,
soil type and moisture content, and the distance between the detector
head and ground surface. Performance is also affected by the
proficiency of the operator, including such factors as maintaining
the correct height and speed of the detector head as it is swept back
and forth in the search for targets, the level of training, and the
operator's ability to pick up audio and visual cues that can help
indicate the presence of a mine. In addition, as suggested by test
results, different detectors of the same model can vary in
performance.
While tests, by their nature, are conducted under controlled
conditions to provide for valid data collection and analysis,
technical and operational tests are conducted under different
circumstances and are interpreted differently. Technical testing is
intended to determine the technical capabilities of a detector under
ideal conditions. While such testing can eliminate detectors that do
not have the ability to meet performance requirements, it is not
intended to assess performance under field conditions. Operational
testing is much more realistic than technical testing, as it can
introduce more factors that affect performance results, most
importantly, the operator-machine interface. The two operational
tests of portable mine detectors the Army conducted in 1991 are
illustrative of how difficult it is to isolate detector performance
from other factors when comparing test results. Their test
conditions are compared in table 4.
Table 4
Test Conditions During 1991 Operational
Testing
Condition March 1991 test Sept.-Oct. 1991 test
---------------------- ---------------------- ----------------------
Location Ft. Hunter-Liggett, Ft. Leonard Wood, Mo.
Calif.
Soil type Sandy loam Red clay
Temperature range 40-55 degrees 55-75 degrees
Fahrenheit Fahrenheit
Detectors tested AN/PSS-11, Schiebel AN/PSS-11, Schiebel,
Foerster
Targets used M-14 pin, PMN-6 Simulated Soviet PMN
striker pin, metal- mine, metal-clad M-8
clad M-15 and M-16 and TMN-46
Specialty of operator Varied Combat engineers\a
Operator training for 12-14 hours per 40 hours per operator
test operator
----------------------------------------------------------------------
\a Combat engineers are trained in the use of countermine equipment.
Some tests are actually demonstrations, which fall somewhere between
technical and operational testing, although they do not necessarily
provide the discipline or data to support statistically valid or
independent data analysis. Demonstrations of portable mine detectors
have been conducted in a field environment; however, the detectors
have been operated by contractor personnel or Army civilian
personnel. Again, as in operational testing, they must contend with
a variety of factors that can affect detector performance. While
demonstrations do not enable conclusions to be drawn about a
detector's ability to meet military requirements, they are a vehicle
for quickly gauging a detector's potential performance in the field.
While the use of portable mine detectors in actual operations
provides realistic information on detection performance, the number
of mines detected are not usually recorded, and the number of mines
missed, absent maps and records, may not be known. Results can also
be site specific as to soil type, moisture content, and temperatures.
Thus, these operations do not lend themselves to quantification of a
detector's performance. Moreover, one mishap can prove fatal. The
AN/PSS-12 was used by U.S. forces in Somalia and by U.S.
contractors in Kuwait and is currently deployed with U.S. forces in
Bosnia. An Army after-action report from operations in Somalia
states that the AN/PSS-12 could not detect low-metallic mines, but it
offers no specifics on the detection shortfalls. Although many
landmines were reportedly found by U.S. contractors in Kuwait using
the Schiebel and other metal detectors, the fact that they were
buried in sand and in patterns made them easier to find than might be
the case in other situations. These operations do not provide
information on the percentage or number and types of mines that were
found by the metal detectors, nor do they indicate what mines were
not detected.
The performance of the AN/PSS-11 in several tests conducted since
1983 illustrates how the measured performance of a detector can vary
from one test to the next. In a 1983 field test outdoors at Fort
Belvoir, Virginia, the AN/PSS-11 detected 80 percent of M-14 mine
targets. In 1985 testing at the Fort Belvoir indoor mine lane
facility, prototypes of the product-improved version of the AN/PSS-11
detected none of the M-14 targets buried in sand and 67 percent of
the M-14 targets on the surface. In a 1988 field test to establish
the AN/PSS-11's detection capabilities as a standard for an Army
development of a vehicle-mounted detection system, the AN/PSS-11
detected 82.5 percent of buried M-19 mine targets. As stated
previously, the AN/PSS-11 detected 24.2 percent of M-14 targets, 80.8
percent of PMN-6 striker pins, and 28.9 percent of PMN-6 targets
(without the large washer) in the 1991 operational tests. The data
from these various sources defy a definitive conclusion on the
performance of a detector that has been in the Army's inventory for
30 years.
NO PROBLEMS REPORTED WITH THE
LIMITED USE OF THE AN/PSS-12 IN
BOSNIA
------------------------------------------------------------ Letter :6
According to the Army, U.S.troops have not experienced problems with
the AN/PSS-12 in Bosnia. Army officials have cited the successful
use of the detector by other countries and the detectability of
low-metallic mines in Bosnia as further evidence of the AN/PSS-12's
potential for performance there. However, this information is not
consistent with the Army's 1991 test results and information from
other sources. Consequently, we believe the potential effectiveness
of the AN/PSS-12 against low-metallic mines in Bosnia is
inconclusive. The steps the Army has taken to minimize the threats
posed by landmines there and the resultant infrequent reliance on the
AN/PSS-12 may help to explain why the detector's poor performance
against low-metallic targets in testing has not been exhibited in
Bosnia.
PERFORMANCE OF THE AN/PSS-12
IN BOSNIA IS INCONCLUSIVE
---------------------------------------------------------- Letter :6.1
While the Army does not know the percentage of each type of mine
detected by the AN/PSS-12 since deploying to Bosnia, officials said
that when the detector has been used it has worked well. As of July
1996, they reported that no U.S. troop casualties had occurred as a
result of the detector's having failed to detect a mine in Bosnia.
Army officials noted that several other countries purchased the
Schiebel detector before the United States, including Germany,
Canada, Israel, Sweden, and the United Kingdom. They said that
Canada and Sweden successfully used the Schiebel in the former
Yugoslavia before the U.S. troops deployed there. The Schiebel was
the detector of choice by contractors that conducted mine-clearing
operations in Kuwait and by the United Nations in several of its
demining operations. According to Army officials, its division
engineer in Bosnia was not interested in any performance enhancements
as the AN/PSS-12 was performing fine.
The Army also believes that the mines found so far in Bosnia have had
enough metal content to be detectable by the AN/PSS-12. While these
mines are classified as low-metallic mines, they reportedly contain
more metal than the M-14 target used in the March 1991 operational
test. More importantly, the metal contained in the Bosnian mines is
aluminum. Because aluminum is lighter than steel, a piece of
aluminum that weighs the same as a piece of steel would be
considerably larger. Thus, according to Army officials, the aluminum
in the Bosnian mines not only weighs more than the M-14 test
target--it would be physically larger as well.
Other information clouds an overall picture of the AN/PSS-12's use in
operations. During the course of our review, we learned that Germany
has decided to replace its Schiebel detectors with a detector made by
Vallon GmbH of Germany, and the Netherlands is using a Foerster
detector in Bosnia. In 1993, the United Kingdom replaced its
Schiebel detectors in Cambodia. A State Department official
assisting with the international humanitarian demining effort in
Bosnia informed us that the AN/PSS-12 is used only in conjunction
with probes (pointed rods used by hand). The Marine Corps informed
us that it prefers the technology of the AN/PSS-11 and currently uses
the old detector in Guantanamo Bay, Cuba. In April 1996, the Air
Force issued a message to its explosive ordnance technicians deployed
in Bosnia to clear landmines and other explosives from airfields,
cautioning them that the AN/PSS-12 does not have the sensitivity to
detect low-metallic mines they may encounter. The Air Force is
processing an urgent contracting action to purchase another metal
detector to replace its AN/PSS-12s in Bosnia. This action is
unrelated to the Army's near-term effort to evaluate commercial
detectors that combine technologies for potential application to
Bosnia.
We attempted to verify that the aluminum found in mines in Bosnia was
in fact more detectable than the steel targets used in the 1991
testing. We contacted several countermine, testing, and explosive
ordnance organizations within the services and none reported that
they had developed credible data on the comparative detectability of
different metals. At our request, a manufacturer of measurement and
detection equipment compared the detectability of an aluminum target
approximating a 0.4-gram aluminum fuze found in Bosnian mines with a
steel target approximating the M-14 pin used in Army's tests. The
comparison did not show that the aluminum target was unequivocally
more detectable than the steel target. We did not attempt to assess
how the detectability of a 1- to 1.5-gram piece of aluminum found in
some mines in Bosnia would compare with the more substantial PMN-6
striker pin used in testing.
ARMY HAS TAKEN STEPS TO
MINIMIZE THE THREAT POSED BY
LANDMINES IN BOSNIA
---------------------------------------------------------- Letter :6.2
According to information we obtained from Department of State,
Defense Intelligence Agency, and Army officials, several factors have
minimized the risks landmines pose to U.S. troops in Bosnia. These
steps include the following:
-- The former warring parties, who are responsible for removing
landmines, have provided maps, when available, of mined areas so
that these areas can be avoided.
-- For the most part, landmines are believed to be concentrated in
known zones of separation that formerly existed between the
warring factions. These zones are avoided when possible.
However, a State Department official said less is known about
the landmine threat outside these zones.
-- Because U.S. forces are not taking ground as they would in a
combat situation, they can move along established routes or
roads. This gives combat engineers the opportunity to run
rollers down the routes several times to detonate mines before
any attempts are made at dismounted mine detection. Most main
routes are believed to be safe.
-- Some mine survey, route clearance assurance, and site clearance
work has been contracted out.
According to the State Department, areas considered cleared by the
former warring parties must still be verified by peacekeeping forces.
This is because the warring parties (1) are responsible for clearing
areas only within the first 30 days after turning the areas over, (2)
do not necessarily have the best mine detection and clearance
equipment or training, and (3) did not prepare many maps of mined
areas.
Army officials have described their approach to mined areas in Bosnia
as follows. All personnel are provided with extensive mine-awareness
training before going into the theater. Before U.S. forces move
into an area, an intelligence assessment is made. Discussions are
held with the former warring parties to determine whether the area is
mined and if so, what kinds of mines were used. At a more detailed
level, some exploration may be done by engineers using probes to find
sample mines. The troops can then verify whether the mines are
consistent with the initial assessment. Data sheets on the threat
mines are available that describe the characteristics of each mine
and help make an accurate identification. If an area cannot be
accessed by rollers, the combat engineers then assess whether the
mines found can be detected with the AN/PSS-12. Army officials said
they do this by actually checking the detector against the sample
mines found in the ground. If the mines can be detected with the
AN/PSS-12, then U.S. troops can go in dismounted. If the mines are
not detectable, U.S. troops do not go in dismounted. As a last
resort, probes could be used.
RECOMMENDATION
------------------------------------------------------------ Letter :7
Had the 1991 operational testing properly portrayed low-metallic
mines, the Army may have had greater assurance that the detector it
selected as the AN/PSS-12 was the best choice at the time against the
full range of landmines. The limitations of this testing are perhaps
more apparent now than at the time; while the testing became focused
on higher metallic content mines, low-metallic content mines are
prevalent in Bosnia. Although testing may not be able to replicate
all of the conditions expected in actual operations, it should
provide a sound assessment of detection and other performance
capabilities that can serve as a consistent baseline for comparing
results from test to test. Because the 1991 testing did not provide
such a foundation, an assessment of the AN/PSS-12's performance in
Bosnia or any operation is perhaps more subjective than it should be.
Accordingly, we recommend that the Secretary of Defense establish and
enforce realistic and consistent test standards for testing
countermine and mine detection systems that reflect known threat
mines and the conditions under which they are likely to be
encountered. Such standards should be applied not only to the
acquisition of new systems but to the evaluation of near-term or
experimental solutions as well.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :8
DOD concurred with our recommendation to establish realistic and
consistent test standards. It also noted that the research,
development, testing, and evaluation of countermine and mine
detection systems were being reviewed by an unexploded ordnance
clearance executive committee and steering group (see app. I).
Although DOD concurred with our recommendation, it stated that the
soldiers in Bosnia are not in danger due to the performance of the
AN/PSS-12 in the presence of low-metallic mines and disagreed with
any implication to the contrary. DOD reiterated that U.S. forces
avoid mines when possible, using devices such as rollers and probes
in addition to the AN/PSS-12 when mines are encountered, and that
other countries selected the same detector before the Army did.
These points were covered in the draft report. The information
available to date supports DOD's characterization of the relative
safety of U.S. forces operating in the presence of landmines in
Bosnia. The analytical dilemma is in reconciling the poor
performance of the AN/PSS-12 against low-metallic targets in
operational testing with its reported satisfactory performance in
Bosnia where low-metallic mines are prevalent. We believe it is the
prudent steps taken by the Army to avoid and minimize the landmine
threat in Bosnia--more so than the capability of the AN/PSS-12 or the
detectability of the low-metallic content mines there relative to the
test targets--that explains the difference between the detector's
performance in operational testing with its experience in Bosnia.
DOD also noted that an independent technical test conducted in June
1996 within DOD shows that the AN/PSS-12 can consistently detect M-14
low-metallic mines when inert mines are used instead of targets. The
data from this test indicates that the inert M-14 mine is more
detectable by the AN/PSS-12 than the M-14 firing pin first used and
later removed as a target in the 1991 operational testing, although
no detection percentages were obtained to measure consistency. The
improvement is attributed to the fact that the inert mine contains
more metal than the firing pin.
The June 1996 test does raise additional questions about the
usefulness of the information obtained in Army testing since 1983
that used the M-14 firing pin as a target. However, it does not
supplant the 1991 operational test results because it was a limited
technical test and was not intended to replicate a realistic
environment. In the June 1996 test, landmines were not buried but
placed on the ground with the detectors held directly over them. The
essence of the test was to lower the detector over the mine and
record the distance at which the detection was made; no searching was
involved. By comparison, in the pilot test phase of the
September-October 1991 operational test, the Schiebel detector found
only 32.2 percent of the PMN-6 targets, which contained significantly
more metal than the inert M-14 mine. The need to put the June 1996
test results into the proper perspective underscores the value of
establishing realistic and consistent test standards.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :9
To obtain information for this report, we reviewed numerous documents
relating to the test and evaluation of portable mine detectors,
including several military services' test reports since 1983, the
contract file on the AN/PSS-12 procurement, files from previous
investigations of the AN/PSS-12 procurement conducted within DOD, the
after-action report on Somalia, threat publications prepared by the
National Ground Intelligence Center, Army, Navy, and U.S. Marine
Corps evaluations, and evaluations conducted by the Naval Explosive
Ordnance Disposal Technology Division.
We interviewed officials from the Office of the Secretary of Defense;
the Departments of State, the Army, the Navy, and the Air Force; the
U.S. Marine Corps; the Defense Intelligence Agency; the National
Ground Intelligence Center; and the Joint Naval Explosive Ordnance
Disposal Technology Division. We also interviewed current and former
Army program officials, representatives from the Army contracting
office at the Army Aviation and Troop Command, current and former
Army user representatives from the Army Engineer School,
representatives from the Army Test and Experimentation Command
involved with the conduct of both operational tests, and a
representative from the Army Waterways Experimentation Station that
supplied PMN-6 mines for the second operational test. We did not
visit Bosnia-Herzegovina, but information was obtained from Army
officials in direct contact with units there and from other sources
as indicated. We also interviewed representatives from detection
equipment manufacturers and, at our request, the Canadian firm,
Geonics, Ltd., conducted a laboratory test to compare the
detectability of steel and aluminum targets.
We conducted our review from December 1995 to July 1996 in accordance
with generally accepted government auditing standards.
---------------------------------------------------------- Letter :9.1
We are sending copies of this report to other interested
congressional committees and the Secretary of Defense. We will also
make copies available to others upon request.
Please contact me at (202) 512-5140 if you or your staff have any
questions concerning this report. Major contributors to this report
were
Paul L. Francis and James B. Dowd.
Sincerely yours,
Mark E. Gebicke, Director
Military Operations and
Capabilities Issues
(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
============================================================== Letter
(See figure in printed edition.)
*** End of document. ***