Combating Nuclear Smuggling: DHS's Decision to Procure and Deploy
the Next Generation of Radiation Detection Equipment Is Not
Supported by Its Cost-Benefit Analysis (14-MAR-07, GAO-07-581T).
The Department of Homeland Security (DHS) is responsible for
addressing the threat of nuclear smuggling. Radiation detection
portal monitors are key elements in our national defenses against
such threats. DHS has sponsored R&D and testing activities to
develop a "next generation" portal monitor, known as the advanced
spectroscopic portal monitor. However, each one costs 6 times
more than a current portal monitor. In March 2006, we recommended
that DHS conduct a cost-benefit analysis to determine whether the
new portal monitors are worth the additional cost. In June 2006,
DHS issued its analysis. In October 2006, we issued our report
that assessed the DHS study. GAO's statement, based on our
October 2006 report, addresses whether DHS's cost-benefit
analysis provides an adequate basis for its decision to purchase
and deploy the next generation portal monitors.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-581T
ACCNO: A66824
TITLE: Combating Nuclear Smuggling: DHS's Decision to Procure
and Deploy the Next Generation of Radiation Detection Equipment
Is Not Supported by Its Cost-Benefit Analysis
DATE: 03/14/2007
SUBJECT: Comparative analysis
Cost analysis
Cost effectiveness analysis
Evaluation methods
Federal procurement
Financial analysis
Homeland security
Nuclear radiation monitoring
Policy evaluation
Radiation monitoring
Smuggling
Benefit-cost tracking
Cost estimates
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-581T
* [1]Background
* [2]DNDO Ignored Its Own Performance Test Results and Instead Re
* [3]DNDO's Cost-Benefit Analysis was Incomplete and Used Inflate
* [4]GAO Contact and Staff Acknowledgements
* [5]GAO's Mission
* [6]Obtaining Copies of GAO Reports and Testimony
* [7]Order by Mail or Phone
* [8]To Report Fraud, Waste, and Abuse in Federal Programs
* [9]Congressional Relations
* [10]Public Affairs
Testimony
Before the Subcommittee on Emerging Threats, Cybersecurity, and Science
and Technology; Committee on Homeland Security; House of Representatives
United States Government Accountability Office
GAO
For Release on Delivery Expected at 2:00 p.m. EST
Wednesday, March 14, 2007
COMBATING NUCLEAR SMUGGLING
DHS's Decision to Procure and Deploy the Next Generation of Radiation
Detection Equipment Is Not Supported by Its Cost-Benefit Analysis
Statement of Gene Aloise, Director
Natural Resources and Environment
GAO-07-581T
Mr. Chairman and Members of the Subcommittee:
I am pleased to appear here today to discuss our assessment of the
Department of Homeland Security's (DHS) May 2006 cost-benefit analysis
used to support the purchase and deployment of next generation radiation
portal monitors.1 This is an important decision because, if procured,
these new portal monitors will be considerably more expensive than the
portal monitors in use today. Combating nuclear smuggling is one of our
nation's key national security objectives and the deployment of radiation
detection equipment including portal monitors at U.S. ports of entry,
including border crossings and domestic seaports, is an integral part of
this system. DHS, through its Domestic Nuclear Detection Office (DNDO), is
responsible for acquiring and supporting the deployment of radiation
detection equipment, including portal monitors, within the United States.
The Pacific Northwest National Laboratory (PNNL), one of the Department of
Energy's (DOE) national laboratories, is under contract to manage the
deployment of radiation detection equipment for DHS.2 U.S. Customs and
Border Protection (CBP) is responsible for screening cargo as it enters
the nation at our borders, which includes operating radiation detection
equipment to interdict dangerous nuclear and radiological materials.
The radiation portal monitors in use today can detect the presence of
radiation, but they cannot distinguish between types of radiological
material. For example, they cannot tell the difference between harmless
products that emit radiation, such as ceramic tile, and dangerous
materials, such as highly enriched uranium (HEU), that could be used to
construct a nuclear weapon. Generally, CBP's standard procedures require
incoming cargo to pass through one of these radiation portal monitors to
screen for the presence of radiation. This "primary inspection" serves to
alert CBP officers when a radioactive threat might be present. If there is
a potential threat, CBP procedures require a "secondary inspection." To
confirm the presence of radiation, this secondary inspection usually
includes a second screening by a radiation portal monitor as well as a
screening by CBP officers using radioactive isotope identification
devices. These handheld devices are used to differentiate between types of
radioactive material to determine if the radiation being detected is
dangerous. Both the radiation portal monitors and handheld devices are
limited in their abilities to detect and identify nuclear material.
1GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support
the Purchase of New Radiation Detection Portal Monitors Was Not Based on
Available Performance Data and Did Not Fully Evaluate All the Monitors'
Costs and Benefits, [11]GAO-07-133R (Washington, D.C.: Oct. 17, 2006).
GAO, Combating Nuclear Smuggling: DHS Has Made Progress Deploying
Radiation Detection Equipment at U.S. Ports of Entry, but Concerns Remain,
[12]GAO-06-389 (Washington, D.C.: Mar. 22, 2006).
2DOE manages the largest laboratory system of its kind in the world. The
mission of DOE's 22 laboratories has evolved. Originally created to design
and build atomic weapons, these laboratories have since expanded to
conduct research in many disciplines - from high-energy physics to
advanced computing.
DHS would like to improve the capabilities of its portal monitors so that
they can perform the dual roles of detecting radiation and identifying
radiological materials. In this regard, DHS has sponsored research,
development, and testing activities designed to create the next generation
of portal monitors capable of performing both functions. These new,
advanced portals are known as advanced spectroscopic portals (ASPs). In
July 2006, DHS awarded contracts to three vendors for developing the
advanced spectroscopic portals' capabilities. These awards were based
mainly on performance tests conducted at DHS's Nevada Test Site in 2005,
where ten competing advanced spectroscopic vendors' monitors were
evaluated. At the same time, three current technology portal monitors were
also tested.
To ensure that DHS's substantial investment in radiation detection
technology yields the greatest possible level of detection capability at
the lowest possible cost, in a March 2006 GAO report,3 we recommended that
once the costs and capabilities of ASPs were well understood, and before
any of the new equipment was purchased for deployment, the Secretary of
DHS work with the Director of DNDO to analyze the costs and benefits of
deploying ASPs. Further, we recommended that this analysis focus on
determining whether any additional detection capability provided by the
ASPs was worth the considerable additional costs. In response to our
recommendation, DNDO issued its cost-benefit analysis in May 2006, and an
updated, revised version in June 2006. According to senior agency
officials, DNDO believes that the basic conclusions of its cost-benefit
analysis show that the new advanced spectroscopic portal monitors are a
sound investment for the U.S. government.
3GAO, Combating Nuclear Smuggling: DHS Has Made Progress Deploying
Radiation Detection Equipment at U.S. Ports of Entry, but Concerns Remain,
[13]GAO-06-389 (Washington, D.C.: Mar. 22, 2006).
Mr. Chairman, my remarks today focus on the cost-benefit analysis DNDO
used in support of its decision to purchase new ASP portal monitors.
Specifically, I will discuss whether DNDO's June 2006 cost-benefit
analysis provides an adequate basis for the substantial investment that
acquiring and deploying ASPs will necessitate.
My testimony is based upon our October 2006 report that evaluated DNDO's
cost-benefit analysis.4 The work for our report was done in accordance
with generally accepted government auditing standards.
In summary, DNDO's 2006 cost-benefit analysis does not provide a sound
analytical basis for its decision to purchase and deploy the new advanced
spectroscopic portal monitor technology.
Regarding the performance of the portal monitors:
o Instead of using the results of its performance tests conducted
in 2005, DNDO's analysis simply assumed that ASPs could detect
highly enriched uranium 95 percent of the time, a performance
level far exceeding the capabilities of the new technology's
current demonstrated capabilities. The 2005 test results showed
that the best of the three winning vendor monitors could only
identify masked HEU5 about 50 percent of the time.
o To determine the current generation of portal monitors'
performance in detecting HEU, DNDO used data from limited tests
carried out in 2004 that test officials concluded was unreliable
for such purposes. In their written report, test officials
explicitly stated that the data were not indicative of how well
current technology portal monitors might perform in the field
particularly for detecting HEU.
o DNDO's analysis of the new technology portal monitors'
performance was deficient because it focused on detecting and
identifying HEU, but did not fully consider other dangerous
radiological or nuclear materials. DNDO should have assessed the
ASPs' abilities to detect several realistic threat materials.
Regarding cost estimates:
o In comparing the costs of the new and current technologies, the
procurement costs of the current generation portal monitors were
highly inflated because DNDO assumed a unit cost of about
$131,000, while the contract price at the time of the analysis was
about $55,000. According to officials who manage the contract, it
was to expire and while they expected portal monitor prices to
increase, they did not believe the cost would be as much as the
price used in DNDO's analysis.
o DNDO stated that the primary benefit of deploying the new portal
monitors is reducing unnecessary secondary inspections. However,
DNDO's analysis did not fully estimate today's baseline costs for
secondary inspections, which makes it impossible to determine
whether the use of the new portal monitors as currently planned
will result in significant cost savings for these inspections.
o The new portal monitor contract price has exceeded DNDO's total
cost estimate by about $200 million. The cost-benefit analysis
shows the total cost for deploying both current and new portal
monitors to be about $1 billion. However, in July 2006, DHS
announced that it had awarded contracts to develop and purchase up
to $1.2 billion worth of the new portal monitors over 5 years.
o DNDO's cost-benefit analysis omitted many factors that could
affect the cost of new portal monitors, such as understating the
life-cycle costs for operating and maintaining the equipment over
time.
Background
In general, DHS is responsible for providing radiation detection
capabilities at U.S. ports of entry. Until April 2005, CBP managed
this program. However, on April 15, 2005, the President directed
the establishment of DNDO within DHS. DNDO's duties include
acquiring and supporting the deployment of radiation detection
equipment, including portal monitors. CBP continues its
traditional screening function at ports of entry to interdict
dangerous nuclear and radiological materials through the use of
radiation detection equipment. The SAFE Port Act of 2006 formally
authorizes DNDO's creation and operation.6 PNNL manages the
deployment of radiation detection equipment for DHS.
DHS's program to deploy radiation detection equipment at U.S.
ports of entry has two goals. The first is to use this equipment
to screen all cargo, vehicles, and individuals coming into the
United States. The second is to screen this traffic without
delaying its movement into the nation. DHS's current plans call
for completing the deployment of radiation portal monitors at U.S.
ports of entry by September 2013. The current technology portal
monitors, known as plastic scintillators or PVTs, cost about
$55,000 per unit, while the advanced spectroscopic portal
monitors, known as ASPs, will cost around $377,000 per unit.7
In July 2006, DHS announced that it had awarded contracts to three
ASP vendors to further develop and purchase $1.2 billion worth of
ASPs over 5 years. Congress, however, provided that none of DNDO's
appropriated funds for systems acquisition could be obligated for
full procurement of ASPs until the Secretary of DHS certifies
through a report to the Committees on Appropriations for the
Senate and House of Representatives that ASPs would result in a
significant increase in operational effectiveness.8
DNDO Ignored Its Own Performance Test Results and Instead Relied
on the Potential Performance of New Portal Monitors and Unreliable
Estimates of Current Equipment Performance
DHS is developing new portal monitors, known as ASPs that, in
addition to detecting nuclear or radiological material, can also
identify the type of material. In 2005, DNDO conducted
side-by-side testing at the Nevada Test Site (NTS)9 on 10 ASP
systems and 3 PVT systems developed by private sector companies,
including the PVT systems currently deployed. DHS requested that
the National Institutes of Standards and Technology (NIST) provide
assistance by conducting an independent analysis of data acquired
during the last phase of developmental testing of ASPs to help DHS
determine the performance of ASP portal monitors being proposed by
private sector companies. NIST compared the 10 ASP systems, and in
June 2006 submitted a report to DHS on the results of that
testing.10
Performance tests of ASPs showed that they did not meet DNDO's
main performance assumption in the cost-benefit analysis of
correctly identifying HEU 95 percent of the time it passes through
portal monitors. The 95 percent performance assumption included
ASPs' ability to both detect bare, or unmasked, HEU in a container
and HEU masked in a container with a more benign radiological
material.11 Based on NIST's assessment of the performance data,
the ASP prototypes (manufactured by the three companies that won
DNDO's recent ASP procurement contract) tested at NTS identified
bare HEU only 70 to 88 percent of time. Performance tests also
showed that ASPs' ability to identify masked HEU fell far short of
meeting the 95 percent goal established for the cost-benefit
analysis. According to DNDO, identifying masked HEU is the most
difficult case to address. DOE officials told us that benign
radiological materials could be used to hide the presence of HEU.
NIST reported that the best ASP prototype DNDO tested in Nevada
during 2005, and which won a procurement contract, was able to
correctly identify masked HEU and depleted uranium (DU) only 53
percent of the time. Similarly, the ASP prototypes submitted by
the other two companies that won DNDO ASP procurement contracts
were able to identify masked HEU and DU only 45 percent and 17
percent of the time.
Despite these results, DNDO did not use the information from these
tests in its cost-benefit analysis. Instead, DNDO officials told
us that since the new portal monitors cannot meet the 95 percent
detection goal, they relied on the assumption that they will reach
that level of performance sometime in the future. DNDO officials
asserted that the ASPs' current performance levels would improve,
but they provided no additional information as to how the 95
percent goal will be achieved or an estimate of when the
technology would attain this level of performance.
Moreover, DNDO's cost-benefit analysis only considered the
benefits of ASPs' ability to detect and identify HEU and did not
consider the ASPs' ability to detect and identify other nuclear
and radiological materials. The ability of an ASP to identify
specific nuclear or radiological materials depends on whether the
ASP contains software that is specific to those materials. In our
view, a complete cost-benefit analysis would include an assessment
of ASPs' ability to detect and identify a variety of nuclear and
radiological material, not just HEU. By excluding radiological and
nuclear materials other than HEU, DNDO's analysis did not consider
the number of secondary inspections that would be related to these
materials and hence it likely underestimated the costs of ASP use.
Further, DNDO told us the assumptions for the ability of ASP
systems to detect and identify HEU 95 percent of the time came
from the ASP performance specifications. However, we examined the
performance specifications and found no specific requirement for
detecting or identifying HEU with a 95 percent probability.12
While there is a requirement in the performance specification for
the identification for HEU and other special nuclear material, we
found no associated probability of success in performing this
function.
DNDO's cost-benefit analysis also may not accurately reflect the
capabilities of PVTs to detect nuclear or radiological material.
DNDO officials acknowledged that DNDO tested the performance of
PVTs along with the ASPs in 2005, but did not use the results of
these tests in its cost-benefit analysis. According to these
officials and NIST staff who assisted in the testing, the PVT
performance data were unusable because the PVTs' background
settings were not set properly. Consequently, DNDO officials told
us the analysis was based on the performance of PVT monitors that
PNNL tested during 2004 in New York. However, the results from
these field tests are not definitive because, as PNNL noted in its
final report, the tests did not use HEU and, therefore, the
results from the tests did not indicate how well PVT portal
monitors would be able to detect HEU in the field.13 Moreover, the
PVT portal monitors that PNNL used had only one radiation
detection panel as opposed to the four-panel PVT monitors that DHS
currently deploys at U.S. ports of entry. An expert at a national
laboratory told us that larger surface areas are more likely to
detect radiological or nuclear material. DNDO also stated that due
to the nature of the testing at the Nevada Test Site, the tests
did not provide the data needed for the cost-benefit analysis.
According to DNDO officials, this data would come from analysis of
the performance of fielded systems at U.S. ports-of-entry where
the probability to detect threats could be compared to false alarm
rates.
DNDO's director stated twice in testimony before the House
Homeland Security Committee, Subcommittee on Prevention of Nuclear
and Biological Attack--once on June 21, 2005, and again on May 26,
2006--that the ASP and PVT portals would be evaluated against one
another in "extensive high-fidelity" tests. In our view, the
results of such testing are critical to any decision by DNDO to
employ new technology, such as ASPs, that might help protect the
nation from nuclear smuggling. According to DNDO officials, new
tests now underway at the DHS Nevada Test Site are comparing the
performance of ASPs and PVTs side-by-side.
DNDO�s Cost-Benefit Analysis was Incomplete and Used Inflated
Cost Estimates for PVT Equipment
DNDO officials told us they did not follow the DHS guidelines for
performing cost-benefit analyses in conducting their own
cost-benefit analysis.14 These guidelines stipulate, among other
things, that such studies should address all of the major costs
and benefits that could have a material effect on DHS programs.
However, DNDO's analysis omitted many factors that could affect
the cost of new radiation portal monitors. For example, DNDO
officials told us that there are over 12 different types of ASP
monitors, yet they only estimated the cost of cargo portal
monitors that would be used at land border crossings. In reality,
DNDO and CBP plan to deploy different types of ASPs that would
have varying costs, such as portal monitors at seaports which
would have higher costs. Additionally, DNDO did not capture all
the costs related to developing the different types of ASP
monitors. In our view, developing realistic cost estimates should
not be sacrificed in favor of simplicity.
DNDO also underestimated the life-cycle costs for operations and
maintenance for both PVT and ASP equipment over time. DNDO's
analysis assumed a 5-year life-cycle for both PVT and ASP
equipment. However, DNDO officials told us that a 10-year
life-cycle was a more reasonable expectation for PVT and ASP
equipment. The analysis assumes that the annual maintenance costs
for PVT and ASP monitors will each equal 10 percent of their
respective procurement costs. This means that maintenance costs
for PVTs would be about $5,500 per year per unit based on a
$55,000 purchase price and ASP maintenance costs would be about
$38,000 per year per unit based on a $377,000 purchase price.
Given the much higher maintenance costs for ASPs and the doubling
of the life-cycle to 10 years, the long-term implications for
these cost differences would be magnified. Consequently, DNDO's
analysis has not accounted for about $181 million in potential
maintenance costs for ASPs alone.
Furthermore, DNDO did not assess the likelihood that radiation
detection equipment would either misidentify or fail to detect
nuclear and radiological materials. Rather, DNDO's cost-benefit
analysis focuses on the ability of ASPs to reduce false
alarms--alarms that indicate nuclear or radiological material is
in a container when, in fact, the material is actually
non-threatening, such as ceramic tile. Reducing false alarms would
reduce the number of secondary inspections of non-threatening
nuclear and radiological materials and therefore the costs of
those inspections. However, as required by DHS's guide to
performing cost-benefit analyses, DNDO's analysis did not include
all costs. In particular, the analysis did not include the
potentially much bigger cost of "false negatives." False negatives
are instances in which a container possesses a threatening nuclear
or radiological material, but the portal monitor either
misidentifies the material as non-threatening or does not detect
the material at all, thus allowing the material to enter the
country. During the 2005 Nevada tests, the incidence of false
negatives among the three vendors who received contracts ranged
from about 45 percent to slightly more that 80 percent. This
raises concerns because, as explained to us by a scientist at a
national laboratory, at this level of performance, ASPs could
conceivably misidentify HEU as a benign nuclear or radiological
material or not detect it at all, particularly if the HEU is
placed side by side with a non-threatening material such as kitty
litter.
In recent testimonies before Congress, DNDO's Director has cited
the primary benefit of deploying ASP monitors as reducing
unnecessary secondary inspections.15 DNDO's cost-benefit analysis
focused on measuring the benefits of ASP's ability to reduce false
alarms--alarms that indicate nuclear or radiological material is
present when, in fact, it is not or such material is actually
non-threatening. Reducing false alarms would reduce the number of
secondary inspections of non-threatening nuclear and radiological
materials and therefore the costs of those inspections. Even on
this point, however, DNDO's analysis was flawed. For example, it
did not estimate the costs of secondary inspections as they are
carried out today. DNDO's analysis needs these baseline costs to
compare alternatives because without them, it is impossible to
determine whether the use of ASPs, as planned, will result in cost
savings for secondary inspections. While we agree that
facilitating commerce at U.S. ports of entry by reducing
unnecessary secondary inspections is an important goal, we believe
that the primary rationale for deploying portal monitors should be
to protect the nation from nuclear and/or radiological attack. We
found that DNDO's analysis did not even attempt to measure the
level--or value--of security afforded by portal monitors.
In addition, the ASP contract award has exceeded DNDO's estimate
for total cost by about $200 million. The cost-benefit analysis
shows the total cost for deploying PVT and ASP monitors to be
about $1 billion, which covers all costs related to acquisition,
design, maintenance, and physical inspection over 5 years (for
both PVT and ASP). However, in July 2006, DHS announced that it
had awarded contracts to develop and purchase up to $1.2 billion
worth of ASP portal monitors over 5 years. Furthermore, the
cost-benefit analysis underestimates total deployment costs and
does not account for other major costs, such as physical
inspections of cargo containers, an additional procurement of 442
new PVT monitors, installation and integration, and maintenance.
Finally, DNDO overstated the purchase price of PVT monitors.
Although DHS is currently paying an average of about $55,000 per
monitor, DNDO's cost-benefit analysis assumed the PVT would cost
$130,959--the highest published manufacturers' price for the
government.16 According to DNDO's Director, DNDO chose the highest
published price because the current contract for portal monitors
at that time was to expire, and the portal monitors will probably
cost more in the future. However, the information DNDO provided us
does not explain why DNDO assumes that the future price will be
more than double what DHS was currently paying, as assumed in
DNDO's analysis. PNNL officials told us that the future price will
almost certainly be lower than the price used in DNDO's analysis.
4GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support
the Purchase of New Radiation Detection Portal Monitors Was Not Based on
Available Performance Data and Did Not Fully Evaluate All the Monitors'
Costs and Benefits, [20]GAO-07-133R (Washington, D.C.: Oct. 17, 2006).
5"Masking" is an attempt to hide dangerous nuclear or radiological
material by placing it with benign radiological sources.
6Pub. L. No. 109-347, tit. V, 120 Stat. 1884, 1932 (2006).
7Prices include only equipment purchase. Installation costs are extra.
8Department of Homeland Security Appropriations Act for Fiscal Year 2007,
Pub. L. No. 109-295, tit. IV, 120 Stat. 1355, 1376 (2006).
9DHS and DOE are collaborating in building a new Radiological and Nuclear
Countermeasures Test and Evaluation Complex at the Nevada Test Site to
support the development, testing, acquisition, and deployment of radiation
detection equipment. The facility is expected to become fully operational
in early 2007. Currently, an interim facility at NTS is being used to test
radiation detection equipment.
10NIST did not evaluate the PVTs or compare their performance to the
performance of the ASPs.
11The ability to detect masked HEU is based on DOE guidance on performing
the evaluation.
12The performance specifications contain a requirement for detecting, not
identifying, californium-252 with a 95 percent probability.
Californium-252 has similar radiological properties to HEU. In addition,
the performance specifications contain a requirement for detecting, but
not identifying, other radiological materials such as cobalt-57,
cobalt-60, barium-133, cesium-137, and americium-241.
13Pacific Northwest National Laboratory, A Sensitivity Comparison of NaI
and PVT Portal Monitors at a Land-Border Port-of-Entry, p. iii, November
2004. For Official Use Only.
14DHS, Capital Planning and Investment Control, Cost-Benefit Analysis
(CBA) Guidebook 2006, Version 2.0, February 2006. Traditional rules of
performing cost-benefit analyses include assessing the full life-cycle
costs for operation and maintenance, and determining the level of
confidence in cost data.
15Enlisting Foreign Cooperation in U.S. Efforts to Prevent Nuclear
Smuggling: Hearing Before the House Committee on Homeland Security,
Subcommittee on Prevention of Nuclear and Biological Attack, 109th Cong.
(May 25, 2006)(statement of Mr. Vayl S. Oxford, Director, DNDO); Detecting
Smuggled Nuclear Weapons, Hearing Before the Senate Judiciary Committee,
Subcommittee on Terrorism, Technology, and Homeland Security, 109th Cong.
(July 27, 2006)(statement of Mr. Vayl S. Oxford, Director, DNDO).
16DNDO, Cost Benefit Analysis for Next Generation Passive Radiation
Detection of Cargo an the Nation's Border Crossings, May 30, 2006.
In conclusion, DNDO's approach to the cost-benefit analysis
omitted many factors that could affect the cost of new radiation
portal monitors. For these reasons, DHS's cost-benefit analysis
does not meet the intent of our March 2006 report recommendation
to fully assess the costs and benefits before purchasing any new
equipment.
Mr. Chairman, this concludes my prepared statement. I would be
happy to respond to any questions that you or other members of the
Subcommittee may have.
GAO Contact and Staff Acknowledgements
For further information about this testimony, please contact me,
Gene Aloise, at (202) 512-3841 or at [email protected]. Contact
points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this statement. John
Delicath, Jim Shafer, and Eugene Wisnoski made key contributions
to this statement.
GAO�s Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of
newly posted products every afternoon, go to www.gao.gov and
select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW,
Room 7125 Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
www.gao.gov/cgi-bin/getrpt?GAO-07-581T .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Gene Aloise at (202) 512-3841 or
[email protected].
Highlights of [22]GAO-07-581T , a testimony before the Subcommittee on
Emerging Threats, Cybersecurity, and Science and Technology; Committee on
Homeland Security; House of Representatives
March 14, 2007
COMBATING NUCLEAR SMUGGLING
DHS's Decision to Procure and Deploy the Next Generation of Radiation
Detection Equipment Is Not Supported by Its Cost-Benefit Analysis
The Department of Homeland Security (DHS) is responsible for addressing
the threat of nuclear smuggling. Radiation detection portal monitors are
key elements in our national defenses against such threats. DHS has
sponsored R&D and testing activities to develop a "next generation" portal
monitor, known as the advanced spectroscopic portal monitor. However, each
one costs 6 times more than a current portal monitor. In March 2006, we
recommended that DHS conduct a cost-benefit analysis to determine whether
the new portal monitors are worth the additional cost. In June 2006, DHS
issued its analysis. In October 2006, we issued our report that assessed
the DHS study.
GAO's statement, based on our October 2006 report, addresses whether DHS's
cost-benefit analysis provides an adequate basis for its decision to
purchase and deploy the next generation portal monitors.
[23]What GAO Recommends
GAO's October 2006 report included two recommendations designed to improve
the quality and comprehensiveness of the Department's analysis. DHS
neither agreed nor disagreed with our recommendations, but continued to
support its analysis as a solid basis for buying and deploying the new
generation of radiation portal monitors.
DHS's cost-benefit analysis does not provide a sound analytical basis for
its decision to purchase and deploy the new portal monitor technology. Our
review of the analysis determined that it had the following problems:
Regarding the performance of the portal monitors:
o Instead of using the results of its own portal monitor tests
conducted in 2005, DHS assumed that the new portal monitor
technology would correctly detect and identify highly enriched
uranium (HEU) 95 percent of the time--a performance level that far
exceeds the new technology's current capabilities.
o To determine the performance of the current generation of portal
monitors in detecting HEU, DHS used data from limited tests
carried out in 2004 that test officials concluded were unreliable
for such purposes.
o DHS's analysis of the new technology portal monitors was
incomplete because the analysis focused on identifying HEU, but
did not fully consider how well the new portal monitor technology
could correctly detect or identify other dangerous radiological or
nuclear materials.
Regarding cost estimates:
o In comparing the costs of the new and current technologies, the
procurement costs of the current generation portal monitors were
highly inflated because DHS assumed a unit cost of about $131,000.
However, the contract price at the time of the analysis was about
$55,000. According to officials who manage the contract, it was to
expire and they expected portal monitor prices to increase, but
not nearly as much as DHS assumed.
o DHS stated that the primary benefit of deploying the new portal
monitors is reducing unnecessary secondary inspections. However,
DHS's analysis does not fully estimate today's baseline costs for
secondary inspections, which makes it impossible to determine
whether the use of the new portal monitors as currently planned,
will result in significant cost savings for these inspections.
o The new portal monitor contract price has exceeded DHS's total
cost estimate by about $200 million. The cost-benefit analysis
shows the total cost for deploying both current and new portal
monitors to be about $1 billion. However, in July 2006, DHS
announced that it had awarded contracts to develop and purchase up
to $1.2 billion worth of the new portal monitors over 5 years.
o DHS's cost-benefit analysis omitted many factors that could
affect the cost of new portal monitors, such as understating the
life-cycle costs for operating and maintaining the equipment over
time.
For these reasons, DHS's cost-benefit analysis does not meet the intent of
our March 2006 report recommendation to fully assess the costs and
benefits before purchasing any new equipment.
References
Visible links
11. http://www.gao.gov/cgi-bin/getrpt?GAO-07-133R
12. http://www.gao.gov/cgi-bin/getrpt?GAO-06-389
13. http://www.gao.gov/cgi-bin/getrpt?GAO-06-389
20. http://www.gao.gov/cgi-bin/getrpt?GAO-07-133R
22. http://www.gao.gov/cgi-bin/getrpt?GAO-07-581T
*** End of document. ***