Combating Nuclear Smuggling: DHS's Decision to Procure and Deploy
the Next Generation of Radiation Detection Equipment Is Not	 
Supported by Its Cost-Benefit Analysis (14-MAR-07, GAO-07-581T). 
                                                                 
The Department of Homeland Security (DHS) is responsible for	 
addressing the threat of nuclear smuggling. Radiation detection  
portal monitors are key elements in our national defenses against
such threats. DHS has sponsored R&D and testing activities to	 
develop a "next generation" portal monitor, known as the advanced
spectroscopic portal monitor. However, each one costs 6 times	 
more than a current portal monitor. In March 2006, we recommended
that DHS conduct a cost-benefit analysis to determine whether the
new portal monitors are worth the additional cost. In June 2006, 
DHS issued its analysis. In October 2006, we issued our report	 
that assessed the DHS study. GAO's statement, based on our	 
October 2006 report, addresses whether DHS's cost-benefit	 
analysis provides an adequate basis for its decision to purchase 
and deploy the next generation portal monitors. 		 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-581T					        
    ACCNO:   A66824						        
  TITLE:     Combating Nuclear Smuggling: DHS's Decision to Procure   
and Deploy the Next Generation of Radiation Detection Equipment  
Is Not Supported by Its Cost-Benefit Analysis			 
     DATE:   03/14/2007 
  SUBJECT:   Comparative analysis				 
	     Cost analysis					 
	     Cost effectiveness analysis			 
	     Evaluation methods 				 
	     Federal procurement				 
	     Financial analysis 				 
	     Homeland security					 
	     Nuclear radiation monitoring			 
	     Policy evaluation					 
	     Radiation monitoring				 
	     Smuggling						 
	     Benefit-cost tracking				 
	     Cost estimates					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-581T

   

     * [1]Background
     * [2]DNDO Ignored Its Own Performance Test Results and Instead Re
     * [3]DNDO's Cost-Benefit Analysis was Incomplete and Used Inflate
     * [4]GAO Contact and Staff Acknowledgements
     * [5]GAO's Mission
     * [6]Obtaining Copies of GAO Reports and Testimony

          * [7]Order by Mail or Phone

     * [8]To Report Fraud, Waste, and Abuse in Federal Programs
     * [9]Congressional Relations
     * [10]Public Affairs

Testimony

Before the Subcommittee on Emerging Threats, Cybersecurity, and Science
and Technology; Committee on Homeland Security; House of Representatives

United States Government Accountability Office

GAO

For Release on Delivery Expected at 2:00 p.m. EST
Wednesday, March 14, 2007

COMBATING NUCLEAR SMUGGLING

DHS's Decision to Procure and Deploy the Next Generation of Radiation
Detection Equipment Is Not Supported by Its Cost-Benefit Analysis

Statement of Gene Aloise, Director
Natural Resources and Environment

GAO-07-581T

Mr. Chairman and Members of the Subcommittee:

I am pleased to appear here today to discuss our assessment of the
Department of Homeland Security's (DHS) May 2006 cost-benefit analysis
used to support the purchase and deployment of next generation radiation
portal monitors.1 This is an important decision because, if procured,
these new portal monitors will be considerably more expensive than the
portal monitors in use today. Combating nuclear smuggling is one of our
nation's key national security objectives and the deployment of radiation
detection equipment including portal monitors at U.S. ports of entry,
including border crossings and domestic seaports, is an integral part of
this system. DHS, through its Domestic Nuclear Detection Office (DNDO), is
responsible for acquiring and supporting the deployment of radiation
detection equipment, including portal monitors, within the United States.
The Pacific Northwest National Laboratory (PNNL), one of the Department of
Energy's (DOE) national laboratories, is under contract to manage the
deployment of radiation detection equipment for DHS.2 U.S. Customs and
Border Protection (CBP) is responsible for screening cargo as it enters
the nation at our borders, which includes operating radiation detection
equipment to interdict dangerous nuclear and radiological materials.

The radiation portal monitors in use today can detect the presence of
radiation, but they cannot distinguish between types of radiological
material. For example, they cannot tell the difference between harmless
products that emit radiation, such as ceramic tile, and dangerous
materials, such as highly enriched uranium (HEU), that could be used to
construct a nuclear weapon. Generally, CBP's standard procedures require
incoming cargo to pass through one of these radiation portal monitors to
screen for the presence of radiation. This "primary inspection" serves to
alert CBP officers when a radioactive threat might be present. If there is
a potential threat, CBP procedures require a "secondary inspection." To
confirm the presence of radiation, this secondary inspection usually
includes a second screening by a radiation portal monitor as well as a
screening by CBP officers using radioactive isotope identification
devices. These handheld devices are used to differentiate between types of
radioactive material to determine if the radiation being detected is
dangerous. Both the radiation portal monitors and handheld devices are
limited in their abilities to detect and identify nuclear material.

1GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support
the Purchase of New Radiation Detection Portal Monitors Was Not Based on
Available Performance Data and Did Not Fully Evaluate All the Monitors'
Costs and Benefits, [11]GAO-07-133R (Washington, D.C.: Oct. 17, 2006).
GAO, Combating Nuclear Smuggling: DHS Has Made Progress Deploying
Radiation Detection Equipment at U.S. Ports of Entry, but Concerns Remain,
[12]GAO-06-389 (Washington, D.C.: Mar. 22, 2006).

2DOE manages the largest laboratory system of its kind in the world. The
mission of DOE's 22 laboratories has evolved. Originally created to design
and build atomic weapons, these laboratories have since expanded to
conduct research in many disciplines - from high-energy physics to
advanced computing.

DHS would like to improve the capabilities of its portal monitors so that
they can perform the dual roles of detecting radiation and identifying
radiological materials. In this regard, DHS has sponsored research,
development, and testing activities designed to create the next generation
of portal monitors capable of performing both functions. These new,
advanced portals are known as advanced spectroscopic portals (ASPs). In
July 2006, DHS awarded contracts to three vendors for developing the
advanced spectroscopic portals' capabilities. These awards were based
mainly on performance tests conducted at DHS's Nevada Test Site in 2005,
where ten competing advanced spectroscopic vendors' monitors were
evaluated. At the same time, three current technology portal monitors were
also tested.

To ensure that DHS's substantial investment in radiation detection
technology yields the greatest possible level of detection capability at
the lowest possible cost, in a March 2006 GAO report,3 we recommended that
once the costs and capabilities of ASPs were well understood, and before
any of the new equipment was purchased for deployment, the Secretary of
DHS work with the Director of DNDO to analyze the costs and benefits of
deploying ASPs. Further, we recommended that this analysis focus on
determining whether any additional detection capability provided by the
ASPs was worth the considerable additional costs. In response to our
recommendation, DNDO issued its cost-benefit analysis in May 2006, and an
updated, revised version in June 2006. According to senior agency
officials, DNDO believes that the basic conclusions of its cost-benefit
analysis show that the new advanced spectroscopic portal monitors are a
sound investment for the U.S. government.

3GAO, Combating Nuclear Smuggling: DHS Has Made Progress Deploying
Radiation Detection Equipment at U.S. Ports of Entry, but Concerns Remain,
[13]GAO-06-389 (Washington, D.C.: Mar. 22, 2006).

Mr. Chairman, my remarks today focus on the cost-benefit analysis DNDO
used in support of its decision to purchase new ASP portal monitors.
Specifically, I will discuss whether DNDO's June 2006 cost-benefit
analysis provides an adequate basis for the substantial investment that
acquiring and deploying ASPs will necessitate.

My testimony is based upon our October 2006 report that evaluated DNDO's
cost-benefit analysis.4 The work for our report was done in accordance
with generally accepted government auditing standards.

In summary, DNDO's 2006 cost-benefit analysis does not provide a sound
analytical basis for its decision to purchase and deploy the new advanced
spectroscopic portal monitor technology.

Regarding the performance of the portal monitors:

           o Instead of using the results of its performance tests conducted
           in 2005, DNDO's analysis simply assumed that ASPs could detect
           highly enriched uranium 95 percent of the time, a performance
           level far exceeding the capabilities of the new technology's
           current demonstrated capabilities. The 2005 test results showed
           that the best of the three winning vendor monitors could only
           identify masked HEU5 about 50 percent of the time.
           o To determine the current generation of portal monitors'
           performance in detecting HEU, DNDO used data from limited tests
           carried out in 2004 that test officials concluded was unreliable
           for such purposes. In their written report, test officials
           explicitly stated that the data were not indicative of how well
           current technology portal monitors might perform in the field
           particularly for detecting HEU.
           o DNDO's analysis of the new technology portal monitors'
           performance was deficient because it focused on detecting and
           identifying HEU, but did not fully consider other dangerous
           radiological or nuclear materials. DNDO should have assessed the
           ASPs' abilities to detect several realistic threat materials.

           Regarding cost estimates:

           o In comparing the costs of the new and current technologies, the
           procurement costs of the current generation portal monitors were
           highly inflated because DNDO assumed a unit cost of about
           $131,000, while the contract price at the time of the analysis was
           about $55,000. According to officials who manage the contract, it
           was to expire and while they expected portal monitor prices to
           increase, they did not believe the cost would be as much as the
           price used in DNDO's analysis.
           o DNDO stated that the primary benefit of deploying the new portal
           monitors is reducing unnecessary secondary inspections. However,
           DNDO's analysis did not fully estimate today's baseline costs for
           secondary inspections, which makes it impossible to determine
           whether the use of the new portal monitors as currently planned
           will result in significant cost savings for these inspections.
           o The new portal monitor contract price has exceeded DNDO's total
           cost estimate by about $200 million. The cost-benefit analysis
           shows the total cost for deploying both current and new portal
           monitors to be about $1 billion. However, in July 2006, DHS
           announced that it had awarded contracts to develop and purchase up
           to $1.2 billion worth of the new portal monitors over 5 years.
           o DNDO's cost-benefit analysis omitted many factors that could
           affect the cost of new portal monitors, such as understating the
           life-cycle costs for operating and maintaining the equipment over
           time.
			  
			  Background

           In general, DHS is responsible for providing radiation detection
           capabilities at U.S. ports of entry. Until April 2005, CBP managed
           this program. However, on April 15, 2005, the President directed
           the establishment of DNDO within DHS. DNDO's duties include
           acquiring and supporting the deployment of radiation detection
           equipment, including portal monitors. CBP continues its
           traditional screening function at ports of entry to interdict
           dangerous nuclear and radiological materials through the use of
           radiation detection equipment. The SAFE Port Act of 2006 formally
           authorizes DNDO's creation and operation.6 PNNL manages the
           deployment of radiation detection equipment for DHS.

           DHS's program to deploy radiation detection equipment at U.S.
           ports of entry has two goals. The first is to use this equipment
           to screen all cargo, vehicles, and individuals coming into the
           United States. The second is to screen this traffic without
           delaying its movement into the nation. DHS's current plans call
           for completing the deployment of radiation portal monitors at U.S.
           ports of entry by September 2013. The current technology portal
           monitors, known as plastic scintillators or PVTs, cost about
           $55,000 per unit, while the advanced spectroscopic portal
           monitors, known as ASPs, will cost around $377,000 per unit.7

           In July 2006, DHS announced that it had awarded contracts to three
           ASP vendors to further develop and purchase $1.2 billion worth of
           ASPs over 5 years. Congress, however, provided that none of DNDO's
           appropriated funds for systems acquisition could be obligated for
           full procurement of ASPs until the Secretary of DHS certifies
           through a report to the Committees on Appropriations for the
           Senate and House of Representatives that ASPs would result in a
           significant increase in operational effectiveness.8
			  
			  DNDO Ignored Its Own Performance Test Results and Instead Relied
			  on the Potential Performance of New Portal Monitors and Unreliable
			  Estimates of Current Equipment Performance

           DHS is developing new portal monitors, known as ASPs that, in
           addition to detecting nuclear or radiological material, can also
           identify the type of material. In 2005, DNDO conducted
           side-by-side testing at the Nevada Test Site (NTS)9 on 10 ASP
           systems and 3 PVT systems developed by private sector companies,
           including the PVT systems currently deployed. DHS requested that
           the National Institutes of Standards and Technology (NIST) provide
           assistance by conducting an independent analysis of data acquired
           during the last phase of developmental testing of ASPs to help DHS
           determine the performance of ASP portal monitors being proposed by
           private sector companies. NIST compared the 10 ASP systems, and in
           June 2006 submitted a report to DHS on the results of that
           testing.10

           Performance tests of ASPs showed that they did not meet DNDO's
           main performance assumption in the cost-benefit analysis of
           correctly identifying HEU 95 percent of the time it passes through
           portal monitors. The 95 percent performance assumption included
           ASPs' ability to both detect bare, or unmasked, HEU in a container
           and HEU masked in a container with a more benign radiological
           material.11 Based on NIST's assessment of the performance data,
           the ASP prototypes (manufactured by the three companies that won
           DNDO's recent ASP procurement contract) tested at NTS identified
           bare HEU only 70 to 88 percent of time. Performance tests also
           showed that ASPs' ability to identify masked HEU fell far short of
           meeting the 95 percent goal established for the cost-benefit
           analysis. According to DNDO, identifying masked HEU is the most
           difficult case to address. DOE officials told us that benign
           radiological materials could be used to hide the presence of HEU.
           NIST reported that the best ASP prototype DNDO tested in Nevada
           during 2005, and which won a procurement contract, was able to
           correctly identify masked HEU and depleted uranium (DU) only 53
           percent of the time. Similarly, the ASP prototypes submitted by
           the other two companies that won DNDO ASP procurement contracts
           were able to identify masked HEU and DU only 45 percent and 17
           percent of the time.

           Despite these results, DNDO did not use the information from these
           tests in its cost-benefit analysis. Instead, DNDO officials told
           us that since the new portal monitors cannot meet the 95 percent
           detection goal, they relied on the assumption that they will reach
           that level of performance sometime in the future. DNDO officials
           asserted that the ASPs' current performance levels would improve,
           but they provided no additional information as to how the 95
           percent goal will be achieved or an estimate of when the
           technology would attain this level of performance.

           Moreover, DNDO's cost-benefit analysis only considered the
           benefits of ASPs' ability to detect and identify HEU and did not
           consider the ASPs' ability to detect and identify other nuclear
           and radiological materials. The ability of an ASP to identify
           specific nuclear or radiological materials depends on whether the
           ASP contains software that is specific to those materials. In our
           view, a complete cost-benefit analysis would include an assessment
           of ASPs' ability to detect and identify a variety of nuclear and
           radiological material, not just HEU. By excluding radiological and
           nuclear materials other than HEU, DNDO's analysis did not consider
           the number of secondary inspections that would be related to these
           materials and hence it likely underestimated the costs of ASP use.
           Further, DNDO told us the assumptions for the ability of ASP
           systems to detect and identify HEU 95 percent of the time came
           from the ASP performance specifications. However, we examined the
           performance specifications and found no specific requirement for
           detecting or identifying HEU with a 95 percent probability.12
           While there is a requirement in the performance specification for
           the identification for HEU and other special nuclear material, we
           found no associated probability of success in performing this
           function.

           DNDO's cost-benefit analysis also may not accurately reflect the
           capabilities of PVTs to detect nuclear or radiological material.
           DNDO officials acknowledged that DNDO tested the performance of
           PVTs along with the ASPs in 2005, but did not use the results of
           these tests in its cost-benefit analysis. According to these
           officials and NIST staff who assisted in the testing, the PVT
           performance data were unusable because the PVTs' background
           settings were not set properly. Consequently, DNDO officials told
           us the analysis was based on the performance of PVT monitors that
           PNNL tested during 2004 in New York. However, the results from
           these field tests are not definitive because, as PNNL noted in its
           final report, the tests did not use HEU and, therefore, the
           results from the tests did not indicate how well PVT portal
           monitors would be able to detect HEU in the field.13 Moreover, the
           PVT portal monitors that PNNL used had only one radiation
           detection panel as opposed to the four-panel PVT monitors that DHS
           currently deploys at U.S. ports of entry. An expert at a national
           laboratory told us that larger surface areas are more likely to
           detect radiological or nuclear material. DNDO also stated that due
           to the nature of the testing at the Nevada Test Site, the tests
           did not provide the data needed for the cost-benefit analysis.
           According to DNDO officials, this data would come from analysis of
           the performance of fielded systems at U.S. ports-of-entry where
           the probability to detect threats could be compared to false alarm
           rates.

           DNDO's director stated twice in testimony before the House
           Homeland Security Committee, Subcommittee on Prevention of Nuclear
           and Biological Attack--once on June 21, 2005, and again on May 26,
           2006--that the ASP and PVT portals would be evaluated against one
           another in "extensive high-fidelity" tests. In our view, the
           results of such testing are critical to any decision by DNDO to
           employ new technology, such as ASPs, that might help protect the
           nation from nuclear smuggling. According to DNDO officials, new
           tests now underway at the DHS Nevada Test Site are comparing the
           performance of ASPs and PVTs side-by-side.
			  
			  DNDOï¿½s Cost-Benefit Analysis was Incomplete and Used Inflated
			  Cost Estimates for PVT Equipment

           DNDO officials told us they did not follow the DHS guidelines for
           performing cost-benefit analyses in conducting their own
           cost-benefit analysis.14 These guidelines stipulate, among other
           things, that such studies should address all of the major costs
           and benefits that could have a material effect on DHS programs.
           However, DNDO's analysis omitted many factors that could affect
           the cost of new radiation portal monitors. For example, DNDO
           officials told us that there are over 12 different types of ASP
           monitors, yet they only estimated the cost of cargo portal
           monitors that would be used at land border crossings. In reality,
           DNDO and CBP plan to deploy different types of ASPs that would
           have varying costs, such as portal monitors at seaports which
           would have higher costs. Additionally, DNDO did not capture all
           the costs related to developing the different types of ASP
           monitors. In our view, developing realistic cost estimates should
           not be sacrificed in favor of simplicity.

           DNDO also underestimated the life-cycle costs for operations and
           maintenance for both PVT and ASP equipment over time. DNDO's
           analysis assumed a 5-year life-cycle for both PVT and ASP
           equipment. However, DNDO officials told us that a 10-year
           life-cycle was a more reasonable expectation for PVT and ASP
           equipment. The analysis assumes that the annual maintenance costs
           for PVT and ASP monitors will each equal 10 percent of their
           respective procurement costs. This means that maintenance costs
           for PVTs would be about $5,500 per year per unit based on a
           $55,000 purchase price and ASP maintenance costs would be about
           $38,000 per year per unit based on a $377,000 purchase price.
           Given the much higher maintenance costs for ASPs and the doubling
           of the life-cycle to 10 years, the long-term implications for
           these cost differences would be magnified. Consequently, DNDO's
           analysis has not accounted for about $181 million in potential
           maintenance costs for ASPs alone.

           Furthermore, DNDO did not assess the likelihood that radiation
           detection equipment would either misidentify or fail to detect
           nuclear and radiological materials. Rather, DNDO's cost-benefit
           analysis focuses on the ability of ASPs to reduce false
           alarms--alarms that indicate nuclear or radiological material is
           in a container when, in fact, the material is actually
           non-threatening, such as ceramic tile. Reducing false alarms would
           reduce the number of secondary inspections of non-threatening
           nuclear and radiological materials and therefore the costs of
           those inspections. However, as required by DHS's guide to
           performing cost-benefit analyses, DNDO's analysis did not include
           all costs. In particular, the analysis did not include the
           potentially much bigger cost of "false negatives." False negatives
           are instances in which a container possesses a threatening nuclear
           or radiological material, but the portal monitor either
           misidentifies the material as non-threatening or does not detect
           the material at all, thus allowing the material to enter the
           country. During the 2005 Nevada tests, the incidence of false
           negatives among the three vendors who received contracts ranged
           from about 45 percent to slightly more that 80 percent. This
           raises concerns because, as explained to us by a scientist at a
           national laboratory, at this level of performance, ASPs could
           conceivably misidentify HEU as a benign nuclear or radiological
           material or not detect it at all, particularly if the HEU is
           placed side by side with a non-threatening material such as kitty
           litter.

           In recent testimonies before Congress, DNDO's Director has cited
           the primary benefit of deploying ASP monitors as reducing
           unnecessary secondary inspections.15 DNDO's cost-benefit analysis
           focused on measuring the benefits of ASP's ability to reduce false
           alarms--alarms that indicate nuclear or radiological material is
           present when, in fact, it is not or such material is actually
           non-threatening. Reducing false alarms would reduce the number of
           secondary inspections of non-threatening nuclear and radiological
           materials and therefore the costs of those inspections. Even on
           this point, however, DNDO's analysis was flawed. For example, it
           did not estimate the costs of secondary inspections as they are
           carried out today. DNDO's analysis needs these baseline costs to
           compare alternatives because without them, it is impossible to
           determine whether the use of ASPs, as planned, will result in cost
           savings for secondary inspections. While we agree that
           facilitating commerce at U.S. ports of entry by reducing
           unnecessary secondary inspections is an important goal, we believe
           that the primary rationale for deploying portal monitors should be
           to protect the nation from nuclear and/or radiological attack. We
           found that DNDO's analysis did not even attempt to measure the
           level--or value--of security afforded by portal monitors.

           In addition, the ASP contract award has exceeded DNDO's estimate
           for total cost by about $200 million. The cost-benefit analysis
           shows the total cost for deploying PVT and ASP monitors to be
           about $1 billion, which covers all costs related to acquisition,
           design, maintenance, and physical inspection over 5 years (for
           both PVT and ASP). However, in July 2006, DHS announced that it
           had awarded contracts to develop and purchase up to $1.2 billion
           worth of ASP portal monitors over 5 years. Furthermore, the
           cost-benefit analysis underestimates total deployment costs and
           does not account for other major costs, such as physical
           inspections of cargo containers, an additional procurement of 442
           new PVT monitors, installation and integration, and maintenance.

           Finally, DNDO overstated the purchase price of PVT monitors.
           Although DHS is currently paying an average of about $55,000 per
           monitor, DNDO's cost-benefit analysis assumed the PVT would cost
           $130,959--the highest published manufacturers' price for the
           government.16 According to DNDO's Director, DNDO chose the highest
           published price because the current contract for portal monitors
           at that time was to expire, and the portal monitors will probably
           cost more in the future. However, the information DNDO provided us
           does not explain why DNDO assumes that the future price will be
           more than double what DHS was currently paying, as assumed in
           DNDO's analysis. PNNL officials told us that the future price will
           almost certainly be lower than the price used in DNDO's analysis.
			  

4GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support
the Purchase of New Radiation Detection Portal Monitors Was Not Based on
Available Performance Data and Did Not Fully Evaluate All the Monitors'
Costs and Benefits, [20]GAO-07-133R (Washington, D.C.: Oct. 17, 2006).

5"Masking" is an attempt to hide dangerous nuclear or radiological
material by placing it with benign radiological sources.

6Pub. L. No. 109-347, tit. V, 120 Stat. 1884, 1932 (2006).

7Prices include only equipment purchase. Installation costs are extra.

8Department of Homeland Security Appropriations Act for Fiscal Year 2007,
Pub. L. No. 109-295, tit. IV, 120 Stat. 1355, 1376 (2006).

9DHS and DOE are collaborating in building a new Radiological and Nuclear
Countermeasures Test and Evaluation Complex at the Nevada Test Site to
support the development, testing, acquisition, and deployment of radiation
detection equipment. The facility is expected to become fully operational
in early 2007. Currently, an interim facility at NTS is being used to test
radiation detection equipment.

10NIST did not evaluate the PVTs or compare their performance to the
performance of the ASPs.

11The ability to detect masked HEU is based on DOE guidance on performing
the evaluation.

12The performance specifications contain a requirement for detecting, not
identifying, californium-252 with a 95 percent probability.
Californium-252 has similar radiological properties to HEU. In addition,
the performance specifications contain a requirement for detecting, but
not identifying, other radiological materials such as cobalt-57,
cobalt-60, barium-133, cesium-137, and americium-241.

13Pacific Northwest National Laboratory, A Sensitivity Comparison of NaI
and PVT Portal Monitors at a Land-Border Port-of-Entry, p. iii, November
2004. For Official Use Only.

14DHS, Capital Planning and Investment Control, Cost-Benefit Analysis
(CBA) Guidebook 2006, Version 2.0, February 2006. Traditional rules of
performing cost-benefit analyses include assessing the full life-cycle
costs for operation and maintenance, and determining the level of
confidence in cost data.

15Enlisting Foreign Cooperation in U.S. Efforts to Prevent Nuclear
Smuggling: Hearing Before the House Committee on Homeland Security,
Subcommittee on Prevention of Nuclear and Biological Attack, 109th Cong.
(May 25, 2006)(statement of Mr. Vayl S. Oxford, Director, DNDO); Detecting
Smuggled Nuclear Weapons, Hearing Before the Senate Judiciary Committee,
Subcommittee on Terrorism, Technology, and Homeland Security, 109th Cong.
(July 27, 2006)(statement of Mr. Vayl S. Oxford, Director, DNDO).

16DNDO, Cost Benefit Analysis for Next Generation Passive Radiation
Detection of Cargo an the Nation's Border Crossings, May 30, 2006.

           In conclusion, DNDO's approach to the cost-benefit analysis
           omitted many factors that could affect the cost of new radiation
           portal monitors. For these reasons, DHS's cost-benefit analysis
           does not meet the intent of our March 2006 report recommendation
           to fully assess the costs and benefits before purchasing any new
           equipment.

           Mr. Chairman, this concludes my prepared statement. I would be
           happy to respond to any questions that you or other members of the
           Subcommittee may have.
			  
			  GAO Contact and Staff Acknowledgements

           For further information about this testimony, please contact me,
           Gene Aloise, at (202) 512-3841 or at [email protected]. Contact
           points for our Offices of Congressional Relations and Public
           Affairs may be found on the last page of this statement. John
           Delicath, Jim Shafer, and Eugene Wisnoski made key contributions
           to this statement.
			  
			  GAOï¿½s Mission

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.
			  
			  Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each
           weekday, GAO posts newly released reports, testimony, and
           correspondence on its Web site. To have GAO e-mail you a list of
           newly posted products every afternoon, go to www.gao.gov and
           select "Subscribe to Updates."
			  
			  Order by Mail or Phone

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061
			  
			  To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470
			  
			  Congressional Relations

           Gloria Jarmon, Managing Director, [email protected] (202)
           512-4400 U.S. Government Accountability Office, 441 G Street NW,
           Room 7125 Washington, D.C. 20548
			  
			  Public Affairs

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

www.gao.gov/cgi-bin/getrpt?GAO-07-581T .

To view the full product, including the scope

and methodology, click on the link above.

For more information, contact Gene Aloise at (202) 512-3841 or
[email protected].

Highlights of [22]GAO-07-581T , a testimony before the Subcommittee on
Emerging Threats, Cybersecurity, and Science and Technology; Committee on
Homeland Security; House of Representatives

March 14, 2007

COMBATING NUCLEAR SMUGGLING

DHS's Decision to Procure and Deploy the Next Generation of Radiation
Detection Equipment Is Not Supported by Its Cost-Benefit Analysis

The Department of Homeland Security (DHS) is responsible for addressing
the threat of nuclear smuggling. Radiation detection portal monitors are
key elements in our national defenses against such threats. DHS has
sponsored R&D and testing activities to develop a "next generation" portal
monitor, known as the advanced spectroscopic portal monitor. However, each
one costs 6 times more than a current portal monitor. In March 2006, we
recommended that DHS conduct a cost-benefit analysis to determine whether
the new portal monitors are worth the additional cost. In June 2006, DHS
issued its analysis. In October 2006, we issued our report that assessed
the DHS study.

GAO's statement, based on our October 2006 report, addresses whether DHS's
cost-benefit analysis provides an adequate basis for its decision to
purchase and deploy the next generation portal monitors.

[23]What GAO Recommends

GAO's October 2006 report included two recommendations designed to improve
the quality and comprehensiveness of the Department's analysis. DHS
neither agreed nor disagreed with our recommendations, but continued to
support its analysis as a solid basis for buying and deploying the new
generation of radiation portal monitors.

DHS's cost-benefit analysis does not provide a sound analytical basis for
its decision to purchase and deploy the new portal monitor technology. Our
review of the analysis determined that it had the following problems:

Regarding the performance of the portal monitors:

           o Instead of using the results of its own portal monitor tests
           conducted in 2005, DHS assumed that the new portal monitor
           technology would correctly detect and identify highly enriched
           uranium (HEU) 95 percent of the time--a performance level that far
           exceeds the new technology's current capabilities.
           o To determine the performance of the current generation of portal
           monitors in detecting HEU, DHS used data from limited tests
           carried out in 2004 that test officials concluded were unreliable
           for such purposes.
           o DHS's analysis of the new technology portal monitors was
           incomplete because the analysis focused on identifying HEU, but
           did not fully consider how well the new portal monitor technology
           could correctly detect or identify other dangerous radiological or
           nuclear materials.

Regarding cost estimates:

           o In comparing the costs of the new and current technologies, the
           procurement costs of the current generation portal monitors were
           highly inflated because DHS assumed a unit cost of about $131,000.
           However, the contract price at the time of the analysis was about
           $55,000. According to officials who manage the contract, it was to
           expire and they expected portal monitor prices to increase, but
           not nearly as much as DHS assumed.
           o DHS stated that the primary benefit of deploying the new portal
           monitors is reducing unnecessary secondary inspections. However,
           DHS's analysis does not fully estimate today's baseline costs for
           secondary inspections, which makes it impossible to determine
           whether the use of the new portal monitors as currently planned,
           will result in significant cost savings for these inspections.
           o The new portal monitor contract price has exceeded DHS's total
           cost estimate by about $200 million. The cost-benefit analysis
           shows the total cost for deploying both current and new portal
           monitors to be about $1 billion. However, in July 2006, DHS
           announced that it had awarded contracts to develop and purchase up
           to $1.2 billion worth of the new portal monitors over 5 years.
           o DHS's cost-benefit analysis omitted many factors that could
           affect the cost of new portal monitors, such as understating the
           life-cycle costs for operating and maintaining the equipment over
           time.

For these reasons, DHS's cost-benefit analysis does not meet the intent of
our March 2006 report recommendation to fully assess the costs and
benefits before purchasing any new equipment.

References

Visible links
  11. http://www.gao.gov/cgi-bin/getrpt?GAO-07-133R
  12. http://www.gao.gov/cgi-bin/getrpt?GAO-06-389
  13. http://www.gao.gov/cgi-bin/getrpt?GAO-06-389
  20. http://www.gao.gov/cgi-bin/getrpt?GAO-07-133R
  22. http://www.gao.gov/cgi-bin/getrpt?GAO-07-581T
*** End of document. ***