    [House Hearing, 111 Congress]
    [From the U.S. Government Publishing Office]

                            THE SCIENCE OF SECURITY,
                                 PARTS I AND II

    =======================================================================

                                    HEARINGS

                                   BEFORE THE

                       SUBCOMMITTEE ON INVESTIGATIONS AND
                                   OVERSIGHT

                      COMMITTEE ON SCIENCE AND TECHNOLOGY
                            HOUSE OF REPRESENTATIVES

                         ONE HUNDRED ELEVENTH CONGRESS

                                 FIRST SESSION

                                   __________

                                 JUNE 25, 2009
                                      AND
                               NOVEMBER 17, 2009

                                   __________

                               Serial No. 111-38
                                      and
                               Serial No. 111-63

                                   __________

         Printed for the use of the Committee on Science and Technology

         Available via the World Wide Web: http://www.science.house.gov

                                     ______

                            U.S. GOVERNMENT PRINTING OFFICE
    50-324 PDF                      WASHINGTON: 2010
    -----------------------------------------------------------------------
    For Sale by the Superintendent of Documents, U.S. Government Printing Office
    Internet: bookstore.gpo.gov  Phone: toll free (866) 512-1800; (202) 512-1800
    Fax: (202) 512-2104 Mail: Stop IDCC, Washington, DC 20402-0001

                      COMMITTEE ON SCIENCE AND TECHNOLOGY

                       HON. BART GORDON, Tennessee, Chair
    JERRY F. COSTELLO, Illinois          RALPH M. HALL, Texas
    EDDIE BERNICE JOHNSON, Texas         F. JAMES SENSENBRENNER JR.,
    LYNN C. WOOLSEY, California              Wisconsin
    DAVID WU, Oregon                     LAMAR S. SMITH, Texas
    BRIAN BAIRD, Washington              DANA ROHRABACHER, California
    BRAD MILLER, North Carolina          ROSCOE G. BARTLETT, Maryland
    DANIEL LIPINSKI, Illinois            VERNON J. EHLERS, Michigan
    GABRIELLE GIFFORDS, Arizona          FRANK D. LUCAS, Oklahoma
    DONNA F. EDWARDS, Maryland           JUDY BIGGERT, Illinois
    MARCIA L. FUDGE, Ohio                W. TODD AKIN, Missouri
    BEN R. LUJAN, New Mexico             RANDY NEUGEBAUER, Texas
    PAUL D. TONKO, New York              BOB INGLIS, South Carolina
    PARKER GRIFFITH, Alabama             MICHAEL T. MCCAUL, Texas
    STEVEN R. ROTHMAN, New Jersey        MARIO DIAZ-BALART, Florida
    JIM MATHESON, Utah                   BRIAN P. BILBRAY, California
    LINCOLN DAVIS, Tennessee             ADRIAN SMITH, Nebraska
    BEN CHANDLER, Kentucky               PAUL C. BROUN, Georgia
    RUSS CARNAHAN, Missouri              PETE OLSON, Texas
    BARON P. HILL, Indiana
    HARRY E. MITCHELL, Arizona
    CHARLES A. WILSON, Ohio
    KATHLEEN DAHLKEMPER, Pennsylvania
    ALAN GRAYSON, Florida
    SUZANNE M. KOSMAS, Florida
    GARY C. PETERS, Michigan
    VACANCY
                                     ------

                  Subcommittee on Investigations and Oversight

                    HON. BRAD MILLER, North Carolina, Chair
    STEVEN R. ROTHMAN, New Jersey        PAUL C. BROUN, Georgia
    LINCOLN DAVIS, Tennessee             BRIAN P. BILBRAY, California
    CHARLES A. WILSON, Ohio              VACANCY
    KATHY DAHLKEMPER, Pennsylvania
    ALAN GRAYSON, Florida
    BART GORDON, Tennessee               RALPH M. HALL, Texas
                    DAN PEARSON Subcommittee Staff Director
                      EDITH HOLLEMAN Subcommittee Counsel
                JAMES PAUL Democratic Professional Staff Member
           DOUGLAS S. PASTERNAK Democratic Professional Staff Member
               KEN JACOBSON Democratic Professional Staff Member
                        ALEX MATTHEWS Research Assistant
                TOM HAMMOND Republican Professional Staff Member
                          JANE WISE Research Assistant
                                C O N T E N T S

    The Science of Security, Part I: Lessons Learned in Developing, Testing
                   and Operating Advanced Radiation Monitors

                                 June 25, 2009

                                                                       Page
    Witness List.....................................................     2

    Hearing Charter..................................................     3

                               Opening Statements

    Statement by Representative Brad Miller, Chairman, Subcommittee
      on Investigations and Oversight, Committee on Science and
      Technology, U.S. House of Representatives......................     8
        Written Statement............................................     9

    Statement by Representative Paul C. Broun, Ranking Minority
      Member, Subcommittee on Investigations and Oversight, Committee
      on Science and Technology, U.S. House of Representatives.......    10
        Written Statement............................................    11

                                    Panel I:

    Mr. Gene Aloise, Director, Natural Resources and Environment,
      U.S. Government Accountability Office (GAO)
        Oral Statement...............................................    12
        Written Statement............................................    14
        Biography....................................................    22

    Dr. Micah D. Lowenthal, Director, Nuclear Security and Nuclear
      Facility Safety Program, Nuclear and Radiation Studies Board,
      National Research Council, The National Academies
        Oral Statement...............................................    22
        Written Statement............................................    24
        Biography....................................................    28

    Discussion
      Prioritizing Security Needs....................................    29
      Decision-making: Processes and Timelines.......................    30
      ASP in Context: The Global Nuclear Detection Architecture......    31
      Management of ASP Testing and Deployment.......................    32
      Justifying the Cost of the Advanced Spectroscopic Portal
        Program......................................................    33

                                   Panel II:

    Dr. William K. Hagan, Acting Deputy Director, Domestic Nuclear
      Detection Office, Department of Homeland Security
        Oral Statement...............................................    35
        Written Statement............................................    37
        Biography....................................................    40

    Mr. Todd C. Owen, Acting Deputy Assistant Commissioner, Office of
      Field Operations, U.S. Customs and Border Protection,
      Department of Homeland Security
        Oral Statement...............................................    41
        Written Statement............................................    42
        Biography....................................................    46

    Discussion.......................................................    47

                  Appendix: Additional Material for the Record

    Combating Nuclear Smuggling: DHS Improved Testing of Advanced
      Radiation Detection Portal Monitors, but Preliminary Results
      Show Limits of the New Technology, Report to Congressional
      Requesters, U.S. Government Accountability Office, GAO-09-655,
      May 2009.......................................................    54

    Prepublication Copy, Evaluating Testing, Costs, and Benefits of
      Advanced Spectroscopic Portals for Screening Cargo at Ports of
      Entry, Interim Report (Abbreviated Version), Committee on
      Advanced Spectroscopic Portals, Nuclear and Radiation Studies
      Board, Division of Earth and Life Studies, National Research
      Council of the National Academies..............................    86

    The Science of Security, Part II: Technical Problems Continue to Hinder
                          Advanced Radiation Monitors

                               November 17, 2009

    Witness List.....................................................   178

    Hearing Charter..................................................   179

                               Opening Statements

    Statement by Representative Brad Miller, Chairman, Subcommittee
      on Investigations and Oversight, Committee on Science and
      Technology, U.S. House of Representatives......................   184
        Written Statement............................................   185

    Statement by Representative Paul C. Broun, Ranking Minority
      Member, Subcommittee on Investigations and Oversight, Committee
      on Science and Technology, U.S. House of Representatives.......   186
        Written Statement............................................   187

                                   Witnesses:

    Mr. Gene Aloise, Director, Natural Resources and Environment,
      U.S. Government Accountability Office (GAO)
        Oral Statement...............................................   193
        Written Statement............................................   195
        Biography....................................................   199

    Dr. Timothy M. Persons, Chief Scientist, U.S. Government
      Accountability Office (GAO)
        Biography....................................................   200

    Mr. Todd C. Owen, Acting Deputy Assistant Commissioner, Office of
      Field Operations, U.S. Customs and Border Protection,
      Department of Homeland Security
        Oral Statement...............................................   200
        Written Statement............................................   202
        Biography....................................................   206

    Dr. William K. Hagan, Acting Deputy Director, Domestic Nuclear
      Detection Office, Department of Homeland Security
        Oral Statement...............................................   207
        Written Statement............................................   208
        Biography....................................................   211

    Discussion
      CBP Procedures After a Primary Alarm and the Effect of False
        Positives....................................................   212
      Steps Taken to Reduce False Positives and Negatives............   213
      Mission Critical Failure.......................................   215
      Energy Windowing to Improve PVT Performance....................   216
      The CBP Inspection System......................................   217
      The Helium-3 Shortage and Potential Alternative Materials......   218
      In What Circumstances Should ASPs Be Deployed?.................   219
      Expectations of a Cost-benefit Analysis on ASPs................   220
      Metrics and Timelines for Making Decisions About ASPs..........   222

                  Appendix: Answers to Post-Hearing Questions

    Dr. William K. Hagan, Acting Deputy Director, Domestic Nuclear
      Detection Office, Department of Homeland Security; and Mr. Todd
      C. Owen, Acting Deputy Assistant Commissioner, Office of Field
      Operations, U.S. Customs and Border Protection, Department of
      Homeland Security..............................................   230

    Mr. Gene Aloise, Director, Natural Resources and Environment,
      U.S. Government Accountability Office (GAO); Mr. Todd C. Owen,
      Acting Deputy Assistant Commissioner, Office of Field
      Operations, U.S. Customs and Border Protection, Department of
      Homeland Security; and Dr. William K. Hagan, Acting Deputy
      Director, Domestic Nuclear Detection Office, Department of
      Homeland Security..............................................   235

        THE SCIENCE OF SECURITY, PART I: LESSONS LEARNED IN DEVELOPING,
               TESTING, AND OPERATING ADVANCED RADIATION MONITORS

                                  ----------

                            THURSDAY, JUNE 25, 2009

                      House of Representatives,
          Subcommittee on Investigations and Oversight,
                           Committee on Science and Technology,
                                                        Washington, DC.

        The Subcommittee met, pursuant to call, at 10:00 a.m., in
    Room 2318 of the Rayburn House Office Building, Hon. Brad
    Miller [Chairman of the Subcommittee] presiding.
    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

                                hearing charter

                  SUBCOMMITTEE ON INVESTIGATIONS AND OVERSIGHT

                      COMMITTEE ON SCIENCE AND TECHNOLOGY

                         U.S. HOUSE OF REPRESENTATIVES

                    The Science of Security, Part I: Lessons

                      Learned in Developing, Testing, and

                     Operating Advanced Radiation Monitors

                            Thursday, June 25, 2009
                             10:00 a.m.-12:00 p.m.
                       2318 Rayburn House Office Building

    Purpose

        The Subcommittee on Investigations and Oversight meets on June 25,
    2009 to examine problems with the Department of Homeland Security's
    (DHS) efforts to acquire its next generation radiation monitors known
    as Advanced Spectroscopic Portals (ASPs). The ASP program has been
    under scrutiny since 2006 for failing to have clear-cut program
    requirements, an adequate test plan, sufficient timelines and
    development milestones or a transparent and comprehensive cost benefit
    analysis. Since the Domestic Nuclear Detection Office (DNDO), a DHS
    component, was created in 2005, they have been responsible for
    researching, developing, testing and managing the program.
        The hearing will examine two new independent reports--one by the
    Government Accountability Office (GAO) and the other by the National
    Academy of Sciences--that identify ongoing and systematic problems in
    the testing and development of the ASP program. With an estimated
    program cost of $2-to-$3 billion the Subcommittee will evaluate the
    rigor of the overall test program, the technical abilities of the ASPs
    compared to existing radiation portal monitors and search for lessons
    from the ASP program that can be applied to future DHS acquisitions.

    Background

        Since the terrorist attacks of September 11, 2001, protecting the
    Nation from a nuclear or radiological attack has been a top national
    security priority. In 2002, to help address this potential threat, the
    U.S. Customs and Border Protection (CBP) agency began deploying
    radiation monitors at U.S. border sites and ports of entry so its
    officers could screen the more than 23 million containers of cargo that
    enter the country every year for radiological and nuclear materials.
        The equipment used to screen this cargo both then and now are
    polyvinyl toluene (PVT) or ``plastic'' portal monitors able to detect
    the presence of radioactive sources, but unable to identify the type of
    radiation present. The PVT monitors, while relatively inexpensive,
    robust and highly reliable, are unable to distinguish between
    radioactive sources that might be used to construct a nuclear bomb,
    such as Highly Enriched Uranium (HEU), and non-threatening naturally
    occurring radiological materials (NORM) contained in ceramic tiles,
    zirconium sand or kitty liter, for instance. As a result, any time a
    PVT monitor detects a radioactive source the cargo is sent to
    ``secondary'' screening where CBP agents verify the detection of the
    radioactive source with a second PVT monitor and use hand-held
    Radioactive Isotope Identification Devices called RIIDs to help
    identify the source of radiation.
        This method of operation leads to many ``secondary'' inspections
    for naturally occurring radioactive material or radioactive material
    intended for benign purposes, such as radioactive medical isotopes. At
    the Los Angeles/Long Beach port of entry, for instance, PVT monitors
    routinely send up to 600 conveyances of cargo to secondary inspection
    each day. The RIIDs, used in secondary inspections however, are limited
    in their abilities to locate and identify potential threat material in
    large cargo containers. As a result, CBP officers can consult with
    scientists in CBP's Laboratories and Scientific Services (LSS) unit who
    can often help them enhance the ability to correctly identify the
    radioactive material of concern. As a last resort, CBP officers may
    physically search a cargo container by emptying its contents and
    closely scrutinizing it for potentially dangerous radioactive material.
        If terrorists were to try to smuggle nuclear or radiological
    materials in containerized cargo--and there are ample other pathways
    for such smuggling--they would likely try to shield or ``mask'' those
    materials in an attempt to make it more difficult to detect, identify
    and locate the material of concern. Shielding requires that lead or
    steal or other types of metal enclose the radioisotopes to hide its
    radioactive signature. Potential terrorists may also attempt to
    ``mask'' threatening radioactive material by placing it together with
    or alongside other non-threatening material that has a natural
    radioactive signature, such as ceramic material, kitty litter or even
    bananas. Most nuclear security experts believe smuggled radioactive or
    nuclear material would be both shielded and masked in order to conceal
    it from being located and properly identified. Obviously, these efforts
    would make it harder to detect.
        In order to help both improve the flow of commerce by eliminating
    many of the false alarms that send cargo for secondary screening and to
    more accurately identify radioactive or nuclear material, the
    Department of Homeland Security (DHS) began developing Advanced
    Spectroscopic Portals (ASPs) in 2004. The ASPs were intended to both
    detect and identify radioactive material. In April 2005, the Domestic
    Nuclear Detection Office was created by National Security Presidential
    Directive-43/Homeland Security Presidential Directive-14 to, among
    other things, research, develop, test and acquire radiation detection
    equipment to be used by CBP and other federal agencies. The office was
    not formally established until October 2006 under the SAFE Port Act.
        From the very start of the ASP program, DNDO seemed to push for
    acquisition decisions well before the technology had demonstrated that
    it could live up to its promise. On July 14, 2006, Secretary of
    Homeland Security Michael Chertoff and the Director of DNDO, Vayl
    Oxford, announced contract awards to three companies worth an estimated
    $1.2 billion to develop the ASPs, including the Raytheon Company, from
    Massachusetts, the Thermo Electron Company from Santa Fe, New Mexico
    and Canberra Industries from Connecticut. Both Chertoff and Oxford held
    a press conference to announce the billion dollar contract awards just
    a few months after highly critical reviews of the ASPs' abilities by
    the GAO and the National Institute of Standards and Technology (NIST).
        In March 2006, GAO said: ``it is not clear that the benefits of the
    new portals would be worth any increased cost to the program.'' In June
    2006, NIST submitted a report to DHS on results of side-by-side testing
    the previous year at the Nevada Test Site of both ASP and PVT systems.
    The DNDO had assumed that the ASPs would correctly identify HEU 95
    percent of the time for both bare or unmasked HEU and HEU masked in a
    container with more benign radiological material. Yet, NIST found that
    the three best ASP systems tested identified HEU only 70 to 88 percent
    of the time. Their ability to identify ``masked'' HEU was much worse.
    The three ASP manufacturers did this only 53 percent of the time
    (Raytheon), 45 percent of the time (Thermo) and 17 percent of the time
    (Canberra). ``Despite these results,'' the GAO found, ``DNDO did not
    use the information from these tests in its cost-benefit analysis.''
    DNDO claimed that they assumed they would meet the 95 percent
    performance level at some point in the future but provided no data on
    why they reached this conclusion, said GAO.
        At the Chertoff-Oxford press conference in July 2006, then
    Secretary of Homeland Security, Michael Chertoff, said one of the key
    reasons for developing the ASPs and replacing the existing radiation
    monitors was to ``have fewer false positives.'' In September 2007, Vayl
    Oxford, the Director of DNDO reiterated that point in testimony to
    Congress where he emphasized that the ASPs would reduce the number of
    false alarms from the nearly 600 experienced each day by the PVTs at
    the port of Long Beach in California, for instance, to 20 to 25 per day
    with the new ASP monitors.
        That was the hope, anyway. One of the criteria for ASP primary
    screening prior to certification of the new radiation monitors by the
    Secretary of Homeland Security, which is required by the appropriations
    committees, is that the ASPs must refer at least 80 percent fewer
    conveyances for further inspection than the PVTs. But in ``field
    validation tests'' earlier this year, by one of the two remaining
    contractors, the ASPs being tested sent more innocent radioactive
    shipments to secondary screening than the older PVT monitors. The cause
    of the high false positives was apparently due to a software glitch.
    This was a serious concern to the Customs and Border Protection
    personnel who will have to operate and maintain the ASPs if and when
    they are certified and deployed. The contractor has reportedly
    corrected the software issue and intends to return the ASPs to field
    validation testing next month.
        Last fall, ``integration'' testing of the ASPs by the second
    remaining contractor was halted because of different technical troubles
    with its own software. The contractor corrected the problem and its ASP
    machines re-entered integration testing late last year. The contractor
    hopes to finish integration testing and begin field validation testing
    in early August. Still, both contractors are now many months behind
    schedule because technical issues have forced delays.
        Virtually any high-technology research and development program
    experiences bumps in the road, technical troubles and occasional set-
    backs. However, well managed programs have clear technical requirements
    and strategic goals. They ensure that the new technology being
    developed is thoroughly tested and adequately integrated into the
    operational plans and procedures of those who must operate them in the
    field. When these vital components are short changed, when the test
    plan is insufficient and the program's research, development and
    testing methods are marred by scanty scientific rigor, the technical
    tools being developed are bound to suffer as a result. Cutting critical
    corners in the development process serves no one's interests. Yet, from
    the start many of the leaders of the ASP program at DHS seemed more
    interested in fielding this technology then in vigilantly validating
    its performance and effectiveness. At the July 2006 press conference
    unveiling the contractors on the ASP program, for instance, Vayl Oxford
    said: ``the priority for the first year . . . is to get units out
    immediately.'' Three years later, none of these units have yet cleared
    field validation tests.
        The policy governing the ASP program and the disproportionate focus
    on getting the ASP units into the field quickly never matched the
    multiple independent technical assessments of the technology being
    developed and tested. Over the past three years the Government
    Accountability Office has issued six reports on the ASP program and
    testified before Congress multiple times on this matter. Last year the
    Homeland Security Institute, a Federally Funded Research and
    Development Center for DHS, issued a report on the ASPs that also
    criticized the ASP test program, saying it provided insufficient data.
    The National Academy's of Science, which will release an interim report
    on the ASPs that they have just concluded this week, will provide
    testimony at the Subcommittee hearing that echoes many of the concerns
    raised by GAO over the years.

    History of Problems

        In 2006, the GAO issued a harsh critique of the DNDO's cost-
    benefit-analysis (CBA) of the ASPs. The DNDO analysis omitted critical
    test data that identified major technical problems with the ASPs and
    they drastically increased the procurement costs of the PVTs. In short,
    the GAO found DNDO's cost-benefit analysis was ``incomplete,'' based on
    ``unreliable'' data and used ``inflated cost estimates for PVT
    equipment.''
        In 2007, GAO concluded that tests of the ASPs conducted by DNDO
    were ``biased'' and ``were not an objective and rigorous assessment of
    the ASPs' capabilities.'' The tests, for instance, used insufficient
    amounts of materials likely to mask or shield radioactive threat
    sources that terrorists might attempt to smuggle into the country. The
    tests, said GAO, did not attempt to test the limitations of the ASPs
    and ``did not objectively test the performance'' of currently used
    hand-held radiation detectors or RIIDs.
        Last year, in their own independent cost estimate of the ASP
    program, GAO found that the ASPs could cost about $3.1 billion, $1
    billion more than the DNDO's estimate. The GAO also found that the DNDO
    had often changed its deployment strategy, eliminating plans to develop
    ASP portals for rail, airport and seaport cargo screening terminals,
    for instance. As a result, GAO estimated the newest scaled back plan
    reduced the potential costs of the program to about $2 billion from
    2008 to 2017. The only documentation that DNDO provided to GAO for this
    major change in the ASP program was a one-page spread-sheet and DNDO
    has still not released an updated cost-benefit analysis of the program.
        In addition, GAO criticized DNDO's decision not to complete
    computerized simulations or ``injection studies'' of the ASPs prior to
    certification by the Secretary of Homeland Security. The National
    Academy of Sciences has also found that computer modeling is critically
    important to the ASP program since running every potential radioactive
    smuggling scenario in live tests is unrealistic. Computer simulations
    would help provide a clearer assessment of the potential performance of
    the ASPs in actual smuggling incidents and effectiveness at identifying
    threatening radioactive material. DNDO, however, does not plan to
    complete the studies prior to the Secretary of Homeland Security's
    decision on certification, which DNDO expects to occur in October.

    Problems Remain

        While DNDO's past tests have been characterized as being unsound,
    incomplete and limited in scope, the GAO's most recent work on the ASP
    program does point to some improvements in the integrity of the latest
    round of tests. However, they also pinpointed significant technical
    limitations which have not yet been resolved.
        The ASP portals did prove more effective than the PVTs in detecting
    HEU materials concealed by ``light shielding.'' However, differences
    between the ASPs and PVTs became less notable when shielding was
    slightly increased or decreased. In past tests there was virtually no
    difference in the performance of the two machines with regard to
    detecting other kinds of radioactive isotopes, such as those used for
    medical or industrial purposes, according to the GAO, except in one
    case where the ASPs performed worse than the PVTs. Whether these other
    forms of radioactive sources are sensed by a PVT or ASP machine they
    all require secondary inspection to determine why a payload contains
    radioactive material. In detecting HEU, the ASPs performed better only
    in one narrowly defined scenario, which many experts see as an
    unrealistic portrayal of a true attempted nuclear smuggling incident.
    None of the tests run by DNDO, for instance, included scenarios that
    utilized both ``shielding'' and ``masking'' as a means of attempting to
    smuggle radioactive or nuclear material.
        In addition, GAO and others have faulted DNDO for not focusing
    enough on attempting to improve the current radiation portal monitor
    program. Instead, DNDO has been nearly single-mindedly focused on
    developing Advanced Spectroscopic Portals at the expense of other far
    simpler alternatives. Surprisingly, for instance, DNDO has not
    completed efforts to improve the performance of PVTs by a method called
    Energy Windowing that could provide them with some limited, but
    enhanced, performance. Energy Windowing efforts are controversial and
    are believed to only provide modest enhancements to the performance of
    PVTs. But both GAO and CBP has been pushing DNDO to do more on this
    front for years. In addition, DNDO has not made efforts to upgrade the
    software in the hand-held radiation detection units known as RIIDs that
    could also provide a far less expensive alternative to enhancing the
    operational effectiveness of radiation monitors.
        Because both remaining ASP contractors suffered from serious
    technical problems in their last round of testing, Customs and Border
    Protection (CBP) agency personnel fear that if the ASPs are certified,
    procured and deployed that they will encounter many problems in the
    field that will negatively impact their day-to-day operations and
    perhaps the technical effectiveness of the current radiation monitoring
    program to actually detect illicit nuclear or radiological material
    coming into the country. The GAO, National Academy of Sciences and
    others have also criticized DNDO for not seeking input from CBP
    officials on the ASP program from the start. The relationship has
    improved and DNDO does attempt to include CBP in critical decisions
    regarding the ASP program today. But many critics say perhaps one of
    DNDO's biggest failures was the fact that they did not do this from the
    beginning, seeking input from the operational users of the technology
    that DNDO was tasked to research, test and develop.
        As a result of all of these issues, the ASPs continue to suffer
    from key questions about their ability to provide significant improved
    performance over existing radiation detection equipment currently
    fielded at U.S. ports. The Department of Homeland Security has already
    spent more than $235 million on the ASP program. But if the Secretary
    of Homeland Security certifies that the ASP monitors are worth
    investing in this fall--just three to four months from now--then $2
    billion more may be invested to procure ASP radiation monitors. Yet,
    given the continued criticism of the narrowly focused and inadequate
    ASP test program, the limited technical improvements they may offer
    over current radiation monitors and significant increased costs to
    maintain and operate the ASPs compared to the PVTs, the success of the
    program remains in doubt.

    Key Issues

            [bullet]  Go Slow. Uncovering and resolving technical problems
            once newly developed radiation monitors are fielded may hinder
            the ability to detect and identify radioactive or nuclear
            material that poses a potential threat. It could disrupt
            operations at U.S. borders and ports of entry curtailing the
            flow of commerce and it will cost more to rectify these
            problems in the field, rather than in the laboratory or at the
            test range. Yet, rather than carefully testing and validating
            the performance and effectiveness of the ASP monitors before a
            major procurement decision is made DHS has continually sought
            to get the ASPs into the field in spite of critical technical
            flaws identified during testing.

            [bullet]  Cost Benefit Analysis. Even if the technical
            abilities of the ASPs are proven, their relative technical
            capabilities and increased costs must be carefully weighed in
            comparison to the existing radiation monitoring system in place
            today. Replacing a proven, less-costly system that has the
            confidence of its operators, must be given careful
            consideration. The DNDO has not yet provided an updated cost-
            benefit-analysis that would validate a decision to procure the
            multi-billion dollar ASP equipment.

            [bullet]  Judging Performance. As the House Committee on
            Appropriations has said in the past, procurement of the
            Advanced Spectroscopic Portal monitors should not proceed until
            they are deemed to add a ``significant increase in operational
            effectiveness'' over the current PVT system already in place.
            Last July, CBP, DNDO and the DHS management directorate jointly
            issued criteria for determining this increase in effectiveness
            in both ``primary'' and ``secondary'' screening. In primary
            screening the criteria requires ASPs to detect potential
            threats as well as or better than PVTs, show improved detection
            of Highly Enriched Uranium and reduce innocent alarms. In
            secondary screening the criteria requires ASPs to reduce the
            probability of misidentifying special nuclear material (HEU or
            plutonium) and reduce the average time to conduct secondary
            screenings. The Secretary of Homeland Security must certify to
            Congress that the ASPs have met these criteria before funding
            for full-scale procurement of the ASPs goes forward. However,
            the criteria to measure this improvement are weak and rather
            vague.

              Lessons Learned. The Department of Homeland Security
            must make greater efforts to avoid rushing to acquisition
            decisions when the R&D is incomplete. With ASPs, the research
            and development program itself has been hindered by a lack of
            rigorous scientific evaluations, and undemanding testing
            protocols. Moving to acquire systems plagued by such problems
            may endanger security and significantly increase the costs of
            the program. A review of DHS's major programs by GAO last
            November found that 45 of 48 major programs did not adhere to
            the agency's own investment review process that helps provide
            appropriate oversight to address cost, schedule and performance
            problems. In FY 2008, the review found, DHS spent $147.5
            million on the ASP program despite the fact it did not have a
            mission needs statement. The program also lacked operational
            requirements documents and an acquisition program baseline.

    Witnesses

    Panel I:

    Mr. Gene Aloise, Director, Natural Resources and Environment,
    Government Accountability Office

    Dr. Micah Lowenthal, Division on Earth and Life Studies, Nuclear and
    Radiation Studies Board, National Research Council, The National
    Academy of Sciences

    Panel II:

    Dr. William Hagan, Acting Deputy Director, Domestic Nuclear Detection
    Office (DNDO), Department of Homeland Security (DHS)

    Mr. Todd C. Owen, Acting Deputy Assistant Commissioner, Office of Field
    Operations, U.S. Customs & Border Protection (CBP), Department of
    Homeland Security (DHS).
        Chairman Miller. The hearing will now come to order. Good
    morning. Welcome to today's hearing, The Science of Security:
    Lessons Learned in Developing, Testing, and Operating Advanced
    Radiation Monitors.
        In the wake of the terrorist attacks on September 11, 2001,
    preventing the detonation of a nuclear or radiologic device--a
    `dirty bomb'--in the United States has become a top national
    security objective. We have invested billions of dollars since
    9/11 to develop the means to prevent, detect and respond to any
    attack by weapons of mass destruction. We developed radiation
    monitors at our port and border crossings to screen millions of
    cargo containers entering the United States every year, hunting
    for radiological material that could be used for terrorist
    purposes. Since 2004, the Department of Homeland Security has
    spent more than $230 million on a program to develop a new
    radiation detection system called an Advanced Spectroscopic
    Portal, or ASP, that can both detect and identify nuclear
    material.
        Congress expects that the funding federal agencies receive
    will be well spent. When it comes to scientifically challenging
    or technically demanding programs, it pays to have well-
    prepared program requirements, demanding testing protocols and
    an independent and comprehensive cost-benefit analysis. Those
    vital steps, those vital program components help managers make
    informed decisions about whether to move forward with a
    technology development program or to replace a proven
    technology with a new technology. Unfortunately, despite recent
    progress, the ASP program has suffered because it lacked all
    the preparatory steps of a well-managed program. We will hear
    about some of those problems today from the Government
    Accountability Office and the National Academies of Science.
        Over the years, the GAO has released six reports on the ASP
    program. The GAO found that some of the Domestic Nuclear
    Detection Office's tests were biased and did not provide a
    rigorous assessment of ASP's capabilities. The Agency relied on
    incomplete and unreliable data in their cost-benefit analysis,
    omitting critical test data, inflating the cost of current
    radiation detectors and underestimating the cost of ASP
    monitors. The Department failed to produce a requirements
    document or adequate documentation regarding major changes to
    the planned ASP deployment strategy. DNDO never considered the
    option of investigating improvements of the existing radiation
    portal monitoring program, both the PVT monitors and the hand-
    held detectors that Customs and Border Protection agents rely
    upon.
        The National Academy of Science's interim report on the ASP
    program, released yesterday, reflects many of the same
    concerns. The Academy calls for significant restructuring of
    DHS testing procedures for the ASP program. They question the
    criteria being used to judge the ASPs' performance and they
    recommend that DHS not proceed with further ASP procurement
    until they address all the findings and recommendations in
    their report.
        GAO's reports have provided a regular accounting of how the
    ASP program was going wrong. The Academy report provides a
    roadmap to how the program can be put back on track, assuming
    that the Department determines that it is worth the cost and
    effort.
        Radioactive materials have become a normal part of
    commerce. They are used for medical procedures and industrial
    applications. Bananas have radiation. Technology can help us
    detect and identify radioactive sources in cargo containers but
    no technology can sort out good radioactive material intended
    for legitimate purposes from the bad radioactive materials
    intended to do us harm. As a result, human operators will still
    need to make important decisions often informed by intelligence
    efforts to keep the Nation secure. Well-trained, well-equipped
    people, law enforcement officers and Customs and Border
    Protection inspectors will always be critical to the equation.
        This hearing addresses our responsibility to the
    technological part of that equation. Before we move forward
    with a $2 billion or $3 billion program, we must ensure that we
    get our money's worth from the new technology. Put another way,
    if we have $2 billion or $3 billion to spend to enhance our
    security, is this technology really how we should spend it?
        I look forward to hearing from all of our witnesses today.
        And now I recognize Dr. Broun, the Ranking Member from
    Georgia, for an opening statement.
        [The prepared statement of Chairman Miller follows:]
                   Prepared Statement of Chairman Brad Miller
        In the wake of the terrorist attacks of September 11, 2001,
    preventing the detonation of a nuclear or radiological device in the
    U.S. has become a top national security objective.
        We have invested billions of dollars since 9/11 to develop the
    means to prevent, detect and respond to any attack by weapons of mass
    destruction. We have deployed radiation monitors at our ports and
    border crossings to screen millions of cargo containers entering the
    U.S. every year, hunting for radiological material that could be used
    for terrorist purposes. Since 2004, the Department of Homeland Security
    (DHS) has spent more than $230 million on a program to develop a new
    radiation detection system called an Advanced Spectroscopic Portal or
    ASP that can both detect and identify nuclear material.
        Congress expects that the funding federal agencies receive will be
    well spent. When it comes to scientifically challenging or technically
    demanding programs, studies have shown that it pays to have well-
    prepared program requirements, demanding testing protocols and an
    independent and comprehensive cost benefit analysis. These vital
    program components help managers make informed decisions about whether
    to move forward with a technology development program, or to replace a
    proven technology with a new technology. Unfortunately, despite some
    recent progress, the ASP program has suffered because it lacked all the
    preparatory steps of a well managed program. We will hear about some of
    those problems today from both Government Accountability Office (GAO)
    and the National Academies of Science.
        Over the past three years the GAO has released six reports on the
    ASP program. The GAO found that some of the Domestic Nuclear Detection
    Office's (DNDO) tests were ``biased'' and did not provide a rigorous
    assessment of the ASP's capabilities. The agency relied on
    ``incomplete'' and ``unreliable'' data in their cost-benefit analysis--
    omitting critical test data, inflating the costs of the current
    radiation detectors, and underestimating the costs of ASP monitors. The
    Department failed to produce a requirements document or adequate
    documentation regarding major changes to their planned ASP deployment
    strategy. DNDO never considered the option of investing in improvements
    to the existing radiation portal monitoring program--both the current
    polyvinyl toluene (PVT) monitors and the hand-held detectors Customs
    and Border Protection agents rely upon.
        The National Academy of Sciences' interim report on the ASP
    program, released yesterday, reflects many of the same concerns. The
    Academy calls for a significant restructuring of DHS testing procedures
    for the ASP program, they question the criteria being used to judge the
    ASP's performance and they recommend that DHS not proceed with further
    ASP procurement until they address all of the findings and
    recommendations in their report.
        GAO's reports have provided a regular accounting of how the ASP
    program was going wrong; the Academy's report provides a roadmap to how
    the program could be put back on track--assuming that the Department
    determines that is worth the cost and effort.
        Radioactive materials have become a normal part of commerce. They
    are used for medical procedures and industrial applications. Technology
    can help us detect and identify radioactive sources in cargo
    containers, but no technology can sort out ``good'' radioactive
    material intended for legitimate purposes from the ``bad'' radioactive
    material intended to do us harm. As a result, human operators will need
    to make important decisions, often informed by intelligence efforts, to
    keep the Nation secure. Well trained, well equipped people--law
    enforcement officers and customs and border protection inspectors--will
    always be critical to the equation.
        This hearing addresses our responsibility to the technological part
    of that equation. Before we move forward with a two to three billion
    dollar program we must ensure that we get our money's worth from the
    new technology. Put another way, if we have two or three billion
    dollars to spend to enhance our security, is this technology really how
    we should spend it?
        I look forward to hearing from all of our witnesses today.

        Mr. Broun. Thank you, Mr. Chairman. I want to welcome our
    witnesses here today and to thank you for participating in this
    important hearing on the Department of Homeland Security's
    Advanced Spectroscopic Portal program. I know it is hard to
    say, particularly for the non-scientist, but I am a scientist--
    I am a physician.
        Yesterday the House took up consideration of the DHS
    appropriations bill, and later today the Science Committee's
    Technology and Innovation Subcommittee, which I am also a
    Member of, will hold a hearing on cyber security. I also sit on
    the Homeland Security Committee's Subcommittee on Emerging
    Threats, Cyber Security, Science and Technology as well.
    Needless to say, DHS has kept me busy this week.
        This morning we will look into the status of the
    Department's ongoing development of the next-generation
    radiation portal monitors, get an update from GAO on their
    continued work, and receive a report from the National Research
    Council. It goes without saying that this program has been
    followed closely for some time now and thankfully many of the
    testing issues that GAO has brought up in previous reports seem
    to have been mitigated, at least somewhat. However, this
    program is far from out of the woods. In their most recent
    analysis, GAO and the Academy raised new issues relating to the
    rigor of the testing and the certification process and offer
    paths forward for potential acquisition in the future. I hope
    DHS takes these recommendations seriously, and I look forward
    to ensuring that they are not summarily dismissed for the sake
    of arbitrary timetables. We see that frequently here in
    government.
        Looking forward, DHS should conduct a rigorous cost-benefit
    analysis that takes into account updated threat assessments, a
    review of all variations of concepts of operations, potential
    upgrades to existing technologies and independent cost
    estimates. It also needs to weigh the pros and cons not just of
    ASP versus Polyvinyl Toluene (PVT) and Radio-Isotope
    Identification Devices (RIID), but also whether the additional
    capability gained outweighs the needs of other aspects of the
    global nuclear detection architecture. Unfortunately, this may
    be hard to do at this point considering GAO indicated earlier
    this year that the Domestic Nuclear Detection Office had, to
    quote GAO, ``not developed an overarching strategic plan to
    guide its development of a more comprehensive global strategy
    for nuclear detection.''
        All of these factors need to be taken into consideration as
    DHS moves toward an acquisition. Even then I will remain
    cautious, given the Department's track record with past
    acquisitions. Many of the issues we are dealing with today
    could have been prevented by engaging the end-users early on in
    the process and also by clearly defining the requirements and
    simply following existing Department acquisition processes.
        Last week this subcommittee held a hearing on issues
    plaguing NPOESS. The AST program exhibits eerie similarities to
    that program in that it attempted to link research and
    development activities with the acquisition of an operational
    system and had unclear architectural priorities. Let us hope
    other federal programs can learn from these lessons and protect
    taxpayers from future inefficiencies and waste.
        Mr. Chairman, I yield back the balance of my time and I
    look forward to our witnesses' testimony. Thank you.
        [The prepared statement of Mr. Broun follows:]
               Prepared Statement of Representative Paul C. Broun
        Thank you, Mr. Chairman. I want to welcome the witnesses here
    today, and thank them for participating in this important hearing on
    the Department of Homeland Security's (DHS) Advanced Spectroscopic
    Portal (ASP) program.
        Yesterday the House took up consideration of the DHS Appropriation
    bill, and later today the Science Committee's Technology and Innovation
    Subcommittee, which I also sit on, will hold a hearing on cyber
    security. I also sit on the Homeland Security Committee's Subcommittee
    on Emerging Threats, Cyber Security, Science and Technology, as well.
    Needless to say, DHS has kept me busy this week.
        This morning we will look into the status of the Department's
    ongoing development of next-generation Radiation Portal Monitors, get
    an update from General Accountability Office (GAO) on their continuing
    work, and receive a report from the National Academy of Sciences. It
    goes without saying that this program has been followed closely for
    some time now, and thankfully many of the testing issues that GAO
    brought up in previous reports seem to be mitigated. However, this
    program is far from ``out of the woods.'' In their most recent
    analysis, GAO and the Academy raise new issues relating to the rigor of
    the testing and certification process, and offer paths forward for a
    potential acquisition in the future. I hope DHS takes these
    recommendations seriously, and I look forward to ensuring that they are
    not summarily dismissed for the sake of arbitrary timetables.
        Looking forward, DHS should conduct a rigorous Cost-Benefit
    Analysis (CBA) that takes into account updated threat assessments, a
    review of all variations of Concepts of Operations (CONOPS), potential
    upgrades for existing technologies, and independent cost estimates. It
    also needs to weigh the pros and cons of not just ASP versus Polyvinyl
    Toluene (PVT) and Radio-isotope Identification Devices (RIID), but also
    whether the additional capability gained outweighs the needs of other
    aspects of the Global Nuclear Detection Architecture. Unfortunately,
    this may be hard to do at this point considering GAO indicated earlier
    this year that the Domestic Nuclear Detection Office (DNDO) had ``not
    developed an overarching strategic plan to guide its development of a
    more comprehensive global strategy for nuclear detection.''
        All of these factors need to be taken into consideration as DHS
    moves toward an acquisition. Even then, I will remain cautious given
    the Department's track record with past acquisitions. Many of the
    issues we are dealing with today could have been prevented by engaging
    the end-users early in the process, clearly defining requirements, and
    simply following existing Department acquisition processes.
        Last week this subcommittee held a hearing on issues plaguing the
    National Polar-Orbiting Environmental Satellite System (NPOESS). The
    ASP program exhibits eerie similarities to that program in that it
    attempted to link research and development activities with the
    acquisition of an operational system, and had unclear architectural
    priorities. Let's hope other federal programs can learn from these
    lessons and protect taxpayers from future inefficiencies and waste.
        With that, Mr. Chairman, I yield back my time.
        Thank you.

        Chairman Miller. Thank you, Dr. Broun.

                                    Panel I:

        Chairman Miller. I am now pleased to introduce our panel of
    witnesses. Mr. Gene Aloise is the Director of Natural Resources
    and Environment at the Government Accountability Office, GAO,
    and Dr. Micah Lowenthal is the Director of the Nuclear Security
    and Nuclear Facility Safety Program in the Nuclear and
    Radiation Studies Board at the National Research Council of the
    National Academy of Sciences. Is that how you describe your job
    at a cocktail party?
        As our witnesses should know, you each have five minutes
    for your spoken testimony. Your written testimony will be
    included in the record for the hearing. When you have all
    completed your spoken testimony, we will begin with questions
    and each Member will have five minutes to question the panel.
    It is the practice of this subcommittee to receive testimony
    under oath. Do either of you have any objection to taking an
    oath? Okay. You also have the right to be represented by
    counsel. Do either of you have counsel here? If you would
    please stand and raise your right hand? Do you swear to tell
    the truth and nothing but the truth?
        The record will reflect that both Mr. Aloise and Dr.
    Lowenthal took the oath. Mr. Aloise, you may begin.

     STATEMENT OF MR. GENE ALOISE, DIRECTOR, NATURAL RESOURCES AND
        ENVIRONMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE (GAO)

        Mr. Aloise. Thank you, Mr. Chairman.
        Mr. Chairman and Members of the Subcommittee, I am pleased
    to be here today to discuss DHS's plans to develop and test
    advanced portal monitors for use at the Nation's borders to
    prevent nuclear materials from being smuggled into the United
    States. According to DHS, the current system of radiation
    detection equipment is effective and does not impede the flow
    of commerce. However, DHS wants to improve the capabilities of
    the existing equipment with new equipment.
        One of the major drawbacks of the new equipment is the
    substantially higher cost compared to the existing equipment.
    We estimated in September 2008 that the life cycle cost of each
    standard cargo version of the ASP to be about $823,000 compared
    to about $308,000 for the PVT standard cargo portal and that
    the total program cost for the Domestic Nuclear Detection
    Office's (DNDO) latest deployment plan would be about $2
    billion.
        Since 2006, we have issued six reports, and including
    today, four testimonies, and have made 19 recommendations for
    improving DNDO's efforts to develop and test portal monitors.
    Our concerns have focused on the need for realistic and
    objective testing of ASPs, full disclosure and reporting of
    testing limitations, development of cost estimates that
    consider the full cost to deploy the new equipment, and
    development of a sound cost-benefit analysis which determines
    whether the marginal increase in security from the ASPs is
    worth its very high cost.
        My testimony today is based on our recent report, which
    assessed the most recent round of ASP testing. I will also
    discuss the lessons learned from ASP testing. Our work on the
    latest round of ASP testing found that DHS increased the rigor
    in comparison with previous tests, and thereby added
    credibility to the test results. However, we still question
    whether the benefits of the ASP justify the high cost. In
    particular, DHS's criteria for significant increase in
    operational effectiveness require only marginal improvement in
    the detection of certain weapons-usable nuclear material. The
    marginal improvement required of ASPs is particularly notable
    given that DNDO has not completed efforts to enhance the
    performance of the current generation of equipment. Customs and
    Border Protection (CBP) officials have told us that they have
    repeatedly urged DNDO to investigate improving the performance
    of the equipment through what is known as energy windowing.
    DNDO has collected the data necessary to do this, but has not
    yet completed efforts to analyze the data and further improve
    the technique.
        Our analysis of the new test results shows that ASPs
    detected certain nuclear materials better than PVTs when
    shielding approximated DOE threat guidance, which is based on
    light shielding. However, differences between the two systems
    were hard to recognize when shielding was slightly increased or
    decreased. Both systems had difficulty in detecting nuclear
    materials when shielding was somewhat greater than threat
    guidance, which is set to match the extent of the detection
    limits of the PVTs. Importantly, the threat guidance is not a
    realistic approximation of how a terrorist might shield nuclear
    material to successfully smuggle it undetected through a border
    crossing.
        In addition, DNDO underestimated the time needed for ASP
    testing, which was originally supposed to be finished by
    September 2008. DHS's most recent schedule anticipated ASP
    certification around May 2009, but testing has been delayed
    even further, and DHS has not updated its schedule. As far as
    we know, certification is scheduled for some time this fall.
        Of concern to us is the fact that DNDO does not plan to
    complete simulations that could provide additional insights
    into ASP capabilities and limitations prior to certification.
    On this point, DNDO does not seem to have learned from past
    mistakes. A rush to deploy ASPs before all testing is complete
    leads to shortcuts in testing and raises questions among
    stakeholders about the equipment's effectiveness and
    reliability. Furthermore, DNDO has not yet updated its cost-
    benefit analysis, which might show that DNDO's plan to replace
    existing equipment is not justified.
        One of the primary lessons to be learned from our work is
    to avoid the pitfalls in testing that stem from a rush to
    procure new technologies. In the case of the ASP, a push to
    rush to replace existing equipment led to a testing program
    which lacked scientific rigor. Even for the new round of
    testing, DNDO consistently underestimated the time necessary to
    conduct tests and resolve problems, and testing is still not
    completed.
        In closing, we believe that given the importance of this
    new equipment to our national security, that Congress and the
    American taxpayer still need to know three things: does the
    equipment work, how much will it cost; and does the marginal
    increase in security justify its very high cost.
        Mr. Chairman, that concludes my remarks and I would be
    happy to respond to any questions you and the Ranking Member
    may have.
        [The prepared statement of Mr. Aloise follows:]
                       Prepared Statement of Gene Aloise

                      Combating Nuclear Smuggling: Lessons

                      Learned From DHS Testing of Advanced

                      Radiation Detection Portal Monitors

    Mr. Chairman and Members of the Subcommittee:

        I am pleased to be here today to discuss GAO's work on the
    Department of Homeland Security's (DHS) testing of advanced
    spectroscopic portal (ASP) radiation detection monitors. As you are
    aware, the national security mission of U.S. Customs and Border
    Protection (CBP), an agency within DHS, includes screening for smuggled
    nuclear or radiological material that could be used in a nuclear weapon
    or radiological dispersal device (a ``dirty bomb''). To screen cargo at
    ports of entry, CBP conducts primary inspections with radiation
    detection equipment called portal monitors--large stationary detectors
    through which cargo containers and vehicles pass as they enter the
    United States. When radiation is detected, CBP conducts secondary
    inspections using a second portal monitor to confirm the original alarm
    and a hand-held radioactive isotope identification device to identify
    the radiation's source and determine whether it constitutes a threat.
        The polyvinyl toluene (PVT) portal monitors currently in use can
    detect radiation but cannot identify the type of material causing an
    alarm. As a result, the monitors' radiation alarms can be set off even
    by benign, naturally occurring radioactive material. One way to reduce
    the rate of such innocent alarms--and thereby minimize unnecessary
    secondary inspections and enhance the flow of commerce--is to adjust
    the operational thresholds (i.e., operate the PVTs at a reduced level
    of sensitivity). However, reducing the sensitivity may make it more
    difficult to detect certain nuclear materials.
        To address the limitations of current-generation portal monitors,
    DHS's Domestic Nuclear Detection Office (DNDO) in 2005 began to develop
    and test ASPs, which are designed to both detect radiation and identify
    the source.\1\ DNDO hopes to use the new portal monitors to replace at
    least some PVTs currently used for primary screening, as well as PVTs
    and hand-held identification devices currently used for secondary
    screening. However, in September 2008, we estimated the life cycle cost
    of each standard cargo version of the ASP (including deployment costs)
    to be about $822,000, compared with about $308,000 for the PVT standard
    cargo portal, and the total program cost for DNDO's latest plan for
    deploying radiation portal monitors--which relies on a combination of
    ASPs and PVTs and does not deploy radiation portal monitors at all
    border crossings--to be about $2 billion.\2\
    ---------------------------------------------------------------------------
        \1\ DNDO was established within DHS in 2005; its mission includes
    developing, testing, acquiring, and supporting the deployment of
    radiation detection equipment at U.S. ports of entry. CBP began
    deploying portal monitors in 2002, prior to DNDO's creation, under the
    radiation portal monitor project.
        \2\ GAO, Combating Nuclear Smuggling: DHS's Program to Procure and
    Deploy Advanced Radiation Detection Portal Monitors Is Likely to Exceed
    the Department's Previous Cost Estimates, GAO-08-1108R (Washington,
    D.C.: Sept. 22, 2008).
    ---------------------------------------------------------------------------
        Concerned about the performance and cost of the ASP monitors,
    Congress required the Secretary of Homeland Security to certify that
    the monitors will provide a ``significant increase in operational
    effectiveness'' before DNDO obligates funds for full-scale ASP
    procurement.\3\ The Secretary must submit separate certifications for
    primary and secondary inspection. In response, CBP, DNDO, and the DHS
    management directorate jointly issued criteria in July 2008 for
    determining whether the new technology provides a significant increase
    in operational effectiveness. The primary screening criteria require
    that the new portal monitors detect potential threats as well as or
    better than PVTs, show improved performance in detection of highly
    enriched uranium (HEU), and reduce innocent alarms. To meet the
    secondary screening criteria, the new portal monitors must reduce the
    probability of misidentifying special nuclear material (e.g., HEU and
    plutonium) and the average time to conduct secondary screenings.
    ---------------------------------------------------------------------------
        \3\ Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121
    Stat. 1844, 2069 (2007); Consolidated Security, Disaster Assistance,
    and Continuing Appropriations Act, 2009, Pub. L. No. 110-329, 121 Stat.
    3574, 3679 (2008).
    ---------------------------------------------------------------------------
        DNDO designed and coordinated a new series of tests, originally
    scheduled to run from April 2008 through September 2008, to determine
    whether the new portal monitors meet the certification criteria for
    primary and secondary screening and are ready for deployment. Key
    phases of this testing program include concurrent testing led by DNDO
    of the new and current equipment's ability to detect and identify
    threats and of ASPs' readiness to be integrated into operations for
    both primary and secondary screening at ports of entry; field
    validation led by CBP at four northern and southern border crossings
    and two seaports; and an independent evaluation, led by the DHS Science
    and Technology Directorate at one of the seaports, of the new portal
    monitors' effectiveness and suitability.
        Since 2006, we have issued six reports and four testimonies on
    development of radiation detection portal monitors, including today's
    testimony, and have made 19 recommendations for improving DNDO's
    efforts to develop and test portal monitors. Our concerns have focused
    on key areas in which DNDO's efforts have lacked the necessary rigor
    given ASPs' high cost and the importance of the radiation portal
    monitor project to our national security. These areas include objective
    and realistic testing of ASPs' performance in comparison with the
    performance of current-generation equipment; full disclosure and
    reporting of the limitations of tests used to support a decision by the
    Secretary of Homeland Security on ASP certification; development of a
    cost estimate that considers the full costs of the plan for deploying
    radiation detection portal monitors; and development of a cost-benefit
    analysis based on ASPs' demonstrated performance and a complete
    accounting of the portal monitor project's costs. (App. I presents a
    summary of our key findings and recommendations related to ASPs.) As I
    will discuss today, DNDO has improved the rigor of testing but has not
    yet updated the cost-benefit analysis that is critical to a decision on
    whether to replace radiation detection equipment already deployed at
    ports of entry with the significantly more expensive ASPs.
        Specifically, my testimony discusses (1) our key findings on the
    most recent round of ASP testing and (2) lessons from ASP testing that
    can be applied to other DHS technology investments. These findings are
    based on our report released this week and other related GAO
    reports.\4\ We conducted this performance audit work in June 2009 in
    accordance with generally accepted government auditing standards. Those
    standards require that we plan and perform the audit to obtain
    sufficient, appropriate evidence to produce a reasonable basis for our
    findings and conclusions based on our audit objectives. We believe that
    the evidence obtained provides a reasonable basis for our statement
    today.
    ---------------------------------------------------------------------------
        \4\ GAO, Combating Nuclear Smuggling: DHS Improved Testing of
    Advanced Radiation Detection Portal Monitors, but Preliminary Results
    Show Limits of the New Technology, GAO-09-655 (Washington, D.C.: May
    21, 2009).

    The Latest Round of Testing Highlights the Limitations of ASPs

        Our report on the latest round of ASP testing found that DHS
    increased the rigor of ASP testing in comparison with previous tests
    and that a particular area of improvement was in the performance
    testing at the Nevada Test Site, where DNDO compared the capability of
    ASP and current-generation equipment to detect and identify nuclear and
    radiological materials. For example, unlike in prior tests, the plan
    for the 2008 performance test stipulated that there would be no system
    contractor involvement in test execution. Such improvements addressed
    concerns we previously raised about the potential for bias and provided
    credibility to the results.
        Nevertheless, based on the following factors, we continue to
    question whether the benefits of the new portal monitors justify the
    high cost:

            [bullet]  The DHS criteria for a significant increase in
            operational effectiveness. Our chief concern with the criteria
            is that they require a marginal improvement over current-
            generation portal monitors in the detection of certain weapons-
            usable nuclear materials when ASPs are deployed for primary
            screening. DNDO considers detection of such materials to be a
            key limitation of current-generation portal monitors. We are
            particularly concerned about the marginal improvement required
            of ASPs because the detection threshold for the current-
            generation portal monitors does not specify a level of
            radiation shielding that smugglers could realistically use. DOE
            and national laboratory officials told us that DOE's threat
            guidance used to set the current detection threshold is based
            not on an analysis of the capabilities of potential smugglers
            to take effective shielding measures but rather on the limited
            sensitivity of PVTs to detect anything more than certain
            lightly shielded nuclear materials. DNDO officials acknowledge
            that both the new and current-generation portal monitors are
            capable of detecting certain nuclear materials only when
            unshielded or lightly shielded. The marginal improvement in
            detection of such materials required of ASPs is particularly
            notable given that DNDO has not completed efforts to fine-tune
            PVTs' software and thereby improve sensitivity to nuclear
            materials. DNDO officials expect they can achieve small
            improvements in sensitivity, but DNDO has not yet funded
            efforts to fine-tune PVTs' software. In contrast to the
            marginal improvement required in detection of certain nuclear
            materials, the primary screening requirement to reduce the rate
            of innocent alarms could result in hundreds of fewer secondary
            screenings per day, thereby reducing CBP's workload and delays
            to commerce. In addition, the secondary screening criteria,
            which require ASPs to reduce the probability of misidentifying
            special nuclear material by one-half, address the inability of
            relatively small hand-held devices to consistently locate and
            identify potential threats in large cargo containers.

            [bullet]  Preliminary results of performance testing and field
            validation. The preliminary results presented to us by DNDO are
            mixed, particularly in the capability of ASPs used for primary
            screening to detect certain shielded nuclear materials.
            Preliminary results show that the new portal monitors detected
            certain nuclear materials better than PVTs when shielding
            approximated DOE threat guidance, which is based on light
            shielding. In contrast, differences in system performance were
            less notable when shielding was slightly increased or
            decreased: Both the PVTs and ASPs were frequently able to
            detect certain nuclear materials when shielding was below
            threat guidance, and both systems had difficulty detecting such
            materials when shielding was somewhat greater than threat
            guidance. With regard to secondary screening, ASPs performed
            better than hand-held devices in identification of threats when
            masked by naturally occurring radioactive material. However,
            differences in the ability to identify certain shielded nuclear
            materials depended on the level of shielding, with increasing
            levels appearing to reduce any ASP advantages over the hand-
            held identification devices. Other phases of testing uncovered
            multiple problems in meeting requirements for successfully
            integrating the new technology into operations at ports of
            entry. Of the two ASP vendors participating in the 2008 round
            of testing, one has fallen behind due to severe problems
            encountered during testing of ASPs' readiness to be integrated
            into operations at ports of entry (``integration testing'');
            the problems may require that the vendor redo previous test
            phases to be considered for certification. The other vendor's
            system completed integration testing, but CBP suspended field
            validation after two weeks because of serious performance
            problems resulting in an overall increase in the number of
            referrals for secondary screening compared with existing
            equipment.

            [bullet]  DNDO's plans for computer simulations. DNDO does not
            plan to complete injection studies--computer simulations for
            testing the response of ASPs and PVTs to simulated threat
            objects concealed in cargo containers--prior to the Secretary
            of Homeland Security's decision on certification even though
            delays to the ASP test schedule have allowed more time to
            conduct the studies. According to DNDO officials, injection
            studies address the inability of performance testing to
            replicate the wide variety of cargo coming into the United
            States and the inability to place special nuclear material and
            other threat objects in cargo during field validation. DNDO had
            earlier indicated that injection studies could provide
            information comparing the performance of the two systems as
            part of the certification process for both primary and
            secondary screening. However, DNDO subsequently decided that
            performance testing would provide sufficient information to
            support a decision on ASP certification. DNDO officials said
            they would instead use injection studies to support effective
            deployment of the new portal monitors.

            [bullet]  Lack of an updated cost-benefit analysis. DNDO has
            not yet updated its cost-benefit analysis to take into account
            the results of the latest round of ASP testing. An updated
            analysis that takes into account the results from the latest
            round of testing, including injection studies, might show that
            DNDO's plan to replace existing equipment with ASPs is not
            justified, particularly given the marginal improvement in
            detection of certain nuclear materials required of ASPs and the
            potential to improve the current-generation portal monitors'
            sensitivity to nuclear materials, most likely at a lower cost.
            DNDO officials said they are currently updating the ASP cost-
            benefit analysis and plan to complete it prior to a decision on
            certification by the Secretary of Homeland Security.

        Our report recommended that the Secretary of Homeland Security
    direct DNDO to (1) assess whether ASPs meet the criteria for a
    significant increase in operational effectiveness based on a valid
    comparison with PVTs' full performance potential and (2) revise the
    schedule for ASP testing and certification to allow sufficient time for
    review and analysis of results from the final phases of testing and
    completion of all tests, including injection studies. We further
    recommended that, if ASPs are certified, the Secretary direct DNDO to
    develop an initial deployment plan that allows CBP to uncover and
    resolve any additional problems not identified through testing before
    proceeding to full-scale deployment. DHS agreed to a phased deployment
    that should allow time to uncover ASP problems but disagreed with GAO's
    other recommendations, which we continue to believe remain valid.

    Procurement Decisions for New Technologies Require Rigorous Testing and
                        Thorough Analysis of Results

        The challenges DNDO has faced in developing and testing ASPs
    illustrate the importance of following existing DHS policies as well as
    best practices for investments in complex homeland security
    acquisitions and for testing of new technologies. The DHS investment
    review process calls for executive decision-making at key points in an
    investment's life cycle and includes many acquisition best practices
    that, if applied consistently, could help increase the chances for
    successful outcomes. However, we reported in November 2008 that, for
    the period from fiscal year 2004 through the second quarter of fiscal
    year 2008, DHS had not effectively implemented or adhered to its
    investment review process due to a lack of senior management officials'
    involvement as well as limited monitoring and resources.\5\ In
    particular, of DHS's 48 major investments requiring milestone and
    annual reviews under the Department's investment review policy, 45 were
    not assessed in accordance with this policy. In addition, many major
    investments, including DNDO's ASP program, had not met the Department's
    requirements for basic acquisition documents necessary to inform the
    investment review process. As a result, DHS had not consistently
    provided the oversight needed to identify and address cost, schedule,
    and performance problems in its major investments. Among other things,
    our November 2008 report recommended that the Secretary of Homeland
    Security direct component heads, such as the Director of DNDO, to
    ensure that the components have established processes to manage major
    investments consistent with departmental policies. DHS generally
    concurred with our recommendations, and we noted that DHS had begun
    several efforts to address shortcomings in the investment review
    process identified in our report, including issuing an interim
    directive requiring DHS components to align their internal policies and
    procedures by the end of the third quarter of fiscal year 2009. In
    January 2009, DHS issued a memorandum instructing component heads to
    create acquisition executives in their organizations to be responsible
    for management and oversight of component acquisition processes. If
    fully implemented, these steps should help ensure that DHS components
    have established processes to manage major investments.
    ---------------------------------------------------------------------------
        \5\ GAO, Department of Homeland Security: Billions Invested in
    Major Programs Lack Appropriate Oversight, GAO-09-29 (Washington, D.C.:
    Nov. 18, 2008).
    ---------------------------------------------------------------------------
        Based on our body of work on ASP testing, one of the primary
    lessons to be learned is to avoid the pitfalls in testing that stem
    from a rush to procure new technologies. GAO has previously reported on
    the negative consequences of pressures imposed by closely linking
    testing and development programs with decisions to procure and deploy
    new technologies, including the creation of incentives to postpone
    difficult tests and limit open communication about test results.\6\ We
    found that testing programs designed to validate a product's
    performance against increasing standards for different stages in
    product development are a best practice for acquisition strategies for
    new technologies. In the case of ASPs, the push to replace existing
    equipment with the new portal monitors led to a testing program that
    until recently lacked the necessary rigor. Even for the most recent
    round of testing, DNDO's schedule consistently underestimated the time
    required to conduct tests, resolve problems uncovered during testing,
    and complete key documents, including final test reports. In addition,
    DNDO's original working schedule did not anticipate the time required
    to update its cost-benefit analysis to take into account the latest
    test results. The schedule anticipated completion of testing in mid-
    September 2008 and the DHS Secretary's decision on ASP certification
    between September and November 2008. However, testing is still not
    completed, and DNDO took months longer than anticipated to complete the
    final report on performance testing.
    ---------------------------------------------------------------------------
        \6\ GAO, Best Practices: A More Constructive Test Approach Is Key
    to Better Weapon System Outcomes, GAO/NSIAD-00-199 (Washington, D.C.:
    July 31, 2000).

        As previously mentioned, a number of aspects of the latest round of
    ASP testing increased the rigor in comparison with earlier rounds and,
    if properly implemented, could improve the rigor in DHS's testing of
    ---------------------------------------------------------------------------
    other advanced technologies. Key aspects included the following:

            [bullet]  Criteria for ensuring test requirements are met. The
            test and evaluation master plan established criteria requiring
            that the ASPs meet certain requirements before starting or
            completing any test phase. For example, the plan required that
            ASPs have no critical or severe issues rendering them
            completely unusable or impairing their function. The criteria
            provided a formal means to ensure that ASPs met certain basic
            requirements prior to the start of each phase of testing. DNDO
            and CBP adhered to the criteria even though doing so resulted
            in integration testing taking longer than anticipated and
            delaying the start of field validation.

            [bullet]  Participation of the technology end-user. The
            participation of CBP (the end-user of the new portal monitors)
            provided an independent check, within DHS, of DNDO's efforts to
            develop and test the new portal monitors. For example, CBP
            added a final requirement to integration testing before
            proceeding to field validation to demonstrate ASPs' ability to
            operate for 40 hours without additional problems and thereby
            provide for a productive field validation. In addition, the
            participation of CBP officers in the 2008 round of performance
            testing allowed DNDO to adhere more closely than in previous
            tests to CBP's standard operating procedure for conducting a
            secondary inspection using the hand-held identification
            devices, thereby providing for an objective test.

            [bullet]  Participation of an independent test authority. The
            DHS Science and Technology Directorate, which is responsible
            for developing and implementing the Department's test and
            evaluation policies and standards, will have the lead role in
            the final phase of ASP testing and thereby provide an
            additional independent check on testing efforts. The Science
            and Technology Directorate identified two critical questions,
            related to ASPs' operational effectiveness (i.e., detection and
            identification of threats) and suitability (e.g., reliability,
            maintainability, and supportability), and drafted its own test
            plan to address those questions.

        Mr. Chairman, this completes my prepared statement. I would be
    happy to respond to any questions that you or other Members of the
    Subcommittee may have at this time.

    GAO Staff Acknowledgments

        Ned Woodward (Assistant Director), Joseph Cook, and Kevin Tarmann
    made key contributions to this testimony. Dr. Timothy Persons (Chief
    Scientist), James Ashley, Steve Caldwell, John Hutton, Omari Norman,
    Alison O'Neill, Amelia Shachoy, and Rebecca Shea also made important
    contributions.

    Appendix I:

                     Key Findings and Recommendations from

                        Related GAO Products on Testing

                            and Development of ASPs

    Combating Nuclear Smuggling: DHS Has Made Progress Deploying Radiation
            Detection Equipment at U.S. Ports-of-Entry, But Concerns
            Remain. GAO-06-389. Washington, D.C.: March 22, 2006.

              Key findings. Prototypes of advanced spectroscopic
            portals (ASP) were expected to be significantly more expensive
            than current-generation portal monitors but had not been shown
            to be more effective. For example, Domestic Nuclear Detection
            Office (DNDO) officials' preliminary analysis of 10 ASPs tested
            at the Nevada Test Site found that the new portal monitors
            outperformed current-generation equipment in detecting numerous
            small, medium-size, and threat-like radioactive objects and
            were able to identify and dismiss most naturally occurring
            radioactive material. However, the detection capabilities of
            both types of portal monitors converged as the amount of source
            material decreased.

              Recommendations. We recommended that the Secretary of
            Homeland Security work with the Director of DNDO to analyze the
            benefits and costs of deploying ASPs before any of the new
            equipment is purchased to determine whether any additional
            detection capability is worth the additional cost. We also
            recommended that the total program cost estimate for the
            radiation portal monitor project be revised after completion of
            the cost-benefit analysis.

    Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support the
            Purchase of New Radiation Detection Portal Monitors Was Not
            Based on Available Performance Data and Did Not Fully Evaluate
            All the Monitors' Costs and Benefits. GAO-07-133R. Washington,
            D.C.: October 17, 2006.

    Combating Nuclear Smuggling: DHS's Decision to Procure and Deploy the
            Next Generation of Radiation Detection Equipment Is Not
            Supported by Its Cost-Benefit Analysis. GAO-07-581T.
            Washington, D.C.: March 14, 2007.

            [bullet]  Key findings. DNDO's cost-benefit analysis issued in
            response to our March 2006 recommendation did not provide a
            sound analytical basis for DNDO's decision to purchase and
            deploy ASPs. We identified a number of problems with the
            analysis of both the performance of the new portal monitors and
            the costs. With regard to performance, DNDO did not use the
            results of its own tests and instead relied on assumptions of
            the new technology's anticipated performance level. In
            addition, the analysis focused on identifying highly enriched
            uranium (HEU) and did not consider how well the new portal
            monitors can correctly detect or identify other dangerous
            radiological or nuclear materials. With regard to costs, DNDO
            did not follow the DHS guidelines for performing cost-benefit
            analyses and used questionable assumptions about the
            procurement costs of portal monitor technology.

            [bullet]  Recommendations. We recommended that DHS and DNDO
            conduct a new cost-benefit analysis using sound analytical
            methods, including actual performance data and a complete
            accounting of all major costs and benefits as required by DHS
            guidelines, and that DNDO conduct realistic testing for both
            ASPs and current-generation portal monitors.

    Combating Nuclear Smuggling: DNDO Has Not Yet Collected Most of the
            National Laboratories' Test Results on Radiation Portal
            Monitors in Support of DNDO's Testing and Development Program.
            GAO-07-347R. Washington, D.C.: March 9, 2007.

            [bullet]  Key findings. DNDO had not collected a comprehensive
            inventory of testing information on current-generation portal
            monitors. Such information, if collected and used, could
            improve DNDO's understanding of how well portal monitors detect
            different radiological and nuclear materials under varying
            conditions. In turn, this understanding would assist DNDO's
            future testing, development, deployment, and purchases of
            portal monitors.

            [bullet]  Recommendations. We recommended that the Secretary of
            Homeland Security, working with the Director of DNDO, collect
            reports concerning all of the testing of current-generation
            portal monitors and review the test reports in order to develop
            an information database on how the portal monitors perform in
            both laboratory and field tests on a variety of indicators,
            such as their ability to detect specific radiological and
            nuclear materials.

    Combating Nuclear Smuggling: Additional Actions Needed to Ensure
            Adequate Testing of Next Generation Radiation Detection
            Equipment. GAO-07-1247T. Washington, D.C.: September 18, 2007.

            [bullet]  Key findings. We found that tests conducted by DNDO
            in early 2007 were not an objective and rigorous assessment of
            the ASPs' capabilities. Specifically, we raised concerns about
            DNDO using biased test methods that enhanced the apparent
            performance of ASPs; not testing the limitations of ASPs'
            detection capabilities-for example, by not using a sufficient
            amount of the type of materials that would mask or hide
            dangerous sources and that ASPs would likely encounter at ports
            of entry; and not using a critical Customs and Border
            Protection (CBP) standard operating procedure that is
            fundamental to the performance of hand-held radiation detectors
            in the field.

            [bullet]  Recommendations. We recommended that the Secretary of
            Homeland Security delay Secretarial certification and full-
            scale production decisions on ASPs until all relevant tests and
            studies had been completed and limitations to tests and studies
            had been identified and addressed. We further recommended that
            DHS determine the need for additional testing in cooperation
            with CBP and other stakeholders and, if additional testing was
            needed, that the Secretary of DHS appoint an independent group
            within DHS to conduct objective, comprehensive, and transparent
            testing that realistically demonstrates the capabilities and
            limitations of ASPs.

    Combating Nuclear Smuggling: DHS's Program to Procure and Deploy
            Advanced Radiation Detection Portal Monitors Is Likely to
            Exceed the Department's Previous Cost Estimates. GAO-08-1108R.
            Washington, D.C.: September 22, 2008.

            [bullet]  Key findings. Our independent cost estimate suggested
            that from 2007 through 2017 the total cost of DNDO's 2006
            project execution plan (the most recent official documentation
            of the program to equip U.S. ports of entry with radiation
            detection equipment) would likely be about $3.1 billion but
            could range from $2.6 billion to $3.8 billion. In contrast, we
            found that DNDO's cost estimate of $2.1 billion was unreliable
            because it omitted major project costs, such as elements of the
            ASPs' life cycle, and relied on a flawed methodology. DNDO
            officials told us that the agency was no longer following the
            2006 project execution plan and that the scope of the agency's
            ASP deployment strategy had been reduced to only the standard
            cargo portal monitor. Our analysis of DNDO's summary
            information outlining its scaled-back plan indicated the total
            cost to deploy standard cargo portals over the period 2008
            through 2017 would be about $2 billion but could range from
            $1.7 billion to $2.3 billion. Agency officials acknowledged the
            program requirements that would have been fulfilled by the
            discontinued ASPs remained valid, including screening rail cars
            and airport cargo, but the agency had no plans for how such
            screening would be accomplished.

            [bullet]  Recommendations. We recommended that the Secretary of
            Homeland Security direct the Director of DNDO to work with CBP
            to update the projection execution plan to guide the entire
            radiation detection program at U.S. ports of entry, revise the
            estimate of the program's cost and ensure that the estimate
            considers all of the costs associated with its project
            execution plan, and communicate the revised estimate to
            Congress so that it is fully apprised of the program's scope
            and funding requirements.

    Combating Nuclear Smuggling: DHS Needs to Consider the Full Costs and
            Complete All Tests Prior to Making a Decision on Whether to
            Purchase Advanced Portal Monitors. GAO-08-1178T. Washington,
            D.C.: September 25, 2008.

            [bullet]  Key findings. In preliminary observations of the 2008
            round of ASP testing, we found that DNDO had made progress in
            addressing a number of problems we identified in previous
            rounds of ASP testing. However, the DHS criteria for
            significant increase in operational effectiveness appeared to
            set a low bar for improvement--for example, by requiring ASPs
            to perform at least as well as current-generation equipment
            when nuclear material is present in cargo but not specifying an
            actual improvement. In addition, the ASP certification schedule
            did not allow for completion of computer simulations that could
            provide useful data on ASP capabilities prior to the
            Secretary's decision on certification. Finally, we questioned
            the replacement of current-generation equipment with ASPs until
            DNDO demonstrates that any additional increase in security
            would be worth the ASPs' much higher cost.\1\
    ---------------------------------------------------------------------------
        \1\ This testimony also summarized our September 2008 report on the
    life cycle cost estimate to deploy ASPs (GAO-08-1108R).

    Combating Nuclear Smuggling: DHS's Phase 3 Test Report on Advanced
            Portal Monitors Does Not Fully Disclose the Limitations of the
    ---------------------------------------------------------------------------
            Test Results. GAO-08-979. Washington, D.C.: September 30, 2008.

              Key findings. DNDO's report on the second group of
            ASP tests in 2007 (the Phase 3 tests) did not appropriately
            state test limitations. As a result, the report did not
            accurately depict the results and could potentially be
            misleading. The purpose of the Phase 3 tests was to conduct a
            limited number of test runs in order to identify areas in which
            the ASP software needed improvement. While aspects of the Phase
            3 report addressed this purpose, the preponderance of the
            report went beyond the test's original purpose and made
            comparisons of the performance of the ASPs with one another or
            with currently deployed portal monitors. We found that it would
            not be appropriate to use the Phase 3 test report in
            determining whether the ASPs represent a significant
            improvement over currently deployed radiation equipment because
            the limited number of test runs did not support many of the
            comparisons of ASP performance made in the report.

            [bullet]  Recommendations. We recommended that the Secretary of
            DHS use the results of the Phase 3 tests solely for the
            purposes for which they were intended--to identify areas
            needing improvement--and not as a justification for certifying
            whether the ASPs warrant full-scale production. If the
            Secretary intends to consider the results of the Phase 3 tests
            in making a certification decision regarding ASPs, we further
            recommended that the Secretary direct the Director of DNDO to
            revise and clarify the Phase 3 test report to more fully
            disclose and articulate the limitations present in the Phase 3
            tests and clearly state which insights from the Phase 3 report
            are factored into any decision regarding the certification that
            ASPs demonstrate a significant increase in operational
            effectiveness. Finally, we recommended that the Secretary
            direct the Director of DNDO to take steps to ensure that any
            limitations associated with the 2008 round of testing are
            properly disclosed when the results are reported.

    Combating Nuclear Smuggling: DHS Improved Testing of Advanced Radiation
            Detection Portal Monitors, but Preliminary Results Show Limits
            of the New Technology. GAO-09-655. Washington, D.C.: May 21,
            2009.

            [bullet]  Key findings. We reported that the DHS criteria for a
            significant increase in operational effectiveness require a
            large reduction in innocent alarms but a marginal improvement
            in the detection of certain weapons--usable nuclear materials.
            In addition, the criteria do not take the current-generation
            portal monitors' full potential into account because DNDO has
            not completed efforts to improve their performance. With regard
            to ASP testing, we found that DHS increased the rigor in
            comparison with previous tests, thereby adding credibility to
            the test results, but that preliminary results were mixed. The
            results showed that the new portal monitors performed better
            than current-generation portal monitors in detection of certain
            nuclear materials concealed by light shielding approximating
            the threat guidance for setting detection thresholds, but
            differences in sensitivity were less notable when shielding was
            slightly below or above that level. Testing also uncovered
            multiple problems in ASPs meeting the requirements for
            successful integration into operations at ports of entry.
            Finally, we found that DNDO did not plan to complete computer
            simulations that could provide additional insight into ASP
            capabilities and limitations prior to certification even though
            delays to testing allowed more time to conduct the simulations.

              Recommendations. We recommended that the Secretary of
            Homeland Security direct the Director of DNDO to assess whether
            ASPs meet the criteria for a significant increase in
            operational effectiveness based on a valid comparison with
            current-generation portal monitors' full performance potential
            and revise the schedule for ASP testing and certification to
            allow sufficient time for review and analysis of results from
            the final phases of testing and completion of all tests,
            including computer simulations. If ASPs are certified, we
            further recommended that the Secretary of Homeland Security
            direct the Director of DNDO to develop an initial deployment
            plan that allows CBP to uncover and resolve any additional
            problems not identified through testing before proceeding to
            full-scale deployment.

                           Biography for Gene Aloise
        Gene Aloise is a Director in the Natural Resources and Environment
    team at GAO. He is GAO's recognized expert in international nuclear
    nonproliferation and safety issues and completed training on these
    subjects at the University of Virginia and Princeton University. His
    work for GAO has taken him to some of Russia's closed nuclear cities
    and the Chernobyl reactor in Ukraine as well as numerous nuclear
    facilities around the world and in the United States. Mr. Aloise has
    had years of experience developing, leading, and managing GAO domestic
    and international engagements. His diverse experience includes
    assignments with congressional committees as well as various offices
    within GAO. He has received numerous awards for his leadership and
    expertise including GAO's Meritorious Service Award. Mr. Aloise
    received his Bachelor's degree in political science/economics from
    Rowan University and holds a Master of Public Administration from
    Temple University. Mr. Aloise is also a graduate of the Senior
    Executive Fellows Program, John F. Kennedy School of Government,
    Harvard University.

        Chairman Miller. Thank you, Mr. Aloise.
        Dr. Lowenthal.

    STATEMENT OF DR. MICAH D. LOWENTHAL, DIRECTOR, NUCLEAR SECURITY
      AND NUCLEAR FACILITY SAFETY PROGRAM, NUCLEAR AND RADIATION
        STUDIES BOARD, NATIONAL RESEARCH COUNCIL, THE NATIONAL
                               ACADEMIES

        Dr. Lowenthal. Good morning, and thank you, Chairman
    Miller, Ranking Member Broun, Members of the Committee. My name
    is Micah Lowenthal. As you said, I am the Director of the
    Nuclear Security and Nuclear Facility Safety Program at the
    National Research Council's Nuclear and Radiation Studies
    Board. I am here to describe a recently issued interim report
    from a Congressionally mandated National Research Council study
    on Advanced Spectroscopic Portals.
        I am the study director supporting the authoring committee
    of that report, and I will begin by providing background on the
    request for this study and then I will summarize the main
    messages of the report.
        The Department of Homeland Security, or DHS, wants to
    deploy new radiation detector systems called Advanced
    Spectroscopic Portals, or ASPs, to improve scrutiny of
    containerized cargo for nuclear and radiological material. The
    ASPs are intended to replace some or all of the radiation
    portal monitors (RPMs) and hand-held radioisotope identifiers
    (RIIDs) currently used at ports and border crossings across the
    United States. Congress required that the Secretary of Homeland
    Security certify that the ASPs provide a significant increase
    in operational effectiveness over the old systems. Congress
    also directed DHS to ask the National Research Council to
    advise the Secretary on testing, analysis, costs and benefits
    of the ASPs before the certification decision.
        I want to point out that the appropriations committees who
    made these requirements said they appreciate that certification
    will be difficult. The study committee agrees with that, that
    these are hard problems, both for the testing and the cost-
    benefit analysis, so it is not surprising that DHS's work has
    been challenging.
        Here are the main findings and recommendations from the
    interim report. The study committee found that the ASP
    performance tests prior to 2008 had serious flaws. The Domestic
    Nuclear Detection Office, DNDO, acknowledged problems with the
    tests and addressed a number of those flaws in later testing.
    The 2008 performance tests were indeed improved, but
    shortcomings remain. DNDO's current approach to performance
    testing involves physically testing detector performance
    against a small number of configurations of threat objects and
    cargo in one environment. The set of possible combinations of
    threats, cargo and environments is so large and
    multidimensional that DNDO needs an analytical basis for
    understanding the performance of its detector systems, not just
    an empirical basis. Our study committee recommended that DHS
    use a more rigorous approach in which scientists use computer
    models to simulate configurations and detector performance, use
    physical tests to validate and refine the models and use the
    models to select key new physical tests that advance our
    understanding of the detector systems. This iterative modeling
    and testing approach is standard in the development of some
    high-technology equipment and is essential for building
    scientific confidence in detector performance.
        The idea of an iterative approach also extends to
    deployment. The study committee recommends a process for
    incremental deployment and continuous improvement of the ASPs
    with experience in the field leading to refinement in both the
    technologies and the operations for the next deployment. As a
    first step in this process, DHS should deploy the unused ASPs
    it already has, to assess the ASPs' performance in multiple
    environments without investing in a much larger acquisition at
    the outset.
        The ASP cost-benefit analysis was not complete when our
    interim report was written. Preliminary estimates by DNDO
    indicate that the cost increases from replacing the currently
    deployed systems with ASPs outweigh the cost reduction or
    savings from operational efficiencies. Therefore, a careful
    cost-benefit analysis will need to reveal the security
    advantages, if any, of the ASPs over the current systems and
    possible alternatives. Such a cost-benefit analysis should
    include three key elements: a clear statement of the objectives
    of the program, an assessment of meaningful alternatives; and a
    comprehensive, credible and transparent analysis of benefits
    and costs.
        DHS should consider tradeoffs among different options for
    allocating its efforts and funds, looking at the overall system
    for ways to improve defense against nuclear smuggling. The
    study committee recommended that DHS not proceed with further
    procurement of ASPs until it has addressed the findings and
    recommendations from the report, and the ASP has been shown to
    be a favored option in the cost-benefit analysis.
        Now, those are the main messages from the report. For this
    hearing I was also asked to comment on any lessons learned
    regarding processes by which the ASPs have been researched,
    developed and tested to date. Although the study committee only
    examined the testing of ASPs, not a broader portfolio, I think
    there are three lessons to be learned based on the study.
    First, the process of modeling and testing iteratively can be
    applied more broadly to other complex technology development
    programs. Second, incremental deployment with continuous
    improvement is a good strategy for deployments of systems that
    have not fully matured, especially if they are envisioned to
    have an ongoing mission. And third, the systems-level approach
    that is examining how to optimize choices within the overall
    system, rather than focusing narrowly on one tradeoff decision,
    is applicable to almost every use of equipment in security
    applications.
        Thank you for the opportunity to testify. I would be glad
    to elaborate my testimony in response to questions.
        [The prepared statement of Dr. Lowenthal follows:]
                    Prepared Statement of Micah D. Lowenthal
        Good morning Chairman Miller, Ranking Member Broun, and Members of
    the Committee. My name is Micah Lowenthal and I am the Director of the
    program on Nuclear Security and Nuclear Facility Safety in the National
    Research Council's Nuclear and Radiation Studies Board.\1\ I am here to
    describe the recently issued interim report from a congressionally
    mandated National Research Council study on advanced spectroscopic
    portals (ASPs). I am the study director supporting the authoring
    committee of that report.\2\ The full report is classified, but an
    abbreviated version was also produced for unrestricted public
    release.\3\ My testimony is based on the abbreviated version. I will
    begin by providing background on the request for this study. I will
    then summarize the main messages of the report and discuss some of the
    points most relevant to this hearing.
    ---------------------------------------------------------------------------
        \1\ The National Research Council is the operating arm of the
    National Academy of Sciences, the National Academy of Engineering, and
    the Institute of Medicine of the National Academies, chartered by
    Congress in 1863 to advise the government on matters of science and
    technology. The Nuclear and Radiation Studies Board is responsible for
    oversight of National Research Council studies on safety and security
    of nuclear materials and waste.
        \2\ Dr. Robert Dynes, a physicist at the University of California,
    member of the National Academy of Sciences, and former President of the
    University of California, chaired this study.
        \3\ The report is titled Evaluating Testing, Costs, and Benefits of
    Advanced Spectroscopic Portals for Screening Cargo at Ports of Entry:
    Interim Report. The abbreviated version of the report is available
    online at http://www.nap.edu/catalog.php?record-id=XXXX

    BACKGROUND ON THE REQUEST FOR THE STUDY

        Containerized cargo entering the United States at sea ports and
    land-border crossings for trucks is currently screened for radiation
    using detectors, called radiation portal monitors (RPMs), in
    conjunction with hand-held radioisotope identifiers (RIIDs). The
    Department of Homeland Security (DHS) is seeking to deploy new
    radiation detectors, called advanced spectroscopic portals (ASPs), to
    replace the current RPM and RIID combination, which has known
    deficiencies. The ASPs consist of new detector equipment and new
    software, including algorithms for isotope identification.
        Following some controversy over the testing and evaluation of the
    new ASPs, Congress required in Title IV of Division E of the
    Consolidated Appropriations Act, 2008 (Public Law 110-161) that the
    Secretary of Homeland Security submit to Congress a report certifying
    that ASPs would provide a ``significant increase in operational
    effectiveness'' over continued use of existing screening devices. This
    certification is a precondition for proceeding with full-scale
    procurement of ASPs. Congress also directed DHS to request that the
    National Academies advise the Secretary on the certification decision
    by helping to validate testing completed to date, providing support for
    future testing, assessing the costs and benefits of this technology,
    and bringing robustness and scientific rigor to the procurement
    process. Due to delays in the test and evaluation program, the
    Academies and DHS agreed that the study committee would issue an
    interim report that provides (1) the committee's evaluation of testing
    plans and execution it has seen, and (2) advice on how the Domestic
    Nuclear Detection Office (DNDO) can complete and make more rigorous its
    ASP evaluation for the Secretary and the Nation.
        This interim report is based on testing done before 2008 (referred
    to as past tests), plans for and preliminary results from performance
    tests carried out in 2008, and the agency's draft cost-benefit analysis
    as of October 2008. The committee received briefings on the performance
    test results and analysis and on the cost-benefit analysis but it did
    not receive any written reports on those topics by February 2009, when
    the interim report entered the Academies peer review process.
        I will now discuss each element of the study task below.

    SUMMARY OF THE MAIN MESSAGES OF THE REPORT

        First, I want to note that the committee focused much of its
    attention on performance testing. This is not because the other tests
    are unimportant--the portals will be of little use if they are
    incompatible with CBP's computer systems, for example--but the design,
    execution, and evaluation of these tests are comparatively routine,
    even if solutions to problems revealed by the tests are not. The
    design, execution, and evaluation of performance tests for the ASPs is
    more challenging and involves more of the science and engineering
    principles on which the committee has advice to offer.

    Past Performance Testing

        Performance tests prior to 2008 had serious flaws that were
    identified by the Government Accountability Office and the Secretary's
    ASP Independent Review Team. All truck-conveyed containers at ports and
    border crossings currently pass through primary screening, which is
    conducted with a radiation portal monitor (RPM). Containers that
    trigger an alarm are sent to secondary screening, which is conducted
    with an RPM and a RIID. The tests prior to 2008 did not adequately
    assess the capabilities of the ASP systems in primary and secondary
    screening compared with the currently deployed RPM and RIID screening
    systems, nor whether the ASP systems met performance criteria for
    procurement.
        There were serious flaws in the testing protocol. Notably, DNDO
    utilized the same radiation sources in performance testing that were
    used to set up and calibrate this testing. Device setup and any
    calibration must use separate radiation sources from those used for
    testing. Also, standard operating procedures for the use of RlIDs in
    secondary screening were not followed in the performance tests, which
    disadvantaged the RIIDs in comparisons with ASPs.

    2008 Performance Testing

        DNDO staff acknowledged several pre-2008 deficiencies and designed
    its 2008 test plan to correct them. The study committee examined the
    revised test plan, observed tests, and questioned test personnel, and
    the committee concluded that DNDO did address those problems.
        Because of the ASP configurations and the size of their detectors,
    ASPs would be expected to improve isotope identification, provide
    greater consistency and coverage in screening, , and increase speed of
    screening compared to the current RPM-RIID combination when used in
    secondary screening. Consequently, DNDO's 2008 performance tests of
    ASPs in secondary screening focused on confirming and quantifying that
    advantage for several threat objects, cargoes, and configurations.
        When used for primary screening, an ASP system must be compared to
    the existing RPM-RIID combination for primary and secondary screening.
    This is because ASPs perform an isotope identification function in
    primary screening. Isotope identification is only possible in secondary
    screening with the current RPM-RIID system. DNDO's preliminary analysis
    did account for this difference.
        The study committee found that the 2008 performance tests were an
    improvement over previous tests. They enabled DNDO to identify and
    physically test some of the performance limits of ASP systems. However,
    the committee identified several shortcomings of these tests: (1) The
    selected test configurations were too limited to assess the performance
    of ASP systems against the range of threat objects, cargoes, and
    configurations that could be encountered during cargo screening
    operations at ports without modeling to complement the physical
    experiments; (2) the sample sizes (the number of test runs of each
    case) are small and limit the confidence that can be placed in
    comparing ASP and RPM-RIID performance; and (3) in its analysis, some
    of the performance metrics are not the correct ones for comparing
    operational performance of cargo screening systems. These shortcomings
    are described in greater detail in our report. In the committee's
    judgment, DHS cannot determine whether ASPs can consistently outperform
    current RPM-RlID systems in routine practice until these shortcomings
    are addressed. Better physical measurement and characterization of the
    performance of the systems are a necessary first step but may not be
    sufficient to enable DHS to conclude that the ASPs meet the criteria it
    has defined for achieving a ``significant increase in operational
    effectiveness.''
        The committee recommends modifications to the testing procedures
    that are being used by DHS. These modifications would influence
    subsequent procurement steps, as described in the recommendations for
    the procurement process.

    Recommended Approach for Testing and Evaluation

        To make the testing and evaluation more scientifically rigorous,
    the committee recommends an iterative approach involving modeling and
    physical testing. The threat space that is, the set of possible threat
    objects, configurations, surrounding cargoes, and conditions of
    transport--is so large and multi-dimensional that DNDO needs an
    analytical basis for understanding the capabilities of ASP detectors
    for screening cargo. DNDO's current approach is to physically test
    small portions of the threat space and to use other experimental data
    to interpolate and extrapolate to other important parts of the threat
    space to test the identification algorithms in ASP systems.
        The committee recommends that DNDO use computer models of threat
    objects, radiation transport, and detector response to simulate ASP
    performance. Then DNDO can use physical experiments to validate these
    computer models, which would allow a critique of the models' fidelity
    to reality and show where model refinements were needed. Physical
    testing and model refinement would proceed iteratively until the model
    provided an acceptably accurate depiction of reality. With validated
    models, DNDO could evaluate the performance of ASP systems over a
    larger, more meaningful range of the threat space than is feasible with
    physical tests alone.
        This kind of interaction between computer models and physical tests
    is standard in the development and deployment of some high-technology
    equipment and is essential for building scientific confidence in
    technology performance. The performance tests conducted in 2008, and
    even prior to 2008, can be used to help refine and validate models. The
    committee also notes the skills required to proceed exist in the
    National Laboratories.

    Recommended Approach for the Procurement Process

        The idea of an iterative approach also extends to deployment of ASP
    systems at ports of entry. The committee noted that DHS' testing
    philosophy is oriented toward a one-time certification decision in the
    near future. However, the mandate for passive radiation screening of
    cargo at ports of entry is expected to continue indefinitely. Rather
    than focusing on a one-time decision about the deployment of ASPs, the
    committee suggests that current testing be viewed as a first step in a
    continuous process of system improvement and adaptation to changes in
    the threat environment, composition of container cargo, technological
    and analytical capabilities, and the nature of commerce at ports of
    entry. These factors have changed significantly over the last decade
    and can be expected to evolve--in both predictable and unpredictable
    ways--in the future. The committee recommends that DHS should develop a
    process for incremental deployment and continuous improvement, with
    experience leading to refinements in both technologies and operations
    over time, rather than a single product purchase to replace current
    screening technologies. The ASP deployment process should be developed
    to address and exploit changes. This would enable DNDO to adapt and
    continually update its screening systems so that they do not become
    outdated as they would after a one-time deployment.
        As the first step in this process, the committee recommends that
    DHS deploy its currently unused low-rate initial production ASPs for
    primary and secondary inspection at various sites under a program of
    extended operational testing. Such deployment, even on a limited scale,
    would provide valuable data concerning ASP operation, reliability, and
    performance, and would allow DHS to better assess ASP capabilities in
    multiple environments without investing in a much larger acquisition at
    the outset.
        The development of the hardware for radiation detection and the
    software for analyzing detector signals is separable. The current DHS
    procurement process is a competition among vendors to provide the
    combined systems, which has been useful. However, as DHS moves forward,
    the committee recommends that it match the best hardware to the best
    software (particularly the algorithms), drawing on tools developed by
    the competing companies and others, such as the national laboratories.
        The deployment of ASPs will not eliminate the need for hand-held
    detectors with spectroscopic capabilities. Because some of the
    improvement in isotope identification offered by the ASPs over the
    RIIDs is a result of software improvements, the committee recommends
    that these improvements be incorporated into hand-held detectors.
    Improved software might significantly improve RIID performance and
    expand the range of deployment options available to Customs and Border
    Protection for cargo screening.
        By separating the hardware and software elements of the system and
    engaging the broader science and engineering community, DHS would have
    increased confidence in its procurement of the best product available
    with current technology, and simultaneously could advance the state-of-
    the-art.

    Recommended Approach for Cost-Benefit Analysis

        The preliminary analysis presented to the committee suggests that
    benefits of deploying the ASPs may not be clearly greater than the
    costs. Because DNDO's preliminary estimates indicate that the cost
    increases from replacing the RPM-RIID combination with ASPs exceed the
    cost reductions from operational efficiencies, it is important to
    consider carefully the conditions under which the benefits of deploying
    ASPs justify the program costs. A cost-benefit analysis (CBA) can
    provide a structure for evaluating whether a proposed program (such as
    the ASP program) is reasonable and justified.
        The Secretary's decision on ASP certification is based, at least in
    part, on whether the ASPs meet the objectives in DHS' definition of
    ``significant increase in operational effectiveness'' (SlOE); however,
    other factors relating to the costs and benefits of the proposed ASP
    program will also need to be taken into account. DHS' definition of
    SlOE is a modest set of goals: As noted above, the increases in
    operational efficiency do not by themselves appear to outweigh the cost
    increases from replacing the RPM/RIID combination with ASPs, based on
    DNDO's preliminary estimates, and the criteria do not require a
    significantly improved ability to detect special nuclear material (an
    ingredient of a nuclear weapon) in primary screening. If the ASPs meet
    the defined goals and are able to detect the minimum quantities of
    nuclear threat material recommended by the ``DOE guidance,'' DHS still
    will not know whether the benefits of the ASPs outweigh the additional
    costs associated with them, or whether the funds are more effectively
    spent on other elements of the Global Architecture.
        A CBA can provide insight about alternative choices--for example,
    whether the benefits of a given program exceed its costs, and which
    choices are most cost-effective. To be effective, the CBA must include
    three key elements: (1) a clear statement of the objectives of the
    screening program; (2) an assessment of meaningful alternatives to
    deploying ASPs; and (3) a comprehensive, credible and transparent
    analysis of in-scope benefits and costs.
        The CBA should begin with a clear statement of what operational
    problem the ASPs are intended to address. This statement will define
    the role that the system plays in providing a layer in the defense
    against the importation of nuclear or radiological materials. The CBA
    should also include a narrative that clarifies how improving detection
    for containers at ports of entry to the United States fits into a
    larger effort to improve detection capabilities, in recognition of the
    many ways that materials could be brought into the United States
    through ports of entry that are not already screened, or across
    uncontrolled stretches of border. Furthermore, to be useful in a
    procurement decision, a CBA must address whether funds are better spent
    to replace the currently deployed equipment rather than to expand
    coverage for other material pathways that currently have no radiation
    screening. DHS should consider tradeoffs among different options for
    allocating efforts and funds, looking at the overall system for ways to
    improve defenses against nuclear smuggling. Such an analysis is needed
    in the CBA for ASP systems because it is not evident that it has been
    provided elsewhere.
        The CBA also needs to account for meaningful alternatives
    (including non-ASP systems) to reveal the scale of the benefits of ASPs
    for radiation screening and determine whether these benefits outweigh
    the additional costs for procurement and deployment. The complexity of
    the container screening task suggests that there could be many
    different options worthy of consideration. These options include
    variations on ASP deployment configurations and operational processes,
    and application of technologies beyond the current RPM-RIID and ASP
    systems such as deploying hand-held passive detectors with state-of-
    the-art software and advanced methods for detecting nuclear materials.
    Considerations should include active interrogation, improved imaging
    systems, and integration of these existing technologies. These
    alternatives need to be compared to a baseline that reflects as
    realistically as possible the screening capability that DHS currently
    has in place. This baseline should reflect the number and placement of
    current RPM and RIID detectors, sensitivity of these detectors based on
    how they are operated at each port, and performance of existing hand-
    held detectors in the manner they are used in the field. The CBA must
    indicate what capability an investment in ASPs will provide beyond the
    existing systems as they are currently deployed and operated, or beyond
    alternative radiation detection technologies that could be developed
    and deployed at ports of entry.
        In comparing alternatives, it is important that the CBA treat
    benefits and costs in a comprehensive, credible, and transparent
    manner. The benefit assessment should show how the ASP system would
    contribute to improving security with respect to prevention of the
    detonation of a nuclear device or radiological weapon in the United
    States. Because this is the primary objective of the ASP program, a CBA
    that is silent on this subject would be incomplete. Such an assessment
    is difficult and no assessment of such benefits will be definitive or
    unassailable. The cost assessment should cover all phases of the
    acquisition life cycle in a manner that is independent of contractor or
    program office biases, and it should also assess the risk of cost
    escalation associated with the estimate.
        The committee recommends that DHS not proceed with further
    procurement until it has addressed the findings and recommendations in
    this report, and then only if the ASP is shown to be a favored option
    in the CBA.

    ``Lessons Learned''

        For this hearing I was also asked to comment on ``lessons learned''
    regarding the processes by which the ASPs have been researched,
    developed, and tested. Although the study committee only examined the
    testing of ASPs, my personal view is that three lessons that could be
    learned:

        First, the process of iterative modeling and testing can be applied
    more broadly to other complex technology development programs. Modeling
    coupled to validating experiments is a necessity for some technology
    applications because of the complex conditions in which these
    technologies must operate.
        Second, incremental deployment with continuous improvement is a
    good strategy for deploying systems that have not fully matured,
    especially if they are envisioned to have an ongoing mission.
        Third, a systems-level approach--that is, examining how to optimize
    choices within the overall system rather than narrowly focusing on one
    tradeoff decision--is applicable to almost every use of equipment in
    security applications.
        This concludes my testimony to the Committee. Thank you for the
    opportunity to testify on this important topic. I would be happy to
    elaborate on any of my comments during the question and answer period.

                        Biography for Micah D. Lowenthal
        Micah Lowenthal is the Director of the Nuclear Security and Nuclear
    Facility Safety Program in the Nuclear and Radiation Studies Board at
    the National Research Council of the National Academies. At the
    Academies, he has directed domestic and international studies on
    nuclear and radiological safety and security, nuclear nonproliferation,
    nuclear fuel cycles, radioactive waste, and management of contaminated
    environments. Before joining the National Academies' staff in 2001, Dr.
    Lowenthal was a researcher and lecturer at the University of California
    at Berkeley. In 1996 he was an American Association for the Advancement
    of Science Environmental Science and Engineering Fellow. Dr. Lowenthal
    received an A.B. degree in physics and a Ph.D. degree in nuclear
    engineering, both from the U.C.-Berkeley.

        Chairman Miller. Thank you, Dr. Lowenthal. Unfortunately,
    you all probably know that our days here are somewhat
    scattered, and I need to go to the Floor. We have tried to
    arrange another Member of the Majority who is a Member of the
    Subcommittee to preside in my absence. Ms. Dahlkemper has been
    an especially conscientious Member and she may arrive between
    now and 11:00, which is when I expect to come back, and if that
    is the case, we will come out of recess and she will preside or
    any other Member of the Majority who is a Member of the
    Subcommittee.
        Mr. Broun. Mr. Chairman, I would be glad to preside if you
    would like me to.
        Chairman Miller. Well, I thank Dr. Broun for that generous
    offer but I think we will do it the way that every other
    committee and subcommittee in Congress does it. So with that,
    we will need to stand in recess. I apologize again for the
    inconvenience. Believe me, it is more inconvenient for me than
    it is for you, and we will be back in session but I appreciate
    your being here and I thank you for your indulgence.
        [Recess.]

                                   Discussion

        Chairman Miller. We are now out of recess. Floor colloquies
    are undoubtedly one of the most peculiar stylized aspects of
    Congressional service. I have heard of political campaigns that
    80 percent of all the effort is wasted, the problem is, no one
    knows what the 20 percent is that is not. I suspect that there
    is at least that ratio of wasted effort in Congressional
    service and I very much suspect Floor colloquies are in the
    wasted effort category. But it was something I had to do.
        Mr. Broun. Floor debate, isn't it?
        Chairman Miller. I will resist temptation to point out
    other aspects of Congressional service that may be wasted
    efforts. If we may now begin our rounds of questioning, Mr.
    Aloise, Dr. Lowenthal, Dr. Broun, I now recognize myself for
    five minutes.

                          Prioritizing Security Needs

        Dr. Broun raised the point that there are other priorities
    that we have to decide between, priorities even within
    security, so it is not security versus other things government
    does. In October of 2007, this committee held a hearing on an
    exercise on a scenario of a terrorist attack assuming that a
    dirty bomb was detonated in two American cities, and what we
    found was that we were probably better prepared, certainly
    better prepared for Katrina than we were for dirty-bomb
    attacks, despite the fact that if there is one punch that has
    obviously been telegraphed by terrorists, by al-Qaeda in
    particular, it is a dirty-bomb attack. We did not have the
    capacity to conduct the test to see what the level of
    contamination was, to let our people know that they can go back
    into offices, office buildings--that we might shut down,
    indefinitely, 10, 15, 20 square blocks of an American city, the
    midtown of an American city or perhaps two--that would probably
    take six years to complete the necessary environmental tests
    for an attack in one city, nor testing on individuals to see if
    they had been exposed to radiation. That might take a couple
    years. We would have to tell parents that we could not test
    their children to see if they had been exposed to radiation for
    a couple years. But, you know, if the child's hair started
    falling out, call us and maybe we can move them up in line. And
    the failing there seemed to be simply one of funding. We are
    not prepared to respond to a radiological attack, and here we
    see we are spending $2 or $3 billion to develop a technology
    that those who will use it really question whether it does much
    if any good at all, certainly not enough to justify $2 or $3
    billion. DNDO expects the Secretary of Homeland Security to
    make a decision this October, four months from now, about
    whether to certify that it is worth investing the $2 billion in
    the ASPs. Do either of you believe that there will be enough
    information available this fall for DHS to make that decision,
    and how confident are you that the costs of the ASPs justify
    the investment? Mr. Aloise.
        Mr. Aloise. Well, that is the question. In our view, until
    integration testing is done, field validation is done,
    operational testing is done, the injection studies are
    complete, an analysis of energy windowing versus the ASPs is
    done, until all the testing is done and we know whether or not
    this marginal increase in security is justified through a
    sound, independent cost-benefit analysis, it is our belief that
    we shouldn't spend any money on this program or go forward with
    it.
        Chairman Miller. And do you think there is any realistic
    chance those preconditions may be satisfied between now and
    October?
        Mr. Aloise. At this point it does not look that way.
        Chairman Miller. Dr. Lowenthal.
        Dr. Lowenthal. We agree that these are the preconditions
    for going ahead with certification, that there is a great deal
    of information that is needed. I am not really in a position to
    tell you how much time it will take them to get that
    information together. There is some information that they
    presumably have already put together, some analysis that they
    have already done that we haven't seen. We will be happy to
    look at it when they provide it. But in terms of timelines, we
    would not be inclined to guess when they will be ready.

                    Decision-making: Processes and Timelines

        Chairman Miller. Both of you appear to disagree with DHS's
    view that they will be able to obtain the data they need from
    field evaluation tests that are to begin next month. Why do you
    not believe that those tests will be good enough for DHS to
    base a decision on? Mr. Aloise.
        Mr. Aloise. Well, that is only part of, as I just said,
    what they need to base their decision on. You know, field
    validation testing has not gone well so far. They suspended it
    after two weeks because they were actually sending more--the
    ASPs were actually sending more to secondary than the PVTs so
    they had to work out a number of those problems and I am
    assuming they have. It has been a while since the initial
    testing. We just have to see how it goes, and we are going to
    be looking to CBP for a lot of the answers because they are the
    agency that has to live with this equipment, and I--we feel
    fairly confident that they are not going to go forward until
    they are satisfied that the testing has gone right at this
    point.
        Chairman Miller. Dr. Lowenthal.
        Dr. Lowenthal. The testing that they are going to do for
    field validation is going to tell them something that they need
    to know but it won't tell them everything that they need to
    know, and the committee--the study committee's recommendations
    were to enhance what they are going to be able to learn from
    these sorts of tests with modeling and simulations. The range
    of environments that they are going to be operating in is very
    limited. They are only doing field validation testing in four
    different sites, and there are a lot of other conditions, a lot
    of other kinds of cargo and so on that are worth learning
    about. Some of that can be done with simulation and some of it
    can be done with the deployment that the committee recommended.
        Chairman Miller. Thank you. My time has expired.
        Dr. Broun for five minutes.
        Mr. Broun. Thank you, Chairman.

           ASP in Context: The Global Nuclear Detection Architecture

        How would y'all--``y'all'' is a southern word. How would
    you all prioritize the upgrade from the PVT RIID system to the
    ASP system within the entire global nuclear detection
    architecture? Let us start with Dr. Lowenthal.
        Dr. Lowenthal. I should point out that the committee really
    only looked at the testing of the ASPs and talked about what
    kinds of information a decision-maker would want to have; and
    so the committee recommended that they (DHS) consider these
    other components of the global architecture--and there are gaps
    that are known and have been pointed out by others within that
    architecture--but the committee did not study that question,
    itself. What the committee did say is that if you look at this
    from the perspective of an adversary who is trying to sneak
    something into the United States, if you can--if you reinforce
    the places where you already are screening but you leave other
    avenues totally open, you may not be gaining much improvement
    in security.
        Mr. Broun. Mr. Aloise.
        Mr. Aloise. We agree. As I mentioned to Dr. Lowenthal a
    minute ago, we are spending a lot of money upgrading locks on
    the front door but the windows and the back door are wide open.
    Does that make sense to do with limited resources, so----
        Mr. Broun. I agree wholeheartedly. Are there other areas
    that would benefit greater from the same level of investment,
    Mr. Aloise?
        Mr. Aloise. Well, there are a lot of vulnerabilities that
    need to be looked at that have not been properly addressed in
    the architecture. The border crossing is really not the
    greatest threat, so I would--we believe that we should be
    looking equally hard at all the other vulnerabilities in the
    architecture because as it was just said, that is probably not
    where they are going to come through. If they have taken the
    time and trouble to get this material, it is unlikely they are
    going to stick it in a cargo container and drive it across
    through a portal monitor.
        Mr. Broun. And with open borders, they can just carry it in
    wherever they want to carry it.
        Dr. Lowenthal, particular--and let me ask you to answer
    your question in view of comments you made in your testimony
    about that we have ASPs sitting there now, if you would include
    that discussion?
        Dr. Lowenthal. Well, given the scope of our study, I
    wouldn't be comfortable to try to identify what the most
    promising area is. What the committee recommended is that DHS
    do that analysis, look at the entire system. We have some
    concerns that that hasn't been done, and--I will separate this
    into two parts. One is that the committee recommended that DHS
    look at that whole system in addition to just the tradeoff
    between the ASPs and the PVTs, because as a decision-maker, the
    people in the Administration and in Congress will want to know
    whether the money is well spent. The analysis does not appear
    to have been done elsewhere and so the committee recommends
    that it be included in this ASP decision. Now, the other
    component of that is the deployment of the ones that they
    already own. That is more for a learning exercise. This is
    something where they can learn more about the performance of
    the systems in the field and improve them as they go forward. I
    think these are directed at slightly different points.
        Mr. Broun. Thank you. My time disappeared, so I guess I am
    out of time. I am not sure where we stand. Thank you, Mr.
    Chairman.
        Chairman Miller. Well, you will have time for a second
    round because I was planning to ask at least one more question.
    I now recognize myself for a second round of questions.

                    Management of ASP Testing and Deployment

        Your criticism, for both of you, the criticism appears to
    be as much that the problems with the ASPs was not the
    technical challenges but that the whole program was just poorly
    managed. This is a recurring theme for this committee, you
    know, how could management--how could programs be managed
    better. But if you think the whole testing program has been
    troubled from the start, and that the management more than
    anything else is a problem, what lessons should we learn? How
    should this program be managed differently? How can it be
    managed differently in the future, this and other similar
    programs?
        Mr. Aloise. Well, I will take a shot at that. Back in 2006
    when we issued our first report on the deployment of PVTs, we
    learned about the ASPs and we knew then that there would only
    be a marginal improvement in security. That is why we called
    for a cost-benefit analysis before anything went further; is it
    worth the marginal increase in security you are going to get
    from the ASPs? Do that first, then go talk to the Congress and
    the decision-makers can decide whether it is worth it or not,
    and we know the story. It was poorly done, and it was based on
    assumptions and not data, and that leads us to where we are
    today. It was a rush to push technology through that was
    immature, and that has got to be the true lesson of this, is
    that without proper and scientifically rigorous testing,
    without a technology that is mature, it should not be generally
    rushed to deployment--especially when you have a system already
    there that is working.
        Chairman Miller. Right. Dr. Lowenthal.
        Dr. Lowenthal. I should point out that our study committee
    was silent on the matter of whether this was managed well or
    not, but I think that the report highlights many of the points
    that you made in your opening remarks about what should be done
    in the future. I think you hit on all of those--a well-planned-
    out testing program with criteria established in advance and an
    idea as you go in for what you are going to learn from each
    stage of this, making sure that it matches up with what the
    customer is looking for, as Mr. Aloise mentioned. All of those
    components lead to having a testing and evaluation program that
    will lead to procurement that gives the customer what he needs
    and the Nation what it needs.
        Chairman Miller. Would it be a fair summary to say that the
    problem is that we began with the conclusion and then found the
    analysis to support it, rather than beginning with the
    analysis, and then finding what conclusions would come from the
    analysis?
        Mr. Aloise. I think so, Mr. Chairman. I think we had a
    solution in search of a problem.
        Chairman Miller. There is a problem but there may have been
    another solution. Our clock is not working, so each Member
    needs to show self-restraint. I will now go to Dr. Broun for a
    second round. Ms. Dahlkemper has arrived and I think is now
    collecting herself.
        Dr. Broun for five minutes, however that might be measured.
    Five minutes based on the honor system.
        Mr. Broun. I hope Ms. Dahlkemper can collect herself. I
    trust that she can. She is very good at doing that.
        I am very interested to hear Dr. Lowenthal's comments about
    whether we can take the ASP units that are available now, put
    them in a parallel screening process out there in the field, as
    the appropriators evidently are preventing us from doing. Would
    you recommend that we do so, and is that something that you
    could put in your report to us, so that we can look at that
    possibility and maybe even make that recommendation?
        Dr. Lowenthal. Yes. Our study committee recommended that
    they take the--they procured a number of these low rate initial
    production [units] so that they can test and evaluate them.
    They have a number of them that have been sitting moth-balled.
    I understand that some of them have been cannibalized for parts
    for other ones, and we don't have an exact number of how many
    they have, but they have some number of them that they already
    own that they could deploy in the field. Our study committee's
    recommendation, explicitly in the report, is that these should
    be deployed at selected sites, and the experience gained from
    those will contribute to both their understanding of how they
    perform, but also how they might be able to modify the
    operations at the sites, and also the technology, what
    improvements they might want to put in there. So that is
    explicitly in the report.
        Mr. Broun. Mr. Aloise, any comments?
        Mr. Aloise. Just that we would agree with that. We think
    that is a smart idea.
        Mr. Broun. Thank you. I hope that the CBP people can be
    consulted because I think the end-user has not been consulted
    enough in this process, and with that, Mr. Chairman, I will
    yield back.
        Chairman Miller. Thank you. You may or may not have had
    time to yield back. You probably did.
        Ms. Dahlkemper for more or less five minutes.
        Ms. Dahlkemper. Thank you very much, and I apologize for
    being late. I had a markup in another committee, but I
    appreciate the opportunity to ask a question of both of you if
    you could just address this.

           Justifying the Cost of the Advanced Spectroscopic Portal
                                    Program

        Can either of you provide some insight into whether
    improving portal radiation detection is worth $2 billion to $3
    billion? Assuming that ASPs prove to be robust enough to use,
    and they deliver real improvement in detection in commercial
    flow, would that still be enough to justify deploying them,
    given all the other needs out there and our limited resources?
        Mr. Aloise. I have to go back to, and I know I sound like a
    broken record on this, a cost-benefit analysis that we have
    been calling for a couple years, although those factors that
    you mentioned need to be in there so that the decision-makers
    can look at and see whether the marginal improvement is worth
    it, considering all the vulnerabilities we have in the
    architecture already that we need to address.
        Ms. Dahlkemper. Dr. Lowenthal.
        Dr. Lowenthal. We agree entirely that a cost-benefit
    analysis will be needed to show that. We are not in a position
    to judge whether they are going to be worth it, partly because
    our study didn't examine the results of their tests for this
    interim report. It looked at the approach. Once they have the
    results and they can factor those into their cost-benefit
    analysis, then a judgment can be made as to whether they are
    worth it.
        Ms. Dahlkemper. Mr. Aloise, the GAO has done a lot of work
    on how agencies should do a cost-benefit analysis. Can you lay
    out some principles that should be guiding DHS as they begin
    their own cost-benefit analysis on whether to acquire ASPs or
    not?
        Mr. Aloise. Yes. In fact, we just developed this about a
    year ago on how to develop a cost-benefit analysis. It was
    based on working with 90 representatives from all the federal
    agencies including DHS and it lays out exactly what the best
    practices are. But some of the things must be based on fact,
    must be fact based, based on data, not assumptions, must have
    proper documentation all along the way, and those are the kinds
    of things that we are missing from the ones we looked at for
    DNDO.
        Ms. Dahlkemper. Anything specific to this particular----
        Mr. Aloise. In particular, test results, what do the test
    results show.
        Ms. Dahlkemper. All right. Dr. Lowenthal, do you want to
    comment on that?
        Dr. Lowenthal. If I can just add one point, we give some
    advice on how to complete the cost-benefit analysis but one of
    the things that is pointed out in our report is that no cost-
    benefit analysis at the end that deals with security matters is
    going to be totally unassailable. They are going to have to
    produce something that they can defend, but you are always
    going to be able to pick at some part of it. This is a
    difficult problem. This is not an easy issue to work through.
        Ms. Dahlkemper. Thank you. I yield back.
        Chairman Miller. Thank you, Ms. Dahlkemper. You still had
    five minutes.
        That concludes the questioning of this first panel. Again,
    we thank both Mr. Aloise and Dr. Lowenthal, and we will stand
    at ease for a second while this panel can step down and the
    next panel can come up.

                                   Panel II:

        At this time I would like to introduce our second panel.
    Dr. William Hagan is the Acting Deputy Director of the Domestic
    Nuclear Detection Office at the Department of Homeland
    Security. That is also a mouthful of a title. Mr. Todd C. Owen
    is the Acting Deputy Assistant Commissioner of the Office of
    Field Operations at the U.S. Customs and Border Protection,
    Department of Homeland Security. You each have five minutes for
    your spoken testimony. Your written testimony will be included
    in the record. When we have completed your spoken testimony, we
    will begin with questions. Each Member will have five minutes
    to question the panel. It is the practice of this subcommittee
    to receive testimony under oath. Do either of you have any
    objection to taking an oath? Both witnesses indicated that they
    do not. If not, you also have the right to be represented by
    counsel. Do either of you have counsel here? And both indicated
    they do not have counsel present. Please stand and raise your
    right hand. Do you swear to tell the truth and nothing but the
    truth?
        The record should reflect that both witnesses took the
    oath. We will begin with Dr. Hagan.

      STATEMENT OF DR. WILLIAM K. HAGAN, ACTING DEPUTY DIRECTOR,
       DOMESTIC NUCLEAR DETECTION OFFICE, DEPARTMENT OF HOMELAND
                                SECURITY

        Dr. Hagan. Good morning, Chairman Miller, Ranking Member
    Broun and distinguished Members of the Subcommittee. As Acting
    Deputy Director of the Domestic Nuclear Detection Office, DNDO,
    at the Department of Homeland Security, I am honored to be here
    with my colleague from U.S. Customs and Border Protection
    (CBP), Mr. Todd Owen. I would like to thank the Committee for
    the opportunity to share lessons learned and progress to date
    on our Advanced Spectroscopic Portal Program. I would also like
    to thank the Committee for its support of DNDO's mission to
    reduce the risk of radiological and nuclear terrorism to the
    Nation.
        Over the past three years, we have made substantial
    investments in the development of the next generation of
    radiation portal monitor known as the Advanced Spectroscopic
    Portal, or ASP. The goal of the ASP program is to advance
    passive detection for cargo inspection. ASP is expected to
    automatically discriminate threat from non-threat materials for
    unshielded to lightly shielded threats in primary inspection
    while improving identification capability in secondary
    inspection. To ensure that ASP systems achieve the necessary
    technical and operational performance, we are in the midst of
    putting them through a rigorous test campaign. ASP has already
    advanced through several rounds of performance testing, and
    will face field validation at ports of entry, as well as
    independent operational testing and evaluation conducted by the
    DHS Science and Technology Directorate's operational test
    authority. The successful completion of these tests, along with
    other analyses in consultation with the National Academy of
    Sciences, will then inform the Secretary's certification
    decisions for ASP this fall.
        In addition, during the execution of our test campaigns, we
    have cooperated with the Government Accountability Office and
    the National Academy of Sciences to provide both groups with
    information and visibility into our testing and analysis
    processes. The resulting review as issued by the GAO and the
    NAS provide a valuable external assessment of the ASP program.
        The 2008-2009 set of ongoing tests reflect a number of
    steps DNDO has taken to reform test processes. For example, one
    of our most fundamental improvements has been to standardize
    test event planning. We now utilize detailed instructions with
    a six-milestone process to ensure that test planning is done
    openly, that partner inputs are included, and that test events
    are designed to provide the data required to meet the
    objectives of the test and to help make programmatic decisions.
    Additionally, some improvements in the ASP program have been
    the result of DHS-wide enhancements to program management. The
    changes put in place within DHS further ensure that any
    eventual certification or acquisition decisions are consistent
    with DHS priorities and made with a strong acquisition
    management foundation.
        Specifically, we are implementing DHS's new management
    directive 102-01 for large acquisition programs which requires
    a complete set of analysis, testing and documentation as input
    for an acquisition decision. This set includes, for example,
    analysis of alternatives, concept of operations, operational
    requirements, mission need statements, cost-benefit analysis
    and others. We are acutely aware that we must integrate end-
    user requirements and conduct tests and evaluation campaigns.
    We recognize that the development and deployment of new systems
    can be expensive and the cost to taxpayers must be justified by
    the increased capabilities of the equipment. We feel that the
    plans and procedures now in place for the ASP program provide a
    sound foundation for future certification and acquisition
    decisions. ASP systems have been under review and evaluation
    for over three years, and further improvements will always be
    possible, but I believe that after the plan testing and
    analysis is complete and the requirements of MD 102-01 have
    been fulfilled, DHS will be in a position to make an informed
    decision.
        As a final note, I would like to emphasize that while we
    are diligently working to characterize and evaluate the
    performance of ASP to ensure that operators receive appropriate
    equipment for their mission, the ASP program is only one piece
    of the multi-layered solution we call the global nuclear
    detection architecture through which we seek to integrate
    efforts across the government into an overarching strategy to
    improve the Nation's nuclear detection capabilities.
    Accordingly, we plan to continue deployments of detection
    systems at our official ports of entry while also dedicating
    increased time and effort to a wider range of pathways. In
    addition, we have been working to strengthen the architecture
    by filling gaps, improving technologies, building necessary
    infrastructure and raising awareness about radiological and
    nuclear threats and the role of detection systems.
        I look forward to continuing to work with our partners
    within DHS, other federal departments and State and local
    agencies and the Members of this subcommittee and the Congress
    to help keep the Nation safe from radiological and nuclear
    terrorism.
        This concludes my statement. I thank you for your
    attention. I am happy to answer any questions.
        [The prepared statement of Dr. Hagan follows:]
                     Prepared Statement of William K. Hagan

    Introduction:

        Good morning Chairman Miller, Ranking Member Broun, and
    distinguished Members of the Subcommittee. As Acting Deputy Director of
    the Domestic Nuclear Detection Office (DNDO) at the Department of
    Homeland Security (DHS), I would like to thank the Committee for the
    opportunity to share lessons learned and progress to date on our
    Advanced Spectroscopic Portal (ASP) program. I would also like to thank
    the Committee for its support of DNDO's mission to reduce the risk of
    radiological and nuclear terrorism to the Nation.
        DNDO was established to improve the Nation's capability to detect
    and report attempts to import, possess, store, develop, or transport
    nuclear or radiological material for use against the Nation, and to
    further enhance this capability over time. To that end, our work is
    guided by our development of a global nuclear detection architecture
    (GNDA). DNDO has developed a time-phased, multi-layered, defense-in-
    depth GNDA that is predicated on the understanding that no single layer
    of defense can detect all radiological or and nuclear (rad/nuc)
    threats. For this reason, the GNDA provides multiple detection and
    interdiction opportunities overseas, at our borders, and within the
    United States to effectively increase the overall probability of system
    success. DNDO has worked with intra- and interagency partners to
    develop time-phased strategies and plans for improving the probability
    of detecting and interdicting nuclear threats. DNDO will continue to
    enhance the GNDA over time by developing better detection technologies,
    working with our operational partners to improve concepts of operations
    (CONOPs), enabling real-time reporting of detection events, and
    supporting effective response to real threats.
        My testimony today will include a status update and lessons learned
    in DNDO's efforts to address one aspect of the GNDA--scanning cargo
    containers at ports of entry. Specifically, I will focus on the ASP
    program--a program to improve the detectors used to perform this task.

    Role of Container Scanning and ASP in the GNDA:

        The United States border is the first layer within the GNDA where
    the United States has full control over detection and interdiction. As
    such, considerable effort and resources have been placed at this layer
    to provide comprehensive radiological and nuclear detection
    capabilities, particularly at ports of entry (POEs). After 9/11,
    considerable concern was raised about the possibility that terrorists
    could use the enormous volume of cargo flowing into the United States
    as a pathway for bringing in nuclear material or a nuclear weapon. By
    far, the largest mode for incoming cargo is maritime shipping
    containers, with approximately 11 million containers coming into the
    country every year. Additionally, in the Security and Accountability
    for Every (SAFE) Port Act of 2006, Congress mandated that all
    containers coming in through the top 22 ports, by volume, be scanned
    for radiation by the end of 2007.
        A key consideration in rad/nuc detection is the ability to
    effectively detect threats without impeding the flow of legitimate
    trade and travel across the border. United States Customs and Border
    Protection (CBP) currently scans cargo entering at our nation's POEs
    using polyvinyl toluene (PVT)-based radiation portal monitors (RPMs)
    that can detect radiation, but cannot distinguish between threat
    materials and naturally occurring radioactive material (NORM), such as
    kitty litter and ceramic tiles. Narrowing down alarms to just those for
    dangerous materials is especially important for POEs that have a high
    volume of containers, or those that see a high rate of NORM. To address
    this limitation, DNDO is developing next-generation technology--the ASP
    program. The ASPs have shown significantly improved capability to
    distinguish rad/nuc threats from non-threats over the hand-held
    instruments currently used in secondary screening. Thus, the
    introduction of ASP systems is expected to not only reduce the number
    of unnecessary referrals and false positives in primary but increase
    the probability of detecting dangerous materials in secondary.
        As you know, DNDO initiated the ASP program in 2006, building on
    previous work within CBP and the Science and Technology Directorate.
    ASP systems are the next generation of radiation portal monitors. ASP
    units are now being developed by two separate vendors. These units have
    been subjected to rigorous tests and both systems will complete several
    rounds of performance testing and field validation at POEs. Following
    these performance tests, both systems will complete independent
    operational testing and evaluation conducted by the DHS Science and
    Technology Directorate's Operational Testing Authority. Test data will
    be analyzed and provided in support of the Secretary's Certification
    decision. DNDO is also engaged with the National Academy of Sciences
    (NAS) to allow them to review ASP testing and inform the certification
    process, as required in the FY 2008 and 2009 Homeland Security
    Appropriations bills. Indeed, in its most recent report on ASP testing,
    the Government Accountability Office (GAO) acknowledged the many
    enhancements and lessons that DNDO has incorporated into its testing
    programs.

    Reviews to date and lessons learned:

        Since 2006, the ASP program has undergone extensive review from
    outside agencies, including the GAO, an Independent Review Team (IRT)
    established by the previous DHS Secretary, and, most recently, the
    National Academy of Sciences. We have taken each review seriously,
    valued the recommendations that have been provided, and, where we felt
    appropriate, we have incorporated their recommendations into the next
    stages of the program.
        The first reviews conducted by the GAO of the ASP program focused
    on testing conducted in 2005 as part of the original ASP vendor
    selection and the initial cost-benefit analysis used to evaluate
    potential deployment options for ASP systems. In its report, released
    in September 2006, GAO questioned the methods used by DNDO to quantify
    performance capabilities of new systems, and insisted that ASP
    performance be evaluated against system requirements prior to full
    scale deployment. DNDO concurred with the need for additional testing
    prior to full scale deployments, which were underway prior to the
    release of GAO's report. These tests were conducted throughout 2007 and
    focused specifically on evaluating the performance of ASP systems in a
    number of testing environments.
        In September 2007, GAO recommended that DHS establish an
    independent body to conduct additional testing of ASP systems, which
    DNDO agreed to, launching the ASP-Independent Review Team (IRT), a team
    of independent experts, drawn from a wide range of institutions and
    backgrounds. The ASP-IRT Report, delivered in February 2008, provides a
    valuable independent assessment of the ASP program, and served as an
    important source of information, albeit based on the data that was
    available at the time.
        In addition to the reviews of 2007 testing, at the conclusion of
    initial ``field validation'' testing in 2007, CBP identified a number
    of functional improvements, unrelated to detector performance, that
    required modifications to ASP systems prior to deployment. DNDO
    postponed efforts to seek certification at that time, initiated new
    efforts to develop these requested changes , and conducted a new series
    of tests in 2008 and early 2009 to ensure that these changes did not
    detract from detector performance.
        The 2008-2009 test campaign transitions the program from
    developmental to functionality and performance testing, culminating in
    full operational tests. This testing includes:

            1.  System Qualification Testing, designed to demonstrate that
            ASP units are manufactured in accordance with processes and
            controls that meet the specified design requirements;

            2.  Performance Testing at the Nevada Test Site (NTS), designed
            to evaluate ASP, PVT, and radioisotope identification devices
            (RIID) detection and identification performance against
            controlled, realistic threat materials, shielding and masking
            scenarios;

            3.  Integration Testing, designed to determine whether the ASP
            systems are capable of operating and interfacing with the other
            equipment found in operational settings;

            4.  Field Validation Testing, designed to exercise the ASP in a
            stream of commerce environment at POEs;

            5.  Operational Test and Evaluation (OT&E), designed to measure
            the operational effectiveness and suitability of ASP. The OT&E
            will be independently conducted by the DHS Science and
            Technology Directorate (S&T).

        Recently, the GAO released its latest report, focusing on the
    testing conducted in 2008 and 2009. GAO acknowledged improvements in
    ASP testing, but raised some concerns. DNDO agrees that analysis and
    review of test data is necessary. DNDO plans to continue study the
    results of testing as the ASP program progresses. However, DNDO
    believes that the data analysis performed to date and the anticipated
    data from ongoing testing will be sufficient to inform an ASP
    certification decision in the future.
        In addition to the GAO's reviews, DNDO and CBP have provided NAS
    with regular testing updates. Recently, the NAS delivered an interim
    report to the Department and the Appropriations Committees. Like the
    NAS, DNDO sees the intrinsic value of continued testing and incremental
    deployment of ASP systems. However, DNDO must balance the need to
    better understand a complex system like ASP and the need to further
    reduce the risk of certain significant vulnerabilities and operational
    burdens. DNDO believes that the criteria outlined for certification are
    a sufficient threshold for determining when we have reached the point
    where deployment should begin.
        Ultimately, these reviews provide for a valuable external
    assessment of the ASP program. It is only through initial deployments
    that we will continue to learn more about the performance of these
    systems, and most rapidly bring about the improvements that are needed
    to address current limitations. ASP systems have been under review and
    evaluation for over three years now, and, while further improvements
    will always be possible, we should not delay the implementation of
    substantially improved capabilities.

    Acquisition lessons learned and changes made in response:

        DNDO has taken a number of steps to reform processes to ensure the
    success of ASP, as well as other development and acquisition programs.
    At the same time, these reforms are accompanied by a number of similar
    improvements to DHS-wide program management processes.
        With regard to testing, DNDO has taken a number of steps to improve
    internal procedures based on lessons learned from earlier tests.
        One of our most fundamental improvements has been through the
    standardized implementation of DNDO Operating Instruction 1, ``Test
    Event Planning.'' This detailed instruction lays out a six-milestone
    process that ensures that test planning is done openly, that partner
    inputs are included, and that test events are designed to provide the
    data required to meet the objectives of the test, and ultimately to
    help make programmatic decisions.
        We have also taken considerable steps to ensure that any ASP
    testing is responsive not just to DNDO requirements and objectives, but
    to all DHS partners. Prior to the 2008-2009 testing, DHS created a test
    planning working group that included DNDO, CBP, the DHS Operational
    Testing Authority (OTA), the Office of the Under Secretary for
    Management (USM), and the National Laboratories. Collectively, this
    group laid out the test campaign and assigned responsibilities for each
    test event to respective components.
        Finally, in the 2008-2009 ASP test campaign, based on lessons
    learned in the 2007 test campaign series, we instituted strict entrance
    and exit criteria for each of the test events. These criteria were
    developed long before testing began, and were developed jointly with
    our operational partner, CBP. This has given us confidence that as the
    ASP systems have continued through this series of test events,
    requirements are met prior to completion.
        DNDO has also made a number of program management changes based on
    lessons learned throughout the program. In late 2008, DHS decided
    against exercising the next contract option for one of three ASP
    vendors. This decision reduced costs for carrying forward multiple,
    parallel development efforts.
        Finally, DNDO and the ASP program have benefited from a number of
    DHS-wide improvements to program management. DNDO has adopted
    Management Directive (MD) 102-01, which outlines the acquisition
    management process for DHS programs. ASP program plans have been
    adapted to be consistent with the rigorous process outlined in the
    directive. This will further ensure that any eventual certification and
    acquisition decisions are consistent with DHS priorities and made with
    a strong acquisition management foundation.
        There is an important distinction between two key milestones for
    the ASP program--Secretarial certification and the MD 102-01 milestone
    decision for purchase and deployment of ASP. The former is a rather
    loosely defined milestone, which we have clarified by defining a
    ``significant increase in operational effectiveness,'' requiring that
    the Secretary formally state that she believes the ASP system is
    ``better than'' the current system. The latter is a very well defined
    milestone that was developed for all large programs within the
    Department of Homeland Security. Items such as mission needs,
    operational requirements, analysis of alternatives, etc., are part of
    the MD 102-01 process. No ASP production units can be purchased and
    deployed without successfully navigating the MD 102-01 process. The
    Secretarial certification requirement is in addition to and in advance
    of the MD 102-01 deployment decision.
        Together, the lessons learned in testing and program management,
    along with the introduction and adherence to MD 102-01 have
    significantly improved the ASP program. At the same time, it is
    important to note that these lessons learned have not only benefited
    the ASP program; they are being applied across DHS.

    ASP Path Forward:

        The plans and procedures in place for the ASP program provide a
    sound path forward for ASP certification and future acquisition
    decisions. The current path to certification includes testing,
    accompanied by the analysis of results, to ensure that Secretary
    Napolitano has sufficient information for ASP certification. ASP
    systems have been under review and evaluation for over three years now,
    and, while further improvements will always be possible, a
    certification decision will determine whether or not the systems
    address increase the probability of detecting dangerous materials while
    minimizing the operational burdens.
        ASP was designed to improve capabilities in both primary and
    secondary inspections. For primary applications, we have defined a
    ``significant increase in operational effectiveness'' as being a
    quantified reduction in unwanted referrals to secondary inspection,
    while maintaining similar or better sensitivity in detecting materials
    of concern. Ultimately, the degree to which ASP systems meet this
    objective is driven by sensitivity thresholds at which the systems will
    be operated, similar to the threshold used for current PVT systems.
    Therefore, the advantage in primary can be viewed as an improvement in
    efficiency of operations (less unwanted referrals), while maintaining
    the same or better detection efficiency. The relative degree to which
    we realize these benefits with respect to the current systems will vary
    by port, depending on the operational thresholds at which the current
    PVT systems are set.
        The evaluation of ASP systems in secondary inspection is more
    direct. The ability of ASP systems to identify and resolve the source
    of radiation is directly compared to the ability of current
    capabilities to perform the same functions. In this instance, a
    ``significant increase in operational effectiveness'' is defined as a
    quantified improvement in the ability to identify materials of concern.
        Both of these definitions were developed through coordination
    between DNDO, CBP, and USM. This definition has been approved by DHS
    senior leaders and has served as the guide for developing test
    campaigns to meet these test objectives. Again, because of the rigor
    that has gone in to developing and quantifying these improvements, we
    are seeking to remove ambiguity in the evaluation process, and ensure
    that any certification and acquisition decisions are made consistent
    with DHS priorities and objectives.

    Conclusion:

        DNDO will continue to work with CBP and other partners within and
    beyond DHS to improve the Nation's ability to detect radiological and
    nuclear threats at our ports and borders. DHS is facing a challenge at
    our ports and borders as the Department balances facilitating the flow
    of goods and commerce with the need to sufficiently scan cargo for
    radiological or nuclear threats as it enters our nation. As both the
    President and Secretary have said, the Nation will need more technology
    to meet its security challenges and the technologies that DNDO is
    pursuing, of which ASP is but one example, are a critical component in
    addressing that challenge.
        Our efforts to develop and evaluate ASP systems are sound. Current
    test results are capturing the benefits of ASP systems, and the reviews
    to date have provided a valuable assessment of the program and
    identified a number of key lessons learned.
        I welcome and appreciate the Committee's active engagement with
    this program, and look forward to continuing our cooperation as we move
    forward together. Chairman Miller, Ranking Member Broun, and Members of
    the Subcommittee, I thank you for your attention and will be happy to
    answer any questions that you may have.

                         Biography for William K. Hagan
        Dr. William Hagan serves as Acting Deputy Director of the
    Department of Homeland Security's (DHS) Domestic Nuclear Detection
    Office (DNDO). Prior to that, he was Assistant Director for the
    Transformational Research and Development (R&D) Directorate at DNDO. In
    that role Dr. Hagan was responsible for long-term R&D seeking
    technologies that can make a significant or dramatic positive impact on
    the performance, cost, or operational burden of detection components
    and systems.
        Prior to DNDO, he was a Senior Vice President at Science
    Applications International Corporation (SAIC). Business areas included
    nuclear technology (analysis, detection, and applications),
    telecommunications, optics, transportation, system integration, and
    technology assessments during his thirty years at SAIC.
        Dr. Hagan earned a Bachelor of Science in Engineering Physics in
    1974, Master of Science in Physics in 1975, and Master of Science in
    Nuclear Engineering in 1977 from the University of Illinois at Urbana.
    He received his Ph.D. in Physics from the University of California-San
    Diego in 1986. He holds three patents. Dr. Hagan was appointed to
    Senior Executive Service in 2006.

        Chairman Miller. Thank you, Dr. Hagan.
        Mr. Owen for five minutes.

        STATEMENT OF MR. TODD C. OWEN, ACTING DEPUTY ASSISTANT
      COMMISSIONER, OFFICE OF FIELD OPERATIONS, U.S. CUSTOMS AND
           BORDER PROTECTION, DEPARTMENT OF HOMELAND SECURITY

        Mr. Owen. Good morning, Chairman Miller, Ranking Member
    Broun, distinguished Members of the Subcommittee. As the Acting
    Deputy Assistant Commissioner for the Office of Field
    Operations with U.S. Customs and Border Protection, I am
    honored to be here this morning alongside Dr. Hagan to discuss
    the detection of radioactive and nuclear material in cargo
    containers and the future role that Advanced Spectroscopic
    Portal technology will have on CBP operations.
        I would also like to express my gratitude to the Congress
    for its continued support of CBP initiatives. Among the
    numerous priorities that were recognized in the American
    Recovery and Reinvestment Act, Congress provided CBP with $100
    million worth of stimulus funding towards non-intrusive
    inspection equipment. This funding will allow CBP to upgrade
    and expand its successful Non-Intrusive Inspection (NII)
    program and more efficiently inspect containers and vehicles
    crossing the border, allowing them to enter our country and its
    commerce in a safe and prompt manner.
        CBP has made tremendous progress in ensuring that supply
    chains importing goods into the United States are more secure
    against potential exploitation by terrorist groups aiming to
    deliver weapons of mass effect. CBP uses a multi-layered
    approach to ensure the integrity of supply chains from point of
    stuffing through the arrival in the U.S. ports of entry. This
    multi-layered defense is built upon interrelated initiatives,
    which include the 24-hour rule in the Trade Act of 2002, the
    automated targeting system, the use of Non-Intrusive Inspection
    equipment and radiation portal monitors, our container security
    initiative and the Customs Trade Partnership Against Terrorism
    program. These complementary layers enhance security and
    protect our nation.
        Prior to 9/11, not a single radiation portal monitor and
    only 64 large-scale non-intrusive inspection systems were
    deployed to our nation's borders. By October of 2002, CBP had
    deployed the first RPM at the Ambassador Bridge in Detroit.
    Today CBP has well over 1,200 RPMs and 227 large-scale NII
    technology systems deployed nationwide. NII technology allows
    the officers to detect possible anomalies, anomalies which may
    indicate the presence of a weapon of mass effect or some other
    contraband, and to date in fiscal year 2009, CBP NII systems
    have conducted over 3.1 million exams resulting in over 7,800
    narcotic seizures with a total weight of over 2.6 million
    pounds as well as $6.2 million in currency seized.
        In addition to the significant strides made in the area of
    NII equipment, CBP also continues to deploy first-generation
    radiation portal monitors to our ports of entry. Currently, 97
    percent of the trucks and 93 percent of the personally owned
    vehicles arriving from Canada, 100 percent of the trucks and
    vehicles from Mexico and 98 percent of the arriving sea
    containers are scanned by our current radiation detection
    technologies. In total, CBP scans 98 percent of all cargo
    arriving into the United States by land and sea using radiation
    portal monitors. In addition, CBP officers scan 100 percent of
    general aviation aircraft arriving to the United States from
    foreign destinations using hand-held radiation identification
    devices.
        Since the first RPM was deployed in 2002, CBP officers have
    scanned over 368 million conveyances for the presence of
    radiation, and we have resolved 2.1 million radiological alarms
    successfully with minimal or no impact to the flow of
    legitimate trade and travel. CBP continues to closely
    coordinate with key stakeholders to ensure the impact of this
    activity causes minimal disruption to port operations. The
    first-generation RPM systems, although very sensitive, do have
    limitations. While they alert CBP officers to the presence of
    radiation, a secondary exam is necessary to positively identify
    the specific isotope causing the alert. In the event that a CBP
    officer is unable to positively resolve the alert, scientific
    reach-back is available 24 hours a day, seven days a week.
        The ASP is expected to enhance our detection capability
    while significantly reducing the number of secondary
    examinations. This is due to its ability to distinguish between
    actual threats and natural or medical radiation sources that
    are not security threats. CBP has worked closely with the DNDO
    in the development and operational testing of ASP, has provided
    DNDO with functional requirements and has been actively engaged
    in every step of the evaluation process. CBP's focus for
    operational testing is to evaluate the effectiveness for
    systems deployed in our operational environments. We will
    continue to work with DNDO and the DHS Science and Technology
    Directorate towards secretarial certification, and we are also
    working within DHS to ensure that any future ASP acquisitions
    and deployment decisions are consistent with DHS priorities.
    The decision to purchase and deploy ASPs in the operational
    arena will be based on CBP's mission needs, operational
    requirements, comprehensive cost-benefit analysis to include
    the full understanding of maintenance and operation costs, and
    analysis alternatives and other considerations.
        Mr. Chairman, Members of the Subcommittee, today I have
    addressed CBP's commitment to invest in new technologies and
    emerging technology aimed at enhancing cargo security. We must
    continue to maintain our tactical edge by integrating new
    technology into our ports of entry.
        Thank you again for the opportunity to be here this morning
    and I would be happy to answer any questions.
        [The prepared statement of Mr. Owen follows:]
                       Prepared Statement of Todd C. Owen
        Chairman Miller, Ranking Member Broun, esteemed Members of the
    Subcommittee, it is a privilege and an honor to appear before you today
    to discuss the work of U.S. Customs and Border Protection (CBP),
    particularly the detection of radioactive and nuclear material in cargo
    containers and the future role that the Advanced Spectroscopic Portal
    (ASP) program will have on our operations. CBP strives to continually
    improve the security of cargo entering our borders and facilitate the
    flow of legitimate trade and travel. Included in this process, over 98
    percent of all arriving maritime containerized cargo is presently
    scanned for radiation through radiation portal monitors.
        I want to begin by expressing my continuing gratitude to Congress
    for its continued support for the mission and people of CBP. It is
    clear that the Congress is committed to providing CBP the resources we
    need in order to increase and maintain the security of our borders. We
    appreciate your efforts and assistance.
        CBP is the largest uniformed, federal law enforcement agency in the
    country. We station over 20,000 CBP officers at access points around
    the Nation, including at air, land, and sea ports. As of mid-May, we
    have deployed over 19,000 Border Patrol agents between the ports of
    entry. These forces are supplemented with 1,058 Air and Marine agents,
    2,318 agricultural specialists, and other professionals. These
    personnel are key players in the implementation of Secretary
    Napolitano's Southwest Border Security Initiative.
        CBP continues to execute all of its responsibilities, which include
    stemming the illegal flow of drugs, contraband and people, protecting
    our agricultural and economic interests from harmful pests and
    diseases, protecting American businesses from theft of their
    intellectual property, enforcing textile agreements, tracking import
    safety violations, regulating and facilitating international trade,
    collecting import duties, facilitating legitimate travel, and enforcing
    United States trade laws. CBP facilitates lawful immigration, welcoming
    visitors and new immigrants, while making certain those entering this
    country are indeed admissible and taking appropriate action when an
    individual fears being persecuted or tortured if returned to their home
    country. At the same time, our employees maintain a vigilant watch for
    terrorist threats. In FY 2008, CBP processed more than 396 million
    pedestrians and passengers, 122 million conveyances, 29 million trade
    entries, examined 5.6 million sea, rail, and truck containers,
    performed over 25 million agriculture inspections, apprehended over 720
    thousand illegal aliens between our ports of entry, encountered over
    220 thousand inadmissible aliens at the ports of entry, and seized more
    than 2.8 million pounds of illegal drugs.
        We must perform our important security and trade enforcement work
    without stifling the flow of legitimate trade and travel that is so
    important to our nation's economy. These are our twin goals: border
    security and facilitation of legitimate trade and travel.

    CBP OVERVIEW

        I am pleased to appear before the Subcommittee today to highlight
    key accomplishments related to container security, particularly those
    related to new and emerging technology. CBP has made tremendous
    progress in securing the supply chains bringing goods into the United
    States from around the world to prevent their potential use by
    terrorist groups that seek to deliver weapons of mass effect. The use
    of cutting-edge technology has greatly increased the ability of front
    line CBP Officers to successfully detect and interdict illicit
    importations of nuclear and radiological materials. CBP uses a multi-
    layered approach to ensure the integrity of the supply chain from the
    point of stuffing through arrival at a U.S. port of entry. This multi-
    layered approach includes:

            [bullet]  Advanced Information

                      24-Hour Rule

                      Automated Targeting Systems

                      Importer Security Filing

            [bullet]  The Customs Trade Partnership Against Terrorism

            [bullet]  The Container Security Initiative

            [bullet]  The Secure Freight Initiative

            [bullet]  Use of Non-Intrusive Inspection Technology and
            Mandatory Exams for All High-Risk Shipments

        I will discuss each one of these layers in greater detail with
    particular focus on our radiation and nuclear detection capabilities.

    ADVANCE INFORMATION

        CBP requires advanced electronic cargo information as mandated in
    the Trade Act of 2002 (including the 24-hour rule for maritime cargo).
    Advanced cargo information on all inbound shipments for all modes of
    transportation is effectively evaluated using the Automated Targeting
    System (ATS) before arrival in the United States.
        ATS provides decision support functionality for CBP officers
    working in Advanced Targeting Units (ATUs) at our ports of entry and
    CSI ports. The system provides uniform review of cargo shipments for
    identification of the highest threat shipments, and presents data in a
    comprehensive, flexible format to address specific intelligence threats
    and trends. ATS uses a rules-based program to highlight potential risk,
    patterns, and targets. Through rules, the ATS alerts the user to data
    that meets or exceeds certain pre-defined criteria. National targeting
    rule sets have been implemented in ATS to provide threshold targeting
    for national security risks for all modes: sea, truck, rail, and air.
        The Importer Security Filing interim final rule, also known as ``10
    + 2,'' went into effect earlier this year and has already yielded some
    promising results. This program will provide CBP timely information
    about cargo shipments that will enhance our ability to detect and
    interdict high risk shipments. Comments on aspects of this rule were
    accepted until June 1, 2009, and implementation using informed
    compliance will continue until January of next year. Shipments
    determined by CBP to be high-risk are examined either overseas as part
    of our Container Security Initiative or upon arrival at a U.S. port.

    Customs Trade Partnership Against Terrorism

        The Customs Trade Partnership Against Terrorism (C-TPAT) is an
    integral part of the CBP multi-layered strategy, in that CBP works in
    partnership with the trade community to better secure goods moving
    through the international supply chain. C-TPAT has enabled CBP to
    leverage supply chain security throughout international locations where
    CBP has no regulatory reach. In 2009, CBP will continue to expand and
    strengthen the C-TPAT program and ensure that certified member
    companies are fulfilling their commitment to the program by securing
    their goods moving across the international supply chain to the United
    States. To carry-out this critical tenet of C-TPAT in 2009, teams of
    Supply Chain Security Specialists (SCSS) will conduct validations and
    revalidations of C-TPAT members' supply chains to ensure security
    protocols are reliable, accurate, and effective.
        As C-TPAT has evolved, we have steadily added to the rigor of the
    program. CBP has strengthened the C-TPAT program by clearly defining
    the minimum-security requirements for all categories of participants
    wishing to participate in the program and thereby gain trade
    facilitation benefits. As of June 18, 2009, there are 9,286 companies
    certified into the C-TPAT program. CBP's goal is to validate all
    partners within one year of certification, revalidate all companies not
    less than once every three years and revalidate all U.S./Mexico highway
    carriers on an annual basis, based on the risk associated with the
    Southern Border Highway Carrier sector of C-TPAT.

    Container Security Initiative

        To prevent terrorists and their weapons from entering the United
    States, CBP has also partnered with other countries through our
    Container Security Initiative (CSI). In FY 2008 CBP Officers stationed
    at CSI ports reviewed over 11 million bills of lading and conducted
    over 74,000 exams in conjunctions with their host country counterparts.
    Because of the sheer volume of sea container traffic, containerized
    shipping is uniquely vulnerable to terrorist exploitation. Under CSI,
    which is the first program of its kind, we are partnering with foreign
    governments to identify and inspect high-risk cargo containers at
    foreign ports before they are shipped to our seaports and pose a threat
    to the United States and to global trade.
        CBP Officers stationed at foreign CSI ports review 100 percent of
    the manifests originating and/or transiting those foreign ports for
    containers that are destined for the United States. In locations where
    the tremendous volume of bills prevents the CSI team at the port itself
    from performing 100 percent review, or during port shutdowns, CSI
    targeters at the National Targeting Center provide additional support
    to ensure that 100 percent review is accomplished. Utilizing the
    overseas CSI team and the CSI targeters at our National Targeting
    Center, CBP is able to achieve 100 percent manifest review for the CSI
    program.
        Today, CSI is operational in 58 ports covering 86 percent of the
    maritime containerized cargo shipped to the United States.

    Secure Freight Initiative

        The Secure Freight Initiative (SFI) is an unprecedented effort to
    build upon existing port security measures by enhancing the United
    States Government's ability to scan containers for nuclear and
    radiological materials in seaports worldwide and to better assess the
    risk of inbound containers. Secure Freight will provide carriers of
    maritime containerized cargo with greater confidence in the security of
    the shipment they are transporting, and it will increase the likelihood
    of an uninterrupted and secure flow of commerce. This initiative is the
    culmination of our work with other government agencies, foreign
    governments, the trade community, and vendors of leading edge
    technology.
        Moving forward, CBP will prioritize future deployments of scanning
    systems to locations of strategic importance by identifying seaports
    where non-intrusive imaging and radiation detection data would be most
    practical and effective in deterring the movement of weapons of mass
    destruction via containerized cargo. Under this strategy, the
    additional scan data provided by SFI will enhance DHS' risk-based and
    layered approach to securing maritime containerized cargo. We will
    continue to work with Congress to enhance the safety of our nation's
    ports and the security of incoming cargo.

    Non-Intrusive Inspection/Radiation Detection Technology

        Today I will specifically address large-scale X-ray and gamma
    imaging systems and radiation detection devices; technologies that play
    a critical role in our layered enforcement strategy.
        The deployment of imaging systems and radiation detection equipment
    has contributed to CBP's tremendous progress in ensuring that supply
    chains bringing goods into the United States from around the world are
    secure against exploitation by terrorist groups that seek to deliver
    weapons of mass effect.
        Non-Intrusive Inspection (NII) technology serves as a force
    multiplier that allows officers to detect possible anomalies between
    the contents of the container and the manifest. CBP relies heavily on
    the use of NII as it allows us to work smarter and more efficiently in
    recognizing potential threats.
        Prior to 9/11, not a single Radiation Portal Monitor (RPM), and
    only 64 large-scale NII systems, were deployed to our nation's borders.
    By October of 2002, CBP had deployed the first RPM at the Ambassador
    Bridge in Detroit. Today, CBP has deployed 1,250 operational RPMs at
    seaports, land border ports, and mail facilities, 227 large-scale gamma
    ray or x-ray imaging systems and 3,000 small scale NII systems
    nationwide. Additionally, CBP has deployed over 1,382 Radiation Isotope
    Identifier Devices (RIID) and over 17,542 Personal Radiation Detectors
    (PRD). These devices allow CBP to inspect 100 percent of all identified
    high-risk cargo.
        Currently, 97 percent of trucks and 93 percent of personally owned
    vehicles arriving through northern border ports, 100 percent of
    vehicles arriving through southern border ports, and 98 percent of
    arriving sea containers are scanned by our radiation detection
    technologies. CBP uses RPMs to scan 98 percent of all cargo arriving in
    the U.S. by land and sea. In addition, CBP officers now use hand-held
    radiation identification devices to scan 100 percent of private
    aircraft arriving in the U.S. from foreign destinations. As of May
    2009, CBP officers scanned over 368 million conveyances and
    successfully adjudicated 2.1 million radiological alarms.
        It is important to distinguish these deployments from the 100
    percent mandate. These deployments refer to CBP's domestic RPM
    deployments, which perform radiation detection (not imaging) of
    containers that are scanned in the U.S. but prior to release into the
    commerce. The 100 percent mandate requires scanning in a foreign port
    and both imaging and radiation detection.
        The first generation RPM systems, although very sensitive, do have
    limitations. While they alert CBP officers to the presence of
    radiation, a secondary exam is necessary to positively identify the
    location and specific isotope causing the alert. In the event that a
    CBP officer is unable to positively resolve the alert, scientific reach
    back is available on a 24/7 basis through the National Targeting Center
    and CBP's Laboratory & Scientific Services Division located in the
    northern Virginia area.
        Understanding these limitations and the need for more precise
    radiological detection architecture, the DNDO was chartered to develop
    new technologies that will improve CBP's radiation and nuclear
    detection capabilities. One of these new technologies is the next
    generation RPM, or the Advanced Spectroscopic Portal (ASP).
        The ASP is able to distinguish between actual threats and natural
    or medical radiation sources that are not security threats. In doing
    so, the ASP is expected to enhance our detection capability, while
    significantly reducing the burden of responding to the numerous benign,
    nuisance alarms that are mostly generated by everyday products. This
    will allow CBP to focus our staffing and resources on high-risk
    shipments and other border security initiatives.

    CBP COORDINATION WITH THE DOMESTIC NUCLEAR DETECTION OFFICE (DNDO)

        In the course of our collaboration with DNDO, CBP brings knowledge
    of how our ports work, of the support needs of our front-line officers,
    and of the operational requirements for new technologies that must work
    consistently in a broad array of environments. Additionally, we must
    remain attuned to critical factors such as throughput and capacity as
    we seek to maintain an appropriate balance between security and the
    facilitation of cross-border travel and trade.
        CBP has worked closely with DNDO in the developmental and
    operational testing of the ASP. A complete independent operational
    testing and evaluation will be conducted by the DHS S&T Director, T&E
    and Standards Director, Operational T&E, when the system is ready.
    CBP's objective for operational testing is ensuring that systems are
    operationally acceptable and effective and can be deployed in our
    operational environments. Specifically, CBP provided DNDO with
    functional requirements for the ASP and has been actively engaged in
    every step of testing, including performance testing at the Nevada Test
    Site and Integration testing currently ongoing at a mock port of entry
    at the Pacific Northwest National Laboratory.
        During integration testing, CBP works closely with DNDO to assess
    each system's performance as an integrated unit, including reach back
    capability and ancillary equipment such as traffic lights and automated
    gate arms that are essential to maintain positive control of vehicles
    at our congested ports of entry. In addition, CBP works with DNDO to
    assess and categorize each system's defects to ascertain their
    technological impact on performance and their operational impact on
    front-line CBP officers--the users of the system.
        CBP will continue to work with DNDO towards Certification by the
    Secretary, which is dependent on demonstrating a ``significant increase
    in operational effectiveness'' over existing first generation radiation
    detection systems. Only after this Certification has been reach can the
    discussion then turn to potential acquisition and deployments of the
    ASP systems. The decisions to purchase and deploy ASPs in the
    operational arena will be based on mission needs, operational
    requirements, and a full understanding of maintenance and operational
    costs, to include a comprehensive cost benefit analysis, an analysis of
    alternatives, etc.

    CONCLUSION

        In conclusion, I would like to say that technology plays an
    enormous role in securing the supply chain. Security technology is
    continuously evolving, not only in terms of capability but also in
    terms of compatibility, standardization, and integration with
    information systems. It is important to note that there is no single
    technological solution to improving supply chain security. As
    technology matures, it must be evaluated, and adjustments to
    operational plans must be made. Priority should be given to effective
    security solutions that complement and improve the business processes
    already in place, and which build a foundation for secure 21st century
    global trade.
        Mr. Chairman, Members of the Committee, today I have addressed
    CBP's commitment to investing its efforts in the areas of new and
    emerging detection technology, as well as some of the steps we have
    taken towards enhancing cargo security.
        Thank you again for this opportunity to testify. I will be happy to
    answer any of your questions.

                           Biography for Todd C. Owen
        Todd C. Owen is the Executive Director of the Cargo and Conveyance
    Security Office within U.S. Customs and Border Protection's Office of
    Field Operations. As the Executive Director for the Cargo and
    Conveyance Security (CCS) Office since May 2006, Mr. Owen is directly
    responsible for all cargo security programs and policies for CBP,
    including the ``100 percent scanning initiative'' announced in October,
    2007. Included within Mr. Owen's responsibilities are the Container
    Security Initiative (CSI), the Customs-Trade Partnership Against
    Terrorism Program (C-TPAT) Office, all non-intrusive inspection
    technology and radiation portal monitor deployments, the National
    Canine Enforcement Program, and the National Targeting Center, Cargo.
        Mr. Owen also coordinates CBP's maritime cargo enforcement policies
    and activities with the U.S. Coast Guard, and all air cargo efforts
    with the Transportation Security Administration.
        Previously, Mr. Owen was the Director of the C-TPAT program from
    January 2005 through May 2006. During his tenure as C-TPAT Director,
    this 9,000 member strong industry partnership program was strengthened
    by more clearly defining the security measures which must be adopted
    for a member to be eligible to receive the trade facilitation benefits
    afforded by CBP. Strong management controls were implemented and hiring
    was increased, allowing for a significant increase in the level of
    foreign site assessments performed worldwide under this program.
        Prior to arriving in Washington in January 2005, Mr. Owen was the
    CBP Area Port Director in New Orleans. As the Area Port Director, Mr.
    Owen was directly responsible for all CBP operations in New Orleans and
    throughout Louisiana. Two hundred forty officers are stationed in the
    seven Louisiana CBP port offices managed by the Area Director.
        Mr. Owen began his career as an Import Specialist in Cleveland,
    Ohio in 1990, and transferred to Miami in 1992. Through his eight years
    in South Florida, Mr. Owen held various trade related positions within
    Field Operations and the Office of Strategic Trade, before being
    selected for the New Orleans Area Port Director position in 2000.
        Mr. Owen, a career member of the Senior Executive Service, is a
    graduate of John Carroll University in Cleveland, Ohio, and was a
    senior executive fellow at Harvard University's John F. Kennedy School
    of Government. Mr. Owen also holds a Master's degree in Public
    Administration from St. Thomas University in Miami, Florida.

                                   Discussion

        Chairman Miller. I now recognize myself for five minutes.
    Dr. Hagan, you have heard a lot of criticisms of this program
    this morning, that is the Maginot Line of terrorism. It is
    pretty unlikely that terrorists will actually try to go through
    this system when it would be relatively easy to go around it,
    that a lot of the testing seems to be an afterthought to
    justify a conclusion that was already reached. The end-user,
    the Customs and Border Patrol, think that they are doing
    perfectly well and that their existing technology could be
    improved upon relatively cheaply compared to the investment
    that would be required, the spending that would be required for
    ASP. And both of the previous witnesses very much questioned
    the validity, the thoroughness of the testing that was being
    used to justify the decision to deploy ASP. What is the
    urgency? In view of all of that, what is the urgency about
    putting a certification decision before the DHS Secretary in
    October?
        Dr. Hagan. I guess there is a couple things I would like to
    say. First is that there is a sense of urgency regarding some
    of the limitations of the current system. We can't talk about
    these in open session, but we would be happy to do that at a
    later time. But having said that, I don't believe that today
    that there is a rush to deployment nor is there a `do it by
    October or something bad happens.' So what we are doing though
    is pushing forward as aggressively as we can to get to the
    decision point about deployment. In other words, we are not
    saying we have got to deploy these right now. We are saying we
    have got to get, as soon as possible, to the point where we
    make that decision, and as was said by the earlier witnesses,
    the information that is required to make that decision, as I
    mentioned in my oral statement, there is a lot of information,
    a lot of data, a lot of test results that have yet to be
    analyzed and brought together and integrated in this cost-
    benefit analysis, and that would be an important part of the
    decision that will be made in October. So, I guess I would
    disagree that we are rushing to deployment. I would say that we
    are rushing more towards getting all the information as fast as
    we can but doing it in a thoughtful, deliberate, disciplined
    way and through this MD 102-01 process that I mentioned, to get
    to that decision point.
        Chairman Miller. Mr. Owen, we heard again from GAO and the
    National Academies of Science that the work to this point did
    not really instill confidence in them that this new system
    would perform as reliably, as effectively as we would like or
    even as the existing technology works. They seem not concerned
    about a delay. They in fact encouraged a delay. Do you agree
    with their recommendation to stick with what we have got and
    not rush either to deploy or to certify or to decide to deploy?
        Mr. Owen. Sir, as the operator, the end-user of the
    systems, we, as front-line officers, need technology that will
    not only reduce the possibility of a missed threat but as well
    as reduce the rate of innocent alarms that our officers are
    spending their time on. In the port of Long Beach, for example,
    we have between 400 and 600 radiation alarms every single day.
    We have a team of about 100 officers over the three shifts that
    work on just resolving these innocent alarms throughout the
    course of their workday. So having a new piece of technology or
    an enhancement to technology that will allow us to not only
    identify any missed potential threats, but reduce the number of
    innocent alarms, is something that we do support. But as I
    mentioned in our hearing, our testimony, we have been doing
    this for some time with the existing PVTs. We have been able to
    resolve those 2.1 million alarms without a negative throughput
    with the flow of commerce in our ports of entry. So the
    technology that we have is effective but again, as we have
    matured the technology, we are looking for that technology
    which will give us an edge, if you will, a step up from what we
    have, particularly in terms of missing any threats that the
    PVTs may not capture right now.
        Chairman Miller. With respect to the existing technologies,
    the PVTs with the follow-up searches, are there enhancements,
    improvements in that technology that seem available,
    achievable, and have those improvements, are those being
    pursued?
        Mr. Owen. Our current system with the PVT which will
    alert--the container will then go in a secondary where we will
    perform the analysis with the radiation isotope identifier and
    we push that information back to our laboratory and scientific
    folks--it is a process that we have crafted and we feel
    comfortable with. What we have seen in the operational area, in
    terms of the energy windowing that you heard from the first
    panel, is we have seen operational benefits to being able to
    make adjustments to the existing PVTs to account for the
    recurring, burdensome, naturally occurring radioactive
    shipments that come through in legitimate cargo. So we have
    seen the benefits from energy windowing in the operational
    environment.
        Now, as to have we pushed those benefits as far as we can
    through energy windowing, we really need to defer to the
    scientists, to DNDO and our partners at Pacific Northwest
    National Laboratories to tell us if we have gone as far as we
    can with energy windowing. But again, as the operator of these
    systems, we have seen benefits in the ports of entry where we
    have deployed energy windowing. Have we maximized that? I would
    really defer to the scientists to give us that answer.
        Chairman Miller. I understand the principal problem with
    the current technology is as you have pointed out, not so much
    that it would miss--it is less a concern for missing radiation
    than the large number of false positives. Perhaps not false in
    the sense that it is not radiation, but it is not radiation
    that is a problem. How--I understand that the PVT system has
    been tested for that as well. How did it fare compared to the
    existing--I am sorry. The ASP system, how did it fare compared
    to the PVT system in false positives?
        Dr. Hagan. I will take that. The actual test data is
    classified, but I can answer in a qualitative way, that in--
    there are two places in which the ASP is being tested and
    considered. One is in primary inspection and the other is
    secondary inspection. Do you want me to explain what I mean by
    that?
        Chairman Miller. Well, for the record at least, sure.
        Dr. Hagan. Okay.
        Chairman Miller. Briefly.
        Dr. Hagan. So a conveyance comes in and goes through
    primary inspection or screening, and goes through an RPM or
    radiation portal monitor of some sort. If that monitor alarms,
    then that conveyance is referred to secondary inspection where
    it is further examined either with another monitor, a radiation
    portal monitor or a hand-held RIID. And ASP is being considered
    for deployments to both of those, both primary inspection and
    secondary inspection. In the primary application, the ASP is
    a--has the ability to identify the type of--the source of the
    radiation. The PVT is not able to do that. So PVT simply says
    there is radiation, better go to secondary and off it goes. In
    the case of ASP, because of the spectroscopic nature of the
    system, it can discriminate between a threat material and non-
    threat material, or if it can't discriminate, then it sends it
    to secondary as well. So it can dismiss and reduce the number
    of referrals to secondary. In the secondary application, if you
    have ASP there, then you are comparing essentially the
    performance of a large spectroscopic portal which is very, very
    large compared to a very small hand-held detector in which case
    the--again, I can't talk about the specifics of the results but
    the ASP is far superior to the hand-held detector in terms of
    its ability to, in real time, identify the source of the
    radiation and dismiss it as appropriate.
        Chairman Miller. Is it classified to say which did better,
    the PVT or the ASP?
        Dr. Hagan. No. I am sorry. I should have said, yes. So the
    ASP does far better in the secondary application of being able
    to identify----
        Chairman Miller. Well, which--I am sorry. Which system
    produced more false positives, ASP or PVT?
        Dr. Hagan. In the most recent----
        Chairman Miller. February.
        Dr. Hagan. Okay. In the most recent--you are talking about
    field validation testing, I think.
        Chairman Miller. Yes.
        Dr. Hagan. In field validation tests, the ASP system sent
    more--had more referrals to secondary than the PVT did.
    However, we have now--it is actually not surprising. It was the
    first time that ASP had been able to operate in a real
    operating environment, and so just as with PVT, adjustments
    were made to what are called thresholds in the ASP. So we are
    not making software changes, we are simply tuning the system as
    you do with the PVT, and then we have done a lot of analysis
    and simulation, as was mentioned earlier, to verify that these
    changes will in fact improve the performance and give us the
    expected performance.
        Chairman Miller. Okay. It appears that my time has really
    and truly expired.
        Dr. Broun for five minutes.
        Mr. Broun. Thank you, Mr. Chairman. I don't watch the clock
    very closely, as you very well know, and I will always give
    you--almost always give you a lot of leeway unless there is
    some particular reason not to.
        But Dr. Hagan, what you are saying, it is my understanding
    that the ASPs--you just made a statement that they are better.
    It is my understanding that that is only with the lightly
    shielded radiation. Isn't it true that with heavily shielded or
    moderately shielded radiation, that there is not much
    difference between the two programs? And isn't it also true
    that if anybody is going to bring radiation-type special
    nuclear materials in any shipment, aren't they going to be
    shielded and very highly so?
        Dr. Hagan. The answer to the second part first. One might
    speculate that that would be the case but you can't really know
    for sure, and so yes, it is probably likely that someone would
    shield it, but a knowledgeable adversary might not do that for
    other reasons which I will talk about later. But going back to
    the first part of your question, yes, the two systems, both PVT
    monitors and ASP monitors are what are called passive detection
    systems. They both suffer from the limitation that if a nuclear
    material or a weapon is shielded enough, then there just is no
    signal for either detector to detect. Now, so what we are--the
    long-range plan for this, and it is already being implemented
    in part, is to say that if a--by having these passive
    detectors, we are forcing the adversary, knowledgeable
    adversary, anyway, to heavily shield the object. Well, that
    heavily shielded object, then, is a very--well, I shouldn't say
    very but is an easy target or an easy image or easy thing to
    identify in a radiographic image or non-intrusive inspection
    system which Mr. Owen talked about. So by having complementary,
    in the sense, orthogonal type of systems, one that is passive
    system that will detect lightly shielded or unshielded
    material, and another system that will detect shielded
    material, then you have covered--you have done a good job of
    covering the complete spectrum of possibilities. Any system, of
    course, can be defeated by a knowledgeable adversary, but by
    having those two systems combined, then you really do a much
    better job of dealing with that problem. That being said, the
    amount of shielding that is needed, the effectiveness of PVT
    versus ASP for lightly or moderately shielded materials is--it
    depends on what it is you are shielding, and we would have to
    go into particulars and specifics about different types of
    objects, which we can't do here.
        Mr. Broun. Thank you, Dr. Hagan. My time is about up. I
    will yield back.
        Chairman Miller. Ms. Dahlkemper for five minutes.
        Ms. Dahlkemper. Thank you, Mr. Chairman.
        I live on Lake Erie and so I go back and forth to Canada a
    fair amount and actually I felt much safer a few years ago when
    were pulled over. My father had had a stress test two weeks
    prior so I do actually have some personal experience with those
    innocent alarms. I didn't understand what was going on at the
    time. It was the first time I had ever been pulled over.
        Mr. Owen. I hope we didn't cause too much stress.
        Ms. Dahlkemper. No, no, we were fine, but I actually felt
    much better coming back home and telling people who had no idea
    that this was actually being detected as we all would drive
    back home from Canada. But DHS has spent almost half a
    million--a billion dollars deploying PVT monitors at border
    crossings and ports across the country, including the one I
    came across, obviously. Does, Mr. Owen, the Customs and Border
    Protection believe that the current system, using PVTs and
    doing the secondary inspection with hand-held detectors and
    other means, is working well in terms of keeping us safe and
    moving commerce along? And I guess I am wondering what is the
    cost of manpower versus what is the cost of ASP? You know, I am
    looking at the dollars spent here, knowing we have limited
    resources and where are we best spending our money.
        Mr. Owen. And I would just respond, ma'am, that again two
    parts to that equation is, is the PVT finding all of the
    threats or are there threats that could be slipping through
    just because of the physics involved, and that is where ASP can
    help us in that regard. The second piece of that would be, can
    we respond to the number of secondaries that are caused by the
    existing PVT systems, and the answer is yes, we demonstrate
    that. There is a resource impact on this. Again, we would look
    at Los Angeles, we look at what we do up in the Peace Bridge
    there in Buffalo where you probably came through. It is a
    timely process. The officers do have to go through each
    secondary exam and treat it as if it is a real threat. We
    cannot let our guard down and just assume it was your father-
    in-law, that he had medical testing, let you go down. We have
    to take you out of the vehicle, we have to go through that
    whole process. So having a system that reduces the number of
    innocent alarms that are sent into secondary will help us do
    that. As to is the cost of the manpower savings offset by the
    ASP, I think that is some of the work that the DNDO is doing
    with their overall cost-benefit analysis. It will take into
    consideration those variables.
        Ms. Dahlkemper. I wanted to ask you too about the
    maintenance and operation costs of the ASPs and they are
    estimated to be anywhere from five to twelve times more
    expensive than the costs of running the PVTs, and so in the
    worst case ASPs will cost about $100,000 a year to run as
    compared to $8,000 a year for the PVTs. Does Customs have the
    budget to support this kind of system if it were deployed
    nationally, and have you thought about what you would have to
    give up in terms of personnel or other, you know, equipment to
    run this?
        Mr. Owen. Yes, and that is a very good question. That is a
    concern that we do have now. The ASP systems, depending on how
    the final outcome of the cost-benefit and the cost of the
    systems, I think it is well accepted that it is going to cost
    more than what we have with the PVTs. Currently we have about
    1,250 PVTs deployed nationwide. We continue to expand along the
    northern border. We are going to get--up to 100 percent of the
    cargo coming in from Canada this year will be covered by PVTs.
    We will end up with about 1,500 PVTs by the end of this
    calendar year. There is an extensive operation and maintenance
    tail that comes with that. I think any deployment decisions
    going forward with ASPs needs to be cognizant of the impact
    that will have on the operator. There is much talk about the
    initial acquisition cost and the deployment cost to buy it, put
    it in the ground, but I don't think we can forget about the
    operation and maintenance tail that comes along with this, that
    will then fall on the backs of the operator, in this case CBP.
    So it is something that we are concerned with. It is something
    that again is going into the overall cost adjustments for what
    we can expect with the ASP systems.
        Ms. Dahlkemper. Do you have any thoughts on what this would
    do in terms of personnel? Would it free up great numbers for
    other purposes?
        Mr. Owen. Well, my view on this is, when you look at the
    way the layouts are in the seaports, you have to have still an
    officer man those exit gates where you have the radiation
    portal monitors. So whether it is manned with two officers or
    one officer because the alarms are less frequent, there will
    still be a cost figure associated with the manpower. There may
    be some operational adjustments that we can make in the
    secondary areas but I think the cost savings are something that
    we still need to measure once we are confident that the ASPs
    are delivering as we hope they will do. So there will be some
    savings from that. Will that offset the increase in operation
    maintenance costs associated with the ASP? That is something
    that we have to take very careful consideration of.
        Ms. Dahlkemper. Thank you. My time is up. I yield back.
        Chairman Miller. Thank you, Ms. Dahlkemper. That will be
    the last round of questions. Dr. Hagan, I trust that when DHS
    has completed its cost-benefit analysis, that we will get a
    copy that is still warm from the printer.
        Dr. Hagan. Okay.
        Chairman Miller. So under the rules--before we bring the
    hearing to a close, I do want to thank all of our witnesses for
    testifying before the Subcommittee today. Under the rules of
    the Committee, the record will remain open for two weeks for
    additional statements from the Members and for answers to any
    follow-up questions any Member of the Committee may have for
    witnesses, and I understand that Dr. Broun does have questions.
    The witnesses are excused and the hearing is now adjourned.
        [Whereupon, at 11:41 a.m., the Subcommittee was adjourned.]
                                   Appendix:

                                  ----------

                       Additional Material for the Record

    [SKIP PAGES = 000]

    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

        HEARING ON THE SCIENCE OF SECURITY, PART II: TECHNICAL PROBLEMS
                 CONTINUE TO HINDER ADVANCED RADIATION MONITORS

                                  ----------

                           TUESDAY, NOVEMBER 17, 2009

                      House of Representatives,
          Subcommittee on Investigations and Oversight,
                           Committee on Science and Technology,
                                                       Washington, D.C.
        The Subcommittee met, pursuant to call, at 1:05 p.m., in
    Room 2310 of the Rayburn House Office Building, Hon. Brad
    Miller [Chairman of the Subcommittee] presiding.
    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

                                hearing charter

                  SUBCOMMITTEE ON INVESTIGATIONS AND OVERSIGHT

                      COMMITTEE ON SCIENCE AND TECHNOLOGY

                         U.S. HOUSE OF REPRESENTATIVES

                       The Science of Security, Part II:

                         Technical Problems Continue to

                       Hinder Advanced Radiation Monitors

                           Tuesday, November 17, 2009
                              1:00 p.m.-3:00 p.m.
                       2318 Rayburn House Office Building

    Purpose

        The Subcommittee on Investigations and Oversight meets on November
    17, 2009, to examine continuing problems with the Department of
    Homeland Security's (DHS) efforts to acquire its next generation
    radiation monitors known as Advanced Spectroscopic Portals (ASPs). This
    is a follow-up to the hearing the Subcommittee held on June 25, 2009,
    titled: The Science of Security: Lessons Learned in Developing, Testing
    and Operating Advanced Radiation Monitors. Since the Domestic Nuclear
    Detection Office (DNDO), a DHS component, was created in 2005 they have
    been responsible for researching, developing, testing and managing the
    program.
        The ASP program is estimated to cost $2-to-$3 billion and has been
    under scrutiny since 2006 for failing to have clear-cut requirements,
    an adequate test plan, sufficient timelines, development milestones or
    a transparent and comprehensive cost benefit analysis. These problems
    have been identified by the Government Accountability Office, National
    Academy of Sciences, the Homeland Security Institute, a Federally
    Funded Research and Development Center for DHS, and the National
    Institute of Standards and Technology.
        In July, one month after the Subcommittee's last hearing, the ASPs
    went through a second round of Field Validation Tests. During the tests
    the ASPs exhibited several ``false positive'' alarms for special
    nuclear material that did not exist. In another disturbing incident
    during the tests, one ASP monitor stopped working altogether yet the
    system operator remained unaware of this malfunction. Two dozen cargo
    trucks were permitted to go through the non-functioning portal monitor
    in order to be screened for potential radioactive and nuclear material
    until the problem became apparent. DNDO considered this a ``Mission
    Critical Failure.'' No new plans have yet been scheduled to re-test the
    ASPs for the third time. The Subcommittee will examine the results from
    the most recent tests, continuing technical problems with the ASPs,
    supply shortages of a key component for radiation monitors that may
    hinder the eventual deployment of the ASPs and further drive up its
    potential cost, and potential enhancements to the current fleet of
    radiation monitors in use today.

    Background

        Since the September 11, 2001 terrorist attacks, protecting the
    Nation from a nuclear or radiological attack has been a top national
    security priority. In 2002, to help address this threat, the U.S.
    Customs and Border Protection (CBP) agency began deploying radiation
    monitors at U.S. border sites and ports of entry to screen the more
    than 23 million cargo containers that enter the country every year for
    radiological and nuclear materials.
        Polyvinyl toluene (PVT) radiation portal monitors have been used to
    screen this cargo since then. They are able to detect the presence of
    radioactive sources, but unable to identify the type of radiation
    present. The PVT monitors, while relatively inexpensive, robust and
    highly reliable, are unable to distinguish between radioactive sources
    that might be used to construct a nuclear bomb, such as Highly Enriched
    Uranium (HEU), and non-threatening naturally occurring radiological
    materials (NORM) contained in ceramic tiles, zirconium sand or kitty
    liter, for instance. As a result, any time a PVT detects a radioactive
    source the cargo is sent to ``secondary'' screening where CBP agents
    verify the detection of the source with a second PVT monitor and use
    hand-held Radioactive Isotope Identification Devices called RIIDs to
    help identify the source of radiation.
        This method of operation leads to many ``secondary'' inspections
    for naturally occurring radioactive material or radioactive material
    intended for benign purposes, such as radioactive medical isotopes. At
    the Los Angeles/Long Beach port of entry, for instance, PVT monitors
    routinely send up to 600 conveyances of cargo to secondary inspection
    each day. In addition, the RIIDs used in secondary inspections are
    limited in their abilities to locate and identify potential radioactive
    material in large cargo containers.
        In order to help improve the flow of commerce by eliminating many
    of the unnecessary alarms that send cargo for secondary screening and
    to more accurately identify radioactive or nuclear material, the
    Department of Homeland Security (DHS) began developing Advanced
    Spectroscopic Portals (ASPs) in 2004. The ASPs were intended to both
    detect and identify radioactive material. In April 2005, the Domestic
    Nuclear Detection Office was created by National Security Presidential
    Directive-43/Homeland Security Presidential Directive-14 to, among
    other things, research, develop, test and acquire radiation detection
    equipment to be used by CBP and other federal agencies.
        In July 2006, then-Secretary of Homeland Security Michael Chertoff
    and the former Director of DNDO, Vayl Oxford, announced contract awards
    to three companies worth an estimated $1.2 billion to develop the ASPs,
    including the Raytheon Company and the Thermo Electron Company (now
    called Thermo Fisher Scientific Inc.) both headquartered in Waltham,
    Massachusetts and Canberra Industries from Connecticut. Canberra is no
    longer a contractor on the DNDO program.

    ASP Requirements/Criteria

        One of the key reasons for replacing the existing radiation
    monitors with newly developed ASPs in the first place, as articulated
    by Secretary of Homeland Security, Michael Chertoff in July 2006 was to
    ``have fewer false positives.'' In September 2007, Vayl Oxford, then
    the director of DNDO reiterated that point in testimony to Congress
    where he emphasized that the ASPs would reduce the number of false
    alarms from the nearly 600 experienced each day by the PVTs at the port
    of Long Beach in California, for instance, to 20-to-25 per day with the
    new ASP monitors. That was the hope, but it has not been the reality
    during testing of the ASPs and other serious security questions about
    the performance reliability of the ASPs have emerged in the most recent
    round of tests.
        As the House Committee on Appropriations has said in the past,
    procurement of the Advanced Spectroscopic Portal monitors should not
    proceed until they are deemed to add a ``significant increase in
    operational effectiveness'' over the current PVT system already in
    place. In July 2008, CBP, DNDO and the DHS management directorate
    jointly issued criteria for determining this increase in effectiveness
    in both ``primary'' and ``secondary'' screening. In primary screening
    the criteria requires ASPs to detect potential threats as well as or
    better than PVTs, show improved detection of Highly Enriched Uranium
    and reduce innocent alarms. In secondary screening the criteria
    requires ASPs to reduce the probability of misidentifying special
    nuclear material (HEU or plutonium) and reduce the average time to
    conduct secondary screenings. The Secretary of Homeland Security must
    certify to Congress that the ASPs have met these criteria before
    funding for full-scale procurement of the ASPs goes forward. The
    criteria to measure this improvement, however, are weak and rather
    vague.

    Testing Regime

        Significant hurdles remain before ASPs can be certified and fully
    deployed. Both contractors have passed ``integration testing.'' They
    must now successfully make it through Field Validation Tests where they
    operate at ports of entry in tandem with PVT units. So far, only one of
    the two ASP vendors has made it to this stage. The one vendor that has
    made it to this stage will need to make its third attempt to
    successfully pass the Field Validation Tests before it can move
    forward. If and when they successfully pass this stage of testing they
    will then go to ``Solo Operations,'' where they will be tested at a
    port-of-entry operating independently of the PVTs. If they pass those
    two critical tests, then the DHS Directorate of Science & Technology
    which has been mandated the Operational Testing Authority (OTA) of the
    ASPs will put them through a separate series of tests to ensure they
    meet the specified requirements, do not suffer from technical glitches
    and operate efficiently. Once that testing is completed and the S&T
    Directorate signs off on the performance and reliability of the ASPs
    then the DHS Secretary must make a determination about whether the
    costs of the ASPs and the capabilities they provide justifies a
    decision to invest in their full scale deployment. Along the way DNDO
    is supposed to provide a final cost-benefit-analysis of the ASP program
    to help inform the Secretary's decision. This document has been
    promised many times but not yet completed.

    Masking & Shielding

        If terrorists were to try to smuggle nuclear or radiological
    materials into the U.S. via containerized cargo they would likely try
    to shield and/or mask those materials in an attempt to make it more
    difficult to detect, identify and locate the material of concern.
    Shielding requires that lead or other types of metal enclose the
    radioisotopes to hide its radioactive signature. Potential terrorists
    may also attempt to ``mask'' threatening radioactive material by
    placing it together with or alongside other non-threatening material
    that has a natural radioactive signature, such as ceramic material,
    kitty liter or even bananas. Most nuclear security experts believe
    smuggled radioactive or nuclear material would be both shielded and
    masked in order to conceal it from being located and properly
    identified. These efforts would make it harder to detect.
        Many of DNDO's previous tests of the ASPs have been criticized for
    being less than realistic. In one series of tests the ASP portals did
    prove more effective than the PVTs in detecting HEU materials concealed
    by ``light shielding.'' However, differences between the ASPs and PVTs
    became less notable when shielding was slightly increased or decreased.
    In other tests there was virtually no difference in the performance of
    the two machines with regard to detecting other kinds of radioactive
    isotopes, such as those used for medical or industrial purposes,
    according to the GAO, except in one case where the ASPs performed worse
    than the PVTs. In the most recent round of tests in July DNDO says the
    ASPs detected one radioactive source that the PVTs missed.
        In previous attempts to detect HEU during tests, the ASPs performed
    better only in one narrowly defined scenario, which many experts see as
    an unrealistic portrayal of a true attempted nuclear smuggling
    incident. None of the tests run by DNDO, for instance, included
    scenarios that utilized both ``shielding'' and ``masking'' as a means
    of attempting to smuggle radioactive or nuclear material. In addition,
    only one of the vendors has made it to field validation testing. But as
    the contractor has attempted to fix problems that occurred during
    previous tests new, more serious technical issues have emerged.

    Field Validation Tests

        The Raytheon ASPs went through their first round of field tests
    last February, but technical issues hampered their performance. They
    had a large number of false alarms on several radioactive isotopes.
    Overall, in fact, the ASPs sent more cargo for secondary inspection
    than the currently operating PVTs did. Adjustments were made to prepare
    them for another round of field tests. Since the Subcommittee's last
    hearing on the ASP program in June, the ASPs have gone through a second
    Field Validation Test at four U.S. ports of entry in L.A. Long Beach,
    California; the New York Container Terminal in Newark, New Jersey; Port
    Huron, Michigan; and Laredo, Texas.
        On average, the PVTs refer one out of every 40 cargo containers to
    secondary inspection placing a large a burden on the staffing resources
    of CBP. The ASPs are required to send only one out of every 1,000
    inspections to secondary inspection in order to help lessen that
    logistical burden. This is one of the key requirements that must be met
    in order for the Secretary of Homeland Security to permit full scale
    production of the ASPs to proceed. During the Field Validation Testing
    last February, however, the ASPs sent more than five times that number
    of cargo conveyances to secondary inspection based on false alarms.
    During the most recent Field Validation Tests in July the ASPs
    reportedly reduced the number of false alarms compared to the PVTs by
    69 percent bringing them much closer to the 80 percent reduction in
    false alarms that they are required to meet. But new, more serious
    problems also emerged during the field validation tests in July.
        During this second round of field tests the ASPs again failed to
    perform as expected. This time they falsely identified several cargo
    conveyances as having special nuclear material, when they actually had
    none. This is a critical issue, since the actual smuggling of special
    nuclear material presents a serious threat. If it is detected at a
    port-of-entry Customs and Border Protection officers have extensive
    response requirements they must implement. DNDO and the contractor are
    still unclear why the ASPs falsely identified special nuclear material
    during these tests. Their intended fix to this problem has been to
    decrease the sensitivity of the ASP monitors to specific radioactive
    isotopes. The hope is that this will correct the problem, reduce the
    number of false alarms and still ensure that the ASPs are able to
    detect these isotopes. It is a delicate and difficult balance. It also
    decreases the ostensible advantage of having the ASPs replace the PVTs
    in the first place.
        Most unsettling, in one instance during the July tests one ASP
    monitor stopped working altogether yet the system operator remained
    unaware of this malfunction. Two dozen cargo trucks were permitted to
    go through the ASP in order to be screened for potential radioactive
    and nuclear material while it was not operating. DNDO considered this a
    ``Mission Critical Failure.'' Fortunately, during these tests all
    trucks that went through the ASP also went through a PVT monitor. If
    this had occurred during ``solo'' testing of the ASPs or during actual
    deployment of the ASPs, cargo carrying radiological or nuclear threat
    material could have sailed past port security and into the United
    States unchecked. The cause of this problem has reportedly been
    rectified by the contractor.

    Energy Windowing

        Many experts believe significant improvements can be made to the
    existing fleet of PVT radiation monitors without investing billions of
    dollars into new ASPs. Energy windowing is a mathematical algorithm
    that can help improve the sensitivity of PVT radiation monitors,
    enhancing their ability to detect radioactive sources resulting in
    improved operations and capabilities. The technology is currently used
    in some radiation monitors. Both GAO and CBP believe that DNDO should
    much more aggressively invest in this research to improve the
    performance of the currently operating radiation detection monitors.
    Although energy windowing may only lead to modest enhancements in the
    performance of PVTs, that improvement could be significant in terms of
    improving their performance to be more on par with what ASPs are
    supposed to be capable of and at a far less financial cost. Reducing
    the sensitivity of the ASPs to certain types of special nuclear
    material, which was done to resolve the problems that emerged during
    the July tests, should not prevent them from alarming for isotopes that
    were not there in the first place. The only result would be to reduce
    the odds that the ASPs will identify those isotopes when they are
    actually present.

    A Dwindling Supply of Helium-3 (He-3)

        The future deployment of both PVT and ASP monitors is dependent on
    the supply of Helium-3 (He-3), a non-radioactive gas that is a
    byproduct of tritium decay. Tritium is a critical component in nuclear
    weapons used to boost the yield of nuclear warheads. Helium-3 gas is
    used in neutron detector tubes, a component of both PVT and ASP
    radiation portal monitors used to help identify plutonium. He-3 is also
    used in medical imaging, such as MRI machines, the oil and gas industry
    and for high energy research. During the cold war the U.S. had a steady
    supply of He-3 as a result of its nuclear weapons production
    operations. With the end of the cold war the production of nuclear
    weapons ceased and this supply diminished. At the same time, since 9/11
    the demand for radiation monitors skyrocketed and demand for He-3 soon
    out-paced the supply.
        There are no readily available alternatives to He-3. In addition,
    no other technology matches the stability, sensitivity, and ability to
    detect neutron radiation that He-3 neutron tubes currently offers. DNDO
    has estimated that the anticipated supply-to-demand ratio of Helium-3
    in coming years is expected to be 1-to-10. Costs for the rare isotope
    have already begun to rise. By one estimate, a few years ago the cost
    of He-3 was around $100 per liter. Today, He-3 is estimated to cost as
    much as $2,000 per liter. According to a recent Department of Energy
    report, new ASP radiation monitors will use nearly three times more He-
    3 as current PVT monitors do, about 132 liters compared to 44 liters.
    These facts should be carefully considered by the Secretary of DHS when
    making cost-benefit decisions about whether or not to proceed with
    producing the ASPs.

    Cost Benefit Analysis

        Even if the technical abilities of the ASPs are proven, their
    relative technical capabilities and increased costs must be carefully
    weighed in comparison to the existing radiation monitoring system in
    place today. Replacing a proven, less-costly system that has the
    confidence of its operators, must be given careful consideration. The
    DNDO has not yet provided an updated cost-benefit-analysis that would
    validate a decision to procure the multi-billion dollar ASP equipment.
        Virtually any high-technology research and development program
    experiences bumps in the road, technical troubles and occasional set-
    backs. However, well managed programs have clear technical requirements
    and strategic goals. They ensure that the new technology being
    developed is thoroughly tested and adequately integrated into the
    operational plans and procedures of those who must operate them in the
    field. When these vital components are short changed, when the test
    plan is insufficient and the program's research, development and
    testing methods are marred by scanty scientific rigor then the
    technical tools being developed are bound to suffer as a result.
    Cutting critical corners in the development process serves no one's
    interests. Yet, at the start of the ASP program many of the DNDO
    leaders seemed more interested in fielding this technology then in
    effectively validating its performance and effectiveness. At the July
    2006 press conference unveiling the contractors on the ASP program,
    Vayl Oxford then the Director of DNDO said: ``the priority for the
    first year . . . is to get units out immediately.'' Three years later,
    none of the ASPs have yet cleared field validation tests.

    Witnesses:

    Mr. Gene Aloise, Director, Natural Resources and Environment,
    Government Accountability Office (GAO)

    Dr. Timothy M. Persons, Chief Scientist, Government Accountability
    Office (GAO)

    Mr. Todd Owen, Executive Director for Cargo and Conveyance Security,
    U.S. Customs and Border Protection (CBP), Department of Homeland
    Security (DHS)

    Dr. William Hagan, Acting Deputy Director, Domestic Nuclear Detection
    Office (DNDO), Department of Homeland Security (DHS)
        Chairman Miller. Welcome to today's hearing entitled The
    Science of Security, Part II: Technical Problems Continue to
    Hinder Advanced Radiation Monitors.
        Soon after the September 11 attacks, Customs and Border
    Protection, CBP, began operating radiation portal monitors to
    screen cargo entering the United States for radiological and
    nuclear material. They have purchased approximately 1,500
    polyvinyl toluene monitors and deployed them at ports and
    border crossings throughout the United States. Mercifully,
    polyvinyl toluene monitors are generally referred to as PVTs.
        PVTs indicate the presence of a radiation source, but they
    cannot identify the nature of the source. As a result, any
    cargo container that provokes a warning from a PVT is then sent
    to a secondary inspection where customs officers use other
    technology and information to determine what sort of material
    is in the container. There are plenty of innocent sources of
    radiation: kitty litter, medical isotopes, ceramics, bananas,
    and many of these secondary inspections can be handled quickly.
    However, some secondary inspections require that the container
    be opened, and even emptied, in the search for a source. That
    is time consuming, but the PVT seems to be working well to meet
    customs' dual mission to keep us safe while maintaining a
    steady flow of commerce.
        The Department of Homeland Security (DHS) has been
    developing a new radiation portal monitor, championed by the
    Domestic Nuclear Detection Office, DNDO, that advocates believe
    should replace the PVTs. The new monitor, the Advanced
    Spectroscopic Portal monitor, or ASP, would detect the presence
    of radiation in cargo, but it would also identify the type of
    radiation. That would allow more harmless cargo to pass
    unimpeded through the port and require far fewer secondary
    referrals. If it worked as advertised, the ASPs would be more
    likely to identify highly enriched uranium or other materials
    of concern and enhance the flow of commerce while freeing up
    customs officers to tend to other duties besides secondary
    inspections.
        Despite a $230 million investment of taxpayer dollars thus
    far for development, the ASPs haven't performed as expected,
    and the results from recent tests are still worrisome. Last
    June we learned of problems in the first field test in February
    2009. At our June hearing we heard that those issues had been
    fixed and that the July field test would allow the department
    to move forward towards a cost-benefit analysis and
    certification decision for the Secretary. So far, we have seen
    neither.
        The July test highlighted yet another problem with the
    ASPs. The devices detected nuclear materials when none were
    present.
        The ASPs had numerous false positive hits for special
    nuclear material in July, each one of which would have resulted
    in the implementation of mandated security responses by
    Customs, potentially shutting down port operations.
    Fortunately, Customs was also running the PVTs which saw no
    radiation presence, and there was none, and cleared up the
    issue in secondary fairly quickly.
        DNDO has told our staff that they intend to fix this new
    problem by changing the sensitivity of the ASPs to detect
    uranium, but it isn't clear why that should be reassuring,
    changing the sensitivity setting. If we lower the sensitivity
    setting to the very materials the ASP is supposed to be better
    at monitoring or detecting, would the ASP still be better than
    PVTs at detecting those materials? And would we have to go back
    to the Nevada Test Site to prove that you can still detect
    levels of special nuclear material more accurately than the
    PVTs can?
        Second, if the machines detect special nuclear material
    when there is none there, how does changing the sensitivity
    make any difference at all? Detecting ghost isotopes is a
    problem with the operating system, not with the sensitivity
    level, it would appear anyway.
        Since the taxpayers have spent $200,000 for each of the
    1,500 PVTs that are deployed now, the case for ASPs, which will
    run about $800,000 per unit, a total cost of $2 to $3 billion,
    needs to be clear both in terms of better detection performance
    and better support for Customs operations. Add to that greater
    acquisition cost, an annual operating expense of ASPs that is
    at least five times more expensive per unit than PVTs, and the
    need for a convincing case is even greater. As it stands, it is
    hard to see why ASPs should be more than a secondary inspection
    tool.
        In fact, that is the role they play in the Department of
    Energy's Megaport program. The DOE already runs a program that
    inspects cargo leaving 27 major foreign ports for destinations
    anywhere in the world. DOE, which developed portal radiation
    detection technology, uses PVTs for primary inspection and then
    uses the ASPs for secondary inspection to help identify the
    type of isotope to which the PVT responded in the first place.
    The Department of Energy's approach to identifying radiation
    should be instructive to DHS.
        I want to thank all of our witnesses for appearing today. I
    particularly want to thank GAO for continuing their work on
    this matter and for their continued assistance to this
    committee and Congress, and the Appropriations Committee as
    well. It is very helpful for the Appropriations Committee to be
    asking the right questions, and you have helped them and helped
    us ask the right questions. I suspect that this will not be the
    last time that we gather on this subject--we do seem to come
    back to the same subjects again and again--nor the last time we
    hear from witnesses that we still face a long list of tests and
    validations before we can think about replacing the PVTs with
    the ASPs.
        I would now recognize the Ranking Member, Dr. Broun, for
    his opening comment.
        [The prepared statement of Chairman Miller follows:]
                   Prepared Statement of Chairman Brad Miller
        Soon after the September 11 attacks, Customs and Border Protection
    (CBP) began operating radiation portal monitors to screen cargo
    entering the United States for radiological or nuclear material. They
    have purchased approximately 1,500 polyvinyl toluene (PVT) monitors and
    deployed them at ports and border crossings throughout the United
    States.
        PVTs indicate the presence of a radiation source, but they cannot
    identify the nature of the source. As a result, any cargo container
    that provokes a warning from a PVT is then sent to a ``secondary''
    inspection where Customs officers use other technology and information
    to determine what sort of material is in the container. There are
    plenty of innocent sources of radiation--from kitty litter to medical
    isotopes--and many of these secondary inspections can be handled
    quickly. However, some secondary inspections require that the container
    be opened, and even emptied, in the search for a source. While this is
    time consuming, the PVT seems to be working well to meet Customs' dual
    mission to keep us safe while maintaining a steady flow of commerce.
        The Department of Homeland Security has been developing a new
    radiation portal monitor, championed by the Domestic Nuclear Detection
    Office (DNDO), that advocates believe should replace the PVTs. This new
    monitor--the Advanced Spectroscopic Portal monitor or ASP--would detect
    the presence of radiation in cargo, but it would also identify the type
    of radiation. This would allow more harmless cargo to pass unimpeded
    through the port and require far-fewer secondary referrals. If it
    worked as advertised, the ASPs would be more likely to identify highly
    enriched uranium and other materials of concern and enhance the flow of
    commerce while freeing up Customs officers to tend to other duties
    besides secondary inspections.
        Despite a $230 million investment of taxpayer dollars for
    development, the ASPs haven't performed as expected, and the results
    from recent field tests are worrisome. Last June we learned of problems
    in the first field test of February 2009. At our June hearing we heard
    that those issues had been fixed and that the July field test would
    allow the Department to move towards a cost-benefit analysis and
    certification decision for the Secretary. To date, we have seen
    neither.
        The July field test highlighted yet another problem with the ASPs:
    the devices detected nuclear materials when none were present.
        The ASPs had numerous false positive hits for special nuclear
    material in July--each one of which would have resulted in the
    implementation of mandated security responses by Customs, potentially
    shutting down port operations. Fortunately, Customs was also running
    the PVTs (which properly saw no radiation present) and cleared up the
    issue in secondary.
        DNDO has told our staff that they intend to fix this new problem by
    changing the sensitivity of the ASPs to detecting uranium, but it isn't
    clear that should be reassuring. If you lower the sensitivity to the
    very materials the ASP was supposed to be better at detecting, why
    would the ASP will still be better than PVTs at detecting those
    materials? Would you have to go back to the Nevada Test Site to prove
    that you can still detect levels of special nuclear material more
    accurately than the PVTs can?
        Second, if the machines detect special nuclear material where it
    doesn't exist, why should changing the sensitivity make any difference
    at all? Detecting ghost isotopes is a problem with the operating system
    itself, not with the sensitivity level for a particular isotope.
        Since the taxpayers have spent $200,000 for each of the 1,500 PVTs
    already deployed, the case for ASPs, which will run approximately
    $800,000 per unit--for a total cost of $2-3 billion--needs to be clear
    both in terms of better detection performance and better support for
    Customs operations. Add to this greater acquisition cost, an annual
    operating expense of ASPs that is at least five times more expensive
    per unit than PVTs, and the need for a convincing case is even greater.
    As it stands, it is hard to see why ASPs should be more than a
    secondary inspection tool.
        In fact, that is the role they play in the Department of Energy's
    Megaport program. DOE already runs a program that inspects cargo
    leaving 27 major foreign ports for destinations anywhere in the world.
    DOE, which developed portal radiation detection technology, uses PVTs
    for primary inspection and reserves ASPs for secondary inspections to
    help identify the type of isotope to which the PVT responded. The
    Department of Energy's approach to identifying radiation should be
    instructive to DHS.
        I want to thank our witnesses for attending today. I particularly
    want to thank GAO for their continuing work on this matter and for
    their continuing assistance to this committee and Congress. I suspect
    that this will not be the last time we gather on this subject, nor the
    last time we hear from witnesses that we still face a long list of
    tests and validations before we can even speak sensibly about replacing
    PVTs with ASPs.

        Mr. Broun. Thank you, Mr. Chairman. I want to welcome the
    witnesses here today and thank you all for participating in our
    follow-up hearing on the Department of Homeland Security's
    Advanced Spectroscopic Portal program. It is hard for a
    Southerner to say quickly.
        This afternoon we will be brought up to date on the
    Department's ongoing development of the next generation
    radiation portal monitors and get an update from the GAO on
    their continuing work. As I said in our earlier hearing this
    past summer, this program is certainly not out of the woods.
    The latest field validation tests reveal additional problems
    that will have to be overcome before moving forward. I hope
    DNDO will be able to give us some insight today on what we can
    expect from this program in terms of the future paths forward.
    With considerable taxpayer money on the line, questionable
    improvements over current capabilities and outstanding cost
    benefit analysis and a confusing acquisitions history that
    unfortunately has morphed the R&D with procurement, this
    program is rapidly approaching a point where the Federal
    Government has to decide whether it wants to fish or cut bait.
    I am concerned with the fact that considerable public funding
    has been expended on developing a technology that the private
    sector was developing in parallel on its own dime. DHS as a
    whole, and DNDO, CBP, DHS, S&T individually, should be focusing
    on long-term, high-risk, high-reward technology, not providing
    seed money for commercial, off-the-shelf equipment. That being
    said, I realize that DHS's mission is vastly different from
    DOE's, the Department of Defense's and that they have
    additional requirements that demand a more robust system. GAO
    and the Academy made several recommendations over the past few
    years. I trust that DNDO and CBP will be able to update this
    committee on how they are responding to those recommendations
    and where they plan to go from here. The Nation expects a lot
    from the Department, and I hope that we aren't developing
    tunnel vision by focusing too much on one method of conveyance
    and not seeing the forest for the trees. The Department has an
    enormous task of securing our borders and not just at points of
    entry but all along our borders. Spending billions of dollars
    to secure the front door of our house doesn't seem very
    rational if we are just going to leave the back door open and
    all the windows and have a gaping hole in the walls, too. That
    is not to say that we should do nothing at all, but rather,
    everything we do should be put in context of a well-thought-out
    global nuclear detection architecture.
        As I said earlier this summer, many of these issues we are
    dealing with today could have been prevented by engaging the
    end-users earlier in the process. Clearly defining the
    requirements, developing clear architectural priorities and
    simply following a clear acquisition process. This Committee is
    no stranger to programs that have set aside these best
    practices for working in expediency's sake.
        I look forward to working with the Department and the
    Majority to make sure any decision that is made is in the best
    interest of our nation's security, the taxpayer and our
    economy.
        With that, Mr. Chairman, I yield back my time and I thank
    you.
        [The prepared statement of Mr. Broun follows:]
               Prepared Statement of Representative Paul C. Broun
        Thank you, Mr. Chairman. I want to welcome the witnesses here
    today, and thank them for participating in our follow-up hearing on the
    Department of Homeland Security's (DHS) Advanced Spectroscopic Portal
    (ASP) program. This afternoon we will be brought up to date on the
    Department's ongoing development of next generation Radiation Portal
    Monitors and get an update from the General Accountability Office (GAO)
    on their continuing work.
        As I said at our earlier hearing this past summer, this program is
    certainly not out of the woods. The latest Field Validation Test
    revealed additional problems that will have to be overcome before
    moving forward. I hope DNDO will be able to give us some insight today
    on what we can expect from this program in terms of future paths
    forward. With considerable. taxpayer money on the line, questionable
    improvements over current capabilities, an outstanding cost-benefit
    analysis, and a confusing acquisitions history that unfortunately has
    morphed Research and Development (R&D) with procurement, this program
    is rapidly approaching a point where the Federal Government has to
    decide to ``fish or cut bait.''
        I'm also concerned with the fact that considerable public funding
    has been expended on developing a technology that the private sector
    was developing in parallel on its own dime. DHS as a whole (and DNDO,
    CBP, and DHS S&T individually) should be focusing on long-term high-
    risk high-reward technology, not providing seed money for Commercial
    Off-The-Shelf (COTS) equipment. That being said, I realize that DHS'
    mission is vastly different from the Department of Energy's (DOE) and
    the Department of Defense's (DOD), and that they have additional
    requirements that demand a more robust system.
        GAO and the Academy made several recommendations over the last few
    years. I trust that DNDO and CBP will be able to update this committee
    on how they are responding to those recommendations, and where they
    plan to go from here. The Nation expects a lot from the Department, and
    I hope that we. aren't developing tunnel vision by focusing too much on
    one method of conveyance and not seeing the forest through the trees.
    The Department has an enormous task of securing our borders, not just
    at points of entry, but all along our borders. Spending billions of
    dollars to secure the front door or our house, doesn't seem very
    rational if we are just going to leave the back door open: That is not
    to say we should do nothing at all, but rather everything we do should
    be put in the context of a well thought out Global Nuclear Detection
    Architecture.
        As I said earlier this summer, many of the issues we are dealing
    with today could have been prevented by engaging the end-users earlier
    in the process, clearly defining requirements, developing clear
    architectural priorities, and simply following a clear acquisition
    process. This committee is no stranger to programs that have set aside
    these best practices for expediency's sake. I look forward to working
    with the Department and the majority to make sure any decision made is
    in the best interest of our nation's security, the taxpayer, and our
    economy.
        With that, Mr. Chairman, I yield back my time.
        Thank you.

        Chairman Miller. Thank you, Dr. Broun. I ask unanimous
    consent that all additional opening statements submitted by
    Members be included in the record. Without objection, it is so
    ordered. Also, there has been an e-mail exchange between our
    staff and the staff of DNDO, Kimberly Koeppel, and without
    objection, I move that the printed versions of the e-mailed
    questions and answers also be entered into the record. Without
    objection, so ordered.
        [The information follows:]
        [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

        Chairman Miller. It is my pleasure to introduce our
    witnesses at this time. Mr. Gene Aloise is the Director of
    Natural Resources and Environment at the Government
    Accountability Office, GAO. He is an expert in international
    nuclear proliferation and safety issues and holds degrees in
    political science, economics and public administration. Mr.
    Aloise is a recipient of GAO's Meritorious Service and
    Distinguished Services Awards and has served several
    Congressional committees and offices within GAO. This is not
    the first time he has appeared before us on this topic.
        With Mr. Aloise is Dr. Timothy Persons, GAO's Chief
    Scientist. Before entering GAO, before joining GAO, Dr. Persons
    was the Technical Director of the Intelligence Advanced
    Research Projects Activity, I-ARPA, and previously served as a
    technical director at the National Security Agency. Mr.
    Persons, if people ask you at a cocktail party what you did for
    a living, did you just say you worked for the government? He
    holds degrees in nuclear physics, computer science and
    biomedical engineering and has been involved in evaluations of
    many high-tech U.S. Government programs and projects, including
    GAO's work on the Advanced Spectroscopic Portals, ASP monitors.
    We look forward to hearing his testimony today as well.
        Mr. Todd C. Owen is the Acting Deputy Assistant
    Commissioner of the Office of Field Operations for the U.S.
    Customs and Border Protection, Department of Homeland Security.
    Also a mouthful. He also served as the Executive Director of
    the Cargo and Conveyance Security Office, Office of Field
    Operations, since May of 2006. Mr. Owen began his career with
    the Customs and Border Protection in 1990 and has previously
    held the positions of Area Port Director in New Orleans and
    Director of the Customs Trade Partnership Against Terrorism, C-
    TPAT Program.
        And then finally, Dr. William Hagan is the Acting Deputy
    Director of the Domestic Nuclear Detection Office at the
    Department of Homeland Security. Dr. Hagan holds degrees in
    physics and nuclear engineering as well as three patents. He
    spent 30 years with the Science Applications International
    Corporation before coming to DNDO and serving as the Assistant
    Director of the Transformational Research and Development
    Directorate.
        I think all of our witnesses would need to use small print
    for their business cards.
        As our witnesses should know, you will each have five
    minutes for your spoken testimony. Your written testimony will
    be included in the record for the hearing. When you all have
    completed your spoken testimony, we will begin with questions.
    Each Member will have five minutes to question the panel. It is
    the practice of this subcommittee to receive testimony under
    oath. This is an Investigations and Oversight Subcommittee. Do
    any of you have an objection to taking an oath? Let the record
    reflect that none of the witnesses had any objection. You also
    have the right to be represented by counsel. Do any of you have
    counsel here? Let the record reflect that all of the witnesses
    said no. We ask you these questions to put you at ease.
        Would you all now please stand and raise your right hand?
        Mr. Broun. And we are not going to water board them.
        Chairman Miller. Well, that would be the next hearing.
    Please stand and raise your right hand. Do you swear to tell
    the truth and nothing but the truth? Let the record reflect
    that each of the witnesses did take the oath.
        We will begin with Mr. Gene Aloise. Mr. Aloise, please
    begin.

     STATEMENT OF MR. GENE ALOISE, DIRECTOR, NATURAL RESOURCES AND
        ENVIRONMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE (GAO)

        Mr. Aloise. Mr. Chairman and Members of the Subcommittee, I
    am pleased to be here today to discuss DHS's plans to develop
    and test advanced portal monitors, known as ASPs, for use at
    the Nation's borders to prevent nuclear materials from being
    smuggled into the United States. My testimony today focuses on
    the results of DNDO's testing of the ASPs, including the July
    2009 field testing.
        According to DHS, the current system of radiation detection
    equipment, the PVTs, is effective and does not impede the flow
    of commerce. However, DHS wants to improve the capabilities of
    the existing equipment with ASPs. One of the major drawbacks of
    the ASP, as you know, is a substantially higher cost compared
    to the existing equipment. We estimated that the life cycle
    cost of each standard cargo version of the ASP to be about
    $822,000 compared to about $308,000 for the PVT standard cargo
    portal and that the total program cost would be about $2
    billion.
        Earlier this year I testified before this subcommittee on
    DNDO's 2008 round of ASP performance testing at the Nevada Test
    Site. That testing showed that the ASPs performed better than
    the PVTs in detecting certain nuclear materials and met DOE's
    threat guidance. However, the ASP's performance rapidly
    deteriorated once shielding was slightly increased. These test
    results showed what we reported in 2006, that any increase in
    detection of certain nuclear materials would be marginal.
        While the ASP may have met DOE's threat guidance for
    shielded nuclear material, the threat guidance is not a
    realistic approximation of how a terrorist might shield nuclear
    material to successfully smuggle it undetected across our
    borders.
        The latest round of field testing conducted in July
    revealed two critical performance problems with ASPs that the
    Chairman mentioned. First, the ASPs had an unacceptably high
    number of false positives for the detection of high-risk
    nuclear material. In other words, the ASP was seeing nuclear
    material that wasn't there and alarming. CBP officials told us
    that any alarm for this type of nuclear material is very
    disruptive to a port or border crossing and could effectively
    shut down operations until the source of the alarm is found.
    Furthermore, repeated false alarms for nuclear materials could
    have the undesired effect of causing CBP officers to doubt the
    reliability of the ASPs and be skeptical about the credibility
    of future alarms.
        The second critical failure of the ASPs noted in the July
    testing stemmed from a problem with the key component of the
    equipment which led an ASP to in essence shut down. Of great
    concern was the fact that the ASP did not alert the CBP
    official that it had shut down and was not scanning cargo. If
    this were not a controlled test, the CBP officer would have
    allowed the cargo to enter the United States thinking it had
    been scanned when it had not.
        DNDO's proposed solutions to these critical failures raise
    questions about whether the ASPs will provide any meaningful
    increase in the ability to detect certain nuclear materials.
    Specifically, to address the problem of false positives, DNDO
    is modifying the ASP to make it less sensitive to certain
    nuclear materials. While this may fix the problem of false
    alarms, it diminishes even further the ASP's ability to detect
    the nuclear material we are most concerned about.
        To address the second failure, DNDO plans to, among other
    things, install an indicator light on the ASP that will alert
    CBP officials that the ASP has a mission-critical failure. In
    our view, an indicator light is not the solution. The ASP must
    be stable and secure enough to avoid these shut-downs.
        Furthermore, DNDO has not completed efforts to improve the
    PVTs to detect high-risk material through energy windowing. CBP
    has repeatedly urged the completion of this research, because
    an improved PVT could be the more cost-effective way to improve
    detecting certain nuclear materials and have a similar
    performance to a working ASP.
        In closing, the concerns raised by the results of the July
    2009 field testing provide even greater reason for DNDO to
    implement our recommendations from our May 2009 report. In
    particular, our recommendation that DNDO assess whether the
    ASPs meet the criteria for significant increase in operational
    effectiveness based on a valid comparison with the PVT's full
    performance. This is especially relevant given that ASPs
    seemingly will no longer be as effective in detecting certain
    nuclear materials.
        Mr. Chairman, that concludes my remarks. Dr. Persons and I
    would be happy to respond to any questions you may have.
        [The prepared statement of Mr. Aloise follows:]
                       Prepared Statement of Gene Aloise

    Mr. Chairman and Members of the Subcommittee:

        I am pleased to be here today to discuss GAO's work on the
    Department of Homeland Security's (DHS) testing of advanced
    spectroscopic portal (ASP) radiation detection monitors. One mission of
    U.S. Customs and Border Protection (CBP), an agency within DHS,
    includes screening cargo and vehicles coming into this country for
    smuggled nuclear or radiological material that could be used in an
    improvised nuclear device or radiological dispersal device (a ``dirty
    bomb''). To screen cargo at ports of entry, CBP conducts primary
    inspections with radiation detection equipment called portal monitors--
    large stationary detectors through which cargo containers and vehicles
    pass as they enter the United States. When radiation is detected, CBP
    conducts secondary inspections using a second portal monitor to confirm
    the original alarm and a hand-held radioactive isotope identification
    device to identify the radiation's source and determine whether it
    constitutes a threat.
        The polyvinyl toluene (PVT) portal monitors CBP currently uses for
    this screening can detect radiation but cannot identify the type of
    material causing an alarm. As a result, the monitors' radiation alarms
    can be set off even by shipments of bananas, kitty litter, or granite
    tile because these materials contain small amounts of benign, naturally
    occurring radioactive material. To address the limitations of current-
    generation portal monitors, DHS's Domestic Nuclear Detection Office
    (DNDO) in 2005 began to develop and test ASPs, which are designed to
    both detect radiation and identify the source.\1\ DNDO hopes to use the
    new portal monitors to replace at least some PVTs currently used for
    primary screening, as well as PVTs and hand-held identification devices
    currently used for secondary screening.
    ---------------------------------------------------------------------------
        \1\ DNDO was established within DHS in 2005; its mission includes
    developing, testing, acquiring, and supporting the deployment of
    radiation detection equipment at U.S. ports of entry. CBP began
    deploying portal monitors in 2002, prior to DNDO's creation, under the
    radiation portal monitor project.
    ---------------------------------------------------------------------------
        Since 2006, we have been reporting on issues associated with the
    cost and performance of the ASPs and the lack of rigor in testing this
    equipment. For example, we found that tests DNDO conducted in early
    2007 used biased test methods that enhanced the apparent performance of
    ASPs and did not use critical CBP operating procedures that are
    fundamental to the performance of current hand-held radiation
    detectors.\2\ In addition, in 2008 we estimated the life cycle cost of
    each standard cargo version of the ASP (including deployment costs) to
    be about $822,000, compared with about $308,000 for the PVT standard
    cargo portal, and the total program cost for DNDO's latest plan for
    deploying radiation portal monitors--which relies on a combination of
    ASPs and PVTs and does not deploy radiation portal monitors at all
    border crossings--to be about $2 billion.\3\
    ---------------------------------------------------------------------------
        \2\ Combating Nuclear Smuggling: Additional Actions Needed to
    Ensure Adequate Testing of Next Generation Radiation Detection
    Equipment. GAO-07-1247T, (Washington, D.C.: Sept. 18, 2007).
        \3\ Combating Nuclear Smuggling: DHS's Program to Procure and
    Deploy Advanced Radiation Detection Portal Monitors Is Likely to Exceed
    the Department's Previous Cost Estimates. GAO-08-1108R, (Washington,
    D.C.: Sept. 22, 2008).
    ---------------------------------------------------------------------------
        Concerned about the performance and cost of the ASP monitors,
    Congress required the Secretary of Homeland Security to certify that
    the monitors will provide a ``significant increase in operational
    effectiveness'' before DNDO obligates funds for full-scale ASP
    procurement.\4\ In response, CBP, DNDO, and the DHS management
    directorate jointly issued criteria for determining whether the new
    technology provides a significant increase in operational
    effectiveness. The primary screening criteria require that the new
    portal monitors detect potential threats as well as or better than
    PVTs, show improved performance in detection of highly enriched uranium
    (HEU), and reduce by 80 percent the number of innocent alarms that are
    sent to secondary inspection. To meet the secondary screening criteria,
    the new portal monitors must reduce the probability of misidentifying
    special nuclear material (e.g., HEU and plutonium) and the average time
    to conduct secondary screenings.
    ---------------------------------------------------------------------------
        \4\ Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121
    Stat. 1844, 2069 (2007); Consolidated Security, Disaster Assistance,
    and Continuing Appropriations Act, 2009, Pub. L. No. 110-329, 121 Stat.
    3574, 3679 (2008); Department of Homeland Security Appropriations Act,
    2010, Pub. L. No. 111-83, 123 Stat. 2142, 2167 (2009).
    ---------------------------------------------------------------------------
        DNDO designed and coordinated a new series of tests, originally
    scheduled to run from April 2008 through September 2008, to determine
    whether the new portal monitors meet the certification criteria and are
    ready for deployment. Key phases of this round of testing include
    concurrent testing led by DNDO of the new and current equipment's
    ability to detect and identify threats and of ASPs' readiness to be
    integrated into operations for both primary and secondary screening at
    ports of entry; field validation testing led by CBP at four northern
    and southern border crossings and two seaports; and an independent
    evaluation, led by the DHS Science and Technology Directorate at one of
    the seaports, of the new portal monitors' effectiveness and
    suitability.
        In May 2009, we reported on the results of the then-current round
    of ASP testing.\5\ The findings from that report were based on
    completed tests and preliminary results available at the time. Testing
    on ASPs has continued since that report was issued. Today my testimony
    will (1) discuss the principal findings and recommendations from our
    May report and (2) update those findings based on the results of DNDO's
    July 2009 ASP field validation testing. The findings we are presenting
    today are based on our previous ASP reports and updated with
    information collected during interviews with DNDO and CBP officials. We
    also reviewed testing results in a report on the July 2009 tests from
    the ASP Field Validation Advisory Panel, a panel made up of officials
    from CBP, DNDO, and a national laboratory established to examine
    testing results and provide recommendations. On November 12, 2009, we
    briefed DHS, CBP, and DNDO officials on the findings of our updated
    work. During the briefing, CBP and DNDO officials provided oral
    comments and offered additional information and clarifications we
    included in this testimony as appropriate. Both our prior work and our
    updated work were conducted in accordance with generally accepted
    government auditing standards. Those standards require that we plan and
    perform the audit to obtain sufficient, appropriate evidence to produce
    a reasonable basis for our findings and conclusions based on our audit
    objectives. We believe that the evidence obtained provides a reasonable
    basis for our statement today.
    ---------------------------------------------------------------------------
        \5\ Combating Nuclear Smuggling: DHS Improved Testing of Advanced
    Radiation Detection Portal Monitors, but Preliminary Results Show
    Limits of the New Technology, GAO-09-655 (Washington, D.C.: May 21,
    2009).

    Improved Testing Rigor Discussed in Our May 2009 Report Demonstrates
                        Limitations of ASPs

        Our May 2009 report on the then-current round of ASP testing found
    that DHS increased the rigor of ASP testing over that of previous
    tests, and that a particular area of improvement was in the performance
    testing at the Nevada Test Site, where DNDO compared the capability of
    ASP and current-generation equipment to detect and identify nuclear and
    radiological materials. For example, unlike in prior tests, the plan
    for the 2008 performance test stipulated that the contractors who
    developed the equipment would not be involved in test execution. This
    improvement addressed concerns we previously raised about the potential
    for bias and provided increased credibility to the results.
    Nevertheless, based on the following factors, in our report we
    questioned whether the benefits of the new portal monitors justify the
    high cost:

            [bullet]  The DHS criteria for a significant increase in
            operational effectiveness. Our chief concern with the criteria
            is that they require only a marginal improvement over current-
            generation portal monitors in the detection of certain weapons-
            usable nuclear materials during primary screening. DNDO
            considers detection of such materials to be a key limitation of
            current-generation portal monitors. The marginal improvement
            required of ASPs to meet the DHS criteria is problematic
            because the detection threshold for the current-generation
            portal monitors does not specify a level of radiation shielding
            that smugglers could realistically use. Officials from the
            Department of Energy (DOE), which designed the threat guidance
            DHS used to set the detection threshold, and national
            laboratory officials told us that the current threshold is
            based not on an analysis of the capabilities of potential
            smugglers to take effective shielding measures but rather on
            the limited sensitivity of PVTs to detect anything more than
            certain lightly shielded nuclear materials. DNDO officials
            acknowledge that both the new and current-generation portal
            monitors are capable of detecting certain nuclear materials
            only when unshielded or lightly shielded. The marginal
            improvement in detection of such materials required of ASPs is
            particularly notable given that DNDO has not completed efforts
            to fine-tune PVTs' software using a technique called ``energy
            windowing'' that could improve the PVTs' sensitivity to nuclear
            materials. DNDO officials expect they can achieve small
            improvements in sensitivity through energy windowing, but DNDO
            has not yet completed efforts to fine-tune the PVTs' software.
            In contrast to the marginal improvement required in detection
            of certain nuclear materials, the primary screening requirement
            to reduce the rate of innocent alarms by 80 percent could
            result in hundreds of fewer secondary screenings per day,
            thereby reducing CBP's workload. In addition, the secondary
            screening criteria, which require ASPs to reduce the
            probability of misidentifying special nuclear material by one-
            half, address the limitations of relatively small hand-held
            devices in consistently locating and identifying potential
            threats in large cargo containers.

            [bullet]  Results of performance testing and field validation.
            The results of performance tests that DNDO presented to us were
            mixed, particularly in the ASPs' capability to detect certain
            shielded nuclear materials during primary screening. The
            results of performance testing at the Nevada Test Site showed
            that the new portal monitors detected certain nuclear materials
            better than PVTs when shielding approximated DOE threat
            guidance, which is based on light shielding. In contrast,
            differences in system performance were less notable when
            shielding was slightly increased or decreased: both the PVTs
            and ASPs were frequently able to detect certain nuclear
            materials when shielding was below threat guidance, and both
            systems had difficulty detecting such materials when shielding
            was somewhat greater than threat guidance. With regard to
            secondary screening, ASPs performed better than hand-held
            devices in identification of threats when masked by naturally
            occurring radioactive material. However, the differences in the
            ability to identify certain shielded nuclear materials depended
            on the level of shielding, with increasing levels appearing to
            reduce any ASP advantages over the hand-held identification
            devices. Other phases of testing uncovered multiple problems in
            meeting requirements for successfully integrating the new
            technology into operations at ports of entry. Of the two ASP
            contractors participating in the current round of testing, one
            has fallen behind due to severe problems encountered during
            testing of ASPs' readiness to be integrated into operations at
            ports of entry (``integration testing''); the problems may
            require that the vendor redo previous test phases to be
            considered for certification. The other vendor's system
            completed integration testing, but CBP suspended field
            validation testing in January 2009 after two weeks because of
            serious performance problems resulting in an overall increase
            in the number of referrals for secondary screening compared
            with existing equipment.

            [bullet]  DNDO's plans for computer simulations. As of May
            2009, DNDO did not plan to complete injection studies--computer
            simulations for testing the response of ASPs and PVTs to
            simulated threat objects concealed in cargo containers--prior
            to the Secretary of Homeland Security's decision on
            certification even though delays to the ASP test schedule have
            allowed more time to conduct the studies. According to DNDO
            officials, injection studies address the inability of
            performance testing to replicate the wide variety of cargo
            coming into the United States and the inability to place
            special nuclear material and other threat objects in cargo
            during field validation. DNDO had earlier indicated that
            injection studies could provide information comparing the
            performance of the two systems as part of the certification
            process for both primary and secondary screening. However, DNDO
            subsequently decided that performance testing would provide
            sufficient information to support a decision on ASP
            certification. DNDO officials said they would instead use
            injection studies to support effective deployment of the new
            portal monitors.

            [bullet]  Lack of an updated cost-benefit analysis. DNDO had
            not updated its cost-benefit analysis to take into account the
            results of ASP testing. An updated analysis that takes into
            account the testing results, including injection studies, might
            show that DNDO's plan to replace existing equipment with ASPs
            is not justified, particularly given the marginal improvement
            in detection of certain nuclear materials required of ASPs and
            the potential to improve the current-generation portal
            monitors' sensitivity to nuclear materials, most likely at a
            lower cost. DNDO officials said they were updating the ASP
            cost-benefit analysis and planned to complete it prior to a
            decision on certification by the Secretary of Homeland
            Security.

        Our May report recommended that the Secretary of Homeland Security
    direct DNDO to (1) assess whether ASPs meet the criteria for a
    significant increase in operational effectiveness based on a valid
    comparison with PVTs' full performance potential and (2) revise the
    schedule for ASP testing and certification to allow sufficient time for
    review and analysis of results from the final phases of testing and
    completion of all tests, including injection studies. We further
    recommended that, if ASPs are certified, the Secretary direct DNDO to
    develop an initial deployment plan that allows CBP to uncover and
    resolve any additional problems not identified through testing before
    proceeding to full-scale deployment. DHS agreed to a phased deployment
    that should allow time to uncover ASP problems but disagreed with GAO's
    other recommendations, which we continue to believe remain valid.

    Results from July 2009 Testing Raise Continuing Issues

        The results of DNDO's most recent round of field validation
    testing, which it undertook in July 2009, after our May report was
    released, raise new issues. In July 2009, DNDO resumed the field
    testing of ASPs at four CBP ports of entry that it initiated in January
    2009 but suspended because of serious performance problems. However,
    the July tests also revealed ASP performance problems, including two
    critical performance deficiencies. First, the ASP monitors had an
    unacceptably high number of false positive alarms for the detection of
    certain high-risk nuclear materials. According to CBP officials, these
    false alarms are very disruptive in a port environment in that any
    alarm for this type of nuclear material would cause CBP to take
    enhanced security precautions because such materials (1) could be used
    in producing an improvised nuclear device and (2) are rarely part of
    legitimate or routine cargo. Furthermore, once receiving an alarm for
    this type of nuclear material, CBP officers are required to conduct a
    thorough secondary inspection to assure themselves that no nuclear
    materials are present before permitting the cargo to enter the country.
    Repeated false alarms for nuclear materials are also causes for concern
    because such alarms could eventually have the effect of causing CBP
    officers to doubt the reliability of the ASP and be skeptical about the
    credibility of future alarms.
        Secondly, during the July testing the ASP experienced a ``critical
    failure,'' which stemmed from a problem with a key component of the ASP
    and caused the ASP to shut down. Importantly, during this critical
    failure, the ASP did not alert the CBP officer that it had shut down
    and was no longer scanning cargo. As a result, were this not in a
    controlled testing environment, the CBP officer would have permitted
    the cargo to enter the country thinking the cargo had been scanned,
    when it had not. According to CBP officials, resolving this issue is
    important in order to assure the stability and security of the ASP.
        In addition to these key performance problems, the ASP was not able
    to reduce referrals to secondary inspection by 80 percent as required
    by the DHS criteria for a significant increase in operational
    effectiveness. According to the report from the ASP Field Validation
    Advisory Panel, a panel made up of officials from CBP, DNDO, and a
    national laboratory, the ASP was able to reduce referrals to secondary
    inspection by about 69 percent rather than the 80 percent as required
    by the DHS criteria.
        While the performance of the ASP during the July field validation
    testing raises issues about its potential readiness for deployment,
    DNDO's proposed solutions to address these performance problems raise
    additional questions about whether this equipment will provide any
    meaningful increase in the ability to detect certain nuclear materials.
    Specifically, to address the problem of false positive alarms
    indicating the presence of certain nuclear materials, according to DNDO
    officials, DNDO has modified the ASP to make this equipment less
    sensitive to these nuclear materials. While this may address the issue
    of false positive alarms, it will also diminish the ASP capability of
    detecting a key high-risk nuclear material. Since the ASP modification,
    DNDO conducted computer simulations using a vendor-provided system
    called a ``replay tool'' to examine the effect of the modification on
    the ASP's performance. According to DNDO officials, the replay tool
    demonstrated that the modified ASP will still be able to detect certain
    nuclear materials better than the PVT. However, at this point, DNDO
    does not plan to retest the ASP at the Nevada Test Site where it can
    examine the effects of these modifications using actual nuclear
    materials. As we reported earlier this year, the results of the testing
    at the Nevada Test Site demonstrated that the ASPs represented a
    marginal improvement in detecting certain nuclear materials. By
    reducing the sensitivity to these materials and not retesting the
    modified ASPs against actual nuclear materials, it is uncertain exactly
    what improvement in detecting certain nuclear materials these costly
    portal monitors are providing.
        While DNDO is reducing the sensitivity of ASPs to certain nuclear
    materials, it has yet to complete efforts to improve the PVT's ability
    to detect these same materials through energy windowing. For several
    years, CBP officials have repeatedly urged DNDO officials to complete
    this research. However, it was not apparent from our discussions with
    DNDO officials if this effort is making meaningful progress with the
    development of energy windowing or when it will be completed.
    Furthermore, CBP officials stated that, depending on the outcome of
    this research, energy windowing could be the more cost effective way to
    improve detection of certain nuclear materials. In our view, ASPs being
    modified to diminish their capabilities to detect certain nuclear
    materials raises questions about whether energy windowing might be able
    to achieve a similar level of performance against these same materials
    from the PVTs that are already in place.
        Beyond reducing the sensitivity of ASPs to certain nuclear
    materials, DNDO also plans to address the issue of critical failures
    by, among other things, installing an indicator light on the ASP that
    will alert CBP officers that the ASP has experienced a mission-critical
    failure and is no longer scanning cargo. While this should address the
    issue of CBP officers not knowing that the ASP has suffered a critical
    failure, CBP officials stressed to us the need for the ASP to be stable
    and secure enough to avoid these shutdowns.
        In closing, the issues raised by the results of the July 2009 field
    validation tests provide even greater reason for DNDO to address
    recommendations from our May 2009 report. In particular, we reiterate
    the importance of our prior recommendation for DNDO to assess whether
    ASPs meet the criteria for a significant increase in operational
    effectiveness based on a valid comparison with PVTs' full performance
    potential, given that the ASPs will no longer be as effective in
    detecting certain nuclear materials.
        Mr. Chairman, this completes my prepared statement. I would be
    happy to respond to any questions that you or other Members of the
    Subcommittee may have at this time.

    Staff Acknowledgments

        Dr. Timothy Persons (Chief Scientist), Ned Woodward (Assistant
    Director), Joseph Cook, and Kevin Tarmann made key contributions to
    this testimony. Kendall Childers, Karen Keegan, Carol Kolarik, Jonathan
    Kucskar, Omari Norman, Alison O'Neill, and Rebecca Shea also made
    important contributions.

                           Biography for Gene Aloise
        Gene Aloise is a Director in the Natural Resources and Environment
    team at GAO. He is GAO's recognized expert in international nuclear
    nonproliferation and safety issues and completed training on these
    subjects at the University of Virginia and Princeton University. His
    work for GAO has taken him to some of Russia's closed nuclear cities
    and the Chernobyl reactor in Ukraine as well as numerous nuclear
    facilities around the world and in the United States. Mr. Aloise has
    had years of experience developing, leading, and managing GAO domestic
    and international engagements. His diverse experience includes
    assignments with congressional committees as well as various offices
    within GAO. He has received numerous awards for his leadership and
    expertise including GAO's Meritorious Service and Distinguished Service
    Awards. Mr. Aloise received his Bachelor's degree in political science/
    economics from Rowan University and holds a Master of Public
    Administration from Temple University. Mr. Aloise is also a graduate of
    the Senior Executive Fellows Program, John F. Kennedy School of
    Government, Harvard University.

      STATEMENT OF DR. TIMOTHY M. PERSONS, CHIEF SCIENTIST, U.S.
                 GOVERNMENT ACCOUNTABILITY OFFICE (GAO)

        Chairman Miller. Thank you, Mr. Aloise. Dr. Persons. I
    think I said Parsons earlier. Persons. You are here to answer
    questions but the testimony of Mr. Aloise is your testimony as
    well?
        Dr. Persons. That is correct.

                        Biography for Timothy M. Persons
        Dr. Timothy M. Persons was appointed the Chief Scientist of the
    United States Government Accountability Office (GAO--the investigative
    arm of the U.S. Congress) in July of 2008. In this role he is an expert
    advisor and chief consultant to the GAO, Congress, and other federal
    agencies and government programs on cutting-edge science and technology
    (S&T), key highly-specialized national and international systems,
    engineering policies, best practices, and original research studies in
    the fields of engineering, computer, and the physical and biological
    sciences to ensure efficient, effective, and economical use of science
    and technology in government programs. The Chief Scientist also works
    with GAO's Chief Technologist to lead the production of Technology
    Assessments for the U.S. Congress.
        Prior to joining GAO, Dr. Persons was the Technical Director of the
    Intelligence Advanced Research Projects Activity (IARPA) from November
    2002 to July 2008. IARPA's mission is to invest in high-risk/high-
    payoff research with potential to revolutionize the business of
    intelligence collection, processing, analysis, and dissemination. From
    July 2001 to November of 2002, he served as the Technical Director for
    the National Security Agency's (NSA) Human Interface Security Group
    which researches, designs, and tests next-generation biometric
    identification and authentication systems. He has also served as a
    radiation physicist with the University of North Carolina at Chapel
    Hill.
        Dr. Persons is a 2007 Director of National Intelligence S&T Fellow
    whose research focuses on computational imaging systems. He was also
    selected as the James Madison University (JMU) Physics Alumnus of 2007.
    He received his B.Sc. (Physics) from JMU, a M.Sc. (Nuclear Physics)
    from Emory University, and a M.Sc. (Computer Science) and Ph.D.
    (Biomedical Engineering) degrees from Wake Forest University. He is a
    senior member of the Institute for Electrical and Electronic Engineers,
    Association for Computing Machinery, and the Sigma Xi research honor
    society and has authored an array of journal, conference, and technical
    articles. He is also a Ruling Elder in the Presbyterian Church in
    America and lives happily in Maryland with his wife Gena and their two
    children.

        Chairman Miller. Okay. Mr. Owen.

        STATEMENT OF MR. TODD C. OWEN, ACTING DEPUTY ASSISTANT
      COMMISSIONER, OFFICE OF FIELD OPERATIONS, U.S. CUSTOMS AND
           BORDER PROTECTION, DEPARTMENT OF HOMELAND SECURITY

        Mr. Owen. Chairman Miller, Ranking Member Broun,
    distinguished Members of the Subcommittee, I am honored to be
    here this afternoon to provide an update on the role that U.S.
    Customs and Border Protection plays in protecting our nation
    from the illicit introduction of radiological or nuclear
    materials in cargo containers, and on the future role that
    Advanced Spectroscopic Portal technology would have on CBP
    operations.
        I would like to begin by expressing my gratitude to
    Congress for its continued support toward CBP initiatives.
    Among the numerous priorities that were recognized in the
    American Recovery and Reinvestment Act of 2009, Congress
    provided CBP with $100 million of stimulus funding toward
    upgrading and adding non-intrusive inspection equipment. This
    funding will allow CBP to upgrade and expand its successful NII
    program and more effectively inspect containers and vehicles
    crossing our border, allowing them to enter our country and its
    commerce in a safe and prompt manner.
        Everyday over 57,000 maritime containers, truck trailers
    and rail cars cross our border. CBP uses a multi-layered
    approach to ensure the integrity of supply chains from points
    of stuffing through arrival at U.S. ports of entry. This multi-
    layer defense is built upon interrelated initiatives which
    include the 24-Hour Rule and the Trade Act of 2002, the
    Automated Targeting System, Non-Intrusive Inspection (NII)
    equipment and radiation portal monitors, the Container Security
    Initiative and the Customs Trade Partnership Against Terrorism,
    C-TPAT program. These complementary layers enhance security and
    protect our nation.
        Prior to 9-11, not a single radiation portal monitor and
    only 64 large-scale NII systems were deployed to our nation's
    borders. By October of 2002, CBP had deployed the first RPM to
    the Ambassador Bridge in Detroit, and today CBP has just under
    1,400 RPMs and 232 large-scale NII systems deployed nationwide,
    and that is an increase of almost 200 more RPMs and five
    additional large-scale NII systems since I last testified
    before this subcommittee in June.
        NII technology allows the officers to detect possible
    anomalies, anomalies which may indicate the presence of weapons
    of mass effect or some other contraband. In Fiscal Year 2009,
    CBP conducted over 5.2 million examinations using NII
    technology, allowing CBP to meet our twin goals of enhance
    security and trade facilitation.
        In addition to the significant strides made in the area of
    NII equipment, the CBP also continues to deploy first-
    generation radiation portal monitors to our nation's ports of
    entry. Since last testifying before this subcommittee, CBP and
    DNDO have now completed deployments of RPMs along the northern
    border and now scan 100 percent of trucks and passenger
    vehicles arriving from Canada, 100 percent arriving from Mexico
    and 98 percent of arriving sea containers for the presence of
    nuclear radiological materials. Since the first RPM was
    deployed in 2002, CBP officers have scanned over 404 million
    conveyances for the presence of radiation and 2.2 million
    radiological alarms have been successfully adjudicated with
    minimal or no impact on the flow of legitimate trade and
    travel. CBP continues to closely coordinate with key
    stakeholders to ensure the impacts of this activity causes
    minimal disruption on port operations.
        The first generation RPM systems, although very sensitive,
    do have limitations. While they alert the CBP officers to the
    presence of radiation, a secondary exam is necessary to
    positively identify the specific isotope causing the alert. In
    the event that a CBP officer is unable to positively resolve
    the alarm, scientific reach-back is available 24 hours a day,
    seven days a week.
        The ASP is expected to enhance our detection capability
    while significantly reducing the number or secondary
    examinations. This is due to its ability to distinguish between
    actual threats and natural or medical radiation sources that
    are not security threats. CBP has worked closely with the DNDO
    in the development and operational testing of ASP, has provided
    DNDO with functional requirements and has been actively engaged
    in every step of systems evaluation. CBP will continue to work
    with DNDO, and with the DHS Science and Technology Directorate,
    S&T, toward Secretarial Certification, and is also working with
    DHS to ensure that any future acquisition and deployment
    decisions are consistent with DHS priorities. The decision to
    purchase and deploy ASPs in the operational arena will be based
    on CBPs mission needs and operational requirements, a
    comprehensive cost benefit analysis to include a full
    understanding of the maintenance and operational cost and
    analysis of alternatives and other considerations.
        Mr. Chairman and Members of the Subcommittee, today I have
    addressed CBP's commitment to investing in new and emerging
    detection technology aimed at enhancing cargo security. We must
    continue to maintain our tactical edge by integrating new
    technology into our ports of entry.
        Thank you again for the opportunity to testify. I would be
    happy to answer any questions.
        [The prepared statement of Mr. Owen follows:]
                       Prepared Statement of Todd C. Owen
        Chairman Miller, Ranking Member Broun, esteemed Members of the
    Subcommittee, it is a privilege and an honor to appear before you today
    to discuss the work of U.S. Customs and Border Protection (CBP),
    particularly the detection of radioactive and nuclear material in cargo
    containers and the potential future role that the Advanced
    Spectroscopic Portal (ASP) program will have on our operations. CBP is
    responsible for ensuring the security of cargo entering the United
    States at our borders and facilitating the flow of legitimate trade and
    travel. As part of this process, over 98 percent of all arriving
    maritime containerized cargo is currently scanned for radiation through
    radiation portal monitors.
        I want to begin by expressing my gratitude to Congress for its
    continued support for the mission and people of CBP. It is clear that
    Congress is committed to providing CBP the resources we need in order
    to increase and maintain the security of our borders. We appreciate
    your efforts and assistance.
        CBP is the largest uniformed federal law enforcement agency in the
    country. We station over 21,000 CBP officers at access points around
    the Nation, including air, land, and sea ports. We have deployed over
    20,000 Border Patrol agents between the ports of entry. These forces
    are supplemented with 1,266 Air and Marine agents, 2,392 agricultural
    specialists, and other professionals.
        CBP's responsibilities include stemming the illegal flow of drugs,
    contraband and people, protecting our agricultural and economic
    interests from harmful pests and diseases, protecting American
    businesses from theft of their intellectual property, enforcing textile
    agreements, tracking import safety violations, regulating and
    facilitating international trade, collecting import duties,
    facilitating legitimate travel, and enforcing U.S. trade laws. CBP
    facilitates lawful immigration, welcoming visitors and new immigrants,
    while making certain those entering this country are indeed admissible
    and taking appropriate action when an individual fears being persecuted
    or tortured if returned to their home country. At the same time, our
    employees maintain a vigilant watch for terrorist threats. In FY 2009,
    CBP processed more than 361 million pedestrians and passengers, 109
    million conveyances, 25.8 million trade entry summaries, examined 5.2
    million sea, rail, and truck containers, performed over 26.8 million
    agriculture inspections, apprehended over 732,000 illegal aliens
    between our ports of entry, encountered over 224,000 inadmissible
    aliens at the ports of entry, and seized more than 2.8 million pounds
    of illegal drugs.
        We must perform our important security and trade enforcement work
    without stifling the flow of legitimate trade and travel that is so
    important to our nation's economy. These are our twin goals: border
    security and facilitation of legitimate trade and travel.

    CBP OVERVIEW

        I am pleased to appear before the Subcommittee today to highlight
    the continued progress on radiation detection technology. I would also
    like to take this opportunity to bring attention to CBP's holistic
    cargo security programs that are applied to all environments. CBP has
    made tremendous progress toward securing the supply chains that bring
    goods into the United States from around the world, and preventing
    their potential use by terrorist groups, by: using cutting-edge
    technology to increase the ability of front-line CBP Officers to
    successfully detect and interdict illicit importations of nuclear and
    radiological materials; moving resources where they are most needed;
    integrating all CBP offices; sharing information, including actionable
    intelligence, across all aspects of CBP; and utilizing a multi-layered
    approach to ensure the integrity of the supply chain from the point of
    stuffing, through arrival at a U.S. port of entry. This multi-layered
    approach includes the following comprehensive cargo security programs
    that are applied to all modes of transportation:

            [bullet]  Advance Information

                      24-Hour Rule

                      Automated Targeting Systems

                      Importer Security Filing (commonly known as
                    ``10+2'')

            [bullet]  The Customs Trade Partnership Against Terrorism (C-
            TPAT) and Free and Secure Trade (FAST)

            [bullet]  Container Security Initiative (CSI)

            [bullet]  Secure Freight Initiative

            [bullet]  Use of non-intrusive inspection technology and
            mandatory exams for all high risk shipments.

        I will discuss each one of these layers in greater detail with
    particular focus on our radiation and nuclear detection capabilities.

    ADVANCE INFORMATION

        CBP requires advanced electronic cargo information, as mandated in
    the Trade Act of 2002, for all inbound shipments in all modes of
    transportation. This advanced cargo information is evaluated using the
    Automated Targeting System (ATS) before the cargo arrives in the United
    States.
        ATS provides decision support functionality for CBP officers
    working in Advanced Targeting Units at our ports of entry and Container
    Security Initiative ports abroad. The system provides uniform review of
    cargo shipments for identification of the highest threat shipments, and
    presents data in a comprehensive, flexible format to address specific
    intelligence threats and trends. Through rules, the ATS alerts the user
    to data that meets or exceeds certain pre-defined criteria. National
    targeting rule sets have been implemented in ATS to provide threshold
    targeting for national security risks for all modes of transportation:
    sea, truck, rail, and air. The DHS Science and Technology Directorate
    is exploring additional methodologies for conducting risk assessment.
        The Importer Security Filing interim final rule, also more commonly
    known as ``10+2,'' went into effect earlier this year and has already
    yielded promising results. This program will increasingly provide CBP
    timely information about cargo shipments that will enhance our ability
    to detect and interdict high risk shipments. Comments on aspects of
    this rule were accepted until June 1, 2009, and implementation using
    informed compliance will continue until January of next year. Shipments
    determined by CBP to be high-risk are examined either overseas as part
    of our Container Security Initiative, or upon arrival at a U.S. port.

    CUSTOMS TRADE PARTNERSHIP AGAINST TERRORISM (C-TPAT)

        CBP works with the trade community through the Customs Trade
    Partnership Against Terrorism (C-TPAT) to better secure goods moving
    through the international supply chain. C-TPAT has enabled CBP to
    leverage supply chain security throughout international locations where
    CBP has no regulatory reach. Under the C-TPAT program, a prospective
    member submits basic company information and a security profile via an
    Internet-based portal system. CBP conducts records checks on the
    company in its law enforcement and trade databases and evaluates the
    security profile, ensuring the company meets the security criteria for
    its particular business sector. Members who pass initial vetting are
    certified into the program. Using a risk-based approach, CBP Supply
    Chain Security Specialists conduct on-site visits of foreign and
    domestic facilities to confirm that the security practices are in place
    and operational.
        C-TPAT is a voluntary public-private partnership program wherein
    CBP works with the trade community in adopting tighter security
    measures throughout their international supply chain. In return for
    making these enhancements members are afforded benefits to include
    reduced exams, dedicated cargo lanes (FAST), Stratified Exam, an
    assigned Supply Chain Security Specialist (SCSS). Potential members
    initially complete an online application. The SCSS then conducts
    internal record checks and evaluates the security profile provided.
    Upon completion of vetting the company then becomes a Certified member
    who will then be physically validated within a year of certification.
        In 2009, CBP continued to expand and strengthen the C-TPAT program
    and ensure that certified member companies are securing their goods
    moving through the international supply chain to the United States.
    Teams of Supply Chain Security Specialists conducted validations and
    re-validations of C-TPAT members' supply chains to ensure that security
    protocols are reliable, accurate, and effective.
        As C-TPAT has evolved, we have steadily increased both the rigor of
    the program and program membership. CBP has strengthened the C-TPAT
    program by clearly defining the minimum security requirements for all
    categories of participants wishing to take part in the program. As of
    Nov. 6, 2009, there were 9,509 companies certified into the C-TPAT
    program. CBP's goal is to validate all partners within one year of
    certification, re-validate all companies not less than once every three
    years, and re-validate all U.S./Mexico highway carriers on an annual
    basis, due to the risks of compromise of trailers associated with the
    Southern Border Highway Carrier sector of C-TPAT.
        C-TPAT's 9,509 Certified Partners include 4,330 importers, 2,583
    carriers, 821 brokers, 784 consolidators/third party logistic
    providers, 56 Marine Port Authority and Terminal Operators and 935
    foreign manufacturers. C-TPAT has conducted 13,246 on-site validations
    of manufacturing and logistics facilities in 90 countries. 301 C-TPAT
    importer partners have been designated Tier 3, meaning they have
    exceeded the minimum security criteria and have been granted the
    highest level of program benefits.

    CONTAINER SECURITY INITIATIVE

        Because of the sheer volume of sea container traffic, containerized
    shipping is uniquely vulnerable to terrorist exploitation. To prevent
    terrorists and their weapons from entering the United States, CBP has
    also partnered with foreign governments through our Container Security
    Initiative (CSI). CSI, which is the first program of its kind, was
    announced in January 2002 and is currently operational in 58 foreign
    seaports--covering more than 80 percent of the maritime containerized
    cargo shipped to the United States. The program enables CBP to identify
    and inspect high-risk cargo containers at foreign ports before they are
    shipped to our seaports and pose a threat to the United States and to
    global trade. CSI stations multidisciplinary teams of CBP officers to
    work with host country counterparts to identify and examine containers
    that are determined to pose the highest risk for terrorist activity.
        CBP officers stationed at foreign CSI ports review 100 percent of
    the manifests originating and/or transiting those foreign ports for
    containers that are destined for the United States. In locations where
    the tremendous volume of bills prevents the CSI team at the port itself
    from performing 100 percent review, or during port shutdowns, CSI
    targeters at the National Targeting Center provide additional support
    to ensure that 100 percent review is accomplished. Utilizing the
    overseas CSI team and the CSI targeters at our National Targeting
    Center, CBP is able to achieve 100 percent manifest review for the CSI
    program. In FY 2009, CBP officers stationed at CSI ports reviewed over
    nine million bills of lading and conducted over 56,000 exams in
    conjunction with their host country counterparts.

    SECURE FREIGHT INITIATIVE

        The Secure Freight Initiative (SFI) is an effort to build upon
    existing port security measures by enhancing the U.S. Government's
    ability to scan containers for nuclear and radiological materials in
    seaports worldwide and to better assess the risk of inbound containers.
    SFI provides carriers of maritime containerized cargo greater
    confidence in the security of the shipment they are transporting, and
    increases the likelihood of an uninterrupted and secure flow of
    commerce. This initiative is the culmination of our work with other
    U.S. Government agencies, foreign governments, the trade community, and
    vendors of leading-edge technology.
        CBP will prioritize future deployments of scanning systems to
    locations of strategic importance by identifying seaports where non-
    intrusive imaging and radiation detection data would be most practical
    and effective in deterring the movement of weapons of mass destruction
    via containerized cargo. The additional scan data provided by SFI will
    enhance DHS' risk-based and layered approach to securing maritime
    containerized cargo. We will continue to work with Congress to enhance
    the safety of our nation's ports and the security of incoming cargo.

    NON INTRUSIVE INSPECTION/RADIATION DETECTION TECHNOLOGY

        The deployment of imaging systems and radiation detection equipment
    has contributed tremendously to CBP's progress in ensuring that supply
    chains bringing goods into the United States from around the world are
    secure against exploitation by terrorist groups. Non-Intrusive
    Inspection (NII) technology serves as a force multiplier that allows
    officers to detect possible anomalies between the contents of a
    container and the manifest. CBP relies heavily on the use of NII, as it
    allows us to work smarter and more efficiently in recognizing potential
    threats.
        Prior to 9/11, not a single Radiation Portal Monitor (RPM), and
    only 64 large-scale NII systems were deployed to our nation's borders.
    By October 2002, CBP had deployed the first RPM at the Ambassador
    Bridge in Detroit. Today, CBP uses RPMs to scan 99 percent of all cargo
    arriving in the U.S. by land and sea. CBP, in partnership with the DHS
    Domestic Nuclear Detection Office (DNDO) and Pacific Northwest National
    Laboratory (PNNL), has deployed 473 RPMs at northern border land ports
    of entry; 385 RPMs at southern border land ports of entry; 433 RPMs at
    seaports; and 55 RPMs at mail facilities. Currently, CBP has 232 large-
    scale NII systems deployed and 5,816 small-scale NII units.
    Additionally, CBP has deployed 1,515 Radiation Isotope Identifier
    Devices (RIIDs) and 19,365 Personal Radiation Detectors (PRDs). These
    devices allow CBP to examine 100 percent of all identified high-risk
    cargo. To date, CBP has used the deployed systems to conduct over 38
    million examinations, resulting in over 8,300 narcotic seizures, with a
    total weight of over 2.6 million pounds, and over $28.6 million in
    undeclared currency seizures. Since RPM program inception in 2002, CBP
    has scanned over 404 million conveyances for radiological contraband
    resulting in over 2.2 million alarms. CBP's Laboratories and Scientific
    Services spectroscopy group at the National Targeting Center has
    responded to some 21,599 requests from the field for technical
    assistance in resolving alarms. To date 100 percent of all alarms have
    been successfully adjudicated as innocent.
        CBP is pleased to report that the final installation of RPMs along
    our shared northern border was commissioned on Oct. 29 at the Trout
    River, N.Y., port of entry. This milestone represents another critical
    step in the Department's efforts to strengthen the interconnected U.S.
    border security network by deploying technology and personnel to key
    border locations to meet modern-day security needs. This recent
    milestone provides CBP with the ability to utilize radiation detection
    technologies to scan 100 percent of trucks and personally owned
    vehicles arriving through both northern and southern border ports, and
    98 percent of arriving sea containers. In addition, CBP officers now
    use hand-held radiation identification devices to scan 100 percent of
    private aircraft arriving in the U.S. from foreign destinations.
        The first generation RPM systems, although very sensitive, do have
    limitations. While they alert CBP officers to the presence of
    radiation, a secondary exam is necessary to identify the location and
    specific isotope causing the alert. In the event that a CBP officer is
    unable to positively resolve the alert, scientific reach back is
    available on a 24/7 basis through the National Targeting Center and
    CBP's Laboratory & Scientific Services Division located in the northern
    Virginia area.
        Understanding these limitations and the need for more precise
    radiological detection architecture, DNDO was created in 2005 to focus
    on radiological and nuclear threats and develop new technologies that
    will improve the Nation's ability to detect and identify radiological
    and nuclear weapons and material. One of these new technologies is the
    next generation RPM, or the Advanced Spectroscopic Portal (ASP).
        The ASP is able to distinguish between actual threats and natural
    or medical radiation sources that are not security threats. In doing
    so, the ASP would enhance our detection capability, while significantly
    reducing the burden of responding to the numerous benign alarms that
    are mostly generated by everyday products. This would allow CBP to
    focus our staffing and resources on high-risk shipments and other
    border security initiatives.

    CBP COORDINATION WITH DNDO

        In our collaboration with DNDO, CBP brings knowledge of how our
    ports work, of the needs of our front-line officers, and of the
    operational requirements for new technologies that must work
    consistently in a broad array of environments. Additionally, CBP is
    attuned to critical factors such as throughput and capacity as we seek
    to maintain an appropriate balance between security and the
    facilitation of cross-border travel and trade.
        CBP has worked closely with DNDO in the developmental and
    operational testing of the ASP. A complete, independent operational
    testing and evaluation will be conducted by the DHS Science and
    Technology Directorate's Test and Evaluation and Standards Division,
    along with the Director of Operational Test and Evaluation, when the
    system completes the current course of testing. CBP's objective for
    operational testing is to ensure that systems are suitable and
    effective in our operational environments. CBP provided DNDO with
    functional requirements for the ASP and has been actively engaged in
    every step of testing, including performance testing at the Nevada Test
    Site and integration testing currently ongoing at a mock port of entry
    at the Pacific Northwest National Laboratory.
        During CBP-led field validation and integration testing, CBP has
    been working closely with DNDO to assess each ASP system's performance
    as an integrated unit, including reach back capability and ancillary
    equipment such as traffic lights and automated gate arms that are
    essential to maintaining positive control of vehicles at our ports of
    entry. In addition, CBP works with DNDO to assess and categorize each
    system's issues to ascertain their technological impact on performance
    and their operational impact on front-line CBP officers--the users of
    the system.
        CBP will continue to work with DNDO toward certification by the
    Secretary, which is dependent on demonstrating a ``significant increase
    in operational effectiveness'' over existing first-generation radiation
    detection systems. At such time, the discussion could then turn to
    potential acquisition and deployment of ASP systems. Any decision to
    purchase and deploy ASPs in the operational arena will be based on
    mission needs, operational requirements, and a full understanding of
    maintenance and operational costs, to include a comprehensive cost-
    benefit analysis and an analysis of alternatives.

    CONCLUSION

        Technology plays an enormous role in securing the supply chain.
    Security technology is continuously evolving, not only in terms of
    capability but also in terms of compatibility, standardization, and
    integration with information systems. It is important to note that
    there is no single technological solution to supply chain security. As
    technology matures, it must be evaluated, and adjustments to
    operational plans must be made. Priority should be given to effective
    security solutions that complement and improve the business processes
    already in place, and which build a foundation for secure 21st century
    global trade.
        Mr. Chairman, Members of the Subcommittee, thank you again for this
    opportunity to testify about CBP's commitment to investing in new and
    emerging detection technology, as well as some of the steps we have
    taken toward enhancing cargo security. I will be happy to answer any of
    your questions.

                          Biography for Todd C. Owens
        Mr. Todd C. Owen was appointed to Executive Director of the Cargo
    and Conveyance Security office, Office of Field Operations (OFO), in
    May 2006. Mr. Owen is responsible for all cargo security programs and
    policies for CBP, including the Container Security Initiative, the
    Customs-Trade Partnership Against Terrorism Program (C-TPAT) office,
    all non-intrusive inspection technology and radiation portal monitor
    deployments, the national canine enforcement program, the National
    Targeting Center-Cargo, and the 100 percent scanning initiative. Mr.
    Owen also coordinates U.S. Customs and Border Protection's (CBP)
    maritime cargo enforcement policies and activities with the U.S. Coast
    Guard and all air cargo efforts with the Transportation Security
    Administration.
        Mr. Owen previously served as the Director of the C-TPAT program
    from January 2005 until May 2006. As Director, he strengthened
    management control and increased hiring, allowing for a significant
    enhancement in the level of foreign site assessments performed
    worldwide under this 9,000 member partnership program.
        Before arriving in Washington, D.C., Mr. Owen was the Area Port
    Director in New Orleans, where he was directly responsible for all CBP
    operations throughout Louisiana, managing 240 officers at seven port
    offices. Before being selected as Area Port Director, Mr. Owen spent
    eight years in South Florida where he held various trade related
    positions within OFO and the Office of Strategic Trade.
        Mr. Owen began his career with U.S. Customs Service in 1990 as an
    Import Specialist in Cleveland, Ohio.
        Mr. Owen is a career member of the Senior Executive Service and was
    a senior executive fellow at Harvard University's John F. Kennedy
    School of Government.
        He is a graduate of John Carroll University in Cleveland, Ohio, and
    holds a Master's degree in Public Administration from St. Thomas
    University in Miami, Florida.

        Chairman Miller. Thank you, Mr. Owen. Dr. Hagan for five
    minutes.

      STATEMENT OF DR. WILLIAM K. HAGAN, ACTING DEPUTY DIRECTOR,
       DOMESTIC NUCLEAR DETECTION OFFICE, DEPARTMENT OF HOMELAND
                                SECURITY

        Dr. Hagan. I am sorry. I am losing my voice. Good afternoon
    Chairman Miller, Ranking Member Broun and the distinguished
    Members of the Subcommittee. As Acting Deputy Director of the
    Domestic Nuclear Detection Office, DNDO, at the Department of
    Homeland Security, I am honored to be here with the U.S.
    Customs and Border Protection Executive Director for Cargo and
    Conveyance Security, Mr. Todd Owen.
        I would like to thank the Committee for the opportunity to
    provide this update on our Advanced Spectroscopic Portal
    Program, or ASP. Since our last hearing, ASP systems have
    undergone another round of field validation conducted by CBP.
    The purpose of field validation is to produce information about
    the operational characteristics of the system and correct any
    issues that must be resolved before moving onto the last test
    event for the system, the Operational Test and Evaluation
    (OT&E).
        The July field validation identified several issues. We
    worked with CBP to classify the severity of these operational
    and technical issues and DNDO has since addressed all the
    issues initially identified. Because the systems are
    configuration controlled, adjustments were checked and
    validated through use of a replay tool and regression testing
    to ensure that sensitivity to threats was not decreased below
    guidance. Although a few issues remain to be resolved, we are
    optimistic that field validation can be restarted in the near
    future.
        After CBP completes field validation, OT&E will be
    conducted by the DHS Science and Technology Directorate's
    operational testing authority. The current path to
    certification includes field validation and OT&E, as well as
    preparation of all the departmental acquisition program
    requirements, including presentation of the program before the
    DHS Acquisition Review Board, or ARB, for a production
    decision. We are committed to following the Department's
    Management Directive 102-01. As such, we cannot speculate about
    the outcome of the ARB decision. The Secretary's decision
    regarding certification will follow the ARB.
        Beyond testing of the ASP systems, deployment of our
    current, low-rate initial production ASP units is now being
    considered. The fiscal year 2010 appropriations bill signed
    three weeks ago contains specific language encouraging
    incremental deployments of ASP, as did the report from the
    National Academy of Sciences, and we are planning discussions
    with CBP regarding the best way to deploy these units once OT&E
    is completed.
        However, one new obstacle has emerged that will greatly
    impact the path forward. The United States is facing a severe
    helium-3 shortage. Helium-3 is a gas that is used for neutron
    detection within both current and next-generation RPMs. Because
    this decreased supply affects multiple agencies within the
    Federal Government and beyond, the White House has convened an
    Interagency Policy Committee, including DHS, to discuss the
    issue and possible solutions. This group decided in September
    that no more helium-3 will be allocated to RPMs for the time
    being. We are currently leading interagency efforts to identify
    alternative neutron detectors, and we are working to assess the
    impact this will have on our deployment strategy and path
    forward. For now, we plan to continue field validation and
    operational testing evaluation in exploring our options for the
    path forward beyond that.
        Additionally, I have also been asked to address the status
    of improvements to the currently deployed RPM systems, the PVT
    systems. Some have asked if software or algorithm improvements
    including improvements to the energy windowing algorithms that
    are currently deployed could provide enhanced capabilities for
    PVT systems. We are currently funding work at Pacific Northwest
    National Laboratory and the Johns Hopkins University Applied
    Physics Laboratory to improve energy windowing and conduct
    related research to determine what kinds of PVT improvements
    may be possible. This research is still ongoing. That being
    said, it is important to note that any improvements to PVT
    detectors will only improve primary inspection capabilities or
    detection. Current performance limitations, and secondary
    inspection where CBP officers require the ability to identify
    radiological sources, are not addressed by improvements to PVT
    technology. Better energy windowing algorithms will not change
    this fact.
        As a final note, I would like to emphasize that the ASP
    program is only one piece of the multi-layered solution we have
    termed the global nuclear detection architecture. This strategy
    calls for the use of multiple preventive actions at our ports
    as well as security measures along all potential pathways
    beyond, at and within the Nation's borders.
        We will continue to work with our partners within DHS,
    other federal departments, State and local agencies and the
    Members of this subcommittee and the Congress to keep the
    Nation safe from radiological and nuclear terrorism.
        This concludes my statement. I thank you for your
    attention, and I will do my best to answer any questions you
    may have.
        [The prepared statement of Dr. Hagan follows:]
                     Prepared Statement of William K. Hagan

    Introduction:

        Good afternoon Chairman Miller, Ranking Member Broun, and
    distinguished Members of the Subcommittee. As Acting Deputy Director of
    the Domestic Nuclear Detection Office (DNDO) at the Department of
    Homeland Security (DHS), I would like to thank the Committee for the
    opportunity to provide a status update on the Advanced Spectroscopic
    Portal (ASP) program. I would also like to thank the Committee for its
    support of DNDO's mission to reduce the risk of radiological and
    nuclear terrorism to the Nation.
        In late June of this year, I provided testimony about our next-
    generation radiation portal monitors (RPMs), including where RPMs are
    deployed and how DHS' U.S. Customs and Border Protection (CBP) operates
    this technology at our ports of entry (POEs). My testimony today will
    include a status update and the path forward for the ASP program. I
    will also describe operations during field validation and improvements
    for poly-vinyl toluene (PVT) systems.

    Energy Windowing for PVT Systems:

        Before I get into an update on the ASP Program, I have been asked
    to address the status of improvements to PVT systems. For some time,
    there have been various questions about the possibility of exploring
    energy windowing improvements to the currently deployed PVT systems.
    Energy windowing is an algorithmic alarm method that can be applied to
    plastic scintillator-based RPM systems to improve operational
    sensitivity to certain threat sources while reducing the alarm rates
    from naturally occurring radioactive material. Some have asked if
    software or algorithm improvements could provide enhanced capabilities
    for PVT systems that would achieve performance similar to that of the
    ASP systems. Algorithms that provide energy windowing for PVT systems
    were introduced into the currently deployed systems in 2007, and DNDO
    is currently funding work both at the Johns Hopkins University Applied
    Physics Laboratory (APL) and at Pacific Northwest National Laboratory
    (PNNL) to determine if further gains can be made through additional
    energy windowing techniques. We have also asked that, wherever
    possible, the labs work cooperatively on these efforts. Funding for the
    work at PNNL is approximately $1.6 million of Fiscal Year (FY) 2008
    appropriations, and investments in APL for studies related to energy
    windowing are approximately $90,000 of FY 2009 appropriations.
    Scientists are thus far uncertain whether additional gains in
    operational performance can be coaxed from PVT RPMs by using more
    energy windowing, because the limits of passive detection with plastic-
    based detectors are coupled with a need to keep false alarms at a
    minimum. As new techniques are studied and evaluated, it is important
    to note that the scintillation properties of PVT detectors are
    fundamentally different than the ASP technology and we continue to
    evaluate the operational effectiveness of next-generation systems
    concurrently.

    Current Status of the ASP Program:

        DNDO continues to enhance the capability to detect and report
    attempts to import, possess, store, develop, or transport nuclear or
    radiological material for use against the Nation. Part of our work
    involves working with DHS partners to provide the equipment and systems
    they need to perform their missions, including scanning cargo for
    possible radiological or nuclear threats. After 9/11, considerable
    concern was raised about the possibility that terrorists could use the
    enormous volume of cargo flowing into the United States as a means to
    bring in nuclear material or a nuclear weapon. By far, the largest mode
    for incoming cargo is maritime shipping containers, with approximately
    11 million containers coming into the country every year. Additionally,
    in the Security and Accountability for Every (SAFE) Port Act of 2006,
    Congress mandated that all containers coming in through the 22 top
    volume ports be scanned for radiation by the end of 2007. Thus,
    considerable effort and resources have been devoted to this mode of
    transportation to provide comprehensive radiological and nuclear
    detection capabilities, particularly at POEs.
        CBP currently scans cargo entering at our nation's POEs using PVT-
    based radiation portal monitors (RPMs) that can detect radiation, but
    cannot distinguish between threat materials and naturally occurring
    radioactive materials (NORM), such as kitty litter and ceramic tiles.
    Narrowing down alarms to just those for dangerous materials is
    especially important for POEs that have a high volume of containers, or
    those that see a high rate of NORM.
        Building on previous work within CBP and the DHS Science and
    Technology Directorate (S&T), DNDO initiated the ASP program in 2006 to
    develop next-generation technology that can both detect radiological
    material and distinguish between threat and non-threat materials. ASP
    systems have shown significantly improved capability to distinguish
    radiological threats from non-threats over the hand-held instruments
    currently used in secondary screening. Thus, the introduction of ASP
    systems is expected to not only reduce the number of unnecessary
    referrals and false positives in primary scanning but increase the
    probability of detecting dangerous materials in secondary. I want to be
    clear here that ASP cannot and will not be acquired and deployed until
    an Acquisition Review Board (ARB) has been conducted and the Secretary
    of Homeland Security has certified that the technology provides a
    significant improvement in operational effectiveness over current
    systems. Although we are getting closer to these decision points,
    testing and evaluation still remains so that all the data required to
    make informed decisions has been accumulated, analyzed, and documented.
        To date, two ASP vendors have developed systems that have completed
    the following 2008-2009 tests: System Qualification Testing, designed
    to demonstrate that ASP units are manufactured in accordance with
    processes and controls that meet the specified design requirements; and
    Performance Testing at the Nevada Test Site (NTS), designed to evaluate
    ASP, PVT, and radioisotope identification devices (RIID) detection and
    identification performance against controlled, realistic threat
    materials, shielding, and masking scenarios. One vendor has also
    completed Integration Testing, designed to determine whether the ASP
    systems are capable of operating and interfacing with the other
    equipment found in operational settings. This vendor has now begun
    field validation testing, designed to exercise ASP systems in a stream
    of commerce environment at POEs.
        The second vendor has experienced technical and accounting issues
    that have caused its testing schedule to lag behind. DNDO is in the
    process of determining the best path forward with this vendor. The
    remainder of my testimony will focus solely on the first vendor.

    Field Validation:

        Since our last hearing, CBP conducted an additional round of field
    validation at operational POEs. You may recall that CBP operated the
    ASP systems at four field validation sites in January and February of
    2009 for a period of two weeks. During this time, the systems were run
    in tandem with the PVT systems to scan incoming cargo conveyances, and
    were able to collect data in an operational port environment with real
    flow-of-commerce. During these operations, the ASP systems showed
    higher than expected alarm rates for three industrial sources.
        Following this first round of field validation, the ASP thresholds
    were adjusted to more effectively eliminate alarms on benign sources
    and were retested to ensure that the sensitivity for detection of
    special nuclear material and threats did not fall below guidance
    requirements. Utilizing a computer-based replay tool, we were able to
    determine that the adjustments solved the problem without decreasing
    the probability that the system will detect and identify threats.
    Additionally, the systems were put through regression testing to
    determine if the problem had been appropriately addressed.
        After the threshold adjustments were fine-tuned for the three
    specific isotopes that were problematic in the winter, CBP restarted
    field validation in July. This testing identified two additional
    issues. First, there was a single mission-critical fault at one field
    validation site in which the software database system failed to scan
    multiple conveyances, and did not immediately notify the CBP Officer of
    the problem. There was no security threat at this port because ASP
    systems were operating in parallel with the current generation systems.
    A technical representative from the vendor was summoned and the issue
    was rectified. The cause of the problem has since been identified and
    fixed. To assure that the fix did not create any unintended problems,
    the software has been successfully regression tested.
        The second technical issue identified during July field validations
    was a higher than expected rate of false alarms for certain special
    nuclear material (SNM) threats. Since ending this second round of
    testing, the ASP thresholds were adjusted to eliminate this problem,
    and the systems were retested to ensure that the sensitivity for
    detection of special nuclear material and threats was not reduced below
    guidance requirements. Use of the replay tool and regression testing
    validated that the false alarm rate will be reduced below guidance
    requirements, and it is anticipated that another round of tandem
    operation field validation will verify the effectiveness of the
    settings. In this case, the threshold adjustments resulted in an
    acceptable reduction in the sensitivity of the system, while remaining
    more sensitive than required by the original specifications.
        The fact that the SNM false alarms did not occur at the same rate
    in the first round of field validation as seen in the second round is a
    good example of the need for robust, multi-stage testing, since it may
    have revealed a sensitivity to seasonal changes and cargo contents that
    was not expected. Such oddities are typical when a new system
    transitions from the lab to the real world, and we will continue to
    learn and improve as testing progresses. At present, DNDO and CBP are
    in discussions to establish the ground rules for starting the next
    period of tandem field validation evaluations. Following the completion
    of tandem operations, ASP systems will undergo solo field validation
    evaluations.

    Path Forward:

        Once all the phases of field validation are completed to the
    satisfaction of both DNDO and CBP, ASP systems will complete
    independent operational testing and evaluation (OT&E) conducted by the
    S&T Operational Testing Authority. Test data will be analyzed and
    provided to inform the Secretary's certification decision. DNDO is also
    engaged with the National Academy of Sciences (NAS) to perform an
    additional review of ASP testing and inform the certification process,
    as required in the FY 2008, 2009, and 2010 Homeland Security
    Appropriations bills.
        Additionally, DNDO is working to develop a cost-benefit analysis
    that will analyze the cost effectiveness of deploying ASP systems in
    several different configurations. This cost-benefit analysis
    methodology is still being processed with the available data and
    requires additional data from the remaining tests before it can undergo
    review within the Department and be finalized.
        The current path to certification includes testing as described
    above, accompanied by the analysis of results, to ensure that Secretary
    Napolitano has sufficient information and all the Departmental
    acquisition program requirements are met. ASP systems have been under
    review and evaluation for over three years now, but we are focused on
    informing a decision to go forward with acquisition and deployment with
    the appropriate processes. Such a decision will be made only when it
    has been determined that ASP will increase the probability of detecting
    dangerous materials while minimizing operational burdens, rather than
    based on a pre-determined timeline.
        The changes and continued diligence that DNDO exercises in
    conjunction with the ASP program will ensure that any eventual
    certification and acquisition decisions are consistent with DHS
    priorities and made with a documented acquisition management
    foundation. The ASP program will also be presented to the DHS ARB for
    an MD 102-01 milestone decision for purchase and deployment of ASP.
    This is a very well-defined milestone that was developed for all large
    programs within the Department of Homeland Security. Items such as
    mission needs, operational requirements, analysis of alternatives,
    etc., are part of the MD 102-01 process--no production units can be
    purchased and deployed without successfully navigating the process. To
    be clear, the Secretarial certification requirement is in addition to
    the MD 102-01 deployment decision. DNDO intends to present both the
    Secretary and the ARB with all the necessary information to make
    decisions about the operational effectiveness and potential optimal
    deployment of ASP systems.

    Conclusion:

        DNDO will continue to work with CBP and other partners within and
    beyond DHS to improve the Nation's ability to detect radiological and
    nuclear threats at our ports and borders. DHS is facing a challenge; we
    must balance facilitating the flow of commerce at our ports and borders
    with the need to sufficiently scan cargo for radiological or nuclear
    threats before it enters our Nation. As both the President and
    Secretary have said, the Nation needs more technology to meet its
    security challenges and the technologies that DNDO is pursuing, of
    which ASP is but one example, are a critical component in addressing
    that challenge.
        Our efforts to develop and evaluate ASP systems are based on sound,
    proven testing and evaluation processes used with proven success across
    government and academia. Current test results are capturing the
    benefits of ASP systems, and the reviews to date have provided a
    valuable assessment of the program and identified a number of key
    lessons learned. As we collect more operational data for the ASP
    systems, we are better able to determine the optimized settings for
    detecting and identifying threats, while facilitating the flow of
    legitimate commerce.
        I welcome and appreciate the Committee's active engagement with
    this program, and look forward to continuing our cooperation as we move
    forward together. Chairman Miller, Ranking Member Broun, and Members of
    the Subcommittee, I thank you for your attention and will be happy to
    answer any questions that you may have.

                         Biography for William K. Hagan
        Dr. William Hagan serves as the Acting Deputy Director for the
    Department of Homeland Security's (DHS) Domestic Nuclear Detection
    Office (DNDO). Prior to January 2009, Dr. Hagan served as the Assistant
    Director of the Transformational Research and Development (R&D) within
    DNDO. Dr. Hagan was responsible for long-term R&D, seeking technologies
    that can make a significant or dramatic positive impact on the
    performance, cost, or operational burden of detection components and
    systems.
        Before his work with DNDO, he was a Deputy Business Unit Manager at
    Science Applications International Corporation (SAIC). Business areas
    included nuclear technology (analysis, detection, and applications),
    telecommunications, optics, transportation, system integration, and
    technology assessments during his thirty years at SAIC.
        Dr. Hagan earned a Bachelor of Science in Engineering Physics in
    1974, Master of Science in Physics in 1975, and Master of Science in
    Nuclear Engineering in 1977 from the University of Illinois at Urbana.
    He received his Ph.D. in Physics from the University of California-San
    Diego in 1986. He holds three patents. Dr. Hagan was appointed to
    Senior Executive Service in 2006.

                                   Discussion

        Chairman Miller. Thank you, Dr. Hagan. I now recognize
    myself for five minutes.

         CBP Procedures After a Primary Alarm and the Effect of False
                                   Positives

        Mr. Owen, I hope that CBP's procedure is, if you identify
    nuclear materials, the kind of nuclear material that would be
    used in a nuclear weapon or for a dirty bomb, that your
    procedures would require that you act like your hair was on
    fire, that you act with great urgency. Without getting into
    anything that is classified in this setting, could you describe
    what your procedures are?
        Mr. Owen. Yes, sir. Whenever we have any alarm of a PVT on
    primary, the cargo or the passenger vehicle is then sent into
    secondary because again, the PVTs at this point only detect
    radiation, they do not identify what it is.
        In the secondary, we use the hand-held Radiation Isotope
    Identification Device. If that determination is that we have
    special nuclear materials, we then implement what we call SIN
    procedures: secure, isolate, notify. We secure the area, we
    isolate the conveyance, and then we begin our notifications
    back to our Laboratory and Scientific Services folks at our
    National Targeting Center for cargo here in Northern Virginia.
    The CBP officers would transmit the radiation spectra to the
    Laboratory and Scientific Services people to make a
    determination as to what exactly what we have there. So as this
    is going on, again, we are in secondary. It is a more
    controlled arena. Depending on the footprint of the port,
    depending on the layout, the time of day, the traffic, will
    impact how much of a disruption it will be to some port
    operations. But clearly, whenever we have a higher level threat
    such as the special nuclear materials, we do act accordingly.
    There is some disruption to the port operations until we
    resolve that alarm.
        Chairman Miller. You said secure, secure and identify,
    secure the area. How big an area do you secure?
        Mr. Owen. Mostly it is just in the secondary area with what
    we have. We will create a perimeter of secure area if you will.
    It will not require an automatic shutdown of neighboring
    activities or anything like that. As we do the initial
    readings, send it back to our Laboratory and Scientific
    Services folks--clearly, if the issue has escalated, if it is
    of grave concern to us, your exclusion zone would expand and
    the level of activity would increase.
        Chairman Miller. Okay. Do you have any idea how many hours
    ports like New York, New Jersey, Los Angeles, Long Beach, would
    be closed or how they would be affected by false positives for
    special nuclear materials?
        Mr. Owen. I don't have any data as to how many times this
    has happened or was shut down, but those are the procedure. And
    really again, based on the layout of the port, if it is a large
    seaport, you can have just one terminal affected by an alert of
    special nuclear materials through one of our detection devices.
    If it is a smaller land border crossing, you could
    theoretically have a much greater impact on that type of
    operation. So there is an impact depending on the different
    variables as to how much of an impact, but clearly there is an
    impact whenever we have an alert of a special nuclear material.
        Chairman Miller. Okay. Well, obviously one of the most
    immediate problems with false positives is disruption of
    commerce, disruption of ports and borders. But I remember from
    college that the first time the fire alarm went off in the
    dorm, everyone rolled right out of bed and went outside, but by
    the time it happened three or four times because of a prank,
    all anyone did was just put a pillow over their head to make
    the noise--so they could sleep through the noise. I don't ever
    want it to become routine if we have an alert that there is a
    special nuclear material. Obviously, if there is a real fire,
    the fire alarm would have done no good at all.
        Mr. Aloise, the July round of field validation tests, there
    were, you said, false positives. The machinery, the device,
    ASPs detected materials that were not there at all. Can you
    give us the details of how many false positives or was it one
    or two machines, was it all the machines? What happened?
        Mr. Aloise. Yes, I believe, and CBP can confirm this, there
    was four per thousand at the smaller ports and eight per
    thousand, eight false positives per thousand conveyances at the
    larger ports during the testing in July.
        Chairman Miller. And was that across all machines or was
    that just one or two machines that seemed not to be quite
    right?
        Mr. Aloise. I believe it was across all machines, yes.
        Chairman Miller. So it was a problem with technology
    generally not with a problem with a specific----
        Mr. Aloise. With any one machine.
        Chairman Miller. Okay. My time has almost expired, so I
    will recognize Dr. Broun for five minutes.
        Mr. Broun. Thank you, Chairman. I still am going to be very
    generous with time. You have been very generous with me, and I
    will continue to be generous with you at any point. I want to
    make the Chairman----
        Chairman Miller. After serving for two years with Dana
    Rohrabacher as a Member of the Subcommittee, it is really not a
    problem.

              Steps Taken to Reduce False Positives and Negatives

        Mr. Broun. Thank you, Mr. Chairman. Some of the field
    validation tests reveal high levels of false positives for the
    special nuclear materials. This is a serious problem because
    any identification of any special nuclear materials requires a
    robust response from CBP officers. What is DNDO doing to
    address this issue? And you might also--I am concerned not only
    about false positives but false negatives. And I really haven't
    heard anything about that so if you could deal with that, too?
        Dr. Hagan. Regarding what we are doing about the false
    positives, the field validation showed a large number or
    unacceptably high number of calls for SNM (Special Nuclear
    Material). We have adjusted the threshold levels for the ASP
    and run a replay tool to demonstrate that in fact the number of
    false alarms, if these threshold settings had been used instead
    of what was used in July, that almost all of those false alarms
    would have been eliminated. The number would have dropped from
    about, according to my data here, from about 49 to seven. Now,
    no system is flawless or perfect. And so it is not likely that
    we will ever build a system that will produce no false alarms
    in primary. But the levels that we are operating now according
    to the replay tool are within the acceptability range.
        Regarding false negatives, in the flow of real commerce in
    the field validation activity, there is really no way to know
    if you have dismissed something that you shouldn't have. Now,
    we have testing at another place, the Nevada Test Site, where
    we tested that, and I don't recall what the false negative rate
    was, but it was quite low.
        Mr. Broun. As a physician, I am concerned about, and in
    science, as you know, talk about specificity and sensitivity.
    Several independent sources have already indicated that ASP
    only marginally increases our ability to detect lightly
    shielded special nuclear materials. If we manipulate those
    thresholds, won't this decrease that margin even more, and if
    this is the case, how would that affect any future cost benefit
    analysis?
        Dr. Hagan. The change in--when the threshold settings were
    changed, again using the replay tool, we were able to calculate
    what the difference or see what the difference was in the
    sensitivity. It is a reduced sensitivity as with any detector.
    When you make your system--when you reduce, make changes to
    reduce the false alarm rate, the sensitivity is usually
    affected. So this is true of this detector as well. But when we
    compare the sensitivity of the system before the threshold
    settings were made to that after, while it was reduced, the
    reduction was rather slight and the performance of the system
    is still within the specification and the guidance.
        Mr. Broun. In order to verify and validate the threshold
    changes made to ASP systems, a replay tool was developed. Has
    this tool been validated by an independent party?
        Dr. Hagan. Yes, it has.
        Mr. Broun. Okay. And who is that party?
        Dr. Hagan. The Johns Hopkins APL, Applied Physics Lab.
        Mr. Broun. And what was the results of that?
        Dr. Hagan. Okay. Well, maybe the best way to convey that is
    to comment or to give you some of the results from the replay
    tool validation exercise. So when we say a replay tool, we mean
    we want a tool that when given the same input data as the real
    system, it produces the same results as the real system. And so
    the replay tool was given the real data for 11,000 occupancies,
    and the results were compared to what actually came out of the
    system. The results were identical for all 11,000 except for
    eight. Those eight that were found, the reason that the
    differences were slightly different, the results were slightly
    different, is that the codes were run on two different
    computers, and there was a slight round-off error difference
    between the two computers. And because of that, there were
    eight occupancies that yielded slightly different results.
    Other than that, the results were identical.
        Mr. Broun. Thank you. Mr. Aloise or Dr. Persons, do you all
    have a response to that?
        Mr. Aloise. We would say this and then I will let Dr.
    Persons answer as well. As you recall in June when we
    testified, we said there was a marginal improvement in the
    Nevada testing over what the PVTs were, and that marginal
    improvement, because of this reducing of the thresholds has now
    gotten smaller. So when they do the cost benefit analysis, all
    this has to be factored into that cost benefit analysis because
    now the ASP seemingly does not have the performance that it
    initially did when we testified here earlier this year. So that
    is one point.
        The other point is we have not looked at the replay tool.
    We have not talked to Johns Hopkins, so we can't really comment
    on the replay tool. Dr. Persons, would you like to----
        Dr. Persons. Well, just to confirm what Mr. Aloise was
    saying, we have not been able to look at that, but one of our
    key points on this is just the idea of an independent
    verification or validation of such a tool to do that. And in
    interviews with CBP officials, for example, there was a concern
    about perhaps having an institution such as the American
    National Standards Institute, or ANSI, do something more
    standardized oriented body to do that. There is also of course
    when you are dealing with software development issues which are
    implied in these systems with this level of complication, then
    you would look at of course the software models, the software
    standards, how those were constructed, how they execute the
    algorithms and perform under test.
        Mr. Broun. Dr. Hagan, I would like for you to, if you
    would, submit that report for the record for us----
        Dr. Hagan. Sure, in the form of a letter.
        Mr. Broun.--and a discussion about it. Okay. Good. Thank
    you, sir. Mr. Chairman, I yield back.
        Chairman Miller. Thank you, Dr. Broun. The Chair joins Dr.
    Broun in that request.
        Ms. Dahlkemper for five minutes.

                            Mission Critical Failure

        Ms. Dahlkemper. Thank you, Mr. Chairman. Thank you to all
    those here testifying today. I want to go to one of the most
    basic problems and I guess one of the most alarming was the
    mission critical failure when two dozen cargo trucks were
    permitted to go through the ASP in order to be screened and
    actually were not operating at that time. And supposedly this
    has been rectified by the contractor with an indicator light.
    Is that correct?
        Dr. Hagan. No.
        Ms. Dahlkemper. Nothing has been rectified?
        Dr. Hagan. It has been rectified but not with an indicator
    light. There is a message that gets shown to the operator on
    screen that says that the system is in mission critical failure
    and should not be operated.
        Ms. Dahlkemper. So prior to this there was no way to detect
    if there was----
        Dr. Hagan. There was a message but it was confusing and not
    clear, and so that is what led to the incident. By the way, I
    should comment that this only happened one time in one system
    in all the testing in validation runs that have been going on
    for approximately for a year. So we don't think it is going to
    happen again, but we modified the software to properly notify
    the operator.
        Ms. Dahlkemper. I can buy a $15 smoke detector for my house
    and it will tell me when it is beginning to not work, and when
    it doesn't work, it will beep so I have to get up in the middle
    of the night and take the battery out. But anyway, I just
    wanted to make sure that that problem is rectified at this
    point.
        Dr. Hagan. It has been rectified, it has been regression
    tested, and we are confident that the message will be properly
    displayed. We are struggling a bit with this one error or issue
    that has come up because it has only happened once. So we don't
    think it will happen again. But that is the point of field
    validation, is to shake out some of the issues and problems
    before you move on to the actual operational test.
        Ms. Dahlkemper. How may of these are in operation in this
    field testing? How many?
        Dr. Hagan. Six.
        Ms. Dahlkemper. Six? And how many of the PVTs do we have
    currently?
        Mr. Owen. Just under 1,400 deployed nationwide at the land
    borders and seaports and some mail facilities.

                  Energy Windowing to Improve PVT Performance

        Ms. Dahlkemper. Okay. One of the things I wanted to ask,
    Mr. Owen, is on the GAO's previous work on the ASPs, they found
    that the ASPs provide a marginal improvement of the PVTs to
    attack the certain types of radioactive sources under a handful
    of very specific scenarios. But the GAO has also recommended
    that DNDO invest in ways to increase performance of the
    existing radiation monitors known as PVTs by a software
    improvement process called energy windowing. Do you believe
    that more efforts should be made to investigate the benefits of
    the energy windowing?
        Mr. Owen. Yes, our position remains that we need to fully
    maximize what the detection capabilities of the existing
    technology of the PVTs is. So we believe there are gains that
    have already been made with energy windowing, and we would rely
    on our scientific partners to tell us if we have reached the
    edge of that envelope as to how much they can do, either with
    the existing algorithms or with advanced algorithms that may be
    there. So as the operator and end-user of the equipment, yes,
    we would like to etch out as much detection capabilities as we
    can from the PVTs, recognizing that even with energy windowing,
    there will still not be an identification device, just a
    detection device.
        Ms. Dahlkemper. Mr. Aloise, how much has been spent on
    energy windowing work by DNDO or DHS in the last few years
    compared to how much has been spent on developing ASPs?
        Mr. Aloise. I don't have that information, but I do know
    that we have been told that further research has sort of been
    slow-balled, and we would have liked to have seen them a lot
    further by now in where the energy windowing is with the PVTs.
        Ms. Dahlkemper. Could I get that information? Could that
    information be found, Mr. Owen, in terms of how much work has
    been done?
        Dr. Hagan. Yeah, I can give you the numbers right now.
        Ms. Dahlkemper. Okay.
        Dr. Hagan. For energy windowing, we have funded a total of
    about $1.9 million again to Pacific Northwest National
    Laboratory and the Johns Hopkins University Applied Physics
    Laboratory.
        Ms. Dahlkemper. And how does that compare to how much has
    been spent on ASPs?
        Dr. Hagan. It is a lot less. I don't know the exact ratios
    or anything.
        Ms. Dahlkemper. What is the total? You don't know what the
    total has been spent?
        Dr. Hagan. Well, I think the number that was quoted I
    wouldn't argue with. I think it was $230 million over the
    last----
        Ms. Dahlkemper. It is $1.9 million over the last how many
    years?
        Dr. Hagan. Which----
        Ms. Dahlkemper. Last few years. When did we start? I am new
    to Congress so when did we start working on ASPs.
        Dr. Hagan. Oh, ASPs? 2004 or 2005. I actually don't know. I
    wasn't here then.
        Ms. Dahlkemper. So we have spent $230 million in that time
    period on that?
        Dr. Hagan. Right.
        Ms. Dahlkemper. And the energy windowing, you started
    working on that?
        Dr. Hagan. That was in fiscal year 2009. I think the actual
    work started in July of '09 at PNNL.
        Ms. Dahlkemper. So since just this past July we spent $1.9
    million?
        Dr. Hagan. Oh, we haven't spent it. We have obligated it.
        Ms. Dahlkemper. Obligated it?
        Dr. Hagan. I don't think it has been actually expended yet,
    but the work is ongoing.
        Ms. Dahlkemper. Okay. And that just started this year, in
    July?
        Dr. Hagan. Yes, in July, right.
        Ms. Dahlkemper. My time is expired. I yield back.
        Chairman Miller. Thank you, Ms. Dahlkemper. Mr. Bilbray for
    five minutes.
        Mr. Bilbray. Thank you, Mr. Chairman. I apologize for being
    late, and so if my questions have already been answered, you
    can remind me my tardiness was inappropriate.

                           The CBP Inspection System

        Mr. Owen, the issue of inspection, do we have a formal,
    multi-tier inspection system and when we talk about the
    inspection is there any pre-entrance inspection before they get
    to the port facilities?
        Mr. Owen. We do not pre-inspect cargo. What we do have is
    part of our layered cargo security strategy. We receive
    advanced information on all cargo and all modes before it
    arrives in the United States, different timelines, whether it
    is coming by air, by truck, by rail or maritime. With that
    advanced information, we make an initial assessment if the
    cargo poses a risk. In our maritime environment through our
    container security initiative, we have officers deployed in 32
    countries at 58 seaports that can then conduct an inspection on
    the highest risk maritime cargo before it is put on a vessel
    coming to the United States. We do not have a similar program
    in air or in land. So even though we have that advanced
    information before the truck or the train or the airplane
    arrives in the United States, we can identify the highest risk,
    but then it would be looked at at the port of arrival. So we do
    not have pre-inspection overseas for cargo.
        Mr. Bilbray. Okay. So we do have it for the maritime. Do we
    have any structure at all that if there was the ability to
    basically modify the arrival system, in other words hold them
    offshore until we could board the craft and be able to inspect
    it?
        Mr. Owen. Yes, sir. Absolutely. We work very closely with
    the Coast Guard. If there are any threats on a vessel, we will
    keep the vessel out at sea to address those threats or concerns
    with the crew or whatever the case may be. Likewise with the
    air cargo, we receive the information four hours before the
    plane lands, so in most cases if we have a concern that
    aircraft can be diverted to have address that threat, yes, sir.
        Mr. Bilbray. And I am glad to hear that because, let us
    face it, the port facilities tend to be much more centrally
    located in high population areas than either the port of
    entries for vehicles or with the aircraft, so thats preemptive.

           The Helium-3 Shortage and Potential Alternative Materials

        The issue of helium-3, as we are fighting to try to make
    what we have stretch, any discussion of the crisis and what we
    would need to do to crank up the breeder so we can produce more
    and make it available for you?
        Dr. Hagan. There has been a lot of discussion. This issue
    has really risen quickly to the top of a lot of people's lists,
    including the White House. They have formed an Interagency
    Policy Committee to address this that involves not only DHS but
    of course the White House folks, DOE, DOD and so forth. The
    problem is rather severe. The demand will probably outstrip the
    supply by a factor of ten. There is just not very much. And
    there are other uses for this material, for helium-3 that
    involve medical uses and so forth that frankly will probably
    end up having higher priority, although I can't say that for
    sure.
        Mr. Bilbray. So the fact that this is a strategic supply
    problem that we need to address as a nation?
        Dr. Hagan. Yes. I believe that is the case. Now, there
    are--so for some applications, like low-temperature physics
    research, helium-3 is pretty much the only--you are almost
    researching the material itself, so it can't be replaced.
        For our applications in neutron detection, there are other
    potential alternatives which we are, DNDO, is taking the lead
    for the interagency group to explore. We have been funding
    research in this area for several years already. So we are
    looking at alternative means of detecting neutrons. But because
    we don't know what those means are today, there are several, as
    I say, potential alternatives. But it can still take a year or
    two probably to sort of test those, evaluate them and make sure
    that the designs work. So it is a real problem for us. That is
    why it has thrown a real glitch into our planning process.
        Mr. Bilbray. Are there any other jurisdictions or agencies
    around the world that are specifically looking at developing
    this capability?
        Dr. Hagan. To make helium-3?
        Mr. Bilbray. Yeah.
        Dr. Hagan. Well, there are places----
        Mr. Bilbray. We don't have any others domestically. Is
    there anybody domestically looking at it?
        Dr. Hagan. There is some thought to perhaps going into
    natural gas wells and separating some of the helium from that
    and then distilling that, pulling out the helium-3 from that.
    There is a possibility of a source in--well, several sources in
    other countries. I probably shouldn't say much more than that.
        Mr. Bilbray. Okay, Mr. Chairman, I know my time has expired
    but in all fairness, there are a whole lot of issues that the
    Federal Government gets involved with that states, counties,
    cities, international bodies could work on or whatever. Here is
    one where this really is in our lap. I just can't perceive a
    state or a county or a coalition of states being able to
    address this issue, and here is one that I would sure like to
    see the Federal Government make some priority decisions and say
    this is one that only the Federal Government can do so the
    Federal Government has to be more aggressive on this. And I
    will remind all of us as we talk about other issues that could
    be handled by other agencies that we keep forgetting about the
    ones that only we can handle. And maybe we need to focus on
    these items that other people can't handle. So I appreciate
    your testimony and yield back, Mr. Chairman.
        Chairman Miller. Thank you. Mr. Rothman was here and I
    think will be back shortly. I will recognize him out of turn if
    he returns for a first round rather than for a second round.

                 In What Circumstances Should ASPs Be Deployed?

        The Chair of the Appropriations Subcommittee for Homeland
    Security is my colleague from North Carolina, David Price. His
    district adjoins mine. In fact, they kind of wrap around each
    other like a couple playing Twister. If he should ask me in
    what circumstances the new technology of the ASP should be
    deployed, what should I tell him? What are the markers for when
    it is reliable enough to replace the PVTs? Should it never
    replace the PVTs? Should it always be used unless there is some
    breakthrough that we haven't seen yet, some improvement in the
    technology? Should it always be used for secondary screening
    after a PVT has identified something to be concerned about? Or
    is this worth it at all? If the PVTs seem to be working, there
    is some disruption of having a truck pull over to the side and
    going over it by hand or cargo. But is this worth it? Mr.
    Aloise.
        Mr. Aloise. Well, our position has consistently been that
    nothing should be acquired in large scale until all the testing
    is complete, all the results are in, we know whether the system
    works or not, a cost benefit analysis has been done, and we can
    see whether it is worth it given all the other demands we have
    in our architecture, all the gaps in that architecture. After
    that cost benefit analysis is done, we will have a better
    answer to know whether it is worth to pursue with the ASPs
    versus the PVTs. Then the question is where? Does it make sense
    more in secondary? Does it make sense in other places?
        So our position, again, we need to get the testing done,
    all of it done, we need to do the cost benefit analysis. And
    until then we really don't have enough information to make this
    decision on.
        Chairman Miller. Well, the original idea was to replace
    PVTs with the ASPs. Given what we know, even assuming that the
    testing improves, we figure out what all the problems are so
    that it doesn't sort of go off without--it doesn't become non-
    functional without us noticing it until after a lot of trucks
    have gone through or it doesn't produce an amount of false
    positives that is going to shut down major ports for a
    significant period of time. After we have worked out those
    kinks, does it still make sense at all for it to be a primary
    detection device instead of the PVTs?
        Mr. Aloise. We have our doubts.
        Chairman Miller. Mr. Owen.
        Mr. Owen. Well, again, from our perspective, assuming it
    meets all of the technical and scientific endeavors and the
    machines work, then we have a serious concern about the
    operation and maintenance costs, the tail that will come with
    each one of these devices. As you mentioned in your opening
    statement, we are looking at perhaps as much as five times or
    more what it costs to currently operate it. So we also are
    waiting anxiously to see the cost benefit analysis as well as
    the life cycle costs to see if we can even as an agency afford
    these devices should they continue to develop and mature and be
    proven scientifically to work.
        Chairman Miller. Dr. Hagan.
        Dr. Hagan. I agree with the GAO on this. Until we get the
    cost benefit analysis and a lot of other data, by the way, that
    goes into what we call an Acquisition Review Board, it is
    really not prudent to try to presuppose the result of that. We
    absolutely adhere to the notion that doing the testing and
    doing the validation, doing the cost benefit analysis, life
    cycle cost estimates, all that has to be done and fed into this
    process. We have a very rigorous process within the department
    to do that. I wouldn't want to speculate about whether it is
    going to make sense in primary or secondary, either or both.
    But we at DNDO at least are indifferent to that. The point is
    it has to be driven by, you know, real data and analysis.
        Chairman Miller. Three years ago, Vayl Oxford, who was then
    Director of DNDO, said the priority for the first year is to
    get units out immediately. That seems like not the best idea
    right now. Does anyone disagree with that? Does anyone agree
    still with what Vayl Oxford said then, three years ago? Not a
    priority to get this in place immediately? Okay.
        When do we expect we will get the cost benefit analysis?

                Expectations of a Cost-benefit Analysis on ASPs

        Dr. Hagan. The cost benefit analysis is--we are actually
    struggling with this right now because of the helium-3 shortage
    and the impact of that. The cost benefit analysis of course
    depends on what the cost of the system is. We will have to most
    likely replace to helium-3 neutron detectors with some
    alternative. At this point, we don't know what that alternative
    is. As I said before, we have several potential candidates for
    that. But until we know what that is, it is going to be
    difficult to generate a cost benefit analysis because we won't
    know what the cost of the systems is. It may be possible to
    work around that, but frankly we don't know that yet. We are
    still looking at all the various options. We can try to get to
    a CBA, cost benefit analysis, but without that information and
    of course without the results from the operational test, we
    can't do that.
        Chairman Miller. Dr. Persons, actually, most of my
    questions I have directed initially to Mr. Aloise and he used
    the first person plural, and then you nodded. So I didn't
    direct an additional question at you, but you have worked with
    many new technologies. Are there best practices? Are there
    lessons to be learned for how to develop new practices, how to
    develop and deploy new technologies and have those lessons,
    have those best practices been applied with respect to this
    technology?
        Dr. Persons. Yes, sir. I worked with--coming out of an
    environment where, if you are familiar with the Defense Advance
    Research Projects Agency, it is the idea about rapid technology
    development, in the absence of requirements that is normally
    for a very high-risk type, high-reward research and development
    paradigm. And this particular case, as you mentioned earlier,
    there is sort of some relatively mature technologies, it was
    just relatively immature, the concept of operations it was
    going to be asked to be placed under, in other words, a port
    operation with high volume and so on.
        And so I think some of the lessons learned from this is
    what you put your finger on just a moment ago, was what I would
    call the mistake of scheduling invention, in other words,
    setting artificial deadlines. We want to get these out and then
    we will worry about sort of the technology development later
    on. GAO best practices for many years on this topic have shown
    that that is a mistake to do that. You want to risk mitigate
    the way you develop your programs by investing and particularly
    empowering your science and technology folks to do that and do
    that outside a procurement operation. Even though there needs
    to be close ties there, it still needs to be this idea that the
    science and tech folks can do the technology and development,
    and the procurement folks can worry about how to do the
    procurement
        And so keeping that sort of balance of power, doing that,
    the requirements definitions is hugely important. I think in
    this case, the customer ultimately was CBP and what were they
    going to be comfortable with in an operational paradigm? Was
    the con-op really going to be an ASP and primary alone in solo
    mode, or is there going to be a reduction in their manpower
    requirements and so on. or was it an SNM detection paradigm
    alone? Those are some issues that drive requirements through
    CBP out to the end of the program.
        And then thirdly, of course, the independent testing and
    evaluation, we have had a lot of discussion on that. Again, the
    key thing is there is the independence of that and how that is
    done.
        Chairman Miller. Okay.
        Dr. Persons. And so we have been pleased to see DNDO
    talking about interacting with DHS's OT&E office and so on----
        Chairman Miller. Okay.
        Dr. Persons.--at this phase.
        Chairman Miller. One of the lessons learned is that, at
    some point, you look for an off-ramp that if a technology
    seemed like a good idea but just didn't work, that you
    determine is not a matter of just working out kinks, but this
    was not as good idea as it seemed. Are there measures for when
    you should look for an off-ramp?
        Dr. Persons. Right. Well, GAO best practices recommend the
    use of readiness levels, technology readiness levels and so on.
    I think one of the things here that I think could perhaps be
    used is the idea of software readiness levels and so on because
    software an algorithms are such a critical part. I think that
    has manifested itself in the recent conclusion of the field
    validation tests and so on.
        So you have these metrics for doing that, and I think
    organizationally as I believe DHS likely has is some idea about
    what is an acceptable risk of this technology or this system
    for deployment. And there are ways to measure that and just
    give your managers informed decisions so that they can
    determine the risk of something, whether or not it needs to
    move forward and--mode.
        Chairman Miller. Well that is not quite a Dana Rohrabacher
    five minutes, but I am over the traditional five minutes. And
    with that I recognize Dr. Broun for five minutes or more.

             Metrics and Timelines for Making Decisions About ASPs

        Mr. Broun. Thank you, Chairman, and I am going to dovetail
    back into what the Chairman was asking. I would like to know
    the metrics. It seems to me that we keep fooling around here
    and have excuses of not implementing a technology, and not only
    a valid technology that is not only useful but may be
    preferable over our current technology, but there are other
    things that CBP is already doing. And going back to the
    Chairman, you are talking about an off-ramp, it seems to me we
    are getting to the point of fishing or cutting bait or one of
    the two. I mean, we have got to do something. If you all would
    give us some metrics about when we can expect the decision-
    making process, if you can help us to understand at what point
    do we need to pull the plug on the ASP program or implement
    something else or what have you with all of you, and I would
    like to--because I don't have a good feeling about this,
    frankly. I don't know about other Members but I would
    appreciate it if you all would give us some metrics or if you
    have them, please tell me right now. Mr. Aloise?
        Mr. Aloise. Well, you know, DHS came up with their
    ``significant increase in operational effectiveness'' as their
    metric for going forward with this new equipment, and we from
    the beginning thought it was a pretty low bar. I mean, I will
    go back to what I said. You know, even if this ASP works, it is
    a marginal increase. Now, with the adjustments and thresholds,
    that margin has gotten more narrow. So what GAO's role is here
    to evaluate what DHS is doing and provide this information to
    the Congress, of course, and we have made recommendations all
    along to make improvements in the testing, in the reporting,
    but the bottom line, I have to go back to what I have said,
    once the testing is done, we need to do that cost benefit
    analysis and decide whether or not it is worth to go ahead with
    this.
        Now, it has taken longer. You know, you mentioned in 2007
    they were ready to deploy, but the Congress who has oversight
    and getting GAO involved has got us to where we are today. Now,
    where do we go from here? I assume a third round of field
    validation testing. The first round, they made adjustments. The
    second round, we may be seeing the results of those
    adjustments, and we don't know what is going to happen in the
    third round. The question is, is this equipment worth it? And
    that is a decision that DHS has to make. And I mean, we have
    got a lot of information right now, and somebody can make a
    decision or they can put this back into what we belong--where
    it should have been for a long while, and that is R&D, in
    development, and see where--what is the most you can get out of
    this promising piece of equipment. But when you know, we
    started down the path or just started down the path of
    acquisition, deployment and spiral development, they have
    called it, and it has got us to where we are today, where the
    equipment has a lot of problems, it doesn't work very well
    right now. And to take a risk and replace the equipment we have
    got now on the borders that we know works with equipment we
    don't know works, you are always going to have GAO saying,
    whoa, step back. We need to look at this again.
        So I don't know. It is kind of a long-winded answer to your
    question, but we need more information before we could come to
    a conclusion, and that has to be the cost benefit analysis.
        Mr. Broun. Another question with cost benefit analysis, how
    can you do one without first knowing whether the ASP was used
    as a primary screening tool or as a secondary screening tool? I
    think that needs to be determined, too, just doing that cost
    benefit analysis because otherwise you are just shooting in the
    dark.
        Frankly, I see the private sector developing screening
    methods without the taxpayer being on the hook for them, and we
    have this, we have spent a bunch of money and a lot of it is
    sitting in a warehouse and not being deployed and we are
    testing. Mr. Chairman, I think it might be time to have that
    offer out from everything that I can see, but I don't know.
        And I would just like to ask Mr. Owen, would you possibly
    agree with me that it is time to maybe look at other
    technologies?
        Mr. Owen. Well, again, as you mentioned, the PVTs do meet
    our needs right now. They are effective. We can handle the
    level of alarms that we have got there. Hopefully, with any
    advances through the energy windowing, they give us a little
    bit greater technology detection capabilities than we have
    right now. Whether we can say we should pull the plug or not, I
    think again from the CBP standpoint, the jury is still out. We
    need that cost benefit analysis. We need to see how much better
    these in fact perform over the technology that we have before
    we could even make a decision as to should they just be in
    secondary or should they be primary and secondary.
        But I would like to just close by saying again, through all
    of our testing, we have kept the PVTs up and running. So we
    have always had that safety net, that effective equipment that
    we know has worked so that those 27 trucks did not get through
    when the machine went down.
        Mr. Broun. Thank you, Mr. Chairman. I yield back.
        Chairman Miller. Thank you.
        Dr. Hagan. Could I respond to any of this?
        Chairman Miller. Yes. Dr. Hagan.
        Dr. Hagan. Thank you. First of all, I wanted to comment
    that the cost benefit analysis addresses both primary and
    secondary screening, so we are not pre-determining that
    outcome. Second, the ASP, what we have been talking about here
    primarily is about ASP and PVT in primary, and you know, the
    debate will rage on. As I said before, I agree that the CBA is
    the way to determine that, you know, whether that is effective
    or not and worth the money.
        The ASP operating in secondary is another matter. The
    limitations of the current system--CBP officers are using the
    current equipment as well as they possibly can. There is no
    question about that. But there are limitations to those systems
    in both primary and secondary, and let me just focus on
    secondary. In secondary, there are limitations which
    improvements to PVT cannot address. Only ASP can do that. I
    can't talk about the limitations or the changes or improvements
    of ASP compared to hand-helds in an open session. But the
    differences are dramatic. And so I think this talk of an off-
    ramp would be very premature and perhaps a closed session would
    be the way to deal with that. But I would urge you to consider
    the fact that ASP has considerable application and benefit in
    secondary.
        Chairman Miller. It is not really my turn, but I do hold
    the gavel. Mr. Aloise, Dr. Persons, do you all have security
    clearances? Do you have that top secret security, the top? Are
    you familiar with all of the analyses of the comparative
    benefits of PVT and ASP and primary and secondary versus hand-
    held? Are you familiar with all of that as part of the work you
    have done?
        Mr. Aloise. We are familiar with everything we have been
    shown, yes.
        Chairman Miller. Okay. Have you asked for everything?
        Mr. Aloise. Yes, we have.
        Chairman Miller. If there is something you haven't been
    shown, you have asked for it and not been shown it?
        Mr. Aloise. Right, and we don't know that that is the case
    but----
        Chairman Miller. Okay. Right. And you do understand that
    GAO is part of Congress and when they ask, we are asking and we
    need for you to give them anything they have asked for.
        Mr. Aloise. Absolutely.
        Chairman Miller. Okay. Assuming that you have seen
    everything, is there anything--if you know all the tippy-top
    secret stuff, is it really different? Is analysis a lot
    different in the way that Dr. Hagan suggests?
        Mr. Aloise. I would say this. I think you pretty much know
    the story as I have testified and GAO has reported on.
        Now, in regards to secondary, just in discussions among
    ourselves, the ASP is bigger in radiation detection size
    matters. So it is a bigger instrument than the hand-held. So
    just for that--you have longer dwell times in secondary. So
    there might be uses. There might be a possibility where it make
    sense to put it into secondary, but we still don't have all
    that information.
        Chairman Miller. Okay. What is a dwell time? I am sorry.
        Mr. Aloise. You know, the container can sit a little bit
    longer in a secondary inspection, so you have a better account.
        Chairman Miller. I see. You are not trying to move things
    through.
        Mr. Aloise. You are not driving through.
        Chairman Miller. All right. I now recognize Ms. Dahlkemper.
        Ms. Dahlkemper. Staying on this line of questioning, if it
    was used in the secondary manner, how many units would we need?
        Mr. Owen. Yeah, I don't have the date available as to how
    many secondary locations we have in all of the ports of entry,
    but it is far fewer than what we have in primary obviously. But
    even with that, the volume through secondary is relatively low.
    I mean, we have about a one percent alarm rate on the land
    border and about 2.5, 2.6 percent in seaports. So we would have
    to again look at the life cycle costs, the cost benefits as to
    how much better does it offset what it is costing us to buy
    this equipment versus the time savings or the fewer officers to
    address the secondaries.
        Ms. Dahlkemper. Would anyone else like to address that? I
    don't know if anyone else can give you a better--more answer on
    that?
        Dr. Hagan. I can give you sort of a range. Like Mr. Owen, I
    don't have the numbers in front of me, but I believe the number
    of secondary sites is on the order of 400 to 500. Not clear
    that one would put an ASP at every one of those by any means.
    But as Mr. Owen says, it is far fewer than the number of
    primary screening locations. So the----
        Ms. Dahlkemper. I am certain there would be certain points
    that would be much more crucial. Maybe some of the borders--I
    mean, I live near the Canadian border. Some of those ports of
    entry are much more crucial than others.
        Dr. Hagan. Right.
        Ms. Dahlkemper. I did want to ask, though, I did want to
    kind of go back to this helium-3 issue because I think this is
    a crucial issue even beyond this hearing today. Who is in
    charge of the interagency task force on this?
        Dr. Hagan. It is the White House. I can't remember the name
    of the organization. I can give you the name of the person.
    That is about all I could do.
        Ms. Dahlkemper. Who is that?
        Dr. Hagan. Her name is Julie Bentz, B-e-n-t-z.
        Ms. Dahlkemper. Okay. My understanding is the ASP need
    three times the helium-3 as the PVTs?
        Dr. Hagan. The way it was designed was to have even more
    sensitivity for neutrons than the PVT systems did. So we put
    helium-3 tubes in the ASP design. But if we wanted to maintain
    the same level of sensitivity as the PVT for neutrons, it would
    require the same amount of helium-3. There is nothing that
    links the neutron detection technology to whether it is an ASP
    or a PVT.
        Ms. Dahlkemper. As it is currently designed?
        Dr. Hagan. It was designed to have more sensitivity to
    neutrons than the PVT system because more sensitivity is
    better.
        Ms. Dahlkemper. So how much more helium-3 is being utilized
    in an ASP than a PVT?
        Dr. Hagan. About three times more.
        Ms. Dahlkemper. About three times?
        Dr. Hagan. What I am saying though is if we want to go back
    to the same level of sensitivity as we currently have with the
    PVT, then the amount of helium-3 that would be required for an
    ASP unit would be the same as for a PVT unit.
        Ms. Dahlkemper. Well, then why would we have to change the
    unit if the sensitivity wasn't increased?
        Dr. Hagan. Because this is for neutron detection only.
        Ms. Dahlkemper. Okay.
        Dr. Hagan. There is another part of the system that detects
    gamma rays, and that is the part that is different. So maybe I
    should say that the difference between an ASP and a PVT is the
    gamma ray detection part. In one case, with the PVT, it is a
    big block of plastic, and in the case of an ASP it is chunks of
    sodium iodide. But in both cases, the neutron detection part is
    done with tubes of helium-3 gas. And that part is essentially
    the same with both systems.
        Ms. Dahlkemper. So you could redesign it so it would use
    the same amount of helium-3 and not really jeopardize the
    security?
        Dr. Hagan. Right. You wouldn't even have to redesign it.
    You could just take the tubes out, cut the number of tubes down
    by pulling them out.
        Ms. Dahlkemper. Okay.
        Dr. Hagan. It would be simple to do that.
        Ms. Dahlkemper. Okay. That answers my question. Thank you.
    I yield back.
        Chairman Miller. Thank you. I thought there was a Steve
    Rothman sighting, but that was perhaps a false positive. I
    think we now had sufficient questions for now at least. We may
    have an additional hearing. On this, you know, we do have--all
    Members have top-secret clearance. We have in this building and
    in the Cannon Building we have--leaded walls and Members go in
    and have to drop off their Blackberries and their cell phones
    before they go in, and they bring down a cone of silence and we
    hear all the top-secret stuff. And if we need to do that, we
    will, but if we go through that exercise and don't hear
    anything, it is going to be pretty irritating. So let us not do
    that right away, but before we close the hearing out, Dr.
    Hagan, can you tell us exactly where you think we will be on
    April 1? Will we have completed successful field tests, will
    there be a cost benefit analysis, where will we be on April 1?
    Three years ago the then-head of your unit said let us get this
    stuff deployed right now. Where will we be in April?
        Dr. Hagan. Because of the helium-3 issue, I cannot give you
    an answer about the cost benefit analysis. We know how
    important it is, and we may be able to do something, but
    without knowing how much the helium-3, or the neutron detection
    part of the system will cost, it makes that difficult. But we
    are looking at ways to try to get around that. Going back to
    field validation--well, maybe I should let Mr. Owen respond
    about the schedule. But by April, we will be, I suspect, in
    some phase of field validation or operational test.
        Mr. Owen. In terms of the field validation, we have those
    two critical areas that we have been discussing. We feel those
    have been addressed by DNDO. There are a few other minor issues
    that we are working through. Our objective is to restart field
    validation in the first quarter of calendar year 2010. So
    within the next month, we should restart field validation for a
    third round and go forward from there.
        Chairman Miller. Okay. Before we bring this hearing to a
    close, I want to thank our witnesses for testifying, and I
    guess it is a little early to say I look forward to seeing you
    the next time. Under the rules of the Committee, the record
    will remain open for two weeks for additional statements from
    the Members and for any answers to any follow-up questions the
    Committee may have for the witnesses. The witnesses are excused
    and the hearing is now adjourned.
        [Whereupon, at 2:27 p.m., the Subcommittee was adjourned.]
                                   Appendix:

                                  ----------

                       Answers to Post-Hearing Questions

    [SKIP PAGES = 000]

    Responses by William K. Hagan, Acting Deputy Director, Domestic
            Nuclear Detection Office, Department of Homeland
            Security; and Todd C. Owen, Acting Deputy Assistant
            Commissioner, Office of Field Operations, U.S. Customs
            and Border Protection, Department of Homeland Security

    Questions submitted by Representative Paul C. Broun

    Questions for Dr. Hagan, and Mr. Owen:

    Q1. LDo you believe computer modeling through replay tools,
    injection studies, or regression tools are enough for
    certification, or do you believe real-world field testing is
    required?

    A1. Certification requires that the ASP system be both
    effective and suitable for the operation it is designed to
    perform and that it meets the significant increase in
    operational effectiveness (SIOE) criteria. Each of the elements
    listed in the question play a part.

        Real-world testing is vital. During the ASP program the
    systems have undergone or will undergo real-world and near-real
    world testing during controlled performance tests at the Nevada
    Test Site (NTS, real-world targets of interest), Integration
    testing (real-world interface and interoperability testing),
    and both Field Validation activities and Operational Testing
    (real-world stream of commerce and Customs Officer
    involvement), as well as any initial deployments. Real-world
    events allow the system to be exposed to a variety of traffic,
    cargo and radiological conditions that cannot be duplicated in
    any other way. Because real-world data is so important, it is
    vital that the data relating to traffic, cargo and radiological
    conditions be collected so that the impact of certain system
    changes can be evaluated without having to re-experience all
    real-world environments. For ASP systems all of this data is
    collected in what is known as ICD-1 files.
        Replay tools provide the developer with an opportunity to
    re-create the real-world sensor data from previous data
    collection events (ICD-1 files), and to explore how changes in
    systems parameters would have impacted the results from those
    events. Replay tools are not computer modeling. These types of
    tools are used extensively for many types of systems,
    especially when data collection/real-world testing is
    expensive, dangerous, or time consuming. In this case, Replay
    tools can determine the performance of the detector system for
    a previous data collection event under various system parameter
    changes, such as threshold settings. In another example, Replay
    tools are used to determine if the system can continue to
    perform if one or more of its elements are faulty. Since ICD-1
    data files are organized by detector type, the replay tools can
    be used on modified ICD-1 files that have had this data
    removed. These sorts of regression analysis are extremely
    useful and can be used to provide information toward a
    certification decision. Injection studies are a way of
    extending the types and amounts of threat sources into stream
    of commerce data (i.e., ICD-1 files) so that system response to
    a broad spectrum of sources can be estimated. Injection studies
    are a type of computer modeling, and should therefore be
    checked against real, controlled experiments to validate that
    the injection tools closely match the output of real systems
    given the same input. Injection tools are useful to point
    toward potential improvements in systems performance, and
    should be used during continuous improvement planning--For
    certification, the best approach to confidently understanding
    system performance for threat sources is to perform
    measurements that span the threat space as much as practicably
    possible.

    Q2. LHas other ASP equipment been developed outside of this
    process by the private sector?

    A2. DNDO is aware of other vendors, outside the ASP program,
    that have continued to develop spectroscopic portals.

    Q3. LHow much money has DNDO spent on ASP R and D to date?

    A3. $181M.

    Q4. LWhat is the proper role for DNDO in this process?
        Should DNDO be simply testing moderate changes to
    Commercial Off The Shelf Technology, or should it be focusing
    on next generation technology?

    A4. The Congressional and Executive guidance establishing DNDO,
    specifically the SAFE Port Act of 2006, the National Security
    Presidential Directive 43, and Homeland Security Presidential
    Directive 14, charge DNDO with maturing ``transformational
    technologies'' as well as to ``develop . . . and enhance
    national nuclear and radiological capabilities.''

    Q5. LWho should be testing COTS, CBP or DNDO?

    A5. Three entities within DHS cooperate in the testing of COTS
    (and GOTS) systems: DNDO, the end-user (in the case of ASP this
    would be CBP), and S&T. CBP must ensure that the testing of
    systems incorporates realistic operational scenario, DNDO must
    ensure that the testing properly addresses the detection and
    identification requirements of the system within the GNDA and
    also must ensure that any developmental testing has been
    completed prior to Operational Testing. S&T is the DHS
    Operational Testing Authority and as such must assess the
    suitability and effectiveness of the final system configuration
    in real operational environments.

    Q6. LIn addressing previous problems indentified in Field
    Validation Testing, has the agency changed the underlying
    algorithms that were tested at the Nevada Test Site (NTS)?

    A6. No.

    Q7. LWill simply changing isotope thresholds be enough to fix
    these problems, or will algorithms need to be changed?

    A7. As demonstrated by replay and regression testing, we are
    convinced that threshold adjustments, while reducing
    sensitivity, will reduce the false alarm rate while maintaining
    adherence to the ASP specification.

    Q8. LIf algorithms are changed, will the equipment have to go
    back to the NTS?

    A8. If the Department determines that changes, if they are to
    occur, warrant a return, to NTS, then such testing will be
    conducted.

    Q9. LIf so, how much will this cost and how long will it take?
        Is the Department only changing threshold levels (and not
    changing algorithms) simply because it does not want to go back
    to NTS?

    A9. The cost and schedule of such tests, if needed, would be
    dictated by the type of testing required. ASP thresholds were
    changed because the algorithm is performing well, and because
    no reasons to make changes to the algorithm have been
    identified. The thresholds were changed because real-world
    field validation showed that they were not previously set in
    the best configuration. The Department has not intentionally
    avoided making any necessary changes to the algorithm to avoid
    returning to NTS.

    Q10. LAt our previous hearing, we discussed the possibility of
    deploying the Low Rate Production (LRP) ASPs that are currently
    in storage in tandem with PVTs in order to get more data.
        Does the Department currently have the authority to do
    this?

    A10. The FY 2010 Appropriations Bill, and accompanying
    Conference Report, addresses low rate initial production ASP
    systems. Specifically, H.Rept. 111-298 states that, ``the
    conferees encourage DNDO to undertake deployment of low rate
    initial production ASP systems, as appropriate, and use data
    from such deployments to inform future portal monitor
    decisions.''

    Q11. LHas the Department made any progress on this front?

    A11. The Department is working through the logistics of
    completing operational testing to get to Secretarial
    certification before making an LRIP deployment decision.

    Q12. LHow much would this cost and how long would it take?

    A12. There are eight (8) LRIP systems available for deployment.
    Approximately $3 million would be needed for installing and
    maintaining the systems for 1 year. Installation of the 8
    systems would take place over a six (6) month window. Data
    collection and data analysis would take approximately 6-12
    months and would cost $100K-$200K per system. Total for all
    activities would be less than $5 million.

    Q13. LCBP issued a RFI for ASPs several years ago only to have
    it rescinded by UNDO after the office was formed.

        In retrospect, do you believe the Request For Information
    (RFI) should have gone forward, with an eventual Request for
    Proposal (RFP) after that?

    A13. Given the urgency to address perceived limitations in
    fielded systems at the time, DHS still believes that those
    decisions were appropriate. It is difficult to gauge what may
    have come from an RFI/RFP process from 2003. CBP believes that
    the success of any research and development program, leading to
    possible deployments of next generation equipment, must begin
    with requirements defined by the end user/operator, and robust
    field evaluations/development.

    Q14. LDo you believe the requirements developed by DNDO will
    create a system that can stand up to a cost-benefit analysis?

    A14. An initial cost-benefit analysis, based on performance
    assumptions and systems requirements developed by DHS,
    warranted a decision to proceed with the program. Performance
    testing has shown potential benefits for ASP systems over
    current systems, particularly in secondary screening. However,
    the Department will not make any predictions as to the results
    of a final cost-benefit analysis, based on performance data,
    until such all testing and analysis is complete

    Q15. LShould the cost-benefit analysis consider upgrades in
    current Radioisotope Identification Device (RIID) technology
    and Polyvinyl Toluene (PVT) ``energy windowing'' as options?

    A15. No. The CBA will assess the cost and benefits of the ASP
    vs. the current system (which currently utilizes energy
    windowing). Should the ``current system'' change before the CBA
    is complete, the CBA would be adjusted accordingly.

    Q16. LHow is the Department responding to the recommendations
    of GAO and the National Academy of Sciences?

    A16. DNDO has appropriately incorporated or responded to the
    recommendations of both organizations as follows.

        a. Responding to the GAO

        Recommendation regarding significant improvement to
    operational effectiveness (SIOE): The GAO Recommended DNDO
    assess whether ASPs meet SIOE criteria based on valid
    comparison with PVT's full performance potential, including use
    of energy windowing to increase sensitivity to threats. DNDO
    has structured ASP testing to validate that ASP meets SIOE
    criteria. The SIOE criteria require that the comparison be made
    against the ``current system.'' DNDO will and should follow
    this path. We stress that the current PVT based system contains
    energy windowing algorithms that were updated in 2007. DNDO has
    also funded additional research at both Pacific Northwest
    National Laboratory and Johns Hopkins University Applied
    Physics Laboratory to further investigate the potential to
    improve performance of PVT systems based upon algorithm
    enhancements, including new approaches to energy windowing. On
    a side note, energy windowing techniques may also be applied to
    spectroscopic systems like the ASP, which may enhance
    performance there as well.
        Recommendation to revise ASP testing and Certification
    schedules: The GAO recommended altering schedules to allow
    sufficient time for review and analysis of results from final
    testing including injection studies. The ASP program schedule
    incorporates time following completion of Field Validation and
    Operational Testing of
        ASP, before presenting ASP to the DHS Acquisition Review
    Board for approval and then to the DHS Secretary for
    certification. This time interval prior to a decision allows
    all stake holders to thoroughly review test results and for
    proper consideration of the ASP cost-benefit analysis.

        b. Responding to the NAS

        Recommended approach for testing and evaluation: The NAS
    recommended an ``iterative approach with modeling and physical
    testing complimenting each other.'' In this approach, ``theory
    and models of threat objects, radiation transport, and detector
    response'' should be used to ``simulate performance and predict
    outcome,'' using ``physical experiments to validate the
    predictions and allow critique of the models.'' We believe that
    the ASP program, through development activities carried out by
    vendors, to test programs executed by the government, to replay
    tools and injection tools used for analysis, has followed the
    spirit of this recommendation. The performance of the ASP
    systems in controlled performance tests has mirrored
    predictions extremely well. The issue that remains for ASP is
    one of threshold settings against real-world cargo which must
    be determined through empirical evidence.
        Recommended Approach of the Procurement Process: The NAS
    recommended that DNDO employ an ``iterative'' approach for the
    procurement process. Rather than a ``onetime certification
    decision in the near future,'' the NAS recommended a continuous
    process of improvement and adaptation of the system, beginning
    with the deployment of LRIP systems. We agree with the NAS in
    principle and are evaluating this approach. This approach is
    consistent with the ASP Acquisition Plan and will be reflected
    in actual deployments when they begin.
        Recommended Approach for Cost-Benefit Analysis: The NAS
    Committee recommended that DHS not proceed with further
    procurement until ``ASP is shown to be a favored option in the
    cost-benefit analysis.'' This recommendation is incorporated
    into the governance structure of the ASP program. Additionally
    the NAS made recommendations about the nature of the CBA
    itself, stating that it ``needs to include three key elements:
    (1) a clear statement of the objectives of the screening
    program: (2) an assessment of meaningful alternatives to
    deploying ASPs; and (3) a comprehensive, credible and
    transparent analysis of in-scope benefits and costs.'' We
    continue to agree with the NAS that a complete CBA is necessary
    before any decisions are made and the elements suggested by NAS
    are factored into the CBA.

    Q17. LSetting aside the increased costs of ASPs over the
    current PVT/RIDD combination, how much more money will ASP
    Operations and Maintenance (0 and M) cost CBP?

    A17. Current PVT RPM units cost approximately $12,000 per year
    to operate and maintain. While CBP has not seen the final ASP
    maintenance cost estimate, this figure is expected to range
    from $65-100k each year per unit.

    Q18. LGiven dwindling budget estimates, how will these
    increased O&M costs affect overall security?

    A18. If no increase in O&M funds accompanies the deployment of
    ASPs, CBP would have to secure funding from other programs or
    staff positions.

    Q19. LWill CBP even be able to operate ASPs in the current
    budget environment if they are certified?

    A19. CBP anticipates that the appropriate O&M funding would be
    included should the ASPs be deployed.

    Q20. LIf the ASP systems are certified and deployed, will the
    Department be able to find cost savings?

    A20. The deployment of ASPs will result in an overall increase
    in life cycle costs.

    Q21. LDoes the ASP system require less CBP personnel to
    operate?

    A21. There is no appreciable savings in required personnel over
    the entire deployed architecture. In primary scanning
    operations, for example, land border crossings would not see a
    staff reduction. In secondary, the ASPs may require less
    physical labor and less time to operate; however, a CBP Officer
    will still be required to operate the ASPs. It should be noted
    that the ASP will not eliminate the need for RIIDs which will
    be used for more detailed inspections and alarm adjudication.
        Only very limited reductions likewise would be seen in
    seaport deployments.

    Q22. LDoes the ASP system allow for greater flow of commerce,
    or would other factors at ports and border crossings still
    impede commerce?

    A22. No appreciable gains to the flow of commerce would be
    realized over the entire deployed architecture. At land border
    crossings, impediments to the flow of commerce are due to non-
    radiation scanning processes (i.e.--passport control and
    antismuggling inspections). At seaports, the flow of commerce
    is mostly determined by individual terminal procedures.
                       Answers to Post-Hearing Questions
    Responses by Gene Aloise, Director, Natural Resources and Environment,
            U.S. Government Accountability Office (GAO); Todd C. Owen,
            Acting Deputy Assistant Commissioner, Office of Field
            Operations, U.S. Customs and Border Protection, Department of
            Homeland Security; and William K. Hagan, Acting Deputy
            Director, Domestic Nuclear Detection Office, Department of
            Homeland Security

    Questions submitted by Representative Paul C. Broun

    Questions for Mr. Aloise, Dr. Hagan, and Mr. Owen:

    Q1.  Do you believe computer modeling through replay tools, injection
    studies, or regression tools is enough for certification, or do you
    believe real world field testing is required?

    A1. The current campaign of ASP testing, which started in early 2008,
    involved several different types of testing each for its own purpose.
    The August 2008 testing at the Nevada Test Site (NTS) was designed to
    test sensitivity of the ASPs (and PVTs) against actual physical sources
    of special nuclear materials (e.g., uranium and plutonium). For the
    current campaign of testing, the NTS phase of testing was particularly
    important given that the ASPs were expected to have an improved
    capability of detecting special nuclear materials, and, for security
    reasons, NTS was the only place where DNDO could fully physically test
    the ASPs capability of detecting these very dangerous materials.
        To this point, the field validation tests have shown an
    unacceptable level of false alarms for the presence of certain nuclear
    materials. To resolve this, DNDO officials told us that they have made
    changes to the thresholds of the ASP, but not to the underlying
    algorithms (or software) that allows these machines to detect
    radiation. As we testified, this change makes the ASP less capable of
    detecting certain nuclear materials, raising questions about whether
    the ASPs represent a significant improvement in operational
    effectiveness over the PVTs in detecting certain nuclear materials.
        While DNDO officials told us that they were not making changes to
    the ASP algorithms, the report of the Field Validation Advisory Panel
    stated that ``correcting algorithm issues will change ASP referral
    rate'' suggesting that there will be future changes to the ASP
    algorithms. So, there is some confusion about exactly what changes will
    need to be made in order to successfully complete field validation
    testing. This represents a subtle, but important distinction--we regard
    ``algorithm corrections'' as algorithm changes and not simple threshold
    adjustment.
        In our view, prior to any certification decision to move forward
    with procurement of ASPs, there should be additional testing at NTS
    against actual physical sources of special nuclear materials in order
    to (1) understand exactly how the modified ASPs performance compares
    against the performance of PVTs in detecting certain nuclear materials,
    and (2) assure that there are not any unintended consequences in
    detection capabilities resulting from the modifications made to the
    ASPs since the August 2008 testing at NTS.\1\ This is to say, at this
    stage in the development of the ASP, major systems acquisition best
    practices require performance testing with actual special nuclear
    material because of the expected corrections to the software (beyond
    simple threshold adjustment).
    ---------------------------------------------------------------------------
        \1\ Without tests at NTS we will not have True Positive or False
    Negative data with the new software.

    Q2a.  Has other ASP equipment been developed outside of this process by
    ---------------------------------------------------------------------------
    the private sector?

    A2a. Based on the number of proposals submitted to DNDO's initial RFP
    for the ASP, it is quite possible that spectroscopic radiation
    detection systems have been developed outside of DNDO sponsorship. The
    issue here is determining the extent to which those COTS vendors have
    focused development on spectroscopic radiation detection systems that
    could operate in a high-impact port environment (large format and high
    traffic throughput) in support of CBP's mission. Moreover, only DNDO's
    two developers have been afforded the ability to test their systems
    with actual special nuclear material at the NTS.

    Q2b.  How much money has DNDO spent on ASP R&D to date?

    A2b. DNDO would be the best source for this information.

    Q2c.  What is the proper role for DNDO in this process? Should DNDO be
    simply testing moderate changes to Commercial off the Shelf Technology
    or should it be focusing on next generation technology?

    A2c. We believe the Secretary of DHS should determine whether DNDO
    should be a technology development or a major systems acquisition
    organization. If she chooses to keep both functions within the
    organization, then the Secretary should establish a strong separation
    and independent operational construct between the technology
    development and major systems acquisition missions. Ultimately, DNDO's
    goal should be toward addressing the mission needs of the end-user (CBP
    in the case of the ASP program). Organizational best practices hold
    that DNDO should regularly assess the extent to which these needs may
    be met with existing commercial-off-the-shelf technologies, as well as
    the nature and extent of development necessary for emerging or next-
    generation technologies. In the case of ASPs, the mission need has
    become less clear based upon the lessening of the criteria for a
    ``significant increase in operational effectiveness.'' This is to say,
    the primary mission need DNDO put forward was to better detect certain
    special nuclear materials, and the expectation was established that
    ASPs would vastly outperform PVTs in this area as manifested by a
    dramatic reduction of referrals to secondary screening due to NORM.
    However, recent testing has raised questions about both the ASPs
    capabilities to reliably and efficiently detect these materials as well
    as the exact significance of the increase in operational effectiveness
    of these next generation systems that are 2.5 times more expensive per
    unit than the current generation of technology. Moreover, it is not
    clear that the PVT technology has reached its limit of performance
    based upon energy windowing algorithms or other advances in materials
    science for nuclear detection.

    Q2d.  Who should be testing COTS, DNDO, or CBP?

    A2d. GAO sees three parts of a testing regime for these systems: (1)
    software development testing, (2) physical performance testing at NTS
    (supported by follow-up modeling and simulation work), and (3) field
    validation testing. In our view, the first is managed by DNDO and its
    contractors with independent testing and evaluation done by a federally
    funded research and development center (FFRDC), university affiliated
    research center (UARC), or the National Institute of Standards and
    Technology (NIST). The second is also managed by DNDO in partnership
    with the National Nuclear Security Administration's staff at NTS as
    well as the OT&E group of DHS and/or NIST (simulation work can be
    supported by national labs or UARCs). The third should be managed by
    CBP with support from DHS OT&E as the end-user since they are
    responsible for screening operations at a working port of entry.

    Questions for Mr. Aloise and Dr. Hagan:

    Q3a.  In addressing previous problems identified in Field Validation
    Testing, has the agency changed underlying algorithms that were tested
    at the Nevada Test Site (NTS)?

    A3a. As discussed above, while DNDO officials told us that they were
    not making changes to the ASP algorithms, the report of the Field
    Validation Advisory Panel stated that ``correcting algorithm issues
    will change ASP referral rate'' suggesting that there will be future
    changes to the ASP algorithms. So, there is some confusion about
    exactly what changes will need to be made in order to successfully
    complete field validation testing.

    Q3b.  Will simply changing isotope thresholds be enough to fix these
    problems, or will algorithms need to be changed?

    A3b. Only additional testing can answer this. However, once the problem
    with false alarms is fixed, the ability of the ASPs to detect certain
    nuclear materials will be diminished. Understanding whether the ASP can
    detected certain nuclear materials better than the PVTs will be an
    important part of deciding whether to move forward with procurement of
    the ASPs.

    Q3c.  If algorithms are changed, will the equipment have to go back to
    the NTS?

    A3c. In our view, prior to any decision to move forward with
    procurement of ASPs, there should be additional testing at NTS against
    actual physical sources of special nuclear materials in order to (1)
    understand exactly how the modified ASPs performance compares against
    the performance of PVTs in detecting certain nuclear materials, and (2)
    assure that there are not any unintended consequences in detection
    capabilities resulting from the modifications made to the ASPs since
    the August 2008 testing at NTS.

    Q3d.  If so, how much will this cost and how long will it take?

    A3d. DNDO would be the best source for this information. However, we
    were previously told by DNDO that NTS testing costs about $500,000 per
    week. In our view, this testing, while expensive, would be worthwhile
    prior to DNDO committing billions of dollars toward procurement of the
    ASPs.

    Q3e.  Is the Department only changing threshold levels (and not
    changing the algorithms) simply because it does not want to go back to
    NTS?

    A3e. DHS would be the best source for this information.