    Risk Management: Further Refinements Needed to Assess Risks and
    Prioritize Protective Measures at Ports and Other Critical
    Infrastructure (15-DEC-05, GAO-06-91).

    Congress and the President have called for various homeland
    security efforts to be based on risk management--a systematic
    process for assessing threats and taking appropriate steps to
    deal with them. GAO examined how three Department of Homeland
    Security (DHS) components were carrying out this charge: the
    Coast Guard, which has overall responsibility for security in the
    nation's ports; the Office for Domestic Preparedness (ODP), which
    awards grants for port security projects; and the Information
    Analysis and Infrastructure Protection Directorate (IAIP), which
    has responsibility for developing ways to assess risks across all
    types of critical infrastructure. GAO's work focused on
    identifying the progress each DHS component has made on risk
    management and the challenges each faces in moving further.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-91
        ACCNO:   A43417
      TITLE:     Risk Management: Further Refinements Needed to Assess
    Risks and Prioritize Protective Measures at Ports and Other
    Critical Infrastructure
         DATE:   12/15/2005
      SUBJECT:   Agency missions
    	     Critical infrastructure
    	     Critical infrastructure protection
    	     Homeland security
    	     Performance measures
    	     Port security
    	     Risk assessment
    	     Risk management
    	     National Infrastructure Protection Plan

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-91

         * Background
         * Results in Brief
         * Principal Findings
              * Coast Guard Has Made Progress in Using Risk Management, but
              * ODP's Port Security Grant Program Illustrates Both Progress
              * IAIP's Progress in Carrying Out Risk Management Has Been Lim
              * Overall Observations
         * Recommendations for Executive Action
         * Agency Comments and Our Evaluation
         * Risk Management Has a Long History of Use in Industry and Go
         * Application of Risk Management to Homeland Security Is Widel
         * Department of Homeland Security Has Broad and Challenging Re
         * Seaports Are an Important Focus in the Homeland Security Res
         * A Framework for Risk Management
              * Strategic Goals, Objectives, and Constraints
              * Risk Assessment
              * Alternatives Evaluation
              * Management Selection
              * Implementation and Monitoring
         * Objectives, Scope, and Methodology
         * Coast Guard Homeland Security Activities Revolve Heavily aro
         * Goal-Setting: High-Level Goals Are in Place, with Efforts Un
         * Risk Assessments: Progress Has Been Substantial, but Challen
              * Several Sets of Assessments Have Been Completed at the Port
              * Key Challenges Involve Improving Data and Methodology
                   * Limitations of Data on Threats, Vulnerabilities, and Consequ
                   * Methodological Limitations
         * Evaluation of Alternatives: Ability to Evaluate Alternatives
         * Management Selection: Local Strategies Have Been Selected bu
         * Implementation and Monitoring: A Foundation for Continuous I
         * Conclusions
         * Recommendations for Executive Action
         * Agency Comments and Our Evaluation
         * ODP Manages Port Security Grants
         * Goal Setting: Overall Goals Have Been Set; Developing Perfor
         * Risk Assessments: The Funding Distribution Model Is Becoming
              * Procedures for the Fiscal Year 2005 Port-Level and National-
              * Challenges Remain for Improving Methods and Data
                   * Incomplete Data on Threats, Vulnerabilities, and Consequence
              * Methodological Limitations
         * Alternatives Evaluation: ODP Recognizes the Importance of Ev
         * Management Selection: ODP Has Addressed Problems in Document
         * Implementation and Monitoring: ODP Has Instituted a Monitori
         * Conclusions
         * Recommendations for Executive Action
              * Agency Comments and Our Evaluation
         * IAIP Plays a Key Role in Evaluating Risk across Infrastructu
         * Strategic Goals, Objectives, and Constraints: High-Level Goa
         * Risk Assessment: IAIP Faces Substantial Gaps and Has Made Li
         * Progress Is Greatest on Database of Critical Assets
              * Development of Threat Data Faces Challenges
         * Methodology for Cross-Sector Comparisons Has Experienced Set
         * Evaluating Alternatives: Lack of Guidance and Consensus on C
         * Management Selection: IAIP Challenged in Protecting Infrastr
         * Implementation and Monitoring: IAIP Is Also Challenged by It
         * Conclusions
         * Recommendations for Executive Action
         * Agency Comments and Our Evaluation
         * First Observation: Much Remains to Be Done
         * Observation Two: Progress Varies by Component and Reflects K
         * Observation Three: In the Short Term, Further Progress Is St
         * Observation Four: In the Long Term, Progress Rests Heavily o
         * Recommendations for Executive Action
         * Agency Comments and Our Evaluation
         * General Lack of Uniform Guidance on Risk Management
         * Methodology for Developing a Risk Management Framework
         * A Risk Management Framework
         * Strategic Goals, Objectives, and Constraints
         * Risk Assessment
         * Alternative Evaluation
         * Management Selection
         * Implementation and Monitoring
         * Difficulties Applying a Risk Management Framework
         * Glossary of Risk Management Terms
         * GAO Contacts
         * Staff Acknowledgments
              * Critical Infrastructure Protection
              * Maritime Security
              * Risk Management
         * GAO's Mission
         * Obtaining Copies of GAO Reports and Testimony
              * Order by Mail or Phone
         * To Report Fraud, Waste, and Abuse in Federal Programs
         * Congressional Relations
         * Public Affairs

    Report to Congressional Requesters

    United States Government Accountability Office

    GAO

    December 2005

    RISK MANAGEMENT

    Further Refinements Needed to Assess Risks and Prioritize Protective
    Measures at Ports and Other Critical Infrastructure

    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management Risk Management Risk Management Risk Management Risk
    Management Risk Management Risk Management Risk Management Risk Management
    Risk Management

    GAO-06-91

    Contents

    Letter 1

    Executive Summary 3
    Background 4
    Results in Brief 6
    Principal Findings 9
    Recommendations for Executive Action 14
    Agency Comments and Our Evaluation 15
    Chapter 1 Introduction Risk Management Is a Key Tool for Homeland Security
    16
    Chapter 2 The Coast Guard Has Made Progress in Using Risk Management, but
    Challenges Remain 30
    Chapter 3 Stronger Risk Management Approach Could Improve the
    Accountability of the Port Security Grant Program 49
    Chapter 4 IAIP Faces Challenges in Meeting Risk Management
    Responsibilities across All Sectors of the Nation's Infrastructure 68
    Chapter 5 Overall Observations and Recommendations 88
    Appendix I A Risk Management Framework 99
    Appendix II Comments from the Department of Homeland Security 113
    Appendix III GAO Contacts and Staff Acknowledgments 116
    Related GAO Products 117

    Tables

    Table 1: Examples of Risk Management in the Private and Public Sectors 18
    Table 2: Summary of Progress Made and Challenges That Remain in the Coast
    Guard's Risk Management Approach 31
    Table 3: Port-Level Assessments Conducted by the Coast Guard 37
    Table 4: National-Level Assessments Conducted by the Coast Guard 38
    Table 5: Examples of Data-Related Challenges in Coast Guard Risk
    Assessments 39
    Table 6: Summary of Progress and Challenges in the Port Security Grant
    Program 50
    Table 7: Examples of Data-Related Challenges in ODP Risk Assessments 59
    Table 8: Examples of Changes in Funding Decisions 64
    Table 9: Summary of Progress Made and Challenges That Remain in IAIP's
    Risk Management Approach 69
    Table 10: Critical Infrastructure Sectors and Lead Federal Agencies 72
    Table 11: A Risk Management Framework 103

    Figures

    Figure 1: Risk Management Framework 6
    Figure 2: A Framework for Risk Management 24
    Figure 3: Facilities at One of the Nation's Major Ports 52
    Figure 4: Sources of Evaluation Criteria Associated with Risk Management
    Phases 102

    Abbreviations

    ASME American Society of Mechanical Engineers

    CIP critical infrastructure protection

    COSO Committee of Sponsoring Organizations

    DHS Department of Homeland Security

    HITRAC Homeland Infrastructure Threat and Risk Analysis Center

    HSPD Homeland Security Presidential Directive

    IAIP Information Analysis and Infrastructure Protection Directorate

    IG Inspector General

    MARAD Maritime Administration

    MSRAM Maritime Security Risk Analysis Model

    MTSA Maritime Transportation Security Act

    NADB National Assets Database

    NIPP National Infrastructure Protection Plan

    ODP Office for Domestic Preparedness

    OMB Office of Management and Budget

    PS-RAT Port Security Risk Assessment Tool

    PWCS Ports, Waterways, and Coastal Security

    RAMCAP Risk Analysis and Management for Critical Asset Protection

    TSA Transportation Security Administration

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States Government Accountability Office

    Washington, DC 20548

    December 15, 2005

    The Honorable Henry A. Waxman Ranking Minority Member Committee on
    Government Reform House of Representatives

    The Honorable C. A. Dutch Ruppersberger House of Representatives

    The threat of terrorism presents a number of risks to our nation's
    seaports and other types of critical infrastructure. The Department of
    Homeland Security (DHS) has three components responsible for the security
    of critical infrastructure related to ports and other facilities. The U.S.
    Coast Guard has responsibility for port security overall. The Office for
    Domestic Preparedness (ODP) is responsible for providing port security
    grants to selected maritime facility owners. The Information Analysis and
    Infrastructure Protection (IAIP) Directorate is responsible for working
    with other federal, state, local, and private organizations to identify
    and protect critical infrastructure across the nation. Risk management is
    a tool for assessing risks, evaluating alternatives, making decisions, and
    implementing and monitoring protective measures. This report provides an
    evaluation of the progress made, and challenges faced, by the Coast Guard,
    ODP, and IAIP in using risk management to improve homeland security.

    As agreed with your offices, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days
    after its issue date. At that time, we will provide copies of this report
    to appropriate departments and interested congressional committees. This
    report will also be available at no charge on the GAO Web site at
    http://www.gao.gov.

    If you or your staff have any questions about this report, please contact
    me at (415) 904-2200 or at wrightsonm@gao.gov . Key contributors to this
    report are listed in appendix III.

    Sincerely yours,

    Margaret T. Wrightson Director, Homeland Security and Justice Issues

    Executive Summary Executive Summary

    Risk management, a strategy for helping policymakers make decisions about
    assessing risks, allocating resources, and taking actions under conditions
    of uncertainty, has been endorsed by Congress and the President as a way
    to strengthen the nation against possible terrorist attacks. Risk
    management has long been used in such areas as insurance and finance, but
    its application to domestic terrorism has no precedent. Unlike storms and
    accidents, terrorism involves an adversary with deliberate intent to
    destroy, and the probabilities and consequences of a terrorist act are
    poorly understood and difficult to predict. The size and complexity of
    homeland security activities and the number of organizations involved-both
    public and private-add another degree of difficulty to the task. The task
    of managing this complexity centers on the Department of Homeland
    Security, which since its inception in March 2003 has been faced with the
    challenge of transforming 22 agencies into an organization that can plan,
    manage, and carry out operations effectively. Congress likewise has a key
    oversight role to play in ensuring that DHS's course regarding risk
    management reflects a consensus as to the most prudent and cost-effective
    course of action.

    To assist Congress in its oversight, this report focuses on the progress
    made by three DHS components in applying risk management to homeland
    security activities and the challenges each component faces in moving
    further ahead. For two of these components, GAO's review dealt
    specifically with their risk management activities at the nation's
    seaports, while the review for the third component encompassed a wider
    range of infrastructure. GAO decided to focus a considerable amount of
    this review on seaport security because seaports have been viewed as
    potential terrorist targets or as conduits for importing a weapon of mass
    destruction or where terrorists may enter the country. GAO's focus on
    these three components, while not a comprehensive look across the entire
    department, provides perspective on the degree of progress made thus far.
    Risk management has applications for deliberate acts of terror as well as
    natural disasters, such as hurricanes and earthquakes. GAO's research,
    which was conducted prior to Hurricane Katrina, focused on preparations
    for terrorist attacks, not natural disasters. The three components GAO
    studied are:

               o  the Coast Guard, the lead federal agency for port security and
               the agency responsible for developing and coordinating various
               risk-based assessments of critical infrastructure in and around
               ports;

               o  the Office for Domestic Preparedness, administrator of the port
               security grant program, has awarded more than half a billion
               dollars in federal grants to owners and operators of port
               facilities and vessels; and

               o  the Information Analysis and Infrastructure Protection
               Directorate, which has been charged with establishing uniform
               policies, approaches, guidelines, and methodologies for
               integrating infrastructure protection and risk management
               activities within and across key sectors, such as energy, defense,
               and transportation, including airports, railroads, and ports.1

    Besides describing the progress of and challenges for each component, this
    report also presents GAO's observations about what the three components'
    efforts indicate collectively, both with regard to how far the department
    has come in managing homeland security efforts on the basis of risk and
    what steps could help advance the current level of progress.

                                       Background

    Seaport security receives particular attention in this report because
    seaports are widely viewed as representing attractive terrorist targets,
    in part because of their importance to the economy. More than 95 percent
    of the nation's non-North American foreign trade (and 100 percent of
    certain commodities, such as foreign oil) arrives by ship. The estimated
    economic consequences of a successful attack and resulting shutdown of
    this system total billions of dollars. Ports also represent attractive
    targets because they contain a myriad of vulnerabilities. In all, the
    nation's 300-plus ports have about 3,700 cargo and passenger terminals.
    Chemical factories, oil refineries, power plants, and other facilities are
    often located in port areas and add another set of possible targets. Roads
    crisscross many ports, allowing access by land as well as by water, and
    the number of people working in or traveling through ports is in the
    millions. The Coast Guard has the major responsibility for seaport
    security, and the port security grant program administered by ODP adds to
    the resources available for port security projects.

    Relative to the Coast Guard and ODP, IAIP's homeland security
    responsibilities are by far the widest-ranging. The Homeland Security Act
    of 2002 and Homeland Security Presidential Directive 7 (HSPD-7) charge
    IAIP with establishing a risk management framework across the federal
    government to protect the nation's critical infrastructure and key
    resources.2 The scope of this effort is immense, and the effort is one of
    IAIP's central responsibilities. IAIP's task ultimately involves
    developing an approach that can inform decisions on what the nation's
    antiterrorism priorities should be and identifying what strategies and
    programs will do the most good. IAIP's work is done in a setting where
    numerous and substantial gaps in security remain, but resources for
    closing these gaps are limited. More specifically, IAIP is charged with
    examining and comparing relative risks associated with a multitude of
    possible targets, ranging from specific structures (such as dams, chemical
    plants, and nuclear power plants) to major sectors of national
    infrastructure (such as the banking system, computer networks, and water
    systems). IAIP is also responsible for developing policies and guidance
    that other agencies can use in conducting their own risk assessments.

    1On November 14, 2005, DHS reorganized the department. ODP and the
    Infrastructure Protection part of the former IAIP Directorate are now
    components in the Preparedness Directorate. We recognize the recent
    organizational changes, but because ODP and IAIP carried out the work we
    reviewed, we have not changed the name or organizational posture of these
    DHS components in our report.

    While federal law and the presidential directive call for the use of risk
    management in homeland security, little specific federal guidance or
    direction exists as to how risk management should be implemented. To
    provide a basis for analyzing component efforts, GAO developed a framework
    for risk management based on industry best practices and other criteria.
    This framework, shown in figure 1, divides risk management into five major
    phases: (1) setting strategic goals and objectives, and determining
    constraints; (2) assessing the risks; (3) evaluating alternatives for
    addressing these risks; (4) selecting the appropriate alternatives; and
    (5) implementing the alternatives and monitoring the progress made and
    results achieved. For all three components, GAO applied this framework
    after conducting a wide range of interviews with officials, reviewing
    plans and activities of the three components, and visiting port locations.
    As part of our work, GAO briefed officials of the three components about
    the various phases of the framework and the officials generally agreed
    with its structure and intent. The application of risk management to
    homeland security is relatively new, and the framework will likely evolve
    as processes mature and lessons are learned.

    2The Homeland Security Act incorporates the definition of "critical
    infrastructure" used in the USA PATRIOT Act of 2001, meaning "systems and
    assets, whether physical or virtual, so vital to the United States that
    the incapacity or destruction of such systems and assets would have a
    debilitating impact on security, national economic security, national
    public health or safety, or any combination of those matters." The
    Homeland Security Act defines "key resources" as "publicly or privately
    controlled resources essential to the minimal operations of the economy
    and government." 6 U.S.C. S: 101.

    Figure 1: Risk Management Framework

                                    Results in Brief

    Of the three components GAO reviewed, the Coast Guard had made the most
    progress in establishing a foundation for using a risk management
    approach; its next challenges are to further refine and enhance its
    approach. While the Coast Guard has made progress in all five risk
    management phases, its greatest progress has been made in conducting risk
    assessments-that is, evaluating individual threats, the degree of
    vulnerability, and the consequences of a successful attack. However, the
    assessments are limited in their reliability and completeness, and better
    coordination will be needed with the intelligence community so that
    analysts can develop models that better assess the relative probability of
    various threat scenarios. The Coast Guard has developed the ability to
    compare and prioritize risks at individual. However, it cannot yet compare
    and prioritize relative risks of various infrastructure across ports.
    Other challenges include developing performance measures to go along with
    the more general goals already developed for the port security mission,
    further integrating risk into the annual cycle of program and budget
    review, and developing formal policies for reviewing and improving the
    implementation of a risk management approach. The Coast Guard has actions
    under way to address the challenges it faces in each risk management
    phase. Several of these actions are based, in part, on briefings GAO held
    with agency officials.

    ODP has made progress in applying risk management to the port security
    grant program, but like the Coast Guard, it also faces challenges across
    all phases of the risk management framework. For example, ODP has set
    broad risk management goals and has placed more emphasis on using
    risk-based data in its assessments, but it lacks performance measures
    showing what specific outcomes the program aims to achieve, and it still
    faces challenges in such matters as comparing grant applications across
    ports. The grant awards for fiscal year 2004 also illustrate some of the
    challenges in ensuring that criteria for making the awards are transparent
    and consistent. At the end of what was, in part, a risk-based assessment
    process, ODP changed the criteria for awarding grants when it decided to
    give lower priority to applicants from large companies on the assumption
    that the companies were better able than other entities to pay for their
    own improvements. This changed 40 percent of the grants awarded, as
    projects with higher risk but greater potential for self-funding gave way
    to lower-risk projects with more limited funding prospects. In the
    procedures for fiscal year 2005 awards, ODP clarified the criteria it
    would use in making the awards.

    IAIP, which has the broadest risk management responsibilities of the three
    components and faces the greatest challenges, has made the least progress
    in carrying out its complex risk management activities. Its efforts are
    aligned with high-level strategic goals, but ways to measure performance
    in achieving these goals have yet to be developed. IAIP is not as far
    along as ODP and the Coast Guard in conducting risk assessments. While
    IAIP has provided input to ODP for its risk assessment efforts, IAIP's
    risk assessment responsibilities span much broader sectors of the nation's
    infrastructure than seaports alone, making its assessment activities more
    difficult. This difficulty is reflected in the limited progress made. With
    regard to its risk assessment responsibilities, IAIP has yet to
    successfully (1) develop data to determine the relative likelihood of
    various threat scenarios, (2) complete a methodology for comparing and
    prioritizing key assets, or (3) meet requirements set forth in HSPD-7 for
    issuing policies and guidance that other agencies can use in conducting
    their own risk assessments. For example, a DHS consultant issued a risk
    assessment methodology in 2004 for collecting data from industry, but
    adverse comments from reviewers have led to revisions that are still under
    way. IAIP is also challenged in its ability to translate these assessments
    into specific measures to be taken, because after IAIP makes decisions
    about what national priorities should be, it is dependent on the actions
    of others to carry them out. This is particularly true with regard to
    private sector assets where IAIP needs collaboration from the owners and
    operators of private sector infrastructure and their regulators.

    GAO made four main observations regarding the experience of these three
    components.

               o  A considerable degree of effort has been expended thus far, but
               much work remains to be done. This is particularly true in viewing
               risk management strategically-that is, with a view that goes
               beyond just assessing what the risks are and also integrating the
               consideration of risk into the annual cycle of program and budget
               review that is already in place.

               o  The varying degree of progress among the three components tends
               to reflect several characteristics of each component-how long it
               has been at the task of developing a risk management approach, how
               long it has existed as a component and is therefore able to
               function maturely, and how complex its risk management task is.
               For example, IAIP, which has made the least progress, is not only
               a new component established in 2003, but also has the most complex
               risk-related tasks of the three components-addressing risk not
               only at ports, but across all types of infrastructure and with
               multiple federal agencies and nonfederal stakeholders.

               o  In the near term, all three components' success in risk
               management will depend partly on continuing to make progress on
               the challenges described above. This involves continuing to work
               on such matters as performance measures, basic policies, and
               enhancements to existing risk assessment tools.

               o  The final observation is related to a critical longer-term
               need: more guidance and coordination from the department level,
               both to help ensure that individual components such as IAIP are
               carrying out their roles effectively and to ensure that the
               various responses from individual components mesh as effectively
               as possible with one another. In comparing the approaches
               developed by the three components, GAO noted ways in which their
               efforts were not consistent. The danger is that if components
               develop systems and methodologies that are inconsistent, they may
               end up with incompatible systems that have little or no ability to
               inform spending decisions on homeland security. The challenges
               associated with creating a department that can effectively
               administer a coherent risk management approach to the nation's
               homeland security have been widely acknowledged. IAIP recognizes
               that as DHS's individual components begin to mature in their risk
               management efforts, the need increases for ensuring consistency
               and coherence in these efforts. Supplying this necessary guidance
               and coordination is what DHS was set up to do.

                                   Principal Findings

    Coast Guard Has Made Progress in Using Risk Management, but Challenges Remain

    The Coast Guard has made progress across all five phases of risk
    management. In the first phase (goal and objective setting), the Coast
    Guard has established broad strategic goals for port security, including,
    in order of priority: (1) preventing terrorist attacks within, and
    terrorist exploitation of, the maritime domain and (2) reducing America's
    vulnerability to terrorism in the maritime domain. It faces challenges in
    developing objectives that translate these goals into more specific and
    measurable results. Coast Guard officials recognize that developing
    performance measures is a necessary next step and have actions under way
    to develop such measures.

    For the second phase, assessing risks, the Coast Guard has greatly
    expanded the scope of its risk assessment activities since the terrorist
    attacks of September 11, 2001. It has conducted three major security
    assessments at the port level, and collectively these assessments have
    resulted in considerable progress in understanding and prioritizing risks
    within a port. After it initiated port-level assessments, the Coast Guard
    expanded its analysis efforts to the national level to gain a more
    strategic perspective on port security. In all, the Coast Guard has
    conducted three major efforts at the national level, focusing more
    generally on understanding the risk posed to various classes of assets
    (such as bridges or container ships) by various types of attacks (such as
    using explosives or weapons of mass destruction). The assessments are
    limited in their reliability and completeness, however, in the degree to
    which the Coast Guard has (1) formal and systematic input from the
    intelligence community for modeling relative probability and likelihood of
    threat scenarios and (2) risk assessment tools allowing comparison and
    prioritization of specific infrastructure across ports. These limitations
    affect the degree to which the Coast Guard is able to determine how best
    to focus its attention on these threats that, from a national perspective,
    pose the greatest risk within the seaport sector. The Coast Guard has
    initiated actions to address these challenges. For example, the agency has
    initiated contact with the intelligence community to obtain better data on
    threat scenarios, and it plans to complete development of an assessment
    tool that will compare the relative risks of high-value assets at one port
    with risks of assets in a different port.

    Enhancing these first two phases of risk management is key to making
    additional progress on the next two phases-evaluating and selecting
    alternatives that reduce risk. While the Coast Guard's efforts have
    resulted in progress in identifying and evaluating alternatives at the
    individual port level, the lack of measurable objectives and sufficient
    information to fully depict threats, vulnerabilities, and consequences
    limits the ability to target the areas with the greatest gaps or produce
    the most cost-effective decisions. Similarly, buttressing annual budget
    review cycles with risk-based data is in its early development and more
    work remains to be done. Finally, with regard to the fifth
    phase-implementation and monitoring-the Coast Guard has implemented a
    number of activities to mitigate risks and has demonstrated the ability to
    evaluate its efforts and make improvements. The actions taken have
    included establishing maritime intelligence centers on the Atlantic and
    Pacific coasts and working closely with nonfederal stakeholders to reduce
    vulnerabilities in and around facilities and vessels. However, existing
    feedback mechanisms are insufficient to ensure that Coast Guard field
    personnel can make their headquarters managers aware of ways to improve
    the process. The Coast Guard recognizes the value of formal feedback loops
    as a means of improving its risk management processes, and it has plans to
    obtain formal feedback as part of its future efforts.

    ODP's Port Security Grant Program Illustrates Both Progress and Challenges in
    Implementing Risk Management

    Like the Coast Guard, ODP has made progress across all five phases of risk
    management. For example, for the first phase, it has set risk management
    goals that support broader maritime goals, such as protecting critical
    infrastructure in harbors, borders, ports, and coastal approaches. ODP has
    not begun to translate these broad goals into measurable objectives.
    Without them, it is difficult to know what progress has been made in
    reducing risk and what security gaps remain.

    ODP has carried out risk assessments, with input from the Coast Guard and
    IAIP, and evaluated mitigation alternatives-the second and third phases of
    the framework-to help determine which ports should receive priority for
    grants. Using risk assessments, ODP narrowed the number of ports eligible
    for grants from 129 to 66 for fiscal year 2005. Other recent steps include
    placing greater emphasis on using threat, vulnerability, and consequence
    data in prioritizing grant applications. Along with this progress,
    however, are several methodological challenges that limit such things as
    the usefulness of data received from intelligence agencies and ODP's
    ability to compare and prioritize risks among ports. For example, without
    data on the relative probability of various threat scenarios from the
    Coast Guard or IAIP, ODP may not target the most significant security
    needs. ODP has not yet developed approaches for addressing most of these
    challenges.

    While ODP has also made progress in developing a risk-based grant
    selection process and mechanisms to monitor what the grants accomplish,
    grant awards for fiscal year 2004 illustrate the challenges involved in
    actually making risk-based decisions. At the end of its process for
    determining which grants to fund, based in part on risk, ODP decided to
    give lower priority to grants involving projects at large companies, on
    the assumption that the companies were better able than other entities to
    pay for their own improvements. For example, one chemical company's
    application for $225,000 to purchase cameras, fencing, and barricades was
    initially ranked 25th out of 287 applications nationwide, but under the
    revised priorities its ranking fell to 236th. Projects initially ranked
    much lower received funding instead. For example, an application initially
    ranked as 279th out of 287 was approved for funding. In all, the
    application of non-risk criteria changed 40 percent of the grants awarded.
    ODP's changes affected the transparency and consistency of the awards
    process, in that (1) the criteria under which applications were submitted
    and initially considered were changed at the end of the process, and (2)
    the role of risk in evaluating the applications was obscured, because the
    resulting awards may not have addressed the most severe security gaps.
    Additionally, there is no guarantee that large companies would spend their
    own funds for security improvements, and it is unclear whether there are
    incentives, such as minimum standards for security, that would motivate
    them to do so. ODP issued revised criteria for fiscal year 2005 grants,
    and in doing so has made the process more transparent and consistent.

    IAIP's Progress in Carrying Out Risk Management Has Been Limited

    IAIP's progress in all five phases of risk management has been limited. It
    has made some progress in developing goals, having issued an Interim
    National Infrastructure Protection Plan in February 2005 that identifies a
    strategy for identifying, prioritizing, and coordinating the protection of
    critical infrastructure and key resources.3 The interim plan provides some
    guidance in meeting IAIP's broad responsibilities for identifying,
    comparing, and prioritizing critical assets, but it is not a comprehensive
    document, and IAIP faces several challenges in making it more
    comprehensive. These challenges are related to (1) developing performance
    measures that can be used in evaluating progress and (2) establishing
    milestones and time frames for processing and prioritizing assets across
    the many different infrastructure sectors.

    3In November 2005, DHS issued a revised Interim National Infrastructure
    Protection Plan for comment.

    IAIP's progress in risk assessment-the second phase of risk management-has
    been limited in several main respects. For example, IAIP has experienced
    difficulties in carrying out requirements of the Homeland Security Act of
    2002 that charged IAIP with the responsibility of conducting risk
    assessments of critical infrastructure and key resources to determine the
    risks of particular types of terrorist attacks. IAIP's original
    methodology for this task, called the Risk Analysis and Management for
    Critical Asset Protection, required extensive modification after its
    initial issuance in April 2004. IAIP now views it as a tool for engaging
    industry in a risk management dialogue with government. In September 2005,
    IAIP officials said they are developing a National Comparative Risk
    Assessment to meet the immediate need of examining risks within and across
    sectors, and they plan to complete an interim assessment by the end of
    2006. Challenges to carrying out this timetable include the need to obtain
    key information from other federal agencies and the fact that IAIP still
    needs to award the contract for this effort. One specific issue is the
    approach IAIP has taken in assessing the probability of various threat
    scenarios. The Homeland Security Act calls on IAIP to assess the
    probability of success of terrorist attacks, and during the course of
    GAO's review, IAIP officials said they recognize the importance of
    assessing the relative likelihood of an attack in meeting this
    requirement. IAIP officials said that the lack of intelligence analysis
    and data on such things as the capability and intent of terrorist groups
    hinders their ability to assess probability, but that work is under way in
    this regard. IAIP officials also pointed out that some inaccuracy is to be
    expected in examining the intent and capability of an adversary whose
    plans are concealed and that it will be important to reduce the potential
    of low-confidence assessments having undue influence when long-term
    investment decisions are made.

    IAIP's progress in the three other phases of risk management (evaluating
    alternatives, selecting a solution, and implementing and monitoring that
    solution) will remain limited, in part because of the points just
    discussed-performance goals and a complete risk assessment methodology are
    not in place. Beyond these limitations, however, IAIP faces additional
    challenges. For example, IAIP's role in selection, implementation, and
    monitoring is further complicated because in many instances, other
    entities have primary responsibility for selecting the solution. For
    example, other agencies, such as the Department of Defense or the
    Department of Energy, have primary responsibilities for some of the
    infrastructure sectors covered in IAIP's assessments. Additionally, much
    of the critical infrastructure is owned or operated by private industry,
    and while IAIP does not have authority over them, other federal agencies
    do have authority over infrastructure in specific sectors. This condition
    highlights the importance of coordination between IAIP and agencies with
    such regulatory authority. For example, the Nuclear Regulatory Commission,
    which issues licenses to nuclear power plants, has regulatory authority
    over security matters at these facilities. IAIP officials said they use
    their expertise and powers of persuasion to bring about specific actions
    but in most cases cannot compel others to adopt IAIP's recommendations.

    Overall Observations

    A great amount of effort has been applied. However, much more remains to
    be done than has been accomplished so far. Across all three components,
    the most progress has generally been made on fundamental steps, such as
    conducting risk assessments of individual assets, and less progress has
    generally been made on developing ways to translate this information into
    comparisons and priorities across ports and across infrastructure sectors,
    or applying it to new programs. Progress among the three components'
    efforts has been far from consistent and has tended to vary not only with
    the length of time the component has been using a risk-based approach, but
    also with the component's own maturity level and the complexity of its
    risk management task.

    With regard to next steps that would appear to add the most value to
    making further progress, one key observation is that in the short term,
    progress is heavily dependent on continuing to improve basic policies,
    procedures, and methods for applying risk management. Each component has
    an admittedly difficult set of challenges ahead, but progress has to be
    built on taking these incremental steps and doing them well. An area that
    needs further attention by all three entities is working with intelligence
    communities to develop improved analysis and data so that the relative
    probability of various threat scenarios can be further developed.

    The final observation is that in the longer term, progress will become
    increasingly dependent on how well the entire risk management effort is
    coordinated. While absolute compatibility among all components' efforts is
    likely impossible, even with components working in close cooperation,
    strong coordination is important to help ensure that component efforts are
    consistent rather than stovepiped. The risk management efforts GAO
    examined appeared to be fueled by a strong concern to make some headway,
    with coordination and interagency consistency a lesser concern. For
    example, the Coast Guard initiated efforts to set up a methodology for
    assessing and specifying risks before IAIP was created; and in the view of
    IAIP officials, it was important to proceed even though they recognized
    that doing so might lead to approaches that would not mesh cleanly with
    the approach IAIP would eventually develop. That approach appeared prudent
    in the short term, in that if the Coast Guard had waited to begin until
    guidelines had been set, it would still be waiting. Now, however, the need
    for coordination is looming larger, and coordination is essential to the
    success of efforts over time. Some of this coordination needs to come from
    IAIP, which is required under presidential directive to issue guidelines
    for other agencies to use, but it has yet to do so. Beyond IAIP, DHS has
    an active role in this regard. This is a key issue for the department as
    it moves from being an organization that is essentially in its early
    stages to one that is increasingly being expected to respond in a way that
    is more organizationally mature. Since 2003, GAO has designated the
    implementation and transformation of DHS as high risk because of the
    numerous challenges in transforming 22 agencies into one department and
    the serious implications of failure. Translating the concept of risk
    management into applications that are consistent and useful represents one
    of these challenges, and failure to effectively address this could have
    serious consequences for homeland security. In risk management, which the
    department has embraced as the guiding principle behind its policies and
    operations, IAIP's role is to act as an intra-agency and interagency
    coordinator of homeland security activities. Doing so will strengthen its
    ability to weigh risks and inform the decisions made across the homeland
    security responsibilities of the many agencies involved.

                          Recommendations for Executive Action

    GAO is making a number of specific recommendations to the Secretary of DHS
    with regard to the challenges faced by the three components. These
    recommendations, listed specifically at the end of the relevant chapters,
    cover such matters as developing performance goals and measures, improving
    risk assessment methodologies, working with intelligence communities to
    develop better data for risk assessment purposes, and (for IAIP)
    developing guidance for other agencies to use in evaluating risk and
    considering risk mitigation alternatives.

                           Agency Comments and Our Evaluation

    We provided DHS a draft of this report for its review and comment. DHS,
    including the Coast Guard, ODP, and IAIP, generally agreed with our
    findings and recommendations. For instance, DHS said that each DHS
    component we reviewed has actions under way to address recommendations
    made in the report. The comments from each component are summarized at the
    end of the relevant chapters. In addition to commenting on our findings
    and recommendations, DHS provided technical comments under separate cover,
    and we revised the draft report where appropriate. Written comments from
    DHS are reprinted in appendix II.

    Chapter 1: Introduction: Risk Management Is a Key Tool for Homeland
    Security Chapter 1: Introduction: Risk Management Is a Key Tool for
    Homeland Security

    This is a report about the nation's progress in applying risk management
    to key aspects of homeland security. Risk management is a widely endorsed
    strategy for helping policymakers make decisions about allocating finite
    resources and taking actions in conditions of uncertainty. It has been
    widely practiced for years in such areas as insurance, construction, and
    finance. By comparison, its application in homeland security is relatively
    new-much of it coming in the wake of the terrorist attacks of September
    11-and it is a difficult task with little precedent. The goals for using
    it in homeland security include informing strategic decisions on ways to
    reduce the likelihood that adverse events will occur, and mitigate the
    negative impacts of and ensure a speedy recovery from those that do.
    Achieving these goals involves making policy decisions about what the
    nation's homeland security priorities should be-for example what the
    relative security priorities should be among seaports, airports, and
    rail-and basing spending decisions on what approaches or strategies will
    do the most good at narrowing the security gaps that exist. Risk
    management has been widely supported by the President and Congress as a
    management approach for homeland security, and the Secretary of the
    Department of Homeland Security has made it the centerpiece of agency
    policy.

    "Homeland security" is a broad term with connotations that resonate from
    the September 11 attacks and other connotations that now resonate from the
    disaster brought on by Hurricane Katrina in August 2005. Risk management
    has applications for deliberate assaults like the September 11 attacks and
    natural disasters, such as hurricanes and earthquakes. Our research was
    completed and the report largely written before Hurricane Katrina struck.
    Thus, our work concentrated on components' actions in response to
    terrorism.

    This report examines how three DHS components have applied risk management
    to certain aspects of their homeland security responsibilities. The three
    components are the United States Coast Guard, the Office for Domestic
    Preparedness, and the Information Analysis and Infrastructure Protection
    Directorate. This report looks at risk management efforts of the Coast
    Guard and ODP specifically related to seaport security, and for IAIP, it
    looks at risk management efforts related to IAIP's broader
    responsibilities in assessing terrorist threats against all aspects of the
    nation's infrastructure.1

          Risk Management Has a Long History of Use in Industry and Government

    Risk management can be described as the continuous process of assessing
    risks, reducing the potential that an adverse event will occur, and
    putting steps in place to deal with any event that does occur.2 It has
    been used in the private and public sectors for decades (see table 1 for
    examples). For example, insurance companies use a variety of statistical
    techniques to assess the level of risk for what they are insuring. Within
    government, agencies use risk management to set regulations and to protect
    the environment and the health and safety of American taxpayers. Although
    some risk management methodologies and processes can be complex and may
    require expert advice and support, other aspects of risk management-such
    as setting goals and using performance measures to track progress in
    meeting them-are well understood and widely practiced.

    1On November 14, 2005, DHS reorganized the department. ODP and the
    Infrastructure Protection component of the former IAIP Directorate are now
    in the Preparedness Directorate. We recognize the recent organizational
    changes, but because ODP and IAIP carried out the work we reviewed, we
    have not changed the name or organizational posture of these DHS
    components in our report.

    2A more precise description of risk management is that it involves a
    continuous process of managing-through a series of mitigating actions that
    permeate an entity's activities-the likelihood of an adverse event and its
    negative impact. Risk management addresses risk before mitigating action,
    as well as the risk that remains after countermeasures have been taken. A
    glossary of risk management terms is contained at the end of appendix I.

    Table 1: Examples of Risk Management in the Private and Public Sectors

    Source: GAO.

    Application of Risk Management to Homeland Security Is Widely Endorsed and
                                        Accepted

    Risk management was part of the nation's approach to assessing terrorism
    before the events of September 11. For example, in the 1990s, the Defense
    Special Weapons Agency assessed risks to evaluate force protection
    security requirements for mass casualty terrorist incidents at military
    bases. Companies under contract to federal agencies such as the Department
    of Energy, the National Security Agency, and the National Aeronautics and
    Space Administration used risk assessment models and methods to identify
    and prioritize security requirements. The Federal Aviation Administration
    and the Federal Bureau of Investigation did joint threat and vulnerability
    assessments on airports determined to be high risk. When we reviewed two
    of these efforts in the late 1990s, we found a lack of formal risk
    assessment requirements and made several recommendations to integrate
    risk-based data into decision-making processes.3

    What September 11 changed was the intensity and magnitude of this task.
    The September 11 attacks were clearly a transformational event for the
    nation, in that they called attention to vulnerabilities throughout the
    nation's infrastructure, not just in aviation security. While there might
    always have been a concern, for example, about the consequences of an
    accident in a chemical factory in a highly populated area, now, these
    consequences had to be viewed not just from the standpoint of a potential
    accident, but as something a terrorist could exploit. Potential targets
    multiplied, and the scope of work to be done became much greater. Homeland
    security spending rose from about $21 billion in fiscal year 2001 to a
    proposed $50 billion in fiscal year 2006.

    Risk management has received widespread support and interest from
    Congress, the President, and the Secretary of DHS as a tool that can help
    set priorities on how to protect the homeland. In this setting, numerous
    and substantial gaps in security exist, but resources for closing these
    gaps are limited and must compete with other national priorities.
    Policymakers in the legislative and executive branches have endorsed risk
    management as a technique that can inform decisions on setting relative
    priorities and on making spending decisions.

    In view of the widespread support that risk management has gained, federal
    agencies are now required to assess risks. The Homeland Security Act of
    2002 calls for a comprehensive assessment of risk related to
    vulnerabilities of critical infrastructure and key resources, notably (1)
    the risk posed by different forms of terrorist attacks, (2) the
    probability that different forms of an attack might succeed, and (3) the
    feasibility and efficacy of countermeasures to deter or prevent such
    attacks.4 Two congressionally chartered commissions, the 9/11 Commission
    and the Gilmore Commission, support the use of data on risks to help
    inform the difficult decisions that must be made in allocating limited
    federal funds for security measures. The President has issued policies
    directing the heads of seven major departments or agencies to assess
    risks. The past and present Secretaries of DHS have stated that actions of
    the department will be guided through the use of risk management.

    3GAO, Combating Terrorism: Threat and Risk Assessments Can Help Prioritize
    and Target Program Investments, GAO/NSIAD-98-74 (Washington D.C.: Apr. 9,
    1998), and GAO, Combating Terrorism: Need for Comprehensive Threat and
    Risk Assessments of Chemical and Biological Attacks, GAO/NSIAD-99-163
    (Washington, D.C.: Sept. 14, 1999).

    46 U.S.C. 201(d)(1), (2).

     Department of Homeland Security Has Broad and Challenging Responsibilities in
                                Applying Risk Management

    Congress has charged DHS with lead responsibilities in carrying out or
    coordinating homeland security programs and for applying risk management
    in carrying out this responsibility. For two main reasons, integrating a
    risk management approach into its business practices is a major management
    challenge that faces DHS.

               o  First, relative to many other fields such as insurance or
               finance, terrorism is a relatively new application for risk
               management. The sources of the risk are intelligent adversaries
               with malevolent intent with whom there is relatively little
               domestic experience. Unlike the insurance or banking industries,
               which have extensive historical data that are used to assess
               risks, DHS lacks such data on domestic terrorism, and this limits
               any detailed analysis in assessing risk. As a result, the
               probabilities and consequences of a terrorist act are poorly
               understood and difficult to predict and greater reliance on expert
               judgment is required. In January 2005, we identified risk
               management as an emerging high-risk area. At that time, we noted
               that DHS had not completed any risk assessments mandated by the
               Homeland Security Act.5

               o  Also, the size and complexity of homeland security activities
               add another dimension of difficulty to the task. Since its
               inception in March 2003, DHS has been faced with the challenge of
               transforming 22 agencies into one department in a way that results
               in an organization with effective planning, management, and
               operations while carrying out its critical mission of securing the
               homeland. Since 2003, we have designated implementing and
               transforming DHS as high risk, because DHS had to transform these
               many agencies-several with major management challenges-into one
               department.6 Besides the challenge it poses at the federal level,
               risk management also crosses jurisdictional boundaries and
               involves state and local governments and private industry
               stakeholders, and it requires a multidisciplinary approach
               involving intelligence, law enforcement, strategic planning, and
               program activities that address threats and vulnerabilities.

    5GAO, High-Risk Series: An Update, GAO-05-207 , p.29 (Washington, D.C.:
    January 2005).

    6 GAO-05-207 .

    Within DHS, we examined the progress of three DHS components-the Coast
    Guard, ODP, and IAIP-in administering risk management as part of their
    management processes. Two of these components (Coast Guard and ODP) have
    responsibilities related to seaport security. IAIP's responsibilities are
    much broader and more difficult-it is responsible for coordinating and
    assessing homeland risks across the federal government. Here is an
    overview of the three components.

               o  The United States Coast Guard. The Coast Guard is the lead
               federal agency for the security of the nation's ports. Its
               responsibilities include protecting ports, the flow of commerce,
               and the maritime transportation system from terrorism. As the lead
               in domestic maritime security, the Coast Guard has a robust
               presence at the national, regional, and port levels. The Coast
               Guard protects more than 300 ports and 95,000 miles of coastline.
               By providing a secure environment, the Coast Guard keeps maritime
               transportation open for the transit of commercial goods, as well
               as assets and personnel from the armed forces. In carrying out its
               mission, the Coast Guard has, among other activities, conducted
               local and national assessments of security risks at the nation's
               ports. The role of the Coast Guard in applying risk management to
               port security is discussed in more detail in chapter 2.

               o  The Office for Domestic Preparedness. Within the Office of
               State and Local Government Coordination and Preparedness, the
               Office for Domestic Preparedness is responsible for administering
               federal homeland security assistance programs for states and
               localities, including the port security grant program. Since 2002,
               the program has awarded over $500 million in grants to state,
               local, and industry stakeholders to improve security in and around
               their facilities or vessels. The role of ODP in applying risk
               management to port security grants is discussed in more detail in
               chapter 3.

               o  Information Analysis and Infrastructure Protection Directorate.
               The Information Analysis and Infrastructure Protection Directorate
               is responsible for, among other things, identifying and assessing
               current and future threats to the homeland, mapping those threats
               against known vulnerabilities, recommending protective measures,
               issuing warnings, and offering advice on preventive and protective
               action. IAIP is responsible for cataloging key critical
               infrastructure, then analyzing various characteristics to
               prioritize this infrastructure for the entire nation. These
               priorities are then to be used to direct protective measures for
               port security as well as across all other kinds of infrastructure.
               The role of IAIP in applying risk management to ports and other
               infrastructure is discussed in more detail in chapter 4.

           Seaports Are an Important Focus in the Homeland Security Response

    Seaport security receives substantial focus in this report because
    seaports have been widely regarded as vulnerable to attack. One reason is
    that the nation's seaports and inland waterways play a vital role in the
    nation's economy and national security. From an economic perspective,
    ports are critical links in the commercial trade and transportation
    systems, with more than 95 percent of the nation's non-North American
    foreign trade, including 100 percent of foreign oil, entering the country
    through seaports. The range of commodities involved includes not only a
    wide variety of consumer and agricultural products, but also cargo
    considered dangerous such as liquefied petroleum gas. A significant
    portion of this waterborne trade comes via cargo containers that are
    expected to move in and out of ports quickly, in keeping with industry
    expectations of just-in-time delivery. Port facilities are also used to
    ship military cargo abroad, and the Departments of Defense and
    Transportation have designated about 17 ports as "strategic" to support
    wartime mobilization, deployment, and resupply. Finally, not only are
    ports key hubs in our transportation system, they also function as centers
    of industrial, commercial, and financial activity. As such, they are home
    to many assets that are deemed to be among the nation's most critical
    infrastructure, which is to be protected under the USA PATRIOT Act of 2001
    and the Homeland Security Act of 2002.

    A second reason that seaports are potentially vulnerable is the wide range
    of targets and attack possibilities they encompass. Facilities such as
    container terminals, where containers are transferred between ships and
    railroad cars or trucks, must be able to screen vehicles entering the
    facility and routinely check cargo. Chemical factories and other
    installations where hazardous materials are present must be able to
    control access to areas containing dangerous goods or hazardous
    substances. Vessels, ranging from oil tankers and freighters to tugboats
    and passenger ferries, must be able to restrict access to onboard areas,
    such as the bridge or other control stations critical to the vessels'
    operation. Possible terrorist scenarios range from the use of improvised
    explosive devices to attack ferries to the use of recreational boats to
    ram key infrastructure in and around ports.

                            A Framework for Risk Management

    While there is a consensus that risk management should be applied to
    homeland security programs, doing so is a complex task that has few
    precedents and little specific guidance. The Homeland Security Act and
    presidential directives have called for the use of risk management.
    However, they did not define how risk management was to be accomplished.
    Given that there are no established universally agreed upon set of
    requirements or processes for risk management of homeland security, we
    developed a framework that can be broadly applied to a range of settings,
    such as analyzing security in the maritime sector and other environments.
    We did so by gathering, reviewing, and analyzing an extensive amount of
    public and private sector work; interviewing experts from private
    consulting companies in the areas of risk management and risk
    computer-modeling; interviewing experts on terrorism; and utilizing our
    own past work in this area. We also solicited comments and feedback from
    academic experts in risk management. As part of our work, we briefed
    officials of the three DHS components about the various phases of the
    framework, and the officials generally agreed with its structure and
    intent. The application of risk management to homeland security is
    relatively new, and the framework will likely evolve as processes mature
    and lessons are learned.

    The framework we developed is a conceptual synthesis of risk management
    approaches that we use as criteria to assess the adequacy of DHS's risk
    management systems (see fig. 2). For further information on the framework
    and how we developed it, see appendix I.

    Figure 2: A Framework for Risk Management

    This framework may be applied governmentwide and at various organizational
    levels, from departmental down to individual programs. The figure
    illustrates the cyclical nature of this approach, and while the phases are
    generally linear, changes can be made at any step in the process as new
    information becomes available. The five major phases of risk management
    are detailed below.

    Strategic Goals, Objectives, and Constraints

    According to the framework, management decisions are to be made in the
    context of the organization's strategic plan, with clearly articulated
    goals and objectives that flow from the plan. Performance measures that
    are clear, concise, and measurable are linked to the broader goals and can
    be used to measure progress toward these goals. An organization's program
    and risk planning documents address risk-related issues that are central
    to its mission.7 However, various constraints can take many forms and have
    an impact on risk related strategies. For example, some constraints may be
    imposed by statute, organizational policy, or budget restrictions.
    Managers at different levels within an agency or organization may
    encounter various constraints that differ with the scale of the operation.

    7For reasons of security, this identification may not be public knowledge.

    Risk Assessment

    Risk assessment helps decision makers identify and evaluate potential
    risks facing key assets or missions so that countermeasures can be
    designed and implemented to prevent or mitigate the effects of the risks.8
    In our framework, risk assessment is a function of threat, vulnerability,
    and consequence. The product of these elements is used to develop
    scenarios and help inform actions that are best suited to prevent an
    attack or mitigate vulnerabilities to a terrorist attack, in conjunction
    with the risk-based evaluation of alternatives undertaken while
    considering cost and other factors.

               o  Threat is the probability that a specific type of attack will
               be initiated against a particular target/class of targets. It may
               include any indication, circumstance, or event with the potential
               to cause the loss of or damage to an asset. It is based on an
               understanding of an adversary's intention, motivation, history of
               attacks, and capability to do damage. Analysis of threat-related
               data is a critical part of risk assessment. Information for
               characterizing threat can be gained from a variety of sources,
               such as the intelligence and law enforcement community, as well as
               from past activities of various terrorist groups. Understanding an
               underlying pattern of attacks on target types is useful in
               predicting future terrorist events and planning mitigation
               strategies. However, the unexpected threats not contained in the
               historical record of terrorist groups also need to be considered.
               Ultimately, one purpose of assessing threats is to assign relative
               probabilities to various types of attacks.

               o  The vulnerability of an asset is the probability that a
               particular attempted attack will succeed against a particular
               target or class of targets. It is usually measured against some
               set of standards, such as availability/predictability,
               accessibility, countermeasures in place, and target hardness (the
               material construction characteristics of the asset). Each of these
               four elements can be evaluated based on a numerical assignment
               corresponding to the conditional probability of a successful
               attack. The probability that a particular vulnerability could be
               successfully exploited is, in part, a function of the
               effectiveness of the antiterrorism countermeasures.

               o  The consequence of a terrorist attack is characterized as the
               expected worst case or worse reasonable adverse impact of a
               successful attack. The consequence to a particular asset can be
               evaluated when threat and vulnerability are considered together.
               The outcome of a terrorist attack may include many forms, such as
               the loss of human lives, economic costs, and adverse impact on
               national security.

    8A countermeasure is any action taken or physical equipment used
    principally to reduce or eliminate one or more vulnerabilities.

    Another closely related element taken into consideration is criticality
    (that is, the relative importance) of the asset involved. Criticality
    involves the prioritization of assets based on factors such as the
    potential for loss of life and the economic implications for the
    livelihood, resources, or wealth of the area, region, or country if the
    asset were to be lost. Layers of effective security countermeasures
    increase the likelihood that a terrorist attack will be unsuccessful as
    risk is reduced.

    Alternatives Evaluation

    Risks can be reduced through various antiterrorism countermeasures or
    countermeasure systems designed to prevent an attack or mitigate the
    impact of an attack. Two concepts here are key to evaluating
    countermeasure alternatives. The first is that countermeasures should be
    evaluated against specific risk assessments to determine the extent to
    which risks can be reduced by the countermeasure being considered. The
    second concept is the role of costs to both public and private sources, as
    costs are a critical element in the application of countermeasures. In our
    framework, cost-benefit analysis is critical in assessing alternatives,
    because it links the benefits derived from risk-reducing alternatives to
    the costs associated with implementing and maintaining them.

    Management Selection

    Management selection in our framework is informed by the outputs in the
    preceding phases. Having assessed risks and evaluated countermeasure
    options, management selects the blend of intervention strategies and
    activities across the entire spectrum of goals, objectives, and components
    of risk that achieves the greatest expected risk reduction in relation to
    cost for both the short and the long term among the various proposed
    alternatives. However, the technical analysis of alternatives is not
    likely to resolve or fully capture the numerous elements of concern to
    management. Decision makers may employ various risk-reducing strategies.
    However, preferences and value judgments will influence decisions about
    which strategies to employ. For example, corporate culture may influence
    decision makers to concentrate countermeasures on a relatively few
    critical assets, while others may value distributional impacts, that is,
    some organizations may be more willing than others to distribute resources
    over a wider array of assets. Management selection is an important task,
    and decisions are made with the information that is available. Our
    guidelines for effective internal controls dictate that once decisions are
    reached, they, along with the rationales for them, should be documented in
    order to inform future actions.

    Implementation and Monitoring

    This phase in the framework involves the implementation of the selected
    countermeasures. Following implementation, monitoring is essential in
    order to help ensure that the entire risk management process remains
    current and relevant, and reflects changes in the effectiveness of the
    alternative actions and the risk environment in which it operates. It is
    crucial to exploit any and all information sources, exercises, gaming,
    modeling and simulation, analysis of real world events, and sharing of
    information in a data sparse environment. Measurable objectives show the
    degree to which activities, timelines, support functions, service
    delivery, and spending are consistent with goals and implemented in
    accordance with the planning process. Program evaluation is an important
    tool for assessing the efficiency and effectiveness of the program. In
    addition to simply monitoring the implementation of the system and making
    adjustments, the entire risk management planning process should be
    periodically revisited. Since technology and information change at a rapid
    pace, countermeasures in place today may be outdated tomorrow and may
    become more susceptible to being breached. In addition, consultation with
    external subject area experts can provide a current perspective and an
    independent review in the formulation and evaluation of the program.

                           Objectives, Scope, and Methodology

    Our overall aim was to provide a perspective on how three DHS components
    have applied risk management as it relates to homeland security in
    general, or to port security in particular. More specifically, this report
    addresses the following objectives:

               o  What progress has the Coast Guard made in applying risk
               management to its port security mission, and what challenges does
               it face in moving further?

               o  What progress has ODP made in applying risk management to its
               administration of the port security grant program, and what
               challenges does it face in moving further?

               o  What progress has IAIP made in applying risk management to
               comparing and prioritizing critical infrastructure with one
               another and what challenges does it face in moving further?

               o  Are there key observations that can be drawn from all three of
               these efforts with regard to how far the three components have
               come in risk management as it applies to terrorism?

    To determine what progress the Coast Guard has made in applying risk
    management to its port security mission and the challenges that it faces,
    we met with Coast Guard officials responsible for port security risk
    assessment efforts to discuss the progress they have made and the
    challenges that remain. We discussed risk management efforts and
    challenges with Coast Guard officials at four ports-Baltimore, Maryland;
    Charleston, South Carolina; Houston, Texas; and Seattle, Washington-who
    were responsible for risk management activities. We judgmentally selected
    these ports because of their geographic distribution, and the results from
    our interviews cannot be generalized to ports nationwide. In addition, we
    reviewed documents of the Port Security Assessment Program, the Port
    Security Risk Assessment Tool, Area Maritime Security Plans, the National
    Risk Assessment Tool, the National Maritime Security Profile, and the
    National Maritime Strategic Risk Assessment. We also reviewed key
    legislation such as the Maritime Transportation Security Act of 2002 and
    prior GAO reports on maritime security. Finally, we reviewed threat
    assessments produced by the National Maritime Intelligence Center and the
    Transportation Security Agency to gain a more complete understanding of
    the challenges.

    To determine what progress ODP has made in applying risk management to its
    administration of the port security grant program and the challenges that
    it faces, we compared fiscal year 2004 and fiscal year 2005 ODP port
    security grant program procedures. In order to understand the grant
    process and the risks related to individual ports, we reviewed the risk
    assessment tools used by ODP officials, including the Coast Guard's Port
    Security Risk Assessment Tool.9 We reviewed and summarized a database
    listing fiscal year 2004 grant applications and awards to determine the
    extent to which criteria for awards coincided with the receipt of grant
    awards. We reviewed the Inspector General's (IG) January 2005 report of
    the port security grant program and discussed the recommendations
    contained in the report with ODP officials. We examined procedural changes
    made by ODP, in response to the IG recommendations and other factors, to
    the 2005 grant application process. We did not review the fiscal year 2005
    award decisions because we had completed our fieldwork before award
    decisions were announced, in September 2005. We met with Coast Guard,
    Maritime Administration, ODP, and IAIP officials involved in the port
    security grant program process. We also reviewed pertinent legislation,
    such as appropriations for the grant program for successive fiscal years.

    9The Coast Guard Port Security Risk Assessment Tool is designed to be used
    by the Captains of the Ports when making risk-based analyses of assets in
    their area of responsibility.

    To determine what progress IAIP has made in applying risk management to
    comparing and prioritizing critical infrastructure, and what challenges it
    faces in moving ahead, we reviewed key legislative and executive
    documents, such as the Interim National Infrastructure Protection Plan,
    the Homeland Security Act of 2002, Homeland Security Presidential
    Directives 7 and 8, national strategies, and DHS's strategic plan. We met
    with IAIP officials responsible for identifying and prioritizing threats,
    vulnerabilities, and consequences across different types of critical
    infrastructure to determine the obstacles they face in making such
    evaluations and the challenges they face in making progress in this area.
    We reviewed documents, such as the Risk Analysis and Management for
    Critical Asset Protection and the Buffer Zone Protection Program. We
    interviewed Office of Management and Budget (OMB) officials responsible
    for oversight of issues involving infrastructure protection to obtain
    their views on risk management practices across the federal government.

    To determine whether there are key observations that can be drawn from the
    three components we reviewed, we analyzed and synthesized the findings we
    developed to identify challenges that remain in applying risk management
    to homeland security. We compared the three components' progress in
    applying risk management principles to their respective tasks, identified
    common experiences in applying risk management, and drew conclusions about
    issues that may need addressing. We reviewed numerous documents, including
    pertinent statutes and presidential directives, GAO reports on high-risk
    programs in the federal government, the National Strategy for the Physical
    Protection of Critical Infrastructures and Key Assets, and testimony by
    the Secretary of DHS.

    We performed our work in accordance with generally accepted government
    auditing standards between May 2004 and November 2005.

    Chapter 2: The Coast Guard Has Made Progress in Using Risk Management, but
    Challenges Remain Chapter 2: The Coast Guard Has Made Progress in Using
    Risk Management, but Challenges Remain

    The Coast Guard has established a foundation for applying risk management
    to port security; its next challenges are to refine and strengthen its
    approach. The Coast Guard has made progress in all five phases of risk
    management (see table 2). For the first phase-goals and objectives-the
    Coast Guard has established broad strategic goals for port security, and
    its next challenge is to translate these goals into specific, measurable
    objectives that can be used to assess performance. The Coast Guard has
    actions under way to address this challenge-a key step in determining the
    extent to which its actions actually reduce risk. The Coast Guard has made
    the most progress in the second phase-conducting risk assessments. Six
    separate but related assessment efforts, covering both individual ports
    and the nation as a whole, have given the Coast Guard a clearer sense of
    the vulnerabilities that exist. However, current assessments are limited
    in terms of their methodology, and they do not allow the Coast Guard to
    compare and prioritize relative risks across ports-limitations that the
    Coast Guard recognizes and is taking steps to address. Enhancing these
    first two risk management phases is key to making further progress on the
    next two phases-evaluating risk mitigation alternatives and selecting a
    particular alternative for action. Without measurable objectives and more
    complete methodologies, the risk management process may not be able to
    target the most significant security concerns or determine the most
    cost-effective approach to take in providing reasonable protection.
    Additionally, weaving data produced from the risk management process into
    the annual cycle of program review remains a challenge. Finally, with
    regard to the fifth phase-implementation and monitoring-the Coast Guard
    has demonstrated the ability to evaluate its efforts and make
    improvements. However, more extensive and more formal feedback mechanisms
    would help ensure that Coast Guard headquarters managers can inform field
    staff about actions taken as a result of the comments received about the
    risk management process. The Coast Guard has actions under way to improve
    feedback loops.

    Table 2: Summary of Progress Made and Challenges That Remain in the Coast
    Guard's Risk Management Approach

    Source: GAO analysis of the Coast Guard's risk management efforts.

      Coast Guard Homeland Security Activities Revolve Heavily around the Maritime
                                         Domain

    The Coast Guard is the lead federal agency responsible for protecting
    domestic ports. In this role, the Coast Guard must identify, evaluate, and
    mitigate many kinds of security challenges. Ports are often sprawling
    enterprises that contain key infrastructure besides docks, piers, ships,
    barges, and warehouses. Many ports are also home to power plants, chemical
    factories, bridges and tunnels, and a variety of other assets of critical
    importance to the nation's economy and its defense.

    Coast Guard expenditures and activities for port security have risen
    dramatically since the terrorist attacks of September 11. The Coast Guard
    estimates that its budget for port security has jumped from about $250
    million in fiscal year 2001 to about $1.5 billion in fiscal year 2005.
    Since the terrorist attacks, the Coast Guard has carried out a myriad of
    port security activities, including increasing its intelligence
    capabilities, carrying out more harbor patrols and vessel escorts,
    establishing security zones, and working more extensively with federal,
    state, local, and industry stakeholders on port security matters as
    required by the Maritime Transportation Security Act of 2002 (MTSA).

    While much of the Coast Guard's homeland security efforts center
    specifically on ports, these activities are part of the agency's broader
    mission of Ports, Waterways, and Coastal Security (PWCS). This mission
    involves protecting the maritime domain and marine transportation system;
    preventing terrorist attacks, and responding to and recovering from
    attacks that do occur.1 As part of the PWCS mission, the Coast Guard aims
    to develop greater "maritime domain awareness"-that is, improving port
    stakeholders' understanding about anything associated with the global
    maritime environment that could adversely affect the security, safety,
    economy, or environment of the United States. Maritime domain awareness
    seeks to identify threats as soon as possible and far enough away from
    domestic ports to eliminate or mitigate the threat. Several Coast Guard
    efforts are under way to help address both port security and marine domain
    awareness. In particular:

    1U.S. Coast Guard, Maritime Sentinel: Coast Guard Strategic Plan for
    Ports, Waterways, and Coastal Security (Washington D.C.: September 2005).
    According to Homeland Security Presidential Directive 13 on Maritime
    Security Policy (Dec. 21, 2004), maritime domain means all areas and
    things of, on, under relating to, adjacent to, or bordering on a sea,
    ocean, or other navigable waterway, including all maritime related
    activities, infrastructure, people, cargo, and vessels and other
    conveyances.

               o  The Coast Guard is planning to expand its sector command
               centers, where officials can receive data 24 hours a day on
               maritime activities. Currently, the Coast Guard plans to develop
               sector command centers at 35 ports.2

               o  The Coast Guard is involved in a major recapitalization
               effort-called the Integrated Deepwater System-to replace and
               modernize the agency's aging fleet of aircraft and vessels,
               including improved and integrated command, control, communications
               and computers, intelligence, and surveillance and reconnaissance
               capabilities. Since the terrorist attacks of September 11, the
               Coast Guard has revised its plans, and the Deepwater program now
               includes improving maritime domain awareness and maritime security
               capabilities as part of its mission. This program is scheduled to
               take 20 years and cost between $19 billion and $24 billion.

    Federal statutes and presidential directives call for the Coast Guard to
    use risk management in its homeland security efforts. MTSA, for example,
    calls for the Coast Guard and other port security stakeholders to carry
    out a variety of risk-based tasks, including assessing risks and
    developing security plans for ports, facilities, and vessels.3 The Coast
    Guard's progress across the various risk management phases is thus a key
    part of its homeland security mission. The remainder of this chapter
    discusses the Coast Guard's progress and challenges on each of the phases
    in GAO's framework.

    2According to the Coast Guard, there are 30 sectors with command centers
    already in place.

    3See 46 U.S.C. S: 70103(b), (c); 33 C.F.R. S:S: 103.400, 103.500, 104.300,
    104.400, 105.300, and 105.400.

    Goal-Setting: High-Level Goals Are in Place, with Efforts Under Way to Set
                                 Measurable Objectives

    The Coast Guard has been able to make some progress in the first phase of
    the risk management framework, in that it has established high-level
    strategic goals for its PWCS mission. The Coast Guard set its national
    strategic goals for port security in December 2002.4 In order of priority,
    the goals were as follows:

               o  preventing terrorist attacks within, and terrorist exploitation
               of, the maritime domain;
               o  reducing America's vulnerability to terrorism in the maritime
               domain;
               o  protecting population centers, critical infrastructure,
               maritime borders, ports, coastal approaches, and the boundaries
               and seams between them;
               o  protecting the U.S. marine transportation system while
               preserving the freedom of the maritime domain for legitimate
               pursuits; and
               o  minimizing the damage and expediting the recovery from attacks
               that may occur within the maritime domain as either the lead
               federal agency or a supporting agency.

    Working with federal, state, local, and industry stakeholders involved in
    port security, the Coast Guard has also developed security plans for port
    areas across the country. These plans reflect the characteristics and
    needs of the individual ports, and in general, they aim to deter a
    terrorist incident and improve communication among port stakeholders.
    These plans are specific to each port location and are aligned with
    higher-level port security goals.

    While the Coast Guard has set broad goals for its port security mission,
    it still faces challenges in developing objectives into more specific and
    measurable results that measure progress toward these goals. So far, the
    Coast Guard has expressed its port security objectives in terms of
    activity levels, such as conducting patrols, escorting vessels, and
    inspecting cargo. While such activities may have contributed to improved
    security in and around the nation's ports, using them as measures may not
    systematically target areas of higher risk and may not result in the most
    effective use of resources, because these measures are not pointed toward
    outcomes. They describe what levels of activity, or outputs, the Coast
    Guard is providing, but they do not provide an indication of what these
    activities are accomplishing. Doing so requires measures that are clearly
    tied to results. Such measures would indicate the extent of progress made
    and help identify the security gaps that still remain.5

    4U.S. Coast Guard, Maritime Strategy for Homeland Security (Washington,
    D.C.: December 2002).

    Developing measurable objectives is a complex and difficult task, but
    Coast Guard officials recognize that doing so is a necessary next step and
    plan to have such objectives developed in fiscal year 2006. In September
    2005, the Coast Guard stated that it plans to develop a measure of its
    performance that will be based on an assessment of threat, vulnerability,
    and consequence. The Coast Guard plans to develop objectives for reducing
    overall risk. As part of this process, the Coast Guard plans to assess the
    impact of its activities in reducing threats, vulnerabilities, and
    consequences.

    Risk Assessments: Progress Has Been Substantial, but Challenges Remain for
               Addressing Limitations in Assessment Data and Methodology

    The Coast Guard's risk management activities have centered primarily on
    this phase of the risk management process, with assessments being
    conducted at both port and national levels. Further progress on the
    quality of these assessments is challenged by several types of limitations
    in the data and the methodology being used.

    Several Sets of Assessments Have Been Completed at the Port and National Levels

    Following the terrorist attacks of September 11, and consistent with
    MTSA's directives, the Coast Guard has greatly expanded the scope of its
    risk assessment activities. Before the attacks, these assessments centered
    on matters such as ecological damage and general marine safety. In 1999,
    for example, the Coast Guard adopted a risk management approach for its
    marine safety and environmental mission area. Assessments of key
    infrastructure in and around ports came largely after the attacks. Three
    such assessments have been done at the port level (see table 3).6 These
    assessments included data on threats, vulnerabilities, and
    consequences-the three types of information used in evaluating risk:

    5Any goals that the Coast Guard establishes will need to be aligned with
    two other national efforts to protect critical infrastructure. First,
    Homeland Security Presidential Directive-7 establishes a national policy
    for federal departments and agencies to identify and prioritize critical
    infrastructure and key resources, including assets and resources in and
    around ports. Second, Homeland Security Presidential Directive-13
    establishes policy and guidelines for implementing actions that enhance
    maritime security. The policy includes (1) preventing terrorist attacks
    and reducing vulnerabilities to attacks in the maritime domain; (2)
    enhancing security of ports, critical infrastructure, and coastal
    approaches; (3) enhancing international relationships; and (4) ensuring
    coordinated implementation of authorities and responsibilities among
    federal departments and agencies.

               o  Threats. The assessments gathered information on plausible
               threat scenarios, such as using weapons of mass destruction,
               ramming a vessel or facility, or detonating devices underwater. In
               general, local Coast Guard personnel or other federal and
               nonfederal stakeholders decided what threat scenarios to include
               in the assessment based on their knowledge of the port.

               o  Vulnerabilities. The assessments evaluated threat scenarios
               against potential targets, such as passenger vessels, bridges, or
               terminals, to assess the degree to which these potential targets
               were vulnerable to attack.

               o  Consequences. Finally, the three assessments addressed the
               potential outcomes of successfully carrying out a threat against a
               potential target. These consequences included such matters as loss
               of life, damage to the environment, damage to property, and
               economic disruption.

    6We previously reported on two of these efforts-the area maritime security
    assessments and the Port Security Assessment Program. See GAO, Maritime
    Security: Substantial Work Remains to Translate New Planning Requirements
    into Effective Port Security, GAO-04-838 (Washington D.C.: June 30, 2004),
    and Maritime Security: Better Planning Needed to Help Ensure an Effective
    Port Security Assessment Program, GAO-04-1062 (Washington D.C.: Sept. 30,
    2004).

    Table 3: Port-Level Assessments Conducted by the Coast Guard

    Source: GAO analysis of the Coast Guard's port-level assessments.

    These assessments have resulted in considerable progress in understanding
    and prioritizing risks within a port. In particular, the Port Security
    Risk Assessment Tool (PS-RAT), a computer program that includes possible
    threat scenarios, allows the Coast Guard to assess risks and develop
    relative rankings for infrastructure at each port location. At one port,
    PS-RAT was used to compare and inform priorities on over 1,000 critical
    items of infrastructure.

    Three other efforts have focused on assessments at the national level (see
    table 4).7 These efforts have relied in part on data and information
    generated from the local assessments described above, but they have also
    incorporated additional information. For example, the National Maritime
    Security Profile integrated available information from the intelligence
    community-a step that had not been carried out in any of the local risk
    assessments. Another assessment, the National Maritime Strategic Risk
    Assessment, sought to develop risk profiles for each of the Coast Guard's
    strategic goals; it examined specific mission areas, including port
    security, search and rescue, and law enforcement, and it sought input from
    field commanders on ways to mitigate key risks. According to Coast Guard
    staff, this was the first attempt at a large-scale strategic risk
    assessment that sought to assess the status of the maritime domain.

    7In addition to these efforts, the Coast Guard was involved in an
    interagency effort called the National Maritime Transportation Security
    Plan. This effort included the results of the National Maritime Security
    Profile. According to the Coast Guard, the results of the plan will
    contribute to the Coast Guard's outcome measures for its PWCS mission.

    Table 4: National-Level Assessments Conducted by the Coast Guard

    Source: GAO analysis of the Coast Guard's national-level assessments.

    While the port-level assessments focus on specific assets and
    infrastructure at each location, national assessments have focused more
    generally on understanding the risk posed to various classes of assets
    (such as container ships, barges, power plants, or bridges and tunnels) by
    various types of attacks (such as using explosives, taking control of an
    asset, or using weapons of mass destruction). These national assessments
    address, for example, whether the maritime domain is at greater risk from
    a takeover of a power plant or from a weapon of mass destruction planted
    on a container ship. The national assessments do not compare risks faced
    by specific assets at one port with risks faced by specific assets at
    another port.

    Key Challenges Involve Improving Data and Methodology

    The progress in conducting risk assessments is tempered by a number of
    challenges that remain in making these assessments more robust tools for
    informing the risk management process. These challenges are numerous and
    complicated, and this chapter illustrates some of the important issues the
    Coast Guard faces in making its risk assessments more useful. The
    challenges discussed here involve (1) improving the threat, vulnerability,
    and consequence data on which the assessments are based and (2) addressing
    methodological limitations that affect the reliability, completeness, and
    applicability of the risk assessments themselves.

      Limitations of Data on Threats, Vulnerabilities, and Consequences

    Our review of the Coast Guard's processes and our discussions with Coast
    Guard personnel surfaced a number of data-related problems that limit the
    reliability and completeness of the risk assessments (see table 5). For
    example, the tools require information not only about the types of threats
    a facility may face, but also about the probability of such threats
    occurring. The threat data received by the Coast Guard from the
    intelligence community do not allow the Coast Guard to model these
    probabilities, thus limiting the value of the output. The problems we
    identified have implications for the ability to effectively compare risks
    faced by the various types of port-related infrastructure.

    Table 5: Examples of Data-Related Challenges in Coast Guard Risk
    Assessments

    Source: GAO analysis of the Coast Guard's risk assessment efforts.

    These challenges are complex and technical, and the following examples
    illustrate the kinds of limitations they pose:

               o  Limitations in threat-related data. The intelligence
               information the Coast Guard normally receives about threats is not
               specific enough for all of the threat scenarios in the National
               Maritime Security Profile or the PS-RAT.8 For example, the type of
               threat data that Coast Guard personnel could use to model threats
               includes data on the presence of terrorist cells nationally and
               internationally, the capability and intent of terrorist groups as
               they relate to specific types of attack in and around ports, and
               the specific target groups. Increasing the quality of this
               information would improve the quality of the output. For example,
               when the Coast Guard received and integrated higher-quality
               information about some threats from its Intelligence Coordination
               Center, it modified data for 80 of about 300 threat scenarios in
               its National Maritime Security Profile.

               o  Limitations in vulnerability-related data. The Coast Guard's
               risk assessment tools were designed to evaluate existing security
               in and around a building or vessel, but according to Coast Guard
               officials, the baseline established by the tools excludes areawide
               actions the Coast Guard has taken to reduce vulnerabilities in and
               around ports, such as conducting more patrols, creating
               operational centers, or establishing security zones in and around
               key ports. The baseline also excludes actions taken by local port
               stakeholders, such as increasing the number of harbor patrols
               conducted by local law enforcement. The Coast Guard designed its
               tools this way because the primary purpose of the tools was to
               provide a port-level risk ranking of assets. Using this
               information, the Coast Guard could then use tools, such as the
               PS-RAT, to measure the benefit and value of any interventions for
               a specific vessel, facility, or asset type that it initiates
               against the original baseline. Coast Guard officials recognize
               that another tool would be useful in determining the overall
               vulnerabilities that exist after all Coast Guard actions have been
               taken.

               o  Limitations in consequences-related data. The Coast Guard's
               assessment of consequences is limited to direct effects of a
               terrorist attack, such as loss of lives and property damage; it
               does not consider important secondary effects, such as follow-on
               effects to the economy, including loss of jobs or increased energy
               costs that may occur months after an attack. This limitation
               likely understates the overall consequences that result from an
               attack and may distort relative risks associated with various
               threat scenarios. Coast Guard officials note that estimating
               secondary effects is important but difficult since there is a lack
               of accepted methods for doing so. A second limitation is that
               there is no commonly agreed upon value for a consequence such as
               death or injury, or the symbolic effect of destroying a national
               symbol such as the Statue of Liberty. For example, the Coast
               Guard's model places a dollar value of $1 million (in 2005
               dollars) on the loss of a life. Other components, such as the
               Environmental Protection Agency, use a monetary value of $6.1
               million (in 1999 dollars). The value chosen can affect the
               priorities that emerge from using the risk assessment tools.

    8The threat scenarios examined by the Coast Guard are contained in
    documents that are considered sensitive and for official use only.
    Accordingly, we do not provide detailed examples of various threat
    scenarios evaluated by the Coast Guard.

    We discussed these data-related limitations with Coast Guard officials,
    who generally agreed with our observations. The Coast Guard has since
    planned or started several actions designed to address some of these
    limitations. Coast Guard officials said they plan to improve PS-RAT based
    in part on the limitations discussed above.9 For example, they made
    changes in procedures for obtaining information from the Intelligence
    Coordination Center and have focused on improving the quality of
    information received from the intelligence community.10 In addition, the
    Coast Guard plans to improve vulnerability and consequence data. For
    example, the Coast Guard plans to assign weights to different levels of
    consequences. Coast Guard officials said they hope to accomplish these
    changes by the end of 2005.

      Methodological Limitations

    Two key methodological limitations affect the use of risk assessment data
    as a tool for informing decision makers on relative risks across port
    locations. The first limitation relates directly to the ability to compare
    and prioritize one port with another, while the second limitation relates
    to the process used to rank and prioritize individual threats.

               o  Risks cannot be compared between ports. PS-RAT results were not
               designed to compare the risks at one port with risks at another.11
               While PS-RAT allows the Coast Guard to compare and prioritize key
               infrastructure within a port, it does not produce a risk ranking
               that permits the Coast Guard to compare and prioritize
               infrastructure across ports. Interport comparisons, while
               theoretically possible, are difficult to actualize in practice. In
               general terms, the difficulties stem largely from the fact that,
               for each port, multiple scenarios must be considered; the
               scenarios that are deemed most relevant to each port, however,
               will differ from port to port. For example, ports that support
               passenger ferries and container cargo may be exposed to different
               risks than ports that primarily support bulk cargo. Comparisons
               are further limited because Coast Guard personnel at different
               ports use different methods to input data into PS-RAT. For
               example, Coast Guard officials at some field offices have
               summarized information on a type of asset, such as all bridges at
               a port location, while officials at other locations have developed
               data on each bridge in the area of responsibility. We discussed
               this limitation with the Coast Guard officials, and during the
               course of our review, the Coast Guard initiated work on a Maritime
               Security Risk Assessment Model (MSRAM)-a model that should permit
               the Coast Guard to compare the relative risks of high-value assets
               at one port with assets at a different port. The model is to
               include an analysis of various threat scenarios, vulnerabilities,
               and consequences. Overall, the MSRAM is to score and characterize
               risk associated with individual assets, including the estimated
               likelihood of an attack, the vulnerability of the asset should an
               attack occur, and the impacts of a successful attack. The Coast
               Guard has also requested data from the Intelligence Coordination
               Center on intent and capability that could improve the estimate of
               relative likelihood of various threat scenarios. Coast Guard
               officials said they plan to implement the MSRAM by the end of
               2005.

               o  Risks viewed as less probable could surprise the Coast Guard.
               The Coast Guard evaluates hundreds of threat scenarios that are
               deemed plausible-focusing attention, ultimately, on those threats
               that, together with identified vulnerabilities and consequences,
               pose the greatest overall risk. For example, in the National
               Maritime Security Profile, the Coast Guard classifies the risk of
               various threat scenarios as very high, high, medium, low, and very
               low, and it centers attention on scenarios that it estimates are
               very high or high risk. The Coast Guard's approach, although a
               useful starting point, may not be as reliable as the process would
               appear to suggest given the data limitations we describe above.
               How agencies like the Coast Guard deal with scenarios that receive
               low rankings is important in addressing the possibility of
               strategic surprise-an attack scenario that may not be identified
               or given high priority in the initial risk assessment process.
               Without sensitivity analysis or formal feedback loops to reassess
               all scenarios and therefore provide greater assurance that the
               rankings are as reliable as possible, the risk of being unprepared
               for strategic surprise may increase.12 The Coast Guard addresses
               this issue by making appropriate adjustments in priorities when
               tactical and strategic information call for such changes. We
               discussed this issue further with agency officials, and partly on
               the basis of these discussions, the Coast Guard is taking
               additional steps to address this issue. It is doing so by (1)
               increasing coordination among risk stakeholders at all levels to
               improve checks throughout the risk management cycle, (2) making
               refinements to threat data by requesting the Intelligence
               Coordination Center to provide estimates of capability and intent
               of terrorist groups, (3) including time horizons for various
               scenarios, and (4) leveraging independent assessments conducted by
               different subject matter experts as a way of checking its risk
               assessment work.

    9Coast Guard officials said that in addition to making improvements in
    threat, vulnerability, and consequence data, they plan to align some of
    their efforts more closely with Department of Homeland Security risk
    assessment databases and to conduct sensitivity analyses of various
    mitigation strategies.

    10IAIP officials said that the Coast Guard's efforts are a pilot project,
    and that if sufficiently mature, the effort will be more broadly
    implemented by the intelligence community.

    11Similarly, area maritime security plans are not comparable with one
    another, in part because PS-RAT results underpin the plans. In addition to
    the limitations we describe above, the scores for each port are relative
    to that port and scores from one port cannot be compared with the scores
    from another port because, in some cases, local Coast Guard staff
    developed scores for asset types, such as bridges or warehouses, and at
    other port locations, the local staff assigned scores for individual types
    of assets. The Port Security Risk Assessment Program conducted studies at
    55 ports at a cost of about $35 million and these studies are also not
    comparable with one another.

    Evaluation of Alternatives: Ability to Evaluate Alternatives Is Greatest at the
                                      Local Level

    Just as the Coast Guard's ability to assess risk is stronger at the
    individual port level than across ports, its ability to evaluate various
    alternatives for addressing these risks is greater at the port level as
    well. PS-RAT was specifically developed to help local Captains of the Port
    concentrate and prioritize their resources and evaluate alternative
    methods of risk reduction. Data from PS-RAT help identify vulnerabilities
    within a port and can be used in improving security measures related to
    the area maritime security plans. PS-RAT is not designed to work, however,
    above the port level. At the national level, the Coast Guard has conducted
    qualitative evaluations of the potential benefits of various alternatives
    for reducing risk levels, such as improved information sharing through the
    use of interagency operational centers, waterborne patrols, and escorting
    ships. In addition, it is assessing the potential reduction in risk of
    different strategies for improving awareness of the maritime domain.

    The effectiveness of such evaluations, both within and between ports, is
    influenced to a large degree by the performance standards and goals that
    are set, as well as the reliability and completeness of the risk
    assessments that are conducted. To the extent that goals are missing and
    risk assessments produce data that do not completely and reliably depict
    threats and vulnerabilities, the prospective evaluation of benefits and
    costs of future mitigation strategies may not target the areas with the
    greatest gaps or lead to the most cost-effective decisions.

    12See Y. Y. Haimes, S. Kaplan, and J. H. Lambert, "Risk Filtering,
    Ranking, and Management Framework Using Hierarchical Holographic
    Modeling," Risk Analysis, Vol. 22, No. 2, 2002, and M. Leung, J. H.
    Lambert, and A. Mosenthal, "A Risk-Based Approach to Setting Priorities in
    Protecting Bridges Against Terrorist Attacks," Risk Analysis, Vol. 4, No.
    4, 2004.

    In examining various alternatives, a key consideration will be measuring
    the overlapping benefits of different strategies as a means of developing
    data on what set of alternatives may provide the most improved security in
    return for the resources expended. These overlapping benefits and costs
    may involve Coast Guard actions, such as expanding the number of
    operational centers at port locations or procuring new ships and aircraft,
    or actions by others, such as more inspections of container cargo
    shipments or developing better intelligence capability. The Coast Guard's
    evaluation of alternatives generally involves examining its own possible
    actions, such as developing proposals to expand the number of operational
    centers, without placing the cost of such actions in context with other
    alternatives that have already been deployed or that could be used. In
    discussions with us, Coast Guard officials said addressing this issue was
    part of the Coast Guard's efforts to further refine its risk management
    approach. For example, the Coast Guard is working toward examining
    alternatives by assessing the degree to which they reduce risk in exchange
    for the cost involved.

     Management Selection: Local Strategies Have Been Selected but Would Be Better
        Informed by Improvements in Phases for Goal Setting and Risk Assessment

    In regard to management selection-the fourth phase of the framework-the
    Coast Guard has used its port-level assessments to select specific
    mitigation strategies to manage vulnerabilities in and around facilities.
    For example, local Captains of the Port have used the assessment
    information in coordination with input from local stakeholders to (1)
    establish security zones around key port infrastructures; (2) improve
    security in and around passenger vessels; and (3) coordinate security
    improvements, such as fences, cameras, and barriers around port
    infrastructures. At the national level, the Coast Guard is designing and
    planning to implement an array of radar systems, sensors, and information
    systems to identify and track possible threats in the maritime domain. One
    element of this effort is the establishment of maritime intelligence
    fusion centers that cover the Pacific and Atlantic coasts. These fusion
    centers are providing intelligence to the Coast Guard intelligence network
    in the field, and the centers share information with interagency partners,
    such as the Navy and Customs and Border Protection.

    In general, progress in this area is affected heavily by the same factors
    affecting progress in evaluating alternatives: progress in setting
    measurable performance objectives and improving the reliability and
    completeness of risk assessments. The various phases of the risk
    management model build and rely on one another, and in this case the
    quality and reliability of the results are heavily affected by the quality
    and reliability of efforts in the first two phases.

    Another major challenge will be to strategically integrate risk-based data
    into other management systems, such as the annual cycle of program and
    budget review for assessing how to deploy resources among ports. Coast
    Guard officials acknowledged that using risk-based data to inform the
    annual budget cycle is an important step and are taking further steps to
    address them. For example, the Coast Guard's MSRAM tool will compare risks
    at one port with those at another port, and the data produced could inform
    the Coast Guard's annual budget review of programs and resource
    allocation. In addition, based in part on our discussions with Coast Guard
    officials, the Coast Guard's resource planning guide for fiscal years
    2008-2012 calls attention to the importance of joining these processes.
    The guide states that by "employing principles of risk management and
    using our understanding of strategic risks to Coast Guard mission
    performance, we will be able to make better decisions regarding
    investment, re-investment and base management priorities."

      Implementation and Monitoring: A Foundation for Continuous Improvement Is in
         Place; Challenges Remain in Refining It and Developing Formal Policies

    The Coast Guard has made progress in the fifth phase of the
    framework-implementation and monitoring-by improving existing tools and
    systems in a variety of ways. For example:

               o  The Coast Guard has improved PS-RAT; version 2, developed in
               November 2002, by providing such improvements as greater detail on
               consequence data. Additionally, the MSRAM will include
               improvements in the quality of threat information in its analysis.

               o  The Coast Guard used the Port Security Assessment Program to
               further refine area maritime security plans by offering
               intelligence and other information to the local Captains of the
               Port on security issues the plan may not have covered.

               o  In November 2004, the Coast Guard started conducting the
               National Maritime Strategic Risk Assessment to obtain comments and
               feedback from Coast Guard area and headquarters staff on its
               mission areas, including its port security mission. Among other
               things, the staff provided input on ways that the Coast Guard
               could improve the manner in which it carries out its port security
               mission and reduce any gaps in coverage. For example, staff
               identified opportunities for leveraging resources, such as
               strengthening the partnership with nonfederal stakeholders
               involved in maritime security, as well as potential gaps in the
               port security mission, such as training, equipment, and
               technology. This information was summarized by a Coast Guard
               consultant and was provided to managers in Coast Guard
               headquarters.

    In some cases, this progress is limited to a degree by a lack of a formal
    policy for taking action on feedback that is received about ways to
    improve the risk assessment and management approach. For example, when the
    Coast Guard receives feedback from field staff on ways to improve its
    strategic approach, there are no formal policies for addressing the issues
    raised. Without such policies, there is little assurance that the feedback
    received will inform decisions to improve risk management practices and
    ultimately port security. For example, a Coast Guard official at one port
    noted that while there are informal communication channels to offer
    feedback on the PS-RAT, there are no formal procedures for doing so, and
    that such procedures would be useful in addressing concerns of local users
    when using the tool. Recognizing that there was no formal feedback process
    for the PS-RAT, the Coast Guard developed an ad hoc methodology to gather
    feedback from the users of the tool. In March 2005, Coast Guard
    headquarters hosted a workshop to obtain input from Coast Guard field
    offices on ways to improve the PS-RAT. The workshop resulted in a list of
    possible modifications to the PS-RAT and a plan to develop the MSRAM.
    Coast Guard officials acknowledge that a formal feedback process would be
    beneficial, and they plan to include one as part of the Coast Guard's
    MSRAM.

    Identifying and addressing organizational barriers to the Coast Guard's
    ability to improve or carry out its risk management approaches is a final
    consideration for the monitoring and implementation phase. One key
    organizational challenge is building and sustaining expertise and skills
    for effectively designing and using the risk management tools, techniques,
    and models necessary for managing the Coast Guard's efforts in carrying
    out its port security and maritime domain awareness responsibilities. The
    Coast Guard provides training on risk-based decision making at its
    training center. Recently, the Coast Guard revised its officer evaluation
    form by including "risk assessment" as a key leadership competency for its
    officers. Partly on the basis of the briefings we provided to agency
    officials, the Coast Guard plans to carry out workshops to examine ways to
    integrate risk management into the PWCS strategic plan and activities.
    Applying risk management tools and techniques is a complex undertaking,
    and it requires a managed effort to maintain and build organizational
    expertise and skills to do the job well.

                                      Conclusions

    The foundation the Coast Guard has established for risk management is
    generally sound. It represents a strong commitment on the Coast Guard's
    part to using risk management effectively in making decisions. Further,
    the Coast Guard, often acting on discussions held during the course of
    this review, has actions under way to address many of the concerns we
    identified. The most difficult work ahead revolves around systematically
    integrating risk-based information into management systems, including the
    annual cycle of program review, that can help inform decisions about how
    to deploy resources and security measures among ports and to examine the
    value of new programs in addressing security gaps that remain. In this
    regard, the Coast Guard's efforts are still in the early phases of
    development. As the Coast Guard moves forward, it is especially important
    that the agency develop ways to establish a stronger linkage between the
    various local and national risk assessment efforts under way. For example,
    area maritime security plans are not comparable, and security risks at one
    port location cannot be compared with the risks identified at another
    location. As a result, the collective value of these individual efforts is
    diminished. Developing ways to establish a stronger linkage would likely
    increase the value of the work.

                          Recommendations for Executive Action

    We are not making recommendations in those areas where the Coast Guard has
    actions well under way. The recommendations below are designed primarily
    to spotlight those areas in which additional steps are most needed to
    strengthen agency efforts to implement a risk management approach to the
    Coast Guard's port security activities. Accordingly, we recommend that the
    Secretary of Homeland Security direct the Commandant of the Coast Guard to
    take action in the following two areas:

               o  Risk assessment: Develop plans to establish a stronger linkage
               between local and national risk assessment efforts. This effort
               could involve strengthening the ties between local assessment
               efforts, such as area maritime security plans, and national risk
               assessment activities.

               o  Alternatives evaluation and management selection: Ensure that
               procedures for these two processes consider the most efficient use
               of resources. For example, one approach involves refining the
               degree to which risk management information is integrated into the
               annual cycle of program and budget review.

                           Agency Comments and Our Evaluation

    In commenting on a draft of chapter 2, DHS, including the Coast Guard,
    generally agreed with our findings and recommendations. DHS said that the
    report notes that the Coast Guard has made progress in all five risk
    management phases and is taking action to address the challenges that
    remain. In addition to commenting on our recommendations, the Coast Guard
    provided several technical comments under separate cover, and we revised
    the report when appropriate. Written comments from DHS are in appendix II.

    Chapter 3: Stronger Risk Management Approach Could Improve the
    Accountability of the Port Security Grant Program Chapter 3: Stronger Risk
    Management Approach Could Improve the Accountability of the Port Security
    Grant Program

    The Office for Domestic Preparedness within the Department of Homeland
    Security has made progress in applying risk management to the port
    security grant program but faces challenges in strengthening its approach,
    as demonstrated in part by its experience in awarding past grants.
    Examples of progress-and challenges-can be found across all five risk
    management phases (see table 6). ODP has established overall goals for the
    grant program but faces challenges in setting specific and measurable
    program objectives, in part because this effort hinges on similar action
    by other federal agencies. ODP's progress, with input from the Coast Guard
    and IAIP, has been greatest in conducting the actual risk assessments. It
    assessed 129 ports and, using risk-based prioritization, narrowed to 66
    the number of ports eligible to apply for fiscal year 2005 grants. Its
    methods for assessing risks and evaluating mitigation alternatives,
    however, still are limited in their ability to prioritize relative risks
    across ports or to calculate the costs and benefits of various
    alternatives. Finally, while ODP has also made progress in developing a
    risk-based grant selection process and mechanisms to monitor what the
    grants accomplish, it has not always completely relied on a risk-based
    process. For example, grant awards for fiscal year 2004 illustrate the
    trade-offs that occur when attempting to award grants to applicants that
    are at greater risk and, at the same time, provide funds to applicants
    that have the larger financial need. At the end of what was, in part, a
    risk-informed assessment process, ODP change the criteria for awarding
    grants when it decided to give lower priority to grant applications
    involving projects at large companies, on the assumption that these
    companies were better able than other entities to pay for their own
    security improvements. This changed 40 percent of the grants originally
    recommended for an award, as projects with higher risk but greater
    potential for self-funding gave way to lower-risk projects with more
    limited funding prospects. ODP changed the process for fiscal year 2005 by
    clarifying the criteria it would use in awarding grants and by requiring
    applications from private entities to match at least 50 percent of the
    total amount requested.

    Table 6: Summary of Progress and Challenges in the Port Security Grant
    Program

    Source: GAO analysis of ODP's port security grant program.

    aThe Coast Guard provided various data on ports, such as amount of total
    cargo, domestic cargo, international cargo, number of passengers using
    ferries, and number of passengers using cruise ships. ODP prioritized
    ports by evaluating these data to determine a list of the 66 highest-risk
    port areas.

                            ODP Manages Port Security Grants

    The port security grant program was established in fiscal year 2002 under
    the purview of the Transportation Security Administration (TSA), which
    became part of DHS in March 2003.1 Because of organizational changes
    within DHS, the grant program has been administered by ODP within the
    Office of State and Local Government Coordination and Preparedness since
    May 2004. ODP was transferred from the Department of Justice to DHS upon
    passage of the Homeland Security Act of 2002.

    1The legislation enabling the port security grant program was the
    Department of Defense and Emergency Supplemental Appropriations for
    Recovery from and Response to Terrorist Attacks on the United States,
    2002, Pub. L. No. 107-117, 115 Stat. 2230, 2327 (2002).

    The grant program provides assistance to nonfederal stakeholders for
    making security improvements at the nation's ports. During fiscal years
    2002-2004, grants from the program totaled about $560 million and covered
    such concerns as more fencing, cameras, and communications equipment. For
    fiscal year 2005, the appropriations act for DHS provided $150 million for
    port security grants.2 The 2005 program focused on three primary concerns:
    (1) protection against improvised explosive devices carried by small
    craft, underwater craft, or vehicles; (2) enhanced explosives detection
    capabilities for the owners and operators of vehicle ferries and
    associated facilities (as shown in fig. 3); and (3) facility security
    enhancements in the nation's highest-risk ports. The program is designed
    to operate in coordination with federal partner agencies and industry.3
    Grantees are selected through a competitive process.

    2Department of Homeland Security Appropriations Act, 2005, Pub. L. No.
    108-334, 118 Stat. 1298, 1309 (2004).

    3Federal agency and industry partners include the United States Coast
    Guard, the Information Analysis and Infrastructure Protection Directorate,
    the Border and Transportation Security Directorate, and the Transportation
    Security Administration within DHS; the Maritime Administration within the
    Department of Transportation; and the American Association of Port
    Authorities.

    Figure 3: Facilities at One of the Nation's Major Ports

    For fiscal year 2006, the Office of Management and Budget had proposed
    consolidating the port security grant program with other homeland security
    grant programs. Known as the Targeted Infrastructure Protection Program,
    the program would have consolidated funding for ports, transit, and other
    critical infrastructure into one program. However, DHS's appropriations
    act for fiscal year 2006 maintained separate funding for the port security
    grant program. In particular, the act provided a $175 million
    appropriation for the port security grant program that, "shall be awarded
    based on risk and threat."4

    In January 2005, the DHS Office of the Inspector General issued a report
    on the Port Security Grant Program that covered the program's second and
    third rounds of grants-through fiscal year 2003.5 This report made a
    number of recommendations, and in response, ODP initiated a number of
    changes to the program, according to ODP grant program officials. Our
    discussion of the grant program reflects the changes ODP had made at the
    time of our report.

    4Department of Homeland Security Appropriations Act, 2006, Pub. L. No.
    109-90, 119 Stat. 2064, 2075 (2005).

    5Department of Homeland Security, Office of Inspector General, Review of
    the Port Security Grant Program, OIG-05-10 (January 2005). In some years,
    more than one round of grants has been awarded. In all, five rounds of
    grants were awarded between 2002 and 2005.

     Goal Setting: Overall Goals Have Been Set; Developing Performance Measures and
                      Leveraging Federal Dollars Remain Challenges

    Progress has been made on setting goals-the first phase of GAO's risk
    management framework-for the port security grant program. Congress and the
    Administration have laid out broad policy goals for maritime security and
    for the program. Congress's stated purpose in establishing the program was
    to finance the costs of enhancing facility and operational security at
    critical national seaports.6 The Administration has set the program in the
    context of the December 2004 Presidential Directive on Maritime Security
    Policy, which cites several broad policy goals for maritime security,
    including preventing terrorist attacks in the maritime domain and reducing
    vulnerability to such attacks; protecting U.S. population centers,
    critical infrastructure, borders, harbors, ports, and coastal approaches;
    and maximizing awareness of security issues in the maritime domain in
    order to respond to identified threats.7

    DHS's application guidelines for fiscal year 2005 grants reflect the
    context of these broad policy goals. They state that the program reflects
    congressional and executive intent "to create a sustainable effort for the
    protection of critical infrastructure from terrorism, especially
    explosives and nonconventional threats that would cause major disruption
    to commerce and significant loss of life," and they link the program to
    specific national priorities specified in the nation's security planning
    framework.8 Other ways in which the 2005 grant program reflects a
    goal-oriented approach are its efforts to apply the grants to locations
    that are viewed as the nation's highest-risk ports (discussed in more
    detail below) and to focus the grants on such specific concerns as
    protection against improvised explosive devices.

    6The Department of Defense and Emergency Supplemental Appropriations for
    Recovery from and Response to Terrorist Attacks on the United States,
    2002, Pub. L. No. 107-117, 115 Stat. 2230, 2327 (2002).

    7Homeland Security Presidential Directive 13 (Dec. 21, 2004).

    8Office for Domestic Preparedness, U.S. Department of Homeland Security,
    Fiscal Year 2005 Port Security Grant Program: Program Guidelines and
    Application Kit (Washington, D.C.: 2005). The specific national priorities
    cited are (1) chemical, biological, radiological, nuclear, and explosive
    detection and response capabilities and (2) National Infrastructure
    Protection Plan implementation.

    While broad policy and program goals have been set, challenges to further
    progress on this risk management phase take two main forms. The first is
    translating the program's broader goals into measurable objectives. One
    difficulty in doing so is that other federal partner agencies have yet to
    spell out measurable objectives at the national level as related to
    protecting key infrastructure. For example, as discussed in chapter 2, the
    Coast Guard's Maritime Strategy for Homeland Security describes strategic
    approaches and priorities, but it does not include measurable objectives
    as part of its approach.9 In addition, DHS has yet to set performance
    measures for programs related to the implementation of programs for
    protecting critical infrastructures-another program area that the grant
    program supports.10 We discuss this further in chapter 4.

    A second challenge involves determining an appropriate way to consider two
    different federal concerns about grant programs: ensuring that grants
    address key needs while at the same time ensuring that they make the most
    efficient use of federal dollars. This challenge exists for several
    reasons:

               o  Federal and nonfederal partnership for addressing key needs.
               First, the federal government is not the only potential source of
               revenue for addressing security needs. Ports are often a complex
               mixture of public sector and private sector infrastructure. For
               example, public entities such as port authorities or local and
               state governments may own or operate seaport facilities and
               roadways, while private companies and interests may own and
               operate factories, warehouses, oil refineries, and railways.11
               Ports can produce benefits that are public in nature (such as
               general economic well-being) and distinctly private in nature
               (such as generating profits for a particular company). The public
               benefits they produce can also be distinctly local in nature, such
               as sustaining a high level of economic activity in a particular
               state or metropolitan area. Thus, state and local governments,
               like private companies, also have a vested interest in ensuring
               that their ports can act as efficient conduits of trade and
               economic activity. Given that homeland security threats can
               imperil this activity, it can be argued that all of these
               stakeholders should invest in the continued stability of the port.

               o  Leveraging federal dollars. Second, in many federal grant
               programs, the desired outcome is that federal grants supplement
               what other stakeholders are willing to spend. If a grant program
               is not designed to encourage supplementation, the danger is that
               other stakeholders will rely solely on the federal funds and
               choose to use their own funds for other purposes. This practice is
               known as substitution, and the net result is that limited federal
               funds cannot be stretched, or leveraged, to the degree they
               otherwise could be. In prior work addressing this issue in certain
               other grant programs, we found that on average, every additional
               federal grant dollar resulted in about 60 cents of substitution.12

    9U.S. Coast Guard: Maritime Strategy for Homeland Security, (Washington,
    D.C.: December 2002).

    10In some ways, owners and operators of facilities and vessels have set
    their own standards for what an acceptable level of risk is by the
    security plans they have developed under the Maritime Transportation
    Security Act of 2002. As required by the act, 46 U.S.C. 70103(c), facility
    and vessel owners developed over 12,000 plans to reduce vulnerabilities
    around their port areas. The Coast Guard required facility and vessel
    owners to implement them by July 1, 2004.

    11ODP estimates that 90 percent of facilities and vessels are owned by
    industry.

    Although the design of a grant program is not part of the risk management
    framework, it is an important issue because it is one key to accomplishing
    the dual aims of targeting funds to projects that address the highest risk
    while discouraging the replacement of state, local, and private funds with
    federal money. ODP's approach for 2005 has been to formalize a matching
    requirement for private sector stakeholders but not for public sector
    stakeholders. For fiscal year 2005 grants, ODP required that applications
    from industry match at least 50 percent of the total amount requested.

    This situation illustrates the complexity of addressing the most
    significant security needs while considering the degree to which
    nonfederal stakeholders should share in the cost of security improvements.
    For fiscal year 2005, the program encourages but does not require public
    sector or nonprofit entities to match federal funds, according to ODP
    officials. Depending on the value placed on reducing the substitution of
    federal funds for local funds, the design of the port security grant
    program offers a way to improve the fiscal impact of federal dollars.
    There is disagreement among policymakers about where the emphasis should
    be on this aspect of grant programs. Some might see the substitution of
    federal funds for local funds as reasonable given differences in fiscal
    capacity, while others may view homeland security as a shared fiscal
    responsibility. If policymakers place greater value on reducing the
    substitution of federal funds for local funds, strengthening matching
    requirements for such entities offers one option. The 2006 appropriation
    for the port security grant program includes a federal matching
    requirement.13 ODP has not yet issued its 2006 port security grant
    guidance clarifying how it will implement this requirement. One way to
    implement the requirement involves using a sliding scale for matching
    federal funds depending on the fiscal capacity of the grant applicant.
    Such a scale could range, for example, from an 80 percent matching
    requirement for Fortune 500 companies to a 25 percent matching requirement
    for those entities that have less in monetary resources.14

    12GAO, Federal Grants: Design Improvements Could Help Federal Resources Go
    Further, GAO-AIMD-97-7 (Washington, D.C.: Dec. 18, 1996). We include this
    figure here for illustrative purposes only and are not stating that this
    degree of substitution would occur in the port security grant program. For
    discussion of this issue in other public/private arenas, such as federal
    funding for highway investments and freight mobility projects, see GAO,
    Freight Transportation: Strategies Needed to Address Planning and
    Financing Limitations, GAO-04-165 (Washington, D.C.: Dec. 19, 2003);
    Highway and Transit Investments: Options for Improving Information on
    Projects' Benefits and Costs and Increasing Accountability for Results,
    GAO-05-172 (Washington, D.C.: Jan. 24, 2005); and Freight Transportation:
    Short Sea Shipping Option Shows Importance of Systematic Approach to
    Public Investment Decisions, GAO-05-768 (Washington, D.C.: July 29, 2005).

      Risk Assessments: The Funding Distribution Model Is Becoming Better Able to
                  Consider Risk, but Methodological Challenges Remain

    ODP has made progress in carrying out risk assessments-the second part of
    the risk management framework-but the progress made is balanced by the
    additional methodological challenges that remain. ODP's progress is
    reflected in changes made to the program for fiscal year 2005, in both
    port-level and national-level assessments. Among other things, ODP has
    placed greater emphasis on using threat, vulnerability, and consequence
    data in prioritizing applications and port locations. Nonetheless, various
    methodological challenges remain in the assessment tools or the
    assessments themselves. For example, ODP is still limited in its ability
    to compare and prioritize applications from one port with those from a
    different location.

    13Department of Homeland Security Appropriations Act, 2006, Pub. L. No.
    109-90, 119 Stat. 2064, 2075 (2005). The appropriation incorporates by
    reference the federal matching requirement of another federal grant
    program contained at 46 U.S.C. 70107(c). The match provision allows
    federal funding up to 75 percent of the total cost of the project, unless
    the DHS Secretary determines that a proposed project merits support and
    cannot be undertaken without a higher rate of federal support.

    14Fortune magazine ranks the nation's largest companies on the basis of
    revenue.

    Procedures for the Fiscal Year 2005 Port-Level and National-Level Assessments
    Have Been Strengthened

    At the port level, a key difference in ODP's fiscal year 2005 grant
    procedures is that input from other stakeholders plays a more prominent
    role in the award decisions. ODP obtained such input in prior years but
    did not formally consider it in making decisions. For example, in making
    its determinations for fiscal year 2004 grants, ODP sought input from the
    Coast Guard Captains of the Port and from regional directors of the
    Maritime Administration. These officials reviewed and ranked port security
    grant applications based on their knowledge of the port location and the
    results of various assessments, including the PS-RAT. ODP officials said
    they considered this input, but they did not integrate rankings into the
    evaluation forms used in the ODP assessment process. In this regard, the
    DHS IG found that ODP should place greater emphasis on risk reduction as
    part of the grant review process.15 In responding to the IG's findings,
    for fiscal year 2005, the rankings made by Coast Guard, Maritime
    Administration, and state officials are part of the formula ODP uses to
    make final decisions on grant awards.16 Additionally, the 2005 program
    places greater emphasis on applications that are consistent with area
    maritime security plan priorities.

    ODP's adjustments to its fiscal year 2005 procedures are even greater at
    the national level, where it has made a concerted effort to narrow the
    program to ports of greatest concern, and to use threat, vulnerability,
    and consequence data to rank and prioritize both ports and applications.
    For grants made in fiscal year 2004, ODP considered applications from all
    ports in making awards. For the 2005 program, ODP worked with IAIP and the
    Coast Guard to develop a list of eligible ports. The agency first
    identified the largest ports based on volume. According to ODP officials,
    a risk formula was developed to rank each of the ports. In May 2005, ODP
    determined that, on the basis of this assessment, 66 of the 129
    largest-volume ports across the country were eligible for grant awards.17
    ODP further prioritized the 66 ports by dividing them into four tiers-tier
    1 representing those ports with the highest risk and tier 4 representing
    ports with the lower risk. ODP provided a set amount of money to each
    tier, and ports in these tiers competed against each other for funding. In
    carrying out its analyses, ODP also placed greater reliance on threat,
    vulnerability, and consequence information.

    15Management for the fiscal year 2004 program was transferred from the
    Transportation Security Agency to ODP during the middle of the process.
    ODP assumed full responsibility for the program in fiscal year 2005.

    16For the 2005 program, the evaluation at the port level is managed by the
    Coast Guard Captain of the Port and state officials when feasible.
    Applications are reviewed against established criteria and ranked on
    relative risk, with the highest rank going to applications that support
    the national port security priorities.

    17ODP defines risk as credible threats and incidents (from the
    intelligence community), less credible threats and incidents (operational
    indicators), and vessels of interest. It defines vulnerability as distance
    from open water, number of port calls, and presence of tankers, and it
    defines consequence as people, economic, national security, and
    port-specific special considerations (such as hazardous materials or oil).

    Beyond reducing the number of ports eligible for the grant program,
    another change ODP made at the national level was to prioritize possible
    threat scenarios that grant funds should address, using input from Coast
    Guard officials. ODP has given priority to applications that prevent or
    detect threats arising from improvised explosive devices. Such devices
    pose a threat to transportation systems across the nation and have been
    used by terrorists in the past. Specifically, for the 2005 grant program,
    ODP has given priority to the following threat scenarios:

               o  preventing and detecting improvised explosive devices delivered
               via small craft,
               o  preventing and detecting underwater attacks from such devices,
               and
               o  preventing and detecting vehicle-born improvised explosive
               devices on ferries.

    Challenges Remain for Improving Methods and Data

    ODP's assessment methodology, while improved, still faces challenges.
    Progress in using risk assessment data to manage the grant program has
    limitations in methods and data for informing award decisions. These
    challenges, if addressed, would provide decision makers with more reliable
    and complete data on which to prioritize and award grant funds. The
    challenges fall into two main categories: (1) incomplete threat,
    vulnerability, and consequence data and (2) methodological inability to
    fully compare grant applications from one port with those from another
    port.

      Incomplete Data on Threats, Vulnerabilities, and Consequences

    Our review of ODP's risk assessment approach and our discussions with ODP
    and Coast Guard personnel identified several challenges related to data on
    threats, vulnerabilities, and consequences (see table 7).18 Many of these
    challenges mirror the challenges faced by the Coast Guard that we
    described earlier in chapter 2.

    18Although ODP has primary responsibility for administering the grant
    program, the Coast Guard plays a key role in the program. Local Captains
    of the Port use the PS-RAT in evaluating grant applications at the local
    level, and ODP officials consult frequently with Coast Guard officials at
    the national level.

    Table 7: Examples of Data-Related Challenges in ODP Risk Assessments

    Source: GAO analysis of ODP's risk assessment for the port security grant
    program.

    The following examples illustrate the challenges posed by the limitations
    in threat, vulnerability, and consequence data.

               o  Limitations in threat-related data. ODP's current
               characterization of threat cannot be interpreted as an estimate of
               relative probability of threat scenarios-a key element of risk
               assessment-in and around ports.19 The problems we pointed out in
               chapter 2 about threat data are applicable here as well: The
               threat information available from intelligence agencies does not
               provide the type of data that could be used to more fully inform
               ODP's decisions. More complete data would include such things as
               information on the presence of national or international terrorist
               cells, the capability and intent of terrorist groups as they
               relate to types of attack, and on the specific infrastructure
               types that have been attacked. ODP officials said that the
               scarcity of threat data limits their ability to inform the
               decision-making process and that decisions are based on the best
               available combination of data and expert judgment. Because threat
               data are limited, ODP bases its models on several proxies for
               risk. For example, ODP used volume of cargo that moves into and
               out of ports as a way to develop its list of the 129 ports that
               could initially be candidates for the grant program. ODP then
               assessed the relative risk at these ports by using, among other
               things, a threat variable represented by the number of "credible
               threats and incidents" and the number of "vessels of interest"
               (i.e., suspicious vessels) that use a port facility. These data
               originated from the intelligence community, the Coast Guard, and
               IAIP. ODP recognizes, however, that threat data on ports are
               scarce, and data on the number of credible threats and incidents
               and vessels of interest may not represent actual threats, but
               instead could also represent other law enforcement problems, such
               as illegal migrants or theft of goods and merchandise. The
               challenges in developing reliable data on probability affect the
               overall risk assessment for a port area, given a specific attack
               scenario. Without data or models that measure the relative
               probability of various threat scenarios, ODP may not target the
               most significant security concerns.

               Limitations in vulnerability-related data. As we described in
               chapter 2, the Coast Guard's PS-RAT excludes from its analysis
               reductions in vulnerabilities resulting from security measures
               that have already been taken by the Coast Guard, such as
               inspecting passenger vehicles that board ferries, escorting
               high-interest vessels, and establishing security zones around the
               port. ODP's assessment of vulnerabilities involves specific
               aspects of individual port areas themselves, namely, data on the
               length of the channel leading to a port, the number of port calls
               by all ships, and the number of tankers that use a port. While
               such data are representative of the intrinsic vulnerabilities of a
               port's location, they do not include such factors as guards,
               fences, and cameras that are already in place; security zones that
               have been established; or escorts of high-interest vessels that
               occur. As a result, the assessment may overstate vulnerabilities
               for port locations. Also important is the fact that ODP's approach
               to evaluating port area vulnerabilities excludes consideration of
               specific threat scenarios. For example, while they have identified
               certain threat scenarios as priorities, the vulnerability
               indicators are not linked directly to scenarios, such as the use
               of improvised explosive devices. Without this linkage to the
               threat component of the risk assessment process, ODP's approach to
               vulnerability assessments may not consider certain factors, such
               as the number of areas in a port location where recreational
               vessels are unmonitored, and how such factors may be conducive to
               certain threat scenarios, such as loading improvised explosive
               devices on such vessels.

               Limitations in consequences-related data.  The values used by ODP
               to describe the consequences of a terrorist attack may not
               accurately depict the damages resulting from a terrorist attack.
               In valuing the consequences of an attack, ODP focuses on
               people-related, economic-related, and national security-related
               measures.20 For example, ODP uses maximum population density
               within 10 miles of a port and the average number of daily ferry
               passengers to estimate the consequences of a terrorist attack at a
               port, and it aggregates these measures. Similarly, ODP uses
               international and domestic tonnages, the amount of containerized
               cargo, and the dollar value of foreign trade to measure the
               economic consequence of a terrorist attack. While these provide a
               useful starting point, there are two issues related to this
               approach. First, without linking these data to the relative
               probability of various threat scenarios, ODP's estimate of
               consequences is limited to inherent characteristics of a port
               rather than the estimated consequences of various attack
               scenarios. In contrast, agencies such as the Coast Guard estimate
               the number of lives lost and assign a dollar value to the loss of
               human lives. Second, ODP aggregates the estimated number of lives
               lost with the dollar value of foreign trade. Doing so raises
               questions about the reliability and meaning of the final output.

    19ODP relied on threat-related information provided by IAIP for the fiscal
    year 2005 grant program.

    Methodological Limitations

    As was the case with the Coast Guard, a key methodological limitation
    affects one goal of risk assessments-informing decision makers on relative
    risks across port locations. This limitation relates directly to the
    ability to compare the relative risks of facilities at one port with those
    at another port. When field review teams rank and prioritize grant
    applications, they use several sets of data, including information
    provided on the application, personal knowledge of the port, and the
    results of the PS-RAT, which gives ranking information for a given port
    area on vulnerability and consequence.21 As discussed in chapter 2,
    PS-RAT, however, cannot now be used to compare the risks at one port with
    those at another. PS-RAT allows ODP to compare and prioritize key
    infrastructure within a port, but it does not produce a risk ranking that
    permits ODP to compare and prioritize infrastructure across ports. Coast
    Guard efforts to develop a tool that examines relative risks across ports
    will aid ODP in addressing this limitation.

    20ODP worked with the Coast Guard and IAIP in developing the consequence
    measures.

    21When the port security grant program was established, according to a
    program official, it was one of the first programs to use the PS-RAT in
    order to identify risks. The PS-RAT has been very useful on the local
    level as a means for the Coast Guard to look at relative risk to assets
    within a port zone in a systematic way. In addition to being a primary
    tool for the port security grant program, the PS-RAT has also been used by
    members of the Area Transportation Security Committees to interact with
    one another and to plan their work.

          Alternatives Evaluation: ODP Recognizes the Importance of Evaluating
                    Alternatives, but Tools for Doing So Are Limited

    Evaluation of alternatives-the third phase of GAO's risk management
    framework-is an area that ODP recognizes as being an important part of
    awarding grants, and the changes for fiscal year 2005 represent progress
    in this area. One change for fiscal year 2005 involves additional steps to
    consider benefits and costs. When ODP asks local Coast Guard Captains of
    the Port to review applications, one criterion it asks them to apply is
    which projects offer the highest potential for risk reduction for the
    least cost. For 2005, ODP plans to augment evaluation by conducting an
    analysis of costs and benefits of the project (which considers the
    potential for risk reduction), and it shares the results with the Coast
    Guard.

    As part of this assessment, ODP plans to break out applications from the
    66 eligible port locations into four tiers and give applications from
    ports that are in higher tiers more priority and more money. Port areas
    with the highest risk are assigned to tier 1 and port areas with the
    lowest risk are assigned to tier 4.

    ODP's ability to assess proposed security improvements, like the Coast
    Guard's, is influenced by the program goals and performance measures that
    the component sets and the reliability and completeness of the risk
    assessments that it carries out. However, when measurable objectives are
    missing, the degree to which security gaps remain and the extent to which
    progress has been made remain unclear. Similarly, while PS-RAT provides a
    starting point for evaluating the proposed measures and the extent to
    which the measure narrows security gaps within a port, it was not designed
    to compare and prioritize relative risks from one port to relative risks
    in a different port. This condition limits ODP's ability to compare the
    benefits of proposed security measures from an applicant at one port
    location with benefits of proposed measures at a different port.

      Management Selection: ODP Has Addressed Problems in Documenting Differences
        between Initial Selection Recommendations and Final Selection Decisions

    Earlier in this chapter, we discussed the importance-and difficulty-of
    balancing potentially conflicting goals, such as ensuring that funds are
    directed to projects of the greatest risk while at the same time
    stretching limited federal dollars to the maximum degree. ODP's selection
    of grants for 2004 illustrates the challenges in applying a risk-based
    approach to awarding grants and the trade-offs involved in attempting to
    balance risk and financial need as criteria. In order to achieve
    accountability, federal standards for internal controls require that all
    transactions and other major events need to be documented. Basically,
    about 40 percent of ODP's final selection decisions were different from
    the initial recommendations of lower-level evaluators, without
    documentation from reviewers explaining why they disagreed with the
    initial recommendations. According to officials involved in the program,
    before the final selections were made, the Secretary of Homeland Security
    issued guidance indicating that Fortune 500 companies should be able to
    pay for their own security improvements and that ferries and port
    authorities should receive higher priority in the final award decisions
    than other applicants. ODP officials said that the fact that they followed
    this new guidance affected the final ranking of grant applicants. The
    tension between self-funding and security priorities illustrates the need
    for effective internal controls to ensure that procedures are followed and
    that the resulting selection decisions are transparent and clearly
    documented. ODP has taken action to address this problem.

    In the 2004 grant award process, ODP's initial assessments resulted in
    recommendations for funding 154 specific proposals.22 However, our
    analysis of the final grant awards showed that about 40 percent of the
    initial 154 applications that were recommended for a grant award by
    lower-level evaluators that examined grant applications did not receive an
    award. Table 8 shows examples of the types of changes that occurred.

    22ODP worked with TSA staff during the assessment process.

    Table 8: Examples of Changes in Funding Decisions

    Source: GAO analysis of 2004 grant fund database.

    These examples illustrate the trade-off between awarding grants to
    applicants that are assessed, in part, based on risk or on providing funds
    to applicants that have a financial need. The net result is that when
    federal funds are used in this fashion, they may not address the most
    severe security gaps in and around ports because there is no guarantee
    that private sector firms will spend their own funds for security
    improvements since it is unclear whether there are incentives, such as
    minimum standards for security, that would motivate them to do so.23 The
    competing goals of addressing the most significant security needs and
    providing financial assistance to those entities with less fiscal capacity
    call attention to the difficulty of achieving a balance between these
    objectives and the need for selection decisions that clearly document the
    trade-off.

    In order to achieve a transparent process for accountability purposes,
    federal standards for internal control require that all transactions and
    other significant events need to be clearly documented, and that
    documentation should be readily available for examination.24 However,
    internal control procedures for documenting decisions, including changes
    in project ranking, were not followed. The result is that the rationale
    for award decisions was not always available. In cases where an
    applicant's ranking would change by over 100 places, there was no
    documentation that described the reason for the change. In several cases
    where changes such as this occurred, it was noted that the final selection
    board concurred with the original ranking, but no reason was provided for
    the reprioritization. DHS's IG review also found this lack of
    documentation. The IG recommended that the reviewers be required to
    document their decisions in the grants management system, particularly
    when the decisions are inconsistent with recommendations from a lower
    level of review. DHS generally concurred with the IG's recommendations and
    stated that it would require reviewers to document their decisions as part
    of the 2005 grant program. Our work showed that in the fiscal year 2004
    grant program-the most recent round prior to the issuance of the IG's
    report-such documentation was still missing. For fiscal year 2005, ODP
    instituted additional measures to ensure that decisions were documented as
    part of the review process.

    23MTSA required owners and operators of facilities and vessels to develop
    and implement security plans by July 1, 2004. In all, about 12,300 plans
    were developed. The Coast Guard is charged with approving the plans and
    ensuring owners and operators are complying with the actions called for
    and determining whether all vulnerabilities have been identified. Coast
    Guard compliance inspections were scheduled for completion by July 2005.
    Funds requested by owners and operators may reflect additional security
    measures taken that go beyond the minimum requirements called for in the
    security plans or they could involve additional technology improvements.

    24GAO, Standards for Internal Control in the Federal Government,
    GAO/AIMD-00-21 .3.1 (Washington, D.C.: November 1999).

     Implementation and Monitoring: ODP Has Instituted a Monitoring Plan for Grant
                                         Awards

    ODP has made substantial progress in creating procedures that address the
    fifth phase of the framework-implementation and monitoring; the challenge
    lies in carrying them out. As we have described in this chapter, ODP has
    taken a number of actions to base spending decisions on risk-based data;
    it has, among other things, (1) developed procedures for prioritizing port
    locations, (2) prioritized threat scenarios, and (3) more closely aligned
    risk assessments to port locations. In addition, ODP has coordinated its
    efforts with the Coast Guard, the intelligence community, and other key
    stakeholders in developing the procedures for the 2005 grant program. For
    the fiscal year 2005 grant program, ODP has established procedures for
    monitoring recipients after they have been funded. The monitoring consists
    of follow-up after a project has been implemented to help ensure that the
    project has been implemented in accordance with the grant award, including
    timelines, budgets, and programmatic criteria are being met. In addition,
    ODP requires that progress reports be submitted biannually and a final
    project report be submitted within 120 days after the end of the project.

    A challenge that remains this phase of the risk management framework is
    developing processes for feedback to improve the process used in awarding
    grants. For example, our review of the 2004 grant awards show that local
    officials, including Coast Guard officials and grant applicants, did not
    receive feedback from ODP on why projects they designated as high priority
    did not receive funding when lower-priority projects did. According to ODP
    officials, they have instituted processes for providing feedback to and
    obtaining feedback from grant program stakeholders as part of the fiscal
    year 2005 program.

                                      Conclusions

    The principles of risk management apply to the port security grant
    program, and the lessons learned call attention to additional actions that
    could build on the progress that has already been made. Depending on the
    value placed on reducing the substitution of federal funds for local
    funds, there are opportunities to further leverage federal dollars in the
    program's design, and by leveraging federal funds, additional security
    needs in and around ports could be addressed. Additionally, without having
    performance measures in place, it is hard to gauge what progress has been
    made and what security gaps remain. The development of such measures
    should offer greater insights on the extent to which funds are narrowing
    security gaps that exist or helping to identify security needs that
    surface. The lessons of the program also provide numerous insights into
    the way that multiple stakeholders, such as the Coast Guard and IAIP,
    contribute to the way in which ODP uses risk-based data in administering
    the program. As we described in chapter 2, the Coast Guard has several
    efforts under way to improve its risk assessment of ports, including
    developing data on the relative probability of threat scenarios, improving
    data on consequences and vulnerabilities that are linked to various threat
    scenarios, and comparing risks among ports. ODP should be able to use the
    results of such efforts to help in awarding grants that are consistent
    with the priorities identified.

                          Recommendations for Executive Action

    To strengthen ODP efforts to implement a risk management approach to its
    port security grant program, we recommend that the Secretary of Homeland
    Security direct the Executive Director for ODP to undertake the following
    three actions:

               o  Clarify, in its grant guidance, the conditions under which
               greater leveraging of federal dollars should be included as a
               strategic goal for the port security grant program.
               o  Develop measurable objectives for managing the grant program's
               progress toward achieving strategic goals and use these measures
               to gauge progress and make adjustments to the program.
               o  Coordinate efforts with the Coast Guard and IAIP to use more
               reliable risk assessment data as they become available. At a
               minimum, such data should include (1) the relative likelihood of
               various threat scenarios, (2) consequences and vulnerabilities
               that are linked to terrorist scenarios, and (3) a comparison of
               risks across ports.

    Agency Comments and Our Evaluation

    In commenting on a draft of chapter 3, DHS, including ODP, generally
    agreed with the findings and recommendations. Specifically, DHS said that
    the recommendations are reasonable given that most of our review took
    place prior to changes that ODP made to the program in fiscal year 2005.
    DHS said that it appreciated our efforts to review the fiscal year 2005
    port security grant program requirements even though most of our work had
    been completed. DHS stated that several of the recommendations have
    already been addressed, and it noted that the remaining ODP-related
    recommendations will be addressed in the fiscal year 2006 port security
    grant program, at least to the extent possible. In addition to commenting
    on our findings and recommendations, DHS provided technical comments under
    separate cover, and we revised the draft report where appropriate. Written
    comments from DHS are in appendix II.

    Chapter 4: IAIP Faces Challenges in Meeting Risk Management
    Responsibilities across All Sectors of the Nation's Infrastructure Chapter
    4: IAIP Faces Challenges in Meeting Risk Management Responsibilities
    across All Sectors of the Nation's Infrastructure

    The Information Analysis and Infrastructure Protection Directorate within
    DHS faces broad and extensive challenges in meeting its risk management
    responsibilities and thus far has made limited progress. Relative to the
    Coast Guard and ODP, IAIP's risk management responsibilities are much
    wider and more difficult: Instead of comparing risks across port assets,
    it must find ways to compare risks at ports with risks in other sectors,
    such as public health, energy, and banking and finance. Challenges remain
    across all of the phases of GAO's risk management framework (see table 9).
    For the first phase (goals and objectives), while IAIP's efforts are
    anchored to strategic goals in various executive branch strategies and an
    interim national plan, IAIP's challenge is to continue developing the
    national plan to provide performance measures and associated steps and
    milestones. In the second phase, IAIP has begun several key risk
    assessment efforts but has had limited success in completing them. For
    example, IAIP faces challenges in developing data on the relative
    likelihood of various threat scenarios-a key part of the assessments it
    must conduct under the Homeland Security Act of 2002-because the
    information produced by the intelligence community is of limited use for
    risk assessment purposes, according to IAIP officials. IAIP has plans to
    develop such data by coordinating its efforts more closely with the
    intelligence community. Additionally, IAIP has yet to successfully
    complete the difficult task of comparing and prioritizing assets within
    and across sectors, but it plans to have an interim assessment done by the
    end of 2006. IAIP's challenge in the final three phases (evaluating
    alternatives, selecting approaches, and implementation and monitoring) are
    related heavily to IAIP's unique role: It recommends what the security
    priorities should be to other federal agencies and nonfederal stakeholders
    and recommends how best to address them, but it is largely dependent on
    other stakeholders, public or private, to take any actions. The decision
    to implement security improvements is made by stakeholders alone.
    Moreover, IAIP acknowledges that it can further leverage work that has
    already been done in this area by other federal agencies that have
    regulatory authorities over certain private sector infrastructure owners.
    IAIP's challenges center on developing credible guidelines and approaches
    that can leverage work already done and foster concurrence in risk
    analysis results and encourage actions to be taken.

    Table 9: Summary of Progress Made and Challenges That Remain in IAIP's
    Risk Management Approach

    Source: GAO analysis of IAIP's risk management practices.

         IAIP Plays a Key Role in Evaluating Risk across Infrastructure Sectors

    While risk management is one of several tools for the Coast Guard and ODP,
    risk management is central to one of IAIP's key missions, which is to
    establish a risk management framework across the federal government to
    protect the nation's critical infrastructure and key resources.1 The
    Homeland Security Act of 2002 made IAIP responsible for critical
    infrastructure protection (CIP) functions, charging IAIP with broad
    responsibility for developing a comprehensive national plan for securing
    the key resources and critical infrastructure of the United States. IAIP's
    statutory responsibilities require it to conduct risk management
    activities on a national scale and to gather the information needed to do
    so from other federal agencies, state and local government agencies, and
    private sector entities.2 By statute, IAIP is responsible for

               o  identifying, detecting, and understanding threats in light of
               actual and potential vulnerabilities to the homeland;
               o  conducting comprehensive assessments of the vulnerabilities of
               the key resources and critical infrastructure of the United
               States;
               o  conducting risk assessments to determine the risks posed by
               particular types of terrorist attacks and how likely they are to
               succeed, as well as the feasibility and potential efficacy of
               various countermeasures;
               o  identifying priorities for protective and support measure by
               DHS, other federal agencies, state and local governments, the
               private sector, and other entities; and
               o  recommending measures to protect the critical infrastructure
               and key resources of the United States in coordination with other
               federal agencies and in cooperation with state and local
               governments, the private sector, and other entities.3

    In December 2003, the President issued Homeland Security Presidential
    Directive-7, Critical Infrastructure Identification, Prioritization, and
    Protection, which established the framework in which IAIP carries out its
    responsibility of coordinating the overall national CIP effort. Current
    CIP policy, as described in HSPD-7, defines responsibilities for DHS,
    sector-specific agencies, and other departments and agencies. It instructs
    federal departments and agencies to identify, prioritize, and coordinate
    the protection of critical infrastructure to prevent and deter attacks,
    and mitigate the effects of any attacks that may occur.

    1The Homeland Security Act incorporates the definition of "critical
    infrastructure" used in the USA PATRIOT Act of 2001, meaning "systems and
    assets, whether physical or virtual, so vital to the United States that
    the incapacity or destruction of such systems and assets would have a
    debilitating impact on security, national economic security, national
    public health or safety, or any combination of those matters." The
    Homeland Security Act defines "key resources" as "publicly or privately
    controlled resources essential to the minimal operations of the economy
    and government." 6 U.S.C. S: 101.

    2See generally 6 U.S.C. 121(d).

    3Id. 121(d)(1), (2), (3), (6).

    To ensure the coverage of critical sectors, HSPD-7 designated
    sector-specific agencies for the critical infrastructure sectors
    identified in the National Strategy for Homeland Security (see table 10).
    These agencies are responsible for infrastructure protection activities in
    their assigned sectors, which include coordinating and collaborating with
    relevant federal agencies, state and local governments, and the private
    sector to carry out their responsibilities and facilitating the sharing of
    information about physical and cyber threats, vulnerabilities, incidents,
    potential protective measures, and best practices. For example, the
    transportation sector, for which the Department of Homeland Security is
    assigned responsibility, includes the movement of people and assets that
    are vital to the nation's economy, mobility, and security. The maritime
    shipping infrastructure, a component of the transportation sector,
    includes ports and their associated assets, ships, passenger
    transportation systems, and other maritime transportation assets. Each
    sector may also include a number of systems and "key assets"-some of which
    include individual targets whose attack could cause large-scale human
    casualties or property destruction, or profoundly damage national prestige
    and confidence.4

    4The National Strategy for the Physical Protection of Critical
    Infrastructures and Key Assets identifies five key assets. The key assets
    and lead federal agencies are as follows: national monuments and icons
    (Department of the Interior); dams, locks, and levees (Department of
    Homeland Security); government facilities (Department of Homeland
    Security); commercial and community assets (Department of Homeland
    Security); and nuclear reactors, materials, and spent fuel (Department of
    Homeland Security, working with the Nuclear Regulatory Commission and, as
    appropriate, the Department of Energy). When we use the term "critical
    infrastructure" in this chapter, we are referring both to the 13 sectors
    and to the five key assets.

    Table 10: Critical Infrastructure Sectors and Lead Federal Agencies

    Source: GAO analysis of the President's national strategy documents and
    HSPD-7.

    Under HSPD-7, the overall national CIP effort is to be coordinated by DHS,
    a responsibility carried out by IAIP, subject to the DHS Secretary's
    direction and control, as provided in the Homeland Security Act of 2002.
    The DHS Secretary has several overarching CIP responsibilities under
    HSPD-7, including identifying, prioritizing, and coordinating CIP within
    and across sectors, with an emphasis on critical infrastructure and key
    resources that could be exploited to cause catastrophic health effects or
    mass casualties. In addition, the Secretary is required to establish
    uniform policies, approaches, guidelines, and methodologies for
    integrating CIP and risk management activities within and across sectors,
    along with metrics and criteria for related programs and activities.

      Strategic Goals, Objectives, and Constraints: High-Level Goals in Place and
      Interim Plan Drafted, but Performance Measures and Milestones Have Yet to Be
                                       Developed

    A number of strategic goals are in place to guide IAIP's broad
    responsibilities. They stem from the Homeland Security Act of 2002 and the
    following executive branch documents:

               o  The National Strategy for the Physical Protection of Critical
               Infrastructures and Key Assets.5 Issued in February 2003, this
               strategy identified a set of national goals and objectives and
               outlined the guiding principles that underpin efforts to secure
               the critical infrastructure and key resources of the United
               States. The strategy recognizes that adequate protection of
               critical infrastructure requires (1) comprehensive threat
               assessment and analysis; (2) effective and efficient risk
               assessment; and (3) security baselines, standards, and guidelines.

               o  Homeland Security Presidential Directives 7 and 8. Both
               directives were issued on December 17, 2003. HSPD-7 established a
               national policy for federal departments and agencies to enhance
               the protection of critical infrastructure and key resources and
               made DHS responsible for establishing uniform policies,
               approaches, guidelines, and methodologies for integrating
               nationwide infrastructure protection and risk management
               activities. HSPD-8 calls for a national preparedness goal that
               balances the potential threat and magnitude of terrorist attacks
               with the resources required to prevent, respond to, and recover
               from them-a risk management approach calling for an estimate of
               the likelihoods and expected consequence of possible terrorist
               attacks that takes finite resources into account.

               o  The Interim National Infrastructure Protection Plan (interim
               NIPP). The Secretary of DHS assigned IAIP the responsibility for
               developing a national infrastructure protection plan. The interim
               NIPP was released in February 2005.6 It calls for the use of a
               risk management framework that takes into account threats,
               vulnerabilities, and consequences when comparing and prioritizing
               critical infrastructure and deciding what actions to take to
               protect them. This framework is intended to be carried out both
               within sectors and nationally across sectors. It is to contain
               steps for narrowing the overall set of assets to those that are
               most critical at a national level.

    5The National Strategy for the Physical Protection of Critical
    Infrastructures and Key Assets  is complemented by the National Strategy
    to Secure Cyberspace. Also issued in February 2003, the National Strategy
    to Secure Cyberspace  provides direction to the federal government
    departments and agencies that have roles in cyberspace security and
    identifies steps that state and local governments, private companies and
    organizations, and individuals can take to improve collective
    cybersecurity.

    6In November 2005, DHS issued a Draft National Infrastructure Protection
    Plan for comment. We did not evaluate the draft plan because our field
    work had already been completed.

    We have reported that the interim NIPP is not a comprehensive document,
    and IAIP faces several challenges in making it more comprehensive.7 These
    challenges are in two main areas:

               o  Performance measures. Although the interim NIPP did not contain
               performance metrics to measure effectiveness, it recognized that
               they are needed and calls on IAIP and the sector-specific agencies
               to develop them. According to IAIP officials, this is being done
               in two phases. For the first phase, IAIP has identified a set of
               basic core metrics that can be used to evaluate performance across
               all sectors, as called for in the interim NIPP. IAIP is also
               working with agencies responsible for specific sectors to develop
               a supplemental set of metrics for each sector. The intent of this
               measurement process is to provide DHS and the sector-specific
               agencies with feedback on where and how they should focus their
               resources to be most effective. According to IAIP officials, the
               second phase involves, in part, monitoring the progress of each
               sector in implementing the risk management framework laid out in
               the interim NIPP. To date, however, IAIP and the sector-specific
               agencies have not completed the performance metrics called for in
               the interim NIPP.

               o  Milestones and timelines. The interim NIPP did not contain
               milestones for the development of sector-specific plans or
               timelines of target dates for identifying and prioritizing
               critical infrastructure. According to IAIP officials, the final
               version of the NIPP, after undergoing interagency review, will be
               released in 2006, and it will contain milestones and timelines for
               the initial phase of developing performance metrics. It is not
               clear, however, if this final version will contain milestones and
               timelines for sector-specific plans or for completing the process
               of prioritizing critical infrastructure.

    7GAO, Critical Infrastructure Protection: Challenges in Addressing
    Cybersecurity, GAO-05-827T (Washington, D.C.: July 19, 2005).

    Risk Assessment: IAIP Faces Substantial Gaps and Has Made Limited Progress

    IAIP's progress in risk assessment has not been as extensive as the Coast
    Guard's or ODP's. Its area of greatest progress is in developing a
    national database of assets that constitutes the nation's critical
    infrastructure. In two other key respects, however, it faces major
    challenges in carrying out requirements specified in law or policy
    directives. These challenges are in developing adequate data on threats
    and creating a methodology for making cross-sector comparisons.

                  Progress Is Greatest on Database of Critical Assets

    IAIP has developed the National Assets Database (NADB), an inventory of
    approximately 80,000 assets, as a starting point in being able to evaluate
    and prioritize them. The NADB includes such facilities as nuclear power
    plants, pipelines, bridges, stadiums, and locations such as Times Square.
    Assets in the NADB are gathered from a variety of public and private
    sector sources, including federal, state, and local databases; prior
    studies containing lists of infrastructure and resources; and
    sector-specific data collection activities. The database is revised as
    assets are added and removed in collaboration with state and local
    officials and with representatives of sector-specific agencies. According
    to IAIP officials, this database is intended to produce baseline data that
    will later allow for assessments of vulnerabilities by location, within
    sectors, and across sectors.8

    Development of Threat Data Faces Challenges

    IAIP has begun work to develop threat scenarios and analyze them. The
    Homeland Infrastructure Threat and Risk Analysis Center (HITRAC), staffed
    by sector specialists and intelligence analysts with backgrounds from the
    intelligence community, is responsible for generating these plausible
    threat scenarios-called benchmark threats. HITRAC has developed 16
    benchmark threats, such as a suicide bomber, a vehicle-borne improvised
    explosive device, and a weapon of mass destruction. IAIP faces two
    substantial challenges, however, in completing this work.

               o  Relative probability for threat scenarios not yet developed.
               First, IAIP faces challenges in developing a way to differentiate
               the relative probability of various threats. Under the Homeland
               Security Act of 2002, IAIP must perform risk assessments to
               determine the risks posed by particular types of terrorist
               attacks, including an assessment of the probability of success of
               such attacks.9 IAIP officials stated that a lack of intelligence
               data and law enforcement data limits their ability to develop the
               relative probability for various threat scenarios, and for this
               reason, they have focused their initial efforts on developing
               vulnerability and consequence data. According to IAIP officials,
               the intelligence community-including the intelligence components
               of DHS-has been unable to provide detailed intelligence on threats
               to most sectors, infrastructure, assets, or asset types.10
               Assigning equal likelihood to various threat scenarios would mean
               IAIP's risk assessments will not include key threat data on the
               capabilities and intent of terrorist groups, the history of
               terrorist threats on various asset types, or the existence of
               terrorist cells domestically or internationally. And because data
               on the relative likelihood of threat scenarios are not included,
               the assessments will emphasize high-consequence events that may
               have a low probability of occurring. This approach is bound to
               result in potentially unreliable or incomplete data on where to
               establish priorities. In September 2005, IAIP officials told us
               that they recognize the importance of developing data on the
               relative likelihood of an attack type, and that doing so is part
               of their responsibility for meeting the requirements of the
               Homeland Security Act. Officials told us that they plan to assess
               the likelihood of threats by having HITRAC develop consistent
               comparative threat assessments by integrating intelligence
               sources, law enforcement data, and suspicious activity reporting
               with subject matter expertise. IAIP officials caution, however,
               that the directorate may not be able to estimate the relative
               likelihood of some threat scenarios, and as a result, some
               assessments may emphasize high-consequence events that have a low
               probability of occurring. Also, officials indicated that some
               inaccuracy is to be expected when HITRAC examines the intent and
               capability of an adversary whose plans are concealed and that it
               will be important to reduce the potential of low-confidence
               assessments having undue influence when long-term investment
               decisions are made.

               o  Vulnerability assessments not yet developed for many
               infrastructure sectors. Second, IAIP has not yet developed
               vulnerability assessments for the full spectrum of infrastructure
               sectors. As of August 2005, IAIP had managed the development of
               vulnerability assessment questionnaires for two components of
               critical infrastructure-nuclear facilities and chemical
               facilities. Initially, IAIP's contractor was scheduled to assess
               asset types in 8 of the 18 sectors and key assets by the end of
               2005. However, according to IAIP officials this work will be done
               by the end of 2006.11 IAIP officials did not have an estimate as
               to when the assessments would be complete for all other sectors of
               critical infrastructure that it is responsible for assessing.12

    8GAO did not analyze the data in the database and cannot comment on the
    database's reliability. The Congressional Research Service has raised a
    number of issues concerning the data integrity of the IAIP asset database.
    See Congressional Research Service, Risk Management and Critical
    Infrastructure Protection: Assessing, Integrating, and Managing Threats,
    Vulnerabilities, and Consequences (Washington, D.C.: February 2005).

    9See 6 U.S.C.121(d)(2). IAIP considers the combination of the threat
    (relative likelihood of the attack type) and vulnerability (likelihood of
    the adversary's success) to represent the relative likelihood of a
    successful attack occurring.

    10While the intelligence community does rank threats, IAIP does not find
    such information useful in developing data on threats. Intelligence
    components of DHS and other intelligence agencies routinely generate
    strategic threat assessment matrices and reports detailing the most likely
    targets and modes of terrorist attack. These assessments break down
    infrastructure and assets into categories and rank them, numerically or
    otherwise, according to the relative likelihood of attack within and
    across categories. However, IAIP officials said that the information was
    not specific to individual assets.

           Methodology for Cross-Sector Comparisons Has Experienced Setbacks

    IAIP has experienced setbacks in its attempts to meet HSPD-7's requirement
    to develop a strategy for identifying, prioritizing, and coordinating the
    protection of critical infrastructure. To prioritize the protection of
    this infrastructure, IAIP has been working for about 2 years on a
    methodology for assessing vulnerabilities of critical infrastructure to
    help inform comparisons of risks on an intrasector and cross-sector basis.
    This methodology has been delayed because of methodological concerns, and
    IAIP's schedule for completing various other activities needed to meet the
    requirement is dependent on the methodology and now appears uncertain.

    IAIP developed an analytical assessment tool known as the Risk Analysis
    and Management for Critical Assets Protection methodology (RAMCAP). RAMCAP
    was begun in November 2003, when DHS awarded the American Society of
    Mechanical Engineers (ASME) a $1.6-million grant to develop an overarching
    methodological guide for the private sector to assess its terrorism
    security risks. Originally, DHS expected RAMCAP to advance homeland
    security efforts by providing a usable, affordable vulnerability and risk
    assessment methodology for owners and operators to use that would inform
    risk management decision in the private sector. Now, IAIP views it as an
    independent effort that will complement IAIP risk assessment efforts at
    comparing risks within and across sectors.

    11After grouping all components of critical infrastructure by sector, IAIP
    plans to ask the owners or operators to complete a questionnaire-called a
    "top screen"-that provides an assessment of consequential losses arising
    from a worst-case scenario that assumes total loss of the asset. Once this
    information is gathered and assessed, assets considered to be of
    sufficient consequence are evaluated on vulnerability, using three
    elements: potential method of attack, probability of success, and
    consequences of the attack (including secondary and tertiary effects). By
    applying this information to the selected subset of assets, IAIP seeks to
    identify the assets of greatest vulnerability and identify strategies that
    hold the greatest potential benefits. Much like the consequence
    assessment, the vulnerability assessment is carried out by eliciting the
    opinions of experts, usually the owner/operator, for a specific piece of
    infrastructure.

    12IAIP has plans to address the risks involved in protecting additional
    critical infrastructures, such as stadiums, though a tool called the
    Vulnerability, Identification, and Self-Assessment Tool. This tool will
    allow owners and operators to assess vulnerabilities in and around their
    facilities.

    When RAMCAP was released for comment in April 2004, ASME received comments
    from over 100 officials from industry, academia, and government. A peer
    review process produced additional comments. The comments centered on
    several issues, including concern about the amount of resources needed to
    assess risks and the limited benefits in creating self-assessments for
    industry without knowing what the purpose of the effort involved.13 In
    June 2004, ASME altered its approach based on this feedback and began a
    broad outreach effort to involve industry in developing vulnerability
    assessment modules.

    In December 2004, IAIP awarded a $4 million contract to ASME to continue
    its efforts in developing RAMCAP and vulnerability assessment modules for
    owners and operators of asset types in eight sectors.14 After developing
    pilot modules for the chemical and nuclear industries, in August 2005,
    ASME produced a new working draft of RAMCAP. According to IAIP, RAMCAP may
    undergo minor revisions as more modules are completed. IAIP officials
    stated that more substantial revisions to RAMCAP will likely occur when
    IAIP broadens its approach to infrastructure protection by examining the
    entire systems and interdependencies within and across sectors.

    In September 2005, IAIP informed us that it is developing a National
    Comparative Risk Assessment for the 18 critical infrastructure sectors and
    key assets, and it plans to complete an interim assessment by the end of
    2006. The assessment has several phases. It involves sector-specific
    agencies identifying the top 100 high-risk systems and assets in their
    sector based on potential consequences. IAIP officials said it requested
    agencies to develop their lists by July 2005. However, the degree to which
    agencies will be able to fully respond to the request is uncertain. For
    example, as of July 2005, the Transportation Security Agency had begun but
    not yet completed a risk assessment for the passenger rail sector, and it
    did not indicate when the assessment would be done. Additionally, TSA did
    not expect to have the first version of its sector-specific plan-the plan
    that describes its risk assessment methodology-until February 2006. Until
    agencies such as TSA complete their assessments and develop their
    sector-specific plans, they will not be able to determine relative risks
    within their sector in a consistent fashion.15

    13IAIP and the private sector agreed that the goal of the self-assessments
    was to satisfy the information needs of DHS.

    14The eight categories of assets are commercial nuclear power plants,
    commercial spent nuclear fuel facilities, chemical plants, petroleum
    refineries, liquefied natural gas storage facilities, subway systems
    (including bridges and tunnels), railroad systems (including bridges and
    tunnels), and highway systems (including bridges and tunnels). The
    vulnerability assessments for the chemical sector and nuclear power plants
    were pilot projects.

    Whether IAIP will be able to complete this interim assessment by the end
    of 2006 may be complicated by two other factors. First, in September 2005,
    IAIP officials said that IAIP still needs to award the contract for this
    effort, and officials did not provide a schedule for when this would
    occur. Second, the degree and extent to which IAIP will need to obtain
    input from the private sector in developing this assessment tool is
    unclear. In developing the vulnerability assessment modules for
    sector-specific industries, IAIP's consultant and industry groups worked
    extensively with each other in developing the modules. If similar
    coordination and communication efforts are needed to complete the interim
    assessment, it may call for additional time and resources.

    According to IAIP officials, once the initial rankings are made, subject
    matter experts will review the results and the results of previous
    analyses will be used to refine the rankings within a sector and
    ultimately across sectors. IAIP plans to use the interim results to inform
    homeland security grant programs and serve as a basis for further risk
    management efforts. The results of the vulnerability assessment modules
    that are being developed will inform the national assessment and will act
    as a basis for interaction with the private sector, according to IAIP.

    The interim National Comparative Risk Assessment is intended to meet the
    immediate need of examining relative risk within and across sectors.
    However, there are limitations on the degree to which the interim
    assessment will produce comparable, consistent, and reliable data for
    setting national priorities. First, only vulnerability assessments of
    asset types in 8 of 18 sectors and key assets are scheduled for completion
    by the end of 2006, while the schedule for completing assessments on other
    sectors or asset types is unclear-suggesting that vulnerability data on
    some sectors will be more reliable than data on other sectors. Second, the
    interim assessment will likely involve a heavy emphasis on
    consequence-related data, and less information on the relative likelihood
    of threat scenarios and data on vulnerability-two major components of risk
    assessment. Third, as sector-specific agencies develop their list of
    high-risk assets, it is unclear whether they will do so in a consistent
    and uniform fashion, because the overarching framework that guides these
    actions is not yet in place. In September 2005, we reported that
    completing the element of the framework that defines concepts,
    terminology, and metrics for assessing risk limits DHS's ability to
    compare risk across sectors.16 IAIP recognizes that engaging
    sector-specific agencies in assessing risk in a consistent way is key.
    IAIP now intends the National Comparative Risk Assessment to provide such
    guidance, but until this assessment is issued, it is unclear where the
    guidance will come from. Limitations such as these, according to IAIP, are
    driven to some degree by resource and time constraints.

    15GAO, Passenger Rail Security: Enhanced Federal Leadership Needed to
    Prioritize and Guide Security Efforts, GAO-05-851 (Washington D.C.: Sept.
    9, 2005).

     Evaluating Alternatives: Lack of Guidance and Consensus on Costs and Benefits
                                   Creates Challenges

    The alternatives evaluated by IAIP differ somewhat from the alternatives
    evaluated by the Coast Guard (as discussed in chapter 2) or ODP (as
    discussed in chapter 3). The Coast Guard evaluates alternatives related
    directly to how it deploys its own resources, such as increasing security
    patrols, boarding suspicious vessels, creating and enforcing security
    zones, and using its regulatory powers to force maritime facilities to
    adopt protective measures. ODP evaluates alternatives that relate directly
    to its grant awards; the alternatives are the various funding proposals
    for security-related infrastructure improvements. In contrast, the
    alternatives that IAIP evaluates are generally not based on actions IAIP
    can take. Instead, they are based on identifying, from a national
    standpoint, the (1) areas of greatest risk and (2) infrastructure
    protection strategies that offer the greatest benefit for the cost
    involved. In this regard, IAIP faces several challenges.

    16 GAO-05-851 .

    One challenge faced by IAIP in evaluating alternatives is the difficulty
    in valuing costs and benefits in a homeland security setting-an important
    tool for evaluating alternatives. OMB has guidance for federal agencies on
    how to evaluate alternative government actions, but this guidance is of
    limited use in assessing alternatives for risk management for homeland
    security programs. OMB has identified various tools it considers useful in
    planning, such as cost-benefit analysis, capital budgeting, and regulatory
    decision making.17 However, such tools are difficult to apply to homeland
    security expenditures, even when such application is encouraged in the
    National Strategy for Homeland Security, because the benefits of homeland
    security investments are hard to quantify. Because OMB guidance is
    relatively silent on acceptable treatment of nonquantifiable benefits,
    there is a lack of criteria to guide federal analysts in conducting risk
    management. In response to our inquiries, OMB officials told us that they
    have not been involved in implementing Homeland Security Presidential
    Directive-7, which is related to critical infrastructure protection. In
    addition, they said that they have not been developing, nor did they have
    plans to start developing, guidance on risk management methodologies for
    federal agencies to use for homeland security programs. They said that
    they would rely on DHS and IAIP to take the federal government lead in
    developing such methodologies. IAIP officials said they plan to develop
    procedures in fiscal year 2006 for quantifying costs and benefits of
    mitigation strategies.

    Another challenge faced by IAIP in evaluating alternatives, at least in
    terms of ranking assets and protective actions to prevent and mitigate
    attacks, is that other entities responsible for taking such protective
    actions may use different criteria for evaluation. That is, the federal,
    state, local, or private sector entities that own and operate much of the
    nation's critical infrastructure may disagree with IAIP on how to evaluate
    alternatives through assessing benefits and costs or other types of
    evaluation. This lack of consensus could lead to two separate evaluations
    of alternatives-one by IAIP and one by the entity that owns and operates
    the asset. While IAIP may view certain assets or protective actions as
    critical, those responsible for the assets and protective actions may view
    the assets and actions as marginal or not necessary at all, or vice versa.
    According to IAIP officials, state and local government and industry
    stakeholders will benefit from using the same assessments, but the value
    that is placed on the assessment may differ from one stakeholder to
    another. As an example, Washington state emergency management officials
    told us that their initial listing and ranking of critical assets was much
    different than that developed by IAIP. State officials may place greater
    weight on attack scenarios that result in impacts on children or
    disadvantaged communities. Industry may place greater weight on scenarios
    that disrupt the long-term viability of a business. IAIP's challenge is
    discerning whether federal risk concerns are managed appropriately and
    that the costs for managing risks are assigned as much as possible to the
    authority that benefits from the activity. In addition to these
    considerations, Congress also has a role in appropriating federal funding
    of protective programs.

    17OMB Circulars A-11 and A-94.

     Management Selection: IAIP Challenged in Protecting Infrastructure because of
             Its Limited Role in Selecting Alternative Protective Measures

    As with the alternatives evaluation part of GAO's framework, IAIP's
    management selection differs from how the Coast Guard or ODP makes
    decisions. While the Coast Guard decides what protective measures to take
    and ODP decides which applicants receive funding for port security, IAIP
    is not in a position to direct others on how to act. IAIP officials said
    that while they have the responsibility of helping set risk-based
    priorities concerning where resources should be spent for protecting
    critical infrastructure, IAIP does not have the authority to direct the
    management of these resources in many cases. For example, IAIP does not
    have authority to compel owners and operators of critical infrastructure
    to take action. Instead, IAIP recommends the relative priority of critical
    infrastructure and specific protective measures. Other entities, such as
    owners, operators, or agencies with more direct regulatory
    responsibilities, can then act on IAIP recommendations and technical
    advice. The one exception is IAIP's Buffer Zone Protection Program, which
    we discuss in the section following on implementation and monitoring.

    At the departmental level, DHS may use the IAIP priority list to direct
    one of its component agencies to take specific actions. As examples, IAIP
    priorities could be used by U.S. Customs and Border Protection to increase
    the scrutiny of cargo containers at a specific port terminal, by the
    Secret Service to increase security for the President, by the Federal
    Emergency Management Agency to help improve local response capabilities
    near specific facilities, or by ODP to provide grants to specific
    facilities. In some cases, DHS components may have some authority over
    private infrastructure owners-such as the Coast Guard's regulatory powers
    related to implementing the Maritime Transportation Security Act. For
    example, the Coast Guard approves vulnerability assessments and mitigation
    strategies developed by owners and operators of facilities and vessels and
    it reviews whether the owners and operators are complying with their
    plans. However, in most cases, DHS and its components do not have
    authority over the owners and operators of critical infrastructure,
    particularly in the private sector.

    Thus, the challenge for IAIP (and DHS at the department level) is that it
    generally does not have authority over the entities that make management
    decisions to select among alternative protective measures. In such cases,
    IAIP's role is limited to providing information on how it views the
    relative strengths and weaknesses of the alternatives and, sometimes,
    technical advice on how these entities could improve the security of their
    assets. IAIP officials said they use their expertise and powers of
    persuasion to get the owners and operators to take specific protective
    actions. IAIP officials also said that IAIP works closely with industry
    groups to set standards and promote voluntary compliance.

    Certain protective measures could have application across multiple threat
    scenarios. The interim NIPP does not describe what IAIP's plans are for
    analyzing its benchmark threat scenarios and developing information that
    could help owners and operators reduce risk by protecting their facilities
    with countermeasures that address multiple scenarios. For example,
    improving security operations could reduce the risk of multiple threats,
    such as suicide bombers, truck bombs, or weapons of mass destruction.
    Having sufficient emergency supplies could address the consequences of
    casualties or damaged infrastructure that occur from various attack modes.
    By not having plans to develop information on what countermeasures could
    address multiple threat scenarios, IAIP is limited in it ability to
    provide information that informs owners and operators of facilities of
    ways to protect their facilities in a cost-effective fashion. According to
    IAIP, it is considering this in the cost-benefit framework that is being
    developed.

    IAIP's lack of authority over the owners and operators of critical
    infrastructure highlights the importance of coordination among different
    federal agencies. Several federal agencies with lead responsibilities for
    specific sectors of infrastructure do have regulatory authorities that
    could be used to set security standards and compel protective measures.
    For example, the Department of the Treasury and the Securities and
    Exchange Commission and other agencies have regulatory authorities over
    financial markets and can compel them to take certain protective measures.
    While DHS generally does not have authority over many of the assets in
    those sectors in which it has lead responsibility, some other federal
    agencies do. For example, the Nuclear Regulatory Commission, which issues
    licenses to commercial nuclear power plants, has clear regulatory
    authority over security matters at these facilities. While each sector has
    a Government Coordinating Council that includes representatives from
    federal agencies involved in the sector, IAIP recognizes that it can
    further leverage work that has already been done by other agencies that
    have regulatory authority over certain private sector owners. This
    condition brings attention to the need for IAIP to coordinate with these
    agencies to leverage federal authority in areas where oversight already
    exists.

     Implementation and Monitoring: IAIP Is Also Challenged by Its Limited Role in
                Implementation and Limited Information on Effectiveness

    IAIP has a limited role in implementing its own protective programs, as
    well as in monitoring the effectiveness and level of implementation of
    security risk management programs broadly. An example of IAIP's role in
    implementing protective programs is the Buffer Zone Protection Program.
    Managed by the Protective Security Division within IAIP, the program is
    designed to make it more difficult for terrorists to conduct planning
    activities or successfully launch attacks from the immediate vicinity of
    critical infrastructure and key resource targets. The goal of the program
    is to assist state and local government, local law enforcement, and owners
    and operators in preventing, defending against, preparing for, and
    mitigating the impacts of terrorist attacks. The program does so by making
    grants available, through ODP, to state and local law enforcement to
    implement buffer zone protection plans "outside the fence" of private
    facilities, as well as by conducting workshops and technical assistance
    visits.

    However, IAIP's role in implementing protective measures is much more
    limited "inside the fence" because IAIP does not own or operate any
    assets, have regulatory authority over those entities that own or operate
    the assets, or provide funds for such entities. Owners and operators of
    the infrastructure assets-be they federal, state, local, or private-are
    responsible for implementing the protective actions needed. As previous
    GAO work has shown, public policy tools have been used as an approach to
    encourage increased private sector critical infrastructure protection
    efforts even when regulatory authority is lacking.18 In terms of
    monitoring implementation of actions to increase the protection of
    critical infrastructure, however, IAIP does have a role. Its mission is to
    assess the overall state of critical infrastructure protection in the
    nation.

    18GAO, Critical Infrastructure Protection: Challenges for Selected
    Agencies and Industry Sectors, GAO-03-233 (Washington D.C.: Feb. 28,
    2003); and Homeland Security: Information Sharing Responsibilities,
    Challenges, and Key Management Issues, GAO-03-715T (Washington D.C.: May
    8, 2003).

    In its monitoring role, IAIP faces a number of challenges. The first
    challenge is that there are currently no performance measures to evaluate
    the effectiveness of infrastructure protection. The interim NIPP, while
    calling for such measures, does not offer any. According to IAIP
    officials, such performance metrics will not be available before 2006. One
    of the difficulties encountered is that it is hard to develop performance
    measures to gauge homeland security activities that are directed at
    modifying terrorist behavior. For example, it is difficult to determine
    whether measures to improve the protection of critical infrastructure have
    a deterrent effect on terrorists. In some cases, deterring terrorists from
    attacking "hard" targets (those that are heavily protected) might have the
    effect of directing the terrorist toward attacking "soft" targets (those
    that are lightly protected).

                                      Conclusions

    IAIP's role in risk management is critical because of its breadth. IAIP
    has overall responsibility to identify critical assets across all sectors
    of the nation's infrastructure, as well as to play a lead role in
    developing methodologies and guidance for analyzing risks and assessing
    the benefits and costs of security alternatives. Progress in most areas of
    responsibility has been limited, and much challenging work remains to be
    done. In particular:

               o  Good threat data are critical to conducting risk management.
               During the course of our review, IAIP recognized the importance of
               developing information on the relative probability to various
               threat scenarios. Until better data on threats are developed,
               risk-based data may not fully inform decisions on where to
               establish priorities.

               o  Even with better threat data, assets cannot be compared across
               sectors without a methodology for doing so. Currently, IAIP is
               developing a methodology to do this, but it is not yet
               comprehensive, nor has it been applied. Until such a methodology
               is fully developed, IAIP will be challenged in conducting a
               national-level assessment of risks, a key element of IAIP's core
               responsibility. Without a methodology, it will not be possible for
               IAIP to make a determination of relative risks that could help
               inform decisions on resource allocation.

               o  Comprehensive planning and performance measures are necessary
               to clarify what needs to be done and to determine progress in
               critical infrastructure protection. The National Infrastructure
               Protection Plan is still in interim form, leaving open many
               questions about how specific sectors will be protected and how
               performance will be measured. Without target dates for completing
               sector-specific plans and performance measures, it will not be
               possible to determine IAIP's progress in these areas.
               Additionally, without having plans to develop information on what
               protective measures could address multiple threat scenarios,
               IAIP's ability to inform owners and operators of ways to protect
               their facilities in a cost-effective fashion is limited.

    In all, the lack of progress leaves many decision makers basically on
    their own to develop a way to determine where scarce resources need to be
    applied against almost unlimited numbers of assets to maximize the
    protection of critical infrastructure and security for the homeland.

                          Recommendations for Executive Action

    To help ensure the development of risk management approaches to homeland
    security activities, we recommend that the Secretary of Homeland Security
    direct the Undersecretary for IAIP to undertake the following three
    actions:

               o  Work with the intelligence community to develop ways to better
               assess terrorist threats and use available information and expert
               judgment to develop a relative probability for various terrorist
               scenarios and provide this information to sector-specific
               agencies;
               o  As tasked by presidential directive, develop a methodology for
               comparing and prioritizing risks of assets within and across
               infrastructure sectors by including data on the relative
               probability of various threat scenarios;
               o  In completing the National Infrastructure Protection Plan,
               include target dates for completing sector-specific plans,
               developing performance measures, and identifying protective
               measures that could address multiple threat scenarios.

                           Agency Comments and Our Evaluation

    In commenting on a draft of chapter 4, DHS, including IAIP, said that IAIP
    is taking steps to address recommendations in the report. In regard to our
    recommendation on working with the intelligence community to develop
    better threat data, DHS said it is working with the Coast Guard on a pilot
    project to do so. The pilot effort will be evaluated, improved upon, and
    then more broadly applied with stakeholders in the intelligence community
    pending their acceptance, according to DHS. In regard to our
    recommendation on developing a methodology for comparing and prioritizing
    risks within and across sectors, DHS responded that for fiscal year 2006
    grants, a risk analysis methodology was applied that considers a small set
    of assets across sectors. With respect to our recommendation on completing
    the National Infrastructure Protection Plan, DHS stated that a draft plan
    is out for comment and that the issue of identifying protective measures
    that could address multiple threat scenarios is being addressed by IAIP.
    DHS also submitted technical comments under separate cover and we made
    changes where appropriate. Written comments from DHS are in appendix II.

    CRecommendations Chapter 5: Overall Observations and Recommendations

    Taken together, what do the efforts and experiences of these three
    components suggest about where DHS is with regard to managing homeland
    security efforts on the basis of risk? In our view, there are two key
    overall observations related to the degree of progress made, and two more
    related to next steps that need to take place. With regard to progress
    made, the first observation is that while considerable effort has been
    applied, much more remains to be done than has been accomplished so far.
    Across all three components, the most progress has generally been made on
    fundamental steps, such as conducting risk assessments of individual
    assets, and the least amount of progress has generally been made on
    developing ways to translate this information into comparisons and
    priorities across ports or across infrastructure sectors. Second, and
    closely related, progress among the three components' efforts has varied
    not only with the length of time the component has been at the job, but
    also with the complexity of its risk management task. With regard to next
    steps that would appear to add most value to making further progress, one
    key observation is that in the short term, progress is heavily dependent
    on continuing to make steady progress at improving basic policies,
    procedures, and methods for risk assessments and other phases of the risk
    management framework outlined in chapters 2, 3, and 4. Each component has
    an admittedly difficult set of challenges ahead, but progress has to be
    built on taking these incremental steps and doing them well. The final
    observation is related to a critical longer-term need: more guidance,
    direction, and coordination from DHS. The challenges and difficulties
    associated with creating a coordinated, coherent risk management approach
    to the nation's homeland security have been widely acknowledged since the
    events of September 11 and the creation of DHS. One of the presidential
    directives calls on DHS to provide such guidance, but the agency has yet
    to do so. As individual components begin to mature in their risk
    management efforts, the need for consistency and coherence becomes even
    greater. Without it, the prospects increase for efforts to fragment,
    clash, and work at cross purposes.

                       First Observation: Much Remains to Be Done

    There is a long way to go in implementing risk management successfully in
    port security-and an even longer way to go in implementing risk management
    in homeland security in general. One main reason is the sheer amount of
    work that must be done. Five years ago, before the September 11 attacks,
    the various terrorist scenarios seemed more remote and less certain than
    the hard reality brought on that day. September 11 changed this
    perspective dramatically. The work involved is immense and cuts across
    many jurisdictional boundaries. Federal agencies are called on to
    strengthen their partnerships with one another and to work more closely
    with thousands of state, local, and industry stakeholders.

    A second major reason is that applying risk management to terrorism has no
    well-established precedent. Parts of the private and public sectors have
    used risk management principles for decades. However, doing so in a
    homeland security setting is a highly difficult task that remains in its
    embryonic phases. The components we reviewed face daunting challenges in
    weaving a concern for risk into the annual cycle of management review and
    budget decisions. Across the federal government, this challenge is
    magnified and complicated because of the number of agencies charged with
    carrying out risk management. This is an extraordinarily difficult effort
    with no clear road map of ways to strategically integrate a concern for
    risk into management decisions.

    The fact that so much work remains is not the result of inaction by
    federal agencies. In the agencies and programs we examined, activities
    were often extensive and wide-ranging. Some activities, such as IAIP's
    attempts to develop risk assessment criteria for its comparisons across
    risk sectors, have had limited success, compounding the problem. The
    underlying point, however, is that this is an extraordinarily difficult
    effort with no clear and direct precedent to act as a guide. Implementing
    risk management in port and homeland security will take time and care, and
    this challenge will require ingenuity in adopting risk management
    techniques to this new application in a cost-effective way.

    The progress that has occurred to date in the agencies and programs we
    examined has been primarily in the activity that most people would perhaps
    associate most readily with risk management-conducting assessments to
    determine what the risks are at specific ports and facilities. While much
    remains to be done even there, progress has generally been slower on ways
    to approach risk management strategically-that is, with a clear set of
    measurable objectives, a clear knowledge of the options available for
    addressing risks and the trade-offs involved with these options, and
    evaluation and feedback mechanisms for continuing to refine and strengthen
    the approach.

     Observation Two: Progress Varies by Component and Reflects Key Characteristics
             of the Component and the Scope of Its Risk Management Efforts

    The three components we studied have made varying degrees of progress in
    risk management, and to a degree their progress is related to three main
    factors: how long they have been at the task, how organizationally stable
    they are, and the scope of what they are trying to do. The Coast Guard,
    for example, is furthest along among the three components, reflecting in
    part where it stands in relationship to all three of these factors. It has
    been at the task the longest of the three components, having begun work on
    implementing risk management in its port security efforts immediately
    after the September 11 attacks. Its primary risk assessment tool at the
    port level, PS-RAT, was implemented in November 2001, and by August 2002,
    prior to the creation of DHS and the port security framework called for
    under the Maritime Transportation Security Act of 2002, it had begun
    security assessments at major U.S. ports. To a degree, these early efforts
    were learning experiences that required changes, but the Coast Guard was
    able to build on its early start. The Coast Guard also had the greatest
    organizational stability of the three components. It moved into DHS as an
    already established entity with an organizational culture spanning
    decades, and its organization and mission were not significantly altered
    by moving into DHS. Finally, with regard to the scope of its risk
    management activities, the Coast Guard's work is specific to port
    locations, where it has direct and primary responsibility for carrying out
    security responsibilities. With its focus on ports, the Coast Guard does
    not have to address a number of the critical infrastructure sectors laid
    out in national preparedness policy, such as banking and finance,
    information and technology, and public health. Even so, the Coast Guard's
    experience to date shows that as the scope of activity widens, even within
    a single sector, complexities develop. For example, the Coast Guard has
    prioritized risks within individual ports, and it has actions under way to
    assess risks across ports, but using this information to strategically
    inform the annual program review and budget process will require further
    attention.

    ODP has made somewhat less progress than the Coast Guard. Relative to the
    Coast Guard's progress, its progress reflects a later start, an
    organization with much less institutional maturity, and a different role
    from the Coast Guard's in that ODP provides grant money rather than
    directly setting security policy. ODP was transferred from the Department
    of Justice to the Department of Homeland Security in 2003. While ODP's
    early grant approval efforts had some risk management features in place,
    its main strides in risk management have come in the procedures recently
    adopted for the fiscal year 2005 grants. In moving toward risk management,
    ODP has found ways to allow information from the Coast Guard and IAIP to
    inform its decision making. This is an encouraging and important sign,
    because the success of risk management efforts depends in part on the
    ability of agencies with security responsibilities to share and use each
    others' data and expertise. Although both the Coast Guard and the port
    security grant program administered by ODP have port security as their
    focus, ODP's more limited scope of responsibility has also had an effect
    on its risk management efforts. First, because ODP's role is to award
    grants that support federal priorities in port security, its progress in
    risk management depends to a degree on the progress made by federal
    agencies in determining what their own port security performance measures
    should be. Second, ODP's funding priorities are subject to criteria other
    than risk, as the fiscal year 2004 grant awards demonstrate. That year,
    after creating an initial list of awardees based in part on risk, and
    without regard to ability to pay, ODP extensively revised the list and
    awarded grants to entities considered to have fewer funding alternatives.

    Of the three components, IAIP is the least far along in its risk
    management efforts. All three factors have had an effect on this progress:
    IAIP has been at its task for a relatively short time; it is a new
    component; and relative to the Coast Guard and ODP, the scope of its
    efforts is much broader and more difficult. IAIP was created under the
    Homeland Security Act of 2002, giving the directorate little time to
    acquire institutional maturity. In addition to taking on difficult tasks
    like risk management, IAIP faces other institutional challenges, such as
    establishing new management systems, developing sound human capital
    practices, and integrating its efforts with those of the rest of DHS.
    Further, the scope of its risk management activities extends well beyond
    the port-focused activities of the Coast Guard or ODP. IAIP is responsible
    for conducting risk assessments for every critical infrastructure segment
    in the nation. As demonstrated by the experience of its RAMCAP methodology
    for comparing risk across sectors, IAIP remains challenged in meeting that
    responsibility. Its lack of progress reflects the same lesson that emerges
    from the Coast Guard's experience in trying to expand the focus of risk
    assessments beyond a single port: The complexity of risk management
    appears to grow exponentially as the focus expands beyond a single
    location or single type of infrastructure. This complexity may help
    explain IAIP's lack of progress, but IAIP is unable at this time to
    provide adequate assurance to Congress or the country that the federal
    government is in a position to effectively manage risk in national
    security efforts. Steps have been small; by far the biggest work is yet to
    come.

        Observation Three: In the Short Term, Further Progress Is Still Heavily
     Dependent on Completing and Improving Basic Policies, Procedures, and Methods

    Acknowledging that the nation still has far to go in establishing a sound
    risk management approach to security should not obscure the need to
    continue taking small, but critical, steps-building on incremental
    advances. The three components we reviewed have actions under way to
    improve their risk management approach, and their experience indicates
    that much of the immediate work should remain focused on basic steps
    needed to implement all components of the full risk management framework.
    The recommendations we make in chapters 2, 3, and 4 include
    component-specific steps for what needs to be done. In overview, these
    specific recommendations cluster around several major themes related to
    the five phases:

               o  Setting strategic goals, objectives, and constraints: While all
               three components have broad-scale goals in place, none has yet
               tied these goals to specific and measurable results. Without such
               measures, it is difficult to gauge what progress has been made in
               improving security and what security gaps remain. The Coast Guard
               is furthest along: It has tied its goals to activity levels, such
               as the number of patrols conducted or vessels inspected, and it is
               working toward developing outcome-based measures. This is a good
               step, but without such measures in place, it is not possible to
               see how programs reduce risks, improve security, or identify gaps
               in security that remain. All three components would benefit from
               specifying in clear and measurable terms what their efforts are
               designed to accomplish.

               o  Conducting risk assessments: All three components can improve
               their risk assessment techniques. All three were challenged by a
               general lack of detailed information on capabilities and
               intentions of terrorist groups as this relates to various threat
               scenarios. They took different approaches in response: The Coast
               Guard, for example, used threat scenarios as substitutes for
               detailed threat information and is working on assigning likelihood
               to each in order to determine where risks might be greatest, while
               IAIP evaluated the consequences of certain possible attacks and
               focused its analysis of vulnerabilities on the attacks with the
               greatest consequences. Approaches that do not include information
               on the likelihood of various threat scenarios have limitations
               that affect the degree to which agencies are able to determine how
               to best focus their efforts on areas of greatest risk. Efforts to
               strengthen both data, methodology, and policy would increase the
               reliability of their results.

               o  Evaluating alternatives: All three components face problems in
               measuring the costs and benefits of different measures for
               preventing or mitigating terrorist attacks. These include
               developing ways to measure costs incurred by a broad range of
               public and private stakeholders and developing ways to measure
               benefits (such as deterrence) when these benefits may not be
               quantifiable. These difficulties are particularly great for IAIP,
               which must be able to measure costs and benefits associated with
               mitigation strategies that reduce vulnerabilities at critical
               infrastructure in all sectors. These difficulties are compounded
               by the complexity in valuing costs and benefits in the area of
               homeland security when either the costs or the benefits are
               difficult to quantify or are not valued in monetary terms.

               o  Management selection: The three components face different
               challenges in this area, because each has different types of
               alternatives available in making decisions. The Coast Guard has
               the most direct control over security efforts; it can, for
               example, decide what protective measures to take with its own
               assets, and it has authority over other stakeholders to implement
               the Maritime Transportation Security Act of 2002. Its challenges
               lie mainly in what has already been discussed above-strengthening
               methods for risk assessment and alternatives evaluation and
               integrating this effort with the annual cycle of program review-so
               that management can make the most informed decisions about these
               efforts. ODP affects security efforts less directly; it can only
               consider facilities that have applied for grants, and it has no
               direct authority over port facilities in general, as does the
               Coast Guard. ODP has worked with the Coast Guard to receive its
               input into the grant application process. One challenge is to
               consistently apply criteria for management selection in a more
               transparent way. IAIP faces the most challenges in this area,
               because once it makes recommendations about how to prioritize
               assets on a national scale, it is largely dependent on the actions
               of others to carry them out and, particularly for owners and
               operators of private infrastructure, is dependent to a large
               degree on persuasion, market forces, or the work of regulatory
               agencies that have authority over key infrastructure, to ensure
               that protective measures are in place.

               o  Implementation and monitoring: Particularly for the Coast Guard
               and ODP, we have been able to identify instances in which the
               components have moved aggressively to improve their risk
               management approaches-and to continue doing so. Particularly with
               its recent setback on its RAMCAP methodology, IAIP is considerably
               behind these two components in implementing any kind of risk
               management approach. To move forward, it must overcome more basic
               problems with assessing risks and alternatives.

        Observation Four: In the Long Term, Progress Rests Heavily on a Level of
                      Coordination That Has Yet to Be Demonstrated

    In the long term, progress will become increasingly dependent on how well
    the nation's homeland security risk management effort is coordinated. We
    have identified and reported on some notable improvements in coordination
    at the port level, through such mechanisms as intelligence fusion and
    coordination centers, local area maritime security committees, and
    interagency operational centers.1 Replicating such coordination among DHS
    agencies and with state, local, and industry stakeholders is key.

    Currently, various assessment approaches are being used, and in many ways,
    these approaches are neither consistent nor comparable. Our work at IAIP,
    the Coast Guard, and ODP showed examples of these inconsistencies. For
    example, IAIP's initial plans called for treating all threat scenarios as
    equally likely to happen, while the Coast Guard and ODP are attempting to
    integrate the likelihood of various threat scenarios into their analysis
    vulnerabilities. The danger in using different methods is that if agencies
    develop systems and methodologies without some overall coordination, they
    may end up with redundant or incompatible systems that have little or no
    ability to inform one another. Even more important, these systems may
    provide decision makers with unreliable or incomplete data on how to
    allocate resources and protect the American people in a cost-effective
    way. Absolute compatibility is likely impossible given the multiple
    stakeholders at the federal, state, local, and industry levels. For
    example, owners and operators of critical infrastructure may value and act
    on risks differently than the Coast Guard and IAIP. Having a common risk
    management framework is a key consideration for assuring that knowledge
    and data can be transferred to all stakeholders, while permitting
    stakeholders to value risks in different ways. Even if agencies and
    stakeholders were working in close cooperation, lack of coordination is
    likely only to exacerbate the problem. This is particularly true given the
    difficulty of the task and the limited availability of federal risk
    management guidance.

    Until now, having such inconsistencies may have seemed less important than
    just getting risk management efforts under way. To a degree, we found this
    was the case with the larger universe of homeland security actions: When
    we first began reviewing agency actions shortly after the September 11
    attacks, we found agencies at work on many efforts, but signs of
    coordination problems were already apparent. Similarly, the risk
    management efforts that have been conducted to date appear fueled by a
    strong sense of the need to make some headway, with coordination and
    consistency a lesser concern. For example, IAIP, which is charged by
    statute with developing comprehensive assessments of the vulnerabilities
    of critical infrastructure and key resources, did not attempt to guide the
    Coast Guard's efforts in setting up a methodology for assessing
    port-specific risks. IAIP officials told us it was more important for the
    Coast Guard and other agencies to proceed with their risk assessment
    efforts than to delay starting, even though the officials recognized that
    these efforts might create approaches that would not mesh cleanly with the
    approach that IAIP would eventually develop. Given what has occurred to
    date, this course of action appears prudent, in that the Coast Guard has a
    considerable portion of a risk management system in place. If it had
    waited to begin until guidelines and policies had been set, it would still
    be waiting to start.

    1GAO, Maritime Security: New Structures Have Improved Information Sharing,
    but Security Clearance Processing Requires Further Attention, GAO-05-394
    (Washington, D.C.: Apr. 15, 2005).

    Now, however, the need for coordination is looming larger. IAIP has a
    significant role to play in this regard through its responsibility for
    providing agencies with guidance about risk management, but it has made
    limited progress. IAIP has been challenged in establishing uniform
    policies, approaches, guidelines, and methodologies for integrating
    federal infrastructure protection and risk management activities within
    and across sectors, along with metrics and criteria for related programs
    and activities as called for by HSPD-7. While IAIP has coordinated its
    activities with entities such as ODP and the Coast Guard, it has yet to
    issue policies, guidelines, and methodologies as required by the
    directive. Making progress with regard to this challenge is key to an
    effective use of risk management resources, as the National Strategy for
    the Physical Protection of Critical Infrastructure and Key Assets
    recognizes.

    Guidance is also important when agencies integrate a concern for risk into
    the annual cycle of program and budget review. Doing so within an agency
    is a difficult task in that traditional ways of reviewing budgets and
    programs often rely on program data that call for continuing or expanding
    a program without examining the relative risks that are addressed with the
    funds that are expended. Shifting organizations toward this nexus of using
    risk-based data as part of annual management review cycles will take time,
    attention, and leadership. Even in agencies where much progress has been
    made in developing risk management techniques, integrating disparate
    systems such as risk management with budget and program management remains
    a long-term challenge. The Secretary of DHS has said that operations and
    budgets of its agencies will be reviewed through the prism of risk, but
    doing this is made difficult by the level of guidance and coordination
    that has been provided so far.

    DHS has recently reorganized, and the consideration of whether its new
    organization will effectively implement its risk management
    responsibilities is an important one. At the time we conducted our review,
    risk management was the responsibility of IAIP. IAIP's risk management
    efforts were focused mainly on assessing and reducing the vulnerabilities
    that exist in and around specific facilities or assets. But DHS's
    responsibility is broader than this: besides assessing and reducing
    vulnerabilities at specific facilities, it also includes preventing
    attacks from occurring (and in the process protecting people and critical
    infrastructure) and responding to and recovering from natural disasters
    and acts of terrorism. This initial focus on vulnerabilities at specific
    assets had the potential of limiting DHS's ability to achieve the broader
    goal of using risk-based data as a tool to inform management decisions on
    all aspects of its missions. The Secretary of DHS has now moved risk
    management to a new Preparedness Directorate. Although, it is unclear how
    such a move could affect DHS's ability to carry out its risk management
    responsibilities, the new focus on preparedness may result in an emphasis
    that may go too far the other way-that is an emphasis on protection of
    specific assets and response and recovery at the expense of prevention. As
    DHS goes forward, the office in which the risk management responsibility
    resides should have a broad perspective across the department's entire
    mission as well as the necessary authority to hold DHS component agencies
    responsible for carrying out risk management activities in a coordinated
    and consistent manner.

    Beyond DHS, integrating risk with existing systems for budget and program
    review is complicated by the fact that while IAIP has responsibility for
    coordinating this effort, IAIP and the Secretary of DHS are challenged
    because they must depend on others to follow risk management principles
    for programs and budgets at the other six major Departments or agencies
    that have been charged with assessing risks under HSPD-7. In regard to
    this situation, OMB has taken the position that this is what the Homeland
    Security Act and HSPD-7 call for and it does not play a role in this
    process. These conditions increase the uncertainty of implementing risk
    management across federal agencies in a way that informs program and
    budget review processes. Whether such practices will occur within the
    executive branch is unclear because of these organizational barriers.

                          Recommendations for Executive Action

    To strengthen individual agency efforts to implement a risk management
    approach to homeland security activities, we recommend that the Secretary
    of Homeland Security direct the Undersecretary for IAIP to undertake the
    following three actions:

               o  As required by presidential directive, establish uniform
               policies, approaches, guidelines, and methodologies for
               integrating federal infrastructure protection and risk management
               activities within and across sectors, along with metrics and
               criteria for related programs and activities and develop a
               timetable for completing such guidance. Such policies and guidance
               should address the issue of integrating risk management systems
               into existing systems of program and budget review.

               o  As DHS continues to review its organizational structure, work
               with the Secretary's office to determine which office is best
               suited to help ensure that the responsibility for risk management
               policy and implementation has a broad enough perspective on all
               elements of risk, including threats, as well as the necessary
               authority to coordinate with DHS component agencies and hold them
               accountable for risk management activities.

               o  Work with the Office of Management and Budget to examine
               options for holding departments and agencies accountable for
               integrating risk management for homeland security programs and
               activities into the annual cycle of program and budget review.

                           Agency Comments and Our Evaluation

    In commenting on a draft of chapter 5, DHS, including IAIP, generally
    concurred with the recommendations. In regard to our observation that
    there is a long-term need for guidance and coordination, DHS noted that as
    part of the department's second-stage review, a six-point agenda has been
    created to ensure that policies, operations, and structures are best
    aligned to address potential threats. This agenda is a major step in the
    right direction, and as we observe in this chapter, much work remains to
    be done to translate this goal into actions.

    DHS agrees that the application of risk management to domestic terrorism
    has no precedent, and that the probabilities and consequences of terrorist
    acts are difficult to predict. DHS also concurred with our observation
    that the scope of establishing a risk management framework-a former IAIP
    Directorate responsibility-across the federal government is immense. DHS
    acknowledges that IAIP's progress has been limited in part because its
    risk assessment responsibilities span broad sectors of the nation's
    infrastructure rather than seaports alone. DHS also submitted written
    comments under separate cover and we revised the report where appropriate.
    Written comments from DHS are in appendix II.

    A Appendix I: A Risk Management Framework

    This appendix describes how we developed the risk management framework and
    how we used it to evaluate activities related to homeland security and
    combating terrorism. The framework is intended to be a starting point for
    risk management activities and will likely evolve as processes mature and
    lessons are learned. A glossary is included at the end of this appendix.

                  General Lack of Uniform Guidance on Risk Management

    Although the Homeland Security Act and subsequent strategies advocate the
    use of risk management to protect the nation's critical infrastructure and
    key resources, they did not define how this was to be accomplished.
    Homeland Security Presidential Directive 7 (HSPD-7) directed the Secretary
    of the Department of Homeland Security (DHS) to establish uniform
    policies, approaches, guidelines, and methodologies integrating federal
    infrastructure protection and risk management activities. However, no
    further direction or guidance as to the course of action has been
    forthcoming.

    The ability to anticipate future happenings and to choose among
    alternatives lies at the heart of risk management and provides us with a
    guide, based on good management practices and supported by established
    internal controls that can enhance decision making. Although risk
    management has long been used for assessing risk in some sectors, such as
    environmental issues, health care, finance, and the insurance industry,
    the application of risk management principles to the homeland security
    area is relatively new. The many areas and activities under homeland
    security provide untested and difficult challenges because the source of
    the risk is an intelligent adversary with whom there exists little
    domestic experience. As a result, the probabilities and consequences of a
    terrorist attack are difficult to predict. In spite of this high degree of
    uncertainty and the knowledge that not all risk can be eliminated,
    enhancing protection from known or potential threats can help prevent or
    mitigate adverse events.

                 Methodology for Developing a Risk Management Framework

    Given that there is no established universally agreed upon set of
    requirements or processes for a risk management framework specifically
    related to homeland security and combating terrorism, we developed a
    framework that would be applicable by reviewing, analyzing, and
    synthesizing several sources of information.

    We began by reviewing current risk literature and previous GAO reports and
    testimonies.1 We consulted the Government Performance and Results Act
    (GPRA) of 1993; the Government Auditing Standards, 2003 Revision, GAO's
    Standards for Internal Control in the Federal Government (November 1999);
    guidance from the Office of Management and Budget (OMB); the work of the
    President's Commission on Risk Management; consulting papers; and the
    enterprise risk management approach of the Committee of Sponsoring
    Organizations (COSO) of the Treadway Commission. In addition, we consulted
    with experts in the fields of risk management, risk modeling, and
    terrorism. We reviewed numerous frameworks from industry, government, and
    academic sources.

    We synthesized information from these numerous government, industry, and
    academic sources in developing our risk management framework. The
    framework was field-tested on several GAO reviews. The draft framework was
    then reviewed by three academic experts in risk management. No substantial
    changes to the draft framework were recommended.

                              A Risk Management Framework

    The framework should be considered to be a starting point in a field that
    is evolving, and the entire cycle of risk management activities should be
    viewed as a goal. The phases contained in the framework are

               o  strategic goals, objectives, and constraints;
               o  risk assessment;
               o  alternatives evaluation;
               o  management selection; and
               o  implementation and monitoring.

    1GAO, Combating Terrorism: Threat and Risk Assessments Can Help Prioritize
    and Target Program Investments, GAO/NSIAD-98-74 (Washington, D.C.: April
    1998); Combating Terrorism: Evaluation of Selected Characteristics in
    National Strategies Related to Terrorism , GAO-04-408T (Washington, D.C.:
    Feb. 3, 2004); Rail Security: Some Actions Taken to Enhance Passenger and
    Freight Rail Security, but Significant Challenges Remain, GAO-04-598T
    (Washington, D.C.: Mar. 23, 2004); and Homeland Security: Summary of
    Challenges Faced in Targeting Oceangoing Cargo Containers for Inspection,
    GAO-04-557T (Washington, D.C.: Mar. 31, 2004).

    The framework has been developed so that individual phases of the
    approach, such as risk assessment, do not become ends in themselves, but
    provide a full cycle of related activities, from strategic planning
    through implementation and monitoring. The process is dynamic, and
    although the various phases appear linear, new information can be entered
    at any phase. The framework can be used to inform agency officials and
    decision makers of the basic components of a risk management system or can
    be used as a stand-alone guide. Figure 4 illustrates our risk management
    framework and some sources of criteria, such as GAO best practices, Office
    of Management and Budget circulars, GAO guidance on internal controls, and
    the Government and Performance Act of 1993 and their link with management
    processes. While statistical methods and risk-ranking approaches
    frequently underlie risk assessment approaches, different application
    areas tend to develop their own terminologies and their own logical
    sequences for the cause of risk.

    Figure 4: Sources of Evaluation Criteria Associated with Risk Management
    Phases

    The risk management framework is designed to be flexible, in that the
    approach may be applied at various organizational levels ranging from that
    of a department level of a multiagency organization down to that of a
    specific project or program.

    Because there is no one uniformly accepted approach to risk management,
    terms and activities may differ across applications. However, any approach
    that omits the substance of the steps shown in figure 4 is likely to have
    material weaknesses. Table 11 summarizes the phases of our risk management
    framework and provides examples of elements contained in those phases.

    Table 11: A Risk Management Framework

    Source: GAO.

    The following sections provide more detail on the five phases of our risk
    management framework.

                      Strategic Goals, Objectives, and Constraints

    This phase addresses what the strategic goals are attempting to achieve
    and the steps needed to attain those results, including milestones and
    performance measures to permit measurement of progress toward those goals.
    Ideally, management decisions should be made in the context of a strategic
    plan, with clearly articulated goals and objectives that flow from the
    plan. Strategic goals at the highest level could be considered an
    "end-state" followed by a logical hierarchy of major goals and subordinate
    objectives composed of clear, concise, measurable activities and timelines
    to achieve results and ensure accountability. An organization's program or
    plan and risk planning documents should address risk-related issues that
    are central to its mission. Our work related to the Government Performance
    and Results Act of 1993 has produced guidance that identifies risk for the
    congressional oversight of federal agencies' strategic plans.2 The
    consideration of risk in strategic planning may be incorporated into
    planning documents or into specific management strategies.

    Currently, it is difficult to translate plans and actions into a clear
    sense of how we are progressing in making our nation more secure. One
    reason is that homeland security efforts, in general, lack clear goals
    with corresponding performance measures to evaluate progress toward these
    goals.3 As others, such as the Gilmore Commission, have stated, a
    continuing problem for homeland security has been the lack of clear
    strategic guidance about the definition and objectives of preparedness.4

    Risk management allows entities to operate more effectively in
    environments of uncertainty by providing the discipline and structure in
    which to address these issues, since risk management is not an end in
    itself, but an important component of an entity's management process. As
    such, risk management is interrelated with, among other things, an
    entity's governance, performance management, and internal controls. The
    process of risk management provides the rigor and structure necessary to
    identify and select among alternative risk responses whose cumulative
    effect is intended to reduce risk, and the methodologies and techniques
    for making selection decisions. This process enables entities to enhance
    their capability to identify potential adverse events, assess risks, and
    establish integrated responses. Further, this phase in the planning
    process would include support and buy-in from upper levels of management
    and stakeholders. Acceptance for concepts of the model from this group
    provides the groundwork for future discussions.

    2GAO, Agencies' Strategic Plans under GPRA: Key Questions to Facilitate
    Congressional Review, GAO/GGD-10.1.16 (Washington, D.C.: May 1, 1997).

    3GAO, Homeland Security: Key Elements to Unify Efforts Are Underway but
    Uncertainty Remains, GAO-02-610 (Washington, D.C.: June 7, 2002).

    4The Gilmore Commission (also known as the Advisory Panel to Assess
    Domestic Response Capabilities for Terrorism Involving Weapons of Mass
    Destruction), Forging America's New Normalcy (Arlington, Virginia: Dec.
    15, 2003).

    Finally, various constraints may have an impact on risk management plans.
    Some constraints may be imposed by statute, higher-level policy, budget,
    or other factors beyond management's control and may vary with the scale
    of the application. Managers at different levels within an organization
    will have different degrees of flexibility to institute risk management
    countermeasures. An important constraint for federal agencies, such as
    DHS, is the role that Congress plays in authorizing and funding programs.
    For example, Congress may direct specific actions affecting how agencies
    allocate funding.

                                    Risk Assessment

    This phase addresses the process of evaluating the threats and
    vulnerabilities of assets so that countermeasures might be instituted to
    prevent or mitigate risks. Threat, in the risk management model, concerns
    the probability that a specific type of attack will be initiated against a
    specific target. It includes any circumstance or event with the potential
    to cause loss or damage to the asset. Although agencies may not have
    enough information to identify and characterize all threats related to
    their assets, known or imagined adverse events would be characterized in
    some detail. Effective threat analysis is dependent on an understanding of
    an adversary's intention, motivation, historical data, and capability to
    damage. An additional crucial component of risk assessment is
    vulnerability, that is, any weakness that an adversary can exploit to harm
    or damage the asset. An asset may be highly vulnerable to one mode of
    attack but have a low level of vulnerability to another, depending on a
    variety of factors, such as countermeasures already in place. While
    consequence concerns the result of an adverse event on a particular asset,
    criticality is the asset's relative importance to the entity. A
    criticality assessment identifies and prioritizes assets and functions in
    terms of specific criteria, such as their importance to public safety and
    the economy, as a basis for identifying which structures or processes are
    relatively more important to protect from attack. Criticality assessments
    are important because they provide, in combination with the framework's
    threat and risk assessments, the basis for prioritizing which assets
    require greater protection relative to finite resources and provide
    information for later stages in the risk management process. Risk
    assessments should utilize the most appropriate subject matter experts in
    assessing the components of risk. When dealing with Bayesian probability
    estimates, this is critical.5 In addition, mathematical constructs must be
    chosen carefully, specifically tailoring approaches in the context of
    uncertainty and data quality.

                                 Alternative Evaluation

    This phase addresses the evaluation of risk reduction methods by
    consideration of countermeasures or countermeasure systems and the costs
    and benefits associated with them. Ideally, a risk management framework
    would include an evaluation of a risk assessment as a valid decision
    support tool to establish and prioritize a risk management strategy.
    Furthermore, a strategy might include risk management consultants and
    decision-making tools, such as software that simulates a particular type
    of attack. Information developed in previous phases would inform
    decisions.

    Specific countermeasures would be considered and prioritized based on a
    number of factors, such as the degree of risk reduction they afford and
    the cost and difficulty to implement them. Risk assessment may give
    guidance to implementing countermeasures or countermeasure systems that
    may be used for more than one critical asset, or to prevent, mitigate, or
    respond to multiple adverse events occurring simultaneously. In addition,
    external risk consultants can be advantageous at this phase in terms of
    creating a variety of countermeasure options. While external reviewers
    cannot ensure the success of a plan, they can increase the probability of
    selecting the most effective countermeasures for the least cost because of
    their expertise in the area. Further, risk scenarios might also provide
    valuable information pertaining to an entity's ability to respond to a
    terrorist event, evaluate its coordination with other entities, identify
    problems, and institute corrective action.

    Finally, a risk management strategy should include a cost-benefit analysis
    of countermeasure options as they are a critical element of alternatives
    evaluation. Major regulatory actions or capital investments of federal
    expenditures generally require a cost-benefit or cost-effectiveness
    approach.6 This approach can be useful in evaluating alternatives, since
    it links the benefits from risk-reducing countermeasures to the costs
    associated with them. In the development of such analyses, quantitative
    impacts affecting both costs and benefits are, to the extent possible,
    identified in monetary terms.

    5See Yacov Y. Haimes, Risk Modeling, Assessment, and Management, 2nd ed.
    (Hoboken, New Jersey: Wiley and Sons, 2004).

    While the core OMB guidance for evaluating countermeasures for budgetary
    and regulatory purposes focuses on monetary cost-benefit evaluation, OMB
    is essentially silent when costs and benefits cannot be easily quantified
    or monetized. Costs that are not generally estimated or included in
    monetary terms include opportunity costs, that is, the value of
    opportunities forgone because resources are applied to antiterrorism
    countermeasures. These costs are most controversial when considered in
    areas such as public service programs or services curtailed or cancelled.
    Benefits are usually measured in terms of the risk reduction they provide.
    They are considered in terms of the overall effectiveness of the
    countermeasures with respect to the estimated vulnerabilities.

                                  Management Selection

    This phase addresses such issues as determining where resources and
    investments will be made, the sources and types of resources needed, and
    where those resources would be targeted. Management's active participation
    in this phase is important as decisions are of necessity influenced by the
    preferences and value judgments of agency leadership. For example, some
    managers will prefer to concentrate countermeasures on a relatively few
    critical assets, while others may value distributional impacts, that is,
    to distribute resources over a wide variety of assets. Ideally, a risk
    management strategy would also identify appropriate mechanisms to allocate
    resources, such as grants based on identified needs. Furthermore, a key
    factor in the selection of risk reducing measures is risk tolerance, the
    level of comfort management has with various levels of risk. This
    tolerance may change over time, depending on new information, changes in
    financial constraints, and attitude toward risk. The risk management
    strategy, with stakeholder input, will identify what constitutes an
    acceptable level of risk for assets and how resources are delegated.

    6Executive Order 12866 and circulars A-4 and A-94 apply to regulatory
    actions, and circulars A-11 (sect. 7), A-94, and A-130 apply to capital
    investments.

    The risk management strategy reflects consideration as to which risks
    should be managed immediately and to what extent, and which risks can be
    deferred and addressed at a later time. Milestones and timelines for
    implementation are important elements that allow evaluating the extent to
    which progress is being made toward achieving goals. It also illustrates
    the degree of protection that can be obtained and places security and
    costs in perspective. The documentation of management decisions, including
    the rationales that support the decisions, will provide valuable
    information for future adjustments.

                             Implementation and Monitoring

    This phase addresses the degree to which risk management strategies
    contain internal controls and performance measurement guidelines. In
    addition to implementing countermeasures, it may also include implementing
    new organizational policies and procedures, as well as human, physical,
    and technical controls. Countermeasures would be initiated in accordance
    with the timelines in the risk management schedule.

    Monitoring and evaluation include, when and where appropriate, external
    peer review, testing and validation of countermeasures, evaluating the
    effects of these actions on future operation, and identifying unintended
    consequences. GAO has also discussed the importance of these activities as
    they ensure actions are taken to address risks.

    Internal control monitoring should assess the quality of performance over
    time and ensure that the findings of audits and other reviews are promptly
    resolved.7 Internal controls help to ensure that procedures are documented
    and maintained.

                   Difficulties Applying a Risk Management Framework

    Risk management represents a unique set of challenges. Developing such a
    framework is designed to guide the actions of management to prepare for
    and respond to adverse events in an environment of uncertainty. As applied
    to homeland security, the ability to determine the likelihood of
    terrorism-related events occurring and quantifying the resulting outcomes
    is balanced against the benefit as protection (security) provided at an
    acceptable cost.

    7GAO, Standards for Internal Control in the Federal Government,
    GAO/AIMD-00-21 .3.1, (Washington, D.C.: November 1999).

    Generally, a major difficulty in executing and implementing a risk
    management system occurs because of what is commonly called decision
    uncertainty, that is, the assumed goal of minimizing risk does not have a
    common meaning.8 For example, minimizing risk is based on values, the
    measure of risk, and the comparison of values and risk. Decision
    uncertainty arises when there is controversy or ambiguity concerning how
    to compare and weight social objectives. Three major sources of decision
    uncertainty are

               o  Risk measurement-although the selection of risk measurement is
               both an art and a science, it must be technically correct as well
               as both valid and meaningful.

               o  The social cost of risk-in order to make different risks
               comparable, various risks often have to be quantified into
               comparable quantities and placing values on cost and benefits
               involves judgments.

               o  The quantification of social values-uncertainty surrounds the
               level of risk that is acceptable or can be tolerated. That is, how
               much money is to be spent on protection meaning risk reduction and
               what is the cost in terms of opportunities forgone because of
               finite resources. This value is dependent upon determining
               society's risk attitude or tolerance and may change over time.

    Coordination activities suggest that for the results of a risk management
    system to be meaningful and useful, all related agencies should be using
    similar methods. If agencies' methods are not compatible, then comparisons
    between agencies become difficult and sector or national risk assessments
    becomes less reliable. In our earlier work, we concluded that a
    structured, systematic approach to risk management offers the best
    assurance that activities designed to protect the homeland and combat the
    effects of terrorism will produce the most effective and efficient
    results.9 Specific difficulties implementing risk management systems are
    contained in chapters 2, 3, and 4.

    8For a more detailed discussion on this topic, see Yacov Y Haimes, Risk
    Modeling, Assessment, and Management, 2nd ed. (Hoboken, New Jersey: Wiley
    and Sons, 2004).

    9See for example, GAO, Homeland Security: Summary of Challenges Faced in
    Targeting Oceangoing Cargo Containers for Inspection, GAO-04-557T
    (Washington, D.C.: Mar. 31, 2004); Homeland Security: A Risk Management
    Approach Can Guide Preparedness Efforts, GAO-02-208T (Washington, D.C.:
    Oct. 31, 2001); and Key Elements of a Risk Management Approach,
    GAO-02-150T (Washington, D.C.: Oct. 12, 2001).

                           Glossary of Risk Management Terms

    For purposes of our risk management framework, we use the following
    definitions:

               o  Asset--any person, facility, material, information, or activity
               that has a positive or symbolic value. An asset may have a value
               to an adversary as well as to its owner, although the nature and
               magnitude of these values may differ. Assets may be categorized or
               combined in many ways; examples are people, information,
               equipment, facilities, operations, and activities.

               o  Benefit--net outcome, usually translated into monetary terms; a
               benefit may include both direct and indirect effects.

               o  Consequence--the expected worse case or reasonable worse case
               impact of a successful attack. The consequence to a particular
               asset can be evaluated when threat and vulnerability are
               considered together. This loss or damage may be long- or
               short-term in nature.

               o  Cost--input, both direct and indirect.

               o  Cost-benefit analysis--part of the management decision-making
               process in which the costs and benefits of each countermeasure
               alternative are compared and the most appropriate alternative is
               selected. Costs include the price paid for tangible materials and
               the ongoing operational costs associated with implementing the
               countermeasures. Benefits are expressed in terms of the amount of
               risk reduction based on the overall effectiveness of the
               countermeasure with respect to the assessed vulnerabilities.

               o  Countermeasure--any action taken or physical equipment used
               principally to reduce or eliminate one or more vulnerabilities.
               The cost of a countermeasure is usually expressed in monetary
               terms but may include nonmonetary costs such as reduced
               operational effectiveness, unfavorable working conditions, adverse
               publicity and political consequences.

               o  Criticality assessment--identifies and evaluates an entity's
               assets or operations on the basis of a variety of factors,
               including the importance of an asset or function and the
               significance of a system in terms of national security, economic
               activity, and public safety. A criticality assessment provides the
               basis for determining which assets require greater or special
               protection relative to finite resources.

               o  Impact--the amount of loss or damage that can be expected from
               a successful attack on an asset. Loss may be monetary, but may
               include loss of lives and destruction of a symbolic structure.

               o  Monitoring and evaluation--is a continuous repetitive
               assessment process to keep a risk management process current and
               relevant. It includes, among other activities, external peer
               review, testing, and validation.

               o  Opportunity cost-the value of opportunities forgone.

               o  Risk-an event that has a potentially negative impact and the
               possibility that such an event will occur and adversely affect an
               entity's assets, activities, and operations. The principal classes
               of risk from terrorism are to the general public, targets of
               symbolic value, organizational, governmental, and societal
               infrastructure, cyber and physical infrastructure, and economic
               sectors and structures.

               o  Risk assessment--the process of qualitatively or quantitatively
               determining the probability of an adverse event and the severity
               of its impact on an asset. It is a function of threat,
               vulnerability, and consequence. A risk assessment may include
               scenarios in which two or more risks interact to create a greater
               or lesser impact. A risk assessment provides the basis for the
               rank ordering of risks and for establishing priorities for
               applying countermeasures.

               o  Risk management--a continuous process of managing-through a
               series of mitigating actions that permeate an entity's
               activities-the likelihood of an adverse event and its negative
               impact. Risk management addresses risk before mitigating action,
               as well as the risk that remains after countermeasures have been
               taken.

               o  Scenario--the combination of weapon and attack mode on a
               specific target or critical asset (for example, the release of
               sarin gas in a subway train).

               o  Threat--an indication of the likelihood that a specific type of
               attack will be initiated against a specific target or class of
               targets. It may include any indication, circumstance, or event
               with the potential to cause the loss of or damage to an asset. It
               can also be defined as an adversary's intention and capability to
               undertake actions that would be detrimental to a valued asset.

               o  Threat assessment--the identification and evaluation of adverse
               events that can harm or damage an asset. A threat assessment
               includes the probability of an event and the extent of its
               lethality. Threats may be present at the global, national, or
               local level.

               o  Vulnerability--the probability that a particular attempted
               attack will succeed against a particular target or class of
               targets.

               o  Vulnerability assessment--the identification of weaknesses in
               physical structures, personal protection systems, processes or
               other areas that may be exploited. A vulnerability assessment
               identifies inherent states and the extent of their susceptibility
               to exploitation relative to the existence of any countermeasures.

    Appendix II: Comments from the Department of Homeland Security Appendix
    II: Comments from the Department of Homeland Security

    Appendix III: GAOA Appendix III: GAO Contacts and Staff Acknowledgments

    GAO Contacts

    Margaret T. Wrightson (415) 904-2200

    Stephen L. Caldwell (202) 512-9610

    Staff Acknowledgments

    In addition to the persons named above, David Alexander, Neil Asaba, Nancy
    A. Briggs, Christine Davis, Scott Farrow, Kevin Heinz, Emily S. Pickrell,
    Albert Schmidt, Stan Stenersen, April Thompson, and L. James Valverde made
    key contributions to this report.

    Related GA Related GAO Products

    Critical Infrastructure Protection

    Critical Infrastructure Protection: Challenges in Addressing
    Cybersecurity. GAO-05-827T . Washington, D.C.: July 19, 2005

    Critical Infrastructure Protection: Department of Homeland Security Faces
    Challenges in Fulfilling Cybersecurity Responsibilities. GAO-05-434 .
    Washington, D.C.: May 26, 2005.

    Homeland Security: Agency Plans, Implementation, and Challenges Regarding
    the National Strategy for Homeland Security. GAO-05-33 . Washington, D.C.:
    January 14, 2005.

    Homeland Security: Further Actions Needed to Coordinate Federal Agencies'
    Facility Protection Efforts and Promote Key Practices. GAO-05-49 .
    Washington, D.C.: November 30, 2004.

    U.S. Postal Service: Physical Security Measures Have Increased at Some
    Core Facilities, but Security Problems Continue. GAO-05-48 . Washington,
    D.C.: November 16, 2004.

    Drinking Water: Experts' Views on How Federal Funding Can Best Be Spent to
    Improve Security. GAO-04-1098T . Washington, D.C.: September 30, 2004.

    Financial Market Preparedness: Improvements Made, but More Action Needed
    to Prepare for Wide-Scale Disasters. GAO-04-984 . Washington, D.C.:
    September 27, 2004.

    Nuclear Regulatory Commission: Preliminary Observations on Efforts to
    Improve Security at Nuclear Power Plants. GAO-04-1064T . Washington, D.C.:
    September 14, 2004.

    U.S. Postal Service: Better Guidance Is Needed to Ensure an Appropriate
    Response to Anthrax Contamination. GAO-04-239 . Washington, D.C.:
    September 9, 2004.

    Public Key Infrastructure: Examples of Risk and Internal Control
    Objectives Associated with Certification Authorities. GAO-04-1023R .
    Washington, D.C.: August 10, 2004.

    Combating Terrorism: DOD Efforts to Improve Installation Preparedness Can
    Be Enhanced with Clarified Responsibilities and Comprehensive Planning.
    GAO-04-855 . Washington, D.C.: August 9, 2004.

    Homeland Security: Transformation Strategy Needed to Address Challenges
    Facing the Federal Protective Service. GAO-04-537 . Washington, D.C.: July
    14, 2004.

    Critical Infrastructure Protection: Improving Information Sharing with
    Infrastructure Sectors. GAO-04-780 . Washington, D.C.: July 9, 2004.

    Information Security: Agencies Need to Implement Consistent Processes in
    Authorizing Systems for Operation. GAO-04-376 . Washington, D.C.: June 28,
    2004.

    National Nuclear Security Administration: Key Management Structure and
    Workforce Planning Issues Remain as NNSA Conducts Downsizing. GAO-04-545 .
    Washington, D.C.: June 25, 2004.

    Nuclear Security: Several Issues Could Impede Ability of DOE's Office of
    Energy, Science, and Environment to Meet the May 2003 Design Basis Threat.
    GAO-04-894T . Washington, D.C.: June 22, 2004.

    Information Security: Information System Controls at the Federal Deposit
    Insurance Corporation. GAO-04-630 . Washington, D.C.: May 28, 2004.

    Posthearing Questions Related to Fragmentation and Overlap in the Federal
    Food Safety System. GAO-04-832R . Washington, D.C.: May 26, 2004.

    Terrorism Insurance: Effects of Terrorism Risk Insurance Act of 2002.
    GAO-04-720T . Washington, D.C.: April 28, 2004.

    Nuclear Security: DOE Needs to Resolve Significant Issues before It Fully
    Meets the New Design Basis Threat. GAO-04-623 . Washington, D.C.: April
    27, 2004.

    Terrorism Insurance: Implementation of the Terrorism Risk Insurance Act of
    2002. GAO-04-307 . Washington, D.C.: April 23, 2004.

    Critical Infrastructure Protection: Establishing Effective Information
    Sharing with Infrastructure Sectors. GAO-04-699T . Washington, D.C.: April
    21, 2004.

    Homeland Security: Federal Action Needed to Address Security Challenges at
    Chemical Facilities. GAO-04-482T . Washington, D.C.: February 23, 2004.

    Posthearing Questions from the September 17, 2003, Hearing on
    "Implications of Power Blackouts for the Nation's Cybersecurity and
    Critical Infrastructure Protection: The Electric Grid, Critical
    Interdependencies, Vulnerabilities, and Readiness." GAO-04-300R .
    Washington, D.C.: December 8, 2003.

    Security: Counterfeit Identification Raises Homeland Security Concerns.
    GAO-04-133T . Washington, D.C.: October 1, 2003.

    Nuclear Regulatory Commission: Oversight of Security at Commercial Nuclear
    Power Plants Needs to Be Strengthened. GAO-03-752 . Washington, D.C.:
    September 4, 2003.

    Nuclear Security: DOE Faces Security Challenges in the Post September 11,
    2001, Environment. GAO-03-896TNI . Washington, D.C.: June 24, 2003.

    Nuclear Security: NNSA Needs to Better Manage Its Safeguards and Security
    Program. GAO-03-471 . Washington, D.C.: May 30, 2003.

    Information Security: Progress Made, but Challenges Remain to Protect
    Federal Systems and the Nation's Critical Infrastructures. GAO-03-564T .
    Washington, D.C.: April 8, 2003.

    Homeland Security: EPA's Management of Clean Air Act Chemical Facility
    Data. GAO-03-509R . Washington, D.C.: March 14, 2003.

    Homeland Security: Voluntary Initiatives Are Under Way at Chemical
    Facilities, but the Extent of Security Preparedness Is Unknown. GAO-03-439
    . Washington, D.C.: March 14, 2003.

    Potential Terrorist Attacks: Additional Actions Needed to Better Prepare
    Critical Financial Market Participants. GAO-03-414 . Washington, D.C.:
    February 12, 2003.

    Potential Terrorist Attacks: More Actions Needed to Better Prepare
    Critical Financial Markets. GAO-03-468T . Washington, D.C.: February 12,
    2003.

    Potential Terrorist Attacks: Additional Actions Needed to Better Prepare
    Critical Financial Market Participants. GAO-03-251 . Washington, D.C.:
    February 12, 2003.

    High-Risk Series: Protecting Information Systems Supporting the Federal
    Government and the Nation's Critical Infrastructures. GAO-03-121 .
    Washington, D.C.: January 1, 2003.

    Combating Terrorism: Actions Needed to Guide Services' Antiterrorism
    Efforts at Installations. GAO-03-14 . Washington, D.C.: November 1, 2002.

    Homeland Security: Department of Justice's Response to Its Congressional
    Mandate to Assess and Report on Chemical Industry Vulnerabilities.
    GAO-03-24R . Washington, D.C.: October 10, 2002.

    Building Security: Interagency Security Committee Has Had Limited Success
    in Fulfilling Its Responsibilities. GAO-02-1004 . Washington, D.C.:
    September 17, 2002.

    Chemical Safety: Emergency Response Community Views on the Adequacy of
    Federally Required Chemical Information. GAO-02-799 . Washington, D.C.:
    July 31, 2002.

    Critical Infrastructure Protection: Significant Homeland Security
    Challenges Need to Be Addressed. GAO-02-918T . Washington, D.C.: July 9,
    2002.

    Information Security: Corps of Engineers Making Improvements, but
    Weaknesses Continue. GAO-02-589 . Washington, D.C.: June 10, 2002.

    Security Breaches at Federal Buildings in Atlanta, Georgia. GAO-02-668T .
    Washington, D.C.: April 30, 2002.

    National Preparedness: Technologies to Secure Federal Buildings.
    GAO-02-687T . Washington, D.C.: April 25, 2002.

    Diffuse Security Threats: Technologies for Mail Sanitation Exist, but
    Challenges Remain. GAO-02-365 . Washington, D.C.: April 23, 2002.

    Terrorism Insurance: Rising Uninsured Exposure to Attacks Heightens
    Potential Economic Vulnerabilities. GAO-02-472T . Washington, D.C.:
    February 27, 2002.

    Critical Infrastructure Protection: Significant Challenges in Safeguarding
    Government and Privately Controlled Systems from Computer-Based Attacks.
    GAO-01-1168T . Washington, D.C.: September 26, 2001.

    Combating Terrorism: Actions Needed to Improve DOD Antiterrorism Program
    Implementation and Management. GAO-01-909 . Washington, D.C.: September
    19, 2001.

    Critical Infrastructure Protection: Significant Challenges in Protecting
    Federal Systems and Developing Analysis and Warning Capabilities.
    GAO-01-1132T . Washington, D.C.: September 12, 2001.

    Maritime Security

    Coast Guard: Progress Being Made on Addressing Deepwater Legacy Asset
    Condition Issues and Program Management, but Acquisition Challenges
    Remain. GAO-05-757 . Washington, D.C.: July 22, 2005.

    Coast Guard-Electronic Certification Procedures. B-302789. Washington,
    D.C.: July 6, 2005

    Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy
    Assets and Acquisition Management Challenges. GAO-05-651T . Washington,
    D.C.: June 21, 2005.

    Maritime Security: Enhancements Made, but Implementation and
    Sustainability Remain Key Challenges. GAO-05-448T . Washington, D.C.: May
    17, 2005.

    Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy
    Assets and Acquisition Management Challenges. GAO-05-307T. Washington,
    D.C.: April 20, 2005.

    Maritime Security: New Structures Have Improved Information Sharing, but
    Security Clearance Processing Requires Further Attention. GAO-05-394 .
    Washington, D.C.: April 15, 2005.

    Coast Guard: Observations on Agency Priorities in Fiscal Year 2006 Budget
    Request. GAO-05-364T. Washington, D.C.: March 17, 2005.

    Homeland Security: Process for Reporting Lessons Learned from Seaport
    Exercises Needs Further Attention. GAO-05-17 . Washington, D.C.: January
    14, 2005.

    Coast Guard: Station Readiness Improving, but Resources Challenges and
    Management Concerns Remain. GAO-05-161 . Washington, D.C.: January 31,
    2005.

    Port Security: Planning Needed to Develop and Operate Maritime Worker
    Identification Card Program. GAO-05-106 . Washington, D.C.: December 10,
    2004.

    Maritime Security: Better Planning Needed to Help Ensure an Effective Port
    Security Assessment Program. GAO-04-1062 . Washington, D.C.: September 30,
    2004.

    Maritime Security: Partnering Could Reduce Federal Costs and Facilitate
    Implementation of Automatic Vessel Identification System. GAO-04-868 .
    Washington, D.C.: July 23, 2004.

    Maritime Security: Substantial Work Remains to Translate New Planning
    Requirements into Effective Port Security. GAO-04-838 . Washington, D.C.:
    June 30, 2004.

    Coast Guard: Station Spending Requirements Met, but Better Processes
    Needed to Track Designated Funds. GAO-04-695 . June 14, 2004.

    Coast Guard: Key Management and Budget Challenges for Fiscal Year 2005 and
    Beyond. GAO-04-636T . Washington, D.C.: April 7, 2004.

    Homeland Security: Summary of Challenges Faced in Targeting Oceangoing
    Cargo Containers for Inspection. GAO-04-557T . Washington, D.C.: March 31,
    2004.

    Coast Guard Programs: Relationship between Resources Used and Results
    Achieved Needs to Be Clearer. GAO-04-432 . Washington, D.C.: March 22,
    2004.

    Homeland Security: Preliminary Observations on Efforts to Target Security
    Inspections of Cargo Containers. GAO-04-325T . Washington, D.C.: December
    16, 2003.

    Posthearing Questions Related to Aviation and Port Security. GAO-04-315R .
    Washington, D.C.: December 12, 2003.

    Maritime Security: Progress Made in Implementing Maritime Transportation
    Security Act, but Concerns Remain. GAO-03-1155T . Washington, D.C.:
    September 9, 2003.

    Container Security: Expansion of Key Customs Programs Will Require Greater
    Attention to Critical Success Factors. GAO-03-770 . Washington, D.C.: July
    25, 2003.

    Homeland Security: Challenges Facing the Department of Homeland Security
    in Balancing its Border Security and Trade Facilitation Missions.
    GAO-03-902T . Washington, D.C.: June 16, 2003.

    Coast Guard: Challenges during the Transition to the Department of
    Homeland Security. GAO-03-594T . Washington, D.C.: April 1, 2003.

    Transportation Security: Post-September 11th Initiatives and Long-Term
    Challenges. GAO-03-616T . Washington, D.C.: April 1, 2003.

    Coast Guard: Comprehensive Blueprint Needed to Balance and Monitor
    Resource Use and Measure Performance for All Missions. GAO-03-544T .
    Washington, D.C.: March 12, 2003.

    Homeland Security: Challenges Facing the Coast Guard as It Transitions to
    the New Department. GAO-03-467T . Washington, D.C.: February 12, 2003.

    Coast Guard: Strategy Needed for Setting and Monitoring Levels of Effort
    for All Missions. GAO-03-155 . Washington, D.C.: November 12, 2002.

    Port Security: Nation Faces Formidable Challenges in Making New
    Initiatives Successful. GAO-02-993T . Washington, D.C.: August 5, 2002.

    Combating Terrorism: Preliminary Observations on Weaknesses in Force
    Protection for DOD Deployments through Domestic Seaports. GAO-02-955TNI .
    Washington, D.C.: July 23, 2002.

    Risk Management

    Passenger Rail Security: Enhanced Federal Leadership Needed to Prioritize
    and Guide Security Efforts. GAO-05-851 . September 9, 2005

    Homeland Security: Actions Needed to Better Protect National Icons and
    Federal Office Buildings from Terrorism. GAO-05-790 . June 24, 2005.

    Protection of Chemical and Water Infrastructure: Federal Requirements,
    Actions of Selected Facilities, and Remaining Challenges. GAO-05-327 .
    Washington, D.C.: March 28, 2005.

    Transportation Security: Systematic Planning Needed to Optimize Resources.
    GAO-05-357T . February 15, 2005

    Homeland Security: Agency Plans, Implementation, and Challenges Regarding
    the National Strategy for Homeland Security. GAO-05-33 . January 14, 2005.

    Homeland Security: Further Actions Needed to Coordinate Federal Agencies'
    Facility Protection Efforts and Promote Key Practices. GAO-05-49 .
    November 30, 2004

    General Aviation Security: Increased Federal Oversight is Needed, but
    Continued Partnership with the Private Sector is Critical to Long-Term
    Success. GAO-05-144 . November 10, 2004.

    Air Traffic Control: System Management Capabilities Improved, but More Can
    Be Done to Institutionalize Improvements. GAO-04-901 . August 20, 2004

    Aviation Security: Challenges in Using Biometric Technologies. GAO-04-785T
    . May 19, 2004.

    Homeland Security: Summary of Challenges Faced in Targeting Oceangoing
    Cargo Containers for Inspection. GAO-04-557T . March 31, 2004

    Rail Security: Some Actions Taken to Enhance Passenger and Freight Rail
    Security, but Significant Challenges Remain. GAO-04-598T . March 23, 2004.

    Homeland Security: A Risk Management Approach Can Guide Preparedness
    Efforts. GAO-02-208T . Washington, D.C.: October 31, 2001.

    Homeland Security: Key Elements of a Risk Management Approach. GAO-02-150T
    . Washington, D.C.: October 12, 2001.

    (440378)

    GAO's Mission

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    Obtaining Copies of GAO Reports and Testimony

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
    released reports, testimony, and correspondence on its Web site. To have
    GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    To Report Fraud, Waste, and Abuse in Federal Programs

    Contact:

    Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
    Automated answering system: (800) 424-5454 or (202) 512-7470

    Congressional Relations

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Washington,
    D.C. 20548

    Public Affairs

    Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S.
    Government Accountability Office, 441 G Street NW, Room 7149 Washington,
    D.C. 20548

    www.gao.gov/cgi-bin/getrpt? GAO-06-91 .

    To view the full product, including the scope

    and methodology, click on the link above.

    For more information, contact Margaret Wrightson at (415) 904-2200 or
    wrightsonm@gao.gov.

    Highlights of GAO-06-91 , a report to Congressional Requesters

    December 2005

    RISK MANAGEMENT

    Further Refinements Needed to Assess Risks and Prioritize Protective
    Measures at Ports and Other Critical Infrastructure

    Congress and the President have called for various homeland security
    efforts to be based on risk management-a systematic process for assessing
    threats and taking appropriate steps to deal with them. GAO examined how
    three Department of Homeland Security components were carrying out this
    charge:

               o  the Coast Guard, which has overall responsibility for security
               in the nation's ports;
               o  the Office for Domestic Preparedness (ODP), which awards grants
               for port security projects; and
               o  the Information Analysis and Infrastructure Protection
               Directorate (IAIP), which has responsibility for developing ways
               to assess risks across all types of critical infrastructure.

    GAO's work focused on identifying the progress each DHS component has made
    on risk management and the challenges each faces in moving further.

    What GAO Recommends

    This report contains many recommendations aimed at helping the three
    components face their next risk management challenges. DHS, including the
    Coast Guard, ODP, and IAIP, generally concurred with the report and its
    recommendations. DHS said that all three components have actions under way
    to address many of the recommendations in this report.

    The three DHS components GAO studied varied considerably in their progress
    in developing a sound risk management framework for homeland security
    responsibilities. The varied progress reflects, among other things, each
    component's organizational maturity and the complexity of its task (see
    table below). The Coast Guard, which is furthest along, is the component
    of longest standing, being created in 1915, while IAIP came into being
    with the creation of the Department of Homeland Security in 2003. IAIP,
    which has made the least progress, is not only a new component but also
    has the most complex task-addressing not just ports but all types of
    infrastructure. The Coast Guard and ODP have a relatively robust
    methodology in place for assessing risks at ports; IAIP is still
    developing its methodology and has had several setbacks in completing the
    task. All three components, however, have much left to do. In particular,
    each component is limited in its ability to compare and prioritize risks.
    The Coast Guard and ODP can do so within a port but not between ports;
    IAIP has not demonstrated that it can do so either within or between all
    infrastructure sectors.

    Each component faces many challenges in making further progress. Success
    will depend partly on continuing to improve various technical and
    management processes that are part of risk management. For example,
    obtaining better quality data from intelligence agencies would help DHS
    components estimate the relative likelihood of various types of threats-a
    key element of assessing risks. In the longer term, progress will depend
    increasingly on how well risk management is coordinated across agencies,
    because current approaches in many ways are neither consistent nor
    comparable. Also, weaving risk-based data into the annual budget cycle of
    program review will be important. Supplying the necessary guidance and
    coordination is what the Department of Homeland Security was set up to do
    and, as the Secretary of Homeland Security has stated, what it now needs
    increasingly to address. This is a key issue for the department as it
    seeks to identify relative risks and take appropriate actions related to
    the nation's homeland security activities.

    Progress in Risk Management Is Affected by Organizational Maturity and
    Complexity of Risk Management Task

    Source: GAO.
    *** End of document. ***