Risk Management: Further Refinements Needed to Assess Risks and
Prioritize Protective Measures at Ports and Other Critical
Infrastructure (15-DEC-05, GAO-06-91).
Congress and the President have called for various homeland
security efforts to be based on risk management--a systematic
process for assessing threats and taking appropriate steps to
deal with them. GAO examined how three Department of Homeland
Security (DHS) components were carrying out this charge: the
Coast Guard, which has overall responsibility for security in the
nation's ports; the Office for Domestic Preparedness (ODP), which
awards grants for port security projects; and the Information
Analysis and Infrastructure Protection Directorate (IAIP), which
has responsibility for developing ways to assess risks across all
types of critical infrastructure. GAO's work focused on
identifying the progress each DHS component has made on risk
management and the challenges each faces in moving further.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-91
ACCNO: A43417
TITLE: Risk Management: Further Refinements Needed to Assess
Risks and Prioritize Protective Measures at Ports and Other
Critical Infrastructure
DATE: 12/15/2005
SUBJECT: Agency missions
Critical infrastructure
Critical infrastructure protection
Homeland security
Performance measures
Port security
Risk assessment
Risk management
National Infrastructure Protection Plan
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-91
* Background
* Results in Brief
* Principal Findings
* Coast Guard Has Made Progress in Using Risk Management, but
* ODP's Port Security Grant Program Illustrates Both Progress
* IAIP's Progress in Carrying Out Risk Management Has Been Lim
* Overall Observations
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Risk Management Has a Long History of Use in Industry and Go
* Application of Risk Management to Homeland Security Is Widel
* Department of Homeland Security Has Broad and Challenging Re
* Seaports Are an Important Focus in the Homeland Security Res
* A Framework for Risk Management
* Strategic Goals, Objectives, and Constraints
* Risk Assessment
* Alternatives Evaluation
* Management Selection
* Implementation and Monitoring
* Objectives, Scope, and Methodology
* Coast Guard Homeland Security Activities Revolve Heavily aro
* Goal-Setting: High-Level Goals Are in Place, with Efforts Un
* Risk Assessments: Progress Has Been Substantial, but Challen
* Several Sets of Assessments Have Been Completed at the Port
* Key Challenges Involve Improving Data and Methodology
* Limitations of Data on Threats, Vulnerabilities, and Consequ
* Methodological Limitations
* Evaluation of Alternatives: Ability to Evaluate Alternatives
* Management Selection: Local Strategies Have Been Selected bu
* Implementation and Monitoring: A Foundation for Continuous I
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* ODP Manages Port Security Grants
* Goal Setting: Overall Goals Have Been Set; Developing Perfor
* Risk Assessments: The Funding Distribution Model Is Becoming
* Procedures for the Fiscal Year 2005 Port-Level and National-
* Challenges Remain for Improving Methods and Data
* Incomplete Data on Threats, Vulnerabilities, and Consequence
* Methodological Limitations
* Alternatives Evaluation: ODP Recognizes the Importance of Ev
* Management Selection: ODP Has Addressed Problems in Document
* Implementation and Monitoring: ODP Has Instituted a Monitori
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* IAIP Plays a Key Role in Evaluating Risk across Infrastructu
* Strategic Goals, Objectives, and Constraints: High-Level Goa
* Risk Assessment: IAIP Faces Substantial Gaps and Has Made Li
* Progress Is Greatest on Database of Critical Assets
* Development of Threat Data Faces Challenges
* Methodology for Cross-Sector Comparisons Has Experienced Set
* Evaluating Alternatives: Lack of Guidance and Consensus on C
* Management Selection: IAIP Challenged in Protecting Infrastr
* Implementation and Monitoring: IAIP Is Also Challenged by It
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* First Observation: Much Remains to Be Done
* Observation Two: Progress Varies by Component and Reflects K
* Observation Three: In the Short Term, Further Progress Is St
* Observation Four: In the Long Term, Progress Rests Heavily o
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* General Lack of Uniform Guidance on Risk Management
* Methodology for Developing a Risk Management Framework
* A Risk Management Framework
* Strategic Goals, Objectives, and Constraints
* Risk Assessment
* Alternative Evaluation
* Management Selection
* Implementation and Monitoring
* Difficulties Applying a Risk Management Framework
* Glossary of Risk Management Terms
* GAO Contacts
* Staff Acknowledgments
* Critical Infrastructure Protection
* Maritime Security
* Risk Management
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Report to Congressional Requesters
United States Government Accountability Office
GAO
December 2005
RISK MANAGEMENT
Further Refinements Needed to Assess Risks and Prioritize Protective
Measures at Ports and Other Critical Infrastructure
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management Risk Management Risk Management Risk Management Risk
Management Risk Management Risk Management Risk Management Risk Management
Risk Management
GAO-06-91
Contents
Letter 1
Executive Summary 3
Background 4
Results in Brief 6
Principal Findings 9
Recommendations for Executive Action 14
Agency Comments and Our Evaluation 15
Chapter 1 Introduction Risk Management Is a Key Tool for Homeland Security
16
Chapter 2 The Coast Guard Has Made Progress in Using Risk Management, but
Challenges Remain 30
Chapter 3 Stronger Risk Management Approach Could Improve the
Accountability of the Port Security Grant Program 49
Chapter 4 IAIP Faces Challenges in Meeting Risk Management
Responsibilities across All Sectors of the Nation's Infrastructure 68
Chapter 5 Overall Observations and Recommendations 88
Appendix I A Risk Management Framework 99
Appendix II Comments from the Department of Homeland Security 113
Appendix III GAO Contacts and Staff Acknowledgments 116
Related GAO Products 117
Tables
Table 1: Examples of Risk Management in the Private and Public Sectors 18
Table 2: Summary of Progress Made and Challenges That Remain in the Coast
Guard's Risk Management Approach 31
Table 3: Port-Level Assessments Conducted by the Coast Guard 37
Table 4: National-Level Assessments Conducted by the Coast Guard 38
Table 5: Examples of Data-Related Challenges in Coast Guard Risk
Assessments 39
Table 6: Summary of Progress and Challenges in the Port Security Grant
Program 50
Table 7: Examples of Data-Related Challenges in ODP Risk Assessments 59
Table 8: Examples of Changes in Funding Decisions 64
Table 9: Summary of Progress Made and Challenges That Remain in IAIP's
Risk Management Approach 69
Table 10: Critical Infrastructure Sectors and Lead Federal Agencies 72
Table 11: A Risk Management Framework 103
Figures
Figure 1: Risk Management Framework 6
Figure 2: A Framework for Risk Management 24
Figure 3: Facilities at One of the Nation's Major Ports 52
Figure 4: Sources of Evaluation Criteria Associated with Risk Management
Phases 102
Abbreviations
ASME American Society of Mechanical Engineers
CIP critical infrastructure protection
COSO Committee of Sponsoring Organizations
DHS Department of Homeland Security
HITRAC Homeland Infrastructure Threat and Risk Analysis Center
HSPD Homeland Security Presidential Directive
IAIP Information Analysis and Infrastructure Protection Directorate
IG Inspector General
MARAD Maritime Administration
MSRAM Maritime Security Risk Analysis Model
MTSA Maritime Transportation Security Act
NADB National Assets Database
NIPP National Infrastructure Protection Plan
ODP Office for Domestic Preparedness
OMB Office of Management and Budget
PS-RAT Port Security Risk Assessment Tool
PWCS Ports, Waterways, and Coastal Security
RAMCAP Risk Analysis and Management for Critical Asset Protection
TSA Transportation Security Administration
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
December 15, 2005
The Honorable Henry A. Waxman Ranking Minority Member Committee on
Government Reform House of Representatives
The Honorable C. A. Dutch Ruppersberger House of Representatives
The threat of terrorism presents a number of risks to our nation's
seaports and other types of critical infrastructure. The Department of
Homeland Security (DHS) has three components responsible for the security
of critical infrastructure related to ports and other facilities. The U.S.
Coast Guard has responsibility for port security overall. The Office for
Domestic Preparedness (ODP) is responsible for providing port security
grants to selected maritime facility owners. The Information Analysis and
Infrastructure Protection (IAIP) Directorate is responsible for working
with other federal, state, local, and private organizations to identify
and protect critical infrastructure across the nation. Risk management is
a tool for assessing risks, evaluating alternatives, making decisions, and
implementing and monitoring protective measures. This report provides an
evaluation of the progress made, and challenges faced, by the Coast Guard,
ODP, and IAIP in using risk management to improve homeland security.
As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will provide copies of this report
to appropriate departments and interested congressional committees. This
report will also be available at no charge on the GAO Web site at
http://www.gao.gov.
If you or your staff have any questions about this report, please contact
me at (415) 904-2200 or at [email protected] . Key contributors to this
report are listed in appendix III.
Sincerely yours,
Margaret T. Wrightson Director, Homeland Security and Justice Issues
Executive Summary Executive Summary
Risk management, a strategy for helping policymakers make decisions about
assessing risks, allocating resources, and taking actions under conditions
of uncertainty, has been endorsed by Congress and the President as a way
to strengthen the nation against possible terrorist attacks. Risk
management has long been used in such areas as insurance and finance, but
its application to domestic terrorism has no precedent. Unlike storms and
accidents, terrorism involves an adversary with deliberate intent to
destroy, and the probabilities and consequences of a terrorist act are
poorly understood and difficult to predict. The size and complexity of
homeland security activities and the number of organizations involved-both
public and private-add another degree of difficulty to the task. The task
of managing this complexity centers on the Department of Homeland
Security, which since its inception in March 2003 has been faced with the
challenge of transforming 22 agencies into an organization that can plan,
manage, and carry out operations effectively. Congress likewise has a key
oversight role to play in ensuring that DHS's course regarding risk
management reflects a consensus as to the most prudent and cost-effective
course of action.
To assist Congress in its oversight, this report focuses on the progress
made by three DHS components in applying risk management to homeland
security activities and the challenges each component faces in moving
further ahead. For two of these components, GAO's review dealt
specifically with their risk management activities at the nation's
seaports, while the review for the third component encompassed a wider
range of infrastructure. GAO decided to focus a considerable amount of
this review on seaport security because seaports have been viewed as
potential terrorist targets or as conduits for importing a weapon of mass
destruction or where terrorists may enter the country. GAO's focus on
these three components, while not a comprehensive look across the entire
department, provides perspective on the degree of progress made thus far.
Risk management has applications for deliberate acts of terror as well as
natural disasters, such as hurricanes and earthquakes. GAO's research,
which was conducted prior to Hurricane Katrina, focused on preparations
for terrorist attacks, not natural disasters. The three components GAO
studied are:
o the Coast Guard, the lead federal agency for port security and
the agency responsible for developing and coordinating various
risk-based assessments of critical infrastructure in and around
ports;
o the Office for Domestic Preparedness, administrator of the port
security grant program, has awarded more than half a billion
dollars in federal grants to owners and operators of port
facilities and vessels; and
o the Information Analysis and Infrastructure Protection
Directorate, which has been charged with establishing uniform
policies, approaches, guidelines, and methodologies for
integrating infrastructure protection and risk management
activities within and across key sectors, such as energy, defense,
and transportation, including airports, railroads, and ports.1
Besides describing the progress of and challenges for each component, this
report also presents GAO's observations about what the three components'
efforts indicate collectively, both with regard to how far the department
has come in managing homeland security efforts on the basis of risk and
what steps could help advance the current level of progress.
Background
Seaport security receives particular attention in this report because
seaports are widely viewed as representing attractive terrorist targets,
in part because of their importance to the economy. More than 95 percent
of the nation's non-North American foreign trade (and 100 percent of
certain commodities, such as foreign oil) arrives by ship. The estimated
economic consequences of a successful attack and resulting shutdown of
this system total billions of dollars. Ports also represent attractive
targets because they contain a myriad of vulnerabilities. In all, the
nation's 300-plus ports have about 3,700 cargo and passenger terminals.
Chemical factories, oil refineries, power plants, and other facilities are
often located in port areas and add another set of possible targets. Roads
crisscross many ports, allowing access by land as well as by water, and
the number of people working in or traveling through ports is in the
millions. The Coast Guard has the major responsibility for seaport
security, and the port security grant program administered by ODP adds to
the resources available for port security projects.
Relative to the Coast Guard and ODP, IAIP's homeland security
responsibilities are by far the widest-ranging. The Homeland Security Act
of 2002 and Homeland Security Presidential Directive 7 (HSPD-7) charge
IAIP with establishing a risk management framework across the federal
government to protect the nation's critical infrastructure and key
resources.2 The scope of this effort is immense, and the effort is one of
IAIP's central responsibilities. IAIP's task ultimately involves
developing an approach that can inform decisions on what the nation's
antiterrorism priorities should be and identifying what strategies and
programs will do the most good. IAIP's work is done in a setting where
numerous and substantial gaps in security remain, but resources for
closing these gaps are limited. More specifically, IAIP is charged with
examining and comparing relative risks associated with a multitude of
possible targets, ranging from specific structures (such as dams, chemical
plants, and nuclear power plants) to major sectors of national
infrastructure (such as the banking system, computer networks, and water
systems). IAIP is also responsible for developing policies and guidance
that other agencies can use in conducting their own risk assessments.
1On November 14, 2005, DHS reorganized the department. ODP and the
Infrastructure Protection part of the former IAIP Directorate are now
components in the Preparedness Directorate. We recognize the recent
organizational changes, but because ODP and IAIP carried out the work we
reviewed, we have not changed the name or organizational posture of these
DHS components in our report.
While federal law and the presidential directive call for the use of risk
management in homeland security, little specific federal guidance or
direction exists as to how risk management should be implemented. To
provide a basis for analyzing component efforts, GAO developed a framework
for risk management based on industry best practices and other criteria.
This framework, shown in figure 1, divides risk management into five major
phases: (1) setting strategic goals and objectives, and determining
constraints; (2) assessing the risks; (3) evaluating alternatives for
addressing these risks; (4) selecting the appropriate alternatives; and
(5) implementing the alternatives and monitoring the progress made and
results achieved. For all three components, GAO applied this framework
after conducting a wide range of interviews with officials, reviewing
plans and activities of the three components, and visiting port locations.
As part of our work, GAO briefed officials of the three components about
the various phases of the framework and the officials generally agreed
with its structure and intent. The application of risk management to
homeland security is relatively new, and the framework will likely evolve
as processes mature and lessons are learned.
2The Homeland Security Act incorporates the definition of "critical
infrastructure" used in the USA PATRIOT Act of 2001, meaning "systems and
assets, whether physical or virtual, so vital to the United States that
the incapacity or destruction of such systems and assets would have a
debilitating impact on security, national economic security, national
public health or safety, or any combination of those matters." The
Homeland Security Act defines "key resources" as "publicly or privately
controlled resources essential to the minimal operations of the economy
and government." 6 U.S.C. S: 101.
Figure 1: Risk Management Framework
Results in Brief
Of the three components GAO reviewed, the Coast Guard had made the most
progress in establishing a foundation for using a risk management
approach; its next challenges are to further refine and enhance its
approach. While the Coast Guard has made progress in all five risk
management phases, its greatest progress has been made in conducting risk
assessments-that is, evaluating individual threats, the degree of
vulnerability, and the consequences of a successful attack. However, the
assessments are limited in their reliability and completeness, and better
coordination will be needed with the intelligence community so that
analysts can develop models that better assess the relative probability of
various threat scenarios. The Coast Guard has developed the ability to
compare and prioritize risks at individual. However, it cannot yet compare
and prioritize relative risks of various infrastructure across ports.
Other challenges include developing performance measures to go along with
the more general goals already developed for the port security mission,
further integrating risk into the annual cycle of program and budget
review, and developing formal policies for reviewing and improving the
implementation of a risk management approach. The Coast Guard has actions
under way to address the challenges it faces in each risk management
phase. Several of these actions are based, in part, on briefings GAO held
with agency officials.
ODP has made progress in applying risk management to the port security
grant program, but like the Coast Guard, it also faces challenges across
all phases of the risk management framework. For example, ODP has set
broad risk management goals and has placed more emphasis on using
risk-based data in its assessments, but it lacks performance measures
showing what specific outcomes the program aims to achieve, and it still
faces challenges in such matters as comparing grant applications across
ports. The grant awards for fiscal year 2004 also illustrate some of the
challenges in ensuring that criteria for making the awards are transparent
and consistent. At the end of what was, in part, a risk-based assessment
process, ODP changed the criteria for awarding grants when it decided to
give lower priority to applicants from large companies on the assumption
that the companies were better able than other entities to pay for their
own improvements. This changed 40 percent of the grants awarded, as
projects with higher risk but greater potential for self-funding gave way
to lower-risk projects with more limited funding prospects. In the
procedures for fiscal year 2005 awards, ODP clarified the criteria it
would use in making the awards.
IAIP, which has the broadest risk management responsibilities of the three
components and faces the greatest challenges, has made the least progress
in carrying out its complex risk management activities. Its efforts are
aligned with high-level strategic goals, but ways to measure performance
in achieving these goals have yet to be developed. IAIP is not as far
along as ODP and the Coast Guard in conducting risk assessments. While
IAIP has provided input to ODP for its risk assessment efforts, IAIP's
risk assessment responsibilities span much broader sectors of the nation's
infrastructure than seaports alone, making its assessment activities more
difficult. This difficulty is reflected in the limited progress made. With
regard to its risk assessment responsibilities, IAIP has yet to
successfully (1) develop data to determine the relative likelihood of
various threat scenarios, (2) complete a methodology for comparing and
prioritizing key assets, or (3) meet requirements set forth in HSPD-7 for
issuing policies and guidance that other agencies can use in conducting
their own risk assessments. For example, a DHS consultant issued a risk
assessment methodology in 2004 for collecting data from industry, but
adverse comments from reviewers have led to revisions that are still under
way. IAIP is also challenged in its ability to translate these assessments
into specific measures to be taken, because after IAIP makes decisions
about what national priorities should be, it is dependent on the actions
of others to carry them out. This is particularly true with regard to
private sector assets where IAIP needs collaboration from the owners and
operators of private sector infrastructure and their regulators.
GAO made four main observations regarding the experience of these three
components.
o A considerable degree of effort has been expended thus far, but
much work remains to be done. This is particularly true in viewing
risk management strategically-that is, with a view that goes
beyond just assessing what the risks are and also integrating the
consideration of risk into the annual cycle of program and budget
review that is already in place.
o The varying degree of progress among the three components tends
to reflect several characteristics of each component-how long it
has been at the task of developing a risk management approach, how
long it has existed as a component and is therefore able to
function maturely, and how complex its risk management task is.
For example, IAIP, which has made the least progress, is not only
a new component established in 2003, but also has the most complex
risk-related tasks of the three components-addressing risk not
only at ports, but across all types of infrastructure and with
multiple federal agencies and nonfederal stakeholders.
o In the near term, all three components' success in risk
management will depend partly on continuing to make progress on
the challenges described above. This involves continuing to work
on such matters as performance measures, basic policies, and
enhancements to existing risk assessment tools.
o The final observation is related to a critical longer-term
need: more guidance and coordination from the department level,
both to help ensure that individual components such as IAIP are
carrying out their roles effectively and to ensure that the
various responses from individual components mesh as effectively
as possible with one another. In comparing the approaches
developed by the three components, GAO noted ways in which their
efforts were not consistent. The danger is that if components
develop systems and methodologies that are inconsistent, they may
end up with incompatible systems that have little or no ability to
inform spending decisions on homeland security. The challenges
associated with creating a department that can effectively
administer a coherent risk management approach to the nation's
homeland security have been widely acknowledged. IAIP recognizes
that as DHS's individual components begin to mature in their risk
management efforts, the need increases for ensuring consistency
and coherence in these efforts. Supplying this necessary guidance
and coordination is what DHS was set up to do.
Principal Findings
Coast Guard Has Made Progress in Using Risk Management, but Challenges Remain
The Coast Guard has made progress across all five phases of risk
management. In the first phase (goal and objective setting), the Coast
Guard has established broad strategic goals for port security, including,
in order of priority: (1) preventing terrorist attacks within, and
terrorist exploitation of, the maritime domain and (2) reducing America's
vulnerability to terrorism in the maritime domain. It faces challenges in
developing objectives that translate these goals into more specific and
measurable results. Coast Guard officials recognize that developing
performance measures is a necessary next step and have actions under way
to develop such measures.
For the second phase, assessing risks, the Coast Guard has greatly
expanded the scope of its risk assessment activities since the terrorist
attacks of September 11, 2001. It has conducted three major security
assessments at the port level, and collectively these assessments have
resulted in considerable progress in understanding and prioritizing risks
within a port. After it initiated port-level assessments, the Coast Guard
expanded its analysis efforts to the national level to gain a more
strategic perspective on port security. In all, the Coast Guard has
conducted three major efforts at the national level, focusing more
generally on understanding the risk posed to various classes of assets
(such as bridges or container ships) by various types of attacks (such as
using explosives or weapons of mass destruction). The assessments are
limited in their reliability and completeness, however, in the degree to
which the Coast Guard has (1) formal and systematic input from the
intelligence community for modeling relative probability and likelihood of
threat scenarios and (2) risk assessment tools allowing comparison and
prioritization of specific infrastructure across ports. These limitations
affect the degree to which the Coast Guard is able to determine how best
to focus its attention on these threats that, from a national perspective,
pose the greatest risk within the seaport sector. The Coast Guard has
initiated actions to address these challenges. For example, the agency has
initiated contact with the intelligence community to obtain better data on
threat scenarios, and it plans to complete development of an assessment
tool that will compare the relative risks of high-value assets at one port
with risks of assets in a different port.
Enhancing these first two phases of risk management is key to making
additional progress on the next two phases-evaluating and selecting
alternatives that reduce risk. While the Coast Guard's efforts have
resulted in progress in identifying and evaluating alternatives at the
individual port level, the lack of measurable objectives and sufficient
information to fully depict threats, vulnerabilities, and consequences
limits the ability to target the areas with the greatest gaps or produce
the most cost-effective decisions. Similarly, buttressing annual budget
review cycles with risk-based data is in its early development and more
work remains to be done. Finally, with regard to the fifth
phase-implementation and monitoring-the Coast Guard has implemented a
number of activities to mitigate risks and has demonstrated the ability to
evaluate its efforts and make improvements. The actions taken have
included establishing maritime intelligence centers on the Atlantic and
Pacific coasts and working closely with nonfederal stakeholders to reduce
vulnerabilities in and around facilities and vessels. However, existing
feedback mechanisms are insufficient to ensure that Coast Guard field
personnel can make their headquarters managers aware of ways to improve
the process. The Coast Guard recognizes the value of formal feedback loops
as a means of improving its risk management processes, and it has plans to
obtain formal feedback as part of its future efforts.
ODP's Port Security Grant Program Illustrates Both Progress and Challenges in
Implementing Risk Management
Like the Coast Guard, ODP has made progress across all five phases of risk
management. For example, for the first phase, it has set risk management
goals that support broader maritime goals, such as protecting critical
infrastructure in harbors, borders, ports, and coastal approaches. ODP has
not begun to translate these broad goals into measurable objectives.
Without them, it is difficult to know what progress has been made in
reducing risk and what security gaps remain.
ODP has carried out risk assessments, with input from the Coast Guard and
IAIP, and evaluated mitigation alternatives-the second and third phases of
the framework-to help determine which ports should receive priority for
grants. Using risk assessments, ODP narrowed the number of ports eligible
for grants from 129 to 66 for fiscal year 2005. Other recent steps include
placing greater emphasis on using threat, vulnerability, and consequence
data in prioritizing grant applications. Along with this progress,
however, are several methodological challenges that limit such things as
the usefulness of data received from intelligence agencies and ODP's
ability to compare and prioritize risks among ports. For example, without
data on the relative probability of various threat scenarios from the
Coast Guard or IAIP, ODP may not target the most significant security
needs. ODP has not yet developed approaches for addressing most of these
challenges.
While ODP has also made progress in developing a risk-based grant
selection process and mechanisms to monitor what the grants accomplish,
grant awards for fiscal year 2004 illustrate the challenges involved in
actually making risk-based decisions. At the end of its process for
determining which grants to fund, based in part on risk, ODP decided to
give lower priority to grants involving projects at large companies, on
the assumption that the companies were better able than other entities to
pay for their own improvements. For example, one chemical company's
application for $225,000 to purchase cameras, fencing, and barricades was
initially ranked 25th out of 287 applications nationwide, but under the
revised priorities its ranking fell to 236th. Projects initially ranked
much lower received funding instead. For example, an application initially
ranked as 279th out of 287 was approved for funding. In all, the
application of non-risk criteria changed 40 percent of the grants awarded.
ODP's changes affected the transparency and consistency of the awards
process, in that (1) the criteria under which applications were submitted
and initially considered were changed at the end of the process, and (2)
the role of risk in evaluating the applications was obscured, because the
resulting awards may not have addressed the most severe security gaps.
Additionally, there is no guarantee that large companies would spend their
own funds for security improvements, and it is unclear whether there are
incentives, such as minimum standards for security, that would motivate
them to do so. ODP issued revised criteria for fiscal year 2005 grants,
and in doing so has made the process more transparent and consistent.
IAIP's Progress in Carrying Out Risk Management Has Been Limited
IAIP's progress in all five phases of risk management has been limited. It
has made some progress in developing goals, having issued an Interim
National Infrastructure Protection Plan in February 2005 that identifies a
strategy for identifying, prioritizing, and coordinating the protection of
critical infrastructure and key resources.3 The interim plan provides some
guidance in meeting IAIP's broad responsibilities for identifying,
comparing, and prioritizing critical assets, but it is not a comprehensive
document, and IAIP faces several challenges in making it more
comprehensive. These challenges are related to (1) developing performance
measures that can be used in evaluating progress and (2) establishing
milestones and time frames for processing and prioritizing assets across
the many different infrastructure sectors.
3In November 2005, DHS issued a revised Interim National Infrastructure
Protection Plan for comment.
IAIP's progress in risk assessment-the second phase of risk management-has
been limited in several main respects. For example, IAIP has experienced
difficulties in carrying out requirements of the Homeland Security Act of
2002 that charged IAIP with the responsibility of conducting risk
assessments of critical infrastructure and key resources to determine the
risks of particular types of terrorist attacks. IAIP's original
methodology for this task, called the Risk Analysis and Management for
Critical Asset Protection, required extensive modification after its
initial issuance in April 2004. IAIP now views it as a tool for engaging
industry in a risk management dialogue with government. In September 2005,
IAIP officials said they are developing a National Comparative Risk
Assessment to meet the immediate need of examining risks within and across
sectors, and they plan to complete an interim assessment by the end of
2006. Challenges to carrying out this timetable include the need to obtain
key information from other federal agencies and the fact that IAIP still
needs to award the contract for this effort. One specific issue is the
approach IAIP has taken in assessing the probability of various threat
scenarios. The Homeland Security Act calls on IAIP to assess the
probability of success of terrorist attacks, and during the course of
GAO's review, IAIP officials said they recognize the importance of
assessing the relative likelihood of an attack in meeting this
requirement. IAIP officials said that the lack of intelligence analysis
and data on such things as the capability and intent of terrorist groups
hinders their ability to assess probability, but that work is under way in
this regard. IAIP officials also pointed out that some inaccuracy is to be
expected in examining the intent and capability of an adversary whose
plans are concealed and that it will be important to reduce the potential
of low-confidence assessments having undue influence when long-term
investment decisions are made.
IAIP's progress in the three other phases of risk management (evaluating
alternatives, selecting a solution, and implementing and monitoring that
solution) will remain limited, in part because of the points just
discussed-performance goals and a complete risk assessment methodology are
not in place. Beyond these limitations, however, IAIP faces additional
challenges. For example, IAIP's role in selection, implementation, and
monitoring is further complicated because in many instances, other
entities have primary responsibility for selecting the solution. For
example, other agencies, such as the Department of Defense or the
Department of Energy, have primary responsibilities for some of the
infrastructure sectors covered in IAIP's assessments. Additionally, much
of the critical infrastructure is owned or operated by private industry,
and while IAIP does not have authority over them, other federal agencies
do have authority over infrastructure in specific sectors. This condition
highlights the importance of coordination between IAIP and agencies with
such regulatory authority. For example, the Nuclear Regulatory Commission,
which issues licenses to nuclear power plants, has regulatory authority
over security matters at these facilities. IAIP officials said they use
their expertise and powers of persuasion to bring about specific actions
but in most cases cannot compel others to adopt IAIP's recommendations.
Overall Observations
A great amount of effort has been applied. However, much more remains to
be done than has been accomplished so far. Across all three components,
the most progress has generally been made on fundamental steps, such as
conducting risk assessments of individual assets, and less progress has
generally been made on developing ways to translate this information into
comparisons and priorities across ports and across infrastructure sectors,
or applying it to new programs. Progress among the three components'
efforts has been far from consistent and has tended to vary not only with
the length of time the component has been using a risk-based approach, but
also with the component's own maturity level and the complexity of its
risk management task.
With regard to next steps that would appear to add the most value to
making further progress, one key observation is that in the short term,
progress is heavily dependent on continuing to improve basic policies,
procedures, and methods for applying risk management. Each component has
an admittedly difficult set of challenges ahead, but progress has to be
built on taking these incremental steps and doing them well. An area that
needs further attention by all three entities is working with intelligence
communities to develop improved analysis and data so that the relative
probability of various threat scenarios can be further developed.
The final observation is that in the longer term, progress will become
increasingly dependent on how well the entire risk management effort is
coordinated. While absolute compatibility among all components' efforts is
likely impossible, even with components working in close cooperation,
strong coordination is important to help ensure that component efforts are
consistent rather than stovepiped. The risk management efforts GAO
examined appeared to be fueled by a strong concern to make some headway,
with coordination and interagency consistency a lesser concern. For
example, the Coast Guard initiated efforts to set up a methodology for
assessing and specifying risks before IAIP was created; and in the view of
IAIP officials, it was important to proceed even though they recognized
that doing so might lead to approaches that would not mesh cleanly with
the approach IAIP would eventually develop. That approach appeared prudent
in the short term, in that if the Coast Guard had waited to begin until
guidelines had been set, it would still be waiting. Now, however, the need
for coordination is looming larger, and coordination is essential to the
success of efforts over time. Some of this coordination needs to come from
IAIP, which is required under presidential directive to issue guidelines
for other agencies to use, but it has yet to do so. Beyond IAIP, DHS has
an active role in this regard. This is a key issue for the department as
it moves from being an organization that is essentially in its early
stages to one that is increasingly being expected to respond in a way that
is more organizationally mature. Since 2003, GAO has designated the
implementation and transformation of DHS as high risk because of the
numerous challenges in transforming 22 agencies into one department and
the serious implications of failure. Translating the concept of risk
management into applications that are consistent and useful represents one
of these challenges, and failure to effectively address this could have
serious consequences for homeland security. In risk management, which the
department has embraced as the guiding principle behind its policies and
operations, IAIP's role is to act as an intra-agency and interagency
coordinator of homeland security activities. Doing so will strengthen its
ability to weigh risks and inform the decisions made across the homeland
security responsibilities of the many agencies involved.
Recommendations for Executive Action
GAO is making a number of specific recommendations to the Secretary of DHS
with regard to the challenges faced by the three components. These
recommendations, listed specifically at the end of the relevant chapters,
cover such matters as developing performance goals and measures, improving
risk assessment methodologies, working with intelligence communities to
develop better data for risk assessment purposes, and (for IAIP)
developing guidance for other agencies to use in evaluating risk and
considering risk mitigation alternatives.
Agency Comments and Our Evaluation
We provided DHS a draft of this report for its review and comment. DHS,
including the Coast Guard, ODP, and IAIP, generally agreed with our
findings and recommendations. For instance, DHS said that each DHS
component we reviewed has actions under way to address recommendations
made in the report. The comments from each component are summarized at the
end of the relevant chapters. In addition to commenting on our findings
and recommendations, DHS provided technical comments under separate cover,
and we revised the draft report where appropriate. Written comments from
DHS are reprinted in appendix II.
Chapter 1: Introduction: Risk Management Is a Key Tool for Homeland
Security Chapter 1: Introduction: Risk Management Is a Key Tool for
Homeland Security
This is a report about the nation's progress in applying risk management
to key aspects of homeland security. Risk management is a widely endorsed
strategy for helping policymakers make decisions about allocating finite
resources and taking actions in conditions of uncertainty. It has been
widely practiced for years in such areas as insurance, construction, and
finance. By comparison, its application in homeland security is relatively
new-much of it coming in the wake of the terrorist attacks of September
11-and it is a difficult task with little precedent. The goals for using
it in homeland security include informing strategic decisions on ways to
reduce the likelihood that adverse events will occur, and mitigate the
negative impacts of and ensure a speedy recovery from those that do.
Achieving these goals involves making policy decisions about what the
nation's homeland security priorities should be-for example what the
relative security priorities should be among seaports, airports, and
rail-and basing spending decisions on what approaches or strategies will
do the most good at narrowing the security gaps that exist. Risk
management has been widely supported by the President and Congress as a
management approach for homeland security, and the Secretary of the
Department of Homeland Security has made it the centerpiece of agency
policy.
"Homeland security" is a broad term with connotations that resonate from
the September 11 attacks and other connotations that now resonate from the
disaster brought on by Hurricane Katrina in August 2005. Risk management
has applications for deliberate assaults like the September 11 attacks and
natural disasters, such as hurricanes and earthquakes. Our research was
completed and the report largely written before Hurricane Katrina struck.
Thus, our work concentrated on components' actions in response to
terrorism.
This report examines how three DHS components have applied risk management
to certain aspects of their homeland security responsibilities. The three
components are the United States Coast Guard, the Office for Domestic
Preparedness, and the Information Analysis and Infrastructure Protection
Directorate. This report looks at risk management efforts of the Coast
Guard and ODP specifically related to seaport security, and for IAIP, it
looks at risk management efforts related to IAIP's broader
responsibilities in assessing terrorist threats against all aspects of the
nation's infrastructure.1
Risk Management Has a Long History of Use in Industry and Government
Risk management can be described as the continuous process of assessing
risks, reducing the potential that an adverse event will occur, and
putting steps in place to deal with any event that does occur.2 It has
been used in the private and public sectors for decades (see table 1 for
examples). For example, insurance companies use a variety of statistical
techniques to assess the level of risk for what they are insuring. Within
government, agencies use risk management to set regulations and to protect
the environment and the health and safety of American taxpayers. Although
some risk management methodologies and processes can be complex and may
require expert advice and support, other aspects of risk management-such
as setting goals and using performance measures to track progress in
meeting them-are well understood and widely practiced.
1On November 14, 2005, DHS reorganized the department. ODP and the
Infrastructure Protection component of the former IAIP Directorate are now
in the Preparedness Directorate. We recognize the recent organizational
changes, but because ODP and IAIP carried out the work we reviewed, we
have not changed the name or organizational posture of these DHS
components in our report.
2A more precise description of risk management is that it involves a
continuous process of managing-through a series of mitigating actions that
permeate an entity's activities-the likelihood of an adverse event and its
negative impact. Risk management addresses risk before mitigating action,
as well as the risk that remains after countermeasures have been taken. A
glossary of risk management terms is contained at the end of appendix I.
Table 1: Examples of Risk Management in the Private and Public Sectors
Source: GAO.
Application of Risk Management to Homeland Security Is Widely Endorsed and
Accepted
Risk management was part of the nation's approach to assessing terrorism
before the events of September 11. For example, in the 1990s, the Defense
Special Weapons Agency assessed risks to evaluate force protection
security requirements for mass casualty terrorist incidents at military
bases. Companies under contract to federal agencies such as the Department
of Energy, the National Security Agency, and the National Aeronautics and
Space Administration used risk assessment models and methods to identify
and prioritize security requirements. The Federal Aviation Administration
and the Federal Bureau of Investigation did joint threat and vulnerability
assessments on airports determined to be high risk. When we reviewed two
of these efforts in the late 1990s, we found a lack of formal risk
assessment requirements and made several recommendations to integrate
risk-based data into decision-making processes.3
What September 11 changed was the intensity and magnitude of this task.
The September 11 attacks were clearly a transformational event for the
nation, in that they called attention to vulnerabilities throughout the
nation's infrastructure, not just in aviation security. While there might
always have been a concern, for example, about the consequences of an
accident in a chemical factory in a highly populated area, now, these
consequences had to be viewed not just from the standpoint of a potential
accident, but as something a terrorist could exploit. Potential targets
multiplied, and the scope of work to be done became much greater. Homeland
security spending rose from about $21 billion in fiscal year 2001 to a
proposed $50 billion in fiscal year 2006.
Risk management has received widespread support and interest from
Congress, the President, and the Secretary of DHS as a tool that can help
set priorities on how to protect the homeland. In this setting, numerous
and substantial gaps in security exist, but resources for closing these
gaps are limited and must compete with other national priorities.
Policymakers in the legislative and executive branches have endorsed risk
management as a technique that can inform decisions on setting relative
priorities and on making spending decisions.
In view of the widespread support that risk management has gained, federal
agencies are now required to assess risks. The Homeland Security Act of
2002 calls for a comprehensive assessment of risk related to
vulnerabilities of critical infrastructure and key resources, notably (1)
the risk posed by different forms of terrorist attacks, (2) the
probability that different forms of an attack might succeed, and (3) the
feasibility and efficacy of countermeasures to deter or prevent such
attacks.4 Two congressionally chartered commissions, the 9/11 Commission
and the Gilmore Commission, support the use of data on risks to help
inform the difficult decisions that must be made in allocating limited
federal funds for security measures. The President has issued policies
directing the heads of seven major departments or agencies to assess
risks. The past and present Secretaries of DHS have stated that actions of
the department will be guided through the use of risk management.
3GAO, Combating Terrorism: Threat and Risk Assessments Can Help Prioritize
and Target Program Investments, GAO/NSIAD-98-74 (Washington D.C.: Apr. 9,
1998), and GAO, Combating Terrorism: Need for Comprehensive Threat and
Risk Assessments of Chemical and Biological Attacks, GAO/NSIAD-99-163
(Washington, D.C.: Sept. 14, 1999).
46 U.S.C. 201(d)(1), (2).
Department of Homeland Security Has Broad and Challenging Responsibilities in
Applying Risk Management
Congress has charged DHS with lead responsibilities in carrying out or
coordinating homeland security programs and for applying risk management
in carrying out this responsibility. For two main reasons, integrating a
risk management approach into its business practices is a major management
challenge that faces DHS.
o First, relative to many other fields such as insurance or
finance, terrorism is a relatively new application for risk
management. The sources of the risk are intelligent adversaries
with malevolent intent with whom there is relatively little
domestic experience. Unlike the insurance or banking industries,
which have extensive historical data that are used to assess
risks, DHS lacks such data on domestic terrorism, and this limits
any detailed analysis in assessing risk. As a result, the
probabilities and consequences of a terrorist act are poorly
understood and difficult to predict and greater reliance on expert
judgment is required. In January 2005, we identified risk
management as an emerging high-risk area. At that time, we noted
that DHS had not completed any risk assessments mandated by the
Homeland Security Act.5
o Also, the size and complexity of homeland security activities
add another dimension of difficulty to the task. Since its
inception in March 2003, DHS has been faced with the challenge of
transforming 22 agencies into one department in a way that results
in an organization with effective planning, management, and
operations while carrying out its critical mission of securing the
homeland. Since 2003, we have designated implementing and
transforming DHS as high risk, because DHS had to transform these
many agencies-several with major management challenges-into one
department.6 Besides the challenge it poses at the federal level,
risk management also crosses jurisdictional boundaries and
involves state and local governments and private industry
stakeholders, and it requires a multidisciplinary approach
involving intelligence, law enforcement, strategic planning, and
program activities that address threats and vulnerabilities.
5GAO, High-Risk Series: An Update, GAO-05-207 , p.29 (Washington, D.C.:
January 2005).
6 GAO-05-207 .
Within DHS, we examined the progress of three DHS components-the Coast
Guard, ODP, and IAIP-in administering risk management as part of their
management processes. Two of these components (Coast Guard and ODP) have
responsibilities related to seaport security. IAIP's responsibilities are
much broader and more difficult-it is responsible for coordinating and
assessing homeland risks across the federal government. Here is an
overview of the three components.
o The United States Coast Guard. The Coast Guard is the lead
federal agency for the security of the nation's ports. Its
responsibilities include protecting ports, the flow of commerce,
and the maritime transportation system from terrorism. As the lead
in domestic maritime security, the Coast Guard has a robust
presence at the national, regional, and port levels. The Coast
Guard protects more than 300 ports and 95,000 miles of coastline.
By providing a secure environment, the Coast Guard keeps maritime
transportation open for the transit of commercial goods, as well
as assets and personnel from the armed forces. In carrying out its
mission, the Coast Guard has, among other activities, conducted
local and national assessments of security risks at the nation's
ports. The role of the Coast Guard in applying risk management to
port security is discussed in more detail in chapter 2.
o The Office for Domestic Preparedness. Within the Office of
State and Local Government Coordination and Preparedness, the
Office for Domestic Preparedness is responsible for administering
federal homeland security assistance programs for states and
localities, including the port security grant program. Since 2002,
the program has awarded over $500 million in grants to state,
local, and industry stakeholders to improve security in and around
their facilities or vessels. The role of ODP in applying risk
management to port security grants is discussed in more detail in
chapter 3.
o Information Analysis and Infrastructure Protection Directorate.
The Information Analysis and Infrastructure Protection Directorate
is responsible for, among other things, identifying and assessing
current and future threats to the homeland, mapping those threats
against known vulnerabilities, recommending protective measures,
issuing warnings, and offering advice on preventive and protective
action. IAIP is responsible for cataloging key critical
infrastructure, then analyzing various characteristics to
prioritize this infrastructure for the entire nation. These
priorities are then to be used to direct protective measures for
port security as well as across all other kinds of infrastructure.
The role of IAIP in applying risk management to ports and other
infrastructure is discussed in more detail in chapter 4.
Seaports Are an Important Focus in the Homeland Security Response
Seaport security receives substantial focus in this report because
seaports have been widely regarded as vulnerable to attack. One reason is
that the nation's seaports and inland waterways play a vital role in the
nation's economy and national security. From an economic perspective,
ports are critical links in the commercial trade and transportation
systems, with more than 95 percent of the nation's non-North American
foreign trade, including 100 percent of foreign oil, entering the country
through seaports. The range of commodities involved includes not only a
wide variety of consumer and agricultural products, but also cargo
considered dangerous such as liquefied petroleum gas. A significant
portion of this waterborne trade comes via cargo containers that are
expected to move in and out of ports quickly, in keeping with industry
expectations of just-in-time delivery. Port facilities are also used to
ship military cargo abroad, and the Departments of Defense and
Transportation have designated about 17 ports as "strategic" to support
wartime mobilization, deployment, and resupply. Finally, not only are
ports key hubs in our transportation system, they also function as centers
of industrial, commercial, and financial activity. As such, they are home
to many assets that are deemed to be among the nation's most critical
infrastructure, which is to be protected under the USA PATRIOT Act of 2001
and the Homeland Security Act of 2002.
A second reason that seaports are potentially vulnerable is the wide range
of targets and attack possibilities they encompass. Facilities such as
container terminals, where containers are transferred between ships and
railroad cars or trucks, must be able to screen vehicles entering the
facility and routinely check cargo. Chemical factories and other
installations where hazardous materials are present must be able to
control access to areas containing dangerous goods or hazardous
substances. Vessels, ranging from oil tankers and freighters to tugboats
and passenger ferries, must be able to restrict access to onboard areas,
such as the bridge or other control stations critical to the vessels'
operation. Possible terrorist scenarios range from the use of improvised
explosive devices to attack ferries to the use of recreational boats to
ram key infrastructure in and around ports.
A Framework for Risk Management
While there is a consensus that risk management should be applied to
homeland security programs, doing so is a complex task that has few
precedents and little specific guidance. The Homeland Security Act and
presidential directives have called for the use of risk management.
However, they did not define how risk management was to be accomplished.
Given that there are no established universally agreed upon set of
requirements or processes for risk management of homeland security, we
developed a framework that can be broadly applied to a range of settings,
such as analyzing security in the maritime sector and other environments.
We did so by gathering, reviewing, and analyzing an extensive amount of
public and private sector work; interviewing experts from private
consulting companies in the areas of risk management and risk
computer-modeling; interviewing experts on terrorism; and utilizing our
own past work in this area. We also solicited comments and feedback from
academic experts in risk management. As part of our work, we briefed
officials of the three DHS components about the various phases of the
framework, and the officials generally agreed with its structure and
intent. The application of risk management to homeland security is
relatively new, and the framework will likely evolve as processes mature
and lessons are learned.
The framework we developed is a conceptual synthesis of risk management
approaches that we use as criteria to assess the adequacy of DHS's risk
management systems (see fig. 2). For further information on the framework
and how we developed it, see appendix I.
Figure 2: A Framework for Risk Management
This framework may be applied governmentwide and at various organizational
levels, from departmental down to individual programs. The figure
illustrates the cyclical nature of this approach, and while the phases are
generally linear, changes can be made at any step in the process as new
information becomes available. The five major phases of risk management
are detailed below.
Strategic Goals, Objectives, and Constraints
According to the framework, management decisions are to be made in the
context of the organization's strategic plan, with clearly articulated
goals and objectives that flow from the plan. Performance measures that
are clear, concise, and measurable are linked to the broader goals and can
be used to measure progress toward these goals. An organization's program
and risk planning documents address risk-related issues that are central
to its mission.7 However, various constraints can take many forms and have
an impact on risk related strategies. For example, some constraints may be
imposed by statute, organizational policy, or budget restrictions.
Managers at different levels within an agency or organization may
encounter various constraints that differ with the scale of the operation.
7For reasons of security, this identification may not be public knowledge.
Risk Assessment
Risk assessment helps decision makers identify and evaluate potential
risks facing key assets or missions so that countermeasures can be
designed and implemented to prevent or mitigate the effects of the risks.8
In our framework, risk assessment is a function of threat, vulnerability,
and consequence. The product of these elements is used to develop
scenarios and help inform actions that are best suited to prevent an
attack or mitigate vulnerabilities to a terrorist attack, in conjunction
with the risk-based evaluation of alternatives undertaken while
considering cost and other factors.
o Threat is the probability that a specific type of attack will
be initiated against a particular target/class of targets. It may
include any indication, circumstance, or event with the potential
to cause the loss of or damage to an asset. It is based on an
understanding of an adversary's intention, motivation, history of
attacks, and capability to do damage. Analysis of threat-related
data is a critical part of risk assessment. Information for
characterizing threat can be gained from a variety of sources,
such as the intelligence and law enforcement community, as well as
from past activities of various terrorist groups. Understanding an
underlying pattern of attacks on target types is useful in
predicting future terrorist events and planning mitigation
strategies. However, the unexpected threats not contained in the
historical record of terrorist groups also need to be considered.
Ultimately, one purpose of assessing threats is to assign relative
probabilities to various types of attacks.
o The vulnerability of an asset is the probability that a
particular attempted attack will succeed against a particular
target or class of targets. It is usually measured against some
set of standards, such as availability/predictability,
accessibility, countermeasures in place, and target hardness (the
material construction characteristics of the asset). Each of these
four elements can be evaluated based on a numerical assignment
corresponding to the conditional probability of a successful
attack. The probability that a particular vulnerability could be
successfully exploited is, in part, a function of the
effectiveness of the antiterrorism countermeasures.
o The consequence of a terrorist attack is characterized as the
expected worst case or worse reasonable adverse impact of a
successful attack. The consequence to a particular asset can be
evaluated when threat and vulnerability are considered together.
The outcome of a terrorist attack may include many forms, such as
the loss of human lives, economic costs, and adverse impact on
national security.
8A countermeasure is any action taken or physical equipment used
principally to reduce or eliminate one or more vulnerabilities.
Another closely related element taken into consideration is criticality
(that is, the relative importance) of the asset involved. Criticality
involves the prioritization of assets based on factors such as the
potential for loss of life and the economic implications for the
livelihood, resources, or wealth of the area, region, or country if the
asset were to be lost. Layers of effective security countermeasures
increase the likelihood that a terrorist attack will be unsuccessful as
risk is reduced.
Alternatives Evaluation
Risks can be reduced through various antiterrorism countermeasures or
countermeasure systems designed to prevent an attack or mitigate the
impact of an attack. Two concepts here are key to evaluating
countermeasure alternatives. The first is that countermeasures should be
evaluated against specific risk assessments to determine the extent to
which risks can be reduced by the countermeasure being considered. The
second concept is the role of costs to both public and private sources, as
costs are a critical element in the application of countermeasures. In our
framework, cost-benefit analysis is critical in assessing alternatives,
because it links the benefits derived from risk-reducing alternatives to
the costs associated with implementing and maintaining them.
Management Selection
Management selection in our framework is informed by the outputs in the
preceding phases. Having assessed risks and evaluated countermeasure
options, management selects the blend of intervention strategies and
activities across the entire spectrum of goals, objectives, and components
of risk that achieves the greatest expected risk reduction in relation to
cost for both the short and the long term among the various proposed
alternatives. However, the technical analysis of alternatives is not
likely to resolve or fully capture the numerous elements of concern to
management. Decision makers may employ various risk-reducing strategies.
However, preferences and value judgments will influence decisions about
which strategies to employ. For example, corporate culture may influence
decision makers to concentrate countermeasures on a relatively few
critical assets, while others may value distributional impacts, that is,
some organizations may be more willing than others to distribute resources
over a wider array of assets. Management selection is an important task,
and decisions are made with the information that is available. Our
guidelines for effective internal controls dictate that once decisions are
reached, they, along with the rationales for them, should be documented in
order to inform future actions.
Implementation and Monitoring
This phase in the framework involves the implementation of the selected
countermeasures. Following implementation, monitoring is essential in
order to help ensure that the entire risk management process remains
current and relevant, and reflects changes in the effectiveness of the
alternative actions and the risk environment in which it operates. It is
crucial to exploit any and all information sources, exercises, gaming,
modeling and simulation, analysis of real world events, and sharing of
information in a data sparse environment. Measurable objectives show the
degree to which activities, timelines, support functions, service
delivery, and spending are consistent with goals and implemented in
accordance with the planning process. Program evaluation is an important
tool for assessing the efficiency and effectiveness of the program. In
addition to simply monitoring the implementation of the system and making
adjustments, the entire risk management planning process should be
periodically revisited. Since technology and information change at a rapid
pace, countermeasures in place today may be outdated tomorrow and may
become more susceptible to being breached. In addition, consultation with
external subject area experts can provide a current perspective and an
independent review in the formulation and evaluation of the program.
Objectives, Scope, and Methodology
Our overall aim was to provide a perspective on how three DHS components
have applied risk management as it relates to homeland security in
general, or to port security in particular. More specifically, this report
addresses the following objectives:
o What progress has the Coast Guard made in applying risk
management to its port security mission, and what challenges does
it face in moving further?
o What progress has ODP made in applying risk management to its
administration of the port security grant program, and what
challenges does it face in moving further?
o What progress has IAIP made in applying risk management to
comparing and prioritizing critical infrastructure with one
another and what challenges does it face in moving further?
o Are there key observations that can be drawn from all three of
these efforts with regard to how far the three components have
come in risk management as it applies to terrorism?
To determine what progress the Coast Guard has made in applying risk
management to its port security mission and the challenges that it faces,
we met with Coast Guard officials responsible for port security risk
assessment efforts to discuss the progress they have made and the
challenges that remain. We discussed risk management efforts and
challenges with Coast Guard officials at four ports-Baltimore, Maryland;
Charleston, South Carolina; Houston, Texas; and Seattle, Washington-who
were responsible for risk management activities. We judgmentally selected
these ports because of their geographic distribution, and the results from
our interviews cannot be generalized to ports nationwide. In addition, we
reviewed documents of the Port Security Assessment Program, the Port
Security Risk Assessment Tool, Area Maritime Security Plans, the National
Risk Assessment Tool, the National Maritime Security Profile, and the
National Maritime Strategic Risk Assessment. We also reviewed key
legislation such as the Maritime Transportation Security Act of 2002 and
prior GAO reports on maritime security. Finally, we reviewed threat
assessments produced by the National Maritime Intelligence Center and the
Transportation Security Agency to gain a more complete understanding of
the challenges.
To determine what progress ODP has made in applying risk management to its
administration of the port security grant program and the challenges that
it faces, we compared fiscal year 2004 and fiscal year 2005 ODP port
security grant program procedures. In order to understand the grant
process and the risks related to individual ports, we reviewed the risk
assessment tools used by ODP officials, including the Coast Guard's Port
Security Risk Assessment Tool.9 We reviewed and summarized a database
listing fiscal year 2004 grant applications and awards to determine the
extent to which criteria for awards coincided with the receipt of grant
awards. We reviewed the Inspector General's (IG) January 2005 report of
the port security grant program and discussed the recommendations
contained in the report with ODP officials. We examined procedural changes
made by ODP, in response to the IG recommendations and other factors, to
the 2005 grant application process. We did not review the fiscal year 2005
award decisions because we had completed our fieldwork before award
decisions were announced, in September 2005. We met with Coast Guard,
Maritime Administration, ODP, and IAIP officials involved in the port
security grant program process. We also reviewed pertinent legislation,
such as appropriations for the grant program for successive fiscal years.
9The Coast Guard Port Security Risk Assessment Tool is designed to be used
by the Captains of the Ports when making risk-based analyses of assets in
their area of responsibility.
To determine what progress IAIP has made in applying risk management to
comparing and prioritizing critical infrastructure, and what challenges it
faces in moving ahead, we reviewed key legislative and executive
documents, such as the Interim National Infrastructure Protection Plan,
the Homeland Security Act of 2002, Homeland Security Presidential
Directives 7 and 8, national strategies, and DHS's strategic plan. We met
with IAIP officials responsible for identifying and prioritizing threats,
vulnerabilities, and consequences across different types of critical
infrastructure to determine the obstacles they face in making such
evaluations and the challenges they face in making progress in this area.
We reviewed documents, such as the Risk Analysis and Management for
Critical Asset Protection and the Buffer Zone Protection Program. We
interviewed Office of Management and Budget (OMB) officials responsible
for oversight of issues involving infrastructure protection to obtain
their views on risk management practices across the federal government.
To determine whether there are key observations that can be drawn from the
three components we reviewed, we analyzed and synthesized the findings we
developed to identify challenges that remain in applying risk management
to homeland security. We compared the three components' progress in
applying risk management principles to their respective tasks, identified
common experiences in applying risk management, and drew conclusions about
issues that may need addressing. We reviewed numerous documents, including
pertinent statutes and presidential directives, GAO reports on high-risk
programs in the federal government, the National Strategy for the Physical
Protection of Critical Infrastructures and Key Assets, and testimony by
the Secretary of DHS.
We performed our work in accordance with generally accepted government
auditing standards between May 2004 and November 2005.
Chapter 2: The Coast Guard Has Made Progress in Using Risk Management, but
Challenges Remain Chapter 2: The Coast Guard Has Made Progress in Using
Risk Management, but Challenges Remain
The Coast Guard has established a foundation for applying risk management
to port security; its next challenges are to refine and strengthen its
approach. The Coast Guard has made progress in all five phases of risk
management (see table 2). For the first phase-goals and objectives-the
Coast Guard has established broad strategic goals for port security, and
its next challenge is to translate these goals into specific, measurable
objectives that can be used to assess performance. The Coast Guard has
actions under way to address this challenge-a key step in determining the
extent to which its actions actually reduce risk. The Coast Guard has made
the most progress in the second phase-conducting risk assessments. Six
separate but related assessment efforts, covering both individual ports
and the nation as a whole, have given the Coast Guard a clearer sense of
the vulnerabilities that exist. However, current assessments are limited
in terms of their methodology, and they do not allow the Coast Guard to
compare and prioritize relative risks across ports-limitations that the
Coast Guard recognizes and is taking steps to address. Enhancing these
first two risk management phases is key to making further progress on the
next two phases-evaluating risk mitigation alternatives and selecting a
particular alternative for action. Without measurable objectives and more
complete methodologies, the risk management process may not be able to
target the most significant security concerns or determine the most
cost-effective approach to take in providing reasonable protection.
Additionally, weaving data produced from the risk management process into
the annual cycle of program review remains a challenge. Finally, with
regard to the fifth phase-implementation and monitoring-the Coast Guard
has demonstrated the ability to evaluate its efforts and make
improvements. However, more extensive and more formal feedback mechanisms
would help ensure that Coast Guard headquarters managers can inform field
staff about actions taken as a result of the comments received about the
risk management process. The Coast Guard has actions under way to improve
feedback loops.
Table 2: Summary of Progress Made and Challenges That Remain in the Coast
Guard's Risk Management Approach
Source: GAO analysis of the Coast Guard's risk management efforts.
Coast Guard Homeland Security Activities Revolve Heavily around the Maritime
Domain
The Coast Guard is the lead federal agency responsible for protecting
domestic ports. In this role, the Coast Guard must identify, evaluate, and
mitigate many kinds of security challenges. Ports are often sprawling
enterprises that contain key infrastructure besides docks, piers, ships,
barges, and warehouses. Many ports are also home to power plants, chemical
factories, bridges and tunnels, and a variety of other assets of critical
importance to the nation's economy and its defense.
Coast Guard expenditures and activities for port security have risen
dramatically since the terrorist attacks of September 11. The Coast Guard
estimates that its budget for port security has jumped from about $250
million in fiscal year 2001 to about $1.5 billion in fiscal year 2005.
Since the terrorist attacks, the Coast Guard has carried out a myriad of
port security activities, including increasing its intelligence
capabilities, carrying out more harbor patrols and vessel escorts,
establishing security zones, and working more extensively with federal,
state, local, and industry stakeholders on port security matters as
required by the Maritime Transportation Security Act of 2002 (MTSA).
While much of the Coast Guard's homeland security efforts center
specifically on ports, these activities are part of the agency's broader
mission of Ports, Waterways, and Coastal Security (PWCS). This mission
involves protecting the maritime domain and marine transportation system;
preventing terrorist attacks, and responding to and recovering from
attacks that do occur.1 As part of the PWCS mission, the Coast Guard aims
to develop greater "maritime domain awareness"-that is, improving port
stakeholders' understanding about anything associated with the global
maritime environment that could adversely affect the security, safety,
economy, or environment of the United States. Maritime domain awareness
seeks to identify threats as soon as possible and far enough away from
domestic ports to eliminate or mitigate the threat. Several Coast Guard
efforts are under way to help address both port security and marine domain
awareness. In particular:
1U.S. Coast Guard, Maritime Sentinel: Coast Guard Strategic Plan for
Ports, Waterways, and Coastal Security (Washington D.C.: September 2005).
According to Homeland Security Presidential Directive 13 on Maritime
Security Policy (Dec. 21, 2004), maritime domain means all areas and
things of, on, under relating to, adjacent to, or bordering on a sea,
ocean, or other navigable waterway, including all maritime related
activities, infrastructure, people, cargo, and vessels and other
conveyances.
o The Coast Guard is planning to expand its sector command
centers, where officials can receive data 24 hours a day on
maritime activities. Currently, the Coast Guard plans to develop
sector command centers at 35 ports.2
o The Coast Guard is involved in a major recapitalization
effort-called the Integrated Deepwater System-to replace and
modernize the agency's aging fleet of aircraft and vessels,
including improved and integrated command, control, communications
and computers, intelligence, and surveillance and reconnaissance
capabilities. Since the terrorist attacks of September 11, the
Coast Guard has revised its plans, and the Deepwater program now
includes improving maritime domain awareness and maritime security
capabilities as part of its mission. This program is scheduled to
take 20 years and cost between $19 billion and $24 billion.
Federal statutes and presidential directives call for the Coast Guard to
use risk management in its homeland security efforts. MTSA, for example,
calls for the Coast Guard and other port security stakeholders to carry
out a variety of risk-based tasks, including assessing risks and
developing security plans for ports, facilities, and vessels.3 The Coast
Guard's progress across the various risk management phases is thus a key
part of its homeland security mission. The remainder of this chapter
discusses the Coast Guard's progress and challenges on each of the phases
in GAO's framework.
2According to the Coast Guard, there are 30 sectors with command centers
already in place.
3See 46 U.S.C. S: 70103(b), (c); 33 C.F.R. S:S: 103.400, 103.500, 104.300,
104.400, 105.300, and 105.400.
Goal-Setting: High-Level Goals Are in Place, with Efforts Under Way to Set
Measurable Objectives
The Coast Guard has been able to make some progress in the first phase of
the risk management framework, in that it has established high-level
strategic goals for its PWCS mission. The Coast Guard set its national
strategic goals for port security in December 2002.4 In order of priority,
the goals were as follows:
o preventing terrorist attacks within, and terrorist exploitation
of, the maritime domain;
o reducing America's vulnerability to terrorism in the maritime
domain;
o protecting population centers, critical infrastructure,
maritime borders, ports, coastal approaches, and the boundaries
and seams between them;
o protecting the U.S. marine transportation system while
preserving the freedom of the maritime domain for legitimate
pursuits; and
o minimizing the damage and expediting the recovery from attacks
that may occur within the maritime domain as either the lead
federal agency or a supporting agency.
Working with federal, state, local, and industry stakeholders involved in
port security, the Coast Guard has also developed security plans for port
areas across the country. These plans reflect the characteristics and
needs of the individual ports, and in general, they aim to deter a
terrorist incident and improve communication among port stakeholders.
These plans are specific to each port location and are aligned with
higher-level port security goals.
While the Coast Guard has set broad goals for its port security mission,
it still faces challenges in developing objectives into more specific and
measurable results that measure progress toward these goals. So far, the
Coast Guard has expressed its port security objectives in terms of
activity levels, such as conducting patrols, escorting vessels, and
inspecting cargo. While such activities may have contributed to improved
security in and around the nation's ports, using them as measures may not
systematically target areas of higher risk and may not result in the most
effective use of resources, because these measures are not pointed toward
outcomes. They describe what levels of activity, or outputs, the Coast
Guard is providing, but they do not provide an indication of what these
activities are accomplishing. Doing so requires measures that are clearly
tied to results. Such measures would indicate the extent of progress made
and help identify the security gaps that still remain.5
4U.S. Coast Guard, Maritime Strategy for Homeland Security (Washington,
D.C.: December 2002).
Developing measurable objectives is a complex and difficult task, but
Coast Guard officials recognize that doing so is a necessary next step and
plan to have such objectives developed in fiscal year 2006. In September
2005, the Coast Guard stated that it plans to develop a measure of its
performance that will be based on an assessment of threat, vulnerability,
and consequence. The Coast Guard plans to develop objectives for reducing
overall risk. As part of this process, the Coast Guard plans to assess the
impact of its activities in reducing threats, vulnerabilities, and
consequences.
Risk Assessments: Progress Has Been Substantial, but Challenges Remain for
Addressing Limitations in Assessment Data and Methodology
The Coast Guard's risk management activities have centered primarily on
this phase of the risk management process, with assessments being
conducted at both port and national levels. Further progress on the
quality of these assessments is challenged by several types of limitations
in the data and the methodology being used.
Several Sets of Assessments Have Been Completed at the Port and National Levels
Following the terrorist attacks of September 11, and consistent with
MTSA's directives, the Coast Guard has greatly expanded the scope of its
risk assessment activities. Before the attacks, these assessments centered
on matters such as ecological damage and general marine safety. In 1999,
for example, the Coast Guard adopted a risk management approach for its
marine safety and environmental mission area. Assessments of key
infrastructure in and around ports came largely after the attacks. Three
such assessments have been done at the port level (see table 3).6 These
assessments included data on threats, vulnerabilities, and
consequences-the three types of information used in evaluating risk:
5Any goals that the Coast Guard establishes will need to be aligned with
two other national efforts to protect critical infrastructure. First,
Homeland Security Presidential Directive-7 establishes a national policy
for federal departments and agencies to identify and prioritize critical
infrastructure and key resources, including assets and resources in and
around ports. Second, Homeland Security Presidential Directive-13
establishes policy and guidelines for implementing actions that enhance
maritime security. The policy includes (1) preventing terrorist attacks
and reducing vulnerabilities to attacks in the maritime domain; (2)
enhancing security of ports, critical infrastructure, and coastal
approaches; (3) enhancing international relationships; and (4) ensuring
coordinated implementation of authorities and responsibilities among
federal departments and agencies.
o Threats. The assessments gathered information on plausible
threat scenarios, such as using weapons of mass destruction,
ramming a vessel or facility, or detonating devices underwater. In
general, local Coast Guard personnel or other federal and
nonfederal stakeholders decided what threat scenarios to include
in the assessment based on their knowledge of the port.
o Vulnerabilities. The assessments evaluated threat scenarios
against potential targets, such as passenger vessels, bridges, or
terminals, to assess the degree to which these potential targets
were vulnerable to attack.
o Consequences. Finally, the three assessments addressed the
potential outcomes of successfully carrying out a threat against a
potential target. These consequences included such matters as loss
of life, damage to the environment, damage to property, and
economic disruption.
6We previously reported on two of these efforts-the area maritime security
assessments and the Port Security Assessment Program. See GAO, Maritime
Security: Substantial Work Remains to Translate New Planning Requirements
into Effective Port Security, GAO-04-838 (Washington D.C.: June 30, 2004),
and Maritime Security: Better Planning Needed to Help Ensure an Effective
Port Security Assessment Program, GAO-04-1062 (Washington D.C.: Sept. 30,
2004).
Table 3: Port-Level Assessments Conducted by the Coast Guard
Source: GAO analysis of the Coast Guard's port-level assessments.
These assessments have resulted in considerable progress in understanding
and prioritizing risks within a port. In particular, the Port Security
Risk Assessment Tool (PS-RAT), a computer program that includes possible
threat scenarios, allows the Coast Guard to assess risks and develop
relative rankings for infrastructure at each port location. At one port,
PS-RAT was used to compare and inform priorities on over 1,000 critical
items of infrastructure.
Three other efforts have focused on assessments at the national level (see
table 4).7 These efforts have relied in part on data and information
generated from the local assessments described above, but they have also
incorporated additional information. For example, the National Maritime
Security Profile integrated available information from the intelligence
community-a step that had not been carried out in any of the local risk
assessments. Another assessment, the National Maritime Strategic Risk
Assessment, sought to develop risk profiles for each of the Coast Guard's
strategic goals; it examined specific mission areas, including port
security, search and rescue, and law enforcement, and it sought input from
field commanders on ways to mitigate key risks. According to Coast Guard
staff, this was the first attempt at a large-scale strategic risk
assessment that sought to assess the status of the maritime domain.
7In addition to these efforts, the Coast Guard was involved in an
interagency effort called the National Maritime Transportation Security
Plan. This effort included the results of the National Maritime Security
Profile. According to the Coast Guard, the results of the plan will
contribute to the Coast Guard's outcome measures for its PWCS mission.
Table 4: National-Level Assessments Conducted by the Coast Guard
Source: GAO analysis of the Coast Guard's national-level assessments.
While the port-level assessments focus on specific assets and
infrastructure at each location, national assessments have focused more
generally on understanding the risk posed to various classes of assets
(such as container ships, barges, power plants, or bridges and tunnels) by
various types of attacks (such as using explosives, taking control of an
asset, or using weapons of mass destruction). These national assessments
address, for example, whether the maritime domain is at greater risk from
a takeover of a power plant or from a weapon of mass destruction planted
on a container ship. The national assessments do not compare risks faced
by specific assets at one port with risks faced by specific assets at
another port.
Key Challenges Involve Improving Data and Methodology
The progress in conducting risk assessments is tempered by a number of
challenges that remain in making these assessments more robust tools for
informing the risk management process. These challenges are numerous and
complicated, and this chapter illustrates some of the important issues the
Coast Guard faces in making its risk assessments more useful. The
challenges discussed here involve (1) improving the threat, vulnerability,
and consequence data on which the assessments are based and (2) addressing
methodological limitations that affect the reliability, completeness, and
applicability of the risk assessments themselves.
Limitations of Data on Threats, Vulnerabilities, and Consequences
Our review of the Coast Guard's processes and our discussions with Coast
Guard personnel surfaced a number of data-related problems that limit the
reliability and completeness of the risk assessments (see table 5). For
example, the tools require information not only about the types of threats
a facility may face, but also about the probability of such threats
occurring. The threat data received by the Coast Guard from the
intelligence community do not allow the Coast Guard to model these
probabilities, thus limiting the value of the output. The problems we
identified have implications for the ability to effectively compare risks
faced by the various types of port-related infrastructure.
Table 5: Examples of Data-Related Challenges in Coast Guard Risk
Assessments
Source: GAO analysis of the Coast Guard's risk assessment efforts.
These challenges are complex and technical, and the following examples
illustrate the kinds of limitations they pose:
o Limitations in threat-related data. The intelligence
information the Coast Guard normally receives about threats is not
specific enough for all of the threat scenarios in the National
Maritime Security Profile or the PS-RAT.8 For example, the type of
threat data that Coast Guard personnel could use to model threats
includes data on the presence of terrorist cells nationally and
internationally, the capability and intent of terrorist groups as
they relate to specific types of attack in and around ports, and
the specific target groups. Increasing the quality of this
information would improve the quality of the output. For example,
when the Coast Guard received and integrated higher-quality
information about some threats from its Intelligence Coordination
Center, it modified data for 80 of about 300 threat scenarios in
its National Maritime Security Profile.
o Limitations in vulnerability-related data. The Coast Guard's
risk assessment tools were designed to evaluate existing security
in and around a building or vessel, but according to Coast Guard
officials, the baseline established by the tools excludes areawide
actions the Coast Guard has taken to reduce vulnerabilities in and
around ports, such as conducting more patrols, creating
operational centers, or establishing security zones in and around
key ports. The baseline also excludes actions taken by local port
stakeholders, such as increasing the number of harbor patrols
conducted by local law enforcement. The Coast Guard designed its
tools this way because the primary purpose of the tools was to
provide a port-level risk ranking of assets. Using this
information, the Coast Guard could then use tools, such as the
PS-RAT, to measure the benefit and value of any interventions for
a specific vessel, facility, or asset type that it initiates
against the original baseline. Coast Guard officials recognize
that another tool would be useful in determining the overall
vulnerabilities that exist after all Coast Guard actions have been
taken.
o Limitations in consequences-related data. The Coast Guard's
assessment of consequences is limited to direct effects of a
terrorist attack, such as loss of lives and property damage; it
does not consider important secondary effects, such as follow-on
effects to the economy, including loss of jobs or increased energy
costs that may occur months after an attack. This limitation
likely understates the overall consequences that result from an
attack and may distort relative risks associated with various
threat scenarios. Coast Guard officials note that estimating
secondary effects is important but difficult since there is a lack
of accepted methods for doing so. A second limitation is that
there is no commonly agreed upon value for a consequence such as
death or injury, or the symbolic effect of destroying a national
symbol such as the Statue of Liberty. For example, the Coast
Guard's model places a dollar value of $1 million (in 2005
dollars) on the loss of a life. Other components, such as the
Environmental Protection Agency, use a monetary value of $6.1
million (in 1999 dollars). The value chosen can affect the
priorities that emerge from using the risk assessment tools.
8The threat scenarios examined by the Coast Guard are contained in
documents that are considered sensitive and for official use only.
Accordingly, we do not provide detailed examples of various threat
scenarios evaluated by the Coast Guard.
We discussed these data-related limitations with Coast Guard officials,
who generally agreed with our observations. The Coast Guard has since
planned or started several actions designed to address some of these
limitations. Coast Guard officials said they plan to improve PS-RAT based
in part on the limitations discussed above.9 For example, they made
changes in procedures for obtaining information from the Intelligence
Coordination Center and have focused on improving the quality of
information received from the intelligence community.10 In addition, the
Coast Guard plans to improve vulnerability and consequence data. For
example, the Coast Guard plans to assign weights to different levels of
consequences. Coast Guard officials said they hope to accomplish these
changes by the end of 2005.
Methodological Limitations
Two key methodological limitations affect the use of risk assessment data
as a tool for informing decision makers on relative risks across port
locations. The first limitation relates directly to the ability to compare
and prioritize one port with another, while the second limitation relates
to the process used to rank and prioritize individual threats.
o Risks cannot be compared between ports. PS-RAT results were not
designed to compare the risks at one port with risks at another.11
While PS-RAT allows the Coast Guard to compare and prioritize key
infrastructure within a port, it does not produce a risk ranking
that permits the Coast Guard to compare and prioritize
infrastructure across ports. Interport comparisons, while
theoretically possible, are difficult to actualize in practice. In
general terms, the difficulties stem largely from the fact that,
for each port, multiple scenarios must be considered; the
scenarios that are deemed most relevant to each port, however,
will differ from port to port. For example, ports that support
passenger ferries and container cargo may be exposed to different
risks than ports that primarily support bulk cargo. Comparisons
are further limited because Coast Guard personnel at different
ports use different methods to input data into PS-RAT. For
example, Coast Guard officials at some field offices have
summarized information on a type of asset, such as all bridges at
a port location, while officials at other locations have developed
data on each bridge in the area of responsibility. We discussed
this limitation with the Coast Guard officials, and during the
course of our review, the Coast Guard initiated work on a Maritime
Security Risk Assessment Model (MSRAM)-a model that should permit
the Coast Guard to compare the relative risks of high-value assets
at one port with assets at a different port. The model is to
include an analysis of various threat scenarios, vulnerabilities,
and consequences. Overall, the MSRAM is to score and characterize
risk associated with individual assets, including the estimated
likelihood of an attack, the vulnerability of the asset should an
attack occur, and the impacts of a successful attack. The Coast
Guard has also requested data from the Intelligence Coordination
Center on intent and capability that could improve the estimate of
relative likelihood of various threat scenarios. Coast Guard
officials said they plan to implement the MSRAM by the end of
2005.
o Risks viewed as less probable could surprise the Coast Guard.
The Coast Guard evaluates hundreds of threat scenarios that are
deemed plausible-focusing attention, ultimately, on those threats
that, together with identified vulnerabilities and consequences,
pose the greatest overall risk. For example, in the National
Maritime Security Profile, the Coast Guard classifies the risk of
various threat scenarios as very high, high, medium, low, and very
low, and it centers attention on scenarios that it estimates are
very high or high risk. The Coast Guard's approach, although a
useful starting point, may not be as reliable as the process would
appear to suggest given the data limitations we describe above.
How agencies like the Coast Guard deal with scenarios that receive
low rankings is important in addressing the possibility of
strategic surprise-an attack scenario that may not be identified
or given high priority in the initial risk assessment process.
Without sensitivity analysis or formal feedback loops to reassess
all scenarios and therefore provide greater assurance that the
rankings are as reliable as possible, the risk of being unprepared
for strategic surprise may increase.12 The Coast Guard addresses
this issue by making appropriate adjustments in priorities when
tactical and strategic information call for such changes. We
discussed this issue further with agency officials, and partly on
the basis of these discussions, the Coast Guard is taking
additional steps to address this issue. It is doing so by (1)
increasing coordination among risk stakeholders at all levels to
improve checks throughout the risk management cycle, (2) making
refinements to threat data by requesting the Intelligence
Coordination Center to provide estimates of capability and intent
of terrorist groups, (3) including time horizons for various
scenarios, and (4) leveraging independent assessments conducted by
different subject matter experts as a way of checking its risk
assessment work.
9Coast Guard officials said that in addition to making improvements in
threat, vulnerability, and consequence data, they plan to align some of
their efforts more closely with Department of Homeland Security risk
assessment databases and to conduct sensitivity analyses of various
mitigation strategies.
10IAIP officials said that the Coast Guard's efforts are a pilot project,
and that if sufficiently mature, the effort will be more broadly
implemented by the intelligence community.
11Similarly, area maritime security plans are not comparable with one
another, in part because PS-RAT results underpin the plans. In addition to
the limitations we describe above, the scores for each port are relative
to that port and scores from one port cannot be compared with the scores
from another port because, in some cases, local Coast Guard staff
developed scores for asset types, such as bridges or warehouses, and at
other port locations, the local staff assigned scores for individual types
of assets. The Port Security Risk Assessment Program conducted studies at
55 ports at a cost of about $35 million and these studies are also not
comparable with one another.
Evaluation of Alternatives: Ability to Evaluate Alternatives Is Greatest at the
Local Level
Just as the Coast Guard's ability to assess risk is stronger at the
individual port level than across ports, its ability to evaluate various
alternatives for addressing these risks is greater at the port level as
well. PS-RAT was specifically developed to help local Captains of the Port
concentrate and prioritize their resources and evaluate alternative
methods of risk reduction. Data from PS-RAT help identify vulnerabilities
within a port and can be used in improving security measures related to
the area maritime security plans. PS-RAT is not designed to work, however,
above the port level. At the national level, the Coast Guard has conducted
qualitative evaluations of the potential benefits of various alternatives
for reducing risk levels, such as improved information sharing through the
use of interagency operational centers, waterborne patrols, and escorting
ships. In addition, it is assessing the potential reduction in risk of
different strategies for improving awareness of the maritime domain.
The effectiveness of such evaluations, both within and between ports, is
influenced to a large degree by the performance standards and goals that
are set, as well as the reliability and completeness of the risk
assessments that are conducted. To the extent that goals are missing and
risk assessments produce data that do not completely and reliably depict
threats and vulnerabilities, the prospective evaluation of benefits and
costs of future mitigation strategies may not target the areas with the
greatest gaps or lead to the most cost-effective decisions.
12See Y. Y. Haimes, S. Kaplan, and J. H. Lambert, "Risk Filtering,
Ranking, and Management Framework Using Hierarchical Holographic
Modeling," Risk Analysis, Vol. 22, No. 2, 2002, and M. Leung, J. H.
Lambert, and A. Mosenthal, "A Risk-Based Approach to Setting Priorities in
Protecting Bridges Against Terrorist Attacks," Risk Analysis, Vol. 4, No.
4, 2004.
In examining various alternatives, a key consideration will be measuring
the overlapping benefits of different strategies as a means of developing
data on what set of alternatives may provide the most improved security in
return for the resources expended. These overlapping benefits and costs
may involve Coast Guard actions, such as expanding the number of
operational centers at port locations or procuring new ships and aircraft,
or actions by others, such as more inspections of container cargo
shipments or developing better intelligence capability. The Coast Guard's
evaluation of alternatives generally involves examining its own possible
actions, such as developing proposals to expand the number of operational
centers, without placing the cost of such actions in context with other
alternatives that have already been deployed or that could be used. In
discussions with us, Coast Guard officials said addressing this issue was
part of the Coast Guard's efforts to further refine its risk management
approach. For example, the Coast Guard is working toward examining
alternatives by assessing the degree to which they reduce risk in exchange
for the cost involved.
Management Selection: Local Strategies Have Been Selected but Would Be Better
Informed by Improvements in Phases for Goal Setting and Risk Assessment
In regard to management selection-the fourth phase of the framework-the
Coast Guard has used its port-level assessments to select specific
mitigation strategies to manage vulnerabilities in and around facilities.
For example, local Captains of the Port have used the assessment
information in coordination with input from local stakeholders to (1)
establish security zones around key port infrastructures; (2) improve
security in and around passenger vessels; and (3) coordinate security
improvements, such as fences, cameras, and barriers around port
infrastructures. At the national level, the Coast Guard is designing and
planning to implement an array of radar systems, sensors, and information
systems to identify and track possible threats in the maritime domain. One
element of this effort is the establishment of maritime intelligence
fusion centers that cover the Pacific and Atlantic coasts. These fusion
centers are providing intelligence to the Coast Guard intelligence network
in the field, and the centers share information with interagency partners,
such as the Navy and Customs and Border Protection.
In general, progress in this area is affected heavily by the same factors
affecting progress in evaluating alternatives: progress in setting
measurable performance objectives and improving the reliability and
completeness of risk assessments. The various phases of the risk
management model build and rely on one another, and in this case the
quality and reliability of the results are heavily affected by the quality
and reliability of efforts in the first two phases.
Another major challenge will be to strategically integrate risk-based data
into other management systems, such as the annual cycle of program and
budget review for assessing how to deploy resources among ports. Coast
Guard officials acknowledged that using risk-based data to inform the
annual budget cycle is an important step and are taking further steps to
address them. For example, the Coast Guard's MSRAM tool will compare risks
at one port with those at another port, and the data produced could inform
the Coast Guard's annual budget review of programs and resource
allocation. In addition, based in part on our discussions with Coast Guard
officials, the Coast Guard's resource planning guide for fiscal years
2008-2012 calls attention to the importance of joining these processes.
The guide states that by "employing principles of risk management and
using our understanding of strategic risks to Coast Guard mission
performance, we will be able to make better decisions regarding
investment, re-investment and base management priorities."
Implementation and Monitoring: A Foundation for Continuous Improvement Is in
Place; Challenges Remain in Refining It and Developing Formal Policies
The Coast Guard has made progress in the fifth phase of the
framework-implementation and monitoring-by improving existing tools and
systems in a variety of ways. For example:
o The Coast Guard has improved PS-RAT; version 2, developed in
November 2002, by providing such improvements as greater detail on
consequence data. Additionally, the MSRAM will include
improvements in the quality of threat information in its analysis.
o The Coast Guard used the Port Security Assessment Program to
further refine area maritime security plans by offering
intelligence and other information to the local Captains of the
Port on security issues the plan may not have covered.
o In November 2004, the Coast Guard started conducting the
National Maritime Strategic Risk Assessment to obtain comments and
feedback from Coast Guard area and headquarters staff on its
mission areas, including its port security mission. Among other
things, the staff provided input on ways that the Coast Guard
could improve the manner in which it carries out its port security
mission and reduce any gaps in coverage. For example, staff
identified opportunities for leveraging resources, such as
strengthening the partnership with nonfederal stakeholders
involved in maritime security, as well as potential gaps in the
port security mission, such as training, equipment, and
technology. This information was summarized by a Coast Guard
consultant and was provided to managers in Coast Guard
headquarters.
In some cases, this progress is limited to a degree by a lack of a formal
policy for taking action on feedback that is received about ways to
improve the risk assessment and management approach. For example, when the
Coast Guard receives feedback from field staff on ways to improve its
strategic approach, there are no formal policies for addressing the issues
raised. Without such policies, there is little assurance that the feedback
received will inform decisions to improve risk management practices and
ultimately port security. For example, a Coast Guard official at one port
noted that while there are informal communication channels to offer
feedback on the PS-RAT, there are no formal procedures for doing so, and
that such procedures would be useful in addressing concerns of local users
when using the tool. Recognizing that there was no formal feedback process
for the PS-RAT, the Coast Guard developed an ad hoc methodology to gather
feedback from the users of the tool. In March 2005, Coast Guard
headquarters hosted a workshop to obtain input from Coast Guard field
offices on ways to improve the PS-RAT. The workshop resulted in a list of
possible modifications to the PS-RAT and a plan to develop the MSRAM.
Coast Guard officials acknowledge that a formal feedback process would be
beneficial, and they plan to include one as part of the Coast Guard's
MSRAM.
Identifying and addressing organizational barriers to the Coast Guard's
ability to improve or carry out its risk management approaches is a final
consideration for the monitoring and implementation phase. One key
organizational challenge is building and sustaining expertise and skills
for effectively designing and using the risk management tools, techniques,
and models necessary for managing the Coast Guard's efforts in carrying
out its port security and maritime domain awareness responsibilities. The
Coast Guard provides training on risk-based decision making at its
training center. Recently, the Coast Guard revised its officer evaluation
form by including "risk assessment" as a key leadership competency for its
officers. Partly on the basis of the briefings we provided to agency
officials, the Coast Guard plans to carry out workshops to examine ways to
integrate risk management into the PWCS strategic plan and activities.
Applying risk management tools and techniques is a complex undertaking,
and it requires a managed effort to maintain and build organizational
expertise and skills to do the job well.
Conclusions
The foundation the Coast Guard has established for risk management is
generally sound. It represents a strong commitment on the Coast Guard's
part to using risk management effectively in making decisions. Further,
the Coast Guard, often acting on discussions held during the course of
this review, has actions under way to address many of the concerns we
identified. The most difficult work ahead revolves around systematically
integrating risk-based information into management systems, including the
annual cycle of program review, that can help inform decisions about how
to deploy resources and security measures among ports and to examine the
value of new programs in addressing security gaps that remain. In this
regard, the Coast Guard's efforts are still in the early phases of
development. As the Coast Guard moves forward, it is especially important
that the agency develop ways to establish a stronger linkage between the
various local and national risk assessment efforts under way. For example,
area maritime security plans are not comparable, and security risks at one
port location cannot be compared with the risks identified at another
location. As a result, the collective value of these individual efforts is
diminished. Developing ways to establish a stronger linkage would likely
increase the value of the work.
Recommendations for Executive Action
We are not making recommendations in those areas where the Coast Guard has
actions well under way. The recommendations below are designed primarily
to spotlight those areas in which additional steps are most needed to
strengthen agency efforts to implement a risk management approach to the
Coast Guard's port security activities. Accordingly, we recommend that the
Secretary of Homeland Security direct the Commandant of the Coast Guard to
take action in the following two areas:
o Risk assessment: Develop plans to establish a stronger linkage
between local and national risk assessment efforts. This effort
could involve strengthening the ties between local assessment
efforts, such as area maritime security plans, and national risk
assessment activities.
o Alternatives evaluation and management selection: Ensure that
procedures for these two processes consider the most efficient use
of resources. For example, one approach involves refining the
degree to which risk management information is integrated into the
annual cycle of program and budget review.
Agency Comments and Our Evaluation
In commenting on a draft of chapter 2, DHS, including the Coast Guard,
generally agreed with our findings and recommendations. DHS said that the
report notes that the Coast Guard has made progress in all five risk
management phases and is taking action to address the challenges that
remain. In addition to commenting on our recommendations, the Coast Guard
provided several technical comments under separate cover, and we revised
the report when appropriate. Written comments from DHS are in appendix II.
Chapter 3: Stronger Risk Management Approach Could Improve the
Accountability of the Port Security Grant Program Chapter 3: Stronger Risk
Management Approach Could Improve the Accountability of the Port Security
Grant Program
The Office for Domestic Preparedness within the Department of Homeland
Security has made progress in applying risk management to the port
security grant program but faces challenges in strengthening its approach,
as demonstrated in part by its experience in awarding past grants.
Examples of progress-and challenges-can be found across all five risk
management phases (see table 6). ODP has established overall goals for the
grant program but faces challenges in setting specific and measurable
program objectives, in part because this effort hinges on similar action
by other federal agencies. ODP's progress, with input from the Coast Guard
and IAIP, has been greatest in conducting the actual risk assessments. It
assessed 129 ports and, using risk-based prioritization, narrowed to 66
the number of ports eligible to apply for fiscal year 2005 grants. Its
methods for assessing risks and evaluating mitigation alternatives,
however, still are limited in their ability to prioritize relative risks
across ports or to calculate the costs and benefits of various
alternatives. Finally, while ODP has also made progress in developing a
risk-based grant selection process and mechanisms to monitor what the
grants accomplish, it has not always completely relied on a risk-based
process. For example, grant awards for fiscal year 2004 illustrate the
trade-offs that occur when attempting to award grants to applicants that
are at greater risk and, at the same time, provide funds to applicants
that have the larger financial need. At the end of what was, in part, a
risk-informed assessment process, ODP change the criteria for awarding
grants when it decided to give lower priority to grant applications
involving projects at large companies, on the assumption that these
companies were better able than other entities to pay for their own
security improvements. This changed 40 percent of the grants originally
recommended for an award, as projects with higher risk but greater
potential for self-funding gave way to lower-risk projects with more
limited funding prospects. ODP changed the process for fiscal year 2005 by
clarifying the criteria it would use in awarding grants and by requiring
applications from private entities to match at least 50 percent of the
total amount requested.
Table 6: Summary of Progress and Challenges in the Port Security Grant
Program
Source: GAO analysis of ODP's port security grant program.
aThe Coast Guard provided various data on ports, such as amount of total
cargo, domestic cargo, international cargo, number of passengers using
ferries, and number of passengers using cruise ships. ODP prioritized
ports by evaluating these data to determine a list of the 66 highest-risk
port areas.
ODP Manages Port Security Grants
The port security grant program was established in fiscal year 2002 under
the purview of the Transportation Security Administration (TSA), which
became part of DHS in March 2003.1 Because of organizational changes
within DHS, the grant program has been administered by ODP within the
Office of State and Local Government Coordination and Preparedness since
May 2004. ODP was transferred from the Department of Justice to DHS upon
passage of the Homeland Security Act of 2002.
1The legislation enabling the port security grant program was the
Department of Defense and Emergency Supplemental Appropriations for
Recovery from and Response to Terrorist Attacks on the United States,
2002, Pub. L. No. 107-117, 115 Stat. 2230, 2327 (2002).
The grant program provides assistance to nonfederal stakeholders for
making security improvements at the nation's ports. During fiscal years
2002-2004, grants from the program totaled about $560 million and covered
such concerns as more fencing, cameras, and communications equipment. For
fiscal year 2005, the appropriations act for DHS provided $150 million for
port security grants.2 The 2005 program focused on three primary concerns:
(1) protection against improvised explosive devices carried by small
craft, underwater craft, or vehicles; (2) enhanced explosives detection
capabilities for the owners and operators of vehicle ferries and
associated facilities (as shown in fig. 3); and (3) facility security
enhancements in the nation's highest-risk ports. The program is designed
to operate in coordination with federal partner agencies and industry.3
Grantees are selected through a competitive process.
2Department of Homeland Security Appropriations Act, 2005, Pub. L. No.
108-334, 118 Stat. 1298, 1309 (2004).
3Federal agency and industry partners include the United States Coast
Guard, the Information Analysis and Infrastructure Protection Directorate,
the Border and Transportation Security Directorate, and the Transportation
Security Administration within DHS; the Maritime Administration within the
Department of Transportation; and the American Association of Port
Authorities.
Figure 3: Facilities at One of the Nation's Major Ports
For fiscal year 2006, the Office of Management and Budget had proposed
consolidating the port security grant program with other homeland security
grant programs. Known as the Targeted Infrastructure Protection Program,
the program would have consolidated funding for ports, transit, and other
critical infrastructure into one program. However, DHS's appropriations
act for fiscal year 2006 maintained separate funding for the port security
grant program. In particular, the act provided a $175 million
appropriation for the port security grant program that, "shall be awarded
based on risk and threat."4
In January 2005, the DHS Office of the Inspector General issued a report
on the Port Security Grant Program that covered the program's second and
third rounds of grants-through fiscal year 2003.5 This report made a
number of recommendations, and in response, ODP initiated a number of
changes to the program, according to ODP grant program officials. Our
discussion of the grant program reflects the changes ODP had made at the
time of our report.
4Department of Homeland Security Appropriations Act, 2006, Pub. L. No.
109-90, 119 Stat. 2064, 2075 (2005).
5Department of Homeland Security, Office of Inspector General, Review of
the Port Security Grant Program, OIG-05-10 (January 2005). In some years,
more than one round of grants has been awarded. In all, five rounds of
grants were awarded between 2002 and 2005.
Goal Setting: Overall Goals Have Been Set; Developing Performance Measures and
Leveraging Federal Dollars Remain Challenges
Progress has been made on setting goals-the first phase of GAO's risk
management framework-for the port security grant program. Congress and the
Administration have laid out broad policy goals for maritime security and
for the program. Congress's stated purpose in establishing the program was
to finance the costs of enhancing facility and operational security at
critical national seaports.6 The Administration has set the program in the
context of the December 2004 Presidential Directive on Maritime Security
Policy, which cites several broad policy goals for maritime security,
including preventing terrorist attacks in the maritime domain and reducing
vulnerability to such attacks; protecting U.S. population centers,
critical infrastructure, borders, harbors, ports, and coastal approaches;
and maximizing awareness of security issues in the maritime domain in
order to respond to identified threats.7
DHS's application guidelines for fiscal year 2005 grants reflect the
context of these broad policy goals. They state that the program reflects
congressional and executive intent "to create a sustainable effort for the
protection of critical infrastructure from terrorism, especially
explosives and nonconventional threats that would cause major disruption
to commerce and significant loss of life," and they link the program to
specific national priorities specified in the nation's security planning
framework.8 Other ways in which the 2005 grant program reflects a
goal-oriented approach are its efforts to apply the grants to locations
that are viewed as the nation's highest-risk ports (discussed in more
detail below) and to focus the grants on such specific concerns as
protection against improvised explosive devices.
6The Department of Defense and Emergency Supplemental Appropriations for
Recovery from and Response to Terrorist Attacks on the United States,
2002, Pub. L. No. 107-117, 115 Stat. 2230, 2327 (2002).
7Homeland Security Presidential Directive 13 (Dec. 21, 2004).
8Office for Domestic Preparedness, U.S. Department of Homeland Security,
Fiscal Year 2005 Port Security Grant Program: Program Guidelines and
Application Kit (Washington, D.C.: 2005). The specific national priorities
cited are (1) chemical, biological, radiological, nuclear, and explosive
detection and response capabilities and (2) National Infrastructure
Protection Plan implementation.
While broad policy and program goals have been set, challenges to further
progress on this risk management phase take two main forms. The first is
translating the program's broader goals into measurable objectives. One
difficulty in doing so is that other federal partner agencies have yet to
spell out measurable objectives at the national level as related to
protecting key infrastructure. For example, as discussed in chapter 2, the
Coast Guard's Maritime Strategy for Homeland Security describes strategic
approaches and priorities, but it does not include measurable objectives
as part of its approach.9 In addition, DHS has yet to set performance
measures for programs related to the implementation of programs for
protecting critical infrastructures-another program area that the grant
program supports.10 We discuss this further in chapter 4.
A second challenge involves determining an appropriate way to consider two
different federal concerns about grant programs: ensuring that grants
address key needs while at the same time ensuring that they make the most
efficient use of federal dollars. This challenge exists for several
reasons:
o Federal and nonfederal partnership for addressing key needs.
First, the federal government is not the only potential source of
revenue for addressing security needs. Ports are often a complex
mixture of public sector and private sector infrastructure. For
example, public entities such as port authorities or local and
state governments may own or operate seaport facilities and
roadways, while private companies and interests may own and
operate factories, warehouses, oil refineries, and railways.11
Ports can produce benefits that are public in nature (such as
general economic well-being) and distinctly private in nature
(such as generating profits for a particular company). The public
benefits they produce can also be distinctly local in nature, such
as sustaining a high level of economic activity in a particular
state or metropolitan area. Thus, state and local governments,
like private companies, also have a vested interest in ensuring
that their ports can act as efficient conduits of trade and
economic activity. Given that homeland security threats can
imperil this activity, it can be argued that all of these
stakeholders should invest in the continued stability of the port.
o Leveraging federal dollars. Second, in many federal grant
programs, the desired outcome is that federal grants supplement
what other stakeholders are willing to spend. If a grant program
is not designed to encourage supplementation, the danger is that
other stakeholders will rely solely on the federal funds and
choose to use their own funds for other purposes. This practice is
known as substitution, and the net result is that limited federal
funds cannot be stretched, or leveraged, to the degree they
otherwise could be. In prior work addressing this issue in certain
other grant programs, we found that on average, every additional
federal grant dollar resulted in about 60 cents of substitution.12
9U.S. Coast Guard: Maritime Strategy for Homeland Security, (Washington,
D.C.: December 2002).
10In some ways, owners and operators of facilities and vessels have set
their own standards for what an acceptable level of risk is by the
security plans they have developed under the Maritime Transportation
Security Act of 2002. As required by the act, 46 U.S.C. 70103(c), facility
and vessel owners developed over 12,000 plans to reduce vulnerabilities
around their port areas. The Coast Guard required facility and vessel
owners to implement them by July 1, 2004.
11ODP estimates that 90 percent of facilities and vessels are owned by
industry.
Although the design of a grant program is not part of the risk management
framework, it is an important issue because it is one key to accomplishing
the dual aims of targeting funds to projects that address the highest risk
while discouraging the replacement of state, local, and private funds with
federal money. ODP's approach for 2005 has been to formalize a matching
requirement for private sector stakeholders but not for public sector
stakeholders. For fiscal year 2005 grants, ODP required that applications
from industry match at least 50 percent of the total amount requested.
This situation illustrates the complexity of addressing the most
significant security needs while considering the degree to which
nonfederal stakeholders should share in the cost of security improvements.
For fiscal year 2005, the program encourages but does not require public
sector or nonprofit entities to match federal funds, according to ODP
officials. Depending on the value placed on reducing the substitution of
federal funds for local funds, the design of the port security grant
program offers a way to improve the fiscal impact of federal dollars.
There is disagreement among policymakers about where the emphasis should
be on this aspect of grant programs. Some might see the substitution of
federal funds for local funds as reasonable given differences in fiscal
capacity, while others may view homeland security as a shared fiscal
responsibility. If policymakers place greater value on reducing the
substitution of federal funds for local funds, strengthening matching
requirements for such entities offers one option. The 2006 appropriation
for the port security grant program includes a federal matching
requirement.13 ODP has not yet issued its 2006 port security grant
guidance clarifying how it will implement this requirement. One way to
implement the requirement involves using a sliding scale for matching
federal funds depending on the fiscal capacity of the grant applicant.
Such a scale could range, for example, from an 80 percent matching
requirement for Fortune 500 companies to a 25 percent matching requirement
for those entities that have less in monetary resources.14
12GAO, Federal Grants: Design Improvements Could Help Federal Resources Go
Further, GAO-AIMD-97-7 (Washington, D.C.: Dec. 18, 1996). We include this
figure here for illustrative purposes only and are not stating that this
degree of substitution would occur in the port security grant program. For
discussion of this issue in other public/private arenas, such as federal
funding for highway investments and freight mobility projects, see GAO,
Freight Transportation: Strategies Needed to Address Planning and
Financing Limitations, GAO-04-165 (Washington, D.C.: Dec. 19, 2003);
Highway and Transit Investments: Options for Improving Information on
Projects' Benefits and Costs and Increasing Accountability for Results,
GAO-05-172 (Washington, D.C.: Jan. 24, 2005); and Freight Transportation:
Short Sea Shipping Option Shows Importance of Systematic Approach to
Public Investment Decisions, GAO-05-768 (Washington, D.C.: July 29, 2005).
Risk Assessments: The Funding Distribution Model Is Becoming Better Able to
Consider Risk, but Methodological Challenges Remain
ODP has made progress in carrying out risk assessments-the second part of
the risk management framework-but the progress made is balanced by the
additional methodological challenges that remain. ODP's progress is
reflected in changes made to the program for fiscal year 2005, in both
port-level and national-level assessments. Among other things, ODP has
placed greater emphasis on using threat, vulnerability, and consequence
data in prioritizing applications and port locations. Nonetheless, various
methodological challenges remain in the assessment tools or the
assessments themselves. For example, ODP is still limited in its ability
to compare and prioritize applications from one port with those from a
different location.
13Department of Homeland Security Appropriations Act, 2006, Pub. L. No.
109-90, 119 Stat. 2064, 2075 (2005). The appropriation incorporates by
reference the federal matching requirement of another federal grant
program contained at 46 U.S.C. 70107(c). The match provision allows
federal funding up to 75 percent of the total cost of the project, unless
the DHS Secretary determines that a proposed project merits support and
cannot be undertaken without a higher rate of federal support.
14Fortune magazine ranks the nation's largest companies on the basis of
revenue.
Procedures for the Fiscal Year 2005 Port-Level and National-Level Assessments
Have Been Strengthened
At the port level, a key difference in ODP's fiscal year 2005 grant
procedures is that input from other stakeholders plays a more prominent
role in the award decisions. ODP obtained such input in prior years but
did not formally consider it in making decisions. For example, in making
its determinations for fiscal year 2004 grants, ODP sought input from the
Coast Guard Captains of the Port and from regional directors of the
Maritime Administration. These officials reviewed and ranked port security
grant applications based on their knowledge of the port location and the
results of various assessments, including the PS-RAT. ODP officials said
they considered this input, but they did not integrate rankings into the
evaluation forms used in the ODP assessment process. In this regard, the
DHS IG found that ODP should place greater emphasis on risk reduction as
part of the grant review process.15 In responding to the IG's findings,
for fiscal year 2005, the rankings made by Coast Guard, Maritime
Administration, and state officials are part of the formula ODP uses to
make final decisions on grant awards.16 Additionally, the 2005 program
places greater emphasis on applications that are consistent with area
maritime security plan priorities.
ODP's adjustments to its fiscal year 2005 procedures are even greater at
the national level, where it has made a concerted effort to narrow the
program to ports of greatest concern, and to use threat, vulnerability,
and consequence data to rank and prioritize both ports and applications.
For grants made in fiscal year 2004, ODP considered applications from all
ports in making awards. For the 2005 program, ODP worked with IAIP and the
Coast Guard to develop a list of eligible ports. The agency first
identified the largest ports based on volume. According to ODP officials,
a risk formula was developed to rank each of the ports. In May 2005, ODP
determined that, on the basis of this assessment, 66 of the 129
largest-volume ports across the country were eligible for grant awards.17
ODP further prioritized the 66 ports by dividing them into four tiers-tier
1 representing those ports with the highest risk and tier 4 representing
ports with the lower risk. ODP provided a set amount of money to each
tier, and ports in these tiers competed against each other for funding. In
carrying out its analyses, ODP also placed greater reliance on threat,
vulnerability, and consequence information.
15Management for the fiscal year 2004 program was transferred from the
Transportation Security Agency to ODP during the middle of the process.
ODP assumed full responsibility for the program in fiscal year 2005.
16For the 2005 program, the evaluation at the port level is managed by the
Coast Guard Captain of the Port and state officials when feasible.
Applications are reviewed against established criteria and ranked on
relative risk, with the highest rank going to applications that support
the national port security priorities.
17ODP defines risk as credible threats and incidents (from the
intelligence community), less credible threats and incidents (operational
indicators), and vessels of interest. It defines vulnerability as distance
from open water, number of port calls, and presence of tankers, and it
defines consequence as people, economic, national security, and
port-specific special considerations (such as hazardous materials or oil).
Beyond reducing the number of ports eligible for the grant program,
another change ODP made at the national level was to prioritize possible
threat scenarios that grant funds should address, using input from Coast
Guard officials. ODP has given priority to applications that prevent or
detect threats arising from improvised explosive devices. Such devices
pose a threat to transportation systems across the nation and have been
used by terrorists in the past. Specifically, for the 2005 grant program,
ODP has given priority to the following threat scenarios:
o preventing and detecting improvised explosive devices delivered
via small craft,
o preventing and detecting underwater attacks from such devices,
and
o preventing and detecting vehicle-born improvised explosive
devices on ferries.
Challenges Remain for Improving Methods and Data
ODP's assessment methodology, while improved, still faces challenges.
Progress in using risk assessment data to manage the grant program has
limitations in methods and data for informing award decisions. These
challenges, if addressed, would provide decision makers with more reliable
and complete data on which to prioritize and award grant funds. The
challenges fall into two main categories: (1) incomplete threat,
vulnerability, and consequence data and (2) methodological inability to
fully compare grant applications from one port with those from another
port.
Incomplete Data on Threats, Vulnerabilities, and Consequences
Our review of ODP's risk assessment approach and our discussions with ODP
and Coast Guard personnel identified several challenges related to data on
threats, vulnerabilities, and consequences (see table 7).18 Many of these
challenges mirror the challenges faced by the Coast Guard that we
described earlier in chapter 2.
18Although ODP has primary responsibility for administering the grant
program, the Coast Guard plays a key role in the program. Local Captains
of the Port use the PS-RAT in evaluating grant applications at the local
level, and ODP officials consult frequently with Coast Guard officials at
the national level.
Table 7: Examples of Data-Related Challenges in ODP Risk Assessments
Source: GAO analysis of ODP's risk assessment for the port security grant
program.
The following examples illustrate the challenges posed by the limitations
in threat, vulnerability, and consequence data.
o Limitations in threat-related data. ODP's current
characterization of threat cannot be interpreted as an estimate of
relative probability of threat scenarios-a key element of risk
assessment-in and around ports.19 The problems we pointed out in
chapter 2 about threat data are applicable here as well: The
threat information available from intelligence agencies does not
provide the type of data that could be used to more fully inform
ODP's decisions. More complete data would include such things as
information on the presence of national or international terrorist
cells, the capability and intent of terrorist groups as they
relate to types of attack, and on the specific infrastructure
types that have been attacked. ODP officials said that the
scarcity of threat data limits their ability to inform the
decision-making process and that decisions are based on the best
available combination of data and expert judgment. Because threat
data are limited, ODP bases its models on several proxies for
risk. For example, ODP used volume of cargo that moves into and
out of ports as a way to develop its list of the 129 ports that
could initially be candidates for the grant program. ODP then
assessed the relative risk at these ports by using, among other
things, a threat variable represented by the number of "credible
threats and incidents" and the number of "vessels of interest"
(i.e., suspicious vessels) that use a port facility. These data
originated from the intelligence community, the Coast Guard, and
IAIP. ODP recognizes, however, that threat data on ports are
scarce, and data on the number of credible threats and incidents
and vessels of interest may not represent actual threats, but
instead could also represent other law enforcement problems, such
as illegal migrants or theft of goods and merchandise. The
challenges in developing reliable data on probability affect the
overall risk assessment for a port area, given a specific attack
scenario. Without data or models that measure the relative
probability of various threat scenarios, ODP may not target the
most significant security concerns.
Limitations in vulnerability-related data. As we described in
chapter 2, the Coast Guard's PS-RAT excludes from its analysis
reductions in vulnerabilities resulting from security measures
that have already been taken by the Coast Guard, such as
inspecting passenger vehicles that board ferries, escorting
high-interest vessels, and establishing security zones around the
port. ODP's assessment of vulnerabilities involves specific
aspects of individual port areas themselves, namely, data on the
length of the channel leading to a port, the number of port calls
by all ships, and the number of tankers that use a port. While
such data are representative of the intrinsic vulnerabilities of a
port's location, they do not include such factors as guards,
fences, and cameras that are already in place; security zones that
have been established; or escorts of high-interest vessels that
occur. As a result, the assessment may overstate vulnerabilities
for port locations. Also important is the fact that ODP's approach
to evaluating port area vulnerabilities excludes consideration of
specific threat scenarios. For example, while they have identified
certain threat scenarios as priorities, the vulnerability
indicators are not linked directly to scenarios, such as the use
of improvised explosive devices. Without this linkage to the
threat component of the risk assessment process, ODP's approach to
vulnerability assessments may not consider certain factors, such
as the number of areas in a port location where recreational
vessels are unmonitored, and how such factors may be conducive to
certain threat scenarios, such as loading improvised explosive
devices on such vessels.
Limitations in consequences-related data. The values used by ODP
to describe the consequences of a terrorist attack may not
accurately depict the damages resulting from a terrorist attack.
In valuing the consequences of an attack, ODP focuses on
people-related, economic-related, and national security-related
measures.20 For example, ODP uses maximum population density
within 10 miles of a port and the average number of daily ferry
passengers to estimate the consequences of a terrorist attack at a
port, and it aggregates these measures. Similarly, ODP uses
international and domestic tonnages, the amount of containerized
cargo, and the dollar value of foreign trade to measure the
economic consequence of a terrorist attack. While these provide a
useful starting point, there are two issues related to this
approach. First, without linking these data to the relative
probability of various threat scenarios, ODP's estimate of
consequences is limited to inherent characteristics of a port
rather than the estimated consequences of various attack
scenarios. In contrast, agencies such as the Coast Guard estimate
the number of lives lost and assign a dollar value to the loss of
human lives. Second, ODP aggregates the estimated number of lives
lost with the dollar value of foreign trade. Doing so raises
questions about the reliability and meaning of the final output.
19ODP relied on threat-related information provided by IAIP for the fiscal
year 2005 grant program.
Methodological Limitations
As was the case with the Coast Guard, a key methodological limitation
affects one goal of risk assessments-informing decision makers on relative
risks across port locations. This limitation relates directly to the
ability to compare the relative risks of facilities at one port with those
at another port. When field review teams rank and prioritize grant
applications, they use several sets of data, including information
provided on the application, personal knowledge of the port, and the
results of the PS-RAT, which gives ranking information for a given port
area on vulnerability and consequence.21 As discussed in chapter 2,
PS-RAT, however, cannot now be used to compare the risks at one port with
those at another. PS-RAT allows ODP to compare and prioritize key
infrastructure within a port, but it does not produce a risk ranking that
permits ODP to compare and prioritize infrastructure across ports. Coast
Guard efforts to develop a tool that examines relative risks across ports
will aid ODP in addressing this limitation.
20ODP worked with the Coast Guard and IAIP in developing the consequence
measures.
21When the port security grant program was established, according to a
program official, it was one of the first programs to use the PS-RAT in
order to identify risks. The PS-RAT has been very useful on the local
level as a means for the Coast Guard to look at relative risk to assets
within a port zone in a systematic way. In addition to being a primary
tool for the port security grant program, the PS-RAT has also been used by
members of the Area Transportation Security Committees to interact with
one another and to plan their work.
Alternatives Evaluation: ODP Recognizes the Importance of Evaluating
Alternatives, but Tools for Doing So Are Limited
Evaluation of alternatives-the third phase of GAO's risk management
framework-is an area that ODP recognizes as being an important part of
awarding grants, and the changes for fiscal year 2005 represent progress
in this area. One change for fiscal year 2005 involves additional steps to
consider benefits and costs. When ODP asks local Coast Guard Captains of
the Port to review applications, one criterion it asks them to apply is
which projects offer the highest potential for risk reduction for the
least cost. For 2005, ODP plans to augment evaluation by conducting an
analysis of costs and benefits of the project (which considers the
potential for risk reduction), and it shares the results with the Coast
Guard.
As part of this assessment, ODP plans to break out applications from the
66 eligible port locations into four tiers and give applications from
ports that are in higher tiers more priority and more money. Port areas
with the highest risk are assigned to tier 1 and port areas with the
lowest risk are assigned to tier 4.
ODP's ability to assess proposed security improvements, like the Coast
Guard's, is influenced by the program goals and performance measures that
the component sets and the reliability and completeness of the risk
assessments that it carries out. However, when measurable objectives are
missing, the degree to which security gaps remain and the extent to which
progress has been made remain unclear. Similarly, while PS-RAT provides a
starting point for evaluating the proposed measures and the extent to
which the measure narrows security gaps within a port, it was not designed
to compare and prioritize relative risks from one port to relative risks
in a different port. This condition limits ODP's ability to compare the
benefits of proposed security measures from an applicant at one port
location with benefits of proposed measures at a different port.
Management Selection: ODP Has Addressed Problems in Documenting Differences
between Initial Selection Recommendations and Final Selection Decisions
Earlier in this chapter, we discussed the importance-and difficulty-of
balancing potentially conflicting goals, such as ensuring that funds are
directed to projects of the greatest risk while at the same time
stretching limited federal dollars to the maximum degree. ODP's selection
of grants for 2004 illustrates the challenges in applying a risk-based
approach to awarding grants and the trade-offs involved in attempting to
balance risk and financial need as criteria. In order to achieve
accountability, federal standards for internal controls require that all
transactions and other major events need to be documented. Basically,
about 40 percent of ODP's final selection decisions were different from
the initial recommendations of lower-level evaluators, without
documentation from reviewers explaining why they disagreed with the
initial recommendations. According to officials involved in the program,
before the final selections were made, the Secretary of Homeland Security
issued guidance indicating that Fortune 500 companies should be able to
pay for their own security improvements and that ferries and port
authorities should receive higher priority in the final award decisions
than other applicants. ODP officials said that the fact that they followed
this new guidance affected the final ranking of grant applicants. The
tension between self-funding and security priorities illustrates the need
for effective internal controls to ensure that procedures are followed and
that the resulting selection decisions are transparent and clearly
documented. ODP has taken action to address this problem.
In the 2004 grant award process, ODP's initial assessments resulted in
recommendations for funding 154 specific proposals.22 However, our
analysis of the final grant awards showed that about 40 percent of the
initial 154 applications that were recommended for a grant award by
lower-level evaluators that examined grant applications did not receive an
award. Table 8 shows examples of the types of changes that occurred.
22ODP worked with TSA staff during the assessment process.
Table 8: Examples of Changes in Funding Decisions
Source: GAO analysis of 2004 grant fund database.
These examples illustrate the trade-off between awarding grants to
applicants that are assessed, in part, based on risk or on providing funds
to applicants that have a financial need. The net result is that when
federal funds are used in this fashion, they may not address the most
severe security gaps in and around ports because there is no guarantee
that private sector firms will spend their own funds for security
improvements since it is unclear whether there are incentives, such as
minimum standards for security, that would motivate them to do so.23 The
competing goals of addressing the most significant security needs and
providing financial assistance to those entities with less fiscal capacity
call attention to the difficulty of achieving a balance between these
objectives and the need for selection decisions that clearly document the
trade-off.
In order to achieve a transparent process for accountability purposes,
federal standards for internal control require that all transactions and
other significant events need to be clearly documented, and that
documentation should be readily available for examination.24 However,
internal control procedures for documenting decisions, including changes
in project ranking, were not followed. The result is that the rationale
for award decisions was not always available. In cases where an
applicant's ranking would change by over 100 places, there was no
documentation that described the reason for the change. In several cases
where changes such as this occurred, it was noted that the final selection
board concurred with the original ranking, but no reason was provided for
the reprioritization. DHS's IG review also found this lack of
documentation. The IG recommended that the reviewers be required to
document their decisions in the grants management system, particularly
when the decisions are inconsistent with recommendations from a lower
level of review. DHS generally concurred with the IG's recommendations and
stated that it would require reviewers to document their decisions as part
of the 2005 grant program. Our work showed that in the fiscal year 2004
grant program-the most recent round prior to the issuance of the IG's
report-such documentation was still missing. For fiscal year 2005, ODP
instituted additional measures to ensure that decisions were documented as
part of the review process.
23MTSA required owners and operators of facilities and vessels to develop
and implement security plans by July 1, 2004. In all, about 12,300 plans
were developed. The Coast Guard is charged with approving the plans and
ensuring owners and operators are complying with the actions called for
and determining whether all vulnerabilities have been identified. Coast
Guard compliance inspections were scheduled for completion by July 2005.
Funds requested by owners and operators may reflect additional security
measures taken that go beyond the minimum requirements called for in the
security plans or they could involve additional technology improvements.
24GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21 .3.1 (Washington, D.C.: November 1999).
Implementation and Monitoring: ODP Has Instituted a Monitoring Plan for Grant
Awards
ODP has made substantial progress in creating procedures that address the
fifth phase of the framework-implementation and monitoring; the challenge
lies in carrying them out. As we have described in this chapter, ODP has
taken a number of actions to base spending decisions on risk-based data;
it has, among other things, (1) developed procedures for prioritizing port
locations, (2) prioritized threat scenarios, and (3) more closely aligned
risk assessments to port locations. In addition, ODP has coordinated its
efforts with the Coast Guard, the intelligence community, and other key
stakeholders in developing the procedures for the 2005 grant program. For
the fiscal year 2005 grant program, ODP has established procedures for
monitoring recipients after they have been funded. The monitoring consists
of follow-up after a project has been implemented to help ensure that the
project has been implemented in accordance with the grant award, including
timelines, budgets, and programmatic criteria are being met. In addition,
ODP requires that progress reports be submitted biannually and a final
project report be submitted within 120 days after the end of the project.
A challenge that remains this phase of the risk management framework is
developing processes for feedback to improve the process used in awarding
grants. For example, our review of the 2004 grant awards show that local
officials, including Coast Guard officials and grant applicants, did not
receive feedback from ODP on why projects they designated as high priority
did not receive funding when lower-priority projects did. According to ODP
officials, they have instituted processes for providing feedback to and
obtaining feedback from grant program stakeholders as part of the fiscal
year 2005 program.
Conclusions
The principles of risk management apply to the port security grant
program, and the lessons learned call attention to additional actions that
could build on the progress that has already been made. Depending on the
value placed on reducing the substitution of federal funds for local
funds, there are opportunities to further leverage federal dollars in the
program's design, and by leveraging federal funds, additional security
needs in and around ports could be addressed. Additionally, without having
performance measures in place, it is hard to gauge what progress has been
made and what security gaps remain. The development of such measures
should offer greater insights on the extent to which funds are narrowing
security gaps that exist or helping to identify security needs that
surface. The lessons of the program also provide numerous insights into
the way that multiple stakeholders, such as the Coast Guard and IAIP,
contribute to the way in which ODP uses risk-based data in administering
the program. As we described in chapter 2, the Coast Guard has several
efforts under way to improve its risk assessment of ports, including
developing data on the relative probability of threat scenarios, improving
data on consequences and vulnerabilities that are linked to various threat
scenarios, and comparing risks among ports. ODP should be able to use the
results of such efforts to help in awarding grants that are consistent
with the priorities identified.
Recommendations for Executive Action
To strengthen ODP efforts to implement a risk management approach to its
port security grant program, we recommend that the Secretary of Homeland
Security direct the Executive Director for ODP to undertake the following
three actions:
o Clarify, in its grant guidance, the conditions under which
greater leveraging of federal dollars should be included as a
strategic goal for the port security grant program.
o Develop measurable objectives for managing the grant program's
progress toward achieving strategic goals and use these measures
to gauge progress and make adjustments to the program.
o Coordinate efforts with the Coast Guard and IAIP to use more
reliable risk assessment data as they become available. At a
minimum, such data should include (1) the relative likelihood of
various threat scenarios, (2) consequences and vulnerabilities
that are linked to terrorist scenarios, and (3) a comparison of
risks across ports.
Agency Comments and Our Evaluation
In commenting on a draft of chapter 3, DHS, including ODP, generally
agreed with the findings and recommendations. Specifically, DHS said that
the recommendations are reasonable given that most of our review took
place prior to changes that ODP made to the program in fiscal year 2005.
DHS said that it appreciated our efforts to review the fiscal year 2005
port security grant program requirements even though most of our work had
been completed. DHS stated that several of the recommendations have
already been addressed, and it noted that the remaining ODP-related
recommendations will be addressed in the fiscal year 2006 port security
grant program, at least to the extent possible. In addition to commenting
on our findings and recommendations, DHS provided technical comments under
separate cover, and we revised the draft report where appropriate. Written
comments from DHS are in appendix II.
Chapter 4: IAIP Faces Challenges in Meeting Risk Management
Responsibilities across All Sectors of the Nation's Infrastructure Chapter
4: IAIP Faces Challenges in Meeting Risk Management Responsibilities
across All Sectors of the Nation's Infrastructure
The Information Analysis and Infrastructure Protection Directorate within
DHS faces broad and extensive challenges in meeting its risk management
responsibilities and thus far has made limited progress. Relative to the
Coast Guard and ODP, IAIP's risk management responsibilities are much
wider and more difficult: Instead of comparing risks across port assets,
it must find ways to compare risks at ports with risks in other sectors,
such as public health, energy, and banking and finance. Challenges remain
across all of the phases of GAO's risk management framework (see table 9).
For the first phase (goals and objectives), while IAIP's efforts are
anchored to strategic goals in various executive branch strategies and an
interim national plan, IAIP's challenge is to continue developing the
national plan to provide performance measures and associated steps and
milestones. In the second phase, IAIP has begun several key risk
assessment efforts but has had limited success in completing them. For
example, IAIP faces challenges in developing data on the relative
likelihood of various threat scenarios-a key part of the assessments it
must conduct under the Homeland Security Act of 2002-because the
information produced by the intelligence community is of limited use for
risk assessment purposes, according to IAIP officials. IAIP has plans to
develop such data by coordinating its efforts more closely with the
intelligence community. Additionally, IAIP has yet to successfully
complete the difficult task of comparing and prioritizing assets within
and across sectors, but it plans to have an interim assessment done by the
end of 2006. IAIP's challenge in the final three phases (evaluating
alternatives, selecting approaches, and implementation and monitoring) are
related heavily to IAIP's unique role: It recommends what the security
priorities should be to other federal agencies and nonfederal stakeholders
and recommends how best to address them, but it is largely dependent on
other stakeholders, public or private, to take any actions. The decision
to implement security improvements is made by stakeholders alone.
Moreover, IAIP acknowledges that it can further leverage work that has
already been done in this area by other federal agencies that have
regulatory authorities over certain private sector infrastructure owners.
IAIP's challenges center on developing credible guidelines and approaches
that can leverage work already done and foster concurrence in risk
analysis results and encourage actions to be taken.
Table 9: Summary of Progress Made and Challenges That Remain in IAIP's
Risk Management Approach
Source: GAO analysis of IAIP's risk management practices.
IAIP Plays a Key Role in Evaluating Risk across Infrastructure Sectors
While risk management is one of several tools for the Coast Guard and ODP,
risk management is central to one of IAIP's key missions, which is to
establish a risk management framework across the federal government to
protect the nation's critical infrastructure and key resources.1 The
Homeland Security Act of 2002 made IAIP responsible for critical
infrastructure protection (CIP) functions, charging IAIP with broad
responsibility for developing a comprehensive national plan for securing
the key resources and critical infrastructure of the United States. IAIP's
statutory responsibilities require it to conduct risk management
activities on a national scale and to gather the information needed to do
so from other federal agencies, state and local government agencies, and
private sector entities.2 By statute, IAIP is responsible for
o identifying, detecting, and understanding threats in light of
actual and potential vulnerabilities to the homeland;
o conducting comprehensive assessments of the vulnerabilities of
the key resources and critical infrastructure of the United
States;
o conducting risk assessments to determine the risks posed by
particular types of terrorist attacks and how likely they are to
succeed, as well as the feasibility and potential efficacy of
various countermeasures;
o identifying priorities for protective and support measure by
DHS, other federal agencies, state and local governments, the
private sector, and other entities; and
o recommending measures to protect the critical infrastructure
and key resources of the United States in coordination with other
federal agencies and in cooperation with state and local
governments, the private sector, and other entities.3
In December 2003, the President issued Homeland Security Presidential
Directive-7, Critical Infrastructure Identification, Prioritization, and
Protection, which established the framework in which IAIP carries out its
responsibility of coordinating the overall national CIP effort. Current
CIP policy, as described in HSPD-7, defines responsibilities for DHS,
sector-specific agencies, and other departments and agencies. It instructs
federal departments and agencies to identify, prioritize, and coordinate
the protection of critical infrastructure to prevent and deter attacks,
and mitigate the effects of any attacks that may occur.
1The Homeland Security Act incorporates the definition of "critical
infrastructure" used in the USA PATRIOT Act of 2001, meaning "systems and
assets, whether physical or virtual, so vital to the United States that
the incapacity or destruction of such systems and assets would have a
debilitating impact on security, national economic security, national
public health or safety, or any combination of those matters." The
Homeland Security Act defines "key resources" as "publicly or privately
controlled resources essential to the minimal operations of the economy
and government." 6 U.S.C. S: 101.
2See generally 6 U.S.C. 121(d).
3Id. 121(d)(1), (2), (3), (6).
To ensure the coverage of critical sectors, HSPD-7 designated
sector-specific agencies for the critical infrastructure sectors
identified in the National Strategy for Homeland Security (see table 10).
These agencies are responsible for infrastructure protection activities in
their assigned sectors, which include coordinating and collaborating with
relevant federal agencies, state and local governments, and the private
sector to carry out their responsibilities and facilitating the sharing of
information about physical and cyber threats, vulnerabilities, incidents,
potential protective measures, and best practices. For example, the
transportation sector, for which the Department of Homeland Security is
assigned responsibility, includes the movement of people and assets that
are vital to the nation's economy, mobility, and security. The maritime
shipping infrastructure, a component of the transportation sector,
includes ports and their associated assets, ships, passenger
transportation systems, and other maritime transportation assets. Each
sector may also include a number of systems and "key assets"-some of which
include individual targets whose attack could cause large-scale human
casualties or property destruction, or profoundly damage national prestige
and confidence.4
4The National Strategy for the Physical Protection of Critical
Infrastructures and Key Assets identifies five key assets. The key assets
and lead federal agencies are as follows: national monuments and icons
(Department of the Interior); dams, locks, and levees (Department of
Homeland Security); government facilities (Department of Homeland
Security); commercial and community assets (Department of Homeland
Security); and nuclear reactors, materials, and spent fuel (Department of
Homeland Security, working with the Nuclear Regulatory Commission and, as
appropriate, the Department of Energy). When we use the term "critical
infrastructure" in this chapter, we are referring both to the 13 sectors
and to the five key assets.
Table 10: Critical Infrastructure Sectors and Lead Federal Agencies
Source: GAO analysis of the President's national strategy documents and
HSPD-7.
Under HSPD-7, the overall national CIP effort is to be coordinated by DHS,
a responsibility carried out by IAIP, subject to the DHS Secretary's
direction and control, as provided in the Homeland Security Act of 2002.
The DHS Secretary has several overarching CIP responsibilities under
HSPD-7, including identifying, prioritizing, and coordinating CIP within
and across sectors, with an emphasis on critical infrastructure and key
resources that could be exploited to cause catastrophic health effects or
mass casualties. In addition, the Secretary is required to establish
uniform policies, approaches, guidelines, and methodologies for
integrating CIP and risk management activities within and across sectors,
along with metrics and criteria for related programs and activities.
Strategic Goals, Objectives, and Constraints: High-Level Goals in Place and
Interim Plan Drafted, but Performance Measures and Milestones Have Yet to Be
Developed
A number of strategic goals are in place to guide IAIP's broad
responsibilities. They stem from the Homeland Security Act of 2002 and the
following executive branch documents:
o The National Strategy for the Physical Protection of Critical
Infrastructures and Key Assets.5 Issued in February 2003, this
strategy identified a set of national goals and objectives and
outlined the guiding principles that underpin efforts to secure
the critical infrastructure and key resources of the United
States. The strategy recognizes that adequate protection of
critical infrastructure requires (1) comprehensive threat
assessment and analysis; (2) effective and efficient risk
assessment; and (3) security baselines, standards, and guidelines.
o Homeland Security Presidential Directives 7 and 8. Both
directives were issued on December 17, 2003. HSPD-7 established a
national policy for federal departments and agencies to enhance
the protection of critical infrastructure and key resources and
made DHS responsible for establishing uniform policies,
approaches, guidelines, and methodologies for integrating
nationwide infrastructure protection and risk management
activities. HSPD-8 calls for a national preparedness goal that
balances the potential threat and magnitude of terrorist attacks
with the resources required to prevent, respond to, and recover
from them-a risk management approach calling for an estimate of
the likelihoods and expected consequence of possible terrorist
attacks that takes finite resources into account.
o The Interim National Infrastructure Protection Plan (interim
NIPP). The Secretary of DHS assigned IAIP the responsibility for
developing a national infrastructure protection plan. The interim
NIPP was released in February 2005.6 It calls for the use of a
risk management framework that takes into account threats,
vulnerabilities, and consequences when comparing and prioritizing
critical infrastructure and deciding what actions to take to
protect them. This framework is intended to be carried out both
within sectors and nationally across sectors. It is to contain
steps for narrowing the overall set of assets to those that are
most critical at a national level.
5The National Strategy for the Physical Protection of Critical
Infrastructures and Key Assets is complemented by the National Strategy
to Secure Cyberspace. Also issued in February 2003, the National Strategy
to Secure Cyberspace provides direction to the federal government
departments and agencies that have roles in cyberspace security and
identifies steps that state and local governments, private companies and
organizations, and individuals can take to improve collective
cybersecurity.
6In November 2005, DHS issued a Draft National Infrastructure Protection
Plan for comment. We did not evaluate the draft plan because our field
work had already been completed.
We have reported that the interim NIPP is not a comprehensive document,
and IAIP faces several challenges in making it more comprehensive.7 These
challenges are in two main areas:
o Performance measures. Although the interim NIPP did not contain
performance metrics to measure effectiveness, it recognized that
they are needed and calls on IAIP and the sector-specific agencies
to develop them. According to IAIP officials, this is being done
in two phases. For the first phase, IAIP has identified a set of
basic core metrics that can be used to evaluate performance across
all sectors, as called for in the interim NIPP. IAIP is also
working with agencies responsible for specific sectors to develop
a supplemental set of metrics for each sector. The intent of this
measurement process is to provide DHS and the sector-specific
agencies with feedback on where and how they should focus their
resources to be most effective. According to IAIP officials, the
second phase involves, in part, monitoring the progress of each
sector in implementing the risk management framework laid out in
the interim NIPP. To date, however, IAIP and the sector-specific
agencies have not completed the performance metrics called for in
the interim NIPP.
o Milestones and timelines. The interim NIPP did not contain
milestones for the development of sector-specific plans or
timelines of target dates for identifying and prioritizing
critical infrastructure. According to IAIP officials, the final
version of the NIPP, after undergoing interagency review, will be
released in 2006, and it will contain milestones and timelines for
the initial phase of developing performance metrics. It is not
clear, however, if this final version will contain milestones and
timelines for sector-specific plans or for completing the process
of prioritizing critical infrastructure.
7GAO, Critical Infrastructure Protection: Challenges in Addressing
Cybersecurity, GAO-05-827T (Washington, D.C.: July 19, 2005).
Risk Assessment: IAIP Faces Substantial Gaps and Has Made Limited Progress
IAIP's progress in risk assessment has not been as extensive as the Coast
Guard's or ODP's. Its area of greatest progress is in developing a
national database of assets that constitutes the nation's critical
infrastructure. In two other key respects, however, it faces major
challenges in carrying out requirements specified in law or policy
directives. These challenges are in developing adequate data on threats
and creating a methodology for making cross-sector comparisons.
Progress Is Greatest on Database of Critical Assets
IAIP has developed the National Assets Database (NADB), an inventory of
approximately 80,000 assets, as a starting point in being able to evaluate
and prioritize them. The NADB includes such facilities as nuclear power
plants, pipelines, bridges, stadiums, and locations such as Times Square.
Assets in the NADB are gathered from a variety of public and private
sector sources, including federal, state, and local databases; prior
studies containing lists of infrastructure and resources; and
sector-specific data collection activities. The database is revised as
assets are added and removed in collaboration with state and local
officials and with representatives of sector-specific agencies. According
to IAIP officials, this database is intended to produce baseline data that
will later allow for assessments of vulnerabilities by location, within
sectors, and across sectors.8
Development of Threat Data Faces Challenges
IAIP has begun work to develop threat scenarios and analyze them. The
Homeland Infrastructure Threat and Risk Analysis Center (HITRAC), staffed
by sector specialists and intelligence analysts with backgrounds from the
intelligence community, is responsible for generating these plausible
threat scenarios-called benchmark threats. HITRAC has developed 16
benchmark threats, such as a suicide bomber, a vehicle-borne improvised
explosive device, and a weapon of mass destruction. IAIP faces two
substantial challenges, however, in completing this work.
o Relative probability for threat scenarios not yet developed.
First, IAIP faces challenges in developing a way to differentiate
the relative probability of various threats. Under the Homeland
Security Act of 2002, IAIP must perform risk assessments to
determine the risks posed by particular types of terrorist
attacks, including an assessment of the probability of success of
such attacks.9 IAIP officials stated that a lack of intelligence
data and law enforcement data limits their ability to develop the
relative probability for various threat scenarios, and for this
reason, they have focused their initial efforts on developing
vulnerability and consequence data. According to IAIP officials,
the intelligence community-including the intelligence components
of DHS-has been unable to provide detailed intelligence on threats
to most sectors, infrastructure, assets, or asset types.10
Assigning equal likelihood to various threat scenarios would mean
IAIP's risk assessments will not include key threat data on the
capabilities and intent of terrorist groups, the history of
terrorist threats on various asset types, or the existence of
terrorist cells domestically or internationally. And because data
on the relative likelihood of threat scenarios are not included,
the assessments will emphasize high-consequence events that may
have a low probability of occurring. This approach is bound to
result in potentially unreliable or incomplete data on where to
establish priorities. In September 2005, IAIP officials told us
that they recognize the importance of developing data on the
relative likelihood of an attack type, and that doing so is part
of their responsibility for meeting the requirements of the
Homeland Security Act. Officials told us that they plan to assess
the likelihood of threats by having HITRAC develop consistent
comparative threat assessments by integrating intelligence
sources, law enforcement data, and suspicious activity reporting
with subject matter expertise. IAIP officials caution, however,
that the directorate may not be able to estimate the relative
likelihood of some threat scenarios, and as a result, some
assessments may emphasize high-consequence events that have a low
probability of occurring. Also, officials indicated that some
inaccuracy is to be expected when HITRAC examines the intent and
capability of an adversary whose plans are concealed and that it
will be important to reduce the potential of low-confidence
assessments having undue influence when long-term investment
decisions are made.
o Vulnerability assessments not yet developed for many
infrastructure sectors. Second, IAIP has not yet developed
vulnerability assessments for the full spectrum of infrastructure
sectors. As of August 2005, IAIP had managed the development of
vulnerability assessment questionnaires for two components of
critical infrastructure-nuclear facilities and chemical
facilities. Initially, IAIP's contractor was scheduled to assess
asset types in 8 of the 18 sectors and key assets by the end of
2005. However, according to IAIP officials this work will be done
by the end of 2006.11 IAIP officials did not have an estimate as
to when the assessments would be complete for all other sectors of
critical infrastructure that it is responsible for assessing.12
8GAO did not analyze the data in the database and cannot comment on the
database's reliability. The Congressional Research Service has raised a
number of issues concerning the data integrity of the IAIP asset database.
See Congressional Research Service, Risk Management and Critical
Infrastructure Protection: Assessing, Integrating, and Managing Threats,
Vulnerabilities, and Consequences (Washington, D.C.: February 2005).
9See 6 U.S.C.121(d)(2). IAIP considers the combination of the threat
(relative likelihood of the attack type) and vulnerability (likelihood of
the adversary's success) to represent the relative likelihood of a
successful attack occurring.
10While the intelligence community does rank threats, IAIP does not find
such information useful in developing data on threats. Intelligence
components of DHS and other intelligence agencies routinely generate
strategic threat assessment matrices and reports detailing the most likely
targets and modes of terrorist attack. These assessments break down
infrastructure and assets into categories and rank them, numerically or
otherwise, according to the relative likelihood of attack within and
across categories. However, IAIP officials said that the information was
not specific to individual assets.
Methodology for Cross-Sector Comparisons Has Experienced Setbacks
IAIP has experienced setbacks in its attempts to meet HSPD-7's requirement
to develop a strategy for identifying, prioritizing, and coordinating the
protection of critical infrastructure. To prioritize the protection of
this infrastructure, IAIP has been working for about 2 years on a
methodology for assessing vulnerabilities of critical infrastructure to
help inform comparisons of risks on an intrasector and cross-sector basis.
This methodology has been delayed because of methodological concerns, and
IAIP's schedule for completing various other activities needed to meet the
requirement is dependent on the methodology and now appears uncertain.
IAIP developed an analytical assessment tool known as the Risk Analysis
and Management for Critical Assets Protection methodology (RAMCAP). RAMCAP
was begun in November 2003, when DHS awarded the American Society of
Mechanical Engineers (ASME) a $1.6-million grant to develop an overarching
methodological guide for the private sector to assess its terrorism
security risks. Originally, DHS expected RAMCAP to advance homeland
security efforts by providing a usable, affordable vulnerability and risk
assessment methodology for owners and operators to use that would inform
risk management decision in the private sector. Now, IAIP views it as an
independent effort that will complement IAIP risk assessment efforts at
comparing risks within and across sectors.
11After grouping all components of critical infrastructure by sector, IAIP
plans to ask the owners or operators to complete a questionnaire-called a
"top screen"-that provides an assessment of consequential losses arising
from a worst-case scenario that assumes total loss of the asset. Once this
information is gathered and assessed, assets considered to be of
sufficient consequence are evaluated on vulnerability, using three
elements: potential method of attack, probability of success, and
consequences of the attack (including secondary and tertiary effects). By
applying this information to the selected subset of assets, IAIP seeks to
identify the assets of greatest vulnerability and identify strategies that
hold the greatest potential benefits. Much like the consequence
assessment, the vulnerability assessment is carried out by eliciting the
opinions of experts, usually the owner/operator, for a specific piece of
infrastructure.
12IAIP has plans to address the risks involved in protecting additional
critical infrastructures, such as stadiums, though a tool called the
Vulnerability, Identification, and Self-Assessment Tool. This tool will
allow owners and operators to assess vulnerabilities in and around their
facilities.
When RAMCAP was released for comment in April 2004, ASME received comments
from over 100 officials from industry, academia, and government. A peer
review process produced additional comments. The comments centered on
several issues, including concern about the amount of resources needed to
assess risks and the limited benefits in creating self-assessments for
industry without knowing what the purpose of the effort involved.13 In
June 2004, ASME altered its approach based on this feedback and began a
broad outreach effort to involve industry in developing vulnerability
assessment modules.
In December 2004, IAIP awarded a $4 million contract to ASME to continue
its efforts in developing RAMCAP and vulnerability assessment modules for
owners and operators of asset types in eight sectors.14 After developing
pilot modules for the chemical and nuclear industries, in August 2005,
ASME produced a new working draft of RAMCAP. According to IAIP, RAMCAP may
undergo minor revisions as more modules are completed. IAIP officials
stated that more substantial revisions to RAMCAP will likely occur when
IAIP broadens its approach to infrastructure protection by examining the
entire systems and interdependencies within and across sectors.
In September 2005, IAIP informed us that it is developing a National
Comparative Risk Assessment for the 18 critical infrastructure sectors and
key assets, and it plans to complete an interim assessment by the end of
2006. The assessment has several phases. It involves sector-specific
agencies identifying the top 100 high-risk systems and assets in their
sector based on potential consequences. IAIP officials said it requested
agencies to develop their lists by July 2005. However, the degree to which
agencies will be able to fully respond to the request is uncertain. For
example, as of July 2005, the Transportation Security Agency had begun but
not yet completed a risk assessment for the passenger rail sector, and it
did not indicate when the assessment would be done. Additionally, TSA did
not expect to have the first version of its sector-specific plan-the plan
that describes its risk assessment methodology-until February 2006. Until
agencies such as TSA complete their assessments and develop their
sector-specific plans, they will not be able to determine relative risks
within their sector in a consistent fashion.15
13IAIP and the private sector agreed that the goal of the self-assessments
was to satisfy the information needs of DHS.
14The eight categories of assets are commercial nuclear power plants,
commercial spent nuclear fuel facilities, chemical plants, petroleum
refineries, liquefied natural gas storage facilities, subway systems
(including bridges and tunnels), railroad systems (including bridges and
tunnels), and highway systems (including bridges and tunnels). The
vulnerability assessments for the chemical sector and nuclear power plants
were pilot projects.
Whether IAIP will be able to complete this interim assessment by the end
of 2006 may be complicated by two other factors. First, in September 2005,
IAIP officials said that IAIP still needs to award the contract for this
effort, and officials did not provide a schedule for when this would
occur. Second, the degree and extent to which IAIP will need to obtain
input from the private sector in developing this assessment tool is
unclear. In developing the vulnerability assessment modules for
sector-specific industries, IAIP's consultant and industry groups worked
extensively with each other in developing the modules. If similar
coordination and communication efforts are needed to complete the interim
assessment, it may call for additional time and resources.
According to IAIP officials, once the initial rankings are made, subject
matter experts will review the results and the results of previous
analyses will be used to refine the rankings within a sector and
ultimately across sectors. IAIP plans to use the interim results to inform
homeland security grant programs and serve as a basis for further risk
management efforts. The results of the vulnerability assessment modules
that are being developed will inform the national assessment and will act
as a basis for interaction with the private sector, according to IAIP.
The interim National Comparative Risk Assessment is intended to meet the
immediate need of examining relative risk within and across sectors.
However, there are limitations on the degree to which the interim
assessment will produce comparable, consistent, and reliable data for
setting national priorities. First, only vulnerability assessments of
asset types in 8 of 18 sectors and key assets are scheduled for completion
by the end of 2006, while the schedule for completing assessments on other
sectors or asset types is unclear-suggesting that vulnerability data on
some sectors will be more reliable than data on other sectors. Second, the
interim assessment will likely involve a heavy emphasis on
consequence-related data, and less information on the relative likelihood
of threat scenarios and data on vulnerability-two major components of risk
assessment. Third, as sector-specific agencies develop their list of
high-risk assets, it is unclear whether they will do so in a consistent
and uniform fashion, because the overarching framework that guides these
actions is not yet in place. In September 2005, we reported that
completing the element of the framework that defines concepts,
terminology, and metrics for assessing risk limits DHS's ability to
compare risk across sectors.16 IAIP recognizes that engaging
sector-specific agencies in assessing risk in a consistent way is key.
IAIP now intends the National Comparative Risk Assessment to provide such
guidance, but until this assessment is issued, it is unclear where the
guidance will come from. Limitations such as these, according to IAIP, are
driven to some degree by resource and time constraints.
15GAO, Passenger Rail Security: Enhanced Federal Leadership Needed to
Prioritize and Guide Security Efforts, GAO-05-851 (Washington D.C.: Sept.
9, 2005).
Evaluating Alternatives: Lack of Guidance and Consensus on Costs and Benefits
Creates Challenges
The alternatives evaluated by IAIP differ somewhat from the alternatives
evaluated by the Coast Guard (as discussed in chapter 2) or ODP (as
discussed in chapter 3). The Coast Guard evaluates alternatives related
directly to how it deploys its own resources, such as increasing security
patrols, boarding suspicious vessels, creating and enforcing security
zones, and using its regulatory powers to force maritime facilities to
adopt protective measures. ODP evaluates alternatives that relate directly
to its grant awards; the alternatives are the various funding proposals
for security-related infrastructure improvements. In contrast, the
alternatives that IAIP evaluates are generally not based on actions IAIP
can take. Instead, they are based on identifying, from a national
standpoint, the (1) areas of greatest risk and (2) infrastructure
protection strategies that offer the greatest benefit for the cost
involved. In this regard, IAIP faces several challenges.
16 GAO-05-851 .
One challenge faced by IAIP in evaluating alternatives is the difficulty
in valuing costs and benefits in a homeland security setting-an important
tool for evaluating alternatives. OMB has guidance for federal agencies on
how to evaluate alternative government actions, but this guidance is of
limited use in assessing alternatives for risk management for homeland
security programs. OMB has identified various tools it considers useful in
planning, such as cost-benefit analysis, capital budgeting, and regulatory
decision making.17 However, such tools are difficult to apply to homeland
security expenditures, even when such application is encouraged in the
National Strategy for Homeland Security, because the benefits of homeland
security investments are hard to quantify. Because OMB guidance is
relatively silent on acceptable treatment of nonquantifiable benefits,
there is a lack of criteria to guide federal analysts in conducting risk
management. In response to our inquiries, OMB officials told us that they
have not been involved in implementing Homeland Security Presidential
Directive-7, which is related to critical infrastructure protection. In
addition, they said that they have not been developing, nor did they have
plans to start developing, guidance on risk management methodologies for
federal agencies to use for homeland security programs. They said that
they would rely on DHS and IAIP to take the federal government lead in
developing such methodologies. IAIP officials said they plan to develop
procedures in fiscal year 2006 for quantifying costs and benefits of
mitigation strategies.
Another challenge faced by IAIP in evaluating alternatives, at least in
terms of ranking assets and protective actions to prevent and mitigate
attacks, is that other entities responsible for taking such protective
actions may use different criteria for evaluation. That is, the federal,
state, local, or private sector entities that own and operate much of the
nation's critical infrastructure may disagree with IAIP on how to evaluate
alternatives through assessing benefits and costs or other types of
evaluation. This lack of consensus could lead to two separate evaluations
of alternatives-one by IAIP and one by the entity that owns and operates
the asset. While IAIP may view certain assets or protective actions as
critical, those responsible for the assets and protective actions may view
the assets and actions as marginal or not necessary at all, or vice versa.
According to IAIP officials, state and local government and industry
stakeholders will benefit from using the same assessments, but the value
that is placed on the assessment may differ from one stakeholder to
another. As an example, Washington state emergency management officials
told us that their initial listing and ranking of critical assets was much
different than that developed by IAIP. State officials may place greater
weight on attack scenarios that result in impacts on children or
disadvantaged communities. Industry may place greater weight on scenarios
that disrupt the long-term viability of a business. IAIP's challenge is
discerning whether federal risk concerns are managed appropriately and
that the costs for managing risks are assigned as much as possible to the
authority that benefits from the activity. In addition to these
considerations, Congress also has a role in appropriating federal funding
of protective programs.
17OMB Circulars A-11 and A-94.
Management Selection: IAIP Challenged in Protecting Infrastructure because of
Its Limited Role in Selecting Alternative Protective Measures
As with the alternatives evaluation part of GAO's framework, IAIP's
management selection differs from how the Coast Guard or ODP makes
decisions. While the Coast Guard decides what protective measures to take
and ODP decides which applicants receive funding for port security, IAIP
is not in a position to direct others on how to act. IAIP officials said
that while they have the responsibility of helping set risk-based
priorities concerning where resources should be spent for protecting
critical infrastructure, IAIP does not have the authority to direct the
management of these resources in many cases. For example, IAIP does not
have authority to compel owners and operators of critical infrastructure
to take action. Instead, IAIP recommends the relative priority of critical
infrastructure and specific protective measures. Other entities, such as
owners, operators, or agencies with more direct regulatory
responsibilities, can then act on IAIP recommendations and technical
advice. The one exception is IAIP's Buffer Zone Protection Program, which
we discuss in the section following on implementation and monitoring.
At the departmental level, DHS may use the IAIP priority list to direct
one of its component agencies to take specific actions. As examples, IAIP
priorities could be used by U.S. Customs and Border Protection to increase
the scrutiny of cargo containers at a specific port terminal, by the
Secret Service to increase security for the President, by the Federal
Emergency Management Agency to help improve local response capabilities
near specific facilities, or by ODP to provide grants to specific
facilities. In some cases, DHS components may have some authority over
private infrastructure owners-such as the Coast Guard's regulatory powers
related to implementing the Maritime Transportation Security Act. For
example, the Coast Guard approves vulnerability assessments and mitigation
strategies developed by owners and operators of facilities and vessels and
it reviews whether the owners and operators are complying with their
plans. However, in most cases, DHS and its components do not have
authority over the owners and operators of critical infrastructure,
particularly in the private sector.
Thus, the challenge for IAIP (and DHS at the department level) is that it
generally does not have authority over the entities that make management
decisions to select among alternative protective measures. In such cases,
IAIP's role is limited to providing information on how it views the
relative strengths and weaknesses of the alternatives and, sometimes,
technical advice on how these entities could improve the security of their
assets. IAIP officials said they use their expertise and powers of
persuasion to get the owners and operators to take specific protective
actions. IAIP officials also said that IAIP works closely with industry
groups to set standards and promote voluntary compliance.
Certain protective measures could have application across multiple threat
scenarios. The interim NIPP does not describe what IAIP's plans are for
analyzing its benchmark threat scenarios and developing information that
could help owners and operators reduce risk by protecting their facilities
with countermeasures that address multiple scenarios. For example,
improving security operations could reduce the risk of multiple threats,
such as suicide bombers, truck bombs, or weapons of mass destruction.
Having sufficient emergency supplies could address the consequences of
casualties or damaged infrastructure that occur from various attack modes.
By not having plans to develop information on what countermeasures could
address multiple threat scenarios, IAIP is limited in it ability to
provide information that informs owners and operators of facilities of
ways to protect their facilities in a cost-effective fashion. According to
IAIP, it is considering this in the cost-benefit framework that is being
developed.
IAIP's lack of authority over the owners and operators of critical
infrastructure highlights the importance of coordination among different
federal agencies. Several federal agencies with lead responsibilities for
specific sectors of infrastructure do have regulatory authorities that
could be used to set security standards and compel protective measures.
For example, the Department of the Treasury and the Securities and
Exchange Commission and other agencies have regulatory authorities over
financial markets and can compel them to take certain protective measures.
While DHS generally does not have authority over many of the assets in
those sectors in which it has lead responsibility, some other federal
agencies do. For example, the Nuclear Regulatory Commission, which issues
licenses to commercial nuclear power plants, has clear regulatory
authority over security matters at these facilities. While each sector has
a Government Coordinating Council that includes representatives from
federal agencies involved in the sector, IAIP recognizes that it can
further leverage work that has already been done by other agencies that
have regulatory authority over certain private sector owners. This
condition brings attention to the need for IAIP to coordinate with these
agencies to leverage federal authority in areas where oversight already
exists.
Implementation and Monitoring: IAIP Is Also Challenged by Its Limited Role in
Implementation and Limited Information on Effectiveness
IAIP has a limited role in implementing its own protective programs, as
well as in monitoring the effectiveness and level of implementation of
security risk management programs broadly. An example of IAIP's role in
implementing protective programs is the Buffer Zone Protection Program.
Managed by the Protective Security Division within IAIP, the program is
designed to make it more difficult for terrorists to conduct planning
activities or successfully launch attacks from the immediate vicinity of
critical infrastructure and key resource targets. The goal of the program
is to assist state and local government, local law enforcement, and owners
and operators in preventing, defending against, preparing for, and
mitigating the impacts of terrorist attacks. The program does so by making
grants available, through ODP, to state and local law enforcement to
implement buffer zone protection plans "outside the fence" of private
facilities, as well as by conducting workshops and technical assistance
visits.
However, IAIP's role in implementing protective measures is much more
limited "inside the fence" because IAIP does not own or operate any
assets, have regulatory authority over those entities that own or operate
the assets, or provide funds for such entities. Owners and operators of
the infrastructure assets-be they federal, state, local, or private-are
responsible for implementing the protective actions needed. As previous
GAO work has shown, public policy tools have been used as an approach to
encourage increased private sector critical infrastructure protection
efforts even when regulatory authority is lacking.18 In terms of
monitoring implementation of actions to increase the protection of
critical infrastructure, however, IAIP does have a role. Its mission is to
assess the overall state of critical infrastructure protection in the
nation.
18GAO, Critical Infrastructure Protection: Challenges for Selected
Agencies and Industry Sectors, GAO-03-233 (Washington D.C.: Feb. 28,
2003); and Homeland Security: Information Sharing Responsibilities,
Challenges, and Key Management Issues, GAO-03-715T (Washington D.C.: May
8, 2003).
In its monitoring role, IAIP faces a number of challenges. The first
challenge is that there are currently no performance measures to evaluate
the effectiveness of infrastructure protection. The interim NIPP, while
calling for such measures, does not offer any. According to IAIP
officials, such performance metrics will not be available before 2006. One
of the difficulties encountered is that it is hard to develop performance
measures to gauge homeland security activities that are directed at
modifying terrorist behavior. For example, it is difficult to determine
whether measures to improve the protection of critical infrastructure have
a deterrent effect on terrorists. In some cases, deterring terrorists from
attacking "hard" targets (those that are heavily protected) might have the
effect of directing the terrorist toward attacking "soft" targets (those
that are lightly protected).
Conclusions
IAIP's role in risk management is critical because of its breadth. IAIP
has overall responsibility to identify critical assets across all sectors
of the nation's infrastructure, as well as to play a lead role in
developing methodologies and guidance for analyzing risks and assessing
the benefits and costs of security alternatives. Progress in most areas of
responsibility has been limited, and much challenging work remains to be
done. In particular:
o Good threat data are critical to conducting risk management.
During the course of our review, IAIP recognized the importance of
developing information on the relative probability to various
threat scenarios. Until better data on threats are developed,
risk-based data may not fully inform decisions on where to
establish priorities.
o Even with better threat data, assets cannot be compared across
sectors without a methodology for doing so. Currently, IAIP is
developing a methodology to do this, but it is not yet
comprehensive, nor has it been applied. Until such a methodology
is fully developed, IAIP will be challenged in conducting a
national-level assessment of risks, a key element of IAIP's core
responsibility. Without a methodology, it will not be possible for
IAIP to make a determination of relative risks that could help
inform decisions on resource allocation.
o Comprehensive planning and performance measures are necessary
to clarify what needs to be done and to determine progress in
critical infrastructure protection. The National Infrastructure
Protection Plan is still in interim form, leaving open many
questions about how specific sectors will be protected and how
performance will be measured. Without target dates for completing
sector-specific plans and performance measures, it will not be
possible to determine IAIP's progress in these areas.
Additionally, without having plans to develop information on what
protective measures could address multiple threat scenarios,
IAIP's ability to inform owners and operators of ways to protect
their facilities in a cost-effective fashion is limited.
In all, the lack of progress leaves many decision makers basically on
their own to develop a way to determine where scarce resources need to be
applied against almost unlimited numbers of assets to maximize the
protection of critical infrastructure and security for the homeland.
Recommendations for Executive Action
To help ensure the development of risk management approaches to homeland
security activities, we recommend that the Secretary of Homeland Security
direct the Undersecretary for IAIP to undertake the following three
actions:
o Work with the intelligence community to develop ways to better
assess terrorist threats and use available information and expert
judgment to develop a relative probability for various terrorist
scenarios and provide this information to sector-specific
agencies;
o As tasked by presidential directive, develop a methodology for
comparing and prioritizing risks of assets within and across
infrastructure sectors by including data on the relative
probability of various threat scenarios;
o In completing the National Infrastructure Protection Plan,
include target dates for completing sector-specific plans,
developing performance measures, and identifying protective
measures that could address multiple threat scenarios.
Agency Comments and Our Evaluation
In commenting on a draft of chapter 4, DHS, including IAIP, said that IAIP
is taking steps to address recommendations in the report. In regard to our
recommendation on working with the intelligence community to develop
better threat data, DHS said it is working with the Coast Guard on a pilot
project to do so. The pilot effort will be evaluated, improved upon, and
then more broadly applied with stakeholders in the intelligence community
pending their acceptance, according to DHS. In regard to our
recommendation on developing a methodology for comparing and prioritizing
risks within and across sectors, DHS responded that for fiscal year 2006
grants, a risk analysis methodology was applied that considers a small set
of assets across sectors. With respect to our recommendation on completing
the National Infrastructure Protection Plan, DHS stated that a draft plan
is out for comment and that the issue of identifying protective measures
that could address multiple threat scenarios is being addressed by IAIP.
DHS also submitted technical comments under separate cover and we made
changes where appropriate. Written comments from DHS are in appendix II.
CRecommendations Chapter 5: Overall Observations and Recommendations
Taken together, what do the efforts and experiences of these three
components suggest about where DHS is with regard to managing homeland
security efforts on the basis of risk? In our view, there are two key
overall observations related to the degree of progress made, and two more
related to next steps that need to take place. With regard to progress
made, the first observation is that while considerable effort has been
applied, much more remains to be done than has been accomplished so far.
Across all three components, the most progress has generally been made on
fundamental steps, such as conducting risk assessments of individual
assets, and the least amount of progress has generally been made on
developing ways to translate this information into comparisons and
priorities across ports or across infrastructure sectors. Second, and
closely related, progress among the three components' efforts has varied
not only with the length of time the component has been at the job, but
also with the complexity of its risk management task. With regard to next
steps that would appear to add most value to making further progress, one
key observation is that in the short term, progress is heavily dependent
on continuing to make steady progress at improving basic policies,
procedures, and methods for risk assessments and other phases of the risk
management framework outlined in chapters 2, 3, and 4. Each component has
an admittedly difficult set of challenges ahead, but progress has to be
built on taking these incremental steps and doing them well. The final
observation is related to a critical longer-term need: more guidance,
direction, and coordination from DHS. The challenges and difficulties
associated with creating a coordinated, coherent risk management approach
to the nation's homeland security have been widely acknowledged since the
events of September 11 and the creation of DHS. One of the presidential
directives calls on DHS to provide such guidance, but the agency has yet
to do so. As individual components begin to mature in their risk
management efforts, the need for consistency and coherence becomes even
greater. Without it, the prospects increase for efforts to fragment,
clash, and work at cross purposes.
First Observation: Much Remains to Be Done
There is a long way to go in implementing risk management successfully in
port security-and an even longer way to go in implementing risk management
in homeland security in general. One main reason is the sheer amount of
work that must be done. Five years ago, before the September 11 attacks,
the various terrorist scenarios seemed more remote and less certain than
the hard reality brought on that day. September 11 changed this
perspective dramatically. The work involved is immense and cuts across
many jurisdictional boundaries. Federal agencies are called on to
strengthen their partnerships with one another and to work more closely
with thousands of state, local, and industry stakeholders.
A second major reason is that applying risk management to terrorism has no
well-established precedent. Parts of the private and public sectors have
used risk management principles for decades. However, doing so in a
homeland security setting is a highly difficult task that remains in its
embryonic phases. The components we reviewed face daunting challenges in
weaving a concern for risk into the annual cycle of management review and
budget decisions. Across the federal government, this challenge is
magnified and complicated because of the number of agencies charged with
carrying out risk management. This is an extraordinarily difficult effort
with no clear road map of ways to strategically integrate a concern for
risk into management decisions.
The fact that so much work remains is not the result of inaction by
federal agencies. In the agencies and programs we examined, activities
were often extensive and wide-ranging. Some activities, such as IAIP's
attempts to develop risk assessment criteria for its comparisons across
risk sectors, have had limited success, compounding the problem. The
underlying point, however, is that this is an extraordinarily difficult
effort with no clear and direct precedent to act as a guide. Implementing
risk management in port and homeland security will take time and care, and
this challenge will require ingenuity in adopting risk management
techniques to this new application in a cost-effective way.
The progress that has occurred to date in the agencies and programs we
examined has been primarily in the activity that most people would perhaps
associate most readily with risk management-conducting assessments to
determine what the risks are at specific ports and facilities. While much
remains to be done even there, progress has generally been slower on ways
to approach risk management strategically-that is, with a clear set of
measurable objectives, a clear knowledge of the options available for
addressing risks and the trade-offs involved with these options, and
evaluation and feedback mechanisms for continuing to refine and strengthen
the approach.
Observation Two: Progress Varies by Component and Reflects Key Characteristics
of the Component and the Scope of Its Risk Management Efforts
The three components we studied have made varying degrees of progress in
risk management, and to a degree their progress is related to three main
factors: how long they have been at the task, how organizationally stable
they are, and the scope of what they are trying to do. The Coast Guard,
for example, is furthest along among the three components, reflecting in
part where it stands in relationship to all three of these factors. It has
been at the task the longest of the three components, having begun work on
implementing risk management in its port security efforts immediately
after the September 11 attacks. Its primary risk assessment tool at the
port level, PS-RAT, was implemented in November 2001, and by August 2002,
prior to the creation of DHS and the port security framework called for
under the Maritime Transportation Security Act of 2002, it had begun
security assessments at major U.S. ports. To a degree, these early efforts
were learning experiences that required changes, but the Coast Guard was
able to build on its early start. The Coast Guard also had the greatest
organizational stability of the three components. It moved into DHS as an
already established entity with an organizational culture spanning
decades, and its organization and mission were not significantly altered
by moving into DHS. Finally, with regard to the scope of its risk
management activities, the Coast Guard's work is specific to port
locations, where it has direct and primary responsibility for carrying out
security responsibilities. With its focus on ports, the Coast Guard does
not have to address a number of the critical infrastructure sectors laid
out in national preparedness policy, such as banking and finance,
information and technology, and public health. Even so, the Coast Guard's
experience to date shows that as the scope of activity widens, even within
a single sector, complexities develop. For example, the Coast Guard has
prioritized risks within individual ports, and it has actions under way to
assess risks across ports, but using this information to strategically
inform the annual program review and budget process will require further
attention.
ODP has made somewhat less progress than the Coast Guard. Relative to the
Coast Guard's progress, its progress reflects a later start, an
organization with much less institutional maturity, and a different role
from the Coast Guard's in that ODP provides grant money rather than
directly setting security policy. ODP was transferred from the Department
of Justice to the Department of Homeland Security in 2003. While ODP's
early grant approval efforts had some risk management features in place,
its main strides in risk management have come in the procedures recently
adopted for the fiscal year 2005 grants. In moving toward risk management,
ODP has found ways to allow information from the Coast Guard and IAIP to
inform its decision making. This is an encouraging and important sign,
because the success of risk management efforts depends in part on the
ability of agencies with security responsibilities to share and use each
others' data and expertise. Although both the Coast Guard and the port
security grant program administered by ODP have port security as their
focus, ODP's more limited scope of responsibility has also had an effect
on its risk management efforts. First, because ODP's role is to award
grants that support federal priorities in port security, its progress in
risk management depends to a degree on the progress made by federal
agencies in determining what their own port security performance measures
should be. Second, ODP's funding priorities are subject to criteria other
than risk, as the fiscal year 2004 grant awards demonstrate. That year,
after creating an initial list of awardees based in part on risk, and
without regard to ability to pay, ODP extensively revised the list and
awarded grants to entities considered to have fewer funding alternatives.
Of the three components, IAIP is the least far along in its risk
management efforts. All three factors have had an effect on this progress:
IAIP has been at its task for a relatively short time; it is a new
component; and relative to the Coast Guard and ODP, the scope of its
efforts is much broader and more difficult. IAIP was created under the
Homeland Security Act of 2002, giving the directorate little time to
acquire institutional maturity. In addition to taking on difficult tasks
like risk management, IAIP faces other institutional challenges, such as
establishing new management systems, developing sound human capital
practices, and integrating its efforts with those of the rest of DHS.
Further, the scope of its risk management activities extends well beyond
the port-focused activities of the Coast Guard or ODP. IAIP is responsible
for conducting risk assessments for every critical infrastructure segment
in the nation. As demonstrated by the experience of its RAMCAP methodology
for comparing risk across sectors, IAIP remains challenged in meeting that
responsibility. Its lack of progress reflects the same lesson that emerges
from the Coast Guard's experience in trying to expand the focus of risk
assessments beyond a single port: The complexity of risk management
appears to grow exponentially as the focus expands beyond a single
location or single type of infrastructure. This complexity may help
explain IAIP's lack of progress, but IAIP is unable at this time to
provide adequate assurance to Congress or the country that the federal
government is in a position to effectively manage risk in national
security efforts. Steps have been small; by far the biggest work is yet to
come.
Observation Three: In the Short Term, Further Progress Is Still Heavily
Dependent on Completing and Improving Basic Policies, Procedures, and Methods
Acknowledging that the nation still has far to go in establishing a sound
risk management approach to security should not obscure the need to
continue taking small, but critical, steps-building on incremental
advances. The three components we reviewed have actions under way to
improve their risk management approach, and their experience indicates
that much of the immediate work should remain focused on basic steps
needed to implement all components of the full risk management framework.
The recommendations we make in chapters 2, 3, and 4 include
component-specific steps for what needs to be done. In overview, these
specific recommendations cluster around several major themes related to
the five phases:
o Setting strategic goals, objectives, and constraints: While all
three components have broad-scale goals in place, none has yet
tied these goals to specific and measurable results. Without such
measures, it is difficult to gauge what progress has been made in
improving security and what security gaps remain. The Coast Guard
is furthest along: It has tied its goals to activity levels, such
as the number of patrols conducted or vessels inspected, and it is
working toward developing outcome-based measures. This is a good
step, but without such measures in place, it is not possible to
see how programs reduce risks, improve security, or identify gaps
in security that remain. All three components would benefit from
specifying in clear and measurable terms what their efforts are
designed to accomplish.
o Conducting risk assessments: All three components can improve
their risk assessment techniques. All three were challenged by a
general lack of detailed information on capabilities and
intentions of terrorist groups as this relates to various threat
scenarios. They took different approaches in response: The Coast
Guard, for example, used threat scenarios as substitutes for
detailed threat information and is working on assigning likelihood
to each in order to determine where risks might be greatest, while
IAIP evaluated the consequences of certain possible attacks and
focused its analysis of vulnerabilities on the attacks with the
greatest consequences. Approaches that do not include information
on the likelihood of various threat scenarios have limitations
that affect the degree to which agencies are able to determine how
to best focus their efforts on areas of greatest risk. Efforts to
strengthen both data, methodology, and policy would increase the
reliability of their results.
o Evaluating alternatives: All three components face problems in
measuring the costs and benefits of different measures for
preventing or mitigating terrorist attacks. These include
developing ways to measure costs incurred by a broad range of
public and private stakeholders and developing ways to measure
benefits (such as deterrence) when these benefits may not be
quantifiable. These difficulties are particularly great for IAIP,
which must be able to measure costs and benefits associated with
mitigation strategies that reduce vulnerabilities at critical
infrastructure in all sectors. These difficulties are compounded
by the complexity in valuing costs and benefits in the area of
homeland security when either the costs or the benefits are
difficult to quantify or are not valued in monetary terms.
o Management selection: The three components face different
challenges in this area, because each has different types of
alternatives available in making decisions. The Coast Guard has
the most direct control over security efforts; it can, for
example, decide what protective measures to take with its own
assets, and it has authority over other stakeholders to implement
the Maritime Transportation Security Act of 2002. Its challenges
lie mainly in what has already been discussed above-strengthening
methods for risk assessment and alternatives evaluation and
integrating this effort with the annual cycle of program review-so
that management can make the most informed decisions about these
efforts. ODP affects security efforts less directly; it can only
consider facilities that have applied for grants, and it has no
direct authority over port facilities in general, as does the
Coast Guard. ODP has worked with the Coast Guard to receive its
input into the grant application process. One challenge is to
consistently apply criteria for management selection in a more
transparent way. IAIP faces the most challenges in this area,
because once it makes recommendations about how to prioritize
assets on a national scale, it is largely dependent on the actions
of others to carry them out and, particularly for owners and
operators of private infrastructure, is dependent to a large
degree on persuasion, market forces, or the work of regulatory
agencies that have authority over key infrastructure, to ensure
that protective measures are in place.
o Implementation and monitoring: Particularly for the Coast Guard
and ODP, we have been able to identify instances in which the
components have moved aggressively to improve their risk
management approaches-and to continue doing so. Particularly with
its recent setback on its RAMCAP methodology, IAIP is considerably
behind these two components in implementing any kind of risk
management approach. To move forward, it must overcome more basic
problems with assessing risks and alternatives.
Observation Four: In the Long Term, Progress Rests Heavily on a Level of
Coordination That Has Yet to Be Demonstrated
In the long term, progress will become increasingly dependent on how well
the nation's homeland security risk management effort is coordinated. We
have identified and reported on some notable improvements in coordination
at the port level, through such mechanisms as intelligence fusion and
coordination centers, local area maritime security committees, and
interagency operational centers.1 Replicating such coordination among DHS
agencies and with state, local, and industry stakeholders is key.
Currently, various assessment approaches are being used, and in many ways,
these approaches are neither consistent nor comparable. Our work at IAIP,
the Coast Guard, and ODP showed examples of these inconsistencies. For
example, IAIP's initial plans called for treating all threat scenarios as
equally likely to happen, while the Coast Guard and ODP are attempting to
integrate the likelihood of various threat scenarios into their analysis
vulnerabilities. The danger in using different methods is that if agencies
develop systems and methodologies without some overall coordination, they
may end up with redundant or incompatible systems that have little or no
ability to inform one another. Even more important, these systems may
provide decision makers with unreliable or incomplete data on how to
allocate resources and protect the American people in a cost-effective
way. Absolute compatibility is likely impossible given the multiple
stakeholders at the federal, state, local, and industry levels. For
example, owners and operators of critical infrastructure may value and act
on risks differently than the Coast Guard and IAIP. Having a common risk
management framework is a key consideration for assuring that knowledge
and data can be transferred to all stakeholders, while permitting
stakeholders to value risks in different ways. Even if agencies and
stakeholders were working in close cooperation, lack of coordination is
likely only to exacerbate the problem. This is particularly true given the
difficulty of the task and the limited availability of federal risk
management guidance.
Until now, having such inconsistencies may have seemed less important than
just getting risk management efforts under way. To a degree, we found this
was the case with the larger universe of homeland security actions: When
we first began reviewing agency actions shortly after the September 11
attacks, we found agencies at work on many efforts, but signs of
coordination problems were already apparent. Similarly, the risk
management efforts that have been conducted to date appear fueled by a
strong sense of the need to make some headway, with coordination and
consistency a lesser concern. For example, IAIP, which is charged by
statute with developing comprehensive assessments of the vulnerabilities
of critical infrastructure and key resources, did not attempt to guide the
Coast Guard's efforts in setting up a methodology for assessing
port-specific risks. IAIP officials told us it was more important for the
Coast Guard and other agencies to proceed with their risk assessment
efforts than to delay starting, even though the officials recognized that
these efforts might create approaches that would not mesh cleanly with the
approach that IAIP would eventually develop. Given what has occurred to
date, this course of action appears prudent, in that the Coast Guard has a
considerable portion of a risk management system in place. If it had
waited to begin until guidelines and policies had been set, it would still
be waiting to start.
1GAO, Maritime Security: New Structures Have Improved Information Sharing,
but Security Clearance Processing Requires Further Attention, GAO-05-394
(Washington, D.C.: Apr. 15, 2005).
Now, however, the need for coordination is looming larger. IAIP has a
significant role to play in this regard through its responsibility for
providing agencies with guidance about risk management, but it has made
limited progress. IAIP has been challenged in establishing uniform
policies, approaches, guidelines, and methodologies for integrating
federal infrastructure protection and risk management activities within
and across sectors, along with metrics and criteria for related programs
and activities as called for by HSPD-7. While IAIP has coordinated its
activities with entities such as ODP and the Coast Guard, it has yet to
issue policies, guidelines, and methodologies as required by the
directive. Making progress with regard to this challenge is key to an
effective use of risk management resources, as the National Strategy for
the Physical Protection of Critical Infrastructure and Key Assets
recognizes.
Guidance is also important when agencies integrate a concern for risk into
the annual cycle of program and budget review. Doing so within an agency
is a difficult task in that traditional ways of reviewing budgets and
programs often rely on program data that call for continuing or expanding
a program without examining the relative risks that are addressed with the
funds that are expended. Shifting organizations toward this nexus of using
risk-based data as part of annual management review cycles will take time,
attention, and leadership. Even in agencies where much progress has been
made in developing risk management techniques, integrating disparate
systems such as risk management with budget and program management remains
a long-term challenge. The Secretary of DHS has said that operations and
budgets of its agencies will be reviewed through the prism of risk, but
doing this is made difficult by the level of guidance and coordination
that has been provided so far.
DHS has recently reorganized, and the consideration of whether its new
organization will effectively implement its risk management
responsibilities is an important one. At the time we conducted our review,
risk management was the responsibility of IAIP. IAIP's risk management
efforts were focused mainly on assessing and reducing the vulnerabilities
that exist in and around specific facilities or assets. But DHS's
responsibility is broader than this: besides assessing and reducing
vulnerabilities at specific facilities, it also includes preventing
attacks from occurring (and in the process protecting people and critical
infrastructure) and responding to and recovering from natural disasters
and acts of terrorism. This initial focus on vulnerabilities at specific
assets had the potential of limiting DHS's ability to achieve the broader
goal of using risk-based data as a tool to inform management decisions on
all aspects of its missions. The Secretary of DHS has now moved risk
management to a new Preparedness Directorate. Although, it is unclear how
such a move could affect DHS's ability to carry out its risk management
responsibilities, the new focus on preparedness may result in an emphasis
that may go too far the other way-that is an emphasis on protection of
specific assets and response and recovery at the expense of prevention. As
DHS goes forward, the office in which the risk management responsibility
resides should have a broad perspective across the department's entire
mission as well as the necessary authority to hold DHS component agencies
responsible for carrying out risk management activities in a coordinated
and consistent manner.
Beyond DHS, integrating risk with existing systems for budget and program
review is complicated by the fact that while IAIP has responsibility for
coordinating this effort, IAIP and the Secretary of DHS are challenged
because they must depend on others to follow risk management principles
for programs and budgets at the other six major Departments or agencies
that have been charged with assessing risks under HSPD-7. In regard to
this situation, OMB has taken the position that this is what the Homeland
Security Act and HSPD-7 call for and it does not play a role in this
process. These conditions increase the uncertainty of implementing risk
management across federal agencies in a way that informs program and
budget review processes. Whether such practices will occur within the
executive branch is unclear because of these organizational barriers.
Recommendations for Executive Action
To strengthen individual agency efforts to implement a risk management
approach to homeland security activities, we recommend that the Secretary
of Homeland Security direct the Undersecretary for IAIP to undertake the
following three actions:
o As required by presidential directive, establish uniform
policies, approaches, guidelines, and methodologies for
integrating federal infrastructure protection and risk management
activities within and across sectors, along with metrics and
criteria for related programs and activities and develop a
timetable for completing such guidance. Such policies and guidance
should address the issue of integrating risk management systems
into existing systems of program and budget review.
o As DHS continues to review its organizational structure, work
with the Secretary's office to determine which office is best
suited to help ensure that the responsibility for risk management
policy and implementation has a broad enough perspective on all
elements of risk, including threats, as well as the necessary
authority to coordinate with DHS component agencies and hold them
accountable for risk management activities.
o Work with the Office of Management and Budget to examine
options for holding departments and agencies accountable for
integrating risk management for homeland security programs and
activities into the annual cycle of program and budget review.
Agency Comments and Our Evaluation
In commenting on a draft of chapter 5, DHS, including IAIP, generally
concurred with the recommendations. In regard to our observation that
there is a long-term need for guidance and coordination, DHS noted that as
part of the department's second-stage review, a six-point agenda has been
created to ensure that policies, operations, and structures are best
aligned to address potential threats. This agenda is a major step in the
right direction, and as we observe in this chapter, much work remains to
be done to translate this goal into actions.
DHS agrees that the application of risk management to domestic terrorism
has no precedent, and that the probabilities and consequences of terrorist
acts are difficult to predict. DHS also concurred with our observation
that the scope of establishing a risk management framework-a former IAIP
Directorate responsibility-across the federal government is immense. DHS
acknowledges that IAIP's progress has been limited in part because its
risk assessment responsibilities span broad sectors of the nation's
infrastructure rather than seaports alone. DHS also submitted written
comments under separate cover and we revised the report where appropriate.
Written comments from DHS are in appendix II.
A Appendix I: A Risk Management Framework
This appendix describes how we developed the risk management framework and
how we used it to evaluate activities related to homeland security and
combating terrorism. The framework is intended to be a starting point for
risk management activities and will likely evolve as processes mature and
lessons are learned. A glossary is included at the end of this appendix.
General Lack of Uniform Guidance on Risk Management
Although the Homeland Security Act and subsequent strategies advocate the
use of risk management to protect the nation's critical infrastructure and
key resources, they did not define how this was to be accomplished.
Homeland Security Presidential Directive 7 (HSPD-7) directed the Secretary
of the Department of Homeland Security (DHS) to establish uniform
policies, approaches, guidelines, and methodologies integrating federal
infrastructure protection and risk management activities. However, no
further direction or guidance as to the course of action has been
forthcoming.
The ability to anticipate future happenings and to choose among
alternatives lies at the heart of risk management and provides us with a
guide, based on good management practices and supported by established
internal controls that can enhance decision making. Although risk
management has long been used for assessing risk in some sectors, such as
environmental issues, health care, finance, and the insurance industry,
the application of risk management principles to the homeland security
area is relatively new. The many areas and activities under homeland
security provide untested and difficult challenges because the source of
the risk is an intelligent adversary with whom there exists little
domestic experience. As a result, the probabilities and consequences of a
terrorist attack are difficult to predict. In spite of this high degree of
uncertainty and the knowledge that not all risk can be eliminated,
enhancing protection from known or potential threats can help prevent or
mitigate adverse events.
Methodology for Developing a Risk Management Framework
Given that there is no established universally agreed upon set of
requirements or processes for a risk management framework specifically
related to homeland security and combating terrorism, we developed a
framework that would be applicable by reviewing, analyzing, and
synthesizing several sources of information.
We began by reviewing current risk literature and previous GAO reports and
testimonies.1 We consulted the Government Performance and Results Act
(GPRA) of 1993; the Government Auditing Standards, 2003 Revision, GAO's
Standards for Internal Control in the Federal Government (November 1999);
guidance from the Office of Management and Budget (OMB); the work of the
President's Commission on Risk Management; consulting papers; and the
enterprise risk management approach of the Committee of Sponsoring
Organizations (COSO) of the Treadway Commission. In addition, we consulted
with experts in the fields of risk management, risk modeling, and
terrorism. We reviewed numerous frameworks from industry, government, and
academic sources.
We synthesized information from these numerous government, industry, and
academic sources in developing our risk management framework. The
framework was field-tested on several GAO reviews. The draft framework was
then reviewed by three academic experts in risk management. No substantial
changes to the draft framework were recommended.
A Risk Management Framework
The framework should be considered to be a starting point in a field that
is evolving, and the entire cycle of risk management activities should be
viewed as a goal. The phases contained in the framework are
o strategic goals, objectives, and constraints;
o risk assessment;
o alternatives evaluation;
o management selection; and
o implementation and monitoring.
1GAO, Combating Terrorism: Threat and Risk Assessments Can Help Prioritize
and Target Program Investments, GAO/NSIAD-98-74 (Washington, D.C.: April
1998); Combating Terrorism: Evaluation of Selected Characteristics in
National Strategies Related to Terrorism , GAO-04-408T (Washington, D.C.:
Feb. 3, 2004); Rail Security: Some Actions Taken to Enhance Passenger and
Freight Rail Security, but Significant Challenges Remain, GAO-04-598T
(Washington, D.C.: Mar. 23, 2004); and Homeland Security: Summary of
Challenges Faced in Targeting Oceangoing Cargo Containers for Inspection,
GAO-04-557T (Washington, D.C.: Mar. 31, 2004).
The framework has been developed so that individual phases of the
approach, such as risk assessment, do not become ends in themselves, but
provide a full cycle of related activities, from strategic planning
through implementation and monitoring. The process is dynamic, and
although the various phases appear linear, new information can be entered
at any phase. The framework can be used to inform agency officials and
decision makers of the basic components of a risk management system or can
be used as a stand-alone guide. Figure 4 illustrates our risk management
framework and some sources of criteria, such as GAO best practices, Office
of Management and Budget circulars, GAO guidance on internal controls, and
the Government and Performance Act of 1993 and their link with management
processes. While statistical methods and risk-ranking approaches
frequently underlie risk assessment approaches, different application
areas tend to develop their own terminologies and their own logical
sequences for the cause of risk.
Figure 4: Sources of Evaluation Criteria Associated with Risk Management
Phases
The risk management framework is designed to be flexible, in that the
approach may be applied at various organizational levels ranging from that
of a department level of a multiagency organization down to that of a
specific project or program.
Because there is no one uniformly accepted approach to risk management,
terms and activities may differ across applications. However, any approach
that omits the substance of the steps shown in figure 4 is likely to have
material weaknesses. Table 11 summarizes the phases of our risk management
framework and provides examples of elements contained in those phases.
Table 11: A Risk Management Framework
Source: GAO.
The following sections provide more detail on the five phases of our risk
management framework.
Strategic Goals, Objectives, and Constraints
This phase addresses what the strategic goals are attempting to achieve
and the steps needed to attain those results, including milestones and
performance measures to permit measurement of progress toward those goals.
Ideally, management decisions should be made in the context of a strategic
plan, with clearly articulated goals and objectives that flow from the
plan. Strategic goals at the highest level could be considered an
"end-state" followed by a logical hierarchy of major goals and subordinate
objectives composed of clear, concise, measurable activities and timelines
to achieve results and ensure accountability. An organization's program or
plan and risk planning documents should address risk-related issues that
are central to its mission. Our work related to the Government Performance
and Results Act of 1993 has produced guidance that identifies risk for the
congressional oversight of federal agencies' strategic plans.2 The
consideration of risk in strategic planning may be incorporated into
planning documents or into specific management strategies.
Currently, it is difficult to translate plans and actions into a clear
sense of how we are progressing in making our nation more secure. One
reason is that homeland security efforts, in general, lack clear goals
with corresponding performance measures to evaluate progress toward these
goals.3 As others, such as the Gilmore Commission, have stated, a
continuing problem for homeland security has been the lack of clear
strategic guidance about the definition and objectives of preparedness.4
Risk management allows entities to operate more effectively in
environments of uncertainty by providing the discipline and structure in
which to address these issues, since risk management is not an end in
itself, but an important component of an entity's management process. As
such, risk management is interrelated with, among other things, an
entity's governance, performance management, and internal controls. The
process of risk management provides the rigor and structure necessary to
identify and select among alternative risk responses whose cumulative
effect is intended to reduce risk, and the methodologies and techniques
for making selection decisions. This process enables entities to enhance
their capability to identify potential adverse events, assess risks, and
establish integrated responses. Further, this phase in the planning
process would include support and buy-in from upper levels of management
and stakeholders. Acceptance for concepts of the model from this group
provides the groundwork for future discussions.
2GAO, Agencies' Strategic Plans under GPRA: Key Questions to Facilitate
Congressional Review, GAO/GGD-10.1.16 (Washington, D.C.: May 1, 1997).
3GAO, Homeland Security: Key Elements to Unify Efforts Are Underway but
Uncertainty Remains, GAO-02-610 (Washington, D.C.: June 7, 2002).
4The Gilmore Commission (also known as the Advisory Panel to Assess
Domestic Response Capabilities for Terrorism Involving Weapons of Mass
Destruction), Forging America's New Normalcy (Arlington, Virginia: Dec.
15, 2003).
Finally, various constraints may have an impact on risk management plans.
Some constraints may be imposed by statute, higher-level policy, budget,
or other factors beyond management's control and may vary with the scale
of the application. Managers at different levels within an organization
will have different degrees of flexibility to institute risk management
countermeasures. An important constraint for federal agencies, such as
DHS, is the role that Congress plays in authorizing and funding programs.
For example, Congress may direct specific actions affecting how agencies
allocate funding.
Risk Assessment
This phase addresses the process of evaluating the threats and
vulnerabilities of assets so that countermeasures might be instituted to
prevent or mitigate risks. Threat, in the risk management model, concerns
the probability that a specific type of attack will be initiated against a
specific target. It includes any circumstance or event with the potential
to cause loss or damage to the asset. Although agencies may not have
enough information to identify and characterize all threats related to
their assets, known or imagined adverse events would be characterized in
some detail. Effective threat analysis is dependent on an understanding of
an adversary's intention, motivation, historical data, and capability to
damage. An additional crucial component of risk assessment is
vulnerability, that is, any weakness that an adversary can exploit to harm
or damage the asset. An asset may be highly vulnerable to one mode of
attack but have a low level of vulnerability to another, depending on a
variety of factors, such as countermeasures already in place. While
consequence concerns the result of an adverse event on a particular asset,
criticality is the asset's relative importance to the entity. A
criticality assessment identifies and prioritizes assets and functions in
terms of specific criteria, such as their importance to public safety and
the economy, as a basis for identifying which structures or processes are
relatively more important to protect from attack. Criticality assessments
are important because they provide, in combination with the framework's
threat and risk assessments, the basis for prioritizing which assets
require greater protection relative to finite resources and provide
information for later stages in the risk management process. Risk
assessments should utilize the most appropriate subject matter experts in
assessing the components of risk. When dealing with Bayesian probability
estimates, this is critical.5 In addition, mathematical constructs must be
chosen carefully, specifically tailoring approaches in the context of
uncertainty and data quality.
Alternative Evaluation
This phase addresses the evaluation of risk reduction methods by
consideration of countermeasures or countermeasure systems and the costs
and benefits associated with them. Ideally, a risk management framework
would include an evaluation of a risk assessment as a valid decision
support tool to establish and prioritize a risk management strategy.
Furthermore, a strategy might include risk management consultants and
decision-making tools, such as software that simulates a particular type
of attack. Information developed in previous phases would inform
decisions.
Specific countermeasures would be considered and prioritized based on a
number of factors, such as the degree of risk reduction they afford and
the cost and difficulty to implement them. Risk assessment may give
guidance to implementing countermeasures or countermeasure systems that
may be used for more than one critical asset, or to prevent, mitigate, or
respond to multiple adverse events occurring simultaneously. In addition,
external risk consultants can be advantageous at this phase in terms of
creating a variety of countermeasure options. While external reviewers
cannot ensure the success of a plan, they can increase the probability of
selecting the most effective countermeasures for the least cost because of
their expertise in the area. Further, risk scenarios might also provide
valuable information pertaining to an entity's ability to respond to a
terrorist event, evaluate its coordination with other entities, identify
problems, and institute corrective action.
Finally, a risk management strategy should include a cost-benefit analysis
of countermeasure options as they are a critical element of alternatives
evaluation. Major regulatory actions or capital investments of federal
expenditures generally require a cost-benefit or cost-effectiveness
approach.6 This approach can be useful in evaluating alternatives, since
it links the benefits from risk-reducing countermeasures to the costs
associated with them. In the development of such analyses, quantitative
impacts affecting both costs and benefits are, to the extent possible,
identified in monetary terms.
5See Yacov Y. Haimes, Risk Modeling, Assessment, and Management, 2nd ed.
(Hoboken, New Jersey: Wiley and Sons, 2004).
While the core OMB guidance for evaluating countermeasures for budgetary
and regulatory purposes focuses on monetary cost-benefit evaluation, OMB
is essentially silent when costs and benefits cannot be easily quantified
or monetized. Costs that are not generally estimated or included in
monetary terms include opportunity costs, that is, the value of
opportunities forgone because resources are applied to antiterrorism
countermeasures. These costs are most controversial when considered in
areas such as public service programs or services curtailed or cancelled.
Benefits are usually measured in terms of the risk reduction they provide.
They are considered in terms of the overall effectiveness of the
countermeasures with respect to the estimated vulnerabilities.
Management Selection
This phase addresses such issues as determining where resources and
investments will be made, the sources and types of resources needed, and
where those resources would be targeted. Management's active participation
in this phase is important as decisions are of necessity influenced by the
preferences and value judgments of agency leadership. For example, some
managers will prefer to concentrate countermeasures on a relatively few
critical assets, while others may value distributional impacts, that is,
to distribute resources over a wide variety of assets. Ideally, a risk
management strategy would also identify appropriate mechanisms to allocate
resources, such as grants based on identified needs. Furthermore, a key
factor in the selection of risk reducing measures is risk tolerance, the
level of comfort management has with various levels of risk. This
tolerance may change over time, depending on new information, changes in
financial constraints, and attitude toward risk. The risk management
strategy, with stakeholder input, will identify what constitutes an
acceptable level of risk for assets and how resources are delegated.
6Executive Order 12866 and circulars A-4 and A-94 apply to regulatory
actions, and circulars A-11 (sect. 7), A-94, and A-130 apply to capital
investments.
The risk management strategy reflects consideration as to which risks
should be managed immediately and to what extent, and which risks can be
deferred and addressed at a later time. Milestones and timelines for
implementation are important elements that allow evaluating the extent to
which progress is being made toward achieving goals. It also illustrates
the degree of protection that can be obtained and places security and
costs in perspective. The documentation of management decisions, including
the rationales that support the decisions, will provide valuable
information for future adjustments.
Implementation and Monitoring
This phase addresses the degree to which risk management strategies
contain internal controls and performance measurement guidelines. In
addition to implementing countermeasures, it may also include implementing
new organizational policies and procedures, as well as human, physical,
and technical controls. Countermeasures would be initiated in accordance
with the timelines in the risk management schedule.
Monitoring and evaluation include, when and where appropriate, external
peer review, testing and validation of countermeasures, evaluating the
effects of these actions on future operation, and identifying unintended
consequences. GAO has also discussed the importance of these activities as
they ensure actions are taken to address risks.
Internal control monitoring should assess the quality of performance over
time and ensure that the findings of audits and other reviews are promptly
resolved.7 Internal controls help to ensure that procedures are documented
and maintained.
Difficulties Applying a Risk Management Framework
Risk management represents a unique set of challenges. Developing such a
framework is designed to guide the actions of management to prepare for
and respond to adverse events in an environment of uncertainty. As applied
to homeland security, the ability to determine the likelihood of
terrorism-related events occurring and quantifying the resulting outcomes
is balanced against the benefit as protection (security) provided at an
acceptable cost.
7GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21 .3.1, (Washington, D.C.: November 1999).
Generally, a major difficulty in executing and implementing a risk
management system occurs because of what is commonly called decision
uncertainty, that is, the assumed goal of minimizing risk does not have a
common meaning.8 For example, minimizing risk is based on values, the
measure of risk, and the comparison of values and risk. Decision
uncertainty arises when there is controversy or ambiguity concerning how
to compare and weight social objectives. Three major sources of decision
uncertainty are
o Risk measurement-although the selection of risk measurement is
both an art and a science, it must be technically correct as well
as both valid and meaningful.
o The social cost of risk-in order to make different risks
comparable, various risks often have to be quantified into
comparable quantities and placing values on cost and benefits
involves judgments.
o The quantification of social values-uncertainty surrounds the
level of risk that is acceptable or can be tolerated. That is, how
much money is to be spent on protection meaning risk reduction and
what is the cost in terms of opportunities forgone because of
finite resources. This value is dependent upon determining
society's risk attitude or tolerance and may change over time.
Coordination activities suggest that for the results of a risk management
system to be meaningful and useful, all related agencies should be using
similar methods. If agencies' methods are not compatible, then comparisons
between agencies become difficult and sector or national risk assessments
becomes less reliable. In our earlier work, we concluded that a
structured, systematic approach to risk management offers the best
assurance that activities designed to protect the homeland and combat the
effects of terrorism will produce the most effective and efficient
results.9 Specific difficulties implementing risk management systems are
contained in chapters 2, 3, and 4.
8For a more detailed discussion on this topic, see Yacov Y Haimes, Risk
Modeling, Assessment, and Management, 2nd ed. (Hoboken, New Jersey: Wiley
and Sons, 2004).
9See for example, GAO, Homeland Security: Summary of Challenges Faced in
Targeting Oceangoing Cargo Containers for Inspection, GAO-04-557T
(Washington, D.C.: Mar. 31, 2004); Homeland Security: A Risk Management
Approach Can Guide Preparedness Efforts, GAO-02-208T (Washington, D.C.:
Oct. 31, 2001); and Key Elements of a Risk Management Approach,
GAO-02-150T (Washington, D.C.: Oct. 12, 2001).
Glossary of Risk Management Terms
For purposes of our risk management framework, we use the following
definitions:
o Asset--any person, facility, material, information, or activity
that has a positive or symbolic value. An asset may have a value
to an adversary as well as to its owner, although the nature and
magnitude of these values may differ. Assets may be categorized or
combined in many ways; examples are people, information,
equipment, facilities, operations, and activities.
o Benefit--net outcome, usually translated into monetary terms; a
benefit may include both direct and indirect effects.
o Consequence--the expected worse case or reasonable worse case
impact of a successful attack. The consequence to a particular
asset can be evaluated when threat and vulnerability are
considered together. This loss or damage may be long- or
short-term in nature.
o Cost--input, both direct and indirect.
o Cost-benefit analysis--part of the management decision-making
process in which the costs and benefits of each countermeasure
alternative are compared and the most appropriate alternative is
selected. Costs include the price paid for tangible materials and
the ongoing operational costs associated with implementing the
countermeasures. Benefits are expressed in terms of the amount of
risk reduction based on the overall effectiveness of the
countermeasure with respect to the assessed vulnerabilities.
o Countermeasure--any action taken or physical equipment used
principally to reduce or eliminate one or more vulnerabilities.
The cost of a countermeasure is usually expressed in monetary
terms but may include nonmonetary costs such as reduced
operational effectiveness, unfavorable working conditions, adverse
publicity and political consequences.
o Criticality assessment--identifies and evaluates an entity's
assets or operations on the basis of a variety of factors,
including the importance of an asset or function and the
significance of a system in terms of national security, economic
activity, and public safety. A criticality assessment provides the
basis for determining which assets require greater or special
protection relative to finite resources.
o Impact--the amount of loss or damage that can be expected from
a successful attack on an asset. Loss may be monetary, but may
include loss of lives and destruction of a symbolic structure.
o Monitoring and evaluation--is a continuous repetitive
assessment process to keep a risk management process current and
relevant. It includes, among other activities, external peer
review, testing, and validation.
o Opportunity cost-the value of opportunities forgone.
o Risk-an event that has a potentially negative impact and the
possibility that such an event will occur and adversely affect an
entity's assets, activities, and operations. The principal classes
of risk from terrorism are to the general public, targets of
symbolic value, organizational, governmental, and societal
infrastructure, cyber and physical infrastructure, and economic
sectors and structures.
o Risk assessment--the process of qualitatively or quantitatively
determining the probability of an adverse event and the severity
of its impact on an asset. It is a function of threat,
vulnerability, and consequence. A risk assessment may include
scenarios in which two or more risks interact to create a greater
or lesser impact. A risk assessment provides the basis for the
rank ordering of risks and for establishing priorities for
applying countermeasures.
o Risk management--a continuous process of managing-through a
series of mitigating actions that permeate an entity's
activities-the likelihood of an adverse event and its negative
impact. Risk management addresses risk before mitigating action,
as well as the risk that remains after countermeasures have been
taken.
o Scenario--the combination of weapon and attack mode on a
specific target or critical asset (for example, the release of
sarin gas in a subway train).
o Threat--an indication of the likelihood that a specific type of
attack will be initiated against a specific target or class of
targets. It may include any indication, circumstance, or event
with the potential to cause the loss of or damage to an asset. It
can also be defined as an adversary's intention and capability to
undertake actions that would be detrimental to a valued asset.
o Threat assessment--the identification and evaluation of adverse
events that can harm or damage an asset. A threat assessment
includes the probability of an event and the extent of its
lethality. Threats may be present at the global, national, or
local level.
o Vulnerability--the probability that a particular attempted
attack will succeed against a particular target or class of
targets.
o Vulnerability assessment--the identification of weaknesses in
physical structures, personal protection systems, processes or
other areas that may be exploited. A vulnerability assessment
identifies inherent states and the extent of their susceptibility
to exploitation relative to the existence of any countermeasures.
Appendix II: Comments from the Department of Homeland Security Appendix
II: Comments from the Department of Homeland Security
Appendix III: GAOA Appendix III: GAO Contacts and Staff Acknowledgments
GAO Contacts
Margaret T. Wrightson (415) 904-2200
Stephen L. Caldwell (202) 512-9610
Staff Acknowledgments
In addition to the persons named above, David Alexander, Neil Asaba, Nancy
A. Briggs, Christine Davis, Scott Farrow, Kevin Heinz, Emily S. Pickrell,
Albert Schmidt, Stan Stenersen, April Thompson, and L. James Valverde made
key contributions to this report.
Related GA Related GAO Products
Critical Infrastructure Protection
Critical Infrastructure Protection: Challenges in Addressing
Cybersecurity. GAO-05-827T . Washington, D.C.: July 19, 2005
Critical Infrastructure Protection: Department of Homeland Security Faces
Challenges in Fulfilling Cybersecurity Responsibilities. GAO-05-434 .
Washington, D.C.: May 26, 2005.
Homeland Security: Agency Plans, Implementation, and Challenges Regarding
the National Strategy for Homeland Security. GAO-05-33 . Washington, D.C.:
January 14, 2005.
Homeland Security: Further Actions Needed to Coordinate Federal Agencies'
Facility Protection Efforts and Promote Key Practices. GAO-05-49 .
Washington, D.C.: November 30, 2004.
U.S. Postal Service: Physical Security Measures Have Increased at Some
Core Facilities, but Security Problems Continue. GAO-05-48 . Washington,
D.C.: November 16, 2004.
Drinking Water: Experts' Views on How Federal Funding Can Best Be Spent to
Improve Security. GAO-04-1098T . Washington, D.C.: September 30, 2004.
Financial Market Preparedness: Improvements Made, but More Action Needed
to Prepare for Wide-Scale Disasters. GAO-04-984 . Washington, D.C.:
September 27, 2004.
Nuclear Regulatory Commission: Preliminary Observations on Efforts to
Improve Security at Nuclear Power Plants. GAO-04-1064T . Washington, D.C.:
September 14, 2004.
U.S. Postal Service: Better Guidance Is Needed to Ensure an Appropriate
Response to Anthrax Contamination. GAO-04-239 . Washington, D.C.:
September 9, 2004.
Public Key Infrastructure: Examples of Risk and Internal Control
Objectives Associated with Certification Authorities. GAO-04-1023R .
Washington, D.C.: August 10, 2004.
Combating Terrorism: DOD Efforts to Improve Installation Preparedness Can
Be Enhanced with Clarified Responsibilities and Comprehensive Planning.
GAO-04-855 . Washington, D.C.: August 9, 2004.
Homeland Security: Transformation Strategy Needed to Address Challenges
Facing the Federal Protective Service. GAO-04-537 . Washington, D.C.: July
14, 2004.
Critical Infrastructure Protection: Improving Information Sharing with
Infrastructure Sectors. GAO-04-780 . Washington, D.C.: July 9, 2004.
Information Security: Agencies Need to Implement Consistent Processes in
Authorizing Systems for Operation. GAO-04-376 . Washington, D.C.: June 28,
2004.
National Nuclear Security Administration: Key Management Structure and
Workforce Planning Issues Remain as NNSA Conducts Downsizing. GAO-04-545 .
Washington, D.C.: June 25, 2004.
Nuclear Security: Several Issues Could Impede Ability of DOE's Office of
Energy, Science, and Environment to Meet the May 2003 Design Basis Threat.
GAO-04-894T . Washington, D.C.: June 22, 2004.
Information Security: Information System Controls at the Federal Deposit
Insurance Corporation. GAO-04-630 . Washington, D.C.: May 28, 2004.
Posthearing Questions Related to Fragmentation and Overlap in the Federal
Food Safety System. GAO-04-832R . Washington, D.C.: May 26, 2004.
Terrorism Insurance: Effects of Terrorism Risk Insurance Act of 2002.
GAO-04-720T . Washington, D.C.: April 28, 2004.
Nuclear Security: DOE Needs to Resolve Significant Issues before It Fully
Meets the New Design Basis Threat. GAO-04-623 . Washington, D.C.: April
27, 2004.
Terrorism Insurance: Implementation of the Terrorism Risk Insurance Act of
2002. GAO-04-307 . Washington, D.C.: April 23, 2004.
Critical Infrastructure Protection: Establishing Effective Information
Sharing with Infrastructure Sectors. GAO-04-699T . Washington, D.C.: April
21, 2004.
Homeland Security: Federal Action Needed to Address Security Challenges at
Chemical Facilities. GAO-04-482T . Washington, D.C.: February 23, 2004.
Posthearing Questions from the September 17, 2003, Hearing on
"Implications of Power Blackouts for the Nation's Cybersecurity and
Critical Infrastructure Protection: The Electric Grid, Critical
Interdependencies, Vulnerabilities, and Readiness." GAO-04-300R .
Washington, D.C.: December 8, 2003.
Security: Counterfeit Identification Raises Homeland Security Concerns.
GAO-04-133T . Washington, D.C.: October 1, 2003.
Nuclear Regulatory Commission: Oversight of Security at Commercial Nuclear
Power Plants Needs to Be Strengthened. GAO-03-752 . Washington, D.C.:
September 4, 2003.
Nuclear Security: DOE Faces Security Challenges in the Post September 11,
2001, Environment. GAO-03-896TNI . Washington, D.C.: June 24, 2003.
Nuclear Security: NNSA Needs to Better Manage Its Safeguards and Security
Program. GAO-03-471 . Washington, D.C.: May 30, 2003.
Information Security: Progress Made, but Challenges Remain to Protect
Federal Systems and the Nation's Critical Infrastructures. GAO-03-564T .
Washington, D.C.: April 8, 2003.
Homeland Security: EPA's Management of Clean Air Act Chemical Facility
Data. GAO-03-509R . Washington, D.C.: March 14, 2003.
Homeland Security: Voluntary Initiatives Are Under Way at Chemical
Facilities, but the Extent of Security Preparedness Is Unknown. GAO-03-439
. Washington, D.C.: March 14, 2003.
Potential Terrorist Attacks: Additional Actions Needed to Better Prepare
Critical Financial Market Participants. GAO-03-414 . Washington, D.C.:
February 12, 2003.
Potential Terrorist Attacks: More Actions Needed to Better Prepare
Critical Financial Markets. GAO-03-468T . Washington, D.C.: February 12,
2003.
Potential Terrorist Attacks: Additional Actions Needed to Better Prepare
Critical Financial Market Participants. GAO-03-251 . Washington, D.C.:
February 12, 2003.
High-Risk Series: Protecting Information Systems Supporting the Federal
Government and the Nation's Critical Infrastructures. GAO-03-121 .
Washington, D.C.: January 1, 2003.
Combating Terrorism: Actions Needed to Guide Services' Antiterrorism
Efforts at Installations. GAO-03-14 . Washington, D.C.: November 1, 2002.
Homeland Security: Department of Justice's Response to Its Congressional
Mandate to Assess and Report on Chemical Industry Vulnerabilities.
GAO-03-24R . Washington, D.C.: October 10, 2002.
Building Security: Interagency Security Committee Has Had Limited Success
in Fulfilling Its Responsibilities. GAO-02-1004 . Washington, D.C.:
September 17, 2002.
Chemical Safety: Emergency Response Community Views on the Adequacy of
Federally Required Chemical Information. GAO-02-799 . Washington, D.C.:
July 31, 2002.
Critical Infrastructure Protection: Significant Homeland Security
Challenges Need to Be Addressed. GAO-02-918T . Washington, D.C.: July 9,
2002.
Information Security: Corps of Engineers Making Improvements, but
Weaknesses Continue. GAO-02-589 . Washington, D.C.: June 10, 2002.
Security Breaches at Federal Buildings in Atlanta, Georgia. GAO-02-668T .
Washington, D.C.: April 30, 2002.
National Preparedness: Technologies to Secure Federal Buildings.
GAO-02-687T . Washington, D.C.: April 25, 2002.
Diffuse Security Threats: Technologies for Mail Sanitation Exist, but
Challenges Remain. GAO-02-365 . Washington, D.C.: April 23, 2002.
Terrorism Insurance: Rising Uninsured Exposure to Attacks Heightens
Potential Economic Vulnerabilities. GAO-02-472T . Washington, D.C.:
February 27, 2002.
Critical Infrastructure Protection: Significant Challenges in Safeguarding
Government and Privately Controlled Systems from Computer-Based Attacks.
GAO-01-1168T . Washington, D.C.: September 26, 2001.
Combating Terrorism: Actions Needed to Improve DOD Antiterrorism Program
Implementation and Management. GAO-01-909 . Washington, D.C.: September
19, 2001.
Critical Infrastructure Protection: Significant Challenges in Protecting
Federal Systems and Developing Analysis and Warning Capabilities.
GAO-01-1132T . Washington, D.C.: September 12, 2001.
Maritime Security
Coast Guard: Progress Being Made on Addressing Deepwater Legacy Asset
Condition Issues and Program Management, but Acquisition Challenges
Remain. GAO-05-757 . Washington, D.C.: July 22, 2005.
Coast Guard-Electronic Certification Procedures. B-302789. Washington,
D.C.: July 6, 2005
Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy
Assets and Acquisition Management Challenges. GAO-05-651T . Washington,
D.C.: June 21, 2005.
Maritime Security: Enhancements Made, but Implementation and
Sustainability Remain Key Challenges. GAO-05-448T . Washington, D.C.: May
17, 2005.
Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy
Assets and Acquisition Management Challenges. GAO-05-307T. Washington,
D.C.: April 20, 2005.
Maritime Security: New Structures Have Improved Information Sharing, but
Security Clearance Processing Requires Further Attention. GAO-05-394 .
Washington, D.C.: April 15, 2005.
Coast Guard: Observations on Agency Priorities in Fiscal Year 2006 Budget
Request. GAO-05-364T. Washington, D.C.: March 17, 2005.
Homeland Security: Process for Reporting Lessons Learned from Seaport
Exercises Needs Further Attention. GAO-05-17 . Washington, D.C.: January
14, 2005.
Coast Guard: Station Readiness Improving, but Resources Challenges and
Management Concerns Remain. GAO-05-161 . Washington, D.C.: January 31,
2005.
Port Security: Planning Needed to Develop and Operate Maritime Worker
Identification Card Program. GAO-05-106 . Washington, D.C.: December 10,
2004.
Maritime Security: Better Planning Needed to Help Ensure an Effective Port
Security Assessment Program. GAO-04-1062 . Washington, D.C.: September 30,
2004.
Maritime Security: Partnering Could Reduce Federal Costs and Facilitate
Implementation of Automatic Vessel Identification System. GAO-04-868 .
Washington, D.C.: July 23, 2004.
Maritime Security: Substantial Work Remains to Translate New Planning
Requirements into Effective Port Security. GAO-04-838 . Washington, D.C.:
June 30, 2004.
Coast Guard: Station Spending Requirements Met, but Better Processes
Needed to Track Designated Funds. GAO-04-695 . June 14, 2004.
Coast Guard: Key Management and Budget Challenges for Fiscal Year 2005 and
Beyond. GAO-04-636T . Washington, D.C.: April 7, 2004.
Homeland Security: Summary of Challenges Faced in Targeting Oceangoing
Cargo Containers for Inspection. GAO-04-557T . Washington, D.C.: March 31,
2004.
Coast Guard Programs: Relationship between Resources Used and Results
Achieved Needs to Be Clearer. GAO-04-432 . Washington, D.C.: March 22,
2004.
Homeland Security: Preliminary Observations on Efforts to Target Security
Inspections of Cargo Containers. GAO-04-325T . Washington, D.C.: December
16, 2003.
Posthearing Questions Related to Aviation and Port Security. GAO-04-315R .
Washington, D.C.: December 12, 2003.
Maritime Security: Progress Made in Implementing Maritime Transportation
Security Act, but Concerns Remain. GAO-03-1155T . Washington, D.C.:
September 9, 2003.
Container Security: Expansion of Key Customs Programs Will Require Greater
Attention to Critical Success Factors. GAO-03-770 . Washington, D.C.: July
25, 2003.
Homeland Security: Challenges Facing the Department of Homeland Security
in Balancing its Border Security and Trade Facilitation Missions.
GAO-03-902T . Washington, D.C.: June 16, 2003.
Coast Guard: Challenges during the Transition to the Department of
Homeland Security. GAO-03-594T . Washington, D.C.: April 1, 2003.
Transportation Security: Post-September 11th Initiatives and Long-Term
Challenges. GAO-03-616T . Washington, D.C.: April 1, 2003.
Coast Guard: Comprehensive Blueprint Needed to Balance and Monitor
Resource Use and Measure Performance for All Missions. GAO-03-544T .
Washington, D.C.: March 12, 2003.
Homeland Security: Challenges Facing the Coast Guard as It Transitions to
the New Department. GAO-03-467T . Washington, D.C.: February 12, 2003.
Coast Guard: Strategy Needed for Setting and Monitoring Levels of Effort
for All Missions. GAO-03-155 . Washington, D.C.: November 12, 2002.
Port Security: Nation Faces Formidable Challenges in Making New
Initiatives Successful. GAO-02-993T . Washington, D.C.: August 5, 2002.
Combating Terrorism: Preliminary Observations on Weaknesses in Force
Protection for DOD Deployments through Domestic Seaports. GAO-02-955TNI .
Washington, D.C.: July 23, 2002.
Risk Management
Passenger Rail Security: Enhanced Federal Leadership Needed to Prioritize
and Guide Security Efforts. GAO-05-851 . September 9, 2005
Homeland Security: Actions Needed to Better Protect National Icons and
Federal Office Buildings from Terrorism. GAO-05-790 . June 24, 2005.
Protection of Chemical and Water Infrastructure: Federal Requirements,
Actions of Selected Facilities, and Remaining Challenges. GAO-05-327 .
Washington, D.C.: March 28, 2005.
Transportation Security: Systematic Planning Needed to Optimize Resources.
GAO-05-357T . February 15, 2005
Homeland Security: Agency Plans, Implementation, and Challenges Regarding
the National Strategy for Homeland Security. GAO-05-33 . January 14, 2005.
Homeland Security: Further Actions Needed to Coordinate Federal Agencies'
Facility Protection Efforts and Promote Key Practices. GAO-05-49 .
November 30, 2004
General Aviation Security: Increased Federal Oversight is Needed, but
Continued Partnership with the Private Sector is Critical to Long-Term
Success. GAO-05-144 . November 10, 2004.
Air Traffic Control: System Management Capabilities Improved, but More Can
Be Done to Institutionalize Improvements. GAO-04-901 . August 20, 2004
Aviation Security: Challenges in Using Biometric Technologies. GAO-04-785T
. May 19, 2004.
Homeland Security: Summary of Challenges Faced in Targeting Oceangoing
Cargo Containers for Inspection. GAO-04-557T . March 31, 2004
Rail Security: Some Actions Taken to Enhance Passenger and Freight Rail
Security, but Significant Challenges Remain. GAO-04-598T . March 23, 2004.
Homeland Security: A Risk Management Approach Can Guide Preparedness
Efforts. GAO-02-208T . Washington, D.C.: October 31, 2001.
Homeland Security: Key Elements of a Risk Management Approach. GAO-02-150T
. Washington, D.C.: October 12, 2001.
(440378)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548
www.gao.gov/cgi-bin/getrpt? GAO-06-91 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Margaret Wrightson at (415) 904-2200 or
[email protected].
Highlights of GAO-06-91 , a report to Congressional Requesters
December 2005
RISK MANAGEMENT
Further Refinements Needed to Assess Risks and Prioritize Protective
Measures at Ports and Other Critical Infrastructure
Congress and the President have called for various homeland security
efforts to be based on risk management-a systematic process for assessing
threats and taking appropriate steps to deal with them. GAO examined how
three Department of Homeland Security components were carrying out this
charge:
o the Coast Guard, which has overall responsibility for security
in the nation's ports;
o the Office for Domestic Preparedness (ODP), which awards grants
for port security projects; and
o the Information Analysis and Infrastructure Protection
Directorate (IAIP), which has responsibility for developing ways
to assess risks across all types of critical infrastructure.
GAO's work focused on identifying the progress each DHS component has made
on risk management and the challenges each faces in moving further.
What GAO Recommends
This report contains many recommendations aimed at helping the three
components face their next risk management challenges. DHS, including the
Coast Guard, ODP, and IAIP, generally concurred with the report and its
recommendations. DHS said that all three components have actions under way
to address many of the recommendations in this report.
The three DHS components GAO studied varied considerably in their progress
in developing a sound risk management framework for homeland security
responsibilities. The varied progress reflects, among other things, each
component's organizational maturity and the complexity of its task (see
table below). The Coast Guard, which is furthest along, is the component
of longest standing, being created in 1915, while IAIP came into being
with the creation of the Department of Homeland Security in 2003. IAIP,
which has made the least progress, is not only a new component but also
has the most complex task-addressing not just ports but all types of
infrastructure. The Coast Guard and ODP have a relatively robust
methodology in place for assessing risks at ports; IAIP is still
developing its methodology and has had several setbacks in completing the
task. All three components, however, have much left to do. In particular,
each component is limited in its ability to compare and prioritize risks.
The Coast Guard and ODP can do so within a port but not between ports;
IAIP has not demonstrated that it can do so either within or between all
infrastructure sectors.
Each component faces many challenges in making further progress. Success
will depend partly on continuing to improve various technical and
management processes that are part of risk management. For example,
obtaining better quality data from intelligence agencies would help DHS
components estimate the relative likelihood of various types of threats-a
key element of assessing risks. In the longer term, progress will depend
increasingly on how well risk management is coordinated across agencies,
because current approaches in many ways are neither consistent nor
comparable. Also, weaving risk-based data into the annual budget cycle of
program review will be important. Supplying the necessary guidance and
coordination is what the Department of Homeland Security was set up to do
and, as the Secretary of Homeland Security has stated, what it now needs
increasingly to address. This is a key issue for the department as it
seeks to identify relative risks and take appropriate actions related to
the nation's homeland security activities.
Progress in Risk Management Is Affected by Organizational Maturity and
Complexity of Risk Management Task
Source: GAO.
*** End of document. ***