    Homeland Security: Process for Reporting Lessons Learned from
    Seaport Exercises Needs Further Attention (14-JAN-05,
    GAO-05-170).

    Seaports are a critical vulnerability in the nation's defense
    against terrorism. They are potential entry points for bombs or
    other devices smuggled into cargo ships and ports'
    often-sprawling nature presents many potential targets for
    attack. To assess the response procedures that would be
    implemented in an attack or security incident, officials conduct
    port-specific exercises. Many federal, state, and local agencies
    may potentially be involved. The Coast Guard has primary
    responsibility for coordinating these exercises and analyzing the
    results. GAO examined (1) the emerging framework for coordinating
    entities involved in security responses, (2) legal and
    operational issues emerging from exercises conducted to date, and
    (3) Coast Guard management of reports analyzing exercises. GAO
    reviewed reports on 82 exercises from fiscal year 2004 and
    observed 4 exercises as they were being conducted.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-05-170
        ACCNO:   A15500
      TITLE:     Homeland Security: Process for Reporting Lessons Learned
    from Seaport Exercises Needs Further Attention
         DATE:   01/14/2005
      SUBJECT:   Counterterrorism
    	     Defense contingency planning
    	     Facility security
    	     Harbors
    	     National preparedness
    	     Interagency relations
    	     National defense operations
    	     Reporting requirements
    	     Strategic planning
    	     Terrorism
    	     Homeland security
    	     Timeliness

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-05-170

    United States Government Accountability Office

    GAO

                           Report to Congressional Requesters

    January 2005

    HOMELAND SECURITY

    Process for Reporting Lessons Learned from Seaport Exercises Needs Further
                                       Attention

    GAO-05-170

    [IMG]

    January 2005

    HOMELAND SECURITY

    Process for Reporting Lessons Learned from Seaport Exercises Needs Further
    Attention

                                     What GAO Found

    The framework under which federal agencies would manage a port-terrorism
    incident is still evolving. The primary guidance for response, the
    National Response Plan, was just issued in January 2005, and the National
    Incident Management System, the structure for multiagency coordination, is
    still being put in place. As a result, it is too early to determine how
    well the complete framework will function in an actual incident.

    GAO's review of fiscal year 2004 terrorism-related reports and exercises
    identified relatively few legal issues, and none of these issues produced
    recommendations for statutory changes. Most issues have instead been
    operational in nature and have surfaced in nearly every exercise. They are
    of four main types: difficulties in sharing or accessing information,
    inadequate coordination of resources, difficulties in coordinating
    effectively in a command and control environment, and lack of knowledge
    about who has jurisdictional or decision-making authority.

    Reports on the exercises often do not meet the Coast Guard's standards for
    timeliness or completeness. Sixty-one percent of the reports were not
    submitted within 60 days of completing the exercise-the Coast Guard
    standard. The Coast Guard has implemented a new system for tracking the
    reports, but after a year of use, timeliness remains a concern. The Coast
    Guard has requirements for what the reports should contain, but 18 percent
    of the reports did not meet the requirement to assess each objective of
    the exercise. The Coast Guard has cited several planned actions that may
    allow for improving completeness. These actions are still in development,
    and it is too early to determine how much they will help.

    Port Terrorism Exercise Conducted at the Port of Los Angeles/Long Beach

                     United States Government Accountability Office

    Contents

           Letter                                                               1
                                            Background                          3
                                             Results                            4
                                           Conclusions                          8
                               Recommendation for Executive Action              8
                                         Agency Comments                        8
         Appendix I      Briefing on Legal and Operational Issues Identified
                                from U.S. Port Terrorism Exercises

      Appendix II Objectives, Scope, and Methodology

    Abbreviations

    AAR after-action reports
    CG Coast Guard
    CONPLAN Interagency Domestic Concept of Operations Plan
    CPS Contingency Preparedness System
    DHS Department of Homeland Security
    DOD Department of Defense
    DOJ Department of Justice
    FAA Federal Aviation Administration
    FEMA Federal Emergency Management Agency
    FBI Federal Bureau of Investigation
    FRERP Federal Radiological Emergency Response Plan
    FRP Federal Response Plan
    HSPD-5 Homeland Security Presidential Directive-5
    ICS Incident Command Structure
    INRP Initial National Response Plan
    NCP National Contingency Plan
    NIMS National Incident Management System
    NRP National Response Plan

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States Government Accountability Office Washington, DC 20548

    January 14, 2005

    The Honorable Bennie G. Thompson Ranking Minority Member Committee on
    Homeland Security House of Representatives

    The Honorable Robert E. Andrews House of Representatives

    Seaports have emerged as a critical vulnerability in the nation's defense
    against terrorism. More than 95 percent of the nation's overseas trade
    passes through these ports, and the nation's economy is highly dependent
    on an efficient transfer of goods flowing into and out of these gateways.
    Seaports are vulnerable entry points for bombs or other devices smuggled
    into cargo ships, and ports' often-sprawling nature presents many
    potential targets for attack. But, the repercussions of such an attack go
    far beyond the port immediately affected; an attack at one port could
    disrupt the broader flow of goods, creating economic consequences in the
    billions of dollars.

    In the wake of the terrorist attacks of September 11, 2001, law
    enforcement and other agencies have become more aware of the threats to
    America's strategic targets and have taken steps to make ports more
    secure. To assess coordination and response procedures that would be
    implemented in the event of a terrorist attack, officials in U.S. ports
    have conducted exercises that simulate a potential threat, attack, or
    incident. These exercises have addressed such scenarios as the explosion
    of a "dirty bomb" that releases radioactive materials, threats of an
    approaching ship that may have a bomb or other hazardous material aboard,
    or disruptive attacks on critical infrastructure or specific facilities
    within a port. In fiscal year 2004, the United States Coast Guard, which
    is the Department of Homeland Security agency with primary responsibility
    for port security, conducted 85 port-based terrorism exercises.

    Responding effectively in such exercise scenarios can be difficult.
    Depending on the nature of the incident and the particular port involved,
    dozens of federal, state, and local agencies may be involved. The incident
    may also require close coordination across many jurisdictions, raising
    issues about who has authority or how agency personnel can communicate
    effectively when they have different chains of command,

    operating procedures, and equipment. After these exercises are conducted,
    the Coast Guard requires that the unit participating in the exercise
    submit an "after-action" report describing the results and highlighting
    any lessons learned. These reports are approved for content and lessons
    learned by the commanding officer of that unit. Upon approval, each report
    is electronically transmitted to Coast Guard headquarters for inclusion in
    a master exercise database. Analysis of these reports presents an
    opportunity to identify potential barriers to an effective response during
    an actual threat or incident. These reports can also provide valuable
    input for future exercises conducted by the Coast Guard or other agencies.

    You asked that we examine what has been learned to date in the nation's
    attempts to use such exercises to coordinate effective port security
    procedures. Our specific objectives were to (1) describe the emerging
    framework under which the federal government coordinates with state and
    local entities to address a terrorist incident in a U.S. port; (2)
    identify the issues, if any, regarding federal agencies' legal authority
    that have emerged from port security exercises and what statutory actions
    might address them; (3) describe the types of operational issues being
    identified through these exercises; and (4) identify any management issues
    related to Coast Guard-developed after-action reports. On December 8,
    2004, we briefed your offices on the results of our work. Appendix I
    contains the briefing slides we presented at that meeting.1

    Our description of the emerging framework for coordinating an effective
    response to a terrorist incident is based on our review of statutes,
    regulations, directives, plans, and agency guidance, as well as interviews
    with officials from various agencies at the federal, state, and local
    levels. Our analysis of statutory and operational issues arising from
    exercises is based on a review of 82 Coast Guard after-action reports on
    terrorismrelated exercises conducted at U.S. ports in fiscal year 2004.2
    In addition, to obtain a more in-depth understanding of exercise issues,
    we attended

    1On January 6, 2005, as this report was being finalized for printing, the
    Department of Homeland Security issued the final version of the National
    Response Plan, a plan discussed in several of the briefing slides. Some
    details of the plan were changed in the final version and thus differ in a
    few respects from the information presented in the slides. These
    differences do not affect the conclusions or recommendations contained in
    this report.

    2As of December 1, 2004, all 85 after-action reports were required to be
    submitted to the Coast Guard. Three reports had not been submitted by this
    date. We conducted our content analysis on the 82 available reports.

    four exercises at various locations around the country. Our evaluation of
    management issues related to Coast Guard after-action reports included
    interviews with officials at Coast Guard headquarters and in the field,
    analysis of Coast Guard and Department of Homeland Security guidance, and
    reliance on our prior work related to managing and evaluating exercises.
    While our audit work allowed us to review a large number of exercises for
    potential legal and operational issues, our analysis is limited to the
    degree that, (1) for most exercises, we had to rely on
    Coast-Guarddeveloped after-action reports rather than direct observation
    and (2) the scope of the exercises themselves was designed around specific
    plan elements and subject to resource constraints. Appendix II describes
    our scope and methodology in more detail. We conducted our work from June
    to December 2004 in accordance with generally accepted government auditing
    standards.

                                       Background

    On November 25, 2002, the President signed into law the Homeland Security
    Act,3 which created the new federal Department of Homeland Security, and
    the Maritime Transportation Security Act,4 which created a consistent
    security program specifically for the nation's seaports. Since that time,
    and in keeping with the provisions of these new laws, the federal
    government has been developing a variety of new national policies and
    procedures for improving the nation's response to domestic emergencies.
    These policies and procedures are designed to work together to provide a
    cohesive framework for preparing for, responding to, and recovering from
    domestic incidents.

    A key element of this new response framework is the use of exercises to
    test and evaluate federal agencies' policies and procedures, response
    capabilities, and skill levels. The Coast Guard has primary responsibility
    for such testing and evaluation in the nation's ports and waterways, and
    as part of its response, it has added multiagency and multicontingency
    terrorism exercises to its training program. These exercises vary in size
    and scope and are designed to test specific aspects of the Coast Guard's
    terrorism response plans, such as communicating with state and local
    responders, raising maritime security levels, or responding to incidents
    within the port. For each exercise the Coast Guard conducts, an after

    3Pub.L. 107-296. 4Pub.L. 107-295.

    action report detailing the objectives, participants, and lessons learned
    must be produced within 60 days.

      Results

        Coordination Framework Is Still Evolving

    The framework under which federal agencies would coordinate with state and
    local entities to manage a port-terrorism incident is still evolving. As
    directed by Homeland Security Presidential Directive/HSPD-5, issued in
    February 2003, this framework is designed to address all types of
    responses to national emergencies, not just port-related events. Key
    elements of the framework have been released over the past 2 years. For
    example, the Department of Homeland Security released the Interim National
    Response Plan in September 2003 and was in the final approval stage for a
    more comprehensive National Response Plan in November 2004, as our work
    was drawing to a close. DHS announced the completion of the National
    Response Plan on January 6, 2005, too late for a substantive review to be
    included in this report. However, the finalized plan is designed to be the
    primary operational guidance for incident management and, when fully
    implemented, will incorporate or supersede existing federal interagency
    response plans. According to the updated implementation schedule in the
    National Response Plan, federal agencies will have up to 120 days to bring
    their existing plans, protocols, and training into accordance with the new
    plan. In March 2004, the department also put in place a system, called the
    National Incident Management System, which requires common principles,
    structures, and terminology for incident management and multiagency
    coordination. Although the framework that will be brought about by the
    final plan, the management system, and other actions is still in the
    implementation phase, some of the protocols and procedures contained in
    this framework were already evident at the port exercises we observed.
    However, it is still too early to determine how well the complete
    framework will function in coordinating an effective response to a
    port-related threat or incident.

    Exercises Identified Few Port security exercises have identified
    relatively few issues related to

    Legal Issues 	federal agencies' legal authority, and none of these issues
    were statutory problems according to exercise participants and agency
    officials. Our review of fiscal year 2004 after-action reports and
    observation of specific exercises showed that exercise participants
    encountered seven legal issues, but exercise participants and agency
    officials we interviewed did not recommend statutory changes to address
    these issues. In three

    instances, exercise participants made nonstatutory recommendations (such
    as policy clarifications) to assist agencies in better exercising their
    authority, but did not question the adequacy of that authority. In the
    other four instances, no recommendations were made either because
    statutory authority was deemed sufficient or, in one case, because the
    issue involved a constitutional restraint (i.e., under the Fourth
    Amendment, police are prohibited from detaining passengers not suspected
    of terrorism).

    While the exercises were conducted to examine a wide range of issues and
    not specifically to identify gaps in agencies' legal authority, the
    results of the exercises are consistent with the information provided by
    agency officials we interviewed, who indicated that sufficient statutory
    authority exists to respond to a terrorist attack at a seaport. Moreover,
    when Department of Homeland Security officials reviewed the issue of
    statutory authority, as required by Homeland Security Presidential
    Directive/HSPD5, they concluded that federal agencies had sufficient
    authority to implement the National Response Plan and that any
    implementation issues could be addressed by nonstatutory means, such as
    better coordination mechanisms.

        Exercises Identified Four Main Types of Operational Issues

    Most of the issues identified in port security exercises have been
    operational rather than legal in nature. Such issues appeared in most
    afteraction reports we reviewed and in all four of the exercises we
    observed. While such issues are indications that improvements are needed,
    it should be pointed out that the primary purpose of the exercises is to
    identify matters that need attention and that surfacing problems is
    therefore a desirable outcome, not an undesirable one. The operational
    issues can be divided into four main categories, listed in descending
    order of frequency with which they were reported:

    o  	Communication-59 percent of the exercises raised communication issues,
    including problems with interoperable radio communications among first
    responders, failure to adequately share information across agency lines,
    and difficulties in accessing classified information when needed.

    o  	Adequacy or coordination of resources-54 percent of the exercises
    raised concerns with the adequacy or coordination of resources, including
    inadequate facilities or equipment, differing response procedures or
    levels of acceptable risk exposure, and the need for additional training
    in joint agency response.

    o  	Ability of participants to coordinate effectively in a command and
    control environment-41 percent of the exercises raised concerns related to
    command and control, most notably a lack of knowledge or training in the
    incident command structure.

    o  	Lack of knowledge about who has jurisdictional or decision-making
    authority-28 percent of the exercises raised concerns with participants'
    knowledge about who has jurisdiction or decision-making authority. For
    example, agency personnel were sometimes unclear about who had the proper
    authority to raise security levels, board vessels, or detain passengers.

    After-Action Reports Show Our review of the Coast Guard's fiscal year 2004
    after-action reports from Problems Related to port terrorism exercises
    identified problems with timeliness in completing Timeliness and the
    reports and limitations in the information they contained. Specifically,

    Completeness  o  	Timeliness: Coast Guard guidance states that
    after-action reports are an extremely important part of the exercise
    program, and the guidance requires that such reports be submitted to the
    after-action report database (Contingency Preparedness System) within 60
    days of completing the exercise. However, current practice falls short: 61
    percent of the 85 afteraction reports were not submitted within this
    60-day time frame. Late reports were submitted, on average, 61 days past
    the due date. Exercises with late reports include large full-scale
    exercises designed to identify major interagency coordination and response
    capabilities. Not meeting the 60-day requirement can lessen the usefulness
    of these reports. Coast Guard guidance notes, and officials confirm, that
    exercise planners should regularly review past after-action reports when
    planning and designing future exercises, and to the extent that reports
    are unavailable, such review cannot be done. In previous reviews of
    exercises conducted by the Coast Guard and others, we found that timely
    after-action reports were necessary to help ensure that potential lessons
    can be learned and applied after each counterterrorism exercise.5 The main
    problem in producing reports on a more timely basis appeared to be one of
    competing priorities: Coast Guard field personnel indicated that other
    workload priorities were an impediment to completing reports, but most of
    them also said 60 days is a sufficient amount of time to develop and
    submit an after-action report.

    5See GAO, Combating Terrorism: Selected Challenges and Related
    Recommendations, GAO-01-822 (Washington, D.C.: Sept 20, 2001).

    Officials cited the development of the Contingency Preparedness System,
    which is the program for managing exercises and after-action reports, as a
    step allowing for a renewed emphasis on timeliness. Headquarters planning
    staff are able to run reports using this system and regularly notify key
    Coast Guard officials of overdue after-action reports. However, this
    system was implemented more than 1 year ago, in August 2003, and was,
    therefore, in place during the period in which we found a majority of
    afteraction reports were late. We did not compare our results with
    timeliness figures for earlier periods, and we, therefore, do not know the
    extent to which the system may have helped reduce the number of reports
    that are submitted late. Even if the new system has produced improvement,
    however, the overall record is still not in keeping with the Coast Guard's
    60-day requirement.

    o  	Content and quality: Coast Guard guidance also contains criteria for
    the information that should be included in an after-action report. These
    criteria, which are consistent with standards identified in our prior
    work,6 include listing each exercise objective and providing an assessment
    of how well each objective was met. However, 18 percent of the
    after-action reports we reviewed either did not provide such an
    objective-by-objective assessment or identified no issues that emerged
    from the exercise. While the scope of each exercise may contribute to a
    limited number of issues being raised, our past reviews found that
    after-action reports need to accurately capture all exercise results and
    lessons learned; otherwise, agencies may not be benefiting fully from
    exercises in which they participate. Similarly, officials at the
    Department of Defense, which like the Coast Guard conducts a variety of
    exercises as part of its training, said that if their after-action reports
    lack sufficient fundamental content, they cannot be used effectively to
    plan exercises and make necessary revisions to programs and protocols. Our
    review indicated that, in addition to the pressure of other workload
    demands, two additional factors may be contributing to limitations in
    report content and quality-current review procedures and a lack of
    training for planners. Headquarters planning officials noted that local
    commands have primary responsibility for reviewing after-action reports
    and that limited criteria exist at headquarters for evaluating the content
    of reports submitted by these commands. At the field level, many planners
    with whom we spoke said they were unaware of any written documentation or
    exercise-planning guidance they could refer to when developing an
    after-action report.

    6See GAO-01-822.

      Conclusions

    Recommendation for Executive Action

      Agency Comments

    The Coast Guard has cited several planned actions that may allow for
    improved content and quality in after-action reports. These actions
    include updating exercise management guidance and promulgating new
    instructions related to preparing after-action reports and collecting
    lessons learned. While these initiatives may address issues of content and
    quality in after-action reports, they are currently still in the
    development phase.

    A successful response to a terrorist threat or incident in a seaport
    environment clearly requires the effective cooperation and coordination of
    numerous federal, state, local, and private entities-issues that exercises
    and after-action reports are intended to identify. Complete and timely
    analyses of these exercises represent an important opportunity to identify
    and correct barriers to a successful response. The Coast Guard's inability
    to consistently report on these exercises in a timely and complete manner
    represents a lost opportunity to share potentially valuable information
    across the organization. The Coast Guard's existing requirements, which
    include submitting these reports within 60 days and assessing how well
    each objective has been met, appear reasonable but are not being
    consistently met. Coast Guard officials cited a new management system as
    their main effort to making reports more timely, but this system has been
    in place for more than a year, and timeliness remains a problem. It is
    important for Coast Guard officials to examine this situation to determine
    if more needs to be done to meet the standard. The Coast Guard has several
    other steps under development to address issues of report content and
    completeness, and it is too early to assess the effect these actions will
    have. For this set of actions, it will be important for the Coast Guard to
    monitor the situation to help ensure that exercises can achieve their full
    and stated purpose.

    To help ensure that reports on terrorism-related exercises are submitted
    in a timely manner that complies with all Coast Guard requirements, we are
    making one recommendation, that the Commandant of the Coast Guard review
    the Coast Guard's actions for ensuring timeliness and determine if further
    actions are needed.

    We provided DHS, DOJ, and DOD with a draft of this report for review and
    comment. The Coast Guard generally concurred with our findings and
    recommendation and did not provide any formal comments for inclusion in
    the final report. DOJ and DOD also did not have any official comments.

    DOD provided two technical clarifications, which we have incorporated to
    ensure the accuracy of our report.

    As agreed with your offices, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days from
    its issue date. At that time, we will send copies of this report to the
    Secretary of Homeland Security, the Commandant of the Coast Guard,
    appropriate congressional committees, and other interested parties. The
    report will also be available at no charge on GAO's Web site at
    http://www.gao.gov.

    If you or your staffs have any questions about this report, please contact
    me at (415) 904-2200 or by email at wrightsonm@gao.gov, or Steve Caldwell,
    Assistant Director, at (202) 512-9610 or by email at caldwells@gao.gov, or
    Steve Calvo, Assistant Director, at (206) 287-4839 or by email at
    calvos@gao.gov. Other key contributors to this report were Christine
    Davis, Wesley Dunn, Michele Fejfar, Lynn Gibson, Dawn Hoff, David Hudson,
    Dan Klabunde, Ryan Lambert, and Stan Stenersen.

    Margaret T. Wrightson Director, Homeland Security

    and Justice Issues

    Appendix II: Objectives, Scope, and Methodology

    The objectives of this report were to (1) describe the emerging framework
    under which the federal government coordinates with state and local
    entities to address a terrorist incident in a U.S. port; (2) identify the
    issues, if any, regarding federal agencies' legal authority that have
    emerged from port security exercises and what statutory actions might
    address them; (3) describe the types of operational issues being
    identified through these exercises; and (4) identify any management issues
    related to Coast Guarddeveloped after-action reports. To address these
    objectives, we reviewed relevant legislation, regulations, directives and
    plans, analyzed agency operational guidance and Coast Guard after-action
    reports (AARs), interviewed a variety of federal officials, and observed
    several port security exercises.

    To identify the emerging framework to address a terrorist incident in a
    U.S. port, we reviewed relevant statutes such as the Homeland Security Act
    of 2002 and the Maritime Transportation Security Act of 2002 and
    implementing maritime regulations at 33 CFR, parts 101 to 106. We also
    reviewed Homeland Security Presidential Directive/HSPD- 5 and Presidential
    Decision Directive 39. Operational plans that were included in our
    analysis included the Initial National Response Plan, the Interagency
    Domestic Terrorism Concept of Operations plan (CONPLAN), Interim Federal
    Response Plan, and the National Response Plan "Final Draft." We also
    reviewed agency guidance related to exercise planning and evaluation such
    as the Coast Guard Exercise Planning Manual and Contingency Preparedness
    Planning Manual, as well as the Department of Homeland Security/ Office of
    Disaster Preparedness' Exercise and Evaluation Program. Findings were
    supplemented with interviews of key officials in federal agencies,
    including the Coast Guard (CG), the Department of Homeland Security (DHS),
    Department of Defense (DOD), Department of Justice (DOJ), and related
    federal maritime entities such as Project Seahawk.

    To provide a framework for evaluating agencies' legal authority in
    responding to a terrorist incident in a U.S. port, we adopted a case study
    methodology because it afforded a factual context for the emergence of
    legal issues that could confront agencies in the exercise of their
    authority. Our efforts included attending four U.S. port based terrorism
    exercises (Los Angeles, Calif.; Hampton Roads, Va.; Charleston, S.C.;
    Philadelphia, Pa.), reviewing CG AARs for fiscal year 2004, and conducting
    in-person and telephone interviews with DHS, CG, DOJ, DOD, and Project
    Seahawk. The port exercises we selected to visit were geographically
    diverse and each was conducted in either August or September of fiscal
    year 2004. Additional criteria for exercise selection included the
    strategic importance

    Appendix II: Objectives, Scope, and Methodology

    of the port (as defined by the Maritime Administration), the variety of
    terrorism scenarios to be exercised, and the federal, state, and local
    players involved. The AARs we reviewed were based on a list of all fiscal
    year 2004 exercises provided to us by the CG. We focused on any
    contingency that included terrorism and then requested AARs for those
    completed exercises from the CG. According to CG guidance, AARs are
    required to be submitted within 60 days of exercise completion. To
    ascertain compliance with this guidance, CG personnel provided us with the
    dates that AARs for terrorism-related exercises were received at
    headquarters. We used this information, in conjunction with the exercise
    start and stop dates, to determine which reports were on time, which were
    late, and the average time late reports were submitted beyond the 60-day
    requirement.

    While issues of a legal nature did surface during our observation of
    exercises and analysis of AARs, exercise participants and agency officials
    did not recommend statutory changes for these issues. We generally relied
    upon the agency's position as to whether legislation was necessary and did
    not independently assess the need for legislation by auditing the specific
    issues identified in the exercises.

    To identify operational issues that occurred during port terrorism
    exercises, we relied extensively on perspectives gained through our
    observations at the four port terrorism exercises as well as a
    comprehensive review of the available AARs for operational issues based on
    criteria we developed. In order to determine the frequency of various
    operational issues identified in the CG's AARs, we noted instances that
    each subcategory within the major category appeared. These categories and
    subcategories were chosen through exercise observation and an initial
    review of available AARs by two independent analysts. This allowed us to
    identify operational issues that were consistent across the terrorism
    exercises. We used the following major categories and subcategories (which
    appear in parentheses)

    o  	Communication (communication interoperability issues, communication
    policy or protocols between or within agencies, information sharing
    between agencies),

    o  	Command and Control/ Incident Command Structure (NIMS/ICS training,
    UC/IC information flow),

    Appendix II: Objectives, Scope, and Methodology

    o  	Unclear Decision Making/ Jurisdictional Knowledge (unclear decision
    making authority, unclear lead authority, unclear
    authorities/jurisdictions of other agencies), and

    o  	Resource Coordination/ Capabilities (response capabilities, response
    coordination/joint tactics).

    To analyze the reports, two GAO analysts independently reviewed each
    report and coded operational issues based on the above subcategories. The
    results of each analysis were then compared and any discrepancies were
    resolved. Overall percentages for the major categories were determined
    based on whether any of the issues were identified under the respective
    subcategories. The maximum number of observations for any major category
    was equal to one, regardless of the number of times a subcategory was
    recorded.

    To identify management concerns regarding the CG's AARs, we reviewed our
    previous studies on this issue as well as CG and DHS issued guidance on
    exercise management, such as the Coast Guard's Exercise Planning Manual
    and Contingency Preparedness Planning Manual Volume III. Our analysis also
    included in-person interviews with CG exercise management officials from
    headquarters and CG planners in the field to gain additional information
    on how terrorism exercises are planned and evaluated as well as how
    lessons learned are cataloged and disseminated. To ascertain the effect of
    untimely CG AARs (CG AARs are required to be completed within 60 days of
    exercise completion), we also interviewed exercise management experts from
    DOD. We conducted a content analysis of the available AARs to determine
    the weaknesses in the reports and where deviations from CG protocol were
    taking place.

    We conducted our work from June to December 2004 in accordance with
    generally accepted government auditing standards.

      GAO's Mission

    Obtaining Copies of GAO Reports and Testimony

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
    released reports, testimony, and correspondence on its Web site. To have
    GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone 	The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
    Orders for 100 or more copies mailed to a single address are discounted 25
    percent. Orders should be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

      To Report Fraud, Contact:

    Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

    E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
    424-5454 or (202) 512-7470

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
    512-4400Congressional U.S. Government Accountability Office, 441 G Street
    NW, Room 7125 Relations Washington, D.C. 20548

    Public Affairs 	Susan Becker, Acting Manager, BeckerS@gao.gov (202)
    512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

                               PRINTED ON RECYCLED PAPER
    *** End of document. ***