Coast Guard: Changes to Deepwater Plan Appear Sound, and Program
Management Has Improved, but Continued Monitoring Is Warranted
(28-APR-06, GAO-06-546).
The Deepwater program was designed to produce aircraft and
vessels that would function in the Coast Guard's traditional
at-sea roles. After the terrorist attacks of September 11, 2001,
however, the Coast Guard began taking on additional homeland
security missions, and so it revised the Deepwater implementation
plan to provide assets that could better meet these new
responsibilities. While many acknowledge that the Coast Guard's
aging assets need replacement or renovation, concerns exist about
the approach the Coast Guard adopted in launching the Deepwater
program. The subsequent changes in the program's asset mix and
delivery schedules only increased these concerns. This report (1)
compares the revised Deepwater implementation plans with the
original plan in terms of the assets to be replaced or modified,
and the time frames and costs for doing so; (2) assesses the
degree to which the operational effectiveness model and other
analytical methods used by the Coast Guard to develop the revised
Deepwater asset mix are sound and appropriate for such a purpose;
and (3) assesses the progress made in implementing GAO's prior
recommendations regarding program management. GAO is not making
any new recommendations in this report.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-546
ACCNO: A52728
TITLE: Coast Guard: Changes to Deepwater Plan Appear Sound, and
Program Management Has Improved, but Continued Monitoring Is
Warranted
DATE: 04/28/2006
SUBJECT: Homeland security
Interoperability
Procurement planning
Program management
Strategic planning
Equipment upgrades
Program implementation
Coast Guard Deepwater Project
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-546
* Terrorist Attacks Have Led to Increased Emphasis on Homeland
* Revised Plans Propose Replacing Fewer Aircraft and Adjusting
* Delivery Schedules for Deepwater Assets Have Changed
* Estimated Cost of Revised Deepwater Plans is $7 Billion Hig
* Deepwater Costs Could Rise if Funding Deviates from Levels C
* Computer-Based Model Used in Analyzing Capacity Gaps Is Cred
* Other Factors Also Affected Revised Deepwater Asset Mix
* Model Indicates That the Revised Asset Mix Will Provide Impr
* Coast Guard Is Exploring Options for Applying Further Modeli
* Coast Guard's Efforts to Improve Oversight and Program Manag
* Put in Place a Human Capital Plan to Ensure Adequate Staffin
* More Time Needed to Determine Adequacy of Steps Taken to Imp
* Effects of Steps Taken to Control Future Costs through Compe
* Order by Mail or Phone
United States Government Accountability Office
Report to Congressional Requesters
GAO
April 2006
COAST GUARD
Changes to Deepwater Plan Appear Sound, and Program Management Has Improved, but
Continued Monitoring Is Warranted
GAO-06-546
COAST GUARD
Changes to Deepwater Plan Appear Sound, and Program Management Has
Improved, but Continued Monitoring Is Warranted
What GAO Found
The revised Deepwater implementation plans change the balance between new
and legacy assets, alter the delivery schedule for some assets, lengthen
the overall acquisition schedule by 5 years, and increase the projected
program cost from $17 billion to $24 billion. The higher cost generally
relates to upgrading assets to reflect added homeland security mission
requirements. Upgrades to vessels account for the single largest area of
increase; with upgrades to the command, control, communications and other
capabilities being second highest. In contrast, because the revised plans
upgrade rather than replace most legacy aircraft and reduce the number of
unmanned aircraft, the cost for Deepwater aircraft drops. The revised
plans, like the original plan, are heavily dependent on receiving full
funding each year. Coast Guard officials state that a shortfall in funding
in any year could substantially increase total costs.
The Coast Guard's analytical methods were appropriate for determining if
the revised asset mix would provide greater mission performance and
whether the mix is appropriate for meeting Deepwater missions. GAO and
other independent experts found the Coast Guard's methods were reliable
for assessing the effects of changing the asset mix and a Department of
Defense review board facilitated accreditation of the Coast Guard's
approach. Because the model has proved useful for guiding Coast Guard
decisions on the proper asset mix for achieving Deepwater performance
goals, the Coast Guard is considering ways to expand the model to guide
decisions on meeting its Coast Guard-wide performance goals.
Actions by the Coast Guard and the system integrator have fully
implemented three of the eight GAO recommendations that were not fully
addressed during GAO's review in 2005, and three more recommendations
appear to be nearly implemented. The remaining two have unresolved
concerns, but the Coast Guard is taking steps to resolve them. A program
of this size, however, will likely experience other challenges beyond
those that have emerged so far, making continued monitoring by the Coast
Guard important.
United States Government Accountability Office
Contents
Letter Results in Brief BackgroundModel Used to 1 3
Determine Revised Asset Mix Is Reliable, and the 5 20
Coast Guard Hopes to Expand Its Use Progress 28
Continues in Making Recommended Improvements 39
Concluding Observations Agency Comments 39
Appendix I Objectives, Scope, and Methodology 41
Appendix II GAO Contacts and Staff Acknowledgments 44
GAO Related Products 45
Tables Table 1: Key Changes in Asset Capabilities
between the Original and Revised Deepwater
Implementation Plans Table 2: Number of
Deepwater Aircraft and Vessels Currently in
Operation Compared to Those Planned under the
Original and Revised Deepwater Implementation
Plans Table 3: Summary of Cost Differences
between the Original Deepwater Implementation
Plan and the 2005 and 2005 Revised Plans Table
4: Projected Performance of the Asset Mix from
the Revised Implementation Plans Compared to the
Asset Mix from the Original Deepwater Plan Based 10
on Latest DMOES Modeling Table 5: Deepwater 12
Missions, Performance Metrics, and Assessment 18
Results from Fiscal Year 2004 Table 6: Status of 25
GAO Recommendations to the U.S. Coast Guard 28
Regarding Management of the Deepwater Program 30
Figures
Figure 1: Comparison of the Delivery Schedules for Aircraft under
the Original (2002) and the 2005 and 2006 Revised
Deepwater Implementation Plans 14
Figure 2: Comparison of the Delivery Schedules for Vessels
under
the Original (2002) and the 2005 and 2006 Revised
Deepwater Implementation Plans 16
Abbreviations
ASI Acquisitions Solutions, Inc.
DHS Department of Homeland Security
DMOES Deepwater Maritime Operational Effectiveness Simulation
DOD Department of Defense
GPRA Government Performance and Results Act
IPT integrated product teams
MSMP Modeling and Simulation Master Plan
PGA performance gap analysis
TOC total ownership cost
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
April 28, 2006
The Honorable Olympia J. Snowe Chairwoman, Subcommittee on Fisheries
and the Coast Guard United States Senate
The Honorable Maria Cantwell Ranking Member, Subcommittee on Fisheries
and the Coast Guard United States Senate
The nation's new homeland security realities have affected plans for
modernizing the Coast Guard's fleet of aircraft and vessels. For about a
decade, the Coast Guard has been developing an Integrated Deepwater System
(or Deepwater) acquisition program, a long-term plan to replace or
modernize this fleet. Many of these legacy assets are at or approaching
the end of their estimated service lives. 1 As originally conceived,
Deepwater was designed around producing aircraft and vessels that would
function in the Coast Guard's traditional at-sea roles, such as
interdicting illicit drug shipments or rescuing mariners from difficulty
at sea. After the terrorist attacks on September 11, 2001, however, these
aircraft and vessels began taking on additional missions related to
protection of ports, waterways, and coastal areas. As a result, the Coast
Guard began revising the Deepwater implementation plan to provide
replacement assets that could better address these added responsibilities.
To do so, the Coast Guard used an operational effectiveness model and
other methods to help determine what mix of assets it needed and what
their capabilities should be. 2 In August 2005, the Coast Guard issued the
revised Deepwater implementation plan detailing the assets it planned to
modify or acquire,
1
For purposes of this report, we use the term "legacy assets" to refer to
the existing fleet of Deepwater aircraft and vessels. These legacy assets
include the HC-130H and HU-25 fixedwing aircraft, the HH-60 and HH-65
rotary-wing aircraft, the 378-foot high-endurance cutters, the 210-foot
and 270-foot medium-endurance cutters, and the 110-foot and 123-foot
patrol boats. In addition, three other vessels are generally included as
part of the Deepwater fleet-the 213-foot Acushnet, the 230-foot Storis,
and the 282-foot Alex Haley.
2
"Capacity" models produce a force size and mix of assets designed to meet
a specified level of demand based on certain assumptions. The "operational
effectiveness" model used by the Coast Guard simulated Deepwater
operations and projected the extent to which the asset mix could meet
specified program goals.
along with the proposed cost and schedule for doing so. Then, in February
2006, the Coast Guard again updated its Deepwater plan to align with its
fiscal year 2007 budget submissions.
While there is widespread acknowledgment that many of the Coast Guard's
aging assets need replacement or renovation, concerns also exist about the
acquisition approach the Coast Guard adopted in launching the Deepwater
program. From the outset, we have expressed concern about the risks
involved with the Coast Guard's acquisition strategy, which involves
relying on a prime contractor (or system integrator) to identify the
assets needed and then using tiers of subcontractors to design and build
the actual assets. The subsequent changes in the Deepwater asset mix and
delivery schedules only increase these concerns. In 2004 we reported that
well into the contract's second year, key components needed to manage the
program and oversee the system integrator's performance had not been
effectively implemented. 3 Accordingly, we made 11 recommendations to
address three broad areas of concern: improving program management,
strengthening contractor accountability, and promoting cost control
through greater competition among potential subcontractors.
This report, prepared at your request, examines the changes the Coast
Guard has made in the Deepwater program to address its broader scope, as
well as addressing the concerns we raised in 2004. More specifically, it
o compares the revised Deepwater implementation plan issued in August
2005 and the February 2006 updated plan with the original (August
2002) plan in terms of the assets to be replaced or modified, and the
time frames and costs for doing so;
o assesses the degree to which the operational effectiveness model and
other analytical methods used by the Coast Guard to develop the
revised Deepwater asset mix are sound and appropriate for such a
purpose; and
o assesses the progress made in implementing our prior recommendations
regarding Deepwater program management.
GAO, Contract Management: Coast Guard's Deepwater Program Needs Increased
Attention to Management and Contractor Oversight, GAO-04-380 (Washington,
D.C.: Mar. 9, 2004).
Page 2 GAO-06-546 Coast Guard
Results in Brief
Our work included extensive reviews and analyses of (1) the original and
revised Deepwater implementation plans, (2) the Coast Guard's operational
effectiveness models and other analytical tools used for determining the
proposed Deepwater asset mix, and (3) documentation provided by the Coast
Guard on its progress in addressing our recommendations. We supplemented
our document reviews and analyses with extensive discussions with
officials at the Deepwater Program Executive Office, as well as with
interviews of key Coast Guard operations and maintenance officials,
contract monitors, and representatives of the system integrator. We
conducted our work between August 2005 and March 2006 in accordance with
generally accepted governmental auditing standards. Appendix I describes
our objectives, scope, and methodology in greater detail.
Results in Brief
To reflect added homeland security responsibilities based on the terrorist
attacks of September 11, 2001, the Coast Guard's August 2005 revision and
February 2006 update to the Deepwater implementation plan change the
balance between new assets to be acquired and legacy assets to be upgraded
and alter the schedule for delivering many of these assets. Overall, the
acquisition schedule has been lengthened by 5 years, with the final assets
scheduled for delivery in 2027. Further, the revised plans increase
overall program costs from the original estimate of $17 billion to $24
billion. The higher costs of the revised plans relate generally to
upgrading the Deepwater assets to reflect post September 11, 2001 mission
requirements and include such things as improved capabilities to operate
in conditions of chemical, biological, and radiological contamination;
greater antiterrorism weaponry; development of airborne use of force
capabilities; improved communications systems; and enhanced flight decks.
Costs for enhancing and upgrading the capabilities of the planned
Deepwater replacement vessels account for the largest portion of the $7
billion increase. Specifically, vessels account for $5.5 billion of the
increase in the 2005 plan and for $5.9 billion in the 2006 plan. Further,
upgrades to the command, control, communications, computers, intelligence,
surveillance, and reconnaissance (C4ISR) and maritime domain awareness
capabilities of the Deepwater assets account for the second largest
portion of the increase, accounting for an increase of $1.1 billion in the
2005 revised plan and $663 million in the 2006 plan. In contrast, because
the revised plans call for upgrading many of the legacy aircraft rather
than replacing them with new assets, as called for in the original plan,
and acquisitions of unmanned aerial vehicles have been scaled back, the
overall costs for Deepwater aviation assets have dropped in the revised
plans-by about $600 million in the 2005 plan and by about $400 million in
the 2006 plan. Affordability, however, continues to be a risk for the
Deepwater program. Like the original plan, the revised Deepwater plans are
heavily dependent on receiving a sustained level of funding at planned
levels over the life of the program. Coast Guard officials stated that a
shortfall in funding in any given year-for example, because of competing
budget priorities-could cause costs for the Deepwater program to rise
substantially.
The operational effectiveness model and other analytical methods the Coast
Guard used have proved useful for guiding decisions on the revised
Deepwater asset mix. The primary tool the Coast Guard used for determining
the revised asset mix was a computer simulation model that projected the
operational effectiveness of a variety of potential Deepwater force
structures. Using this tool, the Coast Guard determined that the revised
asset mix would provide greater mission performance. In performing our
review of the Coast Guard's model, we reviewed computer simulation model
criteria developed by an authority in the field of simulation modeling and
found that the model successfully addressed these criteria. Further, a
Department of Defense review board facilitated accreditation of the model
and another group with expertise in this type of modeling has studied the
Coast Guard's approach and concluded that it is reliable. Because the
model has proved useful for guiding Coast Guard decisions on the proper
asset mix for enhancing the mission performance of the Deepwater assets,
the Coast Guard is considering ways to expand the model to guide decisions
on meeting Coast Guard-wide performance goals.
The Coast Guard, in conjunction with its system integrator, has taken
steps to fully implement three of the eight recommendations that were not
sufficiently addressed as of our last review in 2005. These deal with
putting in place a human capital plan to help ensure adequate staffing for
the Deepwater program, improving input from Coast Guard representatives
who assess the system integrator's performance to Deepwater program
managers, and holding the system integrator accountable for improving the
effectiveness of integrated product teams. Three other recommendations
appear close to being fully implemented in that the actions taken appear
to be sufficient, but results are not yet known or final procedural steps
(such as issuing a policy currently in draft form) have not been
completed. The remaining two recommendations, both of which deal with
implementing effective program management and contractor oversight, remain
problematic. For example, effective management of the Deepwater program
depends heavily on strong collaboration among the Coast Guard, the system
integrator, and the
Background
subcontractors. However, despite a number of Coast Guard actions to
facilitate communication to and collaboration with the system integrator
and subcontractors, Coast Guard Deepwater performance monitors note that
collaboration among subcontractors remains inconsistent. The Coast Guard
has initiated the steps needed to address this issue, but it is too early
to tell if these will effectively eliminate the problems.
We provided a draft copy of this report to the Department of Homeland
Security and the U.S. Coast Guard for review. The U.S. Coast Guard
provided technical comments, which have been incorporated where
appropriate.
Background
As the lead federal agency for maritime homeland security within the
Department of Homeland Security, the Coast Guard is responsible for a
variety of missions, including ensuring security in ports and waterways
and along coastlines, conducting search and rescue missions, interdicting
drug shipments and illegal aliens, enforcing fisheries laws, and
responding to reports of pollution. The Deepwater fleet, which currently
consists of 186 aircraft and 88 vessels of various sizes and capabilities,
plays a critical role in all of these missions.
Some Coast Guard Deepwater vessels were built in the 1960s.
Notwithstanding extensive overhauls and other upgrades, a number of the
vessels are nearing the end of their estimated service lives. Similarly,
while a number of the Deepwater legacy aircraft have received upgrades in
engines, operating systems, and sensor equipment since they were
originally built, they too have limitations in their operating
capabilities. The Integrated Deepwater System acquisition program, which
the Coast Guard began developing in 1996, is its major effort to replace
or modernize these aircraft and vessels. This Deepwater program is
designed to replace some assets-such as deteriorating vessels-with new
assets, and to upgrade other assets-such as some types of helicopters-so
they can meet new performance requirements.
The Deepwater program represents a unique approach to a major acquisition
in that the Coast Guard is using a prime contractor-the system
integrator-to identify and deliver the assets needed to meet a set of
mission requirements the Coast Guard has specified. 4 In 2002, the Coast
Guard awarded a contract to Integrated Coast Guard Systems (ICGS), a joint
venture of Lockheed Martin and Northrop Grumman, as the system integrator
for the Deepwater program. Lockheed Martin and Northrop Grumman, as the
two main subcontractors, in turn contract with other subcontractors.
Rather than using the traditional approach of replacing classes of ships
or aircraft through a series of individual acquisitions, the Coast Guard
chose to employ a system-of-systems acquisition strategy that would
replace its deteriorating Deepwater assets with a single, integrated
package of new or modernized assets. This system-of-systems approach is
designed to provide an improved, integrated system of aircraft, vessels,
and unmanned aerial vehicles to be linked effectively through systems that
provide command, control, communications, computer, intelligence,
surveillance, reconnaissance, and supporting logistics. The Deepwater
program's three overarching goals are to maximize operational
effectiveness, minimize total ownership cost, 5 and satisfy the customer-
the operational commanders, aircraft pilots, cutter crews, maintenance
personnel, and others who will use the assets.
We have been reviewing the Deepwater program for several years, pointing
out successes as well as difficulties and expressing concern over a number
of facets comprising the program. In 2001, we identified several areas of
risk for Deepwater. 6 First, the Coast Guard faced potential risk in the
overall management and day-to-day administration of the contract. At the
time, we reported on the major challenges in developing and implementing
plans for establishing effective human capital practices, having key
management and oversight processes and procedures in place, and tracking
data to measure system integrator performance. In addition, we expressed
concerns about the potential lack of competition during the program's
later years and the reliance on a single system integrator for procuring
the Deepwater assets. We also reported there was little evidence
4
The mission requirements include such things as he ability to (1) respond
to 90 percent of all distress incidents within 2 hours; (2) detect and
track targets of any material such that the probability of detection is at
least 90 percent for small targets, such as a person in the water or a
single-engine civil aircraft; and (3) respond to National Emergency
Response Operations with 48 hours.
5
Total ownership cost is the sum of all costs associated with the research,
development, procurement, personnel, training, operation, logistical
support, and disposal of the entire Deepwater system.
6
GAO, Coast Guard: Progress Being Made on Deepwater Project, but Risks
Remain, GAO-01-564 (Washington, D.C.: May 2, 2001).
that the Coast Guard had analyzed whether the approach carried any
inherent risks for ensuring the best value to the government and if so,
what to do about them.
We reviewed the Deepwater program again in 2004 and found many of the same
concerns. 7 Specifically, we reported that key components needed to manage
the program and oversee the system integrator's performance had not been
effectively implemented. The Coast Guard's primary tool for overseeing the
system integrator, integrated product teams (IPT), were struggling to
effectively collaborate and accomplish their missions because of changing
membership, understaffing, insufficient training, and inadequate
communication among members. Also, the Coast Guard had not adequately
addressed the frequent turnover of personnel in the program and the
transition from existing assets to those assets that will be part of the
Deepwater program moving forward. Further, the Coast Guard's assessment of
the system integrator's performance in the first year of the contract
lacked rigor, and the factors that formed the basis for the award fee were
unsupported by quantifiable measures. This resulted in the system
integrator receiving an award fee of $4.0 million out of a maximum of $4.6
million despite documented problems in schedule, performance, cost
controls, and contract administration.
At the time of our 2004 report, the Coast Guard had begun to develop
models to measure the extent to which Deepwater was achieving operational
effectiveness and had reduced total ownership cost, but it had not made a
decision as to which specific models would be used. Further, Coast Guard
officials were not able to project a time frame for when the Coast Guard
would be able to hold the contractor accountable for progress toward the
goals of maximizing operational effectiveness, minimizing total ownership
cost, and increasing customer satisfaction. Additionally, the Coast Guard
had not measured the extent of competition among suppliers of Deepwater
assets or held the system integrator accountable for taking steps to
achieve competition. At the time, the Coast Guard's lack of progress on
these issues had contributed to our concerns about the Coast Guard's
ability to rely on competition as a means to control future programmatic
costs. In response to these concerns, we made a number of recommendations
to improve Deepwater management and oversight of the system integrator. In
2005, we reported that the Coast
GAO, Coast Guard: Key Management and Budget Challenges for Fiscal Year
2005 and Beyond, GAO-04-636T (Washington, D.C.: Apr. 7, 2004); and
GAO-04-380.
Page 7 GAO-06-546 Coast Guard
Guard had fully addressed three of the recommendations and had actions
underway on others. 8
For the past several years, the Coast Guard has been revising its
Deepwater plan to incorporate expanded homeland security requirements it
received after the terrorist attacks of September 11, 2001. On May 31,
2005, the Coast Guard submitted a revised implementation plan to the House
Subcommittee on Homeland Security, Committee on Appropriations, which
included both a 20-year and a 25-year plan. The House Appropriations
Committee directed the Department of Homeland Security and the Coast Guard
to select a single revised implementation plan to accompany the Deepwater
fiscal year 2006 budget request. In compliance with the Committee's
direction, the Coast Guard Commandant testified on July 21, 2005 to the
25-year revised Deepwater implementation plan. Further, in February 2006,
the Coast Guard submitted an updated Deepwater implementation plan to
align with its fiscal year 2007 budget submission. These 2005 and 2006
revised plans are the ones we are using to compare to the Coast Guard's
August 26, 2002, original implementation plan. 9
To reflect added homeland security responsibilities based on the terrorist
attacks of September 11, 2001, the August 2005 revision and February 2006
update to the Deepwater implementation plan change the balance of upgraded
legacy versus new assets, the delivery schedules, and program costs from
the original 2002 plan. For aircraft, the revised plans include upgrading
many of the legacy aircraft rather than replacing them with new assets as
called for in the original plan. For vessels, the revised plans maintain
the original plan's strategy of replacing all of the legacy vessels, but
include some changes in the number of small boats being acquired. Overall,
the revised plan (1) increases the program length by 5 years, to a total
of 25 years; (2) changes the delivery schedules for a number of assets;
and (3) increases overall costs to $24 billion, $7 billion more than
earlier estimates. The program's higher costs largely reflect the Coast
Guard's expanded homeland security responsibilities and cover such
8GAO, Coast Guard: Progress Being Made on Addressing Deepwater Legacy
Asset Condition Issues and Program Management, but Acquisition Challenges
Remain,
GAO-05-757 (Washington, D.C.: July 22, 2005).
For purposes of this report, we will refer to the 2005 revised plan and
the 2006 update of the implementation plan as revised plans.
Page 8 GAO-06-546 Coast Guard
Terrorist Attacks Have Led to Increased Emphasis on Homeland Security and
Enhanced Deepwater Asset Capabilities
changes as greater weaponry, improved communications systems, and greater
operating capabilities. Coast Guard officials caution, however, that this
25-year program is heavily dependent on receiving the anticipated budget
amount each fiscal year. If full funding is not available in any given
year-for example, because of competing budget priorities-the shortfall
could have cascading effects on overall costs for the Deepwater program.
Terrorist Attacks Have Led to Increased Emphasis on Homeland Security and
Enhanced Deepwater Asset Capabilities
The original Deepwater plan, while published in 2002, was developed before
the terrorist attacks of September 11, 2001. It reflected an emphasis on
the Coast Guard's traditional Deepwater missions, such as conducting
search and rescue operations at sea, preventing and mitigating oil spills
and other threats to the marine environment, inspecting foreign vessels,
protecting important fishing grounds, and stemming the flow of illegal
drugs and migrants into the United States. After the events of September
11, 2001, the revised plans took into account the increased security
threats by incorporating a new mission to provide greater security for
ports, waterways, and coastal areas and enhancing the capabilities of the
Deepwater assets to better meet the increased threats. In particular, the
revised plans call for equipping Deepwater helicopters to provide warning
and disabling weapons fire at sea and in ports, waterways, and coastal
areas. Further, while the original plan called for assets to have
Deepwater interoperability-meaning that all Deepwater aircraft and vessels
could communicate with one another-the revised plans call for Deepwater
assets to also have interoperability with assets from the Departments of
Homeland Security and Defense, as well as with the Coast Guard's Rescue21
(R21) project. 10 According to Coast Guard officials, this increased
interoperability involves such things as adding circuits and data
transmission capability to allow for more reliable and secure
communication. Table 1 provides further information on some of the key
differences between Deepwater asset capabilities in the original and
revised plans.
10
Rescue-21 is a coastal command and control communication system designed
to improve search and rescue efforts and other missions, such as
interdiction of drugs and migrants. The program includes very
high-frequency-FM radios, communication towers, and communication centers.
Page 9 GAO-06-546 Coast Guard
Table 1: Key Changes in Asset Capabilities between the Original and Revised
Deepwater Implementation Plans
Deepwater asset Key changes in asset capabilities
Aircraft
HC-130 Long-Range Surveillance Aircraft Interoperability expanded to
include DOD, DHS, Rescue-21 assets, and local first respondersa Equipment
installed for detecting chemical, biological, and radiological dangers
Equipment installed for enhancing maritime patrol surveillance
capabilities Increased ability to provide nationwide strategic airlift
capabilities
CN-235 Medium-Range Surveillance Aircraft Interoperability expanded to
include DOD, DHS, Rescue-21 assets, and local first responders Equipment
installed for detecting chemical, biological, and radiological dangers
Equipment installed for enhancing maritime patrol surveillance
capabilities
HH-60 Medium-Range Recovery Helicopter Interoperability expanded to
include DOD, DHS, Rescue-21 assets, and local first responders Equipment
installed for detecting chemical, biological, and radiological dangers
Equipment installed for enhancing maritime patrol surveillance
capabilities Upgraded to provide airborne use of force and vertical
insertion and delivery capabilities
HH-65 Multi-Mission Cutter Helicopter Interoperability expanded to include
DOD, DHS, Rescue-21 assets, and local first responders Equipment installed
for detecting chemical, biological, and radiological dangers
Equipment installed for enhancing maritime patrol surveillance
capabilities Upgraded to provide airborne use of force and vertical
insertion and delivery capabilities
HV-911 Vertical Takeoff and Landing Equipment installed for detecting
chemical, biological, and radiological dangers Unmanned Aerial Vehicle
RQ-4A High-Altitude Endurance Unmanned No changes from the original plan.
Aerial Vehicle
Vessels
National Security Cutter Interoperability expanded to include DOD, DHS,
Rescue-21 assets, and local first responders Equipment installed for
detecting chemical, biological, and radiological dangers Flight deck
increased and enhanced to accommodate DOD and DHS helicopters Weapon
systems upgraded Improved classified communication capabilities Underwater
detection capabilities enhanced
Deepwater asset Key changes in asset capabilities
Offshore Patrol Cutter Interoperability expanded to include DOD, DHS,
Rescue-21 assets, and local first responders Equipment installed for
detecting chemical, biological, and radiological dangers Flight deck
increased and enhanced to accommodate DOD and DHS helicopters Weapon
systems upgraded Improved classified communication capabilities Cruising
speed increased from 22 to 28 knots
Fast Response Cutter Interoperability expanded to include DOD, DHS,
Rescue-21 assets, and local first responders Equipment installed for
detecting chemical, biological, and radiological dangers Use of 40-year
composite hull rather than steel hull Underwater detection capabilities
enhanced Increase in transit speed from 28 to 30 knots
Long-Range Interceptor Interoperability expanded to include DOD, DHS,
Rescue-21 assets, and local first responders
Short-Range Prosecutor Interoperability expanded to include DOD, DHS,
Rescue-21 assets, and local first responders
Revised Plans Propose Replacing Fewer Aircraft and Adjusting the Mix of
Vessels to Be Acquired
Source: GAO analysis of documentation provided by the U. S. Coast Guard
a
Local first responders include police, firefighters, and emergency medical
professionals.
Revised Plans Propose Replacing Fewer Aircraft and Adjusting the Mix of
Vessels to Be Acquired
The revised plans change the final mix of Deepwater aircraft more
significantly than the mix of vessels. For example, the original plan
called for replacing all 41 HH-60 Medium-Range Recovery Helicopters with
34 AB-139 helicopters. Under the revised plans, the Coast Guard will
upgrade the HH-60s and not purchase any AB-139 helicopters. Coast Guard
officials said they elected to retain the HH-60s because they determined
that the AB-139 aircraft was unsuitable to meet new requirements for
weaponry and for tactical operations. Retaining and upgrading HH-60
helicopters cost $500 million less than replacing them. Another major
change in aircraft involved retaining more HC-130s to meet long-range
surveillance, search and rescue, and airlift needs. For vessels, the
revised plans retain the original plan's approach of replacing all cutters
and patrol boats. The only change to the number of vessels is that the
revised plans include nine additional 25-foot short range boats and nine
fewer 35-foot long range boats than were included in the original plan.
Table 2 compares the number and types of Deepwater assets under the
original and revised plans.
Table 2: Number of Deepwater Aircraft and Vessels Currently in Operation
Compared to Those Planned under the Original and Revised Deepwater
Implementation Plans
Under original Under 2005 and Aircraft Currenta plan 2006 revised plans
HC-130 Long-Range Surveillance Aircraft 27 6 22
HU-25 Medium-Range Surveillance Aircraft 23 0
HH-60 Medium-Range Recovery Helicopter 41 0 42
HH-65 Multi-Mission Cutter Helicopter 95 93
CN-235 Medium-Range Surveillance Aircraft 0 35
AB-139 Medium-Range Recovery Helicopter 0 34
HV-911 Vertical Takeoff and Landing Unmanned Aerial Vehicle 0 69
RQ-4A High-Altitude Endurance Unmanned Aerial Vehicle 0 7
Aircraft totals 186 244
Vessels
378-foot High-Endurance Cutter 12 0
425-foot National Security Cutterb 0 8
270-foot Medium-Endurance Cutter 13 0
350-foot Offshore Patrol Cutterb 0 25
210-foot Medium-Endurance Cutter 14 0
140-foot Fast Response Cutterb 0 58
110-foot and 123-foot Patrol Boats 49 0
35-foot Long-Range Interceptorb 0 42
25-foot Short-Range Prosecutorb 0 82
Vessel totals 88 215
Delivery Schedules for Deepwater Assets Have Changed
Source: Developed by GAO from data provided by the U.S. Coast Guard.
Note: The number of assets did not change between the 2005 and 2006
revised plans.
a
As of February 2006.
b
The length of all new vessels Is subject to change based on the maturity
of the design.
Delivery Schedules for Deepwater Assets Have Changed
Estimated delivery schedules for the Deepwater assets have changed. For
some of the aircraft, deliveries have been projected for later years than
were estimated in the original plan. For example, the Coast Guard now
plans for delivery of its first 3 CN-235 Medium-Range Surveillance
Aircraft during calendar year 2008. Under the original plan, the Coast
Guard had anticipated delivery of the first 12 in 2006, with a total of 18
delivered by the end of 2008. Final deliveries of the CN-235s under the
2006 revised plan are now scheduled for 2027, as opposed to 2012 under the
original plan. According to the Coast Guard, the delivery schedule for the
CN-235 Medium-Range Surveillance Aircraft was delayed because the Coast
Guard did not receive the anticipated level of funding in fiscal years
2002 and 2003, which required renegotiations. Figure 1 shows the original
and revised delivery schedules for Deepwater aircraft.
Figure 1: Comparison of the Delivery Schedules for Aircraft under the
Original (2002) and the 2005 and 2006 Revised Deepwater Implementation
Plans
Source: GAO (analysis), U.S. Coast Guard (data).
For vessels, the revised plans generally spread out deliveries of each
class of vessel over a larger number of years, as shown in figure 2. For
example, the original plan called for delivery of 58 of the 140-foot Fast
Response Cutters between 2018 and 2022. The revised plans call for
delivering the first Fast Response Cutter in 2007 or 2008, with additional
cutters being delivered every year from 2009 through 2027-a span of 21
years. The Coast Guard originally planned to convert its legacy 110-foot
patrol boats to 123-foot patrol boats and, beginning in 2018, replace the
123-foot patrol boats with 140-foot Fast Response Cutters. However, the
patrol boat conversion project was halted after the first 8 patrol boats
because the 123-foot patrol boats could not meet post September 11, 2001
mission requirements and were experiencing technical difficulties. Because
of this, the Coast Guard needed to advance the delivery of the Fast
Response Cutters.
Figure 2: Comparison of the Delivery Schedules for Vessels under the
Original (2002) and the 2005 and 2006 Revised Deepwater Implementation
Plans
Source: GAO (analysis), U.S. Coast Guard (data).
Estimated Cost of Revised Deepwater Plans is $7 Billion Higher, Largely
Reflecting Increased Homeland Security Mission Requirements
The total estimated cost of the revised Deepwater plans increased by $7
billion over the original plan-from $17 billion to $24 billion. According
to the Coast Guard, most of the $7 billion increase is due to enhanced
homeland security mission requirements brought about by the events of
September 11, 2001. In particular, data provided by the Coast Guard show
that most of the $7 billion increase is attributable to costs for
enhancing and upgrading the capabilities of the planned Deepwater
replacement vessels. More specifically, as shown in table 3, upgrades to
the Deepwater vessels account for about $5.5 billion of the increase in
the 2005 plan, and $5.9 billion in the 2006 update.
Beyond the increases related solely to vessels, upgrades to the C4ISR and
maritime domain awareness capabilities to improve interoperability between
the Coast Guard and other Department of Homeland Security components, as
well as with the Department of Defense, account for the second largest
category of cost increases-increasing by $1.1 billion in the 2005 revised
plan, and by $663 million in the 2006 plan. In contrast, because the
revised plans include upgrading the HC-130 aircraft and the HH-60
helicopter rather than replacing them as called for in the original plan
and for scaling back on the number of unmanned aerial vehicles to be
acquired, costs for Deepwater aircraft decreased from the original plan to
the revised plan. Overall, costs for Deepwater aircraft were reduced by
about $600 million in the 2005 plan and by about $400 million in the 2006
plan from the amount included in the original plan.
Table3: Summary of Cost Differences between the Original Deepwater
Implementation Plan and the 2005 and 2005 Revised Plans
FY 2002 FY 2005 Change from FY 2006 Change from Deepwater asset Aircraft
original plana revised plan original plan revised plan original plan
HC-130 Long-Range Surveillance Aircraft 48.1 $392.6 $344.5 $380.5 $332.4
CN-235 Medium-Range Surveillance Aircraft 1,270.4 1,590.2 319.8 1,637.7
AB-139 Medium-Range Recovery Helicopter 896.6 (896.6) (896.6)
HH-60 Medium-Range Recovery Helicopter 100.1 446.1 346.0 454.0
HH-65 Multi-Mission Cutter Helicopter 1,140.3 575.0 (565.3) 560.6 (579.7)
HV-911 Vertical Takeoff and Landing Unmanned Aerial Vehicle 624.8 503.3
(121.5) 521.0 (103.8)
Airborne Use of Force (HH-60/HH-
65) b b
90.2 90.2
Aircraft subtotals ($0.6 billion) ($0.4 billion)
Vessels
National Security Cutter $1,838.1 $2,875.1 $1,037.0 $2,875.9 $1037.8
Offshore Patrol Cutter 4,204.2 7,055.7 2,851.5 7,228.0 3,023.8
110-foot to 123-foot patrol boat conversion 363.3 (363.3) 178.5 (184.8)
Fast Response Cutter 1,496.3 3,226.3 1,730.0 3,297.7 1,801.4
Small boats (Long-Range Interceptor and Short-Rang Prosecutor) 130.3 78.9
(51.4) 80.0 (50.3)
Legacy Cutter Sustainment 26.3 338.0 311.7 315.2
Vessels subtotals $5.5 billion $5.9 billion
Other
Technology Obsolescence Prevention 1,106.1 1,630.6 524.5 1,642.6
C41SR Capability/Maritime Domain Awareness 748.0 1,847.5 1,099,5 1,411.4
Integrated Logistics System 343.5 430.7 87.2 452.0 108.5
Systems Engineering and Integration 1,209.7 1,397.6 187.9 1,069.5 (140.2)
Government Program Management 1,441.4 1,526.1 84.7 1,810.1 368.7
Other subtotals 2.0 billion 1.5 billion
Totals 17.0 billion 23.9 billion 6.9 billion 24.0 billion 7.0 billion
Source: GAO analysis based on documentation provided by the U. S. Coast
Guard.
Notes: All costs are in millions of dollars unless otherwise noted.
The RQ-4A High-Altitude Endurance Unmanned Aerial Vehicle is not included
in this table since this aircraft is not being acquired as a capital
investment; rather the Coast Guard plans to procure the surveillance data
that the aircraft will provide.
a
The costs for the fiscal year 2002 plan reflected in this table were
updated in fiscal year 2004 dollars to reflect the lack of funding enacted
in fiscal years 2002 and 2003.
b
The costs shown for Airborne Use of Force ($75.2 million) were included as
part of the total costs for the HH-60 and HH-65 helicopters.
According to the Coast Guard, the primary elements of the enhanced
homeland security mission requirements that contributed to the $7 billion
increase include the following:
o Chemical, biological, and radiological detection and defense. For this
element, the additional capabilities included in the revised plans
vary by asset. The most extensive are for the National Security
Cutter, which is to have a sealed section within which crew can
operate the ship in a contaminated environment for limited time
periods. In the event an area is contaminated, such as from a
terrorist attack, the crew can use radar, heat-seeking sensors, and
other equipment to determine what is occurring-such as whether engines
are operating, vessels are being moved, or people are alive. Other
Deepwater vessels and aircraft are to be equipped with exposure suits
and storage for those suits.
o Antiterrorism and force protection. The revised plans call for more
powerful weapons for National Security Cutters, Offshore Patrol
Cutters, and Fast Response Cutters. Manual gun mounts on cutters will
be replaced with selected sensor-integrated, remote-operated, and
semi-automated gun systems. This weaponry is to give the Coast Guard
enhanced capabilities to protect its own cutters and other high value
assets by, for example, providing cutters with the ability to stop
terrorists who have taken control of a ship by disabling that ship's
propulsion with precision fire.
o Airborne use of force and vertical insertion and delivery. The revised
plans call for the Deepwater helicopters to be fitted with weapons and
equipment that will enable armed teams to land on a vessel, such as in
the event a hostile group has taken over the vessel. Crew members can
use machine guns to provide cover while a team travels by rope from
the hovering helicopter to the vessel's deck. Additionally, for
certain terrorist and criminal scenarios, the helicopter can use
disabling fire to stop an illegally operated boat. In the event of a
terrorist attack and the right circumstances, the disabling fire can
be changed to deadly fire if necessary to stop terrorists.
o Interoperability with the Departments of Defense and Homeland
Security, as well as Rescue-21 equipment. All Deepwater vessels and
aircraft are to receive C4ISR enhancements that make them
interoperable with other DHS entities, DOD assets, and local first
responders. These enhancements include added circuits and equipment
that provide full voice communication and limited data communications
between these entities.
o Extended/enhanced flight deck. The flight decks of the National
Security Cutter and Offshore Patrol Cutter are to be enlarged so that
helicopters from other Department of Homeland Security components and
from DOD can land on the cutters.
Deepwater Costs Could Rise if Funding Deviates from Levels Called For in the
Plans
In May 2001, we reported that affordability was the biggest risk for the
Deepwater program because the Coast Guard's contracting approach depends
on a sustained level of funding each fiscal year over the life of the
program. 11 For the 2005 revised implementation plan, these funding levels
average over $1 billion per year and range from $650 million to over $1.5
billion per year through fiscal year 2026. According to Coast Guard
officials, any significant or sustained deviation from the planned funding
levels would be costly to the Coast Guard in the short term and set off
ripples affecting the acquisition of Deepwater equipment for years to
come. The officials added that significant shortfalls would likely result
in increased costs, late delivery of equipment, and degradation of
Deepwater asset performance.
Model Used to Determine Revised Asset Mix Is Reliable, and the Coast Guard
Hopes to Expand Its Use
In revising the Deepwater asset mix to meet new mission demands, the Coast
Guard undertook a series of analyses that used a computer simulation model
to project the operational effectiveness of a variety of potential
Deepwater force structures or asset mixes. We found that this model
contains reliable information and is useful for guiding decisions on the
revised Deepwater asset mix. Further, a Department of Defense review board
facilitated accreditation of the model and another group with expertise in
this type of modeling has studied the Coast Guard's approach and concluded
that it is reliable. Through use of this model, the Coast Guard projects
that the Deepwater asset mix in the $24 billion revised implementation
plan will provide greater mission performance than the
11
GAO, Coast Guard: Actions Needed to Mitigate Deepwater Project Risks,
GAO-01-659T (Washington, D.C.: May 3, 2001).
Page 20 GAO-06-546 Coast Guard
Computer-Based Model Used in Analyzing Capacity Gaps Is Credible
asset mix in the original plan. Other factors beyond this model, such as
decisions of internal working groups and projected funding, also
contributed to the adoption of the revised Deepwater asset mix. Because
the model has proved useful for guiding Coast Guard decisions on the
proper asset mix for enhancing the mission performance of the Deepwater
assets, the Coast Guard is considering ways to expand the model to guide
decisions on meeting its Coast Guard-wide Government Performance and
Results Act (GPRA) performance goals. 12
Computer-Based Model Used in Analyzing Capacity Gaps Is Credible
After the events of September 11, 2001, the Coast Guard undertook a series
of analyses intended to determine what capability and capacity gaps would
exist if the asset mix in the original Deepwater plan were applied to the
revised Deepwater missions. To conduct this analysis, the Coast Guard
projected the performance of a variety of asset mixes using a
computerbased operational effectiveness simulation model known as the
Deepwater Maritime Operational Effectiveness Simulation (DMOES). 13 Using
three different capacity models, the Coast Guard generated three different
versions of the asset mix needed to meet Coast Guard performance targets.
14 The resulting force structures were then modeled in DMOES to
12
Government Performance and Results Act of 1993, Pub. L. No. 103-62: 107
Stat. 285 (1993).
13
Simulation is a technology that allows the analysis of complex systems
through statistically valid means. Through a software interface, the user
creates a computerized version of a design or process, otherwise known as
a "model." DMOES is a multi-mission "campaign-plus" level model.
"Campaign" level models are used by the Department of Defense to tie
together all aspects of a battlefield for a defined period of time. DMOES,
as a "campaign-plus" level model, simulates Deepwater operations for a
full year under multiple demand levels and is used to translate asset and
capability contributions into system operational effectiveness. The model
used program targets compiled in the Coast Guard's Modeling and Simulation
Master Plan (MSMP). These targets were developed specifically for the
purpose of judging effectiveness of the modeled force structures and,
ultimately, operational effectiveness of the Deepwater asset mix
acquisition. The MSMP targets are based on, but not identical to, the
Coast Guard's GPRA targets and business plan goals. The Coast Guard has
cited two reasons for setting different targets in the MSMP: (1) The MSMP
targets project 25 to 40 years out and anticipate some performance growth,
whereas the actual targets are revised year to year; and (2) because DMOES
is a Deepwater-specific model and does not leverage non-Deepwater asset
contributions reflected in Coast Guardwide GPRA targets, the target values
account for Deepwater asset contribution only.
14
Capacity models produce a force size and mix designed to meet a specified
level of demand, based on certain assumptions. The three capacity models,
each with their own strengths and weaknesses, incorporated Coast Guard
analyses of current and forecast performance capability and capacity gaps
to develop proposed force structures for three levels of demand, using the
capabilities of the asset types in the original Deepwater implementation
plan.
project their operational effectiveness. The results of this assessment
led the Coast Guard to change the asset mix for its revised Deepwater
plan.
We found that DMOES, which provided important evidence for Deepwater
operational effectiveness analyses, contains reliable information for
decision making. Specifically, our review of various statistical aspects
of DMOES indicates that the parameters used in the DMOES model-the
targets, missions, weather events, and probability of target detection
present in the Deepwater environment-appear to be the result of a thorough
and rigorous process that enhanced the model's credibility. In performing
our review of DMOES, we reviewed computer simulation model criteria
developed by an authority in the field of simulation modeling and found
that the DMOES model successfully addressed these criteria. 15 For
example, the parameters used were derived from historical events (e.g.,
target detection or weather events), which helped satisfy the criterion
that interactions between the modeled system and the outside environment
be considered. To ensure use of valid and current data for its major
updates of DMOES, Coast Guard gathered updated historical data and
compared these data to data from past events. Further, because the Coast
Guard modeled target detection capabilities for the assets at less than
their full potential, the asset's target detection capabilities do not
appear to be overstated.
In addition, independent authorities, in their reviews of DMOES, have
assessed the model and have accredited it for force structure planning.
For example, the MITRE Corporation, in an independent analysis of the
performance gap analysis process (of which DMOES was a key component),
found that the process and the resulting analytic results were "likely the
most complete and comprehensive campaign-level study conducted by any
uniformed service in recent times." 16 Further, the Coast Guard submitted
DMOES to a verification, validation, and accreditation 17 review monitored
and facilitated by the Joint Accreditation Support
15
Jim Ledin, Simulation Engineering, CMP Media (Lawrence, Kansas: September 2001).
16
MITRE, Center for Enterprise Modernization, Independent Assessment of U.S.
Coast Guard Deepwater Performance Gap Analysis Process (McLean, Virginia:
March 30, 2004). MITRE is a not-for-profit organization with expertise in
systems engineering, information technology, operational concepts, and
enterprise modernization, chartered to work in the public interest.
17
The simulation modeling criteria defines "verification and validation" as
confirming that the model has been correctly implemented.
Activity. 18 The DMOES Accreditation Review Board, consisting of Coast
Guard officials and external experts in the field of military force
structure determinations and capability-based planning, conducted the
actual review and accredited the DMOES model for acquisition support and
force structure planning.
Other Factors Also Affected Revised Deepwater Asset Mix
While the capability and capacity gaps identified in the performance gap
analysis process were a key input into the decisions leading to the revised
Deepwater asset mix, they were not the only factor. The Coast Guard also
shaped the Deepwater asset mix based on budget considerations and information
developed by an internal working group. In particular:
o Coast Guard officials stated that affordability was
a key factor in shaping
the revised Deepwater asset mix. According to the
officials, Deepwater
was never intended to be an unconstrained
acquisition program, and the
$24 billion force structure was determined through
a process of modeling
performance of anticipated asset mixes, weighed
against expected funding
levels over the life of the program, to come up
with an optimal balance of
performance and affordability. As a result, the
revised Deepwater asset
mix was developed to maximize the system's
capabilities and capacities
within this $24 billion budget. The officials added
that while the $24 billion
budget may not allow for all desired capabilities
on each asset, capabilities
are being designed for later refit, if applicable.
Further, in April 2004, the Assistant Commandant
o for Operations
Capability commissioned an Aviation Legacy
Alternatives Working Group
to analyze possible alternatives to the aviation
force structure in the
original Deepwater plan.19 This working group
provided key data used to
enhance the performance gap analysis process. For
example, as a result of
the working group's analyses, the Coast Guard
decided to convert and
upgrade two of its four legacy aircraft (HC-130 and
HH-60) and replace
18
A Department of Defense resource that provides such services for modeling
and simulation activities.
19
This work was commissioned because, according to the Coast Guard, the
original plan's asset mix would fall short in meeting the Coast Guard's
minimum mission requirements for organic lift capability, maritime domain
awareness, and search and rescue treaty coverage for fixed-wing aircraft,
and airborne use of force, vertical insertion, and Federal Aviation
Administration icing standards for rotary-wing aircraft. All alternative
solutions considered by the working group would meet these requirements.
GAO did not assess any of the analyses prepared by the working group.
Model Indicates That the Revised Asset Mix Will Provide Improved Mission
Performance over the Original Plan
only the HU-25. This strategy was deemed by the Coast Guard to be the most
cost-effective solution for meeting Deepwater mission requirements.
According to the Coast Guard, other alternatives added significant
capacity, but at a greater cost. 20
Model Indicates That the Revised Asset Mix Will Provide Improved Mission
Performance over the Original Plan
The most recent DMOES runs conducted by the Coast Guard, published in
October 2005, project that the revised Deepwater asset mix will provide "a
significant improvement in traditional Coast Guard mission performance"
compared to the original Deepwater asset mix. 21 This marked the first
time that the Coast Guard used DMOES to model the operational
effectiveness of the revised Deepwater asset mix. According to the Coast
Guard, the projected improvement in the overall mission performance of the
revised asset mix is due mainly to increased maritime surveillance
aircraft and, more specifically, to the converted HC-130, which will be
present in greater numbers and with greater capabilities than the
comparable longrange surveillance aircraft from the original Deepwater
plan. Table 4 provides a brief summary of the results of our analysis of
the latest DMOES modeling in terms of how the revised asset mix is
projected to improve performance for the Coast Guard's various Deepwater
missions. Appendix I provides more details on our analysis of the latest
DMOES modeling.
20
For example, one alternative reviewed involved aligning the Deepwater
aircraft mix with capacity and capability levels identified in the
performance gap analysis (PGA) process. This alternative would have
contained greater capacity and capability than the selected alternative
but, according to the Coast Guard, was not adopted because it would be too
costly. This is an example of how potential solutions identified in the
PGA process informed, but did not dictate, the Coast Guard's decision in
adopting its asset mix.
21
MicroSystems Integration, Inc., United States Coast Guard Integrated
Deepwater System Capability and Capacity Performance Gap Analysis (PGA) IV
Part 4 (Pawcatuck, Connecticut: October 17, 2005).
Table 4: Projected Performance of the Asset Mix from the Revised
Implementation Plans Compared to the Asset Mix from the Original Deepwater
Plan Based on Latest DMOES Modeling
GAO assessment of the impact of the revised asset mix on Deepwater mission
Deepwater mission performance
Maritime Safety
Search and rescue Little improvement projected: Increased availability of
fixed- and rotary-wing aircraft to provide event response
International ice patrol No improvement projected: International ice
patrol missions were not modeled because the assets did not change between
the original and revised Deepwater plans
Maritime security
General law enforcement Little to some improvement projected: Increased
availability of fixed- and rotary-wing aircraft to provide event response,
though performance could be decreased in the western United States because
of the relocation of major cutters to other patrol areas and missions.
Alien migrant interdiction operations Some to moderate improvement
projected: Increased availability of fixed- and rotary-wing aircraft to
provide tactical surveillance and increased prosecution capabilities
Drug interdiction Little to some improvement projected: Increased
availability of fixed- and rotary-wing aircraft to provide tactical
surveillance and increased prosecution capabilities
Living marine resources enforcement-exclusive economic zones Moderate to
good improvement projected: Increased availability of fixed- and
rotary-wing aircraft to provide tactical surveillance and to conduct
intercepts of identified targets of interest. Mission performance could be
decreased in Alaska, however, because of a lack of strategic surveillance
capability
Maritime homeland security
Moderate improvement projected: Increased availability of fixed- and
rotary-wing aircraft to provide tactical surveillance as well as increased
naval operational capabilities, such as the increased speed, endurance,
and improved flight decks of the new cutters. This mission may have to be
performed by non-Deepwater assets in some locations, however, because of
competing priorities for the Deepwater assets
Protection of natural resources
Living marine resources enforcement (domestic) Little to some improvement
projected: Increased fixed-wing surveillance capacity and increased
prosecution capabilities. This mission may be negatively affected in some
western locations, however, because of the reallocation of major cutter
patrols, as well as the change in type and number of rotary-wing assets
available to cover high-threat areas.
Foreign vessel inspection No improvement projected. This mission is
largely served by non-Deepwater assets, so the impact of the Deepwater
asset mix is limited under either plan.
Lightering zone enforcement Some improvement projected: Increased
rotary-wing aircraft availability
GAO assessment of the impact of the revised asset mix on Deepwater mission
Deepwater mission performance
Maritime pollution enforcement and response No improvement projected. This
mission is largely served by non-Deepwater assets, so the impact of the
Deepwater asset mix is limited under either plan.
National defense
General defense operations Good improvement projected: Improved naval
operational Maritime Intercept operations capabilities, such as increased
speed, endurance, and improved Marine environmental response operations
flight decks of the new cutters. Port operations security and defense
Coastal sea control operations
Theater security cooperation Good improvement projected: Improved naval
operational capabilities, such as increased speed, endurance, and improved
flight decks of the new cutters
Non-Deepwater aviation demand/Maritime domain awareness
Non-Deepwater aviation demand No improvement projected. The data used to
represent this mission demand in the model was not complete enough to
truly capture all elements of demand; U.S. Coast Guard projects that more
robust data in future model runs will show greater projected performance
under the revised asset mix.
Maritime domain awareness Some improvement projected: Increased
availability of fixed-wing aircraft to provide surveillance.
Coast Guard Is Exploring Options for Applying Further Modeling for
Projecting Coast Guardwide Performance Capabilities
Source: GAO analysis of data provided by the U.S. Coast Guard.
Notes: The categories of projected improvement of the revised Deepwater
asset mix compared to the original Deepwater asset mix are based on GAO
analysis of U.S. Coast Guard data and are as follows:
0% chance of improvement projected = No improvement projected 25% chance
of improvement projected = Little improvement projected 50% chance of
improvement projected = Some improvement projected 75% chance of
improvement projected = Moderate improvement projected 100% chance of
improvement projected = Good improvement projected.
"Improvement" in this context indicates performance of the revised
Deepwater asset mix that is one full standard deviation greater than that
of the original Deepwater asset mix, as presented by the U.S. Coast Guard.
Coast Guard Is Exploring Options for Applying Further Modeling for
Projecting Coast Guardwide Performance Capabilities
Though DMOES was an accredited, rigorous simulation model effective in
supporting Deepwater force structure planning, it does not capture the
impact of non-Deepwater asset contributions and, therefore, does not
provide a means for the Coast Guard to estimate the extent to which its
entire fleet of aircraft and vessels will allow it to meet Coast
Guard-wide GPRA performance targets. The Coast Guard is aware of this
limitation and is exploring options for expanding DMOES to encompass all
Coast Guard assets-both Deepwater and non-Deepwater-in an effort to
provide for a true analysis of Coast Guard-wide mission performance
capabilities. While this has not yet occurred, Coast Guard officials told
us they were reasonably confident that the cumulative effect of merging
the revised Deepwater assets with its non-Deepwater assets would allow the
Coast Guard to meet GPRA targets for those missions involving Deepwater
aircraft and vessels. In the interim, the Coast Guard has taken steps to
measure the impact of Deepwater assets on Deepwater-related metrics. Since
2002, the Coast Guard has annually reviewed-and plans to continue
reviewing-the most recent complete year's worth of data and estimates the
Deepwater-only contribution toward meeting performance goals for seven
particular performance metrics. These performance metrics and results for
the most recent year available are shown in table 5.
Table 5: Deepwater Missions, Performance Metrics, and Assessment Results
from Fiscal Year 2004
Performance Performance
goal results
Search and rescue Percentage of lives saved after 93.0 92.6
Coast Guard
notification in cases with Deepwater
participation
Cocaine seizure rate Percentage of cocaine 6.5 Not determined
shipped through
transit zone that is seized by Deepwater
assets
Percentage
Illegal or undocumented migrant of total
interdiction migrant 32.0 41.9
flow
interdicted
with Deepwater assets
Foreign fishing vessel Percentage of
interdiction foreign fishing 6.7 4.5
vessels
detected in U.S. Economic Enforcement
Zone interdicted with Deepwater assets.
Protection of living marine Percentage of
resources Deepwater living 97 94.8
marine
resources law enforcement boardings
without significant violations.
National defense/military Maintain 100 percent 100.0 99.3
readiness combined readiness of
high-endurance cutters and patrol boats to
support Department of Defense
requirements
International ice patrol Maintain 95 percent 98.0 97.8
accuracy of all limits of
all known ice broadcastsa
Progress Continues in Making Recommended Improvements
Source: Analysis by GAO based on data provided by the U.S. Coast Guard.
a
The Coast Guard regularly collects data on ice conditions whenever
icebergs threaten primary shipping routes between Europe and the United
States and Canada. The Coast Guard uses this information to predict the
drift of icebergs along these shipping routes and broadcasts this
information for the benefit of all vessels transiting the North Atlantic.
These messages are referred to as the "limit of all known ice broadcasts."
Disaggregating performance data to reflect Deepwater-only contributions
provides an estimate of the extent to which the Deepwater fleet is helping
the Coast Guard meet these key targets on an annual basis. For example, as
a result of these efforts, the Coast Guard determined that Deepwater
assets saved 92.6 percent of lives at risk after Coast Guard notification
in fiscal year 2004, which is slightly below the Deepwater asset target
value of 93 percent.
Progress Continues in Making Recommended Improvements
Our past concerns about the Deepwater program have been in three main
areas-ensuring better program management and contractor oversight,
ensuring greater accountability on the part of the system integrator, and
creating sufficient competition to help act as a control on costs-and we
made a total of 11 recommendations to address these concerns. During our
2005 review, we determined that the Coast Guard had addressed and
Page 28 GAO-06-546 Coast Guard
fully implemented 2 of these 11 recommendations. The Coast Guard disagreed
with and declined to implement a separate recommendation that pertained to
updating its cost baseline to determine whether the Deepwater acquisition
approach is costing more than a conventional acquisition approach. While
we stand behind our original recommendation, we decided not to pursue it
further because the Coast Guard determined that the cost to implement this
recommendation was excessive. Thus, at the time we began our current
review, 8 of the 11 recommendations were not yet fully implemented. On the
basis of information we gathered for this review, we consider 3 of these 8
recommendations to be fully implemented. The Coast Guard is in the process
of taking actions to implement 3 more recommendations, but full
implementation is dependent on seeing results or completion of actions
that are not yet in final form. The 2 remaining recommendations, both
relating to overall program management and oversight, remain problematic.
One relates to improving the effectiveness of integrated product teams,
the other to providing field personnel with guidance and training on
transitioning to new Deepwater assets. In each case, the Coast Guard has
taken actions, but our review of program reports and our discussions with
program and field personnel indicate the problems still remain. In all
cases, however, the steps needed to fully implement these recommendations
seem relatively clear.
Table 6 provides an overview of the 11 recommendations. The sections below
discuss the recommendations made in each of the three areas of concern,
describing the initial issue that led to the recommendation, the steps
taken to date to address it, and our rationale for considering the
recommendation as being fully implemented or not. Where we make a
determination that a recommendation has not yet been implemented, we
indicate what actions are needed.
Table 6: Status of GAO Recommendations to the U.S. Coast Guard Regarding
Management of the Deepwater Program
Recommendation Areas of concern Recommendations to the U.S. Coast Guard
status
Key components of management and Put in place a human capital plan to
ensure adequate Implemented oversight are not effectively implemented
staffing of the Deepwater program
Improve integrated product teams responsible for Partially implemented
managing the program by providing better training, approving charters, and
improving systems for sharing information between teams
Provide field personnel with guidance and training on Partially
implemented transitioning to new Deepwater assets
Procedures for ensuring contractor Develop measurable award fee criteria
consistent with Implementeda accountability are inadequate guidance from
the Office of Federal Procurement Policy
Provide for better input from U.S. Coast Guard Implemented performance
monitors
Hold the system integrator accountable for improving Implemented
effectives of the integrated product teams
Establish a baseline for determining whether the Will not be implemented
acquisition approach is costing the government more than the traditional
asset replacement approach
Establish a time frame for putting steps in place to Partially implemented
measure contractor's progress toward improving operational effectiveness
Establish criteria to determine when to adjust the project Partially
implemented baseline and document the reasons for change
Control of future costs through competition For subcontracts over $5
million awarded by the system Implementeda
remains at risk because of week oversight. integrator to the two major
subcontractors, require notification to the Coast Guard about decision to
perform the work in-house rather than contracting it out
Coast Guard's Efforts to Improve Oversight and Program Management Show Mixed
Results
Develop a comprehensive plan for holding the system Partially implemented
integrator accountable for ensuing adequate competition among suppliers
Source: GAO analysis.
aDetermined to be implemented during work performed in 2005 for GAO-05-757.
Coast Guard’s Efforts to Improve Oversight and Program Management
Show Mixed Results
We continue to see mixed results in the Coast Guard's efforts to improve
oversight and management of the Deepwater program. The Coast Guard has put
in place a human capital plan to help ensure adequate staffing of the
Deepwater program and has taken actions to improve the effectiveness of
integrated product teams. However, subcontractor collaboration and
provision of guidance on transitioning to new Deepwater assets to field
personnel, particularly as it pertains to maintenance and logistics
responsibilities, continue to need additional attention.
Put in Place a Human Capital Plan to Ensure Adequate Staffing of the
Deepwater Program
Original issue: As early as 2001, we noted that difficult human capital
challenges would need to be addressed, including the need to recruit and
train sufficient staff to manage and oversee the Deepwater contract. 22
Reviewing this matter again in 2004, we found that the Coast Guard had not
funded the number of staff requested by the Deepwater program and had not
adhered to the processes outlined in its human capital plan for addressing
turnover of Deepwater officials, particularly Coast Guard personnel. 23
These staffing shortfalls contributed to problems in making timely
decisions and keeping pace with the workload.
Steps taken: The Coast Guard took several steps to address this issue. Its
initial steps involved hiring contractors to assist with program support
functions, shifting some positions from being staffed by military
personnel to civilian personnel to mitigate turnover risk, and identifying
the hard-to-fill positions and developing recruitment plans specifically
for them. Subsequent to these changes, the Deepwater program's executive
officer
(1) approved a revised human capital plan in February 2005 emphasizing
workforce planning and (2) is developing ways to leverage institutional
knowledge as staff rotate out of the Deepwater program. The Coast Guard
plans to review the human capital plan annually to ensure continual
alignment between human capital management and actual program performance.
The Coast Guard has also placed added emphasis on staffing when
formulating the program's budget request-for example, in adding
contracting officers and specialists. Finally, the Coast Guard has worked
closely with the Department of Homeland Security and the Defense
Acquisition University to provide training for Deepwater personnel.
Recommendation status: The steps the Coast Guard has taken appear
sufficient to address matters related to adequately staffing the Deepwater
program and mitigating turnover, and therefore we consider this
recommendation to be fully implemented.
Strengthening Integrated Product Teams
Original issue: Effective management of the Deepwater program depends
heavily on strong collaboration among the Coast Guard, the system
integrator, and the subcontractors. Integrated product teams (IPTs), the
Coast Guard's primary tool for managing the Deepwater program, overseeing
contractor activities, and ensuring collaboration have experienced
difficulty from the outset. IPTs, which are generally chaired
22GAO-01-564. 23GAO-04-380.
Page 31 GAO-06-546 Coast Guard
by a subcontractor representative and consist of members representing the
subcontractors and the Coast Guard, are responsible for overall program
planning and management, asset integration, and overseeing delivery of
specific Deepwater assets. In 2004, we reported these teams were
struggling to carry out their missions because of four major issues: lack
of timely charters to provide authority needed for decision making,
inadequate communication among team members, high turnover, and
insufficient training.
Steps taken: In 2005, we found that all IPTs had charters and their
members had received entry-level training. Decision making, however,
continued to be largely compartmented. Since then, the Coast Guard has
established domain management teams to serve as oversight and conflict
resolution entities for the IPTs. According to Coast Guard officials,
these teams are also to enhance collaboration on issues that cut across
several IPTs. Monthly assessments show IPTs have continued to improve
their effectiveness across all performance measures.
Recommendation status: While the Coast Guard has taken some actions, we do
not believe the actions are sufficient to consider the recommendation to
be fully implemented because there are indications that collaboration
among subcontractors remains inconsistent. Last year we pointed out that
ICGS's two major subcontractors, Lockheed Martin and Northrop Grumman,
were operating under their own management systems and that this approach
could lessen the likelihood that a system-of-systems outcome would be
successfully achieved. During our current review, Coast Guard performance
monitors and the program's executive officer reported that collaboration
among the subcontractors continues to be problematic and that ICGS wields
little influence to compel decisions among them. For example, when dealing
with proposed design changes to assets under construction, ICGS submits
the changes as two separate proposals from both first-tier subcontractors
rather than coordinating the separate proposals into one coherent plan.
According to Coast Guard performance monitors, this approach complicates
the Coast Guard's review of the needed design change because the two
proposals often carry overlapping work items, thereby forcing the Coast
Guard to act as the system integrator in these situations.
The Coast Guard has undertaken efforts to address these problems. Coast
Guard officials said they have improved communication with ICGS in this
area so that requirements are more easily discernable. Further, the Coast
Guard is beginning to take steps to collaborate with the system integrator
and the first-tier subcontractors with greater frequency, but it is too
early to tell if these will effectively eliminate the problems
Providing Field Personnel with Guidance and Training on Transitioning to
New Deepwater Assets
Original issue: In 2004, we found the Coast Guard had not effectively
communicated decisions on (1) how new Deepwater and existing assets are to
be integrated during the transition and (2) whether Coast Guard or
contractor personnel (or a combination of the two) will be responsible for
maintenance of the Deepwater assets. For example, Coast Guard field
personnel, including senior-level operators and naval engineering support
command officials, said they had not received information about how they
would be able to continue accomplishing their missions using existing
assets while also being trained on the new assets.
Steps taken: The Coast Guard has taken some steps to improve the level of
communication between the Deepwater program and field operators and
maintenance personnel. A November 2004 analysis of the Deepwater program's
communication process, conducted in coordination with the National
Graduate School, found that the communication and feedback process was
inadequate. Since then, the Coast Guard has placed more emphasis on
outreach to field personnel, including surveys, face-to-face meetings, and
presentations. More recently, officials from the Atlantic and Pacific Area
Commands, Maintenance and Logistics Commands, and the Aircraft Repair and
Supply Center agreed that Deepwater program officials have significantly
improved the frequency and types of information flowing from the program
office to the field. In addition, field personnel are members of several
IPTs and working groups, and ICGS has placed liaisons at several field
locations.
Recommendation status: While the Coast Guard has taken some actions, there
are indications that the actions are not yet sufficient to consider the
recommendation to be fully implemented. In particular, our review of
relevant documents and our discussions with key personnel make clear that
field operators and maintenance personnel are still concerned that their
view are not adequately acknowledged and addressed, and have little
information about maintenance and logistics plans for the new Deepwater
assets. For example, though the first National Security Cutter is to be
delivered in August 2007, field and maintenance officials have yet to
receive information on plans for crew training, necessary shore facility
modifications, or how maintenance and logistics responsibilities will be
divided between the Coast Guard and ICGS. According to Coast Guard
officials, many of these decisions need to be made and communicated very
soon in order to allow for proper planning and preparation in advance of
the cutter's delivery.
More Time Needed to Determine Adequacy of Steps Taken to Improve System
Integrator Accountability
Unlike actions on the previous recommendations, Coast Guard actions to
provide better input from Coast Guard performance monitors and to hold the
system integrator more accountable for performance appear to be largely
sufficient. We cannot determine whether the Coast Guard has implemented
several of our recommendations in this area, however, until more Deepwater
assets are delivered and the results of these actions can be assessed.
Providing Better Input from Coast Guard Performance Monitors
Original issue: In 2004, we reported that the Coast Guard's award fee
evaluation of the first year of ICGS's performance was based on
unsupported calculations and relied heavily on subjective judgments.
Rating procedures used by Coast Guard performance monitors were
inconsistent, as were procedures for calculating scores, leading to
questions about whether the award fee decision was well supported.
Actions taken: The Coast Guard has provided additional guidance and
training to performance monitors, better allowing them to link their
comments with specific examples within their respective areas of
responsibility. The Coast Guard has also improved the consistency of the
format that performance monitors use to provide input about the system
integrator's performance and revised assessment criteria to more clearly
differentiate between objective measures (that is, developed using
automated tools and compared against defined standards) and subjective
evaluations. Weights have been assigned to each set of evaluation factors,
and the Coast Guard continues to adjust these factors to achieve an
appropriate balance between automated results and eyewitness observations.
Recommendation status: The Coast Guard's efforts to provide better
guidance and training, improve the consistency of the format for
performance monitors' input, and clarify performance assessment criteria
appear sufficient for addressing the issue, and therefore we consider this
recommendation to be fully implemented.
Holding the System Integrator Accountable for Improving Effectiveness of
the Integrated Product Teams
Original issue: In 2004, we found that the system integrator, whose
subcontractors chaired the IPT working groups, was not being held
accountable for IPT effectiveness in its performance assessments. Actions
taken: The Coast Guard changed award fee measures to place additional
emphasis on the system integrator's responsibility for making the IPTs
effective. Award fee criteria now incorporate the administration,
management commitment, collaboration, training, and empowerment of these
teams.
Recommendation status: With IPTs' performance now included in the criteria
for measuring the system integrator's performance, we consider this
recommendation to be fully implemented.
Establishing a Time Frame for Putting Steps in Place to Measure
Contractor's Progress toward Improving Operational Effectiveness
Original issue: In 2001, the Coast Guard set a goal of developing
measures, within 1 year after contract award, to conduct annual
assessments of the system integrator's progress toward achieving the three
overarching goals of the Deepwater program, including increased
operational effectiveness. In 2004, we found that the time frame for the
first review of the contractor's performance against the Deepwater goals
had slipped. The former Deepwater chief contracting officer told us that
he anticipated that the metrics would be in place in the fourth year of
the contract, the same year the Coast Guard would decide whether or not to
extend the contract.
Steps taken: The Coast Guard has since developed modeling
capabilities-namely the DMOES model discussed earlier-to simulate the
effect of the new assets' capabilities on the Coast Guard's ability to
meet its missions. Coast Guard officials told us that they are now
beginning to track the operational effectiveness of the Deepwater program
using both the DMOES model and actual mission performance data. Further,
at the Coast Guard's request, the Center for Naval Analyses developed a
tool to measure the "presence" of Deepwater assets-that is the number of
square miles of ocean in which Deepwater aircraft and vessels can detect,
identify, and prosecute targets. In addition, Coast Guard officials have
also begun using mission performance data from 2004, the most recent year
of complete information, to measure the contribution provided by Deepwater
systems or assets in seven mission areas: search and rescue, cocaine
seizure rate, illegal or undocumented migrant interdiction, foreign
fishing vessel interdiction, protection of living marine resources,
national defense/military readiness, and international ice patrol. Coast
Guard officials acknowledge that this is difficult, though, because the
data on mission results and accomplishments do not differentiate between
Deepwater assets and non-Deepwater assets. Coast Guard officials said
doing so should become easier as more Deepwater assets come on line and as
analytical tools are refined.
Recommendation status: Although the models have been developed and are
being refined to measure operational effectiveness, there are too few
Deepwater assets currently in operation to effectively measure the system
integrator's actual performance in improving operational effectiveness. As
a result, we do not consider this recommendation to be fully implemented.
Establishing Criteria to Determine When to Adjust the Project Baseline and
Document the Reasons for Change
o
o
We recognize, though, that as more Deepwater assets and systems come on
line, the amount of data will increase and the analytical tools will be
more refined so that the Coast Guard should be in a better position to
(1) discern the Deepwater program's contribution to operational
effectiveness and (2) fully implement this recommendation.
Establishing Criteria to Determine When to Adjust the Project Baseline
and Document the Reasons for Change
Original issue: Establishing a solid baseline against which to measure
progress in lowering total ownership cost (TOC) is critical to holding the
system integrator accountable. However, during our 2004 review, we found
that the Coast Guard's Deepwater TOC baseline had been significantly
changed from what had been originally envisioned and that further changes
could be made as a result of variables such as fuel costs or vessels'
operating tempo. At the time, Coast Guard officials explained that
proposed changes to the baseline would be approved by the program
executive officer on a case-by-case basis, though the Coast Guard had not
developed criteria for potential upward or downward adjustments to the
baseline.
Steps taken: In response to our concerns, the Coast Guard began using
criteria from its Major Systems Acquisition Manual as the basis for
adjusting the TOC baseline. These criteria allow the baseline to be
adjusted based on significant changes in mission requirements, schedule
changes, or project funding, or for specific congressional actions. Coast
Guard officials also told us that they have also added criteria for making
changes to the baseline, such as:
insufficient program funding or inflationary pressure that exceeds the
assumptions and
natural disasters or periods of national emergency that require a
deviation from the baseline's cost, schedule, or performance parameters.
Coast Guard officials said that approval of revisions to the program's
overall baseline must come through approved decision memorandums from the
Agency Acquisition Executive, who is the Vice Commandant of the Coast
Guard. The Deepwater Program Executive Officer still has authority to
approve baseline revisions at the asset and domain level. Depending on
their severity, baseline changes now are also subject to review and
approval by the Department of Homeland Security (DHS), the Coast Guard's
parent agency. The Coast Guard is required to submit Deepwater program
baseline information to DHS on a quarterly basis, and the project is
subject to an annual review by the DHS Investment Review Board. According
to DHS officials, a baseline breach of 8 percent or more
Effects of Steps Taken to Control Future Costs through Competition Will Take
Time to Assess
Developing a Plan for Holding the System Integrator Accountable for
Ensuring Adequate Competition among Suppliers
would require that the Coast Guard provide information on the causal
factors and propose corrective actions to rectify the breach. 24 The
officials added that, if the baseline breach is considered significant,
the Office of Management and Budget is to be notified that the program
will have to undergo a rebaselining and its funding profile will need to
be altered. Further, as a result of its latest review of the Deepwater
program, the DHS Investment Review Board has asked that, in addition to
overall program baseline information, the Coast Guard also provide
baseline information for each of the Deepwater assets. This will provide
DHS with more insight into the program's cost, schedule, and performance.
Recommendation status: The Coast Guard's steps, combined with DHS's
oversight requirements, should be sufficient to resolve this issue. At
present, however, DHS's policy directive is only in draft form. We will
consider this recommendation to be fully implemented when the management
directive is finalized. 25
Effects of Steps Taken to Control Future Costs through Competition Will
Take Time to Assess
The Coast Guard has taken a number of actions to address the remaining
recommendation in this area, which relates to holding the system
integrator accountable for ensuring competition among subcontractors.
However, until the effects of these actions are more apparent, we are not
able to consider the recommendation as being implemented.
Developing a Plan for Holding the System Integrator Accountable for Ensuring
Adequate Competition among Suppliers
Original issue: Competition is a key component for controlling costs in
the Deepwater program and a guiding principle for DHS's major
acquisitions. In 2004, we found that beyond the initial 5-year contract
period, the Coast Guard had no way to ensure competition was occurring
because it did not have mechanisms in place to measure the extent of
competition or to hold the system integrator accountable for steps taken
to achieve competition. Shortly before, the system integrator had adopted
Lockheed Martin's "open business model" as a corporate policy to help
24
According to DHS officials, a baseline breach occurs when a cost or
schedule threshold is exceeded or when a performance threshold cannot be
met.
25
According to the Coast Guard, DHS considers this interim document final;
however, because the documentation that supports the policy directive is
still in draft form, we do not consider it to be final.
Page 37 GAO-06-546 Coast Guard
ensure competition and keep costs under control. 26 However, the open
business model is not a formal policy involving specific decision points
to ensure that competition will be considered. Further, the first-tier
subcontractors, Lockheed Martin and Northrop Grumman, have largely
continued to follow their own procurement procedures and guidance for
determining whether competition will occur and the suppliers who will be
invited to compete for Deepwater assets.
Steps taken: To address our recommendation about ensuring out-year
competition among second-tier suppliers, the Coast Guard contracted with
Acquisitions Solutions, Inc. (ASI), to assess the amount of second-tier
competition conducted by ICGS during 2004. ASI issued a report in May 2005
that, among other things, found that the open business model had not been
fully embraced by Northrop Grumman despite its being an ICGS corporate
policy. 27 The report made nine recommendations aimed at improving
competition throughout the Deepwater program. According to Deepwater
officials, ICGS developed a plan to adopt all nine recommendations by
March 1, 2006, and is providing training on use of the open business model
to Northrop Grumman personnel working on the Deepwater program. Further,
Coast Guard officials reported that competition will be assessed as a part
of the award fee assessment subjective criteria for the fifth year of the
contract and the Coast Guard will specifically examine the system
integrator's ability to control costs by assessing the degree to which
competition is fostered at the major subcontractor level during the award
term decision process later this year. 28
Recommendation status: While steps already under way appear to be
sufficient to resolve our concerns, we cannot consider this recommendation
as being fully implemented until the Coast Guard has
26
The open business model is to encourage second-tier suppliers to remain
innovative and competitive by directing the first-tier subcontractors to
(1) generally avoid the use of teaming agreements with suppliers and
prohibit teaming agreements based on guaranteed work share, (2) defer
second-tier supplier decisions as long as practicable so that changes in
the market place can be considered, and (3) actively solicit market
information and new suppliers.
27
U.S. Coast Guard, Program Executive Office, Integrated Deepwater System
Program, Competition Assessment, as prepared by Acquisitions Solutions,
Inc., May 27, 2005.
28
During the award term process, the Coast Guard will decide whether to
award the first five-year contract option.
addressed the ASI recommendations and results of the next award term
assessment are known.
Concluding Observations
The Coast Guard has done a commendable job of adapting the Deepwater
program to post-September 11 realities. Our analysis shows that Coast
Guard officials used sound analytical methods to assess the revised needs
for aircraft and vessels. Coast Guard officials have also taken strong
efforts to address concerns about program management and contract
performance and have largely implemented or are in the process of
implementing steps that would help mitigate these concerns. We agree that
the Coast Guard would be well served to continue developing ways to use
its computer modeling to establish clear relationships between its mix of
assets-aircraft and vessels-and its Deepwater and agency-level performance
goals. We have pointed out in past reports that the Coast Guard lacks
clear measures of how its resources are linked to achieving performance
goals, so these steps should help resolve this concern. We realize that
this ongoing effort will likely take some time to complete. While the
Coast Guard has made good progress in addressing our recommendations,
there are aspects of the Deepwater program that will require continued
attention. First, the Deepwater program continues to face a degree of
underlying risk, in part because of the unique approach involving a
system-of-systems approach with the contractor acting as overall
integrator, and in part because it so heavily tied to precise year-to-year
funding requirements over the next two decades. Further, a project of this
magnitude will likely continue to experience other concerns and challenges
beyond those that have emerged so far. It will be important for Coast
Guard managers to continue careful monitoring of contractor performance
and to continue addressing program management concerns as they arise.
Agency Comments
We requested comments on a draft of this report from the Department of
Homeland Security and the U.S. Coast Guard. The U.S. Coast Guard provided
technical comments, which have been incorporated into the report as
appropriate.
We are providing copies of this report to the Secretary of the Department
of Homeland Security, the Commandant of the U.S. Coast Guard, and
interested congressional committees. The report will also be made
available to others upon request. In addition, the report will be
available at no charge on GAO's Web site at http//www.gao.gov.
Page 39 GAO-06-546 Coast Guard
If you or your staff have any questions about this report, please contact
me on (415) 904-2200 or [email protected]. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made major contributions to this report
Margaret T. Wrightson Director, Homeland Security and Justice Issues
Appendix I: Objectives, Scope, and Methodology
This report, which focuses on the Coast Guard's Deepwater management
challenges, provides details on three issues: (1) a comparison of the
revised Deepwater implementation plan issued in August 2005 with the
original (August 2002) plan in terms of cost, time frames, and the balance
of legacy and replacement assets; (2) an assessment of the degree to which
the operational effectiveness model and other analytical methods used by
the Coast Guard to develop the revised Deepwater asset mix are sound and
appropriate for such a purpose; and (3) an assessment of the progress made
in implementing our prior recommendations regarding Deepwater program
management.
To compare the revised Deepwater implementation plans issued in August
2005 and February 2006 with the original (August 2002) Deepwater
implementation plan in terms of cost, time frames, and the balance of
legacy and replacement assets, we analyzed the original and revised
Deepwater implementation plans and related guidance. We also reviewed and
analyzed relevant Coast Guard documentation on changes in missions, costs,
asset mix, asset capabilities, and asset delivery schedules. We
supplemented the documentation reviews and analyses with discussions with
officials from the Deepwater Program Executive Office. Finally, we
discussed the risks associated with the Deepwater program's reliance on a
sustained level of funding through 2027 and the implications of these
risks.
To assess the degree to which the operational effectiveness models and
other analytical methods used to develop the revised Deepwater asset mix
are sound and appropriate for such a purpose, we reviewed the capacity and
operational effectiveness models used in determining the current Deepwater
asset mix to ensure that the approach was sound and that appropriate
assumptions were made in the models' use. This review involved assessing
Coast Guard documentation on how its models were developed and executed,
determining the views of knowledgeable independent parties on the Coast
Guard's operational effectiveness model, and interviewing cognizant Coast
Guard officials. These interviews also included discussions of how these
models, and other factors, were used in developing the current Deepwater
asset mix, as well as whether the Coast Guard has developed an approach
for determining the extent to which the Deepwater asset mix will allow it
to meet its performance targets.
In assessing the Coast Guard's modeling and other analytical methods used
for developing the revised Deepwater asset mix, we paid particular
attention to the most recent performance gap analysis (PGA) study (PGA
IV), which compared the projected performance of the revised Deepwater
asset mix to that of the original Deepwater asset mix-so that we could
gain a better understanding of how these results were used in developing
the revised Deepwater asset mix. Specifically, we reviewed the report's
methodology and requested additional clarifying information to help
determine if the analytic work supported the report's conclusions. As part
of our assessment, we developed an analysis that departs from what the
Coast Guard describes in its report in two important ways. First, and most
important, the Coast Guard assigned a linear scale ranging from 1 to 5 to
five statistical categories describing the distribution of the analysis
data. This range assigned numerical values to the degree to which the
revised asset mix was projected to outperform (or underperform) the
original asset mix, with 1 representing projected performance two or more
standard deviations below that of the original asset mix, up to 5,
representing projected performance two or more standard deviations above
that of the original asset mix. It is our opinion that this type of linear
scale is not appropriate for capturing the variations in projected
performance. Accordingly, we used a weighting scheme for these categories
(known as z-scores) that better reflects the relationship among these
categories. The z-scores take into account the statistical property that
being two standard deviations away from the mean value is almost five
times more difficult than being one standard deviation away from the mean.
Second, we compared our calculated performance measure weights to a
standard in order to assess if our weighting scheme would affect the
study's conclusions. Since the methodology identified three mission
significance categories and four regional mission priority categories, we
compared our recalculated weights based upon the z-score with the weights
we would expect to see if all mission performance measures across all
mission priorities for the four modeled regions had exceed one standard
deviation above the mean in improvement. Despite the different
methodologies used, our results generally aligned with what the Coast
Guard reported in PGA IV.
To determine the status of the Coast Guard's implementation of our prior
recommendations for improving program management, strengthening contractor
accountability, and controlling costs, we reviewed and analyzed briefings
and relevant documentation provided by the Deepwater Program Executive
Office on actions taken to address our concerns. We reviewed and analyzed
documentation on the Coast Guard's assessment of the contractor's system
integration and management performance in the first period of the fourth
year of the contract, including written comments by the performance
monitors. We also reviewed and analyzed information on Deepwater
integrated product teams, including membership lists and briefings
provided by the Coast Guard on measures of effectiveness for the teams. We
analyzed the Coast Guard's plans to increase communications
to field operators, and documentation from field operators and maintenance
personnel regarding these communications. Further, we analyzed the
February 2005 Deepwater revised Human Capital Plan to identify changes
that have been made and discussed Deepwater Program Office staffing plans
with Coast Guard officials. To supplement our analyses of the relevant
documentation, we held several meetings with the Deepwater Program
Executive Officer, the Deputy Program Executive Officer, and a number of
Deepwater staff, including contracting officials and representatives from
the system integrator. We also held discussions with Coast Guard Deepwater
performance monitors to discuss their written comments to the latest award
fee assessment. We also held discussions with officials from the Pacific
Area Command and Pacific Area Maintenance and Logistics Command in
Alameda, California; the Atlantic Area Command and Atlantic Area
Maintenance and Logistics Command in Norfolk, Virginia; and the Aircraft
Repair and Supply Center in Elizabeth City, North Carolina. Further, we
reviewed acquisition guidance and spoke with officials from the Department
of Homeland Security regarding their oversight of the Deepwater
acquisition program baseline.
We performed our review from August 2005 to March 2006 in accordance with
generally accepted government auditing standards.
Appendix II: GAO Contacts and Staff Acknowledgments
GAO Contact
Margaret T. Wrightson (415) 904-2200, [email protected]
In addition to the contact named above, Steven Calvo,
Staff Acknowledgments
Christopher Conrad, Adam Couvillion, Christine Davis, Art James,
Julie Leetch, Michele Mackin, Stan Stenersen, and Linda
Kay Willard made key contributions to this report.
GAO Related Products
Coast Guard's Acquisition Management: Deepwater Project's Justification
and Affordability Need to Be Addressed More Thoroughly, GAO/RCED-99-6 (
Washington, D.C.: Oct. 26, 1998).
Coast Guard: Budget Challenges for 2001 and Beyond, GAO/T-RCED-00-103 (
Washington, D.C.: March 15, 2000).
Coast Guard: Progress Being Made on Deepwater Project, but Risk Remain,
GAO-01-564 ( Washington, D.C.: May 2, 2001).
Coast Guard: Actions Needed to Mitigate Deepwater Project Risks,
GAO-01-659T ( Washington, D.C.: May 3, 2001).
Coast Guard: Strategy Needed for Setting and Monitoring Levels of Effort
for All Missions, GAO-03-155 ( Washington, D.C.: Nov. 12, 2002).
Coast Guard: Comprehensive Blueprint Needed to Balance and Monitor
Resource Use and Measure Performance for All Missions, GAO-03-544T
(Washington, D.C.: March 12, 2003).
Coast Guard: Challenges during the Transition to the Department of
Homeland Security, GAO-03-594T ( Washington, D.C.: April 1, 2003).
Contract Management: Coast Guard's Deepwater Program Needs Increased
Attention to Management and Contractor Oversight, GAO-04-380 ( Washington,
D.C.: March 9, 2004).
Coast Guard: Replacement of HH-65 Helicopter Engine, GAO-04-595
(Washington, D.C.: March 24, 2004).
Coast Guard: Key Management and Budget Challenges for Fiscal Year 2005 and
Beyond, GAO-04-636T (Washington, D.C.: April 7, 2004).
Coast Guard: Deepwater Program Acquisition Schedule Update Needed,
GAO-04-695 ( Washington, D.C.: June 14, 2004).
Coast Guard: Observations and Agency Priorities in Fiscal Year 2006 Budget
Request, GAO-05-364T ( Washington, D.C.: March 17, 2005).
Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy
Assets and Acquisition Management Challenges, GAO-05-307T (Washington,
D.C.: April 20, 2005).
GAO Related Products
Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy
Assets and Acquisition Management Challenges, GAO-05-651T (Washington,
D.C.: June 21, 2005).
Coast Guard: Progress Being Made on Addressing Deepwater Legacy Asset
Condition Issues and Program Management, but Acquisition Challenges
Remain, GAO-05-757 ( Washington, D.C.: July 22, 2005).
(440445)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov) . Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To have GAO e-mail you a list of newly posted products every
afternoon, go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON
RECYCLED PAPER
*** End of document. ***