Contract Management: Coast Guard's Deepwater Program Needs
Increased Attention to Management and Contractor Oversight
(09-MAR-04, GAO-04-380).
The Coast Guard's Deepwater program, the largest acquisition
program in its history, involves modernizing or replacing ships,
aircraft, and communications equipment. The Coast Guard awarded
the Deepwater contract to Integrated Coast Guard Systems (ICGS)
in June 2002. The Coast Guard estimates the program will cost $17
billion over a 30-year period. ICGS is a system integrator, with
responsibility for identifying and delivering an integrated
system of assets to meet the Coast Guard's missions. GAO was
asked to assess whether the Coast Guard is effectively managing
the Deepwater program and overseeing the contractor and to assess
the implications of using the Deepwater contracting model on
opportunities for competition.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-380
ACCNO: A09441
TITLE: Contract Management: Coast Guard's Deepwater Program
Needs Increased Attention to Management and Contractor Oversight
DATE: 03/09/2004
SUBJECT: Competition
Contract administration
Contract oversight
Cost control
Equipment contracts
Federal procurement
Human resources training
Human resources utilization
Performance measures
Personnel management
Procurement policy
Procurement practices
Program management
Coast Guard Deepwater Project
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-380
United States General Accounting Office
GAO Report to the Ranking Minority Member, Committee on Commerce, Science, and
Transportation, U.S. Senate
March 2004
CONTRACT MANAGEMENT
Coast Guard's Deepwater Program Needs Increased Attention to Management and
Contractor Oversight
GAO-04-380
Highlights of GAO-04-380, a report to the Ranking Minority Member,
Committee on Commerce, Science, and Transportation, U.S. Senate
The Coast Guard's Deepwater program, the largest acquisition program in
its history, involves modernizing or replacing ships, aircraft, and
communications equipment. The Coast Guard awarded the Deepwater contract
to Integrated Coast Guard Systems (ICGS) in June 2002. The Coast Guard
estimates the program will cost $17 billion over a 30-year period. ICGS is
a system integrator, with responsibility for identifying and delivering an
integrated system of assets to meet the Coast Guard's missions.
GAO was asked to assess whether the Coast Guard is effectively managing
the Deepwater program and overseeing the contractor and to assess the
implications of using the Deepwater contracting model on opportunities for
competition.
GAO recommends that the Secretary of Homeland Security direct the
Commandant of the Coast Guard to take a number of actions to improve
Deepwater management and contractor oversight. The Department of Homeland
Security forwarded GAO the Coast Guard's written comments on a draft of
this report. The Coast Guard welcomed GAO's observations and concurred
with GAO's recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-04-380.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact William T. Woods at (202)
512-4841 or [email protected].
March 2004
CONTRACT MANAGEMENT
Coast Guard's Deepwater Program Needs Increased Attention to Management and
Contractor Oversight
Over a year and a half into the Deepwater contract, the key components
needed to manage the program and oversee the system integrator's
performance have not been effectively implemented. Integrated product
teams, the Coast Guard's primary tool for overseeing the system
integrator, have struggled to effectively collaborate and accomplish their
missions. They have been hampered by changing membership, understaffing,
insufficient training, and inadequate communication among members. In
addition, the Coast Guard has not adequately addressed the frequent
turnover of personnel in the program and the transition from existing to
Deepwater assets.
The Coast Guard's assessment of the system integrator's performance in the
first year of the contract lacked rigor. For example, comments from the
technical specialist responsible for monitoring the design and delivery of
ships were not included in the evaluation scores. Further, the factors
that formed the basis for the award fee determination were unsupported by
quantifiable metrics. Despite documented problems in schedule,
performance, cost control, and contract administration, ICGS received a
rating of 87 percent, resulting in an award fee of $4.0 million of the
maximum $4.6 million annual award fee.
Further, the Coast Guard has not yet begun to measure the system
integrator's performance on the three overarching goals of the Deepwater
program--operational effectiveness, total ownership cost, and customer
satisfaction. Its original plan of measuring progress on an annual basis
has slipped, and Coast Guard officials have not projected a time frame for
when they will be able to hold the contractor accountable for progress
against these goals. This information will be essential to the Coast
Guard's decision about whether to extend ICGS's contract after the first 5
years.
Competition is critical to controlling costs in the Deepwater program and
a guiding principle of Department of Homeland Security acquisitions.
Concerns about the Coast Guard's ability to rely on competition as a means
to control future costs contributed to GAO's description of the Deepwater
program in 2001 as "risky." Three years later, the Coast Guard has neither
measured the extent of competition among suppliers of Deepwater assets nor
held the system integrator accountable for taking steps to achieve
competition. Deepwater's acquisition structure is such that the two
first-tier subcontractors have sole responsibility for determining whether
to hold competitions for assets or to provide these assets themselves. The
Coast Guard has taken a hands-off approach to "make or buy" decisions made
at the subcontractor level. As a result, questions remain about whether
the government will be able to control costs.
Contents
Letter
Results in Brief
Background
Key Components of Management and Oversight Have Not Been
Effectively Implemented Process for Assessing System Integrator's Ongoing
Performance Lacked Rigor Coast Guard Has Not Begun to Measure Contractor
Performance against Deepwater Program Goals Control of Future Costs
through Competition Remains a Risk
because of Weak Oversight of Subcontractor Decisions Conclusion
Recommendations for Executive Action Agency Comments Scope and Methodology
1
3 5
8
15
18
21 26 27 29 30
Appendix I Comments from the Coast Guard
Appendix II Status of Selected Deepwater Contract Management and
Oversight Plans
Tables
Table 1: Deepwater Program Appropriations Table 2: Comments from Coast
Guard Management Reports on IPT Performance
Table 3: Breakdown of Coast Guard Obligations to ICGS (Includes Planned
Subcontracts to First Tier Subcontractors as of September 30, 2003)
Table 4: Breakdown of ICGS Obligations to Lockheed Martin (Planned
Subcontracts and Second-Tier Obligations as of September 30, 2003)
Table 5: Breakdown of ICGS Obligations to Northrop Grumman (Planned
Subcontracts and Second-Tier Obligations as of September 30, 2003)
7 9
22
23
23
Figure
Figure 1: Contracting Relationship between Coast Guard, ICGS, and
Subcontractors
Abbreviations
COTR contracting officer's technical representative
C4ISR command, control, communications, computer, intelligence,
surveillance, and reconnaissance
DHS Department of Homeland Security
FAR Federal Acquisition Regulation
GAO General Accounting Office
ICGS Integrated Coast Guard Systems, LLC
IPT integrated product team
TOC total ownership cost
VUAV vertical take-off and landing unmanned aerial vehicle
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States General Accounting Office Washington, DC 20548
March 9, 2004
The Honorable Ernest F. Hollings
Ranking Minority Member
Committee on Commerce, Science, and Transportation
United States Senate
Dear Senator Hollings:
The Coast Guard is undergoing a major transformation through its
Deepwater program, the largest acquisition project in its history. The
program involves modernizing or replacing cutters, aircraft, and
communications equipment used for missions that generally occur beyond
50 miles from shore. These missions include drug interdiction, alien
migrant interdiction, search and rescue, overseas port security and
defense, and offshore inspection of foreign vessels bound for U.S. ports.
The three goals of the Deepwater program are to maximize operational
effectiveness, minimize total ownership cost (TOC), and satisfy the
customer. On March 1, 2003, the Coast Guard became part of the
Department of Homeland Security (DHS), and the Deepwater program,
estimated to cost $17 billion over a 30-year period,1 is one of the
largest
acquisitions in the department.
In June 2002, the Coast Guard awarded a contract to Integrated Coast
Guard Systems, LLC (ICGS) as the system integrator for Deepwater to
develop and deliver an improved, integrated system of ships, aircraft,
unmanned aerial vehicles, command, control, communications, computer,
intelligence, surveillance and reconnaissance (C4ISR), and supporting
logistics. The contract has a 5-year base period with five additional
5-year
options. ICGS is a business entity jointly owned by Northrop Grumman
Ship Systems (Northrop Grumman) and Lockheed Martin Corporation
(Lockheed Martin). Northrop Grumman and Lockheed Martin are the two
first-tier subcontractors for the Deepwater program. They, in turn,
provide
Deepwater assets or award second-tier subcontracts for the assets. Figure
1 depicts the relationship of the various parties involved in executing
the
Deepwater program.
1 The Deepwater assets are expected to be delivered within 20 years, and
the contractor is to be retained an additional 10 years to continue
implementing the program.
Figure 1: Contracting Relationship between Coast Guard, ICGS, and
Subcontractors
The Deepwater contract is performance-based, meaning that the Coast Guard
has specified the outcomes it is seeking to achieve and has given the
system integrator responsibility for identifying and delivering the assets
needed to achieve these outcomes. In 2001, we described the
Deepwater project as "risky," due to the unique, untried acquisition
strategy for a project of this magnitude within the Coast Guard.2 We
highlighted concerns about the Coast Guard's ability to ensure that
procedures and personnel would be in place to manage and oversee the
system integrator and raised concerns about the agency's ability to keep
costs under control in future program years by ensuring adequate
competition for Deepwater assets. Now that the Deepwater program is
underway, you asked us to assess whether the Coast Guard is effectively
managing the acquisition program and overseeing the contractor and to
assess the implications of using the Deepwater contracting model on
opportunities for competition. Specifically, we assessed whether the Coast
Guard (1) has a structure in place to effectively manage the program and
oversee the system integrator, (2) has established adequate criteria by
which to assess and reward the system integrator's performance, (3) has
the information needed to make a decision about extending the contract
after the first 5 years, and (4) is addressing the role and extent of
competition for Deepwater assets.
Results in Brief
More than a year and a half into the Deepwater contract, the key
components needed to manage the program and oversee the system
integrator's performance have not been effectively implemented. Integrated
product teams, comprised of Coast Guard, ICGS, and subcontractor
employees, are the Coast Guard's primary tool for managing the program and
overseeing the contractor. However, the effectiveness of the teams has
been weakened due to changing membership, understaffing, insufficient
training, lack of authority for decision making, and inadequate
communication among members. In addition, while the Coast Guard has a
Deepwater human capital plan in place to address turnover, in practice the
plan is not being followed and vacancies exist for key Deepwater
personnel. Finally, although delivery of some of the first assets is
imminent, the Coast Guard has not effectively communicated to its
operational personnel decisions on how new and old assets will be
integrated and how maintenance responsibilities will be divided between
government and contractor personnel. Some of these initial assets have
already experienced schedule delays.
2 Coast Guard: Progress Being Made on Deepwater Project, but Risks Remain,
GAO-01-564 (Washington, D.C.: May 2, 2001).
The Coast Guard has not developed quantifiable metrics or adhered to
effective procedures for holding the system integrator accountable for its
ongoing performance. The process by which the Coast Guard assessed
performance after the first year of the contract lacked rigor. The factors
on which ICGS was rated lacked objective measures on which to assess
performance, and the first annual award fee determination was based
largely on unsupported calculations because the Coast Guard did not fully
adhere to its award fee process. For example, comments from the technical
specialist responsible for monitoring the contractor's performance in
designing and delivering ships were not included in the evaluation scores,
and it was unclear how numerical scores were assigned to the comments made
by another Coast Guard performance monitor. Despite documented problems in
schedule, performance, cost control, and contract administration
throughout the first year of the contract, the program executive officer
awarded the system integrator an overall rating of 87 percent, which falls
in the "very good" range. This rating resulted in an award fee of $4.0
million of the maximum $4.6 million annual award fee.
The Coast Guard has not begun to measure the system integrator's
performance on the three overarching goals of the Deepwater
program-maximizing operational effectiveness, minimizing total ownership
cost, and satisfying the customers. This information is essential for
determining whether to extend ICGS's contract. In 2001, the Coast Guard
planned to develop appropriate metrics and to assess the system
integrator's performance on an annual basis so that, in the fourth year of
the contract, information would be available to determine whether to award
the first 5-year option. Coast Guard officials stated that because it is
early in the Deepwater program and the metrics have not yet been
finalized, they cannot accurately assess the contractor's progress against
the three goals. More troubling is that the Coast Guard has not projected
a time frame for when it will be able to measure progress. In addition,
the baseline for measuring total ownership cost has been fundamentally
changed from the original plan. At present, the Coast Guard is using as a
baseline ICGS's proposed cost for Deepwater plus an additional 10 percent,
rather than using the originally planned baseline of what the total
ownership cost would have been under a traditional procurement approach.3
The Coast Guard is now working with ICGS to develop a
3 Total ownership cost is the sum of all costs associated with the
research, development, procurement, personnel, training, operation,
logistical support, and disposal of the entire Deepwater system.
contingency strategy in the event that the system integrator needs to be
replaced after 5 years. Coast Guard officials indicated, however, that
since ICGS proposed the specific assets that comprise the Deepwater
program, it is unrealistic to believe that the contract could be
terminated without revising the entire Deepwater acquisition strategy.
Although competition among subcontractors is a key vehicle for controlling
costs, the Coast Guard has neither measured the extent of competition
among the suppliers of Deepwater assets nor held the system integrator
accountable for taking steps to achieve competition. Although the
competition that resulted in the initial award of this contract may be
viewed as sufficient for the base period, Deepwater's acquisition
structure provides little protection against cost growth during the option
periods. The two first-tier subcontractors, Lockheed Martin and Northrop
Grumman, have sole responsibility for determining whether to hold
competitions for Deepwater assets or to provide these assets themselves.
ICGS has stated that its use of sourcing guidance, termed the open
business model, guides the subcontractors' decisions about whether to
"make or buy" the Deepwater assets. However, this guidance is a
philosophy-not a formal process with clear criteria and specific decision
points-that encourages, but does not require, competition. In some cases,
teaming agreements formed with second-tier subcontractors during the
initial proposal phase of Deepwater have carried over as sole-source
arrangements. The Coast Guard's hands-off approach to make or buy
decisions and its failure to assess the extent of competition raise
questions about whether the government will be able to control costs in
the Deepwater program.
We are recommending that the Secretary of Homeland Security direct the
Commandant of the Coast Guard to take actions to (1) improve program
management, (2) improve contractor accountability, and (3) facilitate cost
control through competition for Deepwater assets. We received written
comments on a draft of this report from the Coast Guard, forwarded by DHS.
The Coast Guard welcomed our observations and, in a subsequent email,
concurred with the recommendations. The Coast Guard's letter is reproduced
in appendix I.
Background Deepwater is the largest and most complex procurement project
in the Coast Guard's history. The acquisition is scheduled to occur over a
30-year period at a projected cost of $17 billion. It includes the
modernization and replacement of an aging fleet of over 90 cutters and 200
aircraft used for missions that generally occur beyond 50 miles from
shore. Deepwater
currently accounts for almost one-third of the Coast Guard's acquisition
staff and one-third of support personnel funding.
Rather than using the traditional approach of replacing classes of ships
or aircraft through a series of individual acquisitions, the Coast Guard
chose to employ a "system-of-systems" acquisition strategy that would
replace its aging deepwater assets with a single, integrated package of
new or modernized assets. The primary objectives of the Deepwater program
are to maximize operational effectiveness and to minimize TOC while
satisfying the needs of the customer-the operational commanders, aircraft
pilots, cutter crews, maintenance personnel, and others who will use the
assets.
The Deepwater program has been in development for a number of years.
Between 1998 and 2001, three industry teams competed to identify and
provide the assets-aircraft, helicopters, cutters, logistics, and
C4ISR-needed to transform the Coast Guard. In June 2002, the Coast Guard
awarded a contract to ICGS as the system integrator for Deepwater. Under a
5-year base contract with five additional 5-year options, ICGS is
responsible for designing, constructing, deploying, supporting, and
integrating the Deepwater assets to meet Coast Guard requirements. ICGS, a
business entity comprised of a nine-member Board of Directors, is jointly
owned by Northrop Grumman and Lockheed Martin. Northrop Grumman and
Lockheed Martin are the two first-tier subcontractors for the Deepwater
program. They, in turn, provide Deepwater assets themselves or award
second-tier subcontracts for the assets.
In our 2001 report, we identified several areas of risk for Deepwater.
First, the Coast Guard faced potential risk in the overall management and
dayto-day administration of the contract. We reported on the major
challenges in developing and implementing plans for establishing effective
human capital practices, having key management and oversight processes and
procedures in place, and tracking data to measure system integrator
performance. In addition, we, as well as the Office of Management and
Budget, expressed concerns about the potential lack of competition during
the program's later years and the reliance on a single system integrator
for procuring the Deepwater equipment. We also reported there was little
evidence that the Coast Guard had analyzed whether the approach carried
any inherent risks for ensuring the best value to the government and, if
so, what to do about them.
Since fiscal year 2002, Congress has appropriated almost $1.5 billion for
the Deepwater program (see table 1), and as of September 2003, the Coast
Guard had obligated $596 million to ICGS and $120.4 million for government
program management, legacy asset sustainment, and facilities design work.
Table 1: Deepwater Program Appropriations
Dollars in millions
Fiscal year Appropriation
2002 $320
2003 478
2004 668
Total $1,466
Source: Coast Guard.
In response to congressional direction to assess the feasibility of
accelerating the Deepwater program,4 the Coast Guard reported in March
2003 that it could accelerate the implementation schedule from 20 years to
10 years and that this acceleration would provide increased operational
capability sooner to support maritime homeland security.
On March 1, 2003, the Coast Guard became part of DHS. A December 22, 2003,
acquisition decision memorandum from the DHS Investment Review Board's
(the Board) acting chairperson to the Coast Guard Commandant stated that
the Deepwater program has been designated as a level 1 investment, meaning
that it will be reviewed at the highest levels within the department.5
Further, because Deepwater interoperability within DHS and with the
Department of Defense will be a major program challenge, the DHS Joint
Requirements Council6 will be kept informed of Deepwater developments.
While decisions as to specific assets or capabilities have been deferred
to the Coast Guard acquisition executive, the Board will meet to discuss
actual or projected changes to the program that affect
4 Homeland Security Act of 2002, Pub. L. No. 107-296, sec.888(i), 116
Stat. 2135, 2250 (2002).
5 The Investment Review Board was established to provide visibility,
oversight, and accountability for significant, high-cost investments. It
conducts systematic reviews of investment proposals and approves key
decisions.
6 The DHS Joint Requirements Council will conduct program reviews annually
and prior to key decision points to oversee the requirements generation
process, validate mission needs statements, review cross-functional needs
and requirements, and make programmatic recommendations to the Board.
cost, schedule, or performance. Noting that the Coast Guard has proposed
accelerating the Deepwater program in fiscal year 2005, the Board directed
the Coast Guard to ensure that risk management planning receives
appropriate attention, that TOC is kept current, and that cost, schedule,
and performance are monitored by measuring actual data against the
baseline and projections to completion.
Key Components of Management and Oversight Have Not Been Effectively
Implemented
Complex, performance-based contracts such as Deepwater require effective
government oversight to ensure that the intended results are achieved and
that taxpayer dollars are not wasted. Both Coast Guard and ICGS officials
have acknowledged that an unusually large degree of collaboration and
partnership between the government and the system integrator must be in
place for the Deepwater acquisition to be successful. However, a year and
a half into the program, the key management and oversight components
needed to make the program effective have not been effectively
implemented. Integrated product teams (IPT) are the Coast Guard's primary
tool for managing the program and overseeing the contractor, but these
teams have struggled to collaborate effectively and accomplish their
missions. While the Coast Guard has a Deepwater human capital plan in
place to guide strategic planning for turnover among Deepwater personnel,
the plan is not being followed and vacancies exist in key positions.
Further, while it is still early in the program, the transition from
existing Coast Guard assets to the new Deepwater assets has not been
effectively communicated, a particular concern in light of schedule delays
for some of the first assets to be delivered. Finally, a number of plans
integral to the organization and management of the Deepwater program were
finalized much later than anticipated. Appendix II contains additional
information on these plans.
IPTs Have Had Difficulty Fulfilling Their Critical Management Function
IPTs are the Coast Guard's primary tool for managing the Deepwater program
and overseeing the system integrator. More than 30 of these teams,
comprised of Coast Guard, ICGS, and subcontractor employees from Lockheed
Martin and Northrop Grumman, are responsible for overall program planning
and management, asset integration, and overseeing the delivery of specific
Deepwater assets. However, the teams have struggled to effectively carry
out their missions. Our prior work at the Department of Defense has shown
that effective IPTs have (1) expertise to master different facets of
product development, (2) responsibility for day-to-day decisions and
product delivery, (3) key members who are either physically collocated or
connected through virtual means to facilitate team cohesion and the
ability to share information, and (4) control over their
membership, with membership changes driven by each team's need for
different knowledge.7
The Deepwater program manager reported IPT performance shortcomings as an
issue in 14 of the 16 monthly program assessment reports provided to us.
The following comments, made in Deepwater management reports by Coast
Guard officials involved on a number of different IPTs, convey the
difficulties faced by the teams in the first year and a half of the
program. Though the comments in table 2 are not exhaustive, they
demonstrate that the Deepwater IPTs have not been effective.
Table 2: Comments from Coast Guard Management Reports on IPT Performance
June 2002 through Dec. 2002
o Teams are still forming but are hindered with unfamiliarity of
Web-based data environment, integrated design and management processes
immaturity, and geographic separation.
o Because of aggressiveness of schedule, team development and
collaboration have been negatively affected.
o Team is making progress, but most other teams are not yet productive.
Team leaders are challenged by intense pace of work needed to keep up with
asset implementation plan.
Jan. 2003 through June 2003
o High demands and limited resources inhibit commitment to collaboration.
o Team progress is slowed by ineffective collaboration, resulting in
missed milestones.
o Limited collaboration in addressing design and production issues. Team
has been unable to resolve some comments in a timely fashion.
July 2003 through Dec. 2003
o Demands on limited personnel resources have restricted collaboration
in addressing some items in contract data requirements list in a timely
fashion for the 123-foot cutter.
o Team meetings for one asset have been canceled due to workload
associated with another. Team lead has been vacant for several months,
requiring the duties to be assumed by the Aviation Technical Director
and/or Aviation Program Manger. Task order performance has suffered, and
issues have not been resolved in a timely manner.
o Team has been unable to resolve some comments in a timely fashion.
Human resources within the team are taxed due to multitasking.
o There has been a lack of participation by some of the team members.
Meeting minutes, decisions, and such have not been documented as outlined
in the IPT charters. Important action items and risk mitigation plans are
not being consistently addressed, tracked, and resolved in a timely
manner.
o Team disconnects have contributed to delay in delivery of Coast Guard
cutter Matagorda.
Source: Coast Guard.
7 Best Practices: DOD Teaming Practices Not Achieving Potential Results,
GAO-01-510 (Washington, D.C.: Apr. 10, 2001). The Department of Defense
has used IPTs on major programs such as the Advanced Amphibious Assault
Vehicle program, a high-speed amphibious armored personnel carrier.
Based on our review of program reports, we identified four major issues
that have impeded the effective performance of the IPTs.
o Lack of timely charters to vest IPTs with authority for decision
making. Authority for day-to-day decisions-required for program success in
meeting cost, schedule, and performance objectives-is vested in IPTs
through charters; yet charters for most of the Deepwater IPTs were not
developed in a timely manner. In fact, 27 of the 31 IPT charters were not
approved until after the first year of the contract. More than merely a
paperwork exercise, sound IPT charters are critical because they detail
each team's purpose, membership, performance goals, authority,
responsibility, accountability, and relationships with other groups,
resources, and schedules. Between June 2002 and June 2003, 20 delivery
task orders,8 authorized for issuance by the
contracting officer, were executed by IPTs that did not have charters in
place. Similarly, we found that some sub-IPTs, which address specific
issues at a subasset or component level, were operating on an ad hoc basis
without charters. For example, a November 2002 management report states
that sub-IPTs addressed numerous issues concerning requirements for the
national security cutter, even though their charters were not approved
until a year later. In addition, two other sub-IPTs were not chartered.9
o Inadequate communication among members. The Coast Guard's Deepwater
program management plan has identified collocation of IPT members as a key
program success factor, along with effective communications within and
among teams. Face-to-face informal communication enhances information
flow, cohesion, and understanding of other members' roles-all of which
help foster team unity and performance. Yet only 3 of the 31 operating
IPTs are entirely collocated, meaning that every IPT member is in the same
building. The IPTs responsible for assets frequently have members in
multiple locations. For example, the logistics process and policy
development IPT has members in 6 different locations. As noted in table 2,
Coast Guard IPT members have raised geographic separation as an issue of
concern. ICGS developed a Web-based system for government and contractor
employees to regularly access and update technical
8 Delivery task orders are orders for supplies or services placed against
an existing contract. In the context of the Deepwater contract, these
would be orders placed by the Coast Guard with ICGS for the delivery of
specific assets or for other work.
9 These were for Deepwater's integrated logistics system and C4ISR.
information, training materials, and other program information, in part to
mitigate the challenges of having team members in multiple locations.
However, Coast Guard documents indicate that the system is not being
updated or used effectively by IPTs. In fact, the Deepwater program
executive officer reported that, while the system has great potential, it
is a long way from becoming the virtual enterprise and collaborative
environment required by the contractor's statement of work.
o High turnover of IPT membership and understaffing. Most of the
Deepwater IPTs have experienced membership turnover and staffing
difficulties, resulting in a loss of team knowledge, overbooked schedules,
and crisis management. In a few instances, such as the national security
cutter and maritime patrol aircraft, even the IPT leadership has changed.
Also, key system integrator officials serving on the management IPTs have
left the company. Both the Chief Financial Officer and the President of
ICGS left their positions during the latter half of 2003, and an
additional six of the nine ICGS Board members have changed. In addition,
Coast Guard and system integrator representatives have also been staffed
on multiple IPTs, and, in many cases meetings were attended by fewer than
50 percent of IPT members. A December 2002 Coast Guard document
summarizing various programmatic recommendations cited a contractor study
that recommended individuals be assigned to IPTs on a full-time basis and
that they not serve on more than two teams. However, as of December 2003,
15 individuals were serving on three or more IPTs.
o Insufficient training. The system integrator has had difficulty
training IPT members in time to ensure that they could effectively carry
out their duties, and program officials have referred to IPT training as
deficient. IPT charters state that members must complete initial training
before beginning team deliberations regarding execution of new contracts
for Deepwater assets. IPT training is to address, among other issues,
developing team goals and objectives, key processes, use of the Web-based
system, and team rules of behavior. According to a Coast Guard evaluation
report, IPT training was implemented late, which has contributed to a lack
of effective collaboration among team members.
The Coast Guard hired a consultant to survey IPT members concerning teams'
performance from July 2002 to September 2003. Three surveys consisted of
questions about mission, team member cooperation,
Human Capital Issues In our 2001 report,12 we noted that as the Deepwater
program got off the
Pose a Challenge ground, tough human capital challenges would need to be
addressed. A critical challenge we raised was the need to recruit and
train enough staff performance, communication, and integrated product and
process development.10 The final report on the survey results highlighted
the need for improved communications both within and among teams.
Respondents were also concerned that workloads were too high. 11
to manage and oversee the contract. To date, the Coast Guard has not
funded the number of staff requested by the Deepwater program and has not
adhered to the processes outlined in its human capital plan for addressing
turnover of Deepwater officials. These staffing shortfalls have
contributed to the problems IPT members have identified-such as the
struggle to keep pace with the workload and the difficulties in making
decisions due to inconsistent attendance at IPT meetings.
Although the Deepwater program has identified the need for a total of 264
staff in fiscal year 2004, only 224 positions have been funded, and only
209 have been assigned to the program. The Coast Guard's fiscal year 2004
funding for personnel was increased to $70 million; however, the Coast
Guard did not request sufficient funds to fill the 40 positions that the
Deepwater program identified as necessary. According to Coast Guard
officials, $70 million is insufficient to fund their fiscal year 2004
personnel plan because they need $67 million of this amount just to fund
current personnel levels. The assistant commandant has imposed a temporary
hiring freeze and plans to monitor expenditures throughout the year to
identify any available funding for additional positions. Although we
asked, Coast Guard officials did not explain why they did not request
sufficient funds to adequately staff the program.
10 Integrated product and process development is a management technique
that integrates all essential acquisition activities through the use of
multidisciplinary teams to optimize the design, manufacturing, and
supportability processes. One of the key integrated product and process
development tenets is that IPTs facilitate meeting cost and performance
objectives.
11 Due to concerns about low response rates, an analysis was done only on
those IPTs with response rates greater than 30 percent. Only 14 of the 27
teams contacted as part of the latest survey, conducted in September 2003,
had a response rate that allowed their results to be analyzed.
12 GAO-01-564.
In addition, the Coast Guard has not adequately addressed the imminent
departure of Coast Guard officials from the Deepwater program. Coast Guard
officials will leave each year due to the normal rotational cycle of
military members (every 3 to 4 years) and retirements. The Deepwater human
capital plan sets a goal of a 95 percent or higher "fill rate" annually
for both military and civilian positions and proposes using a "floating"
training position that can be filled by replacement personnel reporting
for duty a year before the departure of the military incumbents. This
position is meant to ensure that incoming personnel receive acquisition
training and on-the-job training with experienced Deepwater personnel.
However, the 2004 request for this training position was not funded, nor
was funding provided for additional new positions identified as critical.
In December 2003, the Director of Resources and Metrics and the Chief
Contracting Officer left the Deepwater program, and the program manager is
slated to leave in March 2004. In addition, by July 1, 2004, five key
Coast Guard officials who oversee the work of the asset IPTs are scheduled
to leave. Coast Guard officials told us that they have identified the
military replacements that will join the program in the summer of 2004.
Transition Planning for New Assets Needs Attention
Although Deepwater is still in the early stages, assets will start to be
delivered incrementally to operating units soon. The first Deepwater
assets-the 123-foot cutter and short range prosecutor13-are scheduled to
be delivered to operating divisions in 2004. Operating units will receive
additional ships, aircraft, or C4ISR every year thereafter until the
Deepwater program ends. However, the Coast Guard has not communicated
decisions on how the new and old assets are to be integrated during the
transition and whether Coast Guard or contractor personnel-or both-will be
responsible for maintenance. Coast Guard field personnel, including
senior-level operators and naval engineering support command officials,
told us that they have not received information about how they will be
able to continue meeting their missions using current assets while being
trained on the new assets. They are also unclear as to whether the system
integrator or Coast Guard personnel will be responsible for maintenance of
the new assets. For example, although Deepwater officials have stated that
maintenance on the new assets will
13 The short range prosecutor is a new rigid-hull inflatable small-boat
being introduced for the Deepwater cutters. This boat is 7.7 meters long,
may carry up to 10 personnel and 150 pounds of cargo, and may travel at
speeds up to 33 knots. Each modernized 123-foot cutter and the fast
response cutter will carry one short range prosecutor. The national
security cutter and the offshore patrol cutter will also be able to carry
the short range prosecutor.
be a joint responsibility, naval engineering support command staff had
received no instruction on how this joint responsibility is to be carried
out. Coast Guard officials told us that guidance on joint maintenance
responsibility has not been completely disseminated throughout the Coast
Guard, but said that ICGS has recently added representatives at the key
maintenance and logistics sites to coordinate maintenance issues.
One of the first Deepwater assets to be delivered is the 123-foot cutter.
The Coast Guard is modifying its 110-foot cutter by adding 13 feet of deck
and hull, a stern ramp, a superstructure, and communication equipment. The
123-foot cutter is an example of the transition challenges facing the
Coast Guard. First, there is confusion over which of the cutters will be
modified and when. The contract with ICGS calls for all 49 cutters to be
modified; however, Deepwater officials are considering curtailing the
modification efforts and accelerating the development of the fast response
cutter instead. (The fast response cutter was originally planned to be
delivered in 2018 as a replacement for the 123-foot cutter). In addition,
the Coast Guard identified 22 of the 110-foot cutters that, due to
unexpectedly severe hull corrosion, required additional inspection and
repair separate from the Deepwater modification plans. To date, $14.7
million in non-Deepwater funds has been made available to repair 8 of
these cutters. Further, Coast Guard officials note that there are 4
cutters in operation in the Persian Gulf, which makes them unavailable for
modification at this time. System integrator and Coast Guard officials
expressed confusion about the status of the cutter modifications, hull
repair program, and fast response cutter schedule. For example, ICGS
officials indicated that they did not know what the Coast Guard plans for
the 123-foot cutter modification. The Coast Guard is considering several
options and has not made a final decision on the cutter modification
effort.
Transitioning the staffing and operations of current Coast Guard assets to
Deepwater assets may be further complicated by schedule delays. Reliable
information on the delivery of Deepwater assets is important to the
planning and budgeting efforts of Coast Guard operators and maintenance
personnel to ensure that current missions are met and existing assets are
maintained. Delivery of the first 123-foot cutter and short range
prosecutor is scheduled for March 2004, slipping from the original
delivery date of November 2003, and the rescheduled date of December 2003.
This delay is affecting the schedules for the remaining cutters under
contract, according to the most recent program manager assessment. Program
management reports also indicate that schedule milestones have slipped for
the maritime patrol aircraft. The first two aircraft are currently
scheduled to be delivered to operating divisions in late 2006 or early
2007, compared
Process for Assessing System Integrator's Ongoing Performance Lacked Rigor
with the original plan of 2005. The IPT is working toward design of the
aircraft, even though Coast Guard approval to proceed has not been set
forth in the form of a definitized contract for this asset.14 The target
date for definitizing the contract is now April 2004.
According to Office of Federal Procurement Policy guidance, a
performance-based contract such as Deepwater should have measurable
performance standards and incentives to motivate contractor performance.
Contractors should be rewarded for good performance based on measurement
against predetermined performance standards. In general, the contractor is
to meet the government's performance objectives, at appropriate
performance quality levels, and be rewarded for outstanding work. Further,
sound internal controls are important to ensure that plans, methods, and
procedures are in place to support performance-based management. Relevant
information should be recorded and communicated to management and others
in a form and a time frame that enables them to carry out their
responsibilities.15
The Coast Guard's process and procedures for evaluating the system
integrator's performance during the first year of the contract lacked
rigor in terms of applying quantifiable metrics to assess performance,
gathering input from government performance monitors, and communicating
with and documenting information for the decision makers. The process used
to hold the system integrator accountable for results was not transparent
and, in fact, contained several inconsistencies that raise questions as to
whether the Coast Guard's decision to give the contractor 87 percent of
the award fee16 was based on accurate information.
14 The Federal Acquisition Regulation (FAR) states that a contract is to
be definitized within 180 days after the date of the letter contract or
before completion of 40 percent of the work to be performed, whichever
occurs first. FAR 16.603-2(c).
15 A Guide to Best Practices for Performance-Based Service Contracting,
Office of Federal Procurement Policy, Office of Management and Budget,
Executive Office of the President, Final Edition, October 1998, and
Internal Control: Standards for Internal Control in the Federal Government
(GAO/AIMD-00-21.3.1), U.S. General Accounting Office, November 1999.
16 An award fee is an amount a contractor may earn, either in whole or in
part, based on the contracting agency's judgmental evaluation of the
contractor's performance against established criteria (FAR 16.404,
16.405-2).
Quantifiable Award Fee Metrics Are Still Being Developed
The Coast Guard measures the system integrator's ongoing performance based
on periodic assessments using weighted evaluation factors. The award fee
for the first year of performance of the overall integration and
management of the Deepwater program was based on an evaluation of the
following five factors:17
o overall program management,
o cost monitoring and control,
o quality,
o innovation, and
o flexibility.
These evaluation factors are further defined in the contract's award fee
plan. For example, innovation is the "extent to which innovation, designs,
processes, and concepts have been introduced that result in operational
performance improvements and/or total ownership cost reductions." While
there will inevitably be a degree of subjectivity in award fee decisions,
the Coast Guard lacks quantifiable metrics to make an assessment of the
contractor's performance. Given the lack of specificity of the metrics, it
is not clear how they could be used to make such an assessment,
particularly on a program as complex as Deepwater. Coast Guard officials
acknowledged that the factors need to be better-defined, with supporting
metrics that would provide a more objective basis for future award fee
assessments.
In the meantime, a May 31, 2003, Coast Guard memorandum to ICGS indicates
that the contractor will be rated based on three factors for the second
year of performance rather than five. However, the factors are vague and
undefined: quality, program management, and system engineering. Further,
supporting metrics to measure these performance factors have not been
developed.
17 In a document provided to a congressional committee entitled "DHS
Procurement Policy Impacts on Deepwater," the Coast Guard stated that "the
annual award fee is based on the accomplishments of the small business
subcontracting plan." However, Coast Guard officials subsequently
confirmed that small business subcontracting is not an award fee
evaluation factor.
Information Was Not Gathered and Communicated Accurately
Under the Deepwater contract's award fee plan18 and the program management
plan, technical specialists, known as contracting officers' technical
representatives (COTR), are to provide their observations to a program
evaluation board comprised of the contracting officer, the program
manager, and two COTRs. The Deepwater program executive officer then makes
the final award fee determination based on the board's recommendations.
The Coast Guard's award fee evaluation of the first year of ICGS's
performance was based on unsupported calculations and relied heavily on
subjective considerations. As a result, the basis for the final decision
to provide the contractor an award fee rating of 87 percent, which falls
in the "very good" range, was not well-supported. 19
For example, while all COTRs submitted comments, the assessment did not
include numerical and adjectival ratings from all COTRs. Input from the
COTR responsible for gauging the system integrator's performance for all
efforts related to the design and delivery of ships was not included in
the calculation at all. Prior to speaking with us, the COTR did not know
his input was absent from final performance monitor calculations, which
resulted in a recommended rating of 82.5 percent. Input from two other
COTRs was provided for some but not all of the five evaluation factors. A
fourth COTR provided only adjectival ratings, whereas others provided
numerical scores. Subsequently, and unbeknownst to the COTR, a program
evaluation board member calculated a numerical score for this COTR's
observations. While an adjectival rating of "good," for example, could
range from a score of 71 to 80, the board member scored each of the
factors in the midrange. Scoring this COTR's adjectival ratings in the low
or high end of the range would have produced a different outcome. Program
evaluation board officials were not aware of the inconsistencies in the
calculations until we informed them.
One program evaluation board member raised concerns that the board's
subsequent award fee recommendation of 90 percent was too high and that
the assessment focused disproportionately on the system integrator's
performance in the last part of the year rather than its performance over
the entire year. In addition, the program manager's assessment stated that
"overall program management...needs substantial improvement." Further,
18 Coast Guard contract number DTCG23-02-C-2DW001, section J, attachment
J-14, Award fee plan, awarded June 25, 2002, amended November 8, 2002. The
plan indicates that changes to the award fee plan will be made through a
bilateral contract modification.
19 An adjectival rating of "very good" ranged from a score of 81 to 90.
Coast Guard management reports throughout the first year of the contract
cited various schedule, performance, cost control, and contract
administration problems that required attention. Among the assets cited as
needing attention were the maritime patrol aircraft, the short range
prosecutor, the 123-foot cutter, and the logistics integration management
system. Ultimately, the program executive officer awarded the system
integrator a rating of 87 percent, resulting in an award fee of $4.0
million of the maximum $4.6 million annual award fee.
Coast Guard officials told us that they will now assess ICGS's performance
every 6 months, rather than annually. However, the contract has not been
modified to reflect either the changes to the evaluation factors,
discussed previously, or the new assessment period. The first 6-month
assessment was scheduled for completion in December 2003, but as of March
2004, Coast Guard officials told us it is currently ongoing.
The contractor was eligible for a second award fee of up to $1.5 million
in August 2003 for performance related to the continuous improvement of
elements common to C4ISR and life cycle and logistics engineering for all
assets. Coast Guard officials said that they awarded the system integrator
79 percent of the maximum award fee; however, they did not provide us with
supporting documentation of the award fee determination process.
The Coast Guard is scheduled to decide on extending ICGS's contract by
June 2006, 1 year prior to the end of the first 5-year contract term. In
2001, the Coast Guard set a goal of developing measures, within a year
after contract award, to conduct annual assessments of the system
integrator's progress toward achieving the three overarching goals of the
Deepwater program: increased operational effectiveness, lower TOC, and
customer satisfaction. However, the Coast Guard's time frame for
implementing metrics to gauge progress against these goals has slipped.
Further, the baseline the Coast Guard is using to assess TOC will not
provide the government with critical information it needs about the
efficiencies of using the Deepwater approach. Therefore, the Coast Guard
is not in a position to begin the decision-making process about whether or
not to extend the contract past the 5-year base period.
The time frame for the first review of the contractor's performance
against the Deepwater goals has slipped. It was originally rescheduled for
18 months after contract award (December 2003), 6 months later than
planned. Deepwater officials told us that the performance review is
currently ongoing and is expected to be completed in March 2004. While
Coast Guard Has Not Begun to Measure Contractor Performance against
Deepwater Program Goals
the Coast Guard has begun to develop models to measure the extent to which
Deepwater is achieving increased operational effectiveness and reduced
TOC, a decision has not yet been made as to which specific suite of models
will be used. The former Deepwater chief contracting officer told us he
anticipates that the metrics will be in place in year 4 of the contract,
the same year the decision needs to be made to extend the contract. Other
officials acknowledged that it is difficult to hold the contractor
accountable for progress toward the goals this early in the program, but
could not offer a projection as to when the operational effectiveness and
TOC results would be forthcoming.
Coast Guard officials noted the large degree of complexity involved in
attempting to measure the system integrator's progress toward the
Deepwater goals. In previous work, we found that assessing improvements in
operational effectiveness and TOC may be difficult because performance
data may reflect factors that did not result from the contractor's
actions. Because the Deepwater program includes legacy assets, modified
assets, and new assets, the line of accountability between the government
and the system integrator is blurred. It is not always clear who is
responsible between Coast Guard and ICGS for the change in performance or
costs of Deepwater assets. Measuring the 123-foot cutter's performance,
for example, is complicated by the fact that ICGS is responsible for the
new 13 feet of deck and hull and other modifications, while the engine and
the other 110 feet of the deck and hull are the Coast Guard's
responsibility.
Coast Guard officials said that they are measuring "operational
performance," such as the number of search and rescue, drug interdiction,
and migrant interdiction missions carried out by the current assets.
However, they could not explain how these measures will be used to assess
ICGS's progress toward improving operational effectiveness with Deepwater
assets. The officials stated that the models they are using to measure
operational performance for the various Coast Guard missions lack the
fidelity to capture whether improvements may be due to Coast Guard or
contractor actions, the capability of specific Deepwater assets, or even
outside factors such as improved intelligence on drug smugglers. Program
officials noted that it is difficult to hold the contractor accountable
for operational effectiveness at this point, before Deepwater assets are
delivered.
Establishing a solid baseline against which to measure progress in
lowering TOC is critical to holding the contractor accountable. However,
the Coast Guard is using as the baseline ICGS's own projected cost of
$70.97 billion plus 10 percent (in fiscal year 2002 dollars). Therefore,
the government will not have the TOC information it needs to make a
contract extension decision. Measurement of ICGS's cost as compared to its
own cost proposal will tell the Coast Guard nothing about the efficiencies
it may be getting using the Deepwater performance-based approach. Further,
the baseline the Coast Guard is using has been significantly changed from
that originally envisioned. The Deepwater program management plan,
approved in December 2003, states that the estimated cost to replace
individual Coast Guard assets under a traditional approach, (i.e., without
the ICGS Deepwater "system of systems" solution), is to be the "upper
limit for TOC" that the contractor should not exceed. The officials could
not explain why the program management plan, which sets forth the overall
framework for managing Deepwater, contains a different TOC baseline than
the one they are using.
Further, changes in such variables as fuel costs or cutters' operating
tempo could result in additional changes being made to the TOC baseline.
Coast Guard officials explained that proposed changes to the baseline
would be approved by the program executive officer on a case-by-case
basis. However, the Coast Guard has not developed criteria for potential
upward or downward adjustments to the baseline.
The Coast Guard has only recently begun to address the contractor's
progress in meeting the third overall goal of Deepwater, customer
satisfaction. A January 9, 2004, report indicates that the Coast Guard had
not yet identified the metrics needed to measure this goal. As a start, on
January 12, 2004, a survey was sent to 25 senior leaders and program
managers.
The Coast Guard decided to use the system integrator approach 6 years ago.
In our previous work,20 we found that given the Coast Guard's reliance on
a single system integrator for the Deepwater program, the agency would be
at serious risk if it decides not to extend the contract. Because ICGS
proposed the specific assets that became the Deepwater solution, a
decision not to extend the current contract would require a new Deepwater
acquisition strategy to be developed. Exit strategies and other means to
deal with potential poor performance by the system integrator are
important to mitigate these program risks. However, the Coast Guard is
just beginning an internal review of the system integrator's plan to
20 GAO-01-564.
Control of Future Costs through Competition Remains a Risk because of Weak
Oversight of Subcontractor Decisions
transition out of the program in the event such action would be necessary.
Further, Deepwater program officials indicated that it is not realistic to
believe the Coast Guard would switch system integrators at this point in
the program. They stated that they viewed their relationship with the
contractor as a partnership and are committed to making it work.
Competition is a key component for controlling costs in the Deepwater
program and a guiding principle for DHS's major acquisitions. The benefits
of competition may be viewed as sufficient in the contract's early years
because, for the initial 5-year contract period, prices proposed by ICGS
for equipment and software were based on competitions held among various
subcontractors. However, beyond the first 5-year term, the Coast Guard has
no way to ensure competition is occurring because it does not have
mechanisms in place to measure the extent of competition or to hold the
system integrator accountable for steps taken to achieve competition.
The acquisition structure of the Deepwater program is such that the two
first-tier subcontractors, Lockheed Martin and Northrop Grumman-the
companies that formed ICGS and that developed the Deepwater solution-have
sole responsibility for determining whether to hold competitions for
Deepwater assets or to provide these assets themselves. Over 40 percent of
the funds obligated to Lockheed Martin and Northrop Grumman have either
remained with those companies or been awarded to their subsidiaries.
Further, the system integrator uses a Lockheed Martin sourcing document,
termed the open business model, to guide competition decisions made by the
subcontractors. However, this guidance is a philosophy-not a formal
process involving specific actions-that encourages competition but does
not require it. The lack of transparency into competition and the
government's lack of a mechanism to hold the contractor accountable raise
questions about whether the Coast Guard will be able to control costs.
Coast Guard Lacks Visibility into Subcontractors' Make or Buy Decisions
Neither the Coast Guard nor the system integrator determines how suppliers
for Deepwater assets are chosen. A Coast Guard official told us that the
system integrator was hired to make these decisions because the agency
lacked the expertise to do so. However, Lockheed Martin and Northrop
Grumman, as the subcontractors, are solely responsible for deciding
whether to hold competitions for Deepwater assets or provide them to the
Coast Guard themselves (often referred to as "make or buy" decisions).
Moreover, the Coast Guard has no contractual requirements with ICGS that
provide transparency into significant make or buy decisions. Although the
Coast Guard has decided to include achieving competition as one of the
factors to be considered in decisions about extending the contract for
future option terms, this review will occur after such subcontracting
decisions are made. The subcontractors are not required to notify the
Coast Guard prior to making a decision to provide Deepwater assets
themselves rather than holding a competition. The Coast Guard's review of
competition included in its award term plan21 will not address Lockheed
Martin or Northrop Grumman decisions-increasingly important in subsequent
years-of whether significant equipment should be procured from outside
sources or built in-house.
As of September 30, 2003, the Coast Guard had awarded $596 million in
orders to the system integrator, ICGS. Table 3 shows that over 98 percent
of this amount was then passed through to the two first-tier
subcontractors.
Table 3: Breakdown of Coast Guard Obligations to ICGS (Includes Planned
Subcontracts to First Tier Subcontractors as of September 30, 2003)
Dollars in millions
Dollar value Percentage
ICGSa $9
Subcontracts to Lockheed Martin 393 65.9
Subcontracts to Northrop Grumman 194 32.6
Total $596 100
Source: Coast Guard.
aObligations to ICGS consist of general and administrative charges
exclusively.
To date, the subcontractors managing the acquisition have frequently
performed the work themselves. Based on their respective work scopes, the
two companies either issue orders to second-tier subcontractors or
21 The Deepwater award term plan establishes the criteria for awarding
contract options and the evaluation process and periods.
retain the work for themselves.22 Table 4 shows that, as of September 30,
2003, Lockheed Martin planned to retain 42 percent of its obligated
dollars and to award 58 percent to second-tier subcontractors. Most of
these second-tier dollars will go to major subcontractors, i.e., those
with obligations greater than $5 million.
Table 4: Breakdown of ICGS Obligations to Lockheed Martin (Planned
Subcontracts and Second-Tier Obligations as of September 30, 2003)
Dollars in millions
Subcontractor Dollar value Percentage
Lockheed Martin in-house work $121
Subcontracts to Lockheed Martin subsidiaries 42
Other second-tier subcontractors
Major second-tier subcontractorsa 206
Other suppliers 24
Total $393 100
Source: Coast Guard.
aLockheed Martin defines major subcontractors as those with obligations of
more than $5 million.
As shown in table 5, Northrop Grumman planned to retain 51 percent of its
obligated dollars.
Table 5: Breakdown of ICGS Obligations to Northrop Grumman (Planned
Subcontracts and Second-Tier Obligations as of September 30, 2003)
Dollars in millions
Subcontractor Dollar value Percentage
Northrop Grumman in-house work $96
Subcontracts to Northrop Grumman
subsidiaries 3
Other second-tier subcontractors 95
Total $194 100
Source: Coast Guard.
22 ICGS's limited liability company agreement articulates each member's
work scope. Lockheed Martin "shall be responsible for program management;
systems design; C4ISR assets and capabilities; aviation assets; and
logistics assets and capabilities..." Northrop Grumman (Ingalls) shall be
responsible for program management and logistics for surface assets..."
The System Integrator's Open Business Model Does Not Ensure Competition Is
Considered
The open business model, meant to guide the supplier sourcing process in
the Deepwater program, has been characterized by the system integrator as
a means of ensuring competition for Deepwater assets throughout the life
of the program, thereby keeping costs under control. In October 2003, ICGS
issued a policy statement on the open business model. The stated business
approach of the guidance is to encourage second-tier suppliers to remain
innovative and competitive by directing Lockheed Martin and Northrop
Grumman, as the first-tier subcontractors, to (1) generally avoid the use
of teaming agreements with suppliers and prohibit teaming agreements based
on guaranteed work share,23 (2) defer second-tier supplier decisions as
long as practicable so that changes in the marketplace can be considered,
and (3) actively solicit market information and new suppliers. However,
this guidance is a philosophy-not a formal process involving specific
decision points-that does not ensure that competition will be considered.
The December 2003 Deepwater performance measurement plan requests that the
contractor prepare selfassessments of its efforts to promote competition.
However, the Coast Guard has no means of obtaining insight into the basis
for the contractor's self-assessments. Moreover, the government still
lacks a mechanism to hold the contractor accountable for ensuring that
competition occurs.
To date, there have been varying degrees of competition for the secondtier
subcontracting relationships Lockheed Martin and Northrop Grumman have in
place for the design, development, or production of Deepwater assets.
Lockheed Martin and Northrop Grumman follow their own procurement
procedures and guidance for determining whether competition will occur and
selecting the suppliers who will be invited to compete for Deepwater
assets. The competitions are not "full and open"24 in the way a typical
government procurement would be, nor are they required to be. The federal
procurement system requires "full and open" competition except in cases
where certain statutory exceptions are met. "Full and open" competition
means that all responsible sources are permitted to compete.
23 Teaming agreements are defined in FAR 9.601 as an arrangement between
two or more companies to form a partnership or joint venture to act as a
potential prime contractor or a potential prime contractor agrees with one
or more companies to have them act as its subcontractor. Under guaranteed
work shares, a subcontractor is guaranteed a certain percentage of the
work.
24 10 U.S.C. sec. 2302(3)(D); 41 U.S.C. sec. 403(b).
ICGS officials identified four specific assets for which they believe the
open business model philosophy was effective: the conversion of 110-foot
to 123-foot cutters, the national security cutter, the maritime patrol
aircraft, and the vertical take-off and landing unmanned aerial vehicle
(VUAV). We found that, in some cases, teaming agreements were implemented
in the proposal phase of Deepwater and were carried over when ICGS won the
contract. In other cases, some degree of competition had occurred.
o In December 1998, Lockheed Martin Corporation, Ingalls Shipbuilding,
Inc.,25 and Halter-Bollinger Joint Venture, L.L.C entered into a teaming
agreement that included the 123-foot cutter modification. The agreement
was established to make the capabilities of both Halter and Bollinger
available to Lockheed Martin for all phases of the Deepwater program.
Despite the open business model's prohibition of work share agreements,
such an agreement is in place between Lockheed Martin and
Halter-Bollinger. Halter-Bollinger will be responsible for the design and
construction of all vessels equal to or less than 200 feet in overall
length, with the exception of the national security cutter. For those
ships greater than 200 feet and less than 241 feet, the company's work
share is 25 percent of the total effort.
o The national security cutters are being designed and constructed by
Northrop Grumman. Northrop Grumman awarded a contract to M. Rosenblatt &
Son, Inc. for the cutters' preliminary design, but Northrop Grumman is
responsible for the detailed design and construction. Lockheed Martin is
responsible for the electronics. However, Northrop Grumman plans to hold
competitions for the long-lead materials-such as the gas turbine, bulkhead
seals, stern tubes,26 and rudder-for the cutters and has solicited pricing
proposals from a number of subcontractors.
o Prior to the contract award to ICGS, Lockheed Martin solicited
information from a number of companies for the maritime patrol aircraft,
evaluating 16 aircraft proposals. In October 2000, Lockheed Martin signed
a memorandum of understanding with CASA Aircraft USA, Inc. to provide an
airframe for Deepwater and to help develop and market ICGS's Deepwater
proposal. After some Coast Guard
25 Ingalls Shipbuilding, Inc. was subsequently acquired by Northrop
Grumman. 26 A stern tube is a tube through which the tail shaft passes to
the propeller.
Conclusion
officials expressed concern about the aircraft model that had been
selected for Deepwater, ICGS was awarded a task order to pay for an
evaluation of alternative aircraft. As a result of the evaluation,
Lockheed Martin identified an alternative CASA aircraft to meet the Coast
Guard's maritime patrol aircraft mission.
o For the VUAV, Lockheed Martin conducted a competition between six
models. Bell Helicopter, Inc. was initially identified as the solution.
After ICGS submitted its Deepwater proposal to the Coast Guard, Lockheed
Martin identified a potential Northrop Grumman product based on market
research. However, after evaluation of this alternative, Lockheed Martin
selected one of the Bell products.
The Coast Guard has embarked on a major transformational effort using an
acquisition strategy that allows a system integrator to identify the
Deepwater assets and to manage the acquisition process, with
subcontractors retaining authority for all make or buy decisions. Such a
strategy carries inherent risks that must be mitigated by effective
government oversight of the contractor. The Coast Guard faces a tough
challenge in holding ICGS accountable for results, while facing the
daunting prospect of starting over with a new approach should the
contractor fail. Nevertheless, the integrity of the contractor oversight
process must be enforced through such mechanisms as effective IPTs and a
rigorous and transparent award fee determination process.
Further, the Coast Guard must determine how to hold the contractor
accountable for achieving the basic goals of the Deepwater program in
order to position itself to make a contract extension decision. While
there is no question that the success of Deepwater depends on an effective
partnership between the government and the contractor, the Coast Guard
must preserve its ability to change course if necessary. Solid baselines
need to be developed so actual costs and operational effectiveness of the
Deepwater assets can be accurately measured and reported. The current use
of the contractor's proposed costs, plus 10 percent, as the TOC
baseline-rather than the estimated cost to replace the assets via a
traditional procurement approach-is troublesome. Further, because the
program management plan does not reflect the change to the TOC baseline,
we question whether this decision was well thought-out and in the
government's best interest.
In addition to contractor oversight, the Coast Guard has not invested the
resources needed to ensure that its own personnel are trained and staffed
in sufficient numbers to carry out their duties. The disconnect between
the process outlined in the human capital plan for ensuring a smooth
transition as military personnel rotate out of Deepwater and the current
situation-where key Deepwater officials are leaving the program without a
chance to adequately train their replacements-is cause for concern as the
Deepwater program moves forward. It is unclear why the Coast Guard has not
devoted adequate attention to human capital needs. In addition, although
the first Deepwater assets are just starting to be delivered, the lack of
a solid and well-developed transition plan from legacy to Deepwater assets
is already causing problems, as evidenced by the 123foot cutter
modification difficulties. The schedule delays for several of the assets
further highlight the need for more focus on the transition to Deepwater
assets.
The concerns we raised in 2001 about the Coast Guard's ability to control
costs in future years remain valid today. Without a mechanism to hold the
system integrator accountable for ensuring adequate competition, the Coast
Guard cannot be sure that competition will be used to guard against cost
increases that could jeopardize the program. This situation is especially
risky given the acquisition structure of Deepwater, whereby the
subcontractors, not the system integrator or the Coast Guard, are
responsible for determining whether competition will occur for Deepwater
assets.
Recommendations for We recommend that the Secretary of Homeland Security
direct the Commandant of the Coast Guard to take the following three
actions to
Executive Action address Deepwater program management:
o In collaboration with the system integrator, take the necessary steps
to make IPTs effective, including
o training IPT members in a timely manner,
o chartering the sub-IPTs, and
o making improvements to the electronic information system that would
result in better information sharing among IPT members who are
geographically dispersed.
o Follow the procedures outlined in the human capital plan to ensure
that adequate staffing is in place and turnover among Deepwater personnel
is proactively addressed.
o As Deepwater assets begin to be delivered to operational units, ensure
that field operators and maintenance personnel are provided with
timely information and training on how the transition will occur and how
maintenance responsibilities are to be divided between system integrator
and Coast Guard personnel.
Further, we recommend that the Secretary direct the Commandant to take the
following six actions to improve contractor accountability:
o Develop and adhere to measurable award fee criteria consistent with
the Office of Federal Procurement Policy's guidance.
o In all future award fee assessments, ensure that the input of COTRs is
considered and set forth in a more rigorous manner.
o Hold the system integrator accountable in future award fee
determinations for improving the effectiveness of IPTs.
o Based on the current schedule for delivery of Deepwater assets,
establish a time frame for when the models and metrics will be in place
with the appropriate degree of fidelity to be able to measure the
contractor's progress toward improving operational effectiveness.
o Establish a TOC baseline that can be used to measure whether the
Deepwater acquisition approach is providing the government with increased
efficiencies compared to what it would have cost without this approach.
o Establish criteria to determine when the TOC baseline should be
adjusted and ensure that the reasons for any changes are documented.
To facilitate controlling future costs through competition, we also
recommend that the Secretary direct the Commandant to take the following
two actions:
o Develop a comprehensive plan for holding the system integrator
accountable for ensuring an adequate degree of competition among
second-tier suppliers in future program years. This plan should include
metrics to measure outcomes and consideration of how these outcomes will
be taken into account in future award fee decisions.
o For subcontracts over $5 million awarded by ICGS to Lockheed Martin
and Northrop Grumman, require Lockheed Martin and Northrop Grumman to
notify the Coast Guard of a decision to perform the work themselves rather
than contracting it out. The documentation should include an evaluation of
the alternatives considered.
Agency Comments
DHS forwarded us the Coast Guard's written comments on a draft of this
report, which are reproduced in appendix I. The Coast Guard provided us
with additional technical comments, which we incorporated as appropriate.
In an e-mail sent subsequent to the written comments, the Coast Guard
stated that it agreed with our recommendations.
The Coast Guard noted that the agency is learning and evolving as the
Deepwater program matures and pointed out that many aspects of the
Deepwater program-working with a system integrator, employing IPTs across
multiple acquisition domains, and using a performance-based strategy for
such a long-term undertaking-are new to the Coast Guard. The agency agreed
that, because the IPT structure is new to the Coast Guard, many
adjustments must be made to improve the teams' effectiveness. The Coast
Guard clarified that the IPTs are, for the most part, contractor-led and
that Coast Guard IPT members provide support and oversight. The focus of
this report, however, is on the government's ability to oversee and manage
the contractor. Deepwater management documents assert, as we point out in
our report, that IPTs are the Coast Guard's primary tool for managing and
overseeing the contractor.
Regarding the award fee process, the Coast Guard stated that it has taken
action to assimilate objective factors into future evaluations but
expressed concern that our draft report may not have completely reflected
the rigor that was applied in the first award fee decision. The Coast
Guard states that "no input from any of the monitors was left out of the
evaluation process." While we revised our report to state that all COTRs
submitted comments, the input from the COTR responsible for ships was not
included in the numerical scores, which were then passed on to the fee
determining official. Further, the Coast Guard said that the score of 87
percent is "much lower than industry averages." In our view, however, the
relevant consideration in determining the award fee amount is not industry
averages, but rather the purpose an award fee is intended to serve. The
rationale for offering award fees is to motivate superior effort on
specific task and delivery orders, assets, or system performance
attributes. Of importance here, the narrative description in the Deepwater
award fee plan associated with a score of 87 percent ("very good") is
"very effective performance, fully responsive to contract requirements . .
. only minor deficiencies." As we state in our report, program management
reports throughout the first year of the contract cited various schedule,
performance, cost control, and contract administration problems that
required attention.
The Coast Guard agreed that competition is critical to controlling costs
and indicated that it is planning efforts that will result in greater
visibility and increased accountability to ensure competitive practices
are being used to manage costs.
Scope and Methodology
To determine the steps taken by the Coast Guard to manage the Deepwater
program and oversee system integrator performance, we examined the
Deepwater contract, the program management plan, the human capital plan,
briefings, budget justifications, and monthly and quarterly management
reports. We analyzed IPT charters, membership lists, survey data, and
staffing data, and we observed IPT and working group meetings. We
interviewed various Deepwater program officials representing the Coast
Guard, the system integrator, and the subcontractors, including program
and asset-level program managers, contracting officers, and ICGS
representatives in Arlington, Virginia, and Washington, D.C.. We visited
the First and Seventh Coast Guard Districts in Boston, Massachusetts, and
Miami, Florida, and interviewed operators and systems specialists for
Coast Guard cutters, aircraft, and helicopters at those locations. We also
met with Lockheed Martin and Northrop Grumman employees in Avondale,
Louisiana, and Moorestown, New Jersey. We reviewed our prior reports and
testimonies on the Deepwater project and integrated product teams.
To assess Coast Guard efforts to establish effective criteria to assess
and reward the system integrator's performance after the first year of the
contract, we reviewed the award fee plan, the performance incentives plan,
the interim and final award fee reports for the first year of contract
performance, and other management documents. We interviewed Coast Guard
and ICGS officials. Our analysis of this issue was hindered by the Coast
Guard's failure to provide us with two additional award fee
determinations, despite our repeated requests.
To assess whether the Coast Guard has put in place measures to assess the
contractor's progress in meeting the three overarching goals of Deepwater,
we reviewed the performance measurement plan, the award term plan, and
other performance measurement documents. Additionally, we interviewed the
Deepwater program's Resources and Metrics staff, Coast Guard operations
personnel, and program managers. We also reviewed our prior report on
performance-based contracting attributes.
To determine whether the Coast Guard is addressing the role and extent of
competition for Deepwater assets, we examined ICGS's open business
model policy statement and excerpts from Lockheed Martin's procurement
manual and Northrop Grumman's acquisition policy manual. We discussed the
open business model with officials from the Coast Guard, ICGS, Lockheed
Martin, and Northrop Grumman. In addition, we reviewed financial data,
including contract orders and ICGS spreadsheets. The Coast Guard provided
us with the obligations to ICGS, Lockheed Martin, and Northrop Grumman. We
did not independently verify the financial data.
We performed our work from May 2003 through February 2004 in accordance
with generally accepted government auditing standards.
We are sending copies of this report to other interested congressional
committees, the Secretary of Homeland Security, and the Commandant of the
Coast Guard. We will make copies available to others upon request. In
addition, the report will be available at no charge on the GAO Web site at
http://www.gao.gov.
If you or your staff have any questions regarding this report, please
contact me at (202) 512-4841 or Michele Mackin, Assistant Director, at
(202) 512-4309. Other major contributors to this report were Penny
Berrier, Ramona L. Burton, Christopher Galvin, Lucia DeMaio, Gary
Middleton, and Ralph O. White Jr.
Sincerely yours,
William T. Woods Director, Acquisition
and Sourcing Management
Page 32 GAO-04-380 Deepwater Program
Page 33 GAO-04-380 Deepwater Program
Page 34 GAO-04-380 Deepwater Program
Appendix II: Status of Selected Deepwater
Contract Management and Oversight Plans
Planned
completion date Approved
Scheduled life of plan
Plan Purpose
Award fee plan Establishes criteria and procedures used to Contract award, June
2002 Updated annuallya
evaluate Integrated Coast Guard Systems' (ICGS) management performance,
and to determine the amount of award fee earned
June 2002
Program Provides the management December 2003 Dynamic
framework, December 2002
organizational structure,
schedule, document,
management activities/events,
plan tasking, and process periodically
guidelines required to
implement phase 2 of the updated
program; includes
additional plans as noted
below
Details responsibilities
Contract and processes to implement December 2003
December 2002
management the contract
support plan
Describes activities and
Financial processes to ensure December 2003
December 2002
funding is available to
management execute the program
plan
Establishes structure and
Risk method for identifying and December 2003
"Soon after
managing risks, and
developing and selecting
management contract award"
plan options to mitigate risks
Links the program budget to
Performance performance and December December 2003
2002 Updated on a
measurement provides strategic focus semiannual basis
linking performance
measures to program
plan mission, vision, and
guiding
principles
Outlines the processes and
procedures used to 90 days
Quality after September 2002
implement the program's
assurance plan quality assurance process
contract award
(September 2002)
Establishes a team Not Remains in
Performance to evaluate ICGS's applicable March 2003 effect
efforts to
exceed performance until
incentives plan standards set by terminated,
the contract
and performance changed, or
measurement plan
amended
Establishes
criteria for June 2002 December Duration of
contract term 2003
Award term plan incentives, and
evaluation process contract, with
and periods
procedures for
changes
Source: GAO analysis of Deepwater documents.
aAn updated award fee plan was not available as of December 31, 2003.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Contact:
To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, [email protected] (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***