Defense Acquisitions: Role of Lead Systems Integrator on Future
Combat Systems Program Poses Oversight Challenges (06-JUN-07,
GAO-07-380).
The Army's Future Combat Systems (FCS) program features multiple
new systems linked by a first-of-a-kind information network. The
Army contracted with a lead systems integrator (LSI) for FCS that
could serve in a more expansive role than a typical prime
contractor would. In response to a congressional mandate, this
report addresses (1) why the Army decided to employ an LSI for
the FCS program; (2) the nature of the LSI's working relationship
with the Army; and (3) how FCS contract fees, provisions, and
incentives work. In conducting its work, GAO reviewed extensive
program documentation and held discussions with key officials at
DOD and throughout the FCS program.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-380
ACCNO: A70365
TITLE: Defense Acquisitions: Role of Lead Systems Integrator on
Future Combat Systems Program Poses Oversight Challenges
DATE: 06/06/2007
SUBJECT: Army procurement
Cost analysis
Defense procurement
Department of Defense contractors
Fixed price incentive contracts
Program evaluation
Program management
Systems design
Strategic planning
Cost estimates
Army Future Combat Systems
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-380
* [1]Results in Brief
* [2]Background
* [3]Army Decision to Use LSI Framed by Scope of Program and Work
* [4]Concurrent Development and Integration of FCS Is Most Comple
* [5]Mismatches between Scope and Pace of FCS and Capacity of Arm
* [6]Other Goals in Using an LSI on FCS
* [7]Close Working Relationship between Army and LSI Increases th
* [8]Boeing Is a Key Supplier for Critical Software
* [9]Close Working Relationship May Pose Risks
* [10]FCS Program Can Benefit from Stronger OSD Oversight
* [11]Contract Provides Incentives for Best Effort but Not Account
* [12]FCS Contract Provides the LSI a High Level of Compensation
* [13]Majority of Program Fees and Costs Are Available to the LSI
* [14]Connecting Contract Performance with Program Performance Wil
* [15]Conclusions
* [16]Recommendations
* [17]Agency Comments and Our Evaluation
* [18]GAO's Mission
* [19]Obtaining Copies of GAO Reports and Testimony
* [20]Order by Mail or Phone
* [21]To Report Fraud, Waste, and Abuse in Federal Programs
* [22]Congressional Relations
* [23]Public Affairs
Report to Congressional Committees
United States Government Accountability Office
GAO
June 2007
DEFENSE ACQUISITIONS
Role of Lead Systems Integrator on Future Combat Systems Program Poses
Oversight Challenges
GAO-07-380
Contents
Letter 1
Results in Brief 2
Background 4
Army Decision to Use LSI Framed by Scope of Program and Workforce
Limitations 6
Close Working Relationship between Army and LSI Increases the Burden of
Oversight 10
Contract Provides Incentives for Best Effort but Not Accountability for
Program Outcomes 18
Conclusions 26
Recommendations 27
Agency Comments and Our Evaluation 27
Appendix I Scope and Methodology 31
Appendix II Overview of FAR Provisions Included in Future Combat Systems
Lead Systems Integrator Contract 33
Appendix III Comments from the Department of Defense 35
Table
Table 1: Fee events and schedule for FCS contract 22
Figures
Figure 1: Key Events in FCS Program's Acquisition 6
Figure 2: RDT&E Percentage Increase throughout the Product Development
Cycle for 29 Programs Completed or in Production 23
Abbreviations
CAS Cost Accounting Standards
DFARS Defense Federal Acquisition Regulation
Supplement
DOD Department of Defense
FAR Federal Acquisition Regulation
FCS Future Combat Systems
LSI lead systems integrator
OCI Organizational Conflict of Interest
PIA Procurement Integrity Act
SAIC Science Applications International Corporation
SOSCOE System of Systems Common Operating Environment
TINA Truth in Negotiations Act
WMI Warfighter Machine Interface
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
June 6, 2007
Congressional Committees
With the Future Combat Systems (FCS) program, the Army is embracing a new
warfighting concept by replacing most of its combat systems with a family
of manned and unmanned vehicles and systems linked by an advanced
information network. The warfighting concept embodied by FCS involves
breaking with Army traditions and making unprecedented technological
leaps. After several decades of developing systems through disconnected
warfighter communities within the Army, FCS is proposed as an integrated,
system-of-systems concept rather than having integration occur after
systems are produced. Using this new approach, the Army intends to manage
trade-offs across individual warfighter communities and weapon systems,
save cost, and accelerate delivery. Recent estimates show that the total
investment cost of the FCS program will be at least $163 billion and
perhaps over $200 billion when complementary programs are included.
To achieve its goals for the FCS program, in 2003 the Army decided to
employ a lead systems integrator (LSI) to assist in defining, developing,
and integrating FCS. In the past few years, DOD and other agencies have
applied the LSI concept in a variety of ways. In the case of the FCS
program, the role of the LSI is not simply that of a traditional prime
contractor, but also includes some elements of a partner to the government
in ensuring the design, development and prototype implementation of the
FCS network and family of systems. The pros and cons of LSI arrangements
have been the subject of much debate on the FCS and other programs, such
as the Coast Guard's Deepwater program. The Institute for Defense
Analyses, the Army Audit Agency, and GAO have highlighted potential risks
with the LSI approach on the FCS program. In light of the scope of the FCS
program and its associated cost estimates, as well as concerns over the
role of the LSI, Congress mandated that GAO review and report on the use
of an LSI on FCS.1 Our letter dated March 15, 2007 satisfied the reporting
requirements for that mandate. As agreed with committee staff, this report
addresses the questions from the mandate, specifically: (1) why the Army
decided to employ an LSI for the FCS program; (2) the nature of the LSI's
working relationship with the Army; and (3) how the contract fees,
provisions, and incentives work.
1John Warner National Defense Authorization Act for Fiscal Year 2007, Pub.
L. No. 109-364, S 115 (2006).
In conducting our work, we reviewed extensive program documentation
pertaining to the business arrangement between the Army and LSI. This
included such items as the program contract and acquisition strategy. In
addition, we held discussions with key officials at DOD and throughout the
FCS program, including representatives from the Army, the LSI and various
subcontractors. We also drew significantly on existing GAO work on weapon
system acquisitions. We conducted our work from May 2006 through June 2007
in accordance with generally accepted government accountability standards.
Further detail about our scope and methodology is provided in Appendix I
of this report.
Results in Brief
The Army's decision to contract with a lead systems integrator for the FCS
program was framed by two factors: (1) the ambitious goals of the FCS
program and (2) the Army's capacity to manage it. As envisioned in 2003
when the program started, FCS presented a daunting technical and
management challenge: the concurrent development of multiple weapon
systems whose capabilities would be dependent on an information network
also to be developed. All of this was to take place in about 5 1/2
years--much faster than a single weapon system typically takes. Army
leaders believed the Army did not have the workforce or flexibility to
manage development of FCS on its own within desired timelines. The Army
saw its limitations in meeting this challenge as (1) cultural: difficulty
in crossing traditional organizational lines, (2) capability: shortage of
skills in key areas, such as managing the development of a large
information network, and (3) capacity: insufficient resources to staff,
manage, and synchronize several separate programs. Thus, they used a
contractor-- a Lead Systems Integrator-with significant program management
responsibilities to help it define and develop FCS and reach across
traditional Army mission areas. In May 2003, the Army contracted with the
Boeing Company to serve as the LSI for the FCS system development and
demonstration phase. Boeing subcontracted with Science Applications
International Corporation to assist in performing the LSI functions.
The relationship between the Army and the LSI is complex. On the one hand,
the LSI plays the traditional role of developing a product for its
customer, the Army, and on the other hand the LSI acts like a partner to
the Army in ensuring the design, development and prototype implementation
of the FCS network and family of systems. In forging a close partner-like
relationship with the LSI, the Army sought to gain advantages such as
maintaining flexibility to deal with shifting priorities. The Army saw
this as needed because the LSI was to help define the FCS solution as well
as to develop it. At the same time, this relationship, coupled with the
vast scope of FCS and the synonymy of the program with the future Army,
poses risks for the Army's ability to provide oversight over the long
term. OSD is in a position to provide this oversight, but thus far has
largely accepted the program and its changes as defined by the Army, even
though it is at wide variance from the best practices embodied in OSD's
own acquisition policies. In 2003, OSD approved the FCS for system
development and demonstration despite the program's combination of
immature technologies and short schedule and then declined to follow
through on plans to make a better informed decision 18 months later. OSD
has also allowed the Army to use its cost estimates rather than OSD's own
independent--and significantly higher--cost estimates when submitting
budget requests.
The Army has structured the FCS contract consistent with its desire to
incentivize development efforts and make it financially rewarding for the
LSI for making such efforts. In general, contracts are limited in that
they cannot guarantee a successful outcome. This is true for the FCS
contract, and specific aspects of the contract could make it even more
difficult to tie the LSI's performance to the actual outcomes of the
development effort. By the time the FCS critical design review is
completed in 2011, the Army will have paid out over 80 percent of total
costs of the LSI contract and the LSI will have had the opportunity to
earn more than 80 percent of its total fee. While the Army rationally
notes that it is important to use fees to encourage good performance
early, the experiences of previous weapon systems shows that most cost
growth occurs after the critical design review. The Army shares
responsibility with the LSI for making key decisions and to some extent
the Army's performance may affect the performance of the LSI. For example,
the Army has to bear responsibility for the successful completion of
essential complementary programs and separate technology development
efforts. Accordingly, the Army has to make important judgments about what
contract outcomes and changes it is responsible for versus the LSI. As
with most cost-reimbursable research and development contracts, the LSI is
responsible to put forth its best efforts on the development of the FCS
capability. If, given that effort, the FCS capability falls short of
needs, the LSI is not responsible and is still entitled to have its costs
reimbursed and may still earn its full fee.
We are making several recommendations to the Secretary of Defense,
including: (1) to reassess OSD's approach to overseeing the FCS program,
including asserting its own markers for success, particularly in the areas
of cost, technology maturity, design maturity, and production maturity;
and (2) to assess whether the experience of the LSI on FCS has broader
implications for acquisition management, such as the ability of the DOD
workforce to manage a system-of-systems acquisition. In commenting on the
draft of this report, DOD concurred with our recommendations. However, DOD
believes that the business relationship between the Army and LSI for the
FCS program is typical when compared to other major system acquisitions.
We maintain our position that the scope and complexity of the FCS
system-of-systems creates a business relationship that is not typical of
other weapon system acquisitions. Further, DOD believed that the
characterization of the requirements definition process in the report was
inaccurate. We have clarified our characterization of the LSI role but
maintain our position that the LSI has a significant role in the
requirements definition and refinement process for the FCS program. DOD's
comments are included in their entirety in Appendix III of this report.
Background
In May 2003, the Army and Boeing entered into an "other transaction
agreement" for the system development and demonstration phase of the FCS
program. Other transaction agreements are not subject to the Federal
Acquisition Regulation (FAR) and this gave the Army considerable
flexibility to negotiate the terms and conditions with Boeing as the LSI.
The Army's rationale for using such an agreement was to encourage
innovation and to use its wide latitude in tailoring business,
organizational, and technical relationships to achieve the program goals.
Congress raised concerns over the use of the agreement for the development
of a program as large and risky as FCS, and the Secretary of the Army
directed that the other transaction agreement be converted to a FAR-based
contract. In March 2006, the Army definitized a FAR-based contract with
Boeing for the remainder of FCS development. Science Applications
International Corporation (SAIC) has a contract with Boeing to provide
assistance in performing the LSI functions. All of the work performed from
May 2003 through September 2005 is accounted for under the prior other
transaction agreement, and all work after September 2005 is included under
the new contract. Appendix II of this report provides a brief discussion
of the conversion of the FCS contract from an other transaction agreement
to a FAR-based contract.
The LSI as an entity is intended as a single contractor responsible for
developing and integrating the FCS system of systems within a given budget
and schedule. Furthermore, the LSI was intended to act throughout the
system development and demonstration phase to optimize the FCS capability,
maximize competition, ensure interoperability and maintain commonality to
reduce life-cycle cost. The Army established a number of key tenets that
it wanted to achieve on the FCS program, in partnership with the LSI. They
include:
o create opportunity for best of industry to participate;
o leverage government technology base to maximum extent;
o associate ongoing enabling efforts with LSI-led activity;
o maintain a collaborative environment from design through
life-cycle;
o as a minimum, achieve commonality at subsystem/component level;
o design/plan for technology integration and insertion;
o maintain and shape the industrial base for the future;
o retain competition throughout future force acquisition;
o have appropriate government involvement in procurement
processes;
o achieve consistent and continuous definition of requirements;
o maintain and shape government acquisition community;
o achieve program affordability--balance performance and
sustainment; and
o have a "one team" operating with partnership and teamwork.
An LSI creates an additional tier in the management structure of the FCS
program that would not appear in an acquisition program for a major
individual system. The Army itself does not have a direct contractual
relationship with the prime-item developers as it would when buying a
single system, but rather works through the LSI. This additional tier
serves as a layer of separation between the customer (the Army) and the
platform developers (the tier immediately below the LSI). The Army
believes that this additional layer is required to bring all of the
developers together to a single point of communication and interaction for
the Army. The Army, LSI and platform developers are all members of the
"one-team" structure of the FCS program.
The current contract between the Army and Boeing continues into fiscal
year 2015 and the Army intends to begin low-rate initial production in
2013. The Army intends to start full-rate production in 2016. The Army
plans to achieve initial operational capability in fiscal year 2015 and
full operational capability in fiscal year 2017. The Army intends to
continue FCS procurement through fiscal year 2030, eventually equipping 15
brigade combat teams--about one-third of the current active force.
Figure 1 shows the schedule of key events for the FCS program.
Figure 1: Key Events in FCS Program's Acquisition (in Fiscal Years)
The Army has recently decided to make a number of key changes to the FCS
program as it was considering funding plans for the fiscal 2008-2013
period. These changes include eliminating or deferring certain FCS
systems, increasing quantities of some systems, reducing quantities of
other systems, reducing annual procurement rates, and delaying key program
milestone dates. For example, the Army deleted or deferred four systems
from the FCS system of systems architecture and delayed the start of
initial FCS production by five months.
Army Decision to Use LSI Framed by Scope of Program and Workforce Limitations
The FCS program's complexity and aggressive schedule is unprecedented for
the Army. As we have reported, the program was not near ready for starting
the system development and demonstration phase when it did, primarily
because the majority of the needed technologies were immature. The Army
not only went forward with FCS, it did so with a planned schedule less
than that of a single new system. The Army determined that with its
existing acquisition workforce and organizations, it did not have the
agility, capability, or capacity to manage the program without an LSI to
assist with certain aspects of program management. In using an LSI, the
Army also wanted to structure a development contract that would create
incentives for the contractor to succeed and profit in development and to
increase competition at lower levels in the supplier chain.
Concurrent Development and Integration of FCS Is Most Complex Acquisition Army
Has Ever Undertaken
The sheer scope and complexity of the program was driven by the Army's
desire to concurrently develop and field all systems as an integrated
unit. The backbone of this unit is a ubiquitous network through which all
systems of FCS will operate and communicate--a first-of-a-kind network
that will also have to be developed. FCS represents a huge technological
leap in system development and acquisition. Some of the major technical
challenges faced in the program include:
o The 14 major weapon systems or platforms have to be designed and
integrated simultaneously and within strict size and weight
limitations.
o At least 46 technologies that are considered critical to
achieving critical performance capabilities will need to be
matured and integrated into the system of systems.
o The development, demonstration, and production of as many as 170
complementary systems and associated programs must be synchronized
with FCS content and schedule. This will also involve developing
about 100 network interfaces so the FCS can be interoperable with
other Army and joint forces.
o An estimated 63 million lines of software code, more than 3
times the amount being developed for the Joint Strike Fighter
program.
In addition to the complexity of the integration task, the Army also
outlined an unprecedented timeline for FCS's development--about 5 1/2
years--a shorter timeline than typical for a single weapon system
development. We have pointed out a third source of risk, in addition to
complexity and schedule, namely that the program started before it was
ready for system development and demonstration because the majority of its
critical technologies were immature. Mature technologies are central to
having a sound business case at the start of development. Cognizant of
these risks, the Army believed it could achieve its goals through use of
an LSI because of the technical expertise and workforce flexibility that a
private company could bring to the program. In fact, the Army wanted the
LSI to help it define the FCS solution.
Mismatches between Scope and Pace of FCS and Capacity of Army Workforce
The Army determined it could not meet the challenges of the FCS scope and
accelerated schedule with its workforce alone and with traditional
management approaches. Army leadership saw its workforce as stovepiped
into organizations having areas of expertise that were not a full match
for what FCS needed and not large enough with the right skills to staff
several separate program offices. Army leadership did not see its
workforce as being well-suited to making the tradeoffs and integration
that the FCS program demanded.
As an integrated system of systems, defining requirements and designing
solutions for FCS would necessitate crossing two sets of organizational
lines. The first involves the Army's traditional warfare communities, such
as infantry, armor, artillery, and aviation. In the past, these
communities developed their own requirements and their own weapon systems.
FCS program officials noted that the Army had little success integrating
such separately developed platforms because the individual communities did
not coordinate their development efforts; specifically, they did not
interface across communities during development to ensure systems were
being designed to work with each other. A second organizational line
involves the users, who develop requirements, and the developers, who
manage the weapon systems. Much of the Army's previous experience with
major integration efforts was problematic because of a lack of
coordination between the developers and the users. Each system was
developed separately and integrated into the force after the fact. The
users' needs to have multiple systems integrated and working together in
the field were thus not sufficiently considered in the development
process. The Army believed that an LSI could achieve this more effectively
than an Army program office because a contractor had greater flexibility
to work across these lines.
Capability of the Army's workforce was also a factor in the decision to
use an LSI. The Army's traditional areas of technical expertise, such as
in armored vehicles like tanks, were not sufficient to address all the
needed expertise for FCS. FCS performance is controlled to a great
extent--estimated at 95 percent--by software. Current estimates put the
amount of software needed at 63 million lines--the most ever for a weapon
system--much of which will be needed for the information network that is
the heart of FCS. The Army did not have sufficient skills in software and
networks to manage this effort. Even with the LSI, the relative thinness
of the Army's expertise in these areas is evident in the integrated
product teams through which the Army and the LSI jointly manage the
program. For example, according to an LSI program official, there are 28
LSI representatives for every Army representative on the team responsible
for developing the information network. However, according to data
provided by an Army official, on the manned ground vehicles team, where
the Army has more expertise, there are only about 15 LSI representatives
for every Army representative.2 Service contractors separate from the LSI
are counted among the Army program office representatives. Thus, the
actual numbers of individuals who work for the Army directly is lower than
these numbers represent.
A third factor was workforce capacity. If the FCS platforms were to be
developed as separate programs, each platform would need its own
workforce, meaning several separate Army program offices, each with a full
complement of acquisition and technical staff. The demand for this many
people would have been a challenge in light of the decline in its
acquisition workforce throughout the 1990s. DOD estimates put the decline
of its civilian workforce at 38 percent--much of it in acquisitions--from
1989 through 2002. Hence, the Army would not have the capacity to manage a
multi-system effort like FCS with separate program offices and likely
would have had to turn to contractors to fully staff the program offices.
Other Goals in Using an LSI on FCS
In addition to the complexity and workforce implications of FCS, the Army
saw an opportunity for the LSI to give its best effort in development and
to create more competition at lower supplier levels. Army leadership
involved with setting up the FCS program believed that traditionally,
contractors made much of their profit in production, not in research and
development. Thus, the Army reasoned, the contractors are not as motivated
by research and development as they are by production. Army leadership
believed that by using an LSI that would not necessarily have to be
retained for production, the Army could get the best effort from the
contractor during the system development and demonstration phase, while at
the same time making the effort profitable for the contractor.
Army leadership also set up the FCS program and contract in such a way
that it would create more competition and have more influence over the
selection of suppliers below the LSI. Army leadership noted that
traditionally, once the Army hired a prime contractor, that contractor
would bring its own supplier chains. The Army was not very involved in the
choice of the suppliers. In FCS, the prime contractor--or LSI--is mainly
an integrator, and the Army called for the LSI to hold a competition for
the next tier of contractors. The Army had veto power over these
selections. In addition, the Army directed that the LSI contract with
integrators at lower levels in the program and the Army has been involved
with these selections. These integrators also hold competitions to select
suppliers for those systems. This strategy kept the first tier of
contractors (the one-team) from bringing their own supplier chains and
pushed competition and Army visibility down lower in the supplier chain.
It was also a means for the Army to ensure commonality of key subsystems
across FCS platforms. Thus, for example, each of the manned ground
vehicles would use the same sensors and engines, rather than the past
practice of each vehicle having its own unique set of subsystems.
2These figures exclude staff provided by "one-team" subcontractors on the
integrated product teams.
Close Working Relationship between Army and LSI Increases the Burden of
Oversight
The relationship between the Army and the LSI is complex. On the one hand,
the LSI plays the traditional role of developing a product for its
customer, the Army, and on the other hand, the LSI also performs certain
program management and integration responsibilities for the entire program
and has a partner-like relationship with the Army. In forging a close
partner-like relationship with the LSI, the Army sought to gain advantages
such as maintaining flexibility to deal with shifting priorities. At the
same time, this relationship, coupled with the vast scope of FCS and the
synonymy of the program with the future Army, poses risks for the Army's
ability to provide independent oversight over the long term. OSD is in a
position to provide this oversight, but thus far has largely accepted the
program and its changes as defined by the Army, even though it is at wide
variance from the best practices embodied in OSD's own acquisition
policies.
Boeing Is a Key Supplier for Critical Software
For the FCS program, Boeing serves as a traditional supplier, developing
two software-intensive subsystems for the Army. Specifically, the Boeing
unit that is serving as LSI is developing the System of Systems Common
Operating Environment (SOSCOE). Additionally, a separate Boeing unit is
developing the Warfighter Machine Interface (WMI). Both are critical to
the success of FCS and, as noted by one program manager, will affect the
FCS systems being developed by other contractors.
As part of the original 2003 other transaction agreement to begin system
development and demonstration, the LSI was permitted to internally develop
SOSCOE rather than contracting that work out to a separate supplier. This
make decision was approved by the Army. Referred to in the statement of
work as the "information management backbone" for FCS, SOSCOE has been
likened to a computer operating system like Microsoft Windows(R). All FCS
systems will have to interface with this software to function as a single,
integrated brigade combat team. Ultimately, the success of the FCS program
hinges on the successful development of SOSCOE.
Boeing is also developing WMI. This work was awarded as a separate
competitive subcontract by Boeing as the LSI to a separate unit within the
Boeing Company under the original 2003 other transaction agreement. The
software will provide a common interface for the soldiers in the brigade
combat unit to receive information. The goal of this software is to
provide an integrated presentation of all types of battlefield
information.
Subsequent to the award of WMI, the current FCS contract for system
development and demonstration was definitized with language that provides
a process to mitigate potential conflicts of interest on the part of
Boeing or SAIC as LSI. Under the predecessor other transaction agreement,
an organizational conflict of interest clause required that certain
safeguards be put in place when either a Boeing or SAIC unit wanted to
compete for a subcontract. Under the current contract, an organizational
conflict of interest clause is included that completely prohibits either
company from competing for any work at any tier for any proposed
subcontract under the FCS contract. So, although an award was made to
Boeing under the predecessor other transaction agreement, any further
awards are prohibited for the duration of the contract.
Close Working Relationship May Pose Risks
The government's relationship with a contractor--regardless if it is an
LSI or a more traditional prime contractor-for a major project like a
weapon system can range from a distant, arms-length relationship to a
close, partner-like relationship. An arms-length relationship is
characterized by separation between the government as the customer and the
contractor as the supplier or developer. In this arrangement,
communications between the government and the contractor are more likely
to be periodic and formal. For weapon system programs, these kinds of
relationships can often be found in situations in which the government can
establish detailed technical specifications for the weapon system,
enabling the contractor to design and develop a weapon system to meet the
specifications without much government involvement. An arms-length
relationship optimizes the independence of the government by minimizing
the interaction between its staff and that of the contractor. The downside
of this type of relationship is that information can flow slowly between
the two parties, and decision-making can be sequential and untimely. For
example, if the government were to wait 6 or more months between program
reviews, work done in the interim could go astray from the government's
wishes and have to be redone.
In a partner-like relationship, the government and the contractor work
together on a continual basis to decide what work is to be done. Over the
past 10 years, DOD has attempted to employ more partner-like arrangements
on its programs. For example, in the 1990s, DOD program offices began
employing integrated product teams, which are multidisciplinary teams that
have the cross-functional talent from both the government and the
contractor to make more informed decisions about a product's design,
production, and support.3 In addition, DOD has attempted to increase its
use of performance-based contracting, in which agencies contract for
results rather then processes and leave the determination of how best to
achieve the results to the contractor. DOD guidance on performance-based
contracting states that a positive relationship between the government and
the contractor is essential to that kind of arrangement. For example, the
guidance notes that the government and industry should work together as a
team to communicate expectations, agree on common goals, and identify and
address problems early on to achieve desirable outcomes. Such a
partner-like relationship is intended to enable more real-time, better
informed decisions, reduce rework, and provide increased flexibility to
adjust to new demands.
A partner-like relationship can also pose risks for the government.
Depending on the closeness of the working relationship, the government can
become increasingly vested in the results of shared decisions and runs the
risk of being less able to provide oversight compared with an arms-length
relationship, especially when the government is disadvantaged in terms of
workforce and skills. In the case of FCS, the partner-like relationship
between the Army and the LSI breaks new ground and as such these risks are
present. More specifically, in FCS the Army is more involved in the
selection of subcontractors than we have seen on other programs, which
can, over time, make the Army somewhat responsible for the LSI's
subcontracting network. On the other hand, the LSI is more involved with
influencing the requirements, defining the solution, and testing that
solution than we have seen on other programs. This is not to say that the
level of involvement or collaboration between the Army and the LSI is
inherently improper, but that it may have unintended consequences for
oversight over the long term.
3GAO, Best Practices: DOD Teaming Practices Not Achieving Potential
Results, [24]GAO-01-510 (Washington, D.C.: April 10, 2001).
The degree of the Army's collaboration with the LSI in the FCS program and
the possible risks this poses can be illustrated in the following areas:
Requirements. The Army initially established the operational requirements
for FCS. Based on those requirements, the Army and LSI are collaboratively
refining the FCS system of systems requirements and system-level
requirements (or system specifications). This refinement process has also
resulted in changes and clarifications to the FCS operational
requirements. The collaboration allows both parties to agree on and refine
requirements that they believe are feasible based on system-of-systems
requirements analysis conducted by the LSI and its subcontractors.
Subsequently, the Army and LSI can reach agreement on what requirements
are appropriate to achieve the FCS capability within cost and schedule
goals. For example, the Army and LSI recently collaborated on the
feasibility of the manned ground vehicle weight requirement. As a result
of this collaboration, the Army decided to trade off the original air
transport requirement that FCS manned ground vehicles weigh no more than
24 tons because they did not have enough armor to meet the survivability
requirement. The Army and LSI again collaborated with the Army ultimately
deciding that the requirement for vehicle weight be allowed to grow to as
much as 29 tons to provide the needed armor. This change was significant,
because the FCS vehicles will now have to be transported by a larger
aircraft, the C-17, rather than by the C-130 transporter. Part of the
reason for the change was that, according to program officials, an
advanced armor being developed by the Army did not prove as effective as
expected within desired weight parameters. There are several other key
technologies that are still immature, and to the extent they do not
perform as expected, requirements could continue to be changed to match
what is technically possible. This could help ensure that FCS development
can continue but may produce less value in terms of capability for the
investment.
Subcontract Selections. The Army and the LSI collaborate on subcontract
selection decisions in contracting tiers below the prime contractor level.
Subcontract selections at these levels have normally been made by the
contractors without much government involvement. Army officials
participated in the selection process for the one team subcontracts
awarded by the LSI to build and integrate major platforms.4 The Army also
plays a role in the selection of lower tier subcontractors. For example,
the Army participated in the selection of a subcontractor to build the
Active Protection System to protect vehicles from rocket propelled
grenades. This is a fourth-tier subcontractor. Although the Army is
involved with the selections, the subcontracts are awarded by the LSI or
other lower-tier contractors, so traditional government bid protest
remedies are not available to the losing contractors, as with any
procurement between private entities.5 To the extent that a subcontractor
selected with the Army's involvement underperforms, the Army may bear some
responsibility for the long term consequences of that performance.
Test and Evaluation. The LSI has a lead role in developmental testing and
verification of technical requirements throughout FCS development. For the
FCS program, testing and evaluation of system prototypes will be managed
through a combined test organization co-led by the LSI and the Army and
made of up representatives of the LSI, Army Test and Evaluation Command
and the Army's FCS program management office. In its role co-leading the
test organization, the LSI will coordinate and perform a number of
activities to ensure FCS performance is effectively and efficiently
achieved. Building and testing prototypes is funded through the LSI
contract, and the LSI will recommend how many and what type prototypes
will be fabricated. Typically, the Army test command conducts and/or
monitors system development tests and conducts operational tests of
systems to provide an objective, performance-based evaluation of system
capabilities against expectations in weapons programs. Their independent
role is an important source of information on how well a program is
progressing. In the FCS situation, the Army test command is in the
position of relying on the LSI to plan for and conduct sufficient
developmental testing--as well as proper corrective actions for identified
issues--which is an important precursor to a successful operational test
program. This has led to concerns by members of the Army test community
about their ability to conduct sufficient independent testing, while
having to work so closely with the LSI. It also raises the question of
whether the LSI is too involved with testing its own solution.
4The Defense Contract Management Agency is also involved in most FCS
subcontractor selections.
5Traditional government bid protest remedies allow contractors to protest
decisions made by federal government agencies with regard to contract
awards. Under bid protest provisions, such disputes can be submitted to
the Comptroller General of the United States for resolution within 100
days.
Involvement in Production. According to the FCS program manager, the Army
plans to contract with Boeing during fiscal year 2008 for the initial
production of FCS capabilities to be spun out to the current forces and
for the early production of the FCS non-line-of-sight cannon. The current
LSI development contract for the core FCS systems extends almost 2 years
beyond the FCS initial production decision. The Army does not expect that
the initial brigades outfitted by FCS will meet the upper range of its
requirements, and has made the LSI responsible for planning future FCS
enhancements in the production phase. The LSI is also responsible for
defining and maintaining a FCS growth strategy for integrating new
technologies into brigade combat teams. This role keeps the LSI involved
in the FCS program in the production phase and could make the LSI
indispensable to the Army.
FCS Program Can Benefit from Stronger OSD Oversight
OSD is in a position to provide the arms-length oversight that can
counterbalance some of the potential risks associated with the Army's
level of involvement with both the FCS program and the LSI. Thus far, OSD
has not played an active oversight role but rather has allowed FCS to
proceed according to the Army's plans. It has passed on opportunities to
assert its own positions on knowledge-based acquisition and cost
estimates. In response to a statutory requirement, OSD has committed
itself to a formal decision review of the program following its
preliminary design review in 2009.
In August 2004, the Institute for Defense Analyses expressed concerns that
the collaborative arrangement between the Army and LSI created an inherent
tension between the roles of Army participants as both teammates and
customer representatives. The Institute expressed the need for a corporate
perspective on the FCS program on behalf of the Army, so an independent
eye could be put toward cost, schedule and performance issues. This may be
a difficult principle for the Army to put into practice. The FCS program
is nearly synonymous with the Army's future forces and necessarily
requires the commitment and involvement of Army leadership. FCS represents
the bulk of the Army's investment portfolio.
Additionally, the nature of FCS being made up of several programs that are
large enough to have been individual acquisitions in and of themselves,
reduces the level of granularity of oversight that may have otherwise been
exercised over those programs. Major defense acquisition programs have
certain reporting requirements under law that provide information to
decision-makers about those programs. The programs within FCS are not
designated separately from FCS, so the reporting requirements for them are
not the same as if they were separately designated. Since FCS generally
meets those reporting requirements at the system-of-system level, the
granularity of reporting on individual systems within FCS is less defined.
OSD can help provide the corporate perspective on FCS through its
oversight role. To date, OSD has kept informed of the program and reviews
the program annually. However, it held only one corporate-level decision
meeting on FCS at which it approved the program to begin despite its being
at odds with DOD's own standards for such program initiation. Although OSD
has remained involved in the program, it has thus far largely accepted the
program as defined by the Army. Specifically, in May 2003, the Under
Secretary of Defense (Acquisition, Technology, and Logistics) approved the
FCS program to begin the system development and demonstration phase,
referred to as the milestone B decision. It is DOD policy for programs to
have mature technologies at that point, and for programs to be
evolutionary in nature--that is, an incremental improvement over existing
capabilities. FCS was neither, as all of the program's 49 critical
technologies6 were immature and the program was a revolutionary departure
from existing Army capabilities. Instead, the Army is following its own,
lower standard for technology maturity--achievement by the critical design
review in 2011--over 7 years later than called for by DOD policy.
Upon making that decision, the Under Secretary recognized the FCS
program's immaturity and stated that there would be a milestone B update
review 18 months later. This was to be a decision-making review for which
the Under Secretary had listed several action items that the FCS program
had to complete in order to continue. However, this review never occurred
and the FCS program continued as originally planned. OSD has not since
revisited its decision to approve the program. Since that time, program
costs and schedule have roughly doubled.7 Accordingly, last year, we
recommended that OSD hold a decision-level meeting. However, while OSD
stated that it would have a Defense Acquisition Board review, it would not
commit to making it a milestone decision review. It had not planned
another decision meeting until the FCS production decision, referred to as
milestone C. This would have been too late to have any material effect on
the course of the program, short of cancellation which is extremely rare
at that point in a program. Subsequently, Congress intervened and required
that OSD hold the formal decision meeting, currently scheduled for 2009.8
DOD has since proposed a serious approach to making that decision, which
is encouraging from an oversight perspective.
6Since 2006, a Critical Technology Working-Level Integrated product Team
recommended that the Army remove three critical technologies from its
assessment. The team concluded that these technologies did not conform to
DOD's definition of critical technologies because, in its view, the
technologies did not constitute a unique or novel application.
Recognition and reporting of cost growth is another area in which OSD has
deferred to the Army. The Army has recently restructured the FCS program
to reduce the number of systems and reduce planned production rates to
stay within expected funding levels. This will mark the second
restructuring of the program in 4 years, which has seen program investment
costs increase from $77.2 billion in constant 2003 dollars to $119.2
billion in 2005 according to Army estimates, and again to at least $150.5
billion in 2006 according to an independent cost estimate. The Army
estimates the cost of the recently restructured program to be slightly
different than its 2005 estimate. The cost increases that have occurred
since 2003 have largely been determined by the Army and OSD to be changes
in scope, a distinction that is important for cost reporting purposes. As
we have previously reported,9 DOD has allowed unit cost increases
associated with quantity reductions or increases in capabilities to be
excluded from a determination of a Nunn-McCurdy breach.10 DOD refers to
these as programmatic adjustments and has concluded that nearly all of
FCS' 76 percent cost increase--based solely on Army estimates--falls in
this category. As a result, the Secretary of Defense has not had to carry
out an assessment of the program or make a certification to Congress. Such
an assessment and certification of FCS would have had value from an
oversight perspective.
7Program cost comparison is between the original Army cost estimate--$77.2
billion--at the start of system development and demonstration in 2003 and
the May 2006 independent estimate--$150.5-162.7 billion (base year 2003
dollars).
8Pub. L. No. 109-364, S 214.
9GAO, Defense Acquisitions: Information for Congress on Performance of
Major Programs Can Be More Complete, Timely and Accessible, [25]GAO-05-182
(Washington, D.C.: March 28, 2005).
1010 U.S.C. S 2433 (requires the Secretary concerned to report to Congress
when a program's acquisition unit cost increases by at least 15 percent
over the current baseline estimate or increases by over 30 percent over
the original baseline estimate and requiring the Secretary of Defense to
carry out an assessment of the program and provide a written certification
to Congress when a program's acquisition unit cost increases by at least
25 percent over the current baseline estimate or increases by 50 percent
over the original baseline estimate).
A recent decision not to use an independent cost estimate may have had a
similar effect on cost reporting. In May 2006, the OSD Cost Analysis
Improvement Group submitted an independent cost estimate that showed its
estimate of FCS investment costs to be 24-43 percent higher than the Army
estimate prepared by the FCS program office. OSD did not adopt this
estimate. While OSD is not obligated to adopt its independent estimates,
previous experience has shown these estimates to be more accurate than the
typically optimistic service estimates and could have been become an
additional factor to consider in a Nunn-McCurdy determination.
Contract Provides Incentives for Best Effort but Not Accountability for Program
Outcomes
The Army has structured the FCS contract consistent with its desire to
incentivize development efforts and make it financially rewarding for the
LSI for making such efforts. In general, contracts are limited in that
they cannot guarantee a successful outcome. This is true for the FCS
contract, and specific aspects of the contract could make it even more
difficult to tie the LSI's performance to the actual outcomes of the
development effort. Key demonstrations of the actual capabilities of FCS
systems will take place after the LSI has been able to recoup over 80
percent of its costs and had the opportunity to earn most of its fees. The
Army shares responsibility with the LSI for making some key decisions and
to some extent the Army's performance may affect the performance of the
LSI. As with many cost-reimbursable research and development contracts,
the LSI is responsible to put forth its best effort on the development of
the FCS capability. If, given that effort, the FCS capability falls short
of needs, the LSI is not responsible and still it is entitled to have its
costs reimbursed and may earn its full fee.
FCS Contract Provides the LSI a High Level of Compensation
The current contract for completing FCS's system development and
demonstration phase provides a relatively high level of compensation in
terms of total dollars, fee, and price of labor. The definitized contract
between the Army and the LSI is a cost-reimbursable contract that is
valued at $17.5 billion, comprised of $15.2 billion in cost and up to a
15-percent fee of $2.3 billion. The remaining costs and fees from the
earlier other transaction agreement were separated from the current
FAR-based contract that was definitized in March 2006. The current
contract period, which includes both the remaining work from the other
transaction agreement and the definitized action, effectively runs from
September 2005 through the first quarter of fiscal year 2015. Under the
FCS contract, the LSI is required to put forth its best efforts to ensure
a successful system. The Army will reimburse the LSI's allowable costs and
reward the contractor with profit in the form of a fixed and an incentive
fee for its efforts. The fixed fee is paid annually and the incentive fee
is earned incrementally based on the LSI's demonstrated achievement of
established performance, cost and schedule criteria that are associated
with program events.
The total fee of 15 percent (which includes the potential incentive fees)
is based on the total value of the contract, as estimated at contract
inception. However, the benefit to the LSI is very favorable when
considering the cost of the work the LSI actually performs, versus the
amount that it subcontracts out to other firms. On FCS, the LSI will
actually perform about $8.7 billion worth of the work when combining the
costs under the previous other transaction agreement11 and subsequent
FAR-based contract. Using that as a base, the potential fee of $2.7
billion roughly amounts to a 30 percent profit on the work the LSI
actually does itself. According to an analysis conducted within the Office
of the Secretary of Defense, this is a relatively high ratio of profit to
value of work performed when compared with other large development
programs.
As with most cost reimbursable contracts, the reimbursable costs of the
prime contractor include its costs and the costs and fees of lower tier
subcontractors. The prime's fee is separate from its reimbursable costs.
For example, if a company is awarded a prime development contract for $300
million, that figure includes both costs and fees of the contractor's
subcontracts. The prime contractor's fees are calculated on the $300
million cost figure included in the contract, but are not allowed to go up
if the contract costs increase. Accordingly, if the prime contractor then
awards a subcontract for $100 million of costs and pays the subcontractor
a fee of $15 million, the full $115 million paid to the subcontractor is
part of the $300 million of the prime contractor's estimated reimbursable
costs. The prime contractor is entitled to be reimbursed for the full $300
million in costs from the government plus be paid any fee it has earned.
The result of the FCS LSI arrangement is an additional layer of
subcontractors and associated costs. Thus, the costs and fees of all the
prime item developers and their subcontractors are included in the $15.2
billion in costs reimbursable to the LSI under its contract with the Army.
The LSI's potential $2.3 billion fee is calculated based on these costs,
as with a typical prime contract.
11The other transaction agreement had a total value of $3.3 billion, with
costs of $2.9 billion and fee of $0.4 billion. For the entire FCS system
development and demonstration effort, the total contractual value is $20.8
billion, made up of $18.1 billion in cost and $2.7 billion in fee.
Based on data provided by FCS program officials, the cost of LSI personnel
is high relative to their government counterparts. The Army is paying the
average LSI full-time equivalent about 25 percent more than the average
cost of a federal employee in the senior executive service. These costs
assume salary, benefits, and other costs of maintaining an employee on the
program. We have recently reported that contractor personnel also cost the
Missile Defense Agency about 25 percent more than their government
counterparts.12 However, the comparison data for missile defense personnel
is based on all program personnel, not just the members of the more highly
compensated senior executive service.
Majority of Program Fees and Costs Are Available to the LSI before FCS Systems
Demonstrate Their Performance
Under the terms of the FCS contract, the LSI can earn over 80 percent of
its $2.3 billion fee by the time the program's critical design review is
completed in 2011, and roughly 80 percent of contract costs will have been
paid out by the Army by that point. Yet the actual demonstration of
individual FCS prototypes and the system-of-systems will take place after
the design review. Our work on past weapon system programs shows that most
cost growth--symptomatic of problems--occurs after the critical design
review.
The fee the LSI can earn under the FCS contract is divided between a fixed
fee of $1.13 billion that will be paid in annual installments and an
incentive fee of $1.14 billion that, according to a program official, can
be earned on an incremental basis as the LSI accomplishes certain
performance, cost and schedule criteria associated with each of nine key
program events. Thus, it can earn portions of its incentive fees prior to
occurrence of the event. Typically, incentive fees for weapon acquisition
programs are based largely on how well the contractor achieves cost
targets, but the LSI is eligible to receive a minimum of 50 percent of its
available incentive fee based on performance criteria, not cost.
Additionally, the contract provides for rolling over any unearned
incentive fees to subsequent events. This means that if work under a fee
event is delayed, the Army can decide to delay the associated fee as well
and pay it when the work does get done. To the extent that the contractor
is responsible for the delay, rollover can allow the contractor to get a
second chance to recoup performance fee that it did not perform well
enough to earn according to criteria at the original event, but it will
not recoup the portion of the fee associated with schedule performance. A
high-level program official did tell us that the Army plans to allow
roll-over at only one program event, if the LSI does not earn its full fee
at that event. Previous GAO work on fees highlighted the use of rollover
as an indication that the fee structure for the program lacks the
appropriate incentives, transparency, and accountability for an effective
pay for performance system.13 The nine fee events used to evaluate the
performance of the LSI, along with the fixed and incentive fees that can
be earned are listed in the table below.
12GAO, Defense Acquisitions: Missile Defense Acquisition Strategy
Generates Results but Delivers Less at a Higher Cost, [26]GAO-07-387
(Washington, D.C.: March 15, 2007).
13GAO, Defense Acquisitions: DOD Wastes Billions of Dollars through Poorly
Structured Incentives, [27]GAO-06-409T (Washington, D.C.: December 19,
2005).
Table 1: Fee events and schedule for FCS contract
Dollars in millions
Fiscal Year Event Incentive Fee Fixed Fee Total Fee
2005 No Event 3.4 3.4
2006 In-Process Preliminary 101.1 191.7 292.8
Design Review (Capability
Maturity 0)
2007 No Event 178.0 178.0
2008 Engineering Maturity 1 125.2 168.0 293.2
2009 Preliminary Design Review 255.3 160.6 415.9
(Capability Maturity 1)
2010 Engineering Maturity 2 145.3 148.4 293.7
2011 Capability Maturity 2 317.8 129.1 446.9
(Critical Design Review)
2012 Engineering Maturity 3 59.8 76.4 136.2
2013 Capability Maturity 3 96.8 51.7 148.5
2014 Engineering Maturity 4 22.6 22.0 44.6
2015 Verification Complete 19.9 4.6 24.5
Totals 1,143.6 1,133.9 2,277.7
Source: U.S. Army data and GAO analysis
Note: Although the incentive fees are earned incrementally, the above
incentive fee values are presented with each discreet event they are
associated with for clarity of presentation. The above values reflect the
events which occurred under the current contract, which began in September
2005. Thus, the incentive event in FY 2005, which occurred prior to that
month, is not included above as it was paid under the other transaction
agreement. Also, dollar amounts may not add up to totals because of
rounding.
To date, the LSI has completed one incentive event under the FAR contract
and received 100 percent of the available incentive fee for its efforts.
By the time the Army completes the critical design review in 2011, the LSI
could earn over 80 percent of its incentive fee and over 80 percent of its
total fee. The critical design review is important because our work has
shown that by this point in time, a weapon system's design should be
stable enough to release 90 percent of engineering drawings for
manufacturing. This level of knowledge is demonstrative that the design is
stable and capable of meeting performance requirements. It is the point at
which managers of a program can determine whether or not to build
production-representative prototypes to demonstrate the actual performance
of the design.
We have found that most cost growth on weapon system development programs
occurs after the critical design review. As shown in figure 2, historical
information on 26 major programs that have completed development
experienced about 28 percent cost growth, with almost 20 percent after
critical design review.14
Figure 2: RDT&E Percentage Increase throughout the Product Development
Cycle for 29 Programs Completed or in Production
This pattern of cost growth occurs because most programs hold critical
design review before the design is stable. Subsequent building and testing
of prototypes has led to the discovery of problems that are costly to fix
in the late stages of development. We have already reported that the
critical design review for the FCS program will occur before the program
has attained a sufficient level of knowledge to ensure that technologies
are mature. Moreover, the Army does not plan to build
production-representative prototypes for testing, relying instead on less
mature prototypes and simulations. This sequence of events sets the stage
for much discovery about the FCS's actual performance and potential
problems after the design review and after most of the fee can be paid to
the LSI.
14For a description of the methodology behind this analysis, please see
GAO, Defense Acquisitions: Assessments of Selected Weapon Programs,
[28]GAO-06-391 (Washington, D.C.: March, 31, 2006).
Connecting Contract Performance with Program Performance Will be Difficult
For several reasons, it will be difficult to connect the LSI's performance
on the contract with the success of the program. The contract itself, like
those for other weapon system developments, does not insure the Army
against an unsuccessful outcome. While the Army can gauge the progress
under the contract, the LSI is responsible for providing best efforts, not
successful outcomes. The criteria for fee events are not directly related
to achievement of total program outcomes, and the partner-like involvement
with the LSI creates a situation in which the Army's performance can
affect the LSI's performance.
The FCS contract is a cost-reimbursement research and development
contract. In this respect, it is no different than most contracts to
develop weapon systems. Essentially, under a research and development
contract, the contractor, or LSI in the case of FCS, is required to
provide its best efforts at developing a capability or weapon system to
the Army but is not responsible for actually producing the capability.
Best efforts are measured by the inputs the contractor puts toward
development of the system. Specifically, it must put the resources and
processes in place to demonstrate its best efforts at developing the
Army's desired capability. If the weapon systems, individually or
collectively, fail to provide that capability, the LSI is not responsible
as long as it has put forth best effort.
The contract fee events reflect the best effort nature of the LSI's
performance and do not require the successful demonstration of specific
program knowledge or outcomes. For example, the criteria for the most
recent incentive fee event (which was valued at a total of about $100
million) included such items as an updated force effectiveness analysis,
the update and approval of program technical performance measures, and the
completion of certain requirements and planning products. However, the
incentive fee event criteria do not specify what is expected in terms of
the effectiveness analysis results, the current status of the technical
performance measures, or when and how the requirements process should be
completed. Army program officials point out that this fee structure is
meant to create incentives for the LSI to focus on putting processes in
place to ensure successful development of the system. They also note that
in some past programs, contracts had devoted inadequate resources to such
activities. As noted in previous GAO work15 and in NASA contracting
guidance for major system acquisitions,16 input factors such as those used
as criteria for the fee events in FCS are valuable, but they do not
provide indications of success relative to the desired end result of the
program.
15GAO, NASA Procurement: Use of Award Fees for Achieving Program Outcomes
Should Be Improved, [29]GAO-07-58 (Washington, D.C.: January 17, 2007).
Because of its close involvement with the LSI, the Army has to make
judgments about what contract outcomes and changes it is responsible for
versus the LSI. The Army has already made judgments like these. When the
FCS program was restructured in 2004, the cost estimate and the program
schedule increased significantly as the Army changed the scope of the
program by increasing requirements and adding deferred systems to the
contract. The Army attributed the changes in cost and schedule to the
changes in scope and took responsibility for them, absolving the LSI of
responsibility. Evaluated against the revised cost and schedule estimates,
the Army awarded the LSI the full incentive fee at the next program
evaluation event. Such adjustments in the LSI's contractual
responsibilities are possible in the future as well because the criteria
for each fee event are not set until the year the event occurs and payment
of fee associated with each event is done incrementally based on
accomplishment of specific criteria for each event. This could allow the
Army to adjust the fee criteria based on the status of the program at the
time. Thus, if the LSI and Army determine that a certain segment of work
due to be completed by the time of an event cannot be completed, the
criteria for assessing that segment can be shifted out of that event. This
occurred in the most recent program event where the scope of work
associated with approximately $105,000 in fee was shifted to the
subsequent fee event. The Army and LSI decided this was necessary because
accomplishment of the criteria associated with that fee was better suited
for the next fiscal year.
The Army's own performance may be a factor in these decisions. For
example, the Army is responsible for maturing some of the key technologies
the LSI will need to integrate into the FCS systems. If these technologies
do not succeed, then the expectations of the LSI may have to be adjusted
accordingly. The decision to increase the weight requirement for the
manned ground vehicles is illustrative. Part of the reason for the
decision was the fact that an advanced, lightweight armor the Army was
developing outside the FCS contract was not performing as expected. While
the decision affected the vehicle design, the LSI was not responsible for
development of the armor technology.
16NASA, Award Fee Contracting Guide (Washington, D.C.: June 27, 2001).
Conclusions
Evaluating the use of the LSI on FCS involves consideration of several
intertwined factors. Some, like the best efforts provisions of a
cost-reimbursable research and development contract, are not unique to the
LSI or to FCS. Other factors differ not so much in nature, but in degree
from other programs. For example, FCS is not the first system-of-systems
program DOD has proposed, but it is arguably the most complex. FCS is not
the first program to proceed with immature technologies, but it has more
immature technologies than any other program. FCS is not the first program
to use an LSI, but the extent of the partner-like relationship between the
Army and the LSI breaks new ground. Collectively, they make the LSI
arrangement in the FCS context unique.
We have reported the great costs and risks DOD has accepted by committing
to FCS investments. We have expressed concern that the FCS program moved
forward with insufficient knowledge and, therefore, an insufficient
business case. However, that aside, if one accepts the FCS program for
what it is and where it is in the development cycle, the Army has set up a
contractual relationship that is both consistent with its vision for FCS
and candid with respect to its workforce limitations. The Army has been
thoughtful about what it is trying to accomplish collaboratively with the
LSI, and has been working hard to make progress, including facing up to
difficult tradeoffs. On the other hand, the limits of the contractual
arrangements must also be recognized. Given the unprecedented challenge
FCS represents, it is unrealistic to expect that any contracting approach
alone could assure a successful outcome. Ultimately, the risks of
successful outcomes will be borne by the government. The contractual
arrangements are not a substitute for having the high level of knowledge
that a sound business case requires.
The Army has shown a high tolerance for accepting risk and responsibility
on this program. In addition to accepting high technical risk, the Army
has accepted responsibility for lowering the performance of some
individual systems, deleting some and adding other systems, reducing
quantities, and increasing costs and schedules. The Army has determined
the bulk of cost and schedule changes since 2003 to be programmatic or
scope-related. This determination has had two effects. First, the changes
became the responsibility of the Army, entitling the LSI to earn full fee
thus far. Second, the changes are excluded from a determination of a
Nunn-McCurdy breach and its reporting and certification requirements. Over
time, the Army runs the risk of becoming increasingly vested as it makes
these and other decisions and less able to change course. Yet, the
government must safeguard its ability to change course in the future as
demonstrated knowledge replaces projections.
The foregoing underscores the important role of OSD in providing oversight
on the FCS program and holding the program accountable to its own
policies. While the Army works to manage the program, it is important that
OSD hold the program accountable to best practice standards embedded in
its policies. The go/no-go decision it will hold in 2009 provides an
opportunity for OSD to do so. The use of an LSI on FCS also needs to be
seen as more significant than a contracting arrangement for a single
program. At the very least, a proposal to use an LSI approach on any new
program should be seen as a risk at the outset, not because it is
conceptually flawed, but because it indicates the government may be
pursuing a solution that it does not have the capacity to manage. Such
solutions ought not to be accepted as inevitable or unavoidable. Instead,
they require additional scrutiny before they are approved and increased
oversight if they are approved.
Recommendations
We recommend that the Secretary of Defense:
o reassess OSD's approach to overseeing the FCS program, including
asserting its own policy-based markers for progress, particularly
in the areas of cost, technology maturity, design maturity, and
production maturity.
o ensure that there is the best link possible between the fee
events in the FCS contract and actual FCS demonstrations;
o review major FCS program changes to ensure that determinations
for the government to accept changes as being programmatic or
scope-related in nature are carefully scrutinized; and
o assess whether the experience of the LSI on FCS has broader
implications for acquisition management, such as the ability of
the DOD workforce to manage a system-of-systems acquisition.
Agency Comments and Our Evaluation
DOD concurred with our recommendations. DOD stated that it was updating
its acquisition policy to address markers for progress in a number of
areas including cost, technology maturity, design maturity, and production
maturity. DOD agreed to use a variety of technical assessments to inform
the Defense Acquisition Board on the FCS program's progress against its
policy-based markers. It is important that the Department be as specific
as possible and consistent with its own acquisition policy in setting
expectations that the FCS program must meet. The Department also agreed to
review the FCS award fee plan and to continue scrutinizing FCS program
changes and accurately report against the program baseline. DOD noted that
the FCS program scope has been expanded to add capability and to meet
affordability constraints. In our view, some of the changes in scope were
also made to correct shortcomings in the original acquisition strategy. It
is important for DOD to be able to make such distinctions for reporting
purposes.
In concurring that the Secretary of Defense assess whether the experience
of the LSI on FCS has broader implications for acquisition management, DOD
stated that its acquisition policy is being updated to better manage and
control system and system-of-systems acquisitions. In addition to
exploring how to improve the management of systems-of-systems and LSIs, it
is important for DOD to look at the more strategic questions such as
whether and under what circumstances these approaches should be taken. For
example, are systems-of-systems too large a scope to manage and report as
a single acquisition program? Is using an LSI preferable to getting a
better match between the acquisition programs being conceived and the
acquisition workforce DOD has to manage them? Should DOD be looking at
reducing the scope of programs, increasing the capability of its own
workforce, or both to achieve this match?
DOD also stated that it considers the business relationship for the FCS
development contract to be typical of a prime contract for a major system
because the FCS contractor performs a substantial portion of the
development work for the program. As there is no universally accepted
definition of a LSI, this distinction may be more a matter of opinion than
fact. In our opinion, the role played by the FCS LSI is not typical of a
DOD contractor. Two characteristics, in our view, distinguish a LSI from a
traditional contractor. First, the integrator is managing across what
would traditionally have been program lines, versus subsystems within a
program. Second, in so managing, the integrator is acting on behalf of,
and in the interests of, the government. The Army was specific about
needing a different, partner-like contracting arrangement like this when
it began the FCS program. We also note that while the FCS LSI is
performing substantive work on software systems, its portion of total work
is low relative to major prime contractors elsewhere in DOD and it is not
directly involved in the development of any hardware for the FCS
system-of-systems.
Finally, the Department noted that the role of the FCS prime contractor in
requirements determination is not correctly framed in our draft report and
that we confuse operational requirements with design specifications. We
have characterized the LSI's role in this report as requirements
refinement, rather than requirements definition. The requirements work
being led by the FCS LSI is intended to complete the definition of the
system-of-systems requirements and the system-level requirements. Two
aspects of this role are, in our view, distinctive. First is the fact that
because FCS is a system-of-systems, the functions performed are one level
higher than they would have been for a typical single-system program.
Thus, while the Army determines the operational requirements for the FCS
brigade combat team the LSI is heavily involved with its subcontractors
and the Army in setting the requirements for individual systems. On single
system programs, the Army would have set the requirements for the
individual system. Second, the FCS solution is being formed concurrent
with the development of individual technologies and the design of systems.
Thus, as the limitations of technology and design are discovered, the LSI
works with the Army to change or refine the requirements to conform to
these limitations. While this process is not atypical of weapon system
acquisitions, the vast scope and large technical leaps sought in the FCS
program requires greater involvement by the LSI in the refinement process.
DOD's comments are reprinted in Appendix III. DOD also provided technical
comments, which were addressed throughout the report as appropriate.
We are sending copies of this report to the Secretary of Defense; the
Secretary of the Army; and the Director, Office of Management and Budget.
Copies will also be made available to others on request. In addition, the
report will be available at no charge on the GAO Web site at
http://www.gao.gov .
Please contact me on (202) 512-4841 if you or your staff has any questions
concerning this report. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this report.
Other contributors to this report were Assistant Director William R.
Graveline, Noah B. Bleicher, Lily J. Chin, Brendan S. Culley, Michael D.
O'Neill, Kenneth E. Patton, and Thomas P. Twambly.
Paul L. Francis
Director Acquisition and Sourcing Management
List of Committees
The Honorable Carl Levin
Chairman
The Honorable John McCain
Ranking Member
Committee on Armed Services
United States Senate
The Honorable Daniel K. Inouye
Chairman
The Honorable Ted Stevens
Ranking Member
Subcommittee on Defense
Committee on Appropriations
United States Senate
The Honorable Ike Skelton
Chairman
The Honorable Duncan L. Hunter
Ranking Member
Committee on Armed Services
House of Representatives
The Honorable John P. Murtha, Jr.
Chairman The Honorable C. W. (Bill) Young
Ranking Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives
Appendix I: Scope and Methodology
To identify factors that led to the Army's decision to use an LSI for the
FCS program and to determine the work performed by the LSI, we performed
the following:
o We obtained and analyzed the program documents including the FCS
system development and demonstration contract, statement of work,
Army FCS acquisition strategy report, and FCS operational
requirements document to gain an understanding of the terms and
conditions of LSI responsibilities, the structure and processes of
the program, and goals of the Army. We reviewed FCS subcontracts
to understand the nature of the FCS one team and to ascertain how
Federal Acquisition Regulation clauses were flowed down after FCS
contract conversion. We reviewed Army audits, Defense Contract
Management Agency and Defense Contract Audit Agency reports, as
well as GAO reports and testimonies.
o We reviewed Army and DOD guidance as well as the Federal
Acquisition Regulation to understand government contracting
standards and procedures used for the acquisition of major weapon
systems.
o We interviewed FCS program officials from the Tank-Automotive
and Armaments Command, DOD's Office of Acquisitions, Technology,
and Logistics, Defense Contract Audit Agency, and Defense Contract
Management Agency to gain insight into why the Army chose and LSI
business arrangement, how it is performing, and potential concerns
for the future.
o We interviewed one team partner officials from 12 of the major
platform development offices to receive feedback from those
implementing the program decisions made by the LSI. These
discussions focused on differences, benefits, and drawbacks of the
LSI business approach when compared to more traditional, prime
contractor arrangements. As many of these firms have extensive
experience in defense contracting, we spoke about alternatives
that the Army could have used for the FCS procurement. Finally,
these discussions allow us to gain insight into the implementation
and impact of the other transaction agreement to FAR contract
conversion.
To evaluate the implications of the Army's relationship with the LSI, we
performed the following:
o We reviewed and collected information from the acquisition
strategy report, statement of work, the FAR-based contract,
documents related to source selection decisions, the integrated
master schedule and documents related to the in-process
preliminary design review, and the operational requirements
document to identify the roles and responsibilities of the Army
and LSI.
o We interviewed key Army and LSI program managers, who were
responsible for the overall FCS program, the Army and LSI leaders
of the major integrated product development teams and met with
selected officials from the first tier of major subcontractors to
assess communication and decision making within the program.
To evaluate the Army's criteria for assessing the LSI's performance, we
conducted the following:
o We reviewed the financial terms of the contract, the criteria
for assessing the LSI's performance at program incentive events
contained in the contract and integrated master plan and conducted
quantitative analyses of the contract's fixed and incentive fees;
o We reviewed the LSI's presentations for the Army's assessment
and also interviewed Army officials, who were responsible for
reviewing the LSI's performance. To evaluate the program's
financial reporting systems, we interviewed officials from the
Defense Contract Management Agency and Defense Contract Audit
Agency.
To accomplish our work, we visited and interviewed officials from the Army
Tank and Automotive Command, Warren, Mich.; Army integrated product team
leaders in Huntsville, Ala., Hazelwood, Mo., Fort Picatinny, N.J.; LSI
officials, in Hazelwood, Mo. and Huntington Beach, Calif. In addition, we
interviewed 12 one team partners across the United States. We also
interviewed officials from the Defense Contract Management Agency, Defense
Contract Audit Agency and the Office of the Secretary of Defense's Cost
Analysis Improvement Group.
We conducted our review between May 2006 and June 2007 in accordance with
generally accepted government accounting standards.
Appendix II: Overview of FAR Provisions Included in Future Combat Systems
Lead Systems Integrator Contract
The Federal Acquisition Regulation (FAR) provides uniform policies and
procedures for acquisitions by federal government executive agencies.
Depending on the type of contract entered into, different FAR clauses and
provisions are used to protect the government's interests and define the
terms of the agreement. Likewise, contracting officers structure a
contract appropriately depending on the products or services being
procured. A multitude of FAR provisions and agency FAR supplement
provisions give contracting officers a wide range of options to tailor
government contracts to meet the specific agency needs. While many FAR
clauses are required to be incorporated in all contracts of a particular
type, other provisions are only required to be included as applicable.
The Army's original FCS Other Transaction Agreement was converted into a
FAR-based cost-reimbursable research and development contract in 2006.
According to the Army, the new FCS contract includes the FAR and Defense
Federal Acquisition Regulation Supplement (DFARS) requirements appropriate
for this type of procurement. While GAO confirmed the Army's analysis of
the FAR-based contractual provisions, it did not conduct an independent
detailed examination of every applicable clause in the contract. However,
GAO did confirm the inclusion of several FAR requirements that address
areas of key concern.
o Cost Accounting Standards (CAS) - Two FAR part 12 provisions
pertaining to the use and administration of CAS have been included
in the FCS contract.
o Procurement Integrity Act (PIA) - The FCS contract includes the
two FAR clauses required to address PIA concerns.
o Truth in Negotiations Act (TINA) - TINA standards for cost and
pricing data are addressed in three FAR part 12 provisions.
Additional information regarding exceptions and requirements for
cost and pricing data are included separately in the FCS contract.
o Organizational Conflict of Interest (OCI) - Although the
predecessor other transaction agreement contained an OCI clause
that required certain safeguards be put into place if and when
Boeing and SAIC competed for subcontracts, it did not preclude
them from such competitions. The FCS FAR contract includes an OCI
provision that precludes the Boeing/SAIC LSI team from competing
for any FCS subcontract awards. Though FCS subcontractors may
compete for additional FCS subcontracts, the OCI provision in the
FCS contract requires that steps be taken to ensure an absence of
any organizational conflicts of interest during subcontractor
selection activities. Additionally, this clause provides
instruction on how proprietary information should be protected.
Appendix III: Comments from the Department of Defense
(120555)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
transparent illustrator graphic
www.gao.gov/cgi-bin/getrpt?GAO-07-380 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Paul L. Francis at (202) 512-4841 or
[email protected].
Highlights of [39]GAO-07-380 , a report to congressional committees
June2007
DEFENSE ACQUISITIONS
Role of Lead Systems Integrator on Future Combat Systems Program Poses
Oversight Challenges
The Army's Future Combat Systems (FCS) program features multiple new
systems linked by a first-of-a-kind information network. The Army
contracted with a lead systems integrator (LSI) for FCS that could serve
in a more expansive role than a typical prime contractor would. In
response to a congressional mandate, this report addresses (1) why the
Army decided to employ an LSI for the FCS program; (2) the nature of the
LSI's working relationship with the Army; and (3) how FCS contract fees,
provisions, and incentives work.
In conducting its work, GAO reviewed extensive program documentation and
held discussions with key officials at DOD and throughout the FCS program.
[40]What GAO Recommends
GAO recommends that the Secretary of Defense take steps to strengthen
oversight of the FCS program and assess whether the experience of the LSI
on FCS has broader implications for DOD acquisition management. DOD
concurred with GAO's recommendations.
In 2003, the Army contracted with an LSI for FCS because of the program's
ambitious goals and the Army's belief that it did not have the capacity to
manage the program. The original timeframe for FCS's development was a
shorter time frame than for an individual weapon system program, let alone
a complex systems-of-systems program with a high number of immature
technologies at program start. The Army realized that its
compartmentalized workforce did not lend itself to the kind of
crosscutting work that the FCS program would demand. The Army workforce
also did not have the expertise needed to develop the FCS information
network or enough people to support the program had it been organized into
separate program offices. In contracting with the Boeing Company as LSI,
the Army believed it found a management partner who could define and
develop FCS and reach across the Army's organizations. Boeing
subcontracted with another company, Science Applications International
Corporation, to assist with its responsibilities as LSI.
The working relationship between the LSI and the Army is complex. The LSI
is a traditional contractor in terms of developing a product for its
customer, the Army, but also serves like a partner to the Army in
management of the FCS program. In its management role, the LSI makes
decisions collaboratively with the Army. An advantage of this arrangement
is that the LSI and Army can maintain flexibility when dealing with
shifting priorities. However, that relationship may pose significant risks
to the Army's ability to provide oversight over the long term. The Office
of the Secretary of Defense is in a position to provide this oversight but
thus far has allowed the Army to depart significantly from best practices
and the Office's own policy for weapon system acquisitions. For example,
the Office of the Secretary of Defense has also allowed the Army to use
its own cost estimates rather than independent--and significantly
higher--cost estimates when submitting budget requests. The Army's
experience with the LSI on the FCS program may provide the Office of the
Secretary of Defense insights on broader acquisition management issues.
The Army has structured the FCS contract consistent with its desire to
incentivize development efforts. The definitized cost-reimbursable
research and development contract valued at $17.5 billion contains up to a
15 percent total fixed/incentive fee, or about $2.3 billion. As with many
research and development contracts, the FCS contract obligates the
contractor to put forth its best efforts, but does not assure successful
outcomes. Assuming that critical design review is completed in 2011, the
Army will have paid the LSI over 80 percent to cover the contract costs,
plus a possible 80 percent of its fee or profit. GAO has previously
reported that most cost growth in DOD weapon system programs occurs after
critical design review. Therefore, it is possible for the LSI to have
garnered most of its payouts in costs and fees early next decade, even if
despite its best efforts, the FCS capability ends up falling far short of
the Army's goals. The Army notes that its fee structure is intended to
encourage good performance early in the program.
References
Visible links
24. http://www.gao.gov/cgi-bin/getrpt?GAO-01-510
25. http://www.gao.gov/cgi-bin/getrpt?GAO-05-182
26. http://www.gao.gov/cgi-bin/getrpt?GAO-07-387
27. http://www.gao.gov/cgi-bin/getrpt?GAO-06-409T
28. http://www.gao.gov/cgi-bin/getrpt?GAO-06-391
29. http://www.gao.gov/cgi-bin/getrpt?GAO-07-58
39. http://www.gao.gov/cgi-bin/getrpt?GAO-07-380
*** End of document. ***