Defense Contracting: Questions for the Record (21-DEC-06,
GAO-07-217R).
On September 7, 2006, GAO testified before the Subcommittee on
Defense, Committee on Appropriations, on recent trends in
Department of Defense (DOD) contracting. Specifically, GAO
testified about practices that undermine DOD's ability to
establish sound business arrangements, particularly those
involving the selection and oversight of DOD's contractors and
their performance. This correspondence responds to a number of
post-hearing questions relating to various issues, including
measures that DOD can employ to ensure better contracting
outcomes.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-217R
ACCNO: A64404
TITLE: Defense Contracting: Questions for the Record
DATE: 12/21/2006
SUBJECT: Contract oversight
Defense procurement
Department of Defense contractors
Performance management
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-217R
December 21, 2006
The Honorable C. W. Bill Young Chairman The Honorable John P. Murtha
Ranking Minority Member Subcommittee on Defense Committee on
Appropriations House of Representatives
Subject: Defense Contracting--Questions for the Record
On September 7, 2006, I testified before the Subcommittee on recent trends
in Department of Defense (DOD) contracting. Specifically, I testified
about practices that undermine DOD's ability to establish sound business
arrangements, particularly those involving the selection and oversight of
DOD's contractors and their performance. The need for prudence with
taxpayer funds and long-range fiscal challenges demand that DOD ensure
that its funds are spent wisely, and that it is buying the right things,
the right way.
The Subcommittee requested that I respond to a number of post-hearing
questions relating to various issues, including measures that DOD can
employ to ensure better contracting outcomes. The specific questions and
my responses are attached in appendix I. The responses are based on
previously issued GAO products on DOD acquisitions, all of which were
conducted in accordance with generally accepted government auditing
standards. Because the responses are based on prior work, we did not
obtain comments from DOD.
We will make copies of this letter available to others upon request, and
it will be available at no charge on the GAO Web site at
http://www.gao.gov . If you have any questions about this letter or
need additional information please contact Katherine V. Schinasi at (202)
512-4841 or [email protected] . Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last page
of this letter. Key contributors to this letter were Carol Dawn Petersen,
Chris Kunitz, and Bill Woods.
Sincerely,
David M. Walker
Comptroller General
of the United States
Attachment
Appendix I: GAO Responses to Questions for the Record
Award and Incentive Fees
1. Could you address how these award and incentive fees work, in practical
terms?
When we talk about fees, we are really talking about a government expense
and a form of profit for contractors. Depending on how a contract is
structured, award and incentive fees can account for all or a portion of a
contractor's potential fee or profit. Unlike other contracts, award- and
incentive-fee contracts allow an agency to adjust the amount of fee paid
to contractors based on the contractors' performance. In practical terms,
an award-fee contract sets up a system where the government periodically
assesses the contractor's performance in areas such as quality,
timeliness, technical ingenuity, and cost-effective management. These
criteria are generally subjective and can include inputs as well as
outcomes. Incentive fees, however, are more objective and outcome-based.
The contract identifies specific cost or delivery targets, and/or
performance goals, and the amount of fee earned by the contractor is to be
directly related to achievement of these goals.
2. Do you believe they are an effective way to create incentives with
contractors? In other words, do they improve contractor performance?
Award and incentive fees can be an effective tool if properly designed and
effectively administered. However, for these fees to be effective, DOD
must address the underlying problems in its acquisition system and more
directly link the fees to the outcomes it wants. Because the weapon system
programs that result from this system are in many cases unexecutable
and/or subject to changing "requirements" or funding levels, DOD has been
unwilling to hold its programs or its contractors accountable for
achieving the very acquisition outcomes it has identified. As a result,
fees are paid even when outcomes do not meet expectations. Addressing
these broader acquisition issues and strengthening the link between fees
and acquisition outcomes can increase the accountability of DOD programs
for fees paid, of contractors for results achieved, and the likelihood
that these fees will motivate the contractors and be an effective tool for
the government.
3. Do you believe we should discontinue the use of award and incentive
fees?
No, we do not believe they should be discontinued. Award and incentive
fees can be useful if they are used in the appropriate setting. Each
contract type has a use based on the level of risk involved. The problem
occurs when you proceed into programs without realistic requirements and
sufficiently mature technologies on which to base realistic cost and
schedule estimates and attempt to offset that increased risk by offering
award fees to motivate the contractor to overcome that risk. The important
question is: Have you adequately defined and established appropriate
criteria that enable you to measure outcomes? And finally, how do you
apply those criteria in determining the level of fee that can be
justified? We have made several recommendations to this effect, and DOD
has responded favorably with new guidance to link award fees to
acquisition outcomes. However, as with other recommendations we have made
related to DOD weapon system acquisitions, the key will be how this new
policy is implemented.
Acquisition Policy
1. Given that DOD's acquisition policy seems to embrace practices endorsed
by the Defense Acquisition Performance Assessment, the Defense Science
Board Summer Study 2005, and GAO, what is preventing the services from
consistently executing these policies?
The services are not consistently executing these policies because DOD is
not enforcing a knowledge-based approach, discipline is lacking, and
business cases do not measure up. This is occurring, in part, because
there are no consequences for actions that run counter to the intent of
DOD acquisition polices--officials responsible for approving program
starts are no longer in their positions by the time the consequences of
their actions become evident. The department routinely accepts high levels
of technology risk at the start of major acquisition programs. Mature
technologies are pivotal to developing new products. Without mature
technologies at the outset, a program will almost certainly incur cost and
schedule problems. However, DOD's acquisition community moves forward on
programs with technologies before they are mature and takes on
responsibility for technology development and product development
concurrently. Our work has also shown that DOD allows programs to begin
without establishing a sound business case that matches requirements with
technology, acquisition strategy, time, and funding. And once these
programs begin, their requirements and funding change over time. In fact,
program managers consider shifting requirements--which can result in added
program complexity and costs--and funding instabilities--which occur
throughout the program--to be their biggest obstacles to success.
Fundamentally, DOD will need to reexamine the entirety of its acquisition
process and how it is affected by requirements and funding processes. This
includes making significant changes to program requirements setting,
funding, and execution.
2. What else is needed to ensure that cost, schedule, and performance
outcomes for new weapons are predictable and achievable when these
programs seek approval from Congress?
DOD needs to take additional steps to achieve outcomes on par with best
practices. These include
o developing and implementing an acquisition investment strategy;
o ensuring that individual programs are executable; and
o clearly delineating responsibilities and holding government
employees and contractors accountable for achieving desired
results.
While DOD has incorporated into policy a framework that supports a
knowledge-based acquisition process similar to that used by leading
organizations, it must establish stronger controls to ensure that
decisions on individual programs are informed by demonstrated knowledge.
Moreover, Congressional approval of programs that have not taken these
steps encourages DOD's subsequent requests for additional funding.
3. What are the services doing now to ensure that the technologies in
these programs are achieving the desired level of maturity before they
proceed into system development?
DOD is now required by law to certify to the Congress that technology is
demonstrated to a specific maturity level before being approved for system
development. However, this is a recent statutory requirement and it is too
early to tell whether this requirement will ensure achievement of the
desired technology maturity level before programs are authorized to
proceed into system development. Prior to this new law, some of the
criteria upon which this certification is based were reflected in DOD
policy. However, our prior work showed that DOD written polices are not
always observed in practice.
4. GAO believes a sound business case is needed before a weapon system
begins system development and demonstration yet finds many of the existing
weapons' business cases to be broken. What do you consider key elements of
a business case for starting a development program and how do you ensure
these are followed in each planned development program?
The following are the essential elements of a sound business case:
o A requirement exists that warrants a solution consistent with
national security priorities.
o The developer has the mature technologies and technical
knowledge necessary to meet the requirement.
o The developer has a knowledge-based product development plan
that will attain high levels of design and production maturity at
the right times.
o Reasonable estimates have been developed to execute the product
development and production plan.
o Current and future funding will be available to fully resource
the product development and production plan.
o The agency has the capacity, either in-house or by contractual
arrangement, to manage its programs, including ensuring that
contractors are subject to appropriate independent oversight as to
cost, schedule, and quality.
To ensure that these elements are in place at the individual program
level, DOD needs to (1) begin programs in the context of an overall
funding-constrained, weapon system portfolio, and (2) say "no" when
individual programs do not measure up. DOD will need the support of the
Congress to succeed.
Management of Technological Risk
1. It has often been demonstrated that concurrent technology development
and production in programs lead to wasted funds and undelivered
capability. Do you share in this concern? Please elaborate.
I do share in this concern. It is essential that technology be developed
separately--and before--the development phase of a product or weapon
system. This is a best practice that results in predictable program
outcomes. Our work shows programs that proceed with mature technology
average about 5 percent cost growth; programs with immature technology
experience about 35 percent cost growth. Moreover, immature technology
delays design and production maturity.
2. Your most recent annual report on DOD weapon systems has many cases in
which programs are entering production before TRL Level 7 has been
reached. Can you offer some examples of programs with unnecessary
concurrency?
As we noted in that report ( [3]GAO-06-391 ), allowing technology
development to spill over into product development puts an extra burden on
decision makers and provides a weak foundation for making product
development estimates. We found that programs that began with immature
technologies have experienced average research and development cost growth
of 34.9 percent; programs that began with mature technologies have only
experienced cost growth of 4.8 percent. Examples of programs with
concurrency include the F-22A, the Joint Strike Fighter, the Future Combat
Systems, the VH-71 Presidential Helicopter, the Global Hawk, the CVN-21,
the DDG-1000, the Multi-mission Maritime Aircraft, and the National
Polar-orbiting Operational Environmental Satellite System, among others.
3. What prevents service acquisition executives from demanding that
acquisition programs not go forward until technology is mature?
First, the budgeting process requires that funding for a program is put on
the table 2 to
3 years in advance--this creates pressure to proceed with the program
despite how mature the technology is. Second, the requirements process
tends to settle on ultimate performance, which puts pressure on programs
to reach for exotic technology. Third, it is easier to say "yes" than
"no". There can always be a compelling case made that a particular program
is an exception; the problem is that there are way too many exceptions.
And finally, this is occurring, in part, because there are no consequences
for actions that run counter to the intent of DOD acquisition
polices--officials responsible for approving programs are no longer in
their positions by the time the consequences of their actions become
evident.
4. Should concurrent technology development and system production be an
exception rather than the rule?
Definitely. Sequential phasing of technology development and system
development is DOD policy, best practice, and essential to delivering
capability faster to the warfighter and more economically to the taxpayer.
5. What concrete proposals should be put in place to ensure programs are
not allowed to proceed to production without demonstrated maturity in
technology?
DOD needs to make sure each new program is executable before the
development begins, and well before production is considered. For example,
o Science and technology organizations need the resources,
authority, and tools to mature technology and transition it to
acquisition programs.
o Requirements must be clearly defined and achievable given
available resources, including beginning product development with
mature technologies. Once begun, requirements should not change
without assessing their potential to disrupt the program and
without being fully resourced.
o DOD should say no to programs that do not measure up, including
canceling programs when appropriate.
o If a decision is made to proceed with immature technologies, the
cost and schedule estimates should be increased commensurate with
the additional risk.
In addition, the Congress can help by not funding programs that do not
measure up.
Role of Contractors
1. Would you agree that DOD is relying more heavily on contractors to
manage and deliver weapon systems?
It is clear that DOD continues to increase its reliance on contractors to
provide both products and services. DOD's obligations on service contracts
rose from $82.3 billion in fiscal year 1996 to $141.2 billion in fiscal
year 2005. Based on our work looking at various major systems, we have
observed how DOD is relying on contractors in new ways to manage and
deliver weapon systems. For example, the Army's Future Combat Systems is
using a lead system integrator approach in which the prime contractor has
greater than usual responsibilities in areas such as requirements
definition, system design, and selection of major subsystem contractors.
This could increase taxpayer risk, especially if the government does not
provide adequate oversight.
2. What factors are contributing to this increased dependence on
contractors?
A variety of factors have contributed to the increased dependence on
contractors. In the case of the Future Combat Systems program, for
example, the Army chose to use a lead systems integrator because it did
not believe it had the in-house resources or flexibility to field such a
complex system in the time required. More generally, spending on service
contracting has increased in part to compensate for a declining civilian
workforce. Increased security requirements since September 11 have also
led to more contract spending, particularly in areas such as guard
services.
3. What risks does it pose to protecting the government's interests?
As reliance on contractors has increased, so has the need to ensure that
the government's interests are protected. Organizational conflicts of
interest present one type of risk to the government and these must be
identified and managed, which requires careful government oversight. When
the government is acquiring services, challenges arise when attempting to
define requirements, establish measurable and performance-based outcomes,
and assess contractor performance.
4. Do you think this increased dependence has been a conscious, managed
trend, or has it more or less happened to us?
In the area of service acquisitions in particular, which currently exceeds
the value of major weapon systems, the growth in spending has not been a
managed outcome. For example, when DOD reduced the size of its civilian
workforce in the 1990s, it did so without proactively shaping the
workforce to ensure that it had the specific skills and competencies
needed to accomplish its mission. DOD has acknowledged that it faces
significant workforce challenges that, if not effectively addressed, could
impair the responsiveness and quality of acquisition outcomes. In a recent
report, DEFENSE ACQUISITIONS: Tailored Approach Needed to Improve Service
Acquisition Outcomes, [4]GAO-07-20 , Nov. 2006, we recommended a number of
steps DOD should take to make service acquisitions a managed outcome.
5. What steps do we need to take to protect the government's interests
(that is, on behalf of both the warfighter and the taxpayer) in an
environment of increasing dependence on contractors?
We are currently reviewing a number of aspects of this issue, including
conflict of interest laws and ethical conduct standards for contractor
employees. In addition, the Congress can play an important role through
its oversight function in making sure that taxpayer interests are
protected. On November 17, 2006, we offered a set of issues, based on
GAO's work, for consideration for the agenda of the 110th Congress
(Suggested Areas for Oversight for the 110th Congress, [5]GAO-07-235R ).
The first set suggested targets for near-term oversight, including the
need to address governmentwide acquisition and contracting issues. In this
regard, we noted that the work of the government is increasingly being
performed by contractors and many agencies rely on contractors to carry
out their basic missions. At the same time, GAO's list of government
high-risk areas includes acquisition and contract management issues that
collectively expose hundreds of billions of taxpayer dollars to potential
waste and misuse. Consequently, we suggested that Congress continue to
monitor agencies' efforts to address existing problems, while facilitating
a re-examination of the rules and regulations that govern the
government-contractor relationship in an increasingly blended workforce.
Actions and topics that need continued congressional oversight include:
o requiring agencies to report on mechanisms in place to ensure
that contractors are playing appropriate roles and that agencies
have retained sufficient workforce capacity to monitor contractor
cost, quality, and performance, particularly in such critical
operations as responding to Hurricane Katrina and rebuilding Iraq;
o assessing agencies' efforts to ensure that acquisitions are
performance- and outcome-based, with appropriate risk-sharing
contracts in place.
o requiring agencies with significant acquisition budgets, such as
DOD, to better align requirements, budget, and acquisition
processes to reconcile the differences between wants, needs,
affordability, and sustainability, given current and future
demands and resources; and
o monitoring the implementation of agency action plans to address
the GAO high-risk areas related to acquisition and contract
management.
Incremental Funding
1. Is it a good idea to expand the use of incremental funding for major
weapon system programs? If not, why not?
GAO has advocated full funding for capital asset acquisitions as a way to
increase recognition of implied commitments embodied in budgetary
decisions as compared to the incremental funding approach. That said, I am
not aware of any plans by DOD to expand the use of incremental funding for
major weapon system programs. Nevertheless, in testimony earlier this year
before the Airland Subcommittee of the Senate Armed Services Committee, I
expressed concern about an Air Force plan to use incremental funding for a
multiyear contract for the F-22A program (TACTICAL AIRCRAFT: Questions
Concerning the F-22A's Business Case, [6]GAO-06-991T , July 25, 2006).
Subsequent to that hearing, the Congress authorized multiyear procurement
for the F-22A, but expressly prohibited the use of incremental funding for
the contract.
2. Is it more dangerous to expand the use of incremental funding when the
overall budget is projected to fall or become more competitive? If so,
why?
Inherent in the concept of incremental funding of a program or project is
the expectation that additional funds will be available in subsequent
years to bring the program or project to completion. Declining budgets or
increased competition for available resources in subsequent years would
certainly make it more difficult for that expectation to be realized.
NAVY: DD(X) Destroyer
1. What factors have contributed to driving DD(X) costs up and quantities
down?
DD(X) (now DDG 1000) lead ship procurement costs have increased to $3.3
billion because the Navy underestimated the resources that would be
required to construct (as well as design) a multi-mission ship offering
significant new capabilities related to air defense, land attack, and
undersea warfare. In addition, the Navy did not complete a comprehensive,
program life cycle cost estimate until March 2005. Prior to this estimate,
the Navy relied on program and contractor estimates and the life cycle
cost estimate completed in 1997 for the former DD 21 program.
2. Is this symptomatic of DOD's requirements, budgeting, and acquisition
processes in general?
Yes. Our work has shown that DOD programs often cost more and take longer
to develop than estimated when program managers are forced to make cost
and schedule predictions based on unproven technologies. Alternatively,
initiating product development with a high level of knowledge about
program technologies can help mitigate such risks.
As knowledge has grown in the DD(X) program, the Navy has frequently
discovered that its desired capabilities did not align with the funding it
had available. To compensate, the Navy has steadily reduced planned ship
quantities and land attack capabilities to enable continued funding of
other technology demonstration efforts on the ship.
We have also pointed out that the Navy generally does not hold a milestone
B, which approves entry into the System Development and Demonstration
phase, for ship programs until it is authorizing ship construction. Other
DOD programs are required to complete milestone B before entering System
Development and Demonstration. Completion of an independent cost estimate
is tied to the milestone. As a result the Navy invested billions in DD(X)
research and development without an in-depth analysis of cost. This
practice of delaying the milestone continues today on such programs as
Littoral Combat Ship and amphibious assault ship
(LHA 6).
3. What are the key risks that could jeopardize delivering the DD(X) on
time and within cost?
The Navy's strategy to continue maturing DD(X) technologies while refining
the ship's design and beginning construction of lead ships places the
program at significant risk for additional cost growth and schedule
delays.
Key development risks include remaining work associated with maturing
technologies, especially the volume search radar; completing design of the
integrated deckhouse; and maintaining software release schedules.
Because the Navy plans to begin construction of the lead DD(X) ships with
an incomplete understanding of the ship's critical technologies, design
changes may occur that could require costly rework throughout the
construction phase for the lead ships.
4. What steps need to be taken on DD(X) to ensure that it is delivered on
time and within cost?
The Navy's cost estimate for DD(X) assessed a 45 percent probability of
the lead ship delivering at or below a cost of $3.3 billion. Consequently,
the Navy may be forced to revisit planned capabilities for the ship in the
event that its $3.3 billion cost target becomes unattainable. Moreover,
before construction begins in fiscal year 2008, the Navy should ensure
that all technologies, such as the volume search radar, have been fully
demonstrated.
NAVY/MARINE CORPS: Expeditionary Fighting Vehicle (EFV)
1. What went wrong on the EFV program after it had such a good start?
Program difficulties occurred in part because not enough time was allowed
to demonstrate maturity of the EFV design during the System Development
and Demonstration (SDD) phase. Best practices and current DOD acquisition
policy call for system integration work to be conducted before critical
design review is held. This review represents the commitment to building
full-scale SDD prototypes that are representative of the production
vehicle. In the case of the EFV, however, the SDD critical design review
was held before the system integration work had been completed. While
testing of early prototypes began one year before the SDD critical design
review, it continued for three more years after the decision to begin
building the SDD prototypes. Test schedules for demonstrating design
maturity proved optimistic and success-oriented, and were extended, and
major problems were discovered in testing of the prototypes. Specifically,
the original schedule did not allow adequate time for testing, evaluating
the results, fixing the problems, and retesting to make certain that
problems are fixed before moving forward. The SDD schedule of about three
years proved too short to conduct all necessary planning and to
incorporate the results into design changes, resulting in schedule delays
and cost increases.
2. Could these problems have been avoided?
Yes. The program was positioned for a good start in that it had mature
technologies fairly early. Subsequent problems could have been avoided if
the program's System Development and Demonstration phase followed DOD
policy preferences and best practices as they related to design and
production maturity.
3. Was DOD policy in error, or was the program not following policy?
DOD policy was not in error. In fact, the EFV was following a
knowledge-based approach and other best practices, which had been
incorporated into DOD's guidelines in 2000 for major acquisition and
expanded into its May 2003 policy. In addition to having reached a
relatively high level of technology maturity before starting SDD, the EFV
program had earlier adopted best practices in its implementation of
Integrated Product Teams and had trained its program office staff on this
acquisition improvement initiative. As noted above, the EFV departed from
the tenets of DOD acquisition policy and best practices shortly after SDD
began, when it attempted to do system integration and system demonstration
concurrently.
4. How do strategies that go against sound policy get approved?
Our work on multiple weapon systems shows that most programs proceed with
strategies that do not comport with the knowledge-based approach embodied
in DOD written policies. For example, in our March 2006 assessment of
weapon systems, only 10 percent of the weapon systems reviewed had begun
SDD with mature technologies. There are several reasons for this. First,
although DOD acquisition policy allows individual programs to tailor their
own acquisition approaches, it does not include sufficient controls to
ensure that key aspects of the policy are implemented. Second, programs
proceed with optimistic assumptions about what they can accomplish for
their estimated cost and schedules. Third, as weapon system programs
proceed through development, the desire to protect the program against
disruption intensifies.
5. What lesson from the EFV experience can be drawn and applied to
DOD acquisitions in general?
Using the lens of a knowledge-based business case, the EFV at the
start of SDD was sound on requirements and technology maturity
(knowledge point 1). While design stability was judged to be
attained at the critical design review (knowledge point 2)
immediately after entering SDD, it appears that holding critical
design review so soon was premature. This particular acquisition
strategy did not provide the resources (time and money) necessary
to demonstrate design maturity and production maturity (knowledge
point 3). A key lesson learned is that while it is necessary to
demonstrate one knowledge point before a subsequent one can be
demonstrated, this alone is not sufficient. Attaining one
knowledge point does not guarantee the attainment of the next one.
Rather, the acquisition strategy for any program must adequately
provide for the attainment of each knowledge point even in
programs, such as the EFV, which were in a favorable position at
the start of SDD. If the acquisition strategy does not adequately
provide for the attainment of all knowledge points, the estimates
for cost and schedule will not have a sound basis.
NAVY: Cost Growth in Shipbuilding Programs
1. What did the GAO's work reveal as the major causes of continued
overruns in the Navy's shipbuilding program?
Our February 2005 report ( [7]GAO-05-183 ) on cost growth in shipbuilding
programs cited unrealistic estimates of costs as a key source of cost
growth. Unrealistic estimates in turn led to unrealistic budgets and
contract prices. Increases in labor hours and material costs were common,
with low technology maturity at program start and immature design before
beginning production contributing to these. We also identified
deficiencies in cost reporting as a problem.
2. The Navy has pledged over the past year or so to address this problem.
Have you seen any concrete proposals, and if so, what do you think of
them?
The Navy has cited several actions it has taken to improve cost
performance. These include using risk analysis in developing cost
estimates, budgeting at the Office of the Secretary of Defense Cost
Analysis Improvement Group estimate, using realistic inflation rates in
developing cost estimates, withholding fees if the contractor does not
provide useable and timely cost reports, and introducing more aggressive
contract incentives.
We believe these are positive steps but it remains to be seen whether the
Navy will follow through with these efforts. For example, while the Navy
has said that it will use contract incentives in the form of more
demanding share lines, lower ceiling prices, and lower minimum fees (for
cost plus incentive fee contracts), a key issue is whether these policies
will be sustained over time.
We also believe that further actions are needed. To reduce the risk of
cost growth, the Navy needs to insure that technology risk is not carried
forward into construction. Technology risk, and with it the potential for
design changes, needs to be addressed before construction begins. This has
not been the case with the DD(X) program, where key technologies--such as
the dual band radar--have not been fully demonstrated.
Moreover, we believe the Navy could better ensure realism of budgets and
contracts. We believe funding and contracting for a detailed design of the
ship should be separated from the funding and contracting for
construction. A budget request for ship construction would be informed by
the knowledge gained from the first year or two of detail design.
3. Should ship programs be put on firm fixed price contracts?
Use of fixed price contracts is dependent on how much uncertainty exists.
It is important to reduce uncertainty early in a program so that
technology and design risks are not carried forward into construction.
Following a knowledge-based approach will likely enable follow-on ships to
use fixed price contracts. Use of such contracts would provide the
strongest incentive for improving cost performance--under a fixed price
contract the cost risk of performance generally shifts to the contractor.
4. What specific ideas would you offer to deal with this problem?
In addition to the suggestions discussed above, we believe the Navy should
increase its use of cost-performance trade studies--including
design-to-cost and cost as an independent variable (CAIV) techniques--to
identify cost reduction opportunities in its new ship designs and position
the Navy to achieve "best value" outcomes in its shipbuilding programs.
Establishing design-to-cost goals early in a program encourages
consideration of cost as a key design parameter when evaluating other
parameters such as schedule, performance, and operational capability.
Goals can be set in each phase of the shipbuilding acquisition process and
tracked until desired cost targets have been achieved. In addition,
iterative CAIV analyses can aid Navy efforts to properly balance cost with
performance, design, and system requirements in its shipbuilding programs.
Such analyses can help the Navy quantify and evaluate the degree to which
its planned investments will result in increased utility to the warfighter
and subsequently position the Navy to tailor future ship capabilities in
ways that maximize value.
ARMY: Future Combat Systems (FCS)
1. Was the Army's decision to begin System Development and Demonstration
phase with over 75 percent of FCS critical technology elements immature
consistent with DOD policy?
The decision to enter the System Development and Demonstration phase with
75 percent of technologies immature does not satisfy the basic tenets of
DOD policy and best practices. DOD policy and best practices are very
clear that technologies should be mature before system development begins.
FCS, in particular, is worrisome because it is now three years past the
beginning of development, and most of the technologies remain immature by
best practice standards.
2. In retrospect, what should have been done to bring the program in line
with DOD policy and best practices?
The program should not have entered System Development and Demonstration
with undefined requirements and immature technologies. Rather, it should
have remained in the science and technology community until the enabling
technologies were mature, or requirements should have been traded down to
match the available technologies.
3. What are the implications for other acquisition programs--will they
adhere to DOD policies for awarding System Development and Demonstration
contracts for only those weapons whose critical technologies have achieved
desired maturity levels?
Programs like FCS underscore the challenge in getting better program
outcomes. Despite being out of alignment with the knowledge-based approach
outlined in DOD policy and advocated by best practices, the program was
approved to begin the System Development and Demonstration phase and
funded at high levels. The example of success then becomes not programs
that follow the intent of the policy, but those that do not and still win
approval and funding.
4. What can be done to mitigate risks in FCS, given where it is today?
We have recommended that the Secretary of Defense limit DOD's commitment
to the FCS product development phase and eventual production until a sound
business case that is consistent with DOD acquisition policy and best
practices can be clearly demonstrated. Further, we have recommended a
go/no-go decision be made in 2008 after the preliminary design is done.
This would in essence be the time that the program is ready for a system
development decision, as technologies should have reached a basic level of
maturity and system level requirements should be finalized by this time.
In addition, the Army's fiscal outlook is extremely challenging. Outside
of the Army, competition for production dollars will be great as both the
Air Force and Navy have significant production programs planned for that
time. Outside DOD, there will be an increasing squeeze on discretionary
funds (weapons are discretionary) from other federal sectors.
5. What does the reliance on a lead systems integrator (LSI) say about the
Army's ability to manage a program of this magnitude?
The Army's reliance on a LSI suggests that the FCS is so ambitious in
scope, complexity and schedule, that it exceeded the Army's own ability to
effectively manage such a complex program.
6. What risks does the use of an LSI pose for protecting the interests of
the government?
The use of an LSI can pose a risk with respect to organizational conflicts
of interest. For example, an LSI might favor its own solutions over other
competing solutions. Further, if the LSI stands to benefit from the
continuation of a program into production, it has a financial stake in the
outcome that could compromise judgment. Over time, the government's
ability to oversee and act independently from the LSI could weaken as
experienced government managers retire and dependence on the LSI
increases.
ARMY: Warfighter Information Network-Tactical (WIN-T)
1. Was WIN-T another program that was approved with unrealistic
expectations for requirements, technology, cost, and schedule? Did the
program's original strategy follow DOD policy and best practices?
At the time DOD approved the start of system development in August 2003,
only three of WIN-T's 12 critical technologies were close to full
maturity. Although DOD policy calls for technology maturity by the start
of system development, DOD did not require that the other nine critical
technologies reach a level close to full maturity until the start of
low-rate production.^1 The technical and program risks of moving forward
with immature technologies are exacerbated by the significant
interrelationships among WIN-T's critical technologies. For example,
mobile communications technologies--a critical component of WIN-T's
operational concept--rely on antennas and other technologies to achieve
their performance objectives, and a lag in the development of any of these
technologies would result in a lag in the overall development of mobile
communications.
With respect to requirements, DOD had approved an operational requirements
document for WIN-T shortly before the start of system development.
However, WIN-T's interdependencies with other programs further heighten
the technical challenges and risks associated with the program. In
particular, at the time WIN-T system development began, the Army's
networking requirements for FCS--which WIN-T is expected to heavily
support--were not well understood. As a result, the networking needs of
FCS and the capabilities WIN-T planned to deliver to this program were not
well synchronized. In addition, WIN-T's tightly compressed schedule
assumed nearly flawless execution and did not allow sufficient time for
correcting problems if they arose.
2. What steps did the Army have to take to buy JNN? Why was this approach
necessary?
Rather than undergoing the normal acquisition processes, the Army resorted
to extraordinary measures outside of the normal requirements, acquisition,
and budgeting processes--including the supplemental budget process--to
rapidly acquire and field the Joint Network Node-Network (JNN-N).
Specifically, instead of initiating a formal acquisition program, the Army
utilized existing contracts, originally established for modifying legacy
networking systems, to procure JNN-N equipment in approximately 5 months.
For example, a purchase agreement from the WIN-T program was used to
procure satellite trailers; a General Services Administration contract was
used to arrange training; and existing Army contracts were used to procure
JNN-N in seven spirals, with each spiral equating to approximately one
division's worth of equipment. Supplemental funding totaling $763.6
million for fiscal years 2004 and 2005 was used to purchase the first
seven spirals of JNN-N. Additional supplemental funding totaling $818.7
million for fiscal year 2006 is being used to acquire JNN-N lots 8 through
10.^2
1In mid-2003, the Army developed risk mitigation plans for the nine
immature technologies, but a program review sponsored by the Army in July
2004 concluded that these plans lacked sufficient detail.
The Army's approach for acquiring JNN-N was intended to address an urgent
need identified during Operation Iraqi Freedom and Operation Enduring
Freedom to provide better communications capabilities between soldiers
fighting out of sight of one another. DOD's processes for approving formal
programs are extensive and have not been conducive to rapid acquisition.^3
However, these processes exist to ensure that programs are managed and
overseen, properly funded, and adequately tested prior to fielding. The
Army's decision to acquire the capability through an approach that was
outside these processes did not address the legal and policy requirements
for managing an acquisition program of this magnitude. In particular,
JNN-N procurement did not complete initial operational testing before
going beyond low-rate production, as required by law and DOD policies.
The acquisition of JNN-N through extraordinary measures was made
necessary, in part, by DOD's decision to acquire a revolutionary WIN-T
capability. When the Army opted to pursue large technology advances in
networking capabilities to support the future force through WIN-T, rather
than pursuing a more incremental approach, it accepted a gap in providing
tactical networking capabilities to the warfighter. Moreover, the Army's
optimistic acquisition approach for WIN-T created the impression that this
capability gap was far smaller than it really was. As the Army pursued an
advanced WIN-T capability, its existing communications equipment became
increasingly obsolete and the need for near-term improvements became more
urgent with the onset of the war in Iraq. If the Army had followed DOD's
acquisition policy preferences, which emphasize achieving capabilities in
increments based on mature technologies to get capability into the hands
of the user more quickly, it might have been able to get needed
communications capabilities to the warfighter sooner.
^2In keeping with its transition from a division-based to a modular
brigade-based force, the Army is now procuring JNN-N in batches of brigade
sets. It terms these batches lots, rather than spirals.
^3The Army explored using special rapid acquisition authority to respond
to combat emergencies to formally waive acquisition laws and policies for
JNN-N, but the dollar limit for use of this authority--$100 million per
year for the entire Department of Defense--was far too low to accommodate
the Army's need for JNN-N. See section 806 of the Bob Stump National
Defense Authorization Act for Fiscal Year 2003 (P.L. 107-314), as amended
by section 811 of the Ronald W. Reagan National Defense Authorization Act
for Fiscal Year 2005 (P.L. 108-375).
3. What lessons do the WIN-T and JNN acquisitions have for DOD's
requirements, budgeting, and acquisition processes?
While JNN-N's acquisition was a pragmatic response to an urgent need, the
urgency was brought about, in part, by the Army's decision to use a
revolutionary, rather than evolutionary, approach to acquiring WIN-T. This
approach appeared less risky at the time because optimistic projections
created the impression that the capability gap was narrower than it really
was. In hindsight, focusing on meeting the needs of the future force at
the expense of providing capabilities to the current force proved
detrimental once operations commenced in Iraq. To field capabilities
quickly, the Army did not use DOD's formal requirements setting,
budgeting, and acquisition processes, and over a billion dollars was spent
on JNN-N without it being established as a formal program. This is not a
good model for providing needed capabilities. The WIN-T/JNN-N experience
has implications beyond these two communications systems. The Army's
experience thus far creates opportunities to capture lessons learned and
apply them not only to the Army's two programs but also to DOD's larger
ongoing efforts to reform its acquisition processes. We recently
recommended that the Secretary of Defense examine ways to apply the
lessons learned from the Army's experience with the acquisition of WIN-T
and JNN-N to ongoing DOD-wide efforts to reform its key decision support
processes for setting requirements, allocating resources, and acquiring
weapon systems. DOD concurred with this recommendation.
4. What challenges are presented by the merging of aspects of the WIN-T
and JNN programs?
The Army ultimately intends to create a tactical communications network
that functions as a single tactical backbone. To achieve this, the Army
must now reconcile two separate acquisition efforts intended to meet one
overarching need. Without a sound strategy for meeting near- and long-term
needs for communications and networking capabilities, significant risks
remain. Affordability continues to be an overriding concern, particularly
as the Army and DOD as a whole struggle to meet the needs of current
military operations while making prudent investments in the future in a
difficult fiscal environment. We recently recommended that the Secretary
of Defense direct the Secretary of the Army to develop a sound strategy
for transitioning the Army's tactical communications from JNN-N to WIN-T.
Such a strategy needs to ensure that requirements for both of these
efforts are clearly defined and well integrated, that the fielding
strategy is executable, and that sufficient funding has been committed.
DOD concurred with this recommendation and noted that the Office of the
Secretary of Defense has directed that an approved JNN-N to WIN-T
transition strategy be in place before acquiring additional JNN-N systems.
Air Force/NAVY/Marine Corps: Multi-Service F-35 Joint Strike Fighter (JSF)
1. Were JSF technologies mature at the time of the decision to enter
System Development and Demonstration?
JSF entered development without its eight critical technologies being
mature. Information from the program office indicates that maturity has
progressed. However, seven technologies were not mature at the recent
design review and two of the technologies needed for mission effectiveness
are not expected to be fully mature until 3 years after production begins.
We reported in October 2001 that failure to mature these critical
technologies could result in increases in the production and long-term
ownership costs, schedule delays, and compromised performance as problems
arise in product development.
2. Is the program ready for a low-rate initial procurement decision in FY
2007? What risks does the program face at the time of this decision?
The JSF program is not well positioned to enter production at this time.
At the time of the low-rate production decision, only one early prototype
will have flown and less than one percent of the flight testing will be
complete. In addition, none of the three variants will have a
production-representative aircraft built and in flight testing. The
program intends to invest $26 billion in initial production before a fully
integrated, production-representative aircraft will fly in 2011--4 years
after the production decision. The cost of discovering design problems
during production could be significant if testing shows that large,
structural components of the aircraft require modifications. Furthermore,
the length and scope of the remaining effort make it difficult to
accurately estimate cost and completion schedules. The program has about 7
years to complete the remaining activities of the System Development and
Demonstration phase, such as developing 22 million lines of software code
and completing a 7-year, 12,000-hour flight test program. By investing
heavily in procuring JSF aircraft before flight testing proves it will
perform as expected, DOD has significantly increased its risk of adding
more cost and delaying the delivery of critical capabilities to the
warfighter.
3. What criteria must the program meet to demonstrate readiness for
low-rate production?
Key indicators of a program's readiness for entering production and making
significant investments in tooling, facilities, and materials include
demonstrating that (1) the aircraft's flying qualities function within the
parameters of the flight envelope, (2) the aircraft design is reliable, or
(3) a fully integrated and capable aircraft system can perform as
intended. In addition to a stable design, production processes must be
mature. This point is achieved when the program can manufacture the
aircraft within cost, schedule, and quality targets. Because the program
will lack key design and testing knowledge, DOD plans to use cost
reimbursement contracts for early production aircraft. This type of
contract places significantly greater cost risk on DOD and the taxpayer.
Confidence that investment decisions will deliver expected capability
within the cost and schedule goals is likely to increase when flight
testing proves the JSF will work as expected.
4. When will the first production-representative, fully integrated
prototype aircraft be demonstrated? How many production aircraft will be
on contract by that time?
Under the current schedule, the first production representative, fully
integrated aircraft will not begin flight testing until the 2011
timeframe. This occurs 4 years after production has begun. At that time,
DOD will have committed to buy about 190 aircraft costing $26 billion. By
the time testing is planned to be completed, in 2013, DOD will have
procured more than double that amount--424 aircraft--at an estimated cost
of about $49 billion.
5. What are the cost implications if that prototype indicates the need for
design changes at that stage of the program?
The costs of discovering design changes could be significant if testing
shows that large, structural components of the aircraft require
modifications or mission systems require additional development. For
example, problems discovered late in the development of the F-22A
increased a 4-year flight test program to about 8 years and affected the
program's ability to conduct operational testing and move into production.
In the JSF program, design changes needed in one variant could also ripple
through the other two variants, reducing efficiencies necessary to lower
production costs with common parts and manufacturing processes for the
three variants. Some industry officials have indicated that the cost of
design changes such as these could be 10 to 1,000 times greater, depending
on how far the product has progressed into production.
6. Under a cost-reimbursable production contract, will the government be
responsible for assuming the risk and cost of any needed design changes?
Cost reimbursement contracts provide for payment of allowable incurred
costs, to the extent prescribed by the contract. These contracts are
generally used when uncertainties involved in contract performance do not
permit costs to be estimated with sufficient accuracy to use any type of
fixed price contract. Under a cost reimbursement contract, the government
assumes most of the risks of performance--because it agrees to pay the
contractor its allowable incurred costs to the extent prescribed in the
contract.
Air Force: F-22 Aircraft
1. What fundamental causes led to the F-22's cost, schedule, and
performance problems?
We have reported in the past on the inefficiencies of the F-22 acquisition
program that led to numerous schedule delays and cost increases. This
program provides an excellent example of what can happen when a major
acquisition program is not guided by the principles of evolutionary,
knowledge-based acquisition. The F-22 program failed to match requirements
with resources and make early trade-offs and took on a number of new and
unproven technologies. As a result, the program failed to capture the
appropriate technology, design, and manufacturing knowledge of the F-22
aircraft at the right times during the development process. This created a
challenging and risky acquisition environment that ultimately delayed by
about 10 years to the warfighter the capabilities expected from this new
aircraft. It additionally led to rising development costs, affordability
issues, and changing requirements over the 19 years in development.
2. Has the F-22 followed DOD policy and best practices?
No, the F-22 did not follow a best practices approach to its development
and procurement. As to whether the F-22 followed DOD policy, we have not
specifically assessed the F-22 program's compliance with DOD policy.
However, we reported in 2003 ( [8]GAO-04-53 ) and 2006 ( [9]GAO-06-368 )
that DOD policy includes a good framework for weapons system development
and acquisition, but it does not include sufficient controls to ensure key
elements of the policy are implemented in each phase of a weapons system
program. As a result, our 2006 report found that weapons system
development programs continue to experience cost and schedule growth.
3. What concerns do you have over the multiyear procurement proposal for
buying more F-22s?
In testimony earlier this year before the Airland Subcommittee of the
Senate Armed Services Committee, we expressed a number of concerns about
an Air Force proposal for multiyear procurement of the F-22A (TACTICAL
AIRCRAFT: Questions Concerning the F-22's Business Case, [10]GAO-06-991T ,
July 25, 2006). These concerns related to the savings expected to be
realized, the use of incremental funding, and the cost risk that could
arise if quantities to be ordered continued to decline.. Although Congress
subsequently authorized multiyear procurement of the F-22A, it listed a
number of requirements the Air Force would have to address related to the
concerns we had expressed. In addition, the Congress expressly prohibited
the use of incremental funding.
Questions from Representative Kaptur:
In addition to the questions listed above, the Subcommittee also submitted
to GAO and DOD a list of eleven supplemental questions for the record from
Representative Marcy Kaptur, some of which deal with the use of
contractors in Iraq. These questions and others touch on an emerging issue
of critical importance, that is, the increasing role that contractors play
in conducting the business of government. We appreciate and share the
concern underlying the questions. Although DOD is in a better position
than GAO to provide the information that would be needed to respond to
each of the eleven questions, we have issued reports regarding the use of
private security contractors in Iraq and DOD's use of logistic support
contractors, which we believe are relevant to the issues raised by
Representative Kaptur. The reports are:
REBUILDING IRAQ: Actions Needed to Improve Use of Private Security
Providers, [11]GAO-05-737 , July 2005.
MILITARY OPERATIONS: DOD's Extensive Use of Logistics Support Contracts
Requires Strengthened Oversight, [12]GAO-04-854 , July 2004.
We would be pleased to provide copies of these reports upon request. They
are also available on our Web site, http://www.gao.gov .
(120598)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or (202)
512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
References
Visible links
3. http://www.gao.gov/cgi-bin/getrpt?GAO-06-391
4. http://www.gao.gov/cgi-bin/getrpt?GAO-07-20
5. http://www.gao.gov/cgi-bin/getrpt?GAO-07-235R
6. http://www.gao.gov/cgi-bin/getrpt?GAO-06-991T
7. http://www.gao.gov/cgi-bin/getrpt?GAO-05-183
8. http://www.gao.gov/cgi-bin/getrpt?GAO-04-53
9. http://www.gao.gov/cgi-bin/getrpt?GAO-06-368
10. http://www.gao.gov/cgi-bin/getrpt?GAO-06-991T
11. http://www.gao.gov/cgi-bin/getrpt?GAO-05-737
12. http://www.gao.gov/cgi-bin/getrpt?GAO-04-854
*** End of document. ***