Defense Acquisitions: Issues Concerning Airlift and Tanker
Programs (07-MAR-07, GAO-07-566T).
The Department of Defense (DOD) has continuing efforts to
modernize its airlift and tanker fleets by investing billions of
dollars to modify legacy airlift systems, such as the C-5 and
C-130, and procure new aircraft, such as a tanker replacement.
Acquisition has been on GAO's list as a high risk area since
1990. GAO has reported that elements contributing to a sound
business case for an acquisition are missing or incomplete as DOD
and the services attempt to acquire new capabilities. Those
elements include firm requirements, mature technologies, a
knowledge-based acquisition strategy, a realistic cost estimate,
and sufficient funding. Acquisition problems that include failure
to limit cost growth, schedule delays, and quantity reductions
persist, but fiscal realities will not allow budgets to
accommodate these problems any longer. Today's testimony
addresses (1) the analyses supporting the Department of Defense's
(DOD) mobility capabilities and requirements and (2) actions that
are needed to improve the outcomes of weapon system acquisitions.
For this testimony, GAO drew from issued reports, containing
statements of the scope and methodology used, as well as recently
completed work not yet reported. GAO's work was performed in
accordance with generally accepted government auditing standards.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-566T
ACCNO: A66545
TITLE: Defense Acquisitions: Issues Concerning Airlift and
Tanker Programs
DATE: 03/07/2007
SUBJECT: Accountability
Airlift services
Best practices
Cost analysis
Data collection
Data integrity
Defense capabilities
Defense cost control
Defense procurement
Military aircraft
Military systems analysis
Program evaluation
Reporting requirements
Requirements definition
Risk assessment
Schedule slippages
Strategic planning
Cost estimates
Cost growth
C-130 Aircraft
C-130 Avionics Modernization Program
C-130J Aircraft
C-5 Aircraft
C-5 Avionics Modernization Program
C-5 Reliability Enhancement and
Reengining Program
C-X Aircraft
DOD Mobility Capabilities Study
Galaxy Aircraft
GAO High Risk Series
Hercules Aircraft
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-566T
* [1]Summary
* [2]Background
* [3]Analyses Used to Determine Mobility and Tanker Capabilities
* [4]Mobility Capabilities Study Limitations Raise Questions abou
* [5]Mandatory Analyses Done to Support Passenger and Cargo Capab
* [6]Employing Best Practices Can Shape Better Program Outcomes f
* [7]Outcomes of Certain Airlift Programs
* [8]Airlift Programs Have Not Captured Critical Product Knowledg
* [9]C-5 AMP
* [10]C-5 RERP
* [11]C-130 AMP
* [12]C-130J
* [13]Conclusion
* [14]Contact and Staff Acknowledgements
* [15]Related GAO Products
* [16]Order by Mail or Phone
Testimony
Before the House Committee on Armed Services, Air and Land Forces
Subcommittee
United States Government Accountability Office
GAO
For Release on Delivery Expected at 3:00 p.m. EST
Wednesday, March 7, 2007
DEFENSE ACQUISITIONS
Issues Concerning Airlift and Tanker Programs
Statement of William M. Solis, Director Defense Capabilities and
Management Issues and Michael J. Sullivan, Director Acquisition and
Sourcing Management Issues
GAO-07-566T
Mr. Chairman and Members of the Subcommittee:
We are pleased to be here to discuss the development of aircraft program
requirements and issues related to the acquisition process. We recently
issued our report on high risk areas in the federal government, which
lists DOD's acquisition process as one longstanding area of concern.^1
Acquisition has been on this list since 1990. As we have reported, DOD
knows what to do to achieve more successful outcomes, but finds it
difficult to apply the necessary discipline and controls or assign much
needed accountability. DOD has not been employing a knowledge-based
development approach and business cases have not measured up. DOD has just
begun piloting some corrective actions, so the proof of actual
implementation may be years away. These initiatives also may not
necessarily be applied to programs already under way.
DOD has continuing efforts to modernize its airlift and tanker fleets by
investing billions of dollars to modify legacy airlift systems, such as
the C-5 and C-130, and procure new aircraft, such as the KC-X replacement
tanker. We have reported in the past that a sound business case for an
acquisition contains firm requirements, mature technologies, a
knowledge-based acquisition strategy, a realistic cost estimate, and
sufficient funding. However, we have found many of these elements are
missing or incomplete as DOD and the services attempt to acquire new
capabilities. Persistent acquisition problems include failure to identify
needs versus wants and to limit cost growth, schedule delays, and quantity
reductions, but fiscal realities will not allow budgets to accommodate
these problems any longer.
Today I will highlight for you some issues related to the analyses
supporting the Department of Defense's (DOD) mobility capabilities and
requirements and Mike Sullivan will discuss actions that are needed to
improve the outcomes of weapon system acquisitions. For this testimony, we
drew from issued reports, which contain statements of the scope and
methodology used, as well as recently completed work not yet reported. Our
work was performed in accordance with generally accepted government
auditing standards.
^1 GAO, High Risk Series: An Update, GAO-07-310 (Washington, D.C.: Jan.
2007).
Summary
DOD has an mandate to deliver high-quality products to warfighters, when
they need them and at a price the country can afford. However, our work
shows that acquisition problems will likely persist until DOD provides a
better foundation for buying the right things, the right way. This
foundation begins with setting requirements that are based on adequate and
complete analyses, using current operational data and updated, effective
models. For the past several years, we have reported our concerns with the
analyses done to support requirements and have recently issued two reports
that raise concerns about the quality of analyses underpinning the
programmatic decision-making surrounding DOD's airlift requirements. In
September 2006, we issued our report on DOD's Mobility Capabilities Study
(MCS). The MCS determined that the projected mobility capabilities are
adequate to achieve U.S. objectives with an acceptable level of risk
during the period from fiscal years 2007 through 2013; that is, the
current U.S. inventory of aircraft, ships, prepositioned assets, and other
capabilities are sufficient, in conjunction with host nation support. In
our report, we stated that conclusions of the MCS were based on incomplete
data and inadequate modeling and metrics that did not fully measure stress
on the transportation system.^2 We further observed that, in some cases,
the MCS results were incomplete, unclear, or contingent on further study,
making it difficult to identify findings and evaluate evidence. It is not
clear how the analyses done for the study support DOD's conclusions and we
suggested that decision makers exercise caution in using the results of
this study to make programmatic decisions. This week, we issued a report
on the lack of mandatory analyses to support a passenger and cargo
capability for the new replacement refueling aircraft, the KC-X tanker.^3
Contrary to mandatory Air Force implementing guidance, the Air Force
proposed a capability without an analyses identifying an associated gap,
shortfall, or redundancy. Air Force officials could not provide supporting
information sufficient to explain this discrepancy between the required
analyses and their proposal.
^2 GAO, Defense Transportation: Study Limitations Raise Questions about
the Adequacy and Completeness of the Mobility Capabilities Study and
Report, [17]GAO-06-938 (Washington, D.C.: Sept. 20, 2006).
^3 GAO, Defense Acquisitions: Air Force Decision to Include a Passenger
and Cargo Capability in Its Replacement Refueling Aircraft Was Made
without Required Analyses, [18]GAO-07-367R (Washington, D.C.: Mar. 6,
2007).
Successful acquisition programs make sound decisions based on critical
product knowledge to ensure that program investments are getting promised
returns--on time delivery, within estimated costs, and with expected
capabilities. This is important because DOD expects to invest over $12
billion in new and improved capabilities in four airlift programs
discussed in this testimony between now and 2013--the C-5 Avionics
Modernization Program, C-5 Reliability Enhancement and Reengining Program,
C-130 Avionics Modernization Program, and the C-130J acquisition program.
These four programs have diverged from the best development practices
intended to produce good outcomes and as a result have experienced
significant cost growth and schedule delays. We found that all four
programs failed at basic systems engineering practices to 1) fully
understand the resources needed to integrate proven commercial
technologies, 2) achieve a stable design before beginning system
demonstration, and 3) demonstrate the aircraft would work as required
before making large production investments. As a result, each has
encountered significant delays in delivering capability to the field and
as a group have spent $962.3 million (fiscal year 2007 dollars) more than
planned for development.
Background
DOD must be capable of rapidly deploying armed forces to respond to
contingency and humanitarian operations around the world. Airlift and
tanker aircraft play a vital role in providing this capability. Over the
past 25 years, DOD has invested almost $141 billion to develop, procure,
and modify its airlift and tanker forces with an additional investment
planned for fiscal years 2007 through 2011 of $32 billion. Recent annual
funding levels are at the highest levels in two decades. (See figure 1.)
Figure 1: Planned and Future DOD Investments in Airlift and Tanker Fleets
(1982 to 2011)
Note: Based on DOD's Fiscal Years Defense Program (2007).
In December 2005, DOD issued a report on the study of its mobility
capabilities. The goal of this Mobility Capabilities Study was to identify
and quantify the mobility capabilities needed to support U.S. strategic
objectives into the next decade. The MCS determined that the projected
mobility capabilities are adequate to achieve U.S. objectives with an
acceptable level of risk during the period from fiscal years 2007 through
2013; that is, the current U.S. inventory of aircraft, ships,
prepositioned assets, and other capabilities are sufficient, in
conjunction with host nation support. The MCS emphasized that continued
investment in the mobility system, in line with current departmental
priorities and planned spending, is required to maintain these
capabilities in the future. This includes, for example, fully funding Army
prepositioned assets as planned and completing a planned reengineering of
the C-5 aircraft.
In our previous reports concerning acquisition outcomes and best
practices, we have noted the importance of matching warfighter
requirements with available resources, a responsibility shared by the
requirements and acquisition communities in DOD. As described in Air Force
implementing guidance, there is within DOD a distinct separation between
the requirements authority and acquisition authority.^4 Under this
guidance, this separation requires early and continued collaboration
between both communities.
Analyses Used to Determine Mobility and Tanker Capabilities Were Inadequate
Analyses done for the MCS contained methodological limitations that create
concerns about the adequacy and completeness of the study and decision
makers approving the KC-X tanker proposal lacked required analyses
identifying need and associated risk for a passenger and cargo capability.
Mobility Capabilities Study Limitations Raise Questions about Adequacy and
Completeness
While DOD used an innovative approach in conducting the study and
acknowledged some methodological limitations in its report, it did not
fully disclose how these limitations could affect the MCS conclusions and
recommendations. In September 2006, we reported that DOD's conclusions
were based, in some instances, on incomplete data and inadequate modeling
and metrics that did not fully measure stress on the transportation
system, and that, in some cases, MCS results were incomplete, unclear, or
contingent on further study, making it difficult to identify findings and
evaluate evidence. It is not clear how the analyses done for the study
supported DOD's conclusions, and we suggested that decision makers
exercise caution in using the results of this study to make programmatic
decisions.
As measured against relevant generally accepted research standards, we
identified limitations in the MCS study and report that raise questions.
Among our findings
o Aspects of modeling and data were inadequate in some areas
because data were lacking and some of the models used could not
simulate all relevant aspects of the missions. The report did not
explain how these limitations could affect the study results or
what the effect on the projected mobility capabilities might be.
Relevant research standards require that models used are adequate
for the intended purpose and represent a complete range of
conditions, and also that data used are properly generated and
complete. For example, the MCS modeled hypothetical homeland
defense missions rather than missions for homeland defense demands
from a well-defined and approved concept of operations for
homeland defense because the specific details of the missions were
still being determined, and DOD acknowledged that the data used
may be incomplete. The MCS also was unable to model the flexible
deterrent options/deployment order process to move units and
equipment into theater due to lack of data, but the study assumed
a robust use of this process, which in one scenario accounted for
approximately 60 percent of the airlift prior to beginning combat
operations.^5 In addition, the MCS report contains more than 80
references to the need for improved modeling, and 12 of these
references call for additional data or other refinements.
Additionally, the MCS modeled the year 2012 to determine the
transportation capabilities needed for the years 2007 through
2013. The year 2012 did not place as much demand for mobility
assets in support of smaller military operations, such as
peacekeeping, as other years. However, DOD officials considered
2012--the year modeled--as "most likely" to occur and stated that
statistically it was not different from other years in the 2007 to
2013 period even though the number of smaller military operations
is the least of any of the years reviewed. As I mentioned, we have
reported before on the lack of data available for analysis that
could benefit decision makers. In September 2005, we reported that
the Air Force captured data on short tons transported but did not
systematically collect and analyze information on operational
factors, such as weather and runway length, that impact how much
can be loaded on individual missions.^6 Therefore, Air Force
officials could not know how often it met its secondary goal to
use aircraft capacity as efficiently as possible. Without this
information, Air Mobility Command officials do not know the extent
to which opportunities exist to use aircraft more efficiently and
whether operational tempo, cost, and wear and tear on aircraft
could be reduced. In addition, DOD officials do not have the
benefit of such analysis to determine future airlift requirements
for planning purposes.
o While the MCS concluded that combined U.S. and host nation
transportation assets were adequate to meet U.S. objectives with
acceptable risk, the report, in describing the use of warfighting
metrics in its analyses, does not provide a clear understanding of
the direct relationship of warfighting objectives to
transportation capabilities. Acknowledging this point, the report
stated that further analysis is required to understand the
operational impact of increased or decreased strategic lift on
achieving warfighting objectives. Relevant generally accepted
research standards require that conclusions be supported by
analyses. The use of warfighting metrics is a measure to determine
whether combat tasks, such as achieving air superiority, are
achieved. However, they do not measure whether appropriate
personnel, supplies, and equipment arrived in accordance with
timelines. As a result, we could not determine how the study
concluded that planned transportation assets were adequate because
the study did not contain a transparent analysis to support its
conclusion or a clear roadmap in the report to help decision
makers understand what that conclusion meant in terms of type and
number of mobility assets needed. Previous DOD mobility studies
primarily used mobility metrics, which measured success in terms
of tons of equipment and personnel moved per day to accomplish
military objectives. The use of both warfighting and mobility
metrics to measure success would allow decision makers to know
whether combat tasks were achieved and how much strategic
transportation is needed to accomplish those tasks.
o In some cases, the MCS results were incomplete, unclear, or
contingent on further study, making it difficult to identify
findings and evaluate evidence. Relevant research standards
require results to be presented in a complete, accurate, and
relevant manner. For example, the report contains several
recommendations for further studies and assessments, five of which
are under way. However, at the time of our report, DOD had no
plans to report the effect of these studies on the MCS results
after the studies are complete. In addition, the report contains
qualified information that is not presented clearly, such as
varying assessments of intratheater assets in three different
places in the report. The lack of clarity and conciseness of the
reported results can limit the study's usefulness to decision
makers and stakeholders.
The MCS report also made recommendations to conduct further
studies, develop plans and strategies, and improve data collection
and mobility models. In fact, DOD officials told us at the time
that a Mobility Capabilities Study-2006 was underway, as well as
studies on intratheater lift, aerial refueling, and other mobility
issues. However, unless DOD addresses the concerns I just outlined
for you, decision makers may be unable to clearly understand the
operational implications of the study results and make fully
informed programmatic investment decisions concerning mobility
capabilities. Also, some of the underlying assumptions used in the
MCS have now changed significantly, such as the assumption that
Army prepositioned equipment is in place and fully funded, which
will no longer be the case. Therefore, the MCS analyses and
results, which would be the starting point for any new studies,
may no longer be relevant.
Mandatory Air Force policy requires Air Force organizations to use
a formal capabilities-based approach to identify, evaluate,
develop, field, and sustain capabilities that compete for limited
resources. Contrary to mandatory Air Force implementing guidance,
however, the Air Force proposal for a replacement refueling
aircraft, the KC-X tanker, included a passenger and cargo
capability without analyses identifying an associated gap,
shortfall, or redundant capability. According to mandatory Air
Force implementing guidance, analyses supporting the
decision-making process should assess a capability based on the
effects it seeks to generate and the associated operational risk
of not having it. In this case, the supporting analyses determined
neither need nor risk with regard to a passenger and cargo
capability. Air Force officials could not provide supporting
information sufficient to explain this discrepancy between the
analyses and their proposal. Without sound analyses, the Air Force
may be at risk of spending several billion dollars unnecessarily
for a capability that may not be needed to meet a gap or
shortfall.
Military decision makers approved the passenger and cargo
capability as a requirement although supporting analyses
identified no need or associated risk. Mandatory Air Force
implementing guidance states that senior leaders must use the
documented results of analyses to confirm the identified
capability requirement. The Air Force Requirements for Operational
Capabilities Council validated, and the Chairman of the Joint
Chiefs of Staff's Joint Requirements Oversight Council validated
and approved, KCX tanker proposal with a passenger and cargo
capability. Following the approvals of the oversight councils, DOD
plans to solicit proposals and award a contract for the KC-X
tanker late in fiscal year 2007. At this time, the Under Secretary
of Defense for Acquisition, Technology and Logistics, who
supervises DOD acquisition,^7 must certify, as Milestone Decision
Authority for the proposed tanker acquisition, that, among other
things, the Joint Requirements Oversight Council has accomplished
its statutory duties and that the proposed program is in
compliance with DOD policies and regulations.^8 However, the
absence of analyses identifying a capability gap, shortfall, or
redundancy, and the Joint Requirements Oversight Council approval
of the program without these analyses is contrary to policy and
implementing guidance and could preclude certification of the
program by the Under Secretary. Absent this certification, the
acquisition program for the KC-X tanker cannot begin.^9
In this report, we recommended that the Secretary of Defense
direct the Secretary of the Air Force to accomplish the required
analyses to evaluate the proposed passenger and cargo capability
so as to determine if there is a gap, shortfall, or redundancy,
assess the associated risk, and then submit such documentation to
the Joint Requirements Oversight Council for validation. We also
recommended that, once these analyses are completed, the Secretary
of Defense direct the Chairman, Joint Chiefs of Staff, to formally
notify the Under Secretary of Defense for Acquisition, Technology
and Logistics that such analyses have been completed as required
prior to certification of the program to Congress.
DOD disagreed with our first recommendation to accomplish the
required analyses. In its comments, DOD stated that through the
Joint Capabilities Integration and Development System process, the
Air Force presented analysis and rationale for the passenger and
cargo capability. DOD further stated that its Joint Requirements
Oversight Council and the Air Force concluded that the analysis
was sufficient justification for the capability and the Joint
Requirements Oversight Council validated the requirement. However,
as our report points out, DOD did not perform the required
analyses and failed to identify a gap, shortfall, or redundancy
for the passenger and cargo capability. Considering the
requirement for analyses that separate needs from wants and the
risk of unnecessary expenditures in this multi-year multi-billion
dollar acquisition program, we continue to believe that our
recommendation has merit and that the analyses required by
mandatory guidance are necessary to inform the decision that
begins the acquisition.
DOD agreed with our recommendation to formally notify the Under
Secretary of Defense for Acquisition, Technology and Logistics
once the required analyses have been completed. However, DOD did
not offer assurance that the Air Force would accomplish the
required analyses that evaluate the proposed passenger and cargo
capability as we recommended, and then submit such documentation
to the Joint Requirements Oversight Council for validation. We
believe that the time it could take to accomplish the required
analyses and submit the analyses for revalidation by the Joint
Requirements Oversight Council, could delay the Under Secretary's
certification until just prior to the Milestone B decision, and
may frustrate the congressional oversight that would otherwise be
permitted under section 2366a.^10 We believe that in a program
committing $120 billion over several decades, the review
confirming that needs are justified should occur as far in advance
of program initiation as possible.
In light of the DOD comments on our report, we have put forward a
matter for congressional consideration. Specifically, we are
suggesting that Congress consider requiring
o in addition to the certification described by section 2366a of
title 10, United States Code, the Under Secretary of Defense for
Acquisitions, Technology and Logistics make a specific
certification that the Air Force employed a sound, traceable, and
repeatable process producing analyses that determined if there is
a gap, shortfall, or redundancy and assessed the associated risk
with regard to passenger and cargo capability for the KC-135
Recapitalization, and
o consistent with service policy, these analyses are made
available to the Joint Requirements Oversight Council prior to the
Under Secretary's certification of the program pursuant to section
2366a of title 10, United States Code.
The Air Force intends to replace the fleet of more than 500
tankers and the Mobility Capabilities Study of 2005 set the
requirement for tankers at a range of between 520 to 640 aircraft.
Replacement of this fleet is estimated to cost a minimum of $72
billion. Compared to a refueling aircraft without a passenger and
cargo capability, the inclusion of the capability is estimated,
according to the Analysis of Alternatives done for the KC-X
tanker, to increase costs by 6 percent. The Joint Requirements
Oversight Council approval of the proposal of a replacement tanker
aircraft with the passenger and cargo capability, without an
established need supported by analyses and without an analysis of
risk, could result in an unnecessary expenditure of at least $4.3
billion by our estimates.
In our August 1996 report, U.S. Combat Air Power: Aging Refueling
Aircraft Are Costly to Maintain and Operate, we recommended
consideration of a dual-use aircraft that could conduct both
aerial refueling and airlift operations as a replacement for the
KC-135.^11 We recommended that the Secretary of Defense require
that future studies and analyses of replacement airlift and tanker
aircraft consider accomplishing the missions with a dual-use
aircraft. DOD only partially concurred with this recommendation,
expressing concern at that time about how a dual-use aircraft
would be used and whether one mission area might be degraded to
accomplish the second mission. The lack of analyses done to
support the current proposal still does not give DOD officials
information about how a dual-use aircraft would be used or whether
the primary mission of aerial refueling would be degraded.
Employing Best Practices Can Shape Better Program Outcomes for
DOD�s Airlift Acquisitions
Over the past 25 years, DOD has invested more than $140 billion on
its airlift and tanker forces. Success for acquisitions requires
sound decisions to ensure that program investments are getting
promised returns--on time deliveries to the field, predictable
costs, and sufficient capability. We have reviewed four major
airlift programs and found they did not meet delivery schedules
and were over cost. These programs did not involve huge
technological leaps but presented significant design challenges to
integrate new systems into the older aircraft. A consistent
problem plaguing the programs was an insufficient job of analyzing
the requirements and resources at the programs' outset, a key
systems engineering activity. The divergence between these
programs' experience and best product development practices are
contributing factors to their outcomes.
Outcomes of Certain Airlift Programs
We assessed four airlift programs as part of our annual assessment
of DOD's major acquisition programs and each has experienced cost
growth and schedule delays. Despite being based largely on low
technological risks involving mature systems, these programs have
failed to deliver on the business cases that justified their
initial investment. DOD estimates it will need over $12 billion
between 2007 and 2013 to develop, modify, or procure these
aircraft. The specific airlift programs include
o The Air Force's C-5 Avionics Modernization Program (AMP) is
intended to improve the mission capability rate and transport
capabilities, as well as reduce ownership costs by incorporating
global air traffic management, navigation and safety equipment,
modern digital equipment, and an all-weather flight control
system.
o The Air Force's C-5 Reliability Enhancement and Reengining
Program (RERP) is intended to enhance the reliability,
maintainability, and availability of the C-5 through engine
replacements and modifications to subsystems such as the
electrical and fuel subsystems. The C-5 aircraft will require
installation of the AMP capabilities before the aircraft engines
can be replaced.
o The Air Force's C-130 Avionics Modernization Program (AMP) is
intended to standardize the cockpit configurations and avionics of
different models of C-130 aircraft by providing such things as
communication and navigational system upgrades, terrain avoidance
and warning system, dual flight management systems, and new data
links.
o The C-130J, the latest model of the C-130 aircraft series, is
designed primarily for the transport of cargo and personnel within
a theater of operation. Variants of the C-130J are being acquired
by the Air Force (e.g., Air Mobility Command and Special
Operations Command), Marine Corps, and Coast Guard.
Each of these programs has experienced problems that have impacted
cost and schedule (see table 1). The net effect of the outcomes to
date is that DOD is now paying more to modify or acquire these
systems and the warfighter has had to wait longer than initially
planned before new capability is delivered. For example, the Air
Force now expects by 2011 to have completed the modification of
about 135 fewer C-130 airlift aircraft when compared to its plan 2
years ago.
Table 1: Outcomes of Selected Airlift Programs (as of fiscal year
2006) (FY 2007 dollars in millions)
Outcomes of Certain Airlift Programs
Program
Latest Development Acquisition
Weapon Development Cost Change Unit Cost
Systems Cost Since Start Increases Program schedule delays
C-5 AMP $432.1 17 percent 86 percent Initial operational
capability delayed about
1 year
C-5 RERP $1,342.9 (16 percent) 10 percent Initial operational
capability delayed over
2 years
C-130 AMP $1,627.8 128 percent 43 percent First production
delivery delayed over 2
years
C-130J $262.9 Not applicable 26 percent First production
delivery delayed about 1
1/2 years
Source: GAO analysis of DOD data
Note: C-130J development costs have increased by 2,347 percent but
this includes costs to correct deficiencies and add new
capabilities.
We anticipate there could be additional cost increases and
schedule delays reported in the future. For example, the C-130 AMP
fiscal year 2008 budget indicates that the total program costs
have increased almost $700 million and planned quantities have
been reduced from 434 units to 268 units--nearly doubling the
program acquisition unit costs since December 2005. The program
recently notified Congress of a critical Nunn-McCurdy breach
concerning its unit cost increases.^12 The budget also shows the
Air Force plans to fund the modification of 110 C-5 aircraft with
AMP improvements instead of 59 aircraft as stated in last year's
budget. According to C-5 RERP program officials, total program
costs are expected to increase due to costs with the engine,
pylons, and labor.
Airlift Programs Have Not Captured Critical Product Knowledge
at Key Decision Points
Over the last several years, we have undertaken a body of work
that examines weapon acquisition issues from the perspective that
draws upon lessons learned from best commercial practices for
product development. We have found that a key to successful
product development is the formulation of a business case that
provides demonstrated evidence that (1) the warfighter need exists
and that it can best be met with the chosen concept and (2) the
concept can be developed and produced within existing
resources--including proven technologies, design knowledge,
adequate funding, and adequate time to deliver the product when
needed. The business case is then executed through an acquisition
process that is anchored in knowledge. Leading firms ensure a high
level of knowledge is achieved at key junctures in development,
characterized as knowledge points described below:
o Knowledge point 1: A match must be made between the customer's
needs and the developer's available resources--technology,
engineering knowledge, time, and funding--before a program starts.
o Knowledge point 2: The product's design must be stable and must
meet performance requirements before beginning system
demonstration. This is primarily evidenced by the release of 90
percent of the design drawings by the critical design review and
successful system integration.
o Knowledge point 3: The product must be producible within cost,
schedule, and quality targets and demonstrated to work as intended
before production begins.
There is a synergy in this process, as the attainment of each
successive knowledge point builds on the preceding one. We have
found that if the knowledge based acquisition concept is not
applied, a cascade of negative effects becomes magnified in the
product development and production phases of an acquisition
program leading to cost increases and schedule delays, poor
product quality and reliability, and delays in getting new
capability to the warfighter (see figure 2).
^4Air Force Iinstruction 10-601, Capabilities-Based Requirements
Development, P 1.2 (July 31, 2006).
^5 Deployment orders are issued to deploy specific capabilities as
commitment decisions are made, rather than a deploying unit's full set of
equipment or capabilities. Flexible Deterrent Options (FDOs) provide
escalation options during the initial stages of a conflict. FDOs are
employed under certain conditions to deter adversarial actions contrary to
U.S. interests.
^6 GAO, Defense Transportation: Air Mobility Command Needs to Collect and
Analyze Better Data to Assess Aircraft Utilization. [19]GAO-05-819
(Washington, D.C.: Sept. 29, 2005).
^7 10 U.S.C. S133 (b)(1) (2006).
^8 10 U.S.C. S 2366a S (a)(7) and (a)(10) (2006).
^9 10 U.S.C. S 2366a (a) (2006) and Department of Defense Instruction
5000.2, Operation of the Defense Acquisition System, P 3.7.1.2 (May 12,
2003).
^10 In respect to acquisition programs, milestones are established in DODI
5000.2 and are the points where a recommendation is made and approval is
sought regarding starting or continuing a program into the next phase. In
this instance, the decision at Milestone B is to enter into the system
development and demonstration phase pursuant to guidance prescribed by the
Secretary of Defense and to begin the acquisition program.
^11 GAO, U.S. Combat Air Power: Aging Refueling Aircraft Are Costly to
Maintain and Operate. GAO / NSIAD-96-160, Washington D.C.: (August 8,
1996).
^12 10 U.S.C. S 2433 establishes the requirement for unit cost reports if
certain thresholds for program costs are exceeded (known as unit cost or
Nunn-McCurdy breaches). DOD is required to report to Congress and, if
applicable, certify the program to Congress.
Figure 2: Illustration of Cascading Negative Effects of Failing to Follow
a Knowledge-Based Acquisition Approach
DOD programs often do not capture sufficient knowledge by critical
junctures but decide to move forward regardless. The airlift systems we
reviewed were not immune to this condition and have experienced
unnecessary cost growth and schedule delays as a result. While we do not
have in-depth knowledge on the specific details for these programs, we do
have a broad understanding of the basic underpinnings that led to the
problems. All of the programs were considered low technological risks by
DOD because they planned to rely extensively on proven commercial and
modified off the shelf technology for its new capabilities. However, these
acquisitions have turned out to be more difficult than expected. The
programs did not follow sound systems engineering practices for analyzing
requirements and for ensuring a well integrated design at the right time.
As a result, each program has encountered some difficulty in achieving
design and production maturity as the program moved forward. Some of the
causes to problems encountered include
o Failing to fully analyze the resources needed to integrate
proven commercial technologies and subsystems into a military
system before initiating development.
o Not achieving a stable design before beginning system
demonstration phase resulting in costly design changes and rework.
o Failing to demonstrate the aircraft would work as required
before making large production investments.
In all these instances where appropriate knowledge was not
captured before moving forward, the impact has resulted in a
predictable need for additional resources as shown below in
specific airlift programs.
C-5 AMP
The C-5 AMP entered production without demonstrating that the
system worked as intended and was reliable. The program entered
production just 2 months after flight testing started and ran into
significant design problems while trying to complete development.
Problems uncovered after flight test began required modifications
to the aircraft design which increased by 50 percent the number of
engineering drawings needed for the system. Addressing these
problems delayed the initial operational capability by a year and
contributed to the significant growth in the program's unit costs.
Even today, 4 years after production was initiated, performance
concerns remain with the C-5 AMP. The Director of Operational Test
and Evaluation recently reported that the C-5 AMP is not
operationally suitable because of high component failure rates,
inadequate diagnostics systems, and low reliability rates.
C-5 RERP
The C-5 RERP did not demonstrate design stability before entering
the system demonstration phase which resulted in rework and
schedule delays. At the time the program entered system
demonstration, program officials believed that they had released
90 percent of the design drawings but had not successfully
demonstrated that the subsystems could be integrated onto the C-5
aircraft. During system integration activities the program found
that the "pylon/thrust reverser" had to be redesigned to address
overweight conditions and safety concerns. The program's design
efforts have also been hampered by the fact that its success is
dependent upon the success of the C-5 AMP program. Presently,
according to test officials, the C-5 AMP design is not mature
enough to provide a baseline design for the RERP efforts. These
design issues have contributed to an increase in costs and a
2-year delay in delivering an initial operational capability.
C-130 AMP
The C-130 AMP began development in 2001 without a clear
understanding of the resources needed to integrate proven
commercial technologies into a military system. According to the
program office, they clearly underestimated the complexity of the
engineering efforts that were needed to modify the different
models of the C-130. At the critical design review held in
2005--the point that the design is expected to be stable and ready
to begin the system demonstration phase--the program had not
proven that the subsystems and components could be successfully
integrated into the product.Upon integrating the new avionics into
the test aircraft, program officials realized that it had
significantly underestimated (by 400 percent) the amount of wiring
and the number of harnesses and brackets needed for the
installation. As a result, the design had to be reworked, delaying
the delivery of the test aircraft and increasing costs.
C-130J
The Air Force procured the C-130J without assurances that the
aircraft would work as intended. Program officials believed the
design was mature when procurement began in 1996, largely because
the C-130J evolved from earlier models and was offered as a
commercial item^13. However, the C-130J has encountered numerous
deficiencies that had to be corrected in order to meet the minimum
warfighter requirements delaying the initial aircraft delivery to
the warfighter by about 1.5 years. DOD testing officials still
report performance issues with the aircraft resulting in it being
rated as partially mission capable. The performance issues involve
the aircraft's ability to meet its airdrop operations
requirements, its effectiveness in non-permissive threat
environments, and maintainability issues. Program officials plan
to address the deficiencies as part of a C-130J modernization
effort.
Conclusion
As we said at the beginning, our work shows that acquisition
problems will likely persist until DOD provides a better
foundation for buying the right things the right way. This
involves making tough tradeoff decisions as to which programs
should be pursued, and, more importantly, not pursued, making sure
programs are executable, establishing and locking in needed
requirements before programs are ever started, and making it clear
who is responsible for what and holding people accountable when
these responsibilities are not fulfilled. Recognizing this, DOD
has tried to embrace best practices in its policies, as well as
taking many other actions. However, DOD still has trouble
distinguishing between wants and needs. Because of our concerns
about the analyses done for both the MCS, which has broad
implications for DOD's mobility needs, and the KC-X tanker
requirements, we would urge Congress and other decision makers to
exercise caution when making airlift and tanker investment
decisions.
DOD will continue to face challenges in modernizing its forces
with new demands on the federal dollar created by changing world
conditions. Consequently, it is incumbent upon DOD to find and
adopt best product development practices that can allow it to
manage its weapon system program in the most efficient and
effective way. Success over the long term will depend on following
knowledge-based acquisition practices as well as DOD leadership's
commitment to improving outcomes.
The four acquisition cases we cite in this testimony are not
atypical for all programs. Even with no major technological
invention necessary to meet the warfighters needs in these cases,
acquisition outcomes are not good. There are consequences to these
outcomes. The warfighter does not receive needed capability on
time and the Department and Congress must spend additional
unplanned money to correct mistakes--an expense they can ill
afford. A knowledge-based product development process steeped in
best practices from systems engineering can solve many of these
problems before they start. DOD knows how to do this and, in fact,
informs its acquisition policy with systems engineering rules. It
should redouble its efforts to drive these policies into practice.
^13 In October 2006, the program completed the transition to a
non-commercial negotiated contract.
Mr. Chairman and members of the Subcommittee, this concludes our
prepared statement. We would be pleased to answer any questions
you may have.
Contact and Staff Acknowledgements
For further information about this statement, please contact
William M. Solis at 202-512-8365 or Solisw@gao.gov or Michael
J. Sullivan at 202-512-4841 or Sullivanm@gao.gov . Contact
points for Offices of Congressional Relations and Public Affairs
may be found on the last page of this statement.
GAO staff who made major contributions to this testimony include
Marie Ahearn, Ann Borseth, Cheryl Andrew, Claudia Dickey, Mike
Hazard, Matthew Lea, Oscar Mardis, Sean Merrill, Karen Thornton,
and Steve Woods.
Related GAO Products
Defense Acquisitions: Major Weapon Systems Continue to Experience
Cost and Schedule Problems under DOD's Revised Policy.
[22]GAO-06-368 . Washington, D.C.: April 13, 2006
Defense Acquisitions: Assessments of Selected Major Weapon
Programs. [23]GAO-06-391 . Washington, D.C.:March 31, 2006.
DOD Acquisition Outcomes: A Case for Change. [24]GAO-06-257T .
Washington, D.C.: November 15, 2005.
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. [25]GAO-02-701 . Washington, D.C.:
July 15, 2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices. [26]GAO-02-469T . Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead
to Better Weapon System Outcomes. [27]GAO-01-288 . Washington,
D.C.: March 8, 2001.
Defense Acquisition: Employing Best Practices Can Shape Better
Weapon System Decisions. [28]GAO/T-NSIAD-00-137 . Washington,
D.C.: April 26, 2000.
Best Practices: Better Management of Technology Development Can
Improve Weapon System Outcomes. [29]GAO/NSIAD-99-162 . Washington,
D.C.: July 30, 1999.
Best Practices: Successful Application to Weapon Acquisition
Requires Changes in DOD's Environment. [30]GAO/NSIAD-98-56 .
Washington, D.C.: February 24, 1998.
GAO�s Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of
newly posted products every afternoon, go to www.gao.gov and
select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
fraudnet@gao.gov Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW,
Room 7125 Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
(350999)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
www.gao.gov/cgi-bin/getrpt?GAO-07-566T .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact William M. Solis at (202) 512-8365 or
solisw@gao.gov and MIchael J. Sullivan at (202)-512-4841 or
sullivanm@gao.gov.
Highlights of [38]GAO-07-566T , a testimony to House Committee on Armed
Services, Air and Land Forces Subcommittee
March 7, 2007
DEFENSE ACQUISITIONS
Issues Concerning Airlift and Tanker Programs
The Department of Defense (DOD) has continuing efforts to modernize its
airlift and tanker fleets by investing billions of dollars to modify
legacy airlift systems, such as the C-5 and C-130, and procure new
aircraft, such as a tanker replacement. Acquisition has been on GAO's list
as a high risk area since 1990. GAO has reported that elements
contributing to a sound business case for an acquisition are missing or
incomplete as DOD and the services attempt to acquire new capabilities.
Those elements include firm requirements, mature technologies, a
knowledge-based acquisition strategy, a realistic cost estimate, and
sufficient funding. Acquisition problems that include failure to limit
cost growth, schedule delays, and quantity reductions persist, but fiscal
realities will not allow budgets to accommodate these problems any longer.
Today's testimony addresses (1) the analyses supporting the Department of
Defense's (DOD) mobility capabilities and requirements and (2) actions
that are needed to improve the outcomes of weapon system acquisitions.
For this testimony, GAO drew from issued reports, containing statements of
the scope and methodology used, as well as recently completed work not yet
reported. GAO's work was performed in accordance with generally accepted
government auditing standards.
Past GAO reports, including two recently issued, raise concerns about the
quality of analyses underpinning the programmatic decision-making
surrounding DOD's airlift requirements. In September 2006, GAO issued our
report (GAO-06-938) on DOD's Mobility Capabilities Study (MCS). The MCS
determined that the projected mobility capabilities are adequate to
achieve U.S. objectives with an acceptable level of risk during the period
from fiscal years 2007 through 2013; that is, the current U.S. inventory
of aircraft, ships, prepositioned assets, and other capabilities are
sufficient, in conjunction with host nation support. GAO's report stated
that conclusions of the MCS were based on incomplete data and inadequate
modeling and metrics that did not fully measure stress on the
transportation system. GAO further observed that the MCS results were
incomplete, unclear, or contingent on further study, making it difficult
to identify findings and evaluate evidence. It was not clear how the
analyses done for the study support DOD's conclusions and GAO suggested
that decision makers exercise caution in using the results of this study
to make programmatic decisions. In March 2007, GAO reported (GAO-07-367R)
on the lack of mandatory analyses to support a passenger and cargo
capability for the new replacement refueling aircraft, the KC-X tanker.
Contrary to mandatory Air Force implementing guidance, the Air Force
proposed a capability without analyses identifying an associated gap,
shortfall, or redundancy. GAO believes that without sound analyses, the
Air Force may be at risk of spending several billion dollars unnecessarily
for a capability that may not be needed to meet a gap or shortfall and
made recommendations to the Secretary of Defense that included conducting
the requiring analyses necessary to establish capabilities.
Successful acquisition programs make sound decisions based on critical
product knowledge to ensure that program investments are getting promised
returns--on time delivery, within estimated costs, and with expected
capabilities. However, GAO has shown in its work that DOD practices
diverge from best development practices intended to produce good outcomes
and, as a result, have experienced significant cost growth and schedule
delays. DOD expects to invest over $12 billion in new and improved
capabilities in four airlift programs discussed in this testimony between
now and 2013--C-5 Avionics Modernization Program, C-5 Reliability
Enhancement and Reengining Program, C-130 Avionics Modernization Program,
and the C-130J acquisition program. GAO found that all four programs
failed at basic systems engineering practices to 1) fully analyze the
resources needed to integrate proven commercial technologies, 2) achieve a
stable design before beginning system demonstration, and 3) demonstrate
the aircraft would work as required before making large production
investments.
References
Visible links
17. http://www.gao.gov/cgi-bin/getrpt?GAO-06-938
18. http://www.gao.gov/cgi-bin/getrpt?GAO-07-367R
19. http://www.gao.gov/cgi-bin/getrpt?GAO-05-819
22. http://www.gao.gov/cgi-bin/getrpt?GAO-06-368
23. http://www.gao.gov/cgi-bin/getrpt?GAO-06-391
24. http://www.gao.gov/cgi-bin/getrpt?GAO-06-257T
25. http://www.gao.gov/cgi-bin/getrpt?GAO-02-701
26. http://www.gao.gov/cgi-bin/getrpt?GAO-02-469T
27. http://www.gao.gov/cgi-bin/getrpt?GAO-01-288
28. http://www.gao.gov/cgi-bin/getrpt?GAO/T-NSIAD-00-137
29. http://www.gao.gov/cgi-bin/getrpt?GAO/NSIAD-99-162
30. http://www.gao.gov/cgi-bin/getrpt?GAO/NSIAD-98-56
38. http://www.gao.gov/cgi-bin/getrpt?GAO-07-566T
*** End of document. ***