Defense Acquisitions: Issues Concerning Airlift and Tanker	 
Programs (07-MAR-07, GAO-07-566T).				 
                                                                 
The Department of Defense (DOD) has continuing efforts to	 
modernize its airlift and tanker fleets by investing billions of 
dollars to modify legacy airlift systems, such as the C-5 and	 
C-130, and procure new aircraft, such as a tanker replacement.	 
Acquisition has been on GAO's list as a high risk area since	 
1990. GAO has reported that elements contributing to a sound	 
business case for an acquisition are missing or incomplete as DOD
and the services attempt to acquire new capabilities. Those	 
elements include firm requirements, mature technologies, a	 
knowledge-based acquisition strategy, a realistic cost estimate, 
and sufficient funding. Acquisition problems that include failure
to limit cost growth, schedule delays, and quantity reductions	 
persist, but fiscal realities will not allow budgets to 	 
accommodate these problems any longer. Today's testimony	 
addresses (1) the analyses supporting the Department of Defense's
(DOD) mobility capabilities and requirements and (2) actions that
are needed to improve the outcomes of weapon system acquisitions.
For this testimony, GAO drew from issued reports, containing	 
statements of the scope and methodology used, as well as recently
completed work not yet reported. GAO's work was performed in	 
accordance with generally accepted government auditing standards.
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-566T					        
    ACCNO:   A66545						        
  TITLE:     Defense Acquisitions: Issues Concerning Airlift and      
Tanker Programs 						 
     DATE:   03/07/2007 
  SUBJECT:   Accountability					 
	     Airlift services					 
	     Best practices					 
	     Cost analysis					 
	     Data collection					 
	     Data integrity					 
	     Defense capabilities				 
	     Defense cost control				 
	     Defense procurement				 
	     Military aircraft					 
	     Military systems analysis				 
	     Program evaluation 				 
	     Reporting requirements				 
	     Requirements definition				 
	     Risk assessment					 
	     Schedule slippages 				 
	     Strategic planning 				 
	     Cost estimates					 
	     Cost growth					 
	     C-130 Aircraft					 
	     C-130 Avionics Modernization Program		 
	     C-130J Aircraft					 
	     C-5 Aircraft					 
	     C-5 Avionics Modernization Program 		 
	     C-5 Reliability Enhancement and			 
	     Reengining Program 				 
                                                                 
	     C-X Aircraft					 
	     DOD Mobility Capabilities Study			 
	     Galaxy Aircraft					 
	     GAO High Risk Series				 
	     Hercules Aircraft					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-566T

   

     * [1]Summary
     * [2]Background
     * [3]Analyses Used to Determine Mobility and Tanker Capabilities

          * [4]Mobility Capabilities Study Limitations Raise Questions abou
          * [5]Mandatory Analyses Done to Support Passenger and Cargo Capab

     * [6]Employing Best Practices Can Shape Better Program Outcomes f

          * [7]Outcomes of Certain Airlift Programs
          * [8]Airlift Programs Have Not Captured Critical Product Knowledg

               * [9]C-5 AMP
               * [10]C-5 RERP
               * [11]C-130 AMP
               * [12]C-130J

     * [13]Conclusion
     * [14]Contact and Staff Acknowledgements
     * [15]Related GAO Products

          * [16]Order by Mail or Phone

Testimony

Before the House Committee on Armed Services, Air and Land Forces
Subcommittee

United States Government Accountability Office

GAO

For Release on Delivery Expected at 3:00 p.m. EST
Wednesday, March 7, 2007

DEFENSE ACQUISITIONS

Issues Concerning Airlift and Tanker Programs

Statement of William M. Solis, Director Defense Capabilities and
Management Issues and Michael J. Sullivan, Director Acquisition and
Sourcing Management Issues

GAO-07-566T

Mr. Chairman and Members of the Subcommittee:

We are pleased to be here to discuss the development of aircraft program
requirements and issues related to the acquisition process. We recently
issued our report on high risk areas in the federal government, which
lists DOD's acquisition process as one longstanding area of concern.^1
Acquisition has been on this list since 1990. As we have reported, DOD
knows what to do to achieve more successful outcomes, but finds it
difficult to apply the necessary discipline and controls or assign much
needed accountability. DOD has not been employing a knowledge-based
development approach and business cases have not measured up. DOD has just
begun piloting some corrective actions, so the proof of actual
implementation may be years away. These initiatives also may not
necessarily be applied to programs already under way.

DOD has continuing efforts to modernize its airlift and tanker fleets by
investing billions of dollars to modify legacy airlift systems, such as
the C-5 and C-130, and procure new aircraft, such as the KC-X replacement
tanker. We have reported in the past that a sound business case for an
acquisition contains firm requirements, mature technologies, a
knowledge-based acquisition strategy, a realistic cost estimate, and
sufficient funding. However, we have found many of these elements are
missing or incomplete as DOD and the services attempt to acquire new
capabilities. Persistent acquisition problems include failure to identify
needs versus wants and to limit cost growth, schedule delays, and quantity
reductions, but fiscal realities will not allow budgets to accommodate
these problems any longer.

Today I will highlight for you some issues related to the analyses
supporting the Department of Defense's (DOD) mobility capabilities and
requirements and Mike Sullivan will discuss actions that are needed to
improve the outcomes of weapon system acquisitions. For this testimony, we
drew from issued reports, which contain statements of the scope and
methodology used, as well as recently completed work not yet reported. Our
work was performed in accordance with generally accepted government
auditing standards.

^1 GAO, High Risk Series: An Update, GAO-07-310 (Washington, D.C.: Jan.
2007).

Summary

DOD has an mandate to deliver high-quality products to warfighters, when
they need them and at a price the country can afford. However, our work
shows that acquisition problems will likely persist until DOD provides a
better foundation for buying the right things, the right way. This
foundation begins with setting requirements that are based on adequate and
complete analyses, using current operational data and updated, effective
models. For the past several years, we have reported our concerns with the
analyses done to support requirements and have recently issued two reports
that raise concerns about the quality of analyses underpinning the
programmatic decision-making surrounding DOD's airlift requirements. In
September 2006, we issued our report on DOD's Mobility Capabilities Study
(MCS). The MCS determined that the projected mobility capabilities are
adequate to achieve U.S. objectives with an acceptable level of risk
during the period from fiscal years 2007 through 2013; that is, the
current U.S. inventory of aircraft, ships, prepositioned assets, and other
capabilities are sufficient, in conjunction with host nation support. In
our report, we stated that conclusions of the MCS were based on incomplete
data and inadequate modeling and metrics that did not fully measure stress
on the transportation system.^2 We further observed that, in some cases,
the MCS results were incomplete, unclear, or contingent on further study,
making it difficult to identify findings and evaluate evidence. It is not
clear how the analyses done for the study support DOD's conclusions and we
suggested that decision makers exercise caution in using the results of
this study to make programmatic decisions. This week, we issued a report
on the lack of mandatory analyses to support a passenger and cargo
capability for the new replacement refueling aircraft, the KC-X tanker.^3
Contrary to mandatory Air Force implementing guidance, the Air Force
proposed a capability without an analyses identifying an associated gap,
shortfall, or redundancy. Air Force officials could not provide supporting
information sufficient to explain this discrepancy between the required
analyses and their proposal.

^2 GAO, Defense Transportation: Study Limitations Raise Questions about
the Adequacy and Completeness of the Mobility Capabilities Study and
Report, [17]GAO-06-938 (Washington, D.C.: Sept. 20, 2006).

^3 GAO, Defense Acquisitions: Air Force Decision to Include a Passenger
and Cargo Capability in Its Replacement Refueling Aircraft Was Made
without Required Analyses, [18]GAO-07-367R (Washington, D.C.: Mar. 6,
2007).

Successful acquisition programs make sound decisions based on critical
product knowledge to ensure that program investments are getting promised
returns--on time delivery, within estimated costs, and with expected
capabilities. This is important because DOD expects to invest over $12
billion in new and improved capabilities in four airlift programs
discussed in this testimony between now and 2013--the C-5 Avionics
Modernization Program, C-5 Reliability Enhancement and Reengining Program,
C-130 Avionics Modernization Program, and the C-130J acquisition program.
These four programs have diverged from the best development practices
intended to produce good outcomes and as a result have experienced
significant cost growth and schedule delays. We found that all four
programs failed at basic systems engineering practices to 1) fully
understand the resources needed to integrate proven commercial
technologies, 2) achieve a stable design before beginning system
demonstration, and 3) demonstrate the aircraft would work as required
before making large production investments. As a result, each has
encountered significant delays in delivering capability to the field and
as a group have spent $962.3 million (fiscal year 2007 dollars) more than
planned for development.

Background

DOD must be capable of rapidly deploying armed forces to respond to
contingency and humanitarian operations around the world. Airlift and
tanker aircraft play a vital role in providing this capability. Over the
past 25 years, DOD has invested almost $141 billion to develop, procure,
and modify its airlift and tanker forces with an additional investment
planned for fiscal years 2007 through 2011 of $32 billion. Recent annual
funding levels are at the highest levels in two decades. (See figure 1.)

Figure 1: Planned and Future DOD Investments in Airlift and Tanker Fleets
(1982 to 2011)

Note: Based on DOD's Fiscal Years Defense Program (2007).

In December 2005, DOD issued a report on the study of its mobility
capabilities. The goal of this Mobility Capabilities Study was to identify
and quantify the mobility capabilities needed to support U.S. strategic
objectives into the next decade. The MCS determined that the projected
mobility capabilities are adequate to achieve U.S. objectives with an
acceptable level of risk during the period from fiscal years 2007 through
2013; that is, the current U.S. inventory of aircraft, ships,
prepositioned assets, and other capabilities are sufficient, in
conjunction with host nation support. The MCS emphasized that continued
investment in the mobility system, in line with current departmental
priorities and planned spending, is required to maintain these
capabilities in the future. This includes, for example, fully funding Army
prepositioned assets as planned and completing a planned reengineering of
the C-5 aircraft.

In our previous reports concerning acquisition outcomes and best
practices, we have noted the importance of matching warfighter
requirements with available resources, a responsibility shared by the
requirements and acquisition communities in DOD. As described in Air Force
implementing guidance, there is within DOD a distinct separation between
the requirements authority and acquisition authority.^4 Under this
guidance, this separation requires early and continued collaboration
between both communities.

Analyses Used to Determine Mobility and Tanker Capabilities Were Inadequate

Analyses done for the MCS contained methodological limitations that create
concerns about the adequacy and completeness of the study and decision
makers approving the KC-X tanker proposal lacked required analyses
identifying need and associated risk for a passenger and cargo capability.

Mobility Capabilities Study Limitations Raise Questions about Adequacy and
Completeness

While DOD used an innovative approach in conducting the study and
acknowledged some methodological limitations in its report, it did not
fully disclose how these limitations could affect the MCS conclusions and
recommendations. In September 2006, we reported that DOD's conclusions
were based, in some instances, on incomplete data and inadequate modeling
and metrics that did not fully measure stress on the transportation
system, and that, in some cases, MCS results were incomplete, unclear, or
contingent on further study, making it difficult to identify findings and
evaluate evidence. It is not clear how the analyses done for the study
supported DOD's conclusions, and we suggested that decision makers
exercise caution in using the results of this study to make programmatic
decisions.

As measured against relevant generally accepted research standards, we
identified limitations in the MCS study and report that raise questions.
Among our findings

           o Aspects of modeling and data were inadequate in some areas
           because data were lacking and some of the models used could not
           simulate all relevant aspects of the missions. The report did not
           explain how these limitations could affect the study results or
           what the effect on the projected mobility capabilities might be.
           Relevant research standards require that models used are adequate
           for the intended purpose and represent a complete range of
           conditions, and also that data used are properly generated and
           complete. For example, the MCS modeled hypothetical homeland
           defense missions rather than missions for homeland defense demands
           from a well-defined and approved concept of operations for
           homeland defense because the specific details of the missions were
           still being determined, and DOD acknowledged that the data used
           may be incomplete. The MCS also was unable to model the flexible
           deterrent options/deployment order process to move units and
           equipment into theater due to lack of data, but the study assumed
           a robust use of this process, which in one scenario accounted for
           approximately 60 percent of the airlift prior to beginning combat
           operations.^5 In addition, the MCS report contains more than 80
           references to the need for improved modeling, and 12 of these
           references call for additional data or other refinements.
           Additionally, the MCS modeled the year 2012 to determine the
           transportation capabilities needed for the years 2007 through
           2013. The year 2012 did not place as much demand for mobility
           assets in support of smaller military operations, such as
           peacekeeping, as other years. However, DOD officials considered
           2012--the year modeled--as "most likely" to occur and stated that
           statistically it was not different from other years in the 2007 to
           2013 period even though the number of smaller military operations
           is the least of any of the years reviewed. As I mentioned, we have
           reported before on the lack of data available for analysis that
           could benefit decision makers. In September 2005, we reported that
           the Air Force captured data on short tons transported but did not
           systematically collect and analyze information on operational
           factors, such as weather and runway length, that impact how much
           can be loaded on individual missions.^6 Therefore, Air Force
           officials could not know how often it met its secondary goal to
           use aircraft capacity as efficiently as possible. Without this
           information, Air Mobility Command officials do not know the extent
           to which opportunities exist to use aircraft more efficiently and
           whether operational tempo, cost, and wear and tear on aircraft
           could be reduced. In addition, DOD officials do not have the
           benefit of such analysis to determine future airlift requirements
           for planning purposes.
           o While the MCS concluded that combined U.S. and host nation
           transportation assets were adequate to meet U.S. objectives with
           acceptable risk, the report, in describing the use of warfighting
           metrics in its analyses, does not provide a clear understanding of
           the direct relationship of warfighting objectives to
           transportation capabilities. Acknowledging this point, the report
           stated that further analysis is required to understand the
           operational impact of increased or decreased strategic lift on
           achieving warfighting objectives. Relevant generally accepted
           research standards require that conclusions be supported by
           analyses. The use of warfighting metrics is a measure to determine
           whether combat tasks, such as achieving air superiority, are
           achieved. However, they do not measure whether appropriate
           personnel, supplies, and equipment arrived in accordance with
           timelines. As a result, we could not determine how the study
           concluded that planned transportation assets were adequate because
           the study did not contain a transparent analysis to support its
           conclusion or a clear roadmap in the report to help decision
           makers understand what that conclusion meant in terms of type and
           number of mobility assets needed. Previous DOD mobility studies
           primarily used mobility metrics, which measured success in terms
           of tons of equipment and personnel moved per day to accomplish
           military objectives. The use of both warfighting and mobility
           metrics to measure success would allow decision makers to know
           whether combat tasks were achieved and how much strategic
           transportation is needed to accomplish those tasks.
           o In some cases, the MCS results were incomplete, unclear, or
           contingent on further study, making it difficult to identify
           findings and evaluate evidence. Relevant research standards
           require results to be presented in a complete, accurate, and
           relevant manner. For example, the report contains several
           recommendations for further studies and assessments, five of which
           are under way. However, at the time of our report, DOD had no
           plans to report the effect of these studies on the MCS results
           after the studies are complete. In addition, the report contains
           qualified information that is not presented clearly, such as
           varying assessments of intratheater assets in three different
           places in the report. The lack of clarity and conciseness of the
           reported results can limit the study's usefulness to decision
           makers and stakeholders.

           The MCS report also made recommendations to conduct further
           studies, develop plans and strategies, and improve data collection
           and mobility models. In fact, DOD officials told us at the time
           that a Mobility Capabilities Study-2006 was underway, as well as
           studies on intratheater lift, aerial refueling, and other mobility
           issues. However, unless DOD addresses the concerns I just outlined
           for you, decision makers may be unable to clearly understand the
           operational implications of the study results and make fully
           informed programmatic investment decisions concerning mobility
           capabilities. Also, some of the underlying assumptions used in the
           MCS have now changed significantly, such as the assumption that
           Army prepositioned equipment is in place and fully funded, which
           will no longer be the case. Therefore, the MCS analyses and
           results, which would be the starting point for any new studies,
           may no longer be relevant.

           Mandatory Air Force policy requires Air Force organizations to use
           a formal capabilities-based approach to identify, evaluate,
           develop, field, and sustain capabilities that compete for limited
           resources. Contrary to mandatory Air Force implementing guidance,
           however, the Air Force proposal for a replacement refueling
           aircraft, the KC-X tanker, included a passenger and cargo
           capability without analyses identifying an associated gap,
           shortfall, or redundant capability. According to mandatory Air
           Force implementing guidance, analyses supporting the
           decision-making process should assess a capability based on the
           effects it seeks to generate and the associated operational risk
           of not having it. In this case, the supporting analyses determined
           neither need nor risk with regard to a passenger and cargo
           capability. Air Force officials could not provide supporting
           information sufficient to explain this discrepancy between the
           analyses and their proposal. Without sound analyses, the Air Force
           may be at risk of spending several billion dollars unnecessarily
           for a capability that may not be needed to meet a gap or
           shortfall.

           Military decision makers approved the passenger and cargo
           capability as a requirement although supporting analyses
           identified no need or associated risk. Mandatory Air Force
           implementing guidance states that senior leaders must use the
           documented results of analyses to confirm the identified
           capability requirement. The Air Force Requirements for Operational
           Capabilities Council validated, and the Chairman of the Joint
           Chiefs of Staff's Joint Requirements Oversight Council validated
           and approved, KCX tanker proposal with a passenger and cargo
           capability. Following the approvals of the oversight councils, DOD
           plans to solicit proposals and award a contract for the KC-X
           tanker late in fiscal year 2007. At this time, the Under Secretary
           of Defense for Acquisition, Technology and Logistics, who
           supervises DOD acquisition,^7 must certify, as Milestone Decision
           Authority for the proposed tanker acquisition, that, among other
           things, the Joint Requirements Oversight Council has accomplished
           its statutory duties and that the proposed program is in
           compliance with DOD policies and regulations.^8 However, the
           absence of analyses identifying a capability gap, shortfall, or
           redundancy, and the Joint Requirements Oversight Council approval
           of the program without these analyses is contrary to policy and
           implementing guidance and could preclude certification of the
           program by the Under Secretary. Absent this certification, the
           acquisition program for the KC-X tanker cannot begin.^9

           In this report, we recommended that the Secretary of Defense
           direct the Secretary of the Air Force to accomplish the required
           analyses to evaluate the proposed passenger and cargo capability
           so as to determine if there is a gap, shortfall, or redundancy,
           assess the associated risk, and then submit such documentation to
           the Joint Requirements Oversight Council for validation. We also
           recommended that, once these analyses are completed, the Secretary
           of Defense direct the Chairman, Joint Chiefs of Staff, to formally
           notify the Under Secretary of Defense for Acquisition, Technology
           and Logistics that such analyses have been completed as required
           prior to certification of the program to Congress.

           DOD disagreed with our first recommendation to accomplish the
           required analyses. In its comments, DOD stated that through the
           Joint Capabilities Integration and Development System process, the
           Air Force presented analysis and rationale for the passenger and
           cargo capability. DOD further stated that its Joint Requirements
           Oversight Council and the Air Force concluded that the analysis
           was sufficient justification for the capability and the Joint
           Requirements Oversight Council validated the requirement. However,
           as our report points out, DOD did not perform the required
           analyses and failed to identify a gap, shortfall, or redundancy
           for the passenger and cargo capability. Considering the
           requirement for analyses that separate needs from wants and the
           risk of unnecessary expenditures in this multi-year multi-billion
           dollar acquisition program, we continue to believe that our
           recommendation has merit and that the analyses required by
           mandatory guidance are necessary to inform the decision that
           begins the acquisition.

           DOD agreed with our recommendation to formally notify the Under
           Secretary of Defense for Acquisition, Technology and Logistics
           once the required analyses have been completed. However, DOD did
           not offer assurance that the Air Force would accomplish the
           required analyses that evaluate the proposed passenger and cargo
           capability as we recommended, and then submit such documentation
           to the Joint Requirements Oversight Council for validation. We
           believe that the time it could take to accomplish the required
           analyses and submit the analyses for revalidation by the Joint
           Requirements Oversight Council, could delay the Under Secretary's
           certification until just prior to the Milestone B decision, and
           may frustrate the congressional oversight that would otherwise be
           permitted under section 2366a.^10 We believe that in a program
           committing $120 billion over several decades, the review
           confirming that needs are justified should occur as far in advance
           of program initiation as possible.

           In light of the DOD comments on our report, we have put forward a
           matter for congressional consideration. Specifically, we are
           suggesting that Congress consider requiring

           o in addition to the certification described by section 2366a of
           title 10, United States Code, the Under Secretary of Defense for
           Acquisitions, Technology and Logistics make a specific
           certification that the Air Force employed a sound, traceable, and
           repeatable process producing analyses that determined if there is
           a gap, shortfall, or redundancy and assessed the associated risk
           with regard to passenger and cargo capability for the KC-135
           Recapitalization, and
           o consistent with service policy, these analyses are made
           available to the Joint Requirements Oversight Council prior to the
           Under Secretary's certification of the program pursuant to section
           2366a of title 10, United States Code.

           The Air Force intends to replace the fleet of more than 500
           tankers and the Mobility Capabilities Study of 2005 set the
           requirement for tankers at a range of between 520 to 640 aircraft.
           Replacement of this fleet is estimated to cost a minimum of $72
           billion. Compared to a refueling aircraft without a passenger and
           cargo capability, the inclusion of the capability is estimated,
           according to the Analysis of Alternatives done for the KC-X
           tanker, to increase costs by 6 percent. The Joint Requirements
           Oversight Council approval of the proposal of a replacement tanker
           aircraft with the passenger and cargo capability, without an
           established need supported by analyses and without an analysis of
           risk, could result in an unnecessary expenditure of at least $4.3
           billion by our estimates.

           In our August 1996 report, U.S. Combat Air Power: Aging Refueling
           Aircraft Are Costly to Maintain and Operate, we recommended
           consideration of a dual-use aircraft that could conduct both
           aerial refueling and airlift operations as a replacement for the
           KC-135.^11 We recommended that the Secretary of Defense require
           that future studies and analyses of replacement airlift and tanker
           aircraft consider accomplishing the missions with a dual-use
           aircraft. DOD only partially concurred with this recommendation,
           expressing concern at that time about how a dual-use aircraft
           would be used and whether one mission area might be degraded to
           accomplish the second mission. The lack of analyses done to
           support the current proposal still does not give DOD officials
           information about how a dual-use aircraft would be used or whether
           the primary mission of aerial refueling would be degraded.
			  
			  Employing Best Practices Can Shape Better Program Outcomes for
			  DODï¿½s Airlift Acquisitions

           Over the past 25 years, DOD has invested more than $140 billion on
           its airlift and tanker forces. Success for acquisitions requires
           sound decisions to ensure that program investments are getting
           promised returns--on time deliveries to the field, predictable
           costs, and sufficient capability. We have reviewed four major
           airlift programs and found they did not meet delivery schedules
           and were over cost. These programs did not involve huge
           technological leaps but presented significant design challenges to
           integrate new systems into the older aircraft. A consistent
           problem plaguing the programs was an insufficient job of analyzing
           the requirements and resources at the programs' outset, a key
           systems engineering activity. The divergence between these
           programs' experience and best product development practices are
           contributing factors to their outcomes.
			  
   			  Outcomes of Certain Airlift Programs

           We assessed four airlift programs as part of our annual assessment
           of DOD's major acquisition programs and each has experienced cost
           growth and schedule delays. Despite being based largely on low
           technological risks involving mature systems, these programs have
           failed to deliver on the business cases that justified their
           initial investment. DOD estimates it will need over $12 billion
           between 2007 and 2013 to develop, modify, or procure these
           aircraft. The specific airlift programs include

           o The Air Force's C-5 Avionics Modernization Program (AMP) is
           intended to improve the mission capability rate and transport
           capabilities, as well as reduce ownership costs by incorporating
           global air traffic management, navigation and safety equipment,
           modern digital equipment, and an all-weather flight control
           system.
           o The Air Force's C-5 Reliability Enhancement and Reengining
           Program (RERP) is intended to enhance the reliability,
           maintainability, and availability of the C-5 through engine
           replacements and modifications to subsystems such as the
           electrical and fuel subsystems. The C-5 aircraft will require
           installation of the AMP capabilities before the aircraft engines
           can be replaced.
           o The Air Force's C-130 Avionics Modernization Program (AMP) is
           intended to standardize the cockpit configurations and avionics of
           different models of C-130 aircraft by providing such things as
           communication and navigational system upgrades, terrain avoidance
           and warning system, dual flight management systems, and new data
           links.
           o The C-130J, the latest model of the C-130 aircraft series, is
           designed primarily for the transport of cargo and personnel within
           a theater of operation. Variants of the C-130J are being acquired
           by the Air Force (e.g., Air Mobility Command and Special
           Operations Command), Marine Corps, and Coast Guard.

           Each of these programs has experienced problems that have impacted
           cost and schedule (see table 1). The net effect of the outcomes to
           date is that DOD is now paying more to modify or acquire these
           systems and the warfighter has had to wait longer than initially
           planned before new capability is delivered. For example, the Air
           Force now expects by 2011 to have completed the modification of
           about 135 fewer C-130 airlift aircraft when compared to its plan 2
           years ago.

           Table 1: Outcomes of Selected Airlift Programs (as of fiscal year
           2006) (FY 2007 dollars in millions)
			  
			  Outcomes of Certain Airlift Programs

                                            Program                           
                  Latest    Development Acquisition                           
Weapon    Development    Cost Change   Unit Cost                           
Systems          Cost    Since Start   Increases  Program schedule delays  
C-5 AMP        $432.1     17 percent  86 percent  Initial operational      
                                                     capability delayed about 
                                                     1 year                   
C-5 RERP     $1,342.9   (16 percent)  10 percent  Initial operational      
                                                     capability delayed over  
                                                     2 years                  
C-130 AMP    $1,627.8    128 percent  43 percent  First production         
                                                     delivery delayed over 2  
                                                     years                    
C-130J         $262.9 Not applicable  26 percent  First production         
                                                     delivery delayed about 1 
                                                      1/2 years               

           Source: GAO analysis of DOD data

           Note: C-130J development costs have increased by 2,347 percent but
           this includes costs to correct deficiencies and add new
           capabilities.

           We anticipate there could be additional cost increases and
           schedule delays reported in the future. For example, the C-130 AMP
           fiscal year 2008 budget indicates that the total program costs
           have increased almost $700 million and planned quantities have
           been reduced from 434 units to 268 units--nearly doubling the
           program acquisition unit costs since December 2005. The program
           recently notified Congress of a critical Nunn-McCurdy breach
           concerning its unit cost increases.^12 The budget also shows the
           Air Force plans to fund the modification of 110 C-5 aircraft with
           AMP improvements instead of 59 aircraft as stated in last year's
           budget. According to C-5 RERP program officials, total program
           costs are expected to increase due to costs with the engine,
           pylons, and labor.
			  
     			  Airlift Programs Have Not Captured Critical Product Knowledge
				  at Key Decision Points

           Over the last several years, we have undertaken a body of work
           that examines weapon acquisition issues from the perspective that
           draws upon lessons learned from best commercial practices for
           product development. We have found that a key to successful
           product development is the formulation of a business case that
           provides demonstrated evidence that (1) the warfighter need exists
           and that it can best be met with the chosen concept and (2) the
           concept can be developed and produced within existing
           resources--including proven technologies, design knowledge,
           adequate funding, and adequate time to deliver the product when
           needed. The business case is then executed through an acquisition
           process that is anchored in knowledge. Leading firms ensure a high
           level of knowledge is achieved at key junctures in development,
           characterized as knowledge points described below:

           o Knowledge point 1: A match must be made between the customer's
           needs and the developer's available resources--technology,
           engineering knowledge, time, and funding--before a program starts.
           o Knowledge point 2: The product's design must be stable and must
           meet performance requirements before beginning system
           demonstration. This is primarily evidenced by the release of 90
           percent of the design drawings by the critical design review and
           successful system integration.
           o Knowledge point 3: The product must be producible within cost,
           schedule, and quality targets and demonstrated to work as intended
           before production begins.

           There is a synergy in this process, as the attainment of each
           successive knowledge point builds on the preceding one. We have
           found that if the knowledge based acquisition concept is not
           applied, a cascade of negative effects becomes magnified in the
           product development and production phases of an acquisition
           program leading to cost increases and schedule delays, poor
           product quality and reliability, and delays in getting new
           capability to the warfighter (see figure 2).

^4Air Force Iinstruction 10-601, Capabilities-Based Requirements
Development, P 1.2 (July 31, 2006).

^5 Deployment orders are issued to deploy specific capabilities as
commitment decisions are made, rather than a deploying unit's full set of
equipment or capabilities. Flexible Deterrent Options (FDOs) provide
escalation options during the initial stages of a conflict. FDOs are
employed under certain conditions to deter adversarial actions contrary to
U.S. interests.

^6 GAO, Defense Transportation: Air Mobility Command Needs to Collect and
Analyze Better Data to Assess Aircraft Utilization. [19]GAO-05-819
(Washington, D.C.: Sept. 29, 2005).

^7 10 U.S.C. S133 (b)(1) (2006).

^8 10 U.S.C. S 2366a S (a)(7) and (a)(10) (2006).

^9 10 U.S.C. S 2366a (a) (2006) and Department of Defense Instruction
5000.2, Operation of the Defense Acquisition System, P 3.7.1.2 (May 12,
2003).

^10 In respect to acquisition programs, milestones are established in DODI
5000.2 and are the points where a recommendation is made and approval is
sought regarding starting or continuing a program into the next phase. In
this instance, the decision at Milestone B is to enter into the system
development and demonstration phase pursuant to guidance prescribed by the
Secretary of Defense and to begin the acquisition program.

^11 GAO, U.S. Combat Air Power: Aging Refueling Aircraft Are Costly to
Maintain and Operate. GAO / NSIAD-96-160, Washington D.C.: (August 8,
1996).

^12 10 U.S.C. S 2433 establishes the requirement for unit cost reports if
certain thresholds for program costs are exceeded (known as unit cost or
Nunn-McCurdy breaches). DOD is required to report to Congress and, if
applicable, certify the program to Congress.

Figure 2: Illustration of Cascading Negative Effects of Failing to Follow
a Knowledge-Based Acquisition Approach

DOD programs often do not capture sufficient knowledge by critical
junctures but decide to move forward regardless. The airlift systems we
reviewed were not immune to this condition and have experienced
unnecessary cost growth and schedule delays as a result. While we do not
have in-depth knowledge on the specific details for these programs, we do
have a broad understanding of the basic underpinnings that led to the
problems. All of the programs were considered low technological risks by
DOD because they planned to rely extensively on proven commercial and
modified off the shelf technology for its new capabilities. However, these
acquisitions have turned out to be more difficult than expected. The
programs did not follow sound systems engineering practices for analyzing
requirements and for ensuring a well integrated design at the right time.
As a result, each program has encountered some difficulty in achieving
design and production maturity as the program moved forward. Some of the
causes to problems encountered include

           o Failing to fully analyze the resources needed to integrate
           proven commercial technologies and subsystems into a military
           system before initiating development.
           o Not achieving a stable design before beginning system
           demonstration phase resulting in costly design changes and rework.
           o Failing to demonstrate the aircraft would work as required
           before making large production investments.

           In all these instances where appropriate knowledge was not
           captured before moving forward, the impact has resulted in a
           predictable need for additional resources as shown below in
           specific airlift programs.
			  
			     C-5 AMP

           The C-5 AMP entered production without demonstrating that the
           system worked as intended and was reliable. The program entered
           production just 2 months after flight testing started and ran into
           significant design problems while trying to complete development.
           Problems uncovered after flight test began required modifications
           to the aircraft design which increased by 50 percent the number of
           engineering drawings needed for the system. Addressing these
           problems delayed the initial operational capability by a year and
           contributed to the significant growth in the program's unit costs.
           Even today, 4 years after production was initiated, performance
           concerns remain with the C-5 AMP. The Director of Operational Test
           and Evaluation recently reported that the C-5 AMP is not
           operationally suitable because of high component failure rates,
           inadequate diagnostics systems, and low reliability rates.
			     
				  C-5 RERP

           The C-5 RERP did not demonstrate design stability before entering
           the system demonstration phase which resulted in rework and
           schedule delays. At the time the program entered system
           demonstration, program officials believed that they had released
           90 percent of the design drawings but had not successfully
           demonstrated that the subsystems could be integrated onto the C-5
           aircraft. During system integration activities the program found
           that the "pylon/thrust reverser" had to be redesigned to address
           overweight conditions and safety concerns. The program's design
           efforts have also been hampered by the fact that its success is
           dependent upon the success of the C-5 AMP program. Presently,
           according to test officials, the C-5 AMP design is not mature
           enough to provide a baseline design for the RERP efforts. These
           design issues have contributed to an increase in costs and a
           2-year delay in delivering an initial operational capability.
			  
			     C-130 AMP

           The C-130 AMP began development in 2001 without a clear
           understanding of the resources needed to integrate proven
           commercial technologies into a military system. According to the
           program office, they clearly underestimated the complexity of the
           engineering efforts that were needed to modify the different
           models of the C-130. At the critical design review held in
           2005--the point that the design is expected to be stable and ready
           to begin the system demonstration phase--the program had not
           proven that the subsystems and components could be successfully
           integrated into the product.Upon integrating the new avionics into
           the test aircraft, program officials realized that it had
           significantly underestimated (by 400 percent) the amount of wiring
           and the number of harnesses and brackets needed for the
           installation. As a result, the design had to be reworked, delaying
           the delivery of the test aircraft and increasing costs.
			  
			     C-130J

           The Air Force procured the C-130J without assurances that the
           aircraft would work as intended. Program officials believed the
           design was mature when procurement began in 1996, largely because
           the C-130J evolved from earlier models and was offered as a
           commercial item^13. However, the C-130J has encountered numerous
           deficiencies that had to be corrected in order to meet the minimum
           warfighter requirements delaying the initial aircraft delivery to
           the warfighter by about 1.5 years. DOD testing officials still
           report performance issues with the aircraft resulting in it being
           rated as partially mission capable. The performance issues involve
           the aircraft's ability to meet its airdrop operations
           requirements, its effectiveness in non-permissive threat
           environments, and maintainability issues. Program officials plan
           to address the deficiencies as part of a C-130J modernization
           effort.
			  
			  Conclusion

           As we said at the beginning, our work shows that acquisition
           problems will likely persist until DOD provides a better
           foundation for buying the right things the right way. This
           involves making tough tradeoff decisions as to which programs
           should be pursued, and, more importantly, not pursued, making sure
           programs are executable, establishing and locking in needed
           requirements before programs are ever started, and making it clear
           who is responsible for what and holding people accountable when
           these responsibilities are not fulfilled. Recognizing this, DOD
           has tried to embrace best practices in its policies, as well as
           taking many other actions. However, DOD still has trouble
           distinguishing between wants and needs. Because of our concerns
           about the analyses done for both the MCS, which has broad
           implications for DOD's mobility needs, and the KC-X tanker
           requirements, we would urge Congress and other decision makers to
           exercise caution when making airlift and tanker investment
           decisions.

           DOD will continue to face challenges in modernizing its forces
           with new demands on the federal dollar created by changing world
           conditions. Consequently, it is incumbent upon DOD to find and
           adopt best product development practices that can allow it to
           manage its weapon system program in the most efficient and
           effective way. Success over the long term will depend on following
           knowledge-based acquisition practices as well as DOD leadership's
           commitment to improving outcomes.

           The four acquisition cases we cite in this testimony are not
           atypical for all programs. Even with no major technological
           invention necessary to meet the warfighters needs in these cases,
           acquisition outcomes are not good. There are consequences to these
           outcomes. The warfighter does not receive needed capability on
           time and the Department and Congress must spend additional
           unplanned money to correct mistakes--an expense they can ill
           afford. A knowledge-based product development process steeped in
           best practices from systems engineering can solve many of these
           problems before they start. DOD knows how to do this and, in fact,
           informs its acquisition policy with systems engineering rules. It
           should redouble its efforts to drive these policies into practice.
			  
^13 In October 2006, the program completed the transition to a
non-commercial negotiated contract.

           Mr. Chairman and members of the Subcommittee, this concludes our
           prepared statement. We would be pleased to answer any questions
           you may have.
			  
			  Contact and Staff Acknowledgements

           For further information about this statement, please contact
           William M. Solis at 202-512-8365 or [email protected] or Michael
           J. Sullivan at 202-512-4841 or [email protected] . Contact
           points for Offices of Congressional Relations and Public Affairs
           may be found on the last page of this statement.

           GAO staff who made major contributions to this testimony include
           Marie Ahearn, Ann Borseth, Cheryl Andrew, Claudia Dickey, Mike
           Hazard, Matthew Lea, Oscar Mardis, Sean Merrill, Karen Thornton,
           and Steve Woods.
			  
			  Related GAO Products

           Defense Acquisitions: Major Weapon Systems Continue to Experience
           Cost and Schedule Problems under DOD's Revised Policy.
           [22]GAO-06-368 . Washington, D.C.: April 13, 2006

           Defense Acquisitions: Assessments of Selected Major Weapon
           Programs. [23]GAO-06-391 . Washington, D.C.:March 31, 2006.

           DOD Acquisition Outcomes: A Case for Change. [24]GAO-06-257T .
           Washington, D.C.: November 15, 2005.

           Best Practices: Capturing Design and Manufacturing Knowledge Early
           Improves Acquisition Outcomes. [25]GAO-02-701 . Washington, D.C.:
           July 15, 2002.

           Defense Acquisitions: DOD Faces Challenges in Implementing Best
           Practices. [26]GAO-02-469T . Washington, D.C.: February 27, 2002.

           Best Practices: Better Matching of Needs and Resources Will Lead
           to Better Weapon System Outcomes. [27]GAO-01-288 . Washington,
           D.C.: March 8, 2001.

           Defense Acquisition: Employing Best Practices Can Shape Better
           Weapon System Decisions. [28]GAO/T-NSIAD-00-137 . Washington,
           D.C.: April 26, 2000.

           Best Practices: Better Management of Technology Development Can
           Improve Weapon System Outcomes. [29]GAO/NSIAD-99-162 . Washington,
           D.C.: July 30, 1999.

           Best Practices: Successful Application to Weapon Acquisition
           Requires Changes in DOD's Environment. [30]GAO/NSIAD-98-56 .
           Washington, D.C.: February 24, 1998.
			  
			  GAOï¿½s Mission

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.
			  
			  Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each
           weekday, GAO posts newly released reports, testimony, and
           correspondence on its Web site. To have GAO e-mail you a list of
           newly posted products every afternoon, go to www.gao.gov and
           select "Subscribe to Updates."
			  
			  Order by Mail or Phone

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061
			  
			  To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470
			  
			  Congressional Relations

           Gloria Jarmon, Managing Director, [email protected] (202)
           512-4400 U.S. Government Accountability Office, 441 G Street NW,
           Room 7125 Washington, D.C. 20548
			  
			  Public Affairs

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

(350999)

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

www.gao.gov/cgi-bin/getrpt?GAO-07-566T .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact William M. Solis at (202) 512-8365 or
[email protected] and MIchael J. Sullivan at (202)-512-4841 or
[email protected].

Highlights of [38]GAO-07-566T , a testimony to House Committee on Armed
Services, Air and Land Forces Subcommittee

March 7, 2007

DEFENSE ACQUISITIONS

Issues Concerning Airlift and Tanker Programs

The Department of Defense (DOD) has continuing efforts to modernize its
airlift and tanker fleets by investing billions of dollars to modify
legacy airlift systems, such as the C-5 and C-130, and procure new
aircraft, such as a tanker replacement. Acquisition has been on GAO's list
as a high risk area since 1990. GAO has reported that elements
contributing to a sound business case for an acquisition are missing or
incomplete as DOD and the services attempt to acquire new capabilities.
Those elements include firm requirements, mature technologies, a
knowledge-based acquisition strategy, a realistic cost estimate, and
sufficient funding. Acquisition problems that include failure to limit
cost growth, schedule delays, and quantity reductions persist, but fiscal
realities will not allow budgets to accommodate these problems any longer.

Today's testimony addresses (1) the analyses supporting the Department of
Defense's (DOD) mobility capabilities and requirements and (2) actions
that are needed to improve the outcomes of weapon system acquisitions.

For this testimony, GAO drew from issued reports, containing statements of
the scope and methodology used, as well as recently completed work not yet
reported. GAO's work was performed in accordance with generally accepted
government auditing standards.

Past GAO reports, including two recently issued, raise concerns about the
quality of analyses underpinning the programmatic decision-making
surrounding DOD's airlift requirements. In September 2006, GAO issued our
report (GAO-06-938) on DOD's Mobility Capabilities Study (MCS). The MCS
determined that the projected mobility capabilities are adequate to
achieve U.S. objectives with an acceptable level of risk during the period
from fiscal years 2007 through 2013; that is, the current U.S. inventory
of aircraft, ships, prepositioned assets, and other capabilities are
sufficient, in conjunction with host nation support. GAO's report stated
that conclusions of the MCS were based on incomplete data and inadequate
modeling and metrics that did not fully measure stress on the
transportation system. GAO further observed that the MCS results were
incomplete, unclear, or contingent on further study, making it difficult
to identify findings and evaluate evidence. It was not clear how the
analyses done for the study support DOD's conclusions and GAO suggested
that decision makers exercise caution in using the results of this study
to make programmatic decisions. In March 2007, GAO reported (GAO-07-367R)
on the lack of mandatory analyses to support a passenger and cargo
capability for the new replacement refueling aircraft, the KC-X tanker.
Contrary to mandatory Air Force implementing guidance, the Air Force
proposed a capability without analyses identifying an associated gap,
shortfall, or redundancy. GAO believes that without sound analyses, the
Air Force may be at risk of spending several billion dollars unnecessarily
for a capability that may not be needed to meet a gap or shortfall and
made recommendations to the Secretary of Defense that included conducting
the requiring analyses necessary to establish capabilities.

Successful acquisition programs make sound decisions based on critical
product knowledge to ensure that program investments are getting promised
returns--on time delivery, within estimated costs, and with expected
capabilities. However, GAO has shown in its work that DOD practices
diverge from best development practices intended to produce good outcomes
and, as a result, have experienced significant cost growth and schedule
delays. DOD expects to invest over $12 billion in new and improved
capabilities in four airlift programs discussed in this testimony between
now and 2013--C-5 Avionics Modernization Program, C-5 Reliability
Enhancement and Reengining Program, C-130 Avionics Modernization Program,
and the C-130J acquisition program. GAO found that all four programs
failed at basic systems engineering practices to 1) fully analyze the
resources needed to integrate proven commercial technologies, 2) achieve a
stable design before beginning system demonstration, and 3) demonstrate
the aircraft would work as required before making large production
investments.

References

Visible links

  17. http://www.gao.gov/cgi-bin/getrpt?GAO-06-938
  18. http://www.gao.gov/cgi-bin/getrpt?GAO-07-367R
  19. http://www.gao.gov/cgi-bin/getrpt?GAO-05-819
  22. http://www.gao.gov/cgi-bin/getrpt?GAO-06-368
  23. http://www.gao.gov/cgi-bin/getrpt?GAO-06-391
  24. http://www.gao.gov/cgi-bin/getrpt?GAO-06-257T
  25. http://www.gao.gov/cgi-bin/getrpt?GAO-02-701
  26. http://www.gao.gov/cgi-bin/getrpt?GAO-02-469T
  27. http://www.gao.gov/cgi-bin/getrpt?GAO-01-288
  28. http://www.gao.gov/cgi-bin/getrpt?GAO/T-NSIAD-00-137
  29. http://www.gao.gov/cgi-bin/getrpt?GAO/NSIAD-99-162
  30. http://www.gao.gov/cgi-bin/getrpt?GAO/NSIAD-98-56
  38. http://www.gao.gov/cgi-bin/getrpt?GAO-07-566T
*** End of document. ***