Defense Acquisitions: Changes in E-10A Acquisition Strategy	 
Needed before Development Starts (15-MAR-05, GAO-05-273).	 
                                                                 
The Air Force is on the verge of making a major commitment to the
multi-billion dollar E-10A Multi-sensor Command and Control	 
Aircraft program. Due to the substantial investment needed and	 
technological challenges in developing the aircraft, the	 
Subcommittee on Tactical Air and Land Forces asked GAO to examine
the soundness of the E-10A business case as well as the risks	 
associated with the current acquisition strategy.		 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-05-273 					        
    ACCNO:   A19358						        
  TITLE:     Defense Acquisitions: Changes in E-10A Acquisition       
Strategy Needed before Development Starts			 
     DATE:   03/15/2005 
  SUBJECT:   Financial analysis 				 
	     Military aircraft					 
	     Military research and development			 
	     Performance measures				 
	     Program management 				 
	     Weapons research and development			 
	     Air Force procurement				 
	     Procurement planning				 
	     Operational testing				 
	     Program goals or objectives			 
	     Program implementation				 
	     Air Force E-10A Multi-Sensor Command and		 
	     Control Aircraft					 
                                                                 
	     C-17 Aircraft					 
	     F/A-22 Aircraft					 
	     Globemaster Aircraft				 
	     Joint Surveillance Target Attack Radar		 
	     System						 
                                                                 
	     Osprey Aircraft					 
	     Patriot PAC-3					 
	     V-22 Aircraft					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-05-273

United States Government Accountability Office

GAO	Report to the Chairman, Subcommittee on Tactical Air and Land Forces,

             Committee on Armed Services, House of Representatives

March 2005

DEFENSE ACQUISITIONS

     Changes in E-10A Acquisition Strategy Needed before Development Starts

                                       a

GAO-05-273

[IMG]

March 2005

DEFENSE ACQUISITIONS

Changes in E-10A Acquisition Strategy Needed before Development Starts

                                 What GAO Found

As the E-10A Multi-sensor Command and Control Aircraft program nears its
official starting point, questions remain regarding critical elements of
its business case, including the need for the aircraft, the maturity level
of its technology, and its funding. Plans call for the E-10A to couple a
new radar system with a sophisticated and software intensive battle
management command and control system aboard a Boeing 767. E-10A is
planned to fill a current gap in U.S. capabilities and provide a defense
against weapons such as cruise missiles. The Office of the Secretary of
Defense is still working on a study to determine whether the E-10A program
is the most cost-effective way to fill that gap. E-10A program funding
plans changed dramatically in December 2004 when the DOD proposed reducing
the total program budget by about 45 percent for the next 2 fiscal years.
The business case for starting a development program requires demonstrated
evidence that (1) the warfighter need exists and that it can best be met
with the chosen concept and (2) the concept can be developed and produced
within existing resources--including design knowledge, demonstrated
technologies, adequate funding, and adequate time to deliver the product.
E-10A requirements and resources are still in flux.

GAO found risks associated with the current E-10A acquisition strategy
that could lead to costly changes later in the program. The program is set
to move into production before critical knowledge is acquired. For
example, the first fully assembled E-10A, outfitted with its radar and
battle management command and control systems, would not be delivered in
time to complete testing before the decision is made to begin production.
Testing and production are scheduled to start at the same time in 2010.
Furthermore, four of six E-10As are scheduled to begin production before
the results of testing are available. By not demonstrating that the system
can perform as expected before entering production, the program increases
the risk of changes and delays later in the program. This strategy
requires significant concurrency among the technology development, product
development, and production phases.

E-10A Acquisition Strategy First fully integrated E-10A ready to begin
reliability testing

Program Critical Production Start design review decision

                    Source: GAO analysis of Air Force data.

                 United States Government Accountability Office

Contents

     Letter                                                                 1 
                                    Results in Brief                        2 
                                       Background                           3 
               Uncertainties Exist in E-10A Business Case as It Approaches 
                                                                       Its 
                                     Decision Point                         5 
                      E-10A Acquisition Strategy Does Not Capture Critical 
                                                                 Knowledge 
                             before Program Decision Points                10 
                                       Conclusions                         13 
                          Recommendations for Executive Action             14 
                           Agency Comments and Our Evaluation              14 
Appendix I                     Scope and Methodology                    
Appendix II           Comments from the Department of Defense           

  Figures

Figure 1: Comparison of E-10A Strategy and Best Practices Model 11 Figure
2: E-10A Test and Production Schedule 12

Abbreviations

DOD Department of Defense
OSD Office of the Secretary of Defense

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office Washington, DC 20548

March 15, 2005

The Honorable Curt Weldon
Chairman
Subcommittee on Tactical Air and Land Forces
Committee on Armed Services
House of Representatives

Dear Mr. Chairman:

In April 2005, the Air Force plans to make a major commitment to the
E-10A Multi-sensor Command and Control Aircraft (E-10A) program and
officially begin its development and demonstration phase. The program,
estimated to cost over $7.3 billion through fiscal year 2013, is being
designed to fill a gap in U. S. capabilities and provide a defense against
weapons such as cruise missiles. It is also being designed to be an
airborne
battle management platform capable of directing forces to respond to
moving targets in the air and on the ground. Given the technical
challenges
and significant investment associated with this aircraft, you requested
that
we review the E-10A program to determine whether its business case is
clearly defined and its acquisition process is well executed. This report
examines (1) the soundness of the Air Force's business case1 including
E-10A requirements, technologies, and cost; and (2) the risks associated
with the planned acquisition strategy.

Because the E-10A is not officially a weapons system acquisition program
until the Milestone B2 decision has been approved, the Air Force
considered key elements of the business case as pre-decisional and subject
to change. Therefore, we did not have the opportunity to review all

1 The business case is defined as demonstrated evidence that (1) the
warfighter need exists and that it can best be met with the chosen
concept, and (2) the concept can be developed and produced within existing
resources-including design knowledge, demonstrated technologies, adequate
funding, and adequate time to deliver the product.

2Milestone B is the entrance point for the system development and
demonstration phase, which is considered the initiation of a major defense
systems acquisition program. It requires authorization by the Milestone
Decision Authority based on advice from seniorlevel advisors on the
Defense Acquisition Board. The Defense Acquisition Board meeting to
discuss the E-10A Milestone B is scheduled for April 12, 2005. Department
of Defense Instruction 5000.2, Operation of the Defense Acquisition
System, Paragraph 3.7, System Development and Demonstration, Subparagraphs
3.7.1.2 and 3.7.2.3.

elements related to the critical E-10A technology assessments, the cost
estimate, or the funding profile. The Air Force did provide, however, a
technology readiness assessment of the radar subsystem, the planned
overall acquisition strategy, and the analysis of alternatives supporting
the selection of the specific E-10A aircraft platform. Despite these
limitations, we were able to assess the relative state of the E-10A
business case as compared to the expectations of best practices. We did
this by drawing on other information available that indicated the current
instability of estimated cost and funding and that questioned the maturity
of the technology associated with a key subsystem. We conducted this
review from January 2004 to January 2005 in accordance with generally
accepted government auditing standards. More details about our methodology
are in appendix I.

                                Results in Brief

Questions remain in the E-10A business case over it being the most
cost-effective solution as well as its technology maturity, cost, and
funding. Officials in the Office of the Secretary of Defense (OSD) are
still studying E-10A alternatives to determine if it is the most
cost-effective way to satisfy the planned mission to identify, track, and
target time-sensitive threats like cruise missiles.3 The study is also
assessing on board command and control needs. Current plans are for 25
crew to carry out mission requirements and 2 crew to fly the aircraft. The
OSD expects to present the results of the study by March 2005. We found
that radar technologies are ready for system development, but because the
readiness assessment of the battle management technologies was not
finalized, there was no evidence that these technologies were sufficiently
mature. Review organizations within the Air Force and OSD are examining
Air Force assessments of technology maturity and costs as well as the
annual allocation of funds needed to develop and produce the E-10A. We
were not able to review these assessments because they were not yet
complete. Program officials also stated that the December 2004 reduction
of $600 million in E-10A funding proposed by OSD will require significant
changes to the business case because it would reduce planned funding for
the total program by about 45 percent in fiscal years 2006 and 2007. The
Air Force is determining how this will impact the program schedule and
costs as it will likely cause them to truncate key activities in these
years. An OSD

3 A time-sensitive target is a target of such high priority that it
requires an immediate response, either because it poses a danger to
friendly forces or it is a highly lucrative, fleeting target of
opportunity.

official said the reduction will likely slow development of the E-10A
airframe activities because the available funds will need to be applied to
the radar improvement program that is also supporting the Global Hawk
program.

The Air Force acquisition strategy for acquiring the E-10A calls for
moving through development and into production before critical knowledge
is captured about design, manufacturing, and reliability. For example, the
strategy does not allow for adequate product integration and prototype
demonstration to ensure the design is stable at the critical design review
stage. Additionally, a fully integrated prototype-a working model of the
E-10A with the radar system and command and control computer system
working together-will not be delivered in time to allow testing prior to
the production decision. Both testing and production are scheduled to
concurrently start in 2010. By not demonstrating the system can perform as
expected before entering production, the program risks costly design
changes and delays later in the program. Furthermore, four of six E-10As
are scheduled to begin production before testing is completed. We have
found this to be consistently a high-risk acquisition approach in our past
reviews of Department of Defense (DOD) acquisitions.

We are making recommendations to the Secretary of Defense to ensure that
the open business case questions are answered before starting the E-10A
program. We also recommend that the Secretary direct the Air Force to
revise the acquisition strategy to ensure sufficient time to integrate and
demonstrate the design in flight testing of an E-10A prototype before
moving the program into production. DOD concurred with our first
recommendation, but noted that the open questions will not be fully
resolved until the Milestone B decision, which could be delayed until 2010
due to recent budget reductions. DOD partially concurred with our second
recommendation. It stated that DOD policy did not require integration and
demonstration of a design prior to critical design review. We disagree
with this interpretation because DOD policy requires a demonstration of
the integrated prototype prior to entering demonstration. Nevertheless,
DOD acknowledged that it is restructuring the program to demonstrate the
key technologies in a prototype prior to starting system development and
demonstration.

Background 	The E-10A program comprises three primary elements: the
aircraft, radar, and battle management command and control subsystem. The
aircraft is a Boeing 767-400ER, the largest 767 variant Boeing makes. The
Air Force has only contracted for one aircraft to date, because a final
decision on the

operational platform has not been made. This aircraft is a commercial
product that will be modified for military use and used as a testbed. At
this time there is only 1 unfilled order for the 400 model in the Boeing
assembly line and 25 other unfilled orders for other smaller 767 models.4
If the Boeing production line were to close down before the Air Force is
positioned to make a production decision on the E-10A it would have to
find an alternative. Alternatives could include a different aircraft type
or model or the purchase of 767-400ER aircraft from commercial airline
companies.

The radar planned for the E-10A began development in 1997 as a response to
the growing concern about cruise missile proliferation. Initially, it was
intended to upgrade the radar on the Joint Surveillance Target Attack
Radar System (Joint STARS). The upgraded radar was to have advanced sensor
technology, providing air-to-air capability for cruise missile defense and
significant increases in ground surveillance capability. Shortly after the
program began development, the Air Force restructured the program to
develop a modular, scalable radar suitable for use on a variety of
airborne platforms. OSD approved the development of the multiple platform
radar in 2003. It is being designed for inclusion on the Global Hawk and
E-10A programs.

The Air Force began evaluating the need to improve its airborne battle
management command and control capabilities in 2002. The planned E-10A
battle management command and control subsystem is software intensive and
intended to enable the E-10A to process and display sensor data from the
radar and eventually from off board sensors so that the onboard crew can
take actions against time sensitive targets. The Air Force issued a
contract in September 2004 to begin preliminary design efforts for this
subsystem.

We have a body of work focused on best practices in product development
and weapon systems acquisition.5 This work has found that key to success
is formulation of a business case that matches product requirements to

4 Source: Boeing Website, http://www.boeing.com.

5 GAO, Defense Acquisitions: DOD's Assessments of Major Weapon Programs,
GAO-04-248 (Washington, D.C.: Mar. 31, 2004). Other recent reports
discussing best practices include GAO, Best Practices: Capturing Design
and Manufacturing Knowledge Early Improves Acquisition Outcomes,
GAO-02-701 (Washington, D.C.: July 15, 2002) and Defense Acquisitions: DOD
Faces Challenges in Implementing Best Practices, GAO-02-469T (Washington,
D.C.: Feb. 27, 2002).

available resources-proven technologies, sufficient engineering
capabilities, time, and funding. Several basic factors are critical to
establishing a sound business case for undertaking a new product
development. First, the needs of the party seeking the new product, the
user, must be accurately defined, alternative approaches to satisfying
these needs properly analyzed, and quantities needed for the chosen system
must be well understood. The developed product must be producible at a
cost that matches the users' expectations and budgetary resources.
Finally, the developer must have the resources to design and deliver the
product with the features that the customer wants when it is needed. If
the financial, material and intellectual resources to develop the product
properly are not available, development does not go forward. Additionally,
an evolutionary and knowledge-based acquisition strategy that captures
critical knowledge before key decision points in the program is needed to
execute the business plan. This calls for a realistic assessment of risks
and costs; doing otherwise undermines the intent of the business case and
invites failure. Ultimately, preserving the business case and attaining
critical knowledge in time for decisions strengthens the ability of
managers to say "no" to pressures to accept high risks or unknowns.

If best practices are not followed, we have found a cascade of negative
effects becomes magnified in the product development and production phases
of an acquisition program. These have led to acquisition outcomes that
included significant cost increases and schedule delays, poor product
quality and reliability, and delays in getting the new capability to the
warfigher. These outcomes have been demonstrated in other programs such as
the F/A-22 fighter, C-17 airlifter, V-22 tiltrotor aircraft, PAC-3
missile, and others.

Questions remain as the Air Force develops the E-10A program's business
case to support the decision to begin development in April 2005. The DOD
has identified a need for a cruise missile defense capability and the Air
Force has selected the E-10A to meet this need. There are, however,
unanswered questions in both the requirement and resource elements of the
E-10A business case. OSD is still studying whether the E-10A is the most
cost-effective alternative for the cruise missile requirement and the
extent of battle management command and control needed on board to satisfy
the intended need. Finally, assessments of the technology maturity,
estimated costs, and funding availability are still in process.

  Uncertainties Exist in E-10A Business Case as It Approaches Its Decision Point

    OSD Is Still Studying the Need for the E-10A

OSD officials from the Program Analysis and Evaluation Directorate are not
satisfied that the studies done by the Air Force to select the E-10A
sufficiently analyzed alternative systems.6 As a result, they are
reviewing alternative systems and attempting to determine the most
cost-effective solution to satisfy the warfighter's needs. OSD officials
agree that the E10A could provide an increased capability in identifying
and tracking ground moving and time-sensitive targets. However, they
believe that if there are less costly systems that can provide similar
capabilities, it could be more cost-effective to buy those systems. The
Air Force began efforts in 1997 to develop a radar sensor that would
detect cruise missiles as part of the Joint STARS program. The Air Force
examined different size and power combinations for the radar and which
platforms had the capacity to carry the radar and still perform multiple
missions. These analyses assumed that only manned airborne platforms could
meet these requirements. The Air Force completed a formal analysis of
alternatives in February 2002 of different possible host platforms for the
radar. The study indicated that other aircraft could meet many of the
requirements but were based on older commercial technology that was less
efficient to operate. The Air Force analysis concluded that the Boeing
767-400ER was the optimal choice given the future multi-mission purpose of
the system, and the size, weight, and performance requirements of the
radar.

OSD officials are also uncertain about the degree of battle management
command and control capability needed onboard the E-10A versus
transmitting the information gathered by the E-10A to other command and
control centers. According to the Air Force, the need for an onboard
capability is driven by the large amounts of data that would be collected
and analyzed, the limited bandwidth to transmit the data, and the need to
have line-of-sight communications for time-sensitive targeting,
particularly against cruise missiles. OSD officials said they are looking
at whether the battle management subsystem has to be part of the E-10A
platform to meet the timelines identified by the Air Force. They expect to
present their results by March 2005. Air Force officials told us that some
of the battle management functions are currently performed by ground
units, but these ground units cannot adequately respond to real-time
events involving

6 Both the Air Force and the Joint Theater Air and Missile Defense
Organization conducted analyses to assess future mixtures of systems,
including the E-10A. Both studies determined that if the E-10A performs as
expected it could significantly increase the capability of the
warfighters. However, the Air Force study focused only on ground-moving
target indication and the Joint Theater Air and Missile Defense
Organization focused primarily on cruise missile defense.

moving targets like cruise missiles. The E-10A's primary function will be
battle management command and control of cruise missile detection and
time-sensitive targeting activities. As a result, its battle management
capabilities will be tailored to support those functions. These
capabilities were validated in October 2004 by the Joint Requirements
Oversight Council in preparation for the program's upcoming Milestone B
decision.

To provide these capabilities, an onboard crew will be required. The
current E-10A crew size is estimated at 27 staff-2 flight crew, 21 mission
operators, and 4 technicians. According to the Air Force, the crew size
could change depending on the mission and the degree of automation on the
system. However, the Air Force has not performed any incremental analysis
to show crew size for individual specific missions, such as doing cruise
missile defense only.

    Assessments of Needed Resources to Develop and Produce the E-10A Are
    Incomplete

Status of Technologies

To date, the Air Force has not identified sufficient or available
resources to meet the warfighter's requirements and to start the
development program. The Air Force program office has completed its
assessments of E-10A critical technologies, cost estimates, and funding
needs but these assessments are being reviewed by OSD. While some
resources will meet the requirements, others are either unproven or in a
state of flux. Radar development started under a separate program, the
Radar Technology Insertion Program, and most radar technologies were
reported as mature. Because the Air Force did not provide GAO its
technical assessment of the battle management command and control system
critical technologies, we consider the maturity levels unproven, even
though program officials told us these technologies meet minimum maturity
standards. In addition to technologies, the financial resources for the
program are in a state of flux. The E-10A cost estimate for development
and production is still a work in process and funding was recently reduced
by $600 million for fiscal years 2006 and 2007, which according to DOD
officials will substantially impact the program.

Most radar technologies are at a high level of maturity, but evidence was
not provided to support stated maturity levels of the battle management
command and control subsystem. The Air Force assessed radar technologies
prior to the October 2003 start of the Radar Technology Insertion Program.
The critical technologies identified in the radar improvement program
included the radar architecture, modes, receiver/exciter, and signal
processor among others. Of the nine technologies identified, six were
assessed as mature to our best practice standard; the remaining three were
one level below the best practice

requirement for mature technologies, a level DOD policy states is
sufficient to begin development. These three technologies are the pulse
compression unit, the structure, and the modes. Since the 2003 radar
technology assessment, the radar improvement program completed its final
design review in June 2004. Numerous tests have been conducted on
small-scale radar prototypes to mitigate program risks. These tests
electronically drove a signal through the radar, demonstrating the basic
functionality of the design. However, the radar subsystem being designed
for the E-10A has demonstrated neither form nor fit, nor has it been
integrated on the aircraft platform. Although the integration process is
an inherently high-risk endeavor, Air Force officials stated they have a
process in place to manage these risks. The actual size of the E-10A's
radar will be significantly larger than the tested prototype and will
require the E-10A testbed aircraft in order to complete the demonstration
currently scheduled to occur in 2010. The process of scaling the radar to
the appropriate size and ensuring that all the individual modules work
together has yet to be accomplished. Recognizing this, program officials
have identified the integration of the radar as a critical technology for
the E-10A weapon system. The level of this technology's maturity has not
yet been finalized. OSD officials accepted the Air Force's assessment of
the radar technologies but expect more detailed information on the
technologies when the E-10A weapon system undergoes its Milestone B review
in April 2005.

An assessment of the battle management command and control subsystem
technologies was not provided for our review. This subsystem is complex
and software intensive. E-10A program officials told us these technologies
would meet the minimum DOD standard for starting a program. However, the
Air Force only recently directed the contractor to begin systems
engineering efforts to determine a preliminary design for this subsystem.
Development of critical software needed to demonstrate the technologies
has not started. The first increment of software is not scheduled to be
delivered until January 2008. On other major weapon system development
programs, we have found software development to be a substantial cause for
delays in technology development, system deliveries, and increased costs.
Therefore, even though program officials have stated technologies are
sufficiently mature, we think stronger evidence will be needed to
demonstrate their claim.

Status of Cost Estimate and The Air Force has completed its cost estimate
for the total E-10A program

Funding Needs	and released it to OSD for review. The cost estimates for
each of the three major program elements contain risk. The biggest area of
cost uncertainty is the battle management command and control subsystem.
It is a highly

complex software-intensive system. A contract was issued in September 2004
for about $71 million to begin early design and engineering efforts to
support a preliminary design review in late 2005. Until this initial
design and engineering effort is completed, the program will not be able
to establish high confidence in its estimated costs. In addition, the
aircraft contract only calls for the delivery of one commercial 767-400ER
for testing. To convert this aircraft to military use, there will be
additional costs for installing communication antennas, a refueling
receptacle, hull hardening, and FAA airworthiness certification. According
to the Air Force, these costs have been factored into its latest program
estimate. The initial cost estimate for the radar program, managed
separately from the E-10A program, has grown. Prior to entering system
development, OSD determined that projected costs were understated and
directed the Air Force to increase its funding by $154 million.

The Air Force acknowledges that funding for the E-10A program is also a
major concern. Funding cuts have delayed its start. It has undergone two
congressional budget reductions; the first cut in fiscal year 2003 ($343
million) required a significant program replanning effort. The second cut
in fiscal year 2005 ($115 million) resulted in schedule delays for the
planned test program, system integration lab, testbed aircraft delivery,
and the E-10A's first flight. The Air Force states these cuts have caused
the planned initial operating capability date to slip 3 years to 2015. A
third cut, recently proposed by OSD in December 2004, reduces the
program's budget request by $300 million in both fiscal year 2006 and
2007-a total reduction of $600 million. The program office is in the
process of evaluating the impact of these reductions and officials
indicated that because these represent a reduction of about 45 percent in
each year, they will have a significant impact on the program if they are
sustained. OSD officials indicated that efforts related to aircraft
development and the delivery of the test aircraft will likely bear the
bulk of the reductions. This will have an impact on planned program
milestones. They said it was important to keep the radar program funded
because it is developing the radar planned for the new Global Hawk
unmanned aerial vehicle in addition to the E-10A.

  E-10A Acquisition Strategy Does Not Capture Critical Knowledge before Program
  Decision Points

The E-10A acquisition strategy raises concern as key decisions are planned
before critical product knowledge is available. For example, the strategy
for developing the first E-10A increment does not allow for adequate
integration or prototype demonstration to ensure the design is stable at
the system critical design review. System integration allows program
officials to measure the stability of a product's design and its ability
to meet established requirements. Both commercial companies and DOD
recognize the attainment of this knowledge as being demonstrated by the
completion of most engineering drawings and some demonstration of the
system level capabilities in a prototype. A stable design that meets
requirements should be achieved by critical design review, before system
demonstration and initial manufacturing of production representative
products begins. However, the Air Force does not expect to deliver the
battle management command and control and radar subsystems to the
integration laboratory until 2008 and 2009, after critical design review,
scheduled in 2007. The transition of the battle management command and
control and radar subsystems from the integration lab to the 767-400ER
test airframe is not scheduled to begin until late-2009, nearly 2 years
after the critical design review and only a few months prior to the
program's production commitment decision. As a result, critical knowledge
about the basic performance of key subsystems integrated into an actual
E-10A prototype will not occur until 2010 (see fig. 1).

        Figure 1: Comparison of E-10A Strategy and Best Practices Model

E-10A acquisition strategy

First fully integrated E-10A ready to begin reliability testing

Program start

                            Knowledge-based approach

                   Critical design review Production decision

                Integrated product has demonstrated reliability

          Technology development Product development Integration   Production 
                                 Demonstration                     

Program Critical Production start design review decision

Source: GAO analysis of Air Force data.

Note: In the above figure, technology development is extended into
production because there are several technologies that cannot be assessed
as mature until they are actually integrated and flight tested.

Additionally, the fully integrated E-10A prototype will not be available
for testing prior to the scheduled decision to begin production. This
strategy requires significant concurrency among the technology
development, product development, and production phases and places
decision makers at a disadvantage by not knowing if the E-10A can
demonstrate it meets system performance and reliability requirements
before transitioning into production. In fact, the results of operational
testing are not scheduled to be available until four of the six planned
E-10As are already in production in 2011, greatly increasing the risks of
costly design changes and schedule

delays later in the program (see fig. 2). Our past reviews have found this
to be a high-risk acquisition approach.7

                  Figure 2: E-10A Test and Production Schedule

            2009      2010      2011      2012      2013      2014     2015   

Source: U.S. Air Force.

Note: The schedule depicted above is based on the program plan in effect
at the time of our review. Since that time, the program's budget has been
reduced, necessitating revisions to this schedule. We have not had the
opportunity to review the new program plan.

The Air Force is planning to use an incremental approach to achieve the
E-10A's full capability with each subsequent increment adding capability.
Although an incremental approach can reduce risks, the failure to capture
critical knowledge while developing the first increment will likely reduce
the benefits of such an approach. As currently planned there will be four
distinct E-10A increments. Program officials are planning to conduct major
program decision reviews prior to beginning development and demonstration
of each increment. This approach, if implemented as planned, will provide
decision makers with an opportunity to review the program's progress and
risk before making further investment decisions thus reducing risk in the
program. The first increment is expected to

7GAO, Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes, GAO-02-701 (Washington, D.C.; July 15,
2002).

provide the users with many of the system's basic required capabilities.8
Those capabilities include cruise missile defense and on-board command and
control capability for processing, displaying, and communicating the data
needed to address time-sensitive targets. Subsequent increments will
enhance the system's capabilities, moving them closer to objective levels
by increasing the amount of data processing and analysis done by computers
and decreasing the amount done by human analysts with computer assistance,
thus shortening the time it takes to make decisions. However, if the first
increment falters, the Air Force will likely spend increasing amounts of
time and money to achieve this initial capability, thereby delaying
subsequent increments.

                                  Conclusions

The current conditions surrounding the development of the E-10A business
case portend the potential for poor outcomes if requirement, resource, and
acquisition strategy deficiencies are not resolved before system
development and demonstration begins. The decision to start a major weapon
systems acquisition program for the E-10A requires an executable business
case that demonstrates the E-10A is the best way to satisfy the gap in
warfighter's capability and that the concept can be developed and produced
within existing resources. An evolutionary and knowledge-based acquisition
strategy is needed to ensure this business case can be executed within
planned goals. The Air Force and OSD are still determining if a sound
business case exists. Questions still surrounding the business case
include:

o  Is the E-10A the most cost-effective alternative?

o  	How extensive of a battle management command and control capability is
needed?

o  Are technologies at a high level of maturity?

o  Is there sufficient funding to develop and deliver the capability in
time?

8 The E-10A Capabilities Development Document lists 157 separate
capabilities/ requirements to be achieved by increment 1.3. Of those
requirements, 50 will be upgraded or changed as subsequent increments are
developed. Thus the remaining 107 (68 percent) will be met by increment
1.0. The individual capabilities/requirements were not weighted in this
analysis.

The acquisition strategy also fails to capture critical design,
manufacturing, and reliability data in time to make investment decisions
for moving the program through the development program into production.
The gaps in knowledge increase the likelihood that the Air Force will not
be able to deliver on the cost, schedule, and performance goals in its
business case.

Because gaps exist in the information needed to make a sound business case
to start a major acquisition program, we recommend that the Secretary of
Defense ensure that the open business case questions are answered before a
decision is made to start the E-10A program. Additionally, to ensure a
greater likelihood of success, if the E-10A program is approved to begin,
we recommend the Secretary direct the Air Force to revise the acquisition
strategy to ensure sufficient time is included in the schedule to (1)
integrate and demonstrate the design before moving past the critical
design review and (2) test a production representative E-10A prototype
before starting production.

  Recommendations for Executive Action

                                Agency Comments
                               and Our Evaluation

DOD provided us with written comments on a draft of this report. The
comments appear in appendix II.

DOD concurred with our recommendation that the Secretary ensure that the
open business case questions are answered before a decision is made to
start the E-10A program. DOD provided some information on the current
status of these questions and implies that some of the business case
questions had been answered. We believe that until the OSD/Program
Analysis and Evaluation study is completed and final results are provided
to OSD acquisition decision makers, the business case questions remain
open.

DOD partially concurred with our recommendation that the Secretary direct
the Secretary of the Air Force to revise the E-10A acquisition strategy to
ensure sufficient time is available to (1) integrate and demonstrate the
design before moving past the critical design review and (2) test a
production representative E-10A before starting production.

Regarding (1), DOD stated that OSD policy does not require the integration
and demonstration of a design before critical design review. We disagree.
Section E1.1.14 of Department of Defense Directive 5000.1, The Defense
Acquisition System, states that "PMs...shall reduce integration risk and
demonstrate product design prior to the design readiness review." DOD's

design readiness review is required to end the system integration phase of
system development and demonstration. Additionally, DOD's entrance
criterion for the demonstration phase requires a demonstration of the
integrated product in a prototype. Nonetheless, DOD stated that it is
restructuring the program with the goal of demonstrating the radar and
battle management technologies in a prototype before starting systems
development and demonstration. This approach incorporates the
knowledge-based approach inherent in commercial best practices and
endorsed by DOD policy. In its comments, DOD acknowledges that this
approach will increase confidence in the program's cost estimate and allow
time to evaluate the aircraft platform.

Regarding (2), DOD stated that the Milestone C production decision for low
rate initial production decision will be based on the initial test results
from a representation E-10A aircraft system. While the program schedule in
effect at the time of our review did not indicate this, we believe this
approach is more consistent with a knowledge-based acquisition strategy.
By testing a production representative aircraft prior to committing to
production, DOD will be able to reduce program risks and make informed
decisions based on actual system capabilities and performance information.

DOD also provided technical comments to our report. We made changes where
appropriate but many of these comments were based on a new acquisition
strategy that plans to delay the E-10A program Milestone B decision until
2010. We did not make DOD's recommended changes to the report that
reflected this new schedule because it has not been approved and we have
not had the opportunity to review it.

We are sending copies of this report to the Secretary of Defense and the
Secretaries of the Air Force, the Army, and the Navy. We will also provide
copies to others on request. In addition, the report will be available at
no charge on the GAO Website at http://www.gao.gov.

Please contact me at (202) 512-4841 if you have any questions concerning
this report. Other key contributors to this report were Martin Campbell,
Michael Hazard, Travis Masters, Rae Ann Sapp, David Schilling and John
Krump.

Sincerely yours,

Michael J. Sullivan
Director
Acquisition and Sourcing Management

                       Appendix I: Scope and Methodology

During our review we discussed the E-10A program with officials from the
following organizations in the Office of the Secretary of Defense,
Undersecretary of Defense for Acquisition Technology and Logistics; the
Director, Defense Systems/Developmental Test and Evaluation; the Director,
Operational Test and Evaluation; the Director, Defense Research and
Engineering; and the Director, Program Analysis and Evaluation. We also
discussed the E-10A with the technical director of the Joint Theater Air
Missile Defense Organization. In addition, we discussed the program with
officials from several organizations in the Air Force. These officials
included representatives from the Information Dominance Directorate with
the Office of the Assistant Secretary for Acquisition; the Directorate of
Operational Requirements; the Command Control Communications Intelligence
and Reconnaissance Center at Langley Air Force Base; the Electronic
Systems Center at Hanscom Air Force Base; and the Aeronautical Systems
Center at Wright Patterson Air Force Base.

To determine the progress the Air Force had made in developing the
business case for the E-10A, we obtained available information on the
system's requirements and resources. However, the information we received
on resources such as technology maturity, cost, funding, quantities, and
schedule was limited. We discussed this information with knowledgeable
program office and oversight officials. We also contacted officials
studying force structure issues that could impact the requirements for the
E-10A program. To assess the validity of the proposed business case, we
compared the E-10A information with best commercial practices and DOD
policy guidance for new development programs. Because the E-10A program
has not yet been approved to enter system development and demonstration,
specific information on the system's technology readiness assessment and
total program cost and funding were not available. As a result, we could
not conduct a detailed assessment of these elements of the business case.
However, because of other related information, such as the status of the
software intensive battle management command and control subsystem, the
significant reduction in funding for fiscal years 2006 and 2007, and the
ongoing studies to answer OSD concerns, we were able to conclude that at
the time of our review key business case elements were still not mature
enough to begin product development. For example, complex and software
intensive subsystems in other programs have caused major problems that
have delayed achieving technology maturity and the Air Force has only
recently directed the contractor to begin early systems engineering effort
to determine a preliminary design for the E-10A battle management
subsystem. Additionally, the $600 million reduction in funding planned for
the first 2 years will almost certainly require the program to extend its
planned

Appendix I: Scope and Methodology

schedule resulting in additional costs and funding requirements not yet
estimated. These are business case elements that need to be firmly
established before entering the upcoming Milestone B decision point.

To determine the soundness of the E-10A's acquisition strategy, we
obtained available information on the program's original and revised
acquisition plans from the program office and discussed it with functional
oversight and program officials. In addition, we compared the E-10A's
planned strategy to best commercial practices and DOD's knowledgebased
acquisition policy. However, since our analysis, the program's budget
request was reduced by a total of $600 million in fiscal years 2006 and
2007.

We conducted our review from January 2004 to January 2005 in accordance
with generally accepted government auditing standards.

Appendix II: Comments from the Department of Defense

Appendix II: Comments from the Department of Defense

Appendix II: Comments from the Department of Defense

  GAO's Mission

Obtaining Copies of GAO Reports and Testimony

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone 	The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

  To Report Fraud, Contact:

Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470

Gloria Jarmon, Managing Director, [email protected] (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548

Public Affairs 	Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

                           PRINTED ON RECYCLED PAPER
*** End of document. ***