Uncertainties Remain Concerning the Airborne Laser's Cost and
Military Utility (17-MAY-04, GAO-04-643R).
In 1996, the Air Force launched an acquisition program to develop
and produce a revolutionary laser weapon system, the Airborne
Laser (ABL), capable of defeating an enemy ballistic missile
during the boost phase of its flight. Over the last 8 years, the
program's efforts to develop this technology have resulted in
significant cost growth and schedule delays. These events led
Senate Members to request that we answer the following questions:
(1) how much and why has the ABL's cost increased since the
program's inception; (2) what is the expected military utility of
the initial ABL aircraft; (3) what support systems will be
required when the ABL is fielded and what is the likely cost of
those systems; and (4) have recent program changes resulted in a
more cost effective strategy for developing the weapon?
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-643R
ACCNO: A10095
TITLE: Uncertainties Remain Concerning the Airborne Laser's Cost
and Military Utility
DATE: 05/17/2004
SUBJECT: Cost analysis
Defense capabilities
Defense economic analysis
Defense procurement
Military aircraft
Military cost control
Military systems analysis
Schedule slippages
Weapons research and development
Weapons systems
Cost growth
Lasers
MDA Airborne Laser Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-643R
United States General Accounting Office Washington, DC 20548
May 17, 2004
The Honorable Bill Nelson Ranking Minority Member Subcommittee on
Strategic Forces Committee on Armed Services United States Senate
The Honorable Daniel K. Akaka United States Senate
Subject: Uncertainties Remain Concerning the Airborne Laser's Cost and
Military Utility
In 1996, the Air Force launched an acquisition program to develop and
produce a revolutionary laser weapon system, the Airborne Laser (ABL),
capable of defeating an enemy ballistic missile during the boost phase of
its flight. Over the last 8 years, the program's efforts to develop this
technology have resulted in significant cost growth and schedule delays.
These events led you to request that we answer the following questions:
(1) How much and why has the ABL's cost increased since the program's
inception? (2) What is the expected military utility of the initial ABL
aircraft? (3) What support systems will be required when the ABL is
fielded and what is the likely cost of those systems? (4) Have recent
program changes resulted in a more cost effective strategy for developing
the weapon?
After we began our review, the Missile Defense Agency (MDA) refocused the
ABL program to pursue a more knowledge-based approach, where program
knowledge is demonstrated at critical points in the development cycle.1 We
examined the planned changes to determine whether they will result in a
more cost-effective strategy for developing the ABL element. This report
summarizes that information and transmits the briefing charts that have
been revised to address program changes made by MDA after our presentation
to your staff on March 4, 2004.
In conducting our review, we analyzed Department of Defense (DOD), MDA,
Air Force, and ABL program documents and interviewed key program
officials. To
1 A knowledge-based approach contains three phases that are each
distinguished by the knowledge attained. During technology development,
scientists apply scientific knowledge to a practical engineering problem
and demonstrate that components with the desired form, fit, and function
can be developed. During product development, the second phase, engineers
integrate components into a stable design and demonstrate that the design
will result in a product that meets the customer's needs and can be
produced with the time and money available. The final phase, production,
is the manufacturing of the product.
determine how much ABL's cost increased since the program was initiated
and why this growth occurred, we reviewed both the ABL prime contract
awarded in 1996 and significant modifications made to the contract through
July 2003. We also assessed the contractor's cost and schedule performance
by analyzing contractor cost performance reports using Earned Value
Management principles. To determine the expected military utility of the
initial ABL aircraft, we interviewed key DOD, MDA, and intelligence
officials and analyzed relevant documents, including studies and reports.
To determine the necessary support systems and their costs, we examined
relevant documents and held discussions with the Air Force Air Combat
Command. We also reviewed the Air Force Total Ownership Cost database to
identify support systems and costs for other high value air assets. To
determine whether recent program changes resulted in a more cost-effective
strategy for developing the ABL, we compared the program office's previous
plans, goals and activities with its recent efforts to refocus the
program. We conducted our work from October 2003 through May 2004 in
accordance with generally accepted government auditing standards.
Results in Brief
The prime contractor's costs for developing ABL have nearly doubled from
the Air Force's original estimate and additional cost growth is occurring.
The cost growth occurred primarily because the program did not adequately
plan for and could not fully anticipate the complexities involved in
developing the system. MDA continues to face significant challenges in
developing the ABL's revolutionary technologies and in achieving cost and
schedule stability. From 1996 through 2003, the value of the prime
contract, which accounts for the bulk of the program's cost, increased
from about $1 billion to $2 billion. According to our analysis, costs
could increase between $431 million to $943 million more through first
full demonstration of the ABL system. Cost growth has been spurred by
rework that was necessary because rapid prototyping forced the program to
integrate components before all subcomponents were fully tested. In
addition, fabricating ABL's unique components and developing its complex
software proved more costly and time-consuming than anticipated. Although
ABL's prime contractor has added additional personnel to the contract, the
program is faced with a bow wave of uncompleted work from prior years.
Recognizing that the technology development activities directed by the
contract could not be completed within the contract's cost ceiling, the
ABL program office began development of a new cost estimate for completing
these activities.
Predictions of the military utility of the initial ABL aircraft are still
highly uncertain because these forecasts are not based on any demonstrated
capability of the system, but rather on modeling, simulations, and
analysis. These assessment tools predict that the initial Airborne Laser
will be militarily useful against most theater and intercontinental
ballistic missiles; but flight-test data are not yet available to anchor
these tools. According to test officials, the models and simulations are
adequate for establishing system parameters, but may not be sufficient for
estimating the effectiveness of a fielded system. Additionally, other
factors will influence ABL's military utility, including the availability
of support infrastructure and the number of aircraft available.
When it is fielded, ABL is expected to require unique support for its
laser and beam control and fire control components in addition to the
support burdens attached to all high-value air assets.2 For example, to
carry out a wider range of missions, ABL will need laser fuel production
facilities close to the theater of operations. ABL will also require
unique maintenance, such as re-calibration and re-coating of beam control
and fire control subcomponents. In addition, it will require the typical
support systems needed by other high-value air assets, such as escort
aircraft for protection. MDA has not yet determined the cost of ABL's
unique support systems, but operating costs3 for other high-value air
assets range from about $24,000 per hour to $92,000 per hour.
MDA refocused the ABL program in February 2004 to pursue a more
cost-effective development strategy. The program now plans to follow a
knowledge-based rather than a schedule-driven approach to the element's
development. For example, the program is no longer working to deliver a
contingency sensor capability in Block 2004. Instead it will concentrate
on maturing and demonstrating ABL's critical technologies. In addition,
the program has delayed indefinitely the purchase of a second aircraft and
a test facility.
To provide better information to decision makers as they consider whether
to continue investing in the ABL program, we are recommending that DOD
provide an analysis that quantifies the confidence that decision makers
should have in the new cost estimate. DOD concurred.
Background
From 1996 to 2001, the ABL program was an Air Force major defense
acquisition program. However, in October 2001, the Department of Defense
(DOD) transferred responsibility for the program to the Ballistic Missile
Defense Organization (now MDA) where the ABL became one element of the
Ballistic Missile Defense System (BMDS). The ABL's mission is to destroy
enemy ballistic missiles in the boost phase as part of the layered defense
strategy. Similar to other BMDS elements, the ABL is being developed
incrementally in a series of 2-year blocks. During the first block, known
as Block 2004, MDA's goal is to mature the ABL's critical technologies.
The ABL includes four major components: a modified 747 aircraft, which
serves as the platform for other components; a high-energy chemical laser;
a beam control and fire control system; and a battle management and
command and control system. The high-energy laser is a chemical
oxygen-iodine laser that generates energy through chemical reactions. The
prototype, or demonstrator aircraft, that is under development will
include six laser modules that will be linked together to produce a laser
beam with megawatts of power. The beam control and fire control component
is designed to track and stabilize the beam so that its energy remains
focused on a small area of an enemy missile. ABL uses its high-energy
laser to defeat enemy
2 Examples of other high-value air assets are the Airborne Warning and
Control System and the Joint
Surveillance Target Attack Radar System.
3 According to the U.S. Air Force Total Ownership Costs Database,
operating costs include costs for
mission personnel, unit-level consumption, intermediate maintenance, depot
maintenance, contractor
support, sustaining support, and indirect support.
missiles by rupturing a missile's motor casing, causing the missile to
lose power. MDA expects that the battle management and command and control
component will plan and execute each ABL engagement.
Program Complexity Drives Cost Growth
Because it did not adequately plan for and could not fully anticipate the
complexity of developing and demonstrating ABL's critical technologies,
the program has experienced continual cost growth and schedule delays. The
prime contract, which consumes the bulk of the program's funding, will
reach its cost ceiling in May 2004. The program office is in the process
of developing a new cost estimate as the basis for raising the contract's
ceiling price. Additionally, in recognition of the contract's rising cost,
MDA has increased its cost goal for completing ABL's technology
development phase during Block 2004.
Contract's Current Value Has Doubled Since 1996 and Continues to Increase
From the contract's award in 1996 through 2003, the cost of ABL's primary
research and development contract increased from about $1 billion to about
$2 billion. In fiscal year 2003 alone, work completed by the contractor
cost about $242 million more than expected. The contractor also
experienced schedule delays and was unable to complete $28 million of work
planned for the fiscal year. Based on the contractor's 2003 cost and
schedule performance, we estimate that the prime contract will exceed the
contractor's July 2003 cost estimate of about $2.1 billion by between $431
million and $943 million through first full demonstration of the ABL
system.
Since 1996, the ABL program has experienced several major restructurings
and contract re-baselines, due primarily to the unforeseen complexity in
manufacturing and integrating critical technology. According to program
officials, rapid prototyping (integrating components into a prototype
system prior to demonstrating the maturity of all critical technologies)
limited subcomponent testing, causing rework and changing requirements.
Today the program faces a bow wave of incomplete work from previous years
even though the prime contractor has increased the number of people
devoted to the program and has added additional shifts to bring the work
back on schedule. In addition, unanticipated difficulties in software
coding and integration issues, as well as difficulty in manufacturing
advanced optics and laser components, have caused cost growth.
Increasing Cost of Prime Contract Causes Revisions in Technology
Development Cost Goal
According to program officials, the prime contract will reach its cost
ceiling in May 2004. The ABL program office is in the process of
developing a new cost estimate for the remaining technology development
activities, including component ground tests, beam control and fire
control flight-tests, integration of components into a working ABL
prototype, and a demonstration of the prototype's lethality against a
boosting ballistic missile. The program will use this estimate as the
basis for raising the contract's ceiling price.
MDA also recognized the impact that the prime contract's rising cost is
having on the agency's cost goal for completing ABL's technology
development. In February 2003, MDA reported in its fiscal year 2004 Budget
Estimate Submission4 that its cost goal for maturing and demonstrating
ABL's critical technologies during Block 2004 was about $494 million.
However, in its fiscal year 2005 submission, MDA increased its Block 2004
cost goal by $583 million to $1.077 billion by moving ABL funds from
Blocks 2006 and 2008 to Block 2004. The program manager told us that MDA
plans to continue updating its cost goal for completing technology
development as part of each year's budgeting process. To better assess the
program's annual funding needs, the contractor will plan work in about
12-month increments.
Successful developers have found that it is difficult to make highly
reliable cost and schedule estimates until the maturity of a product's
critical technologies has been demonstrated. The ABL program has not yet
reached this point. One wellestablished tool for providing decision makers
with an increased understanding of the reliability of developed cost
estimates is an "uncertainty analysis." An uncertainty analysis simulates
a model by randomly and repeatedly generating values for certain
variables. After hundreds or thousands of trials, one can view the
statistical results and determine the confidence level in any outcome. For
example, the model may show that there is a 10-percent chance that the
project can be completed for $50 million, a 50-percent chance that it can
be completed for $70 million, or a 90-percent chance that the project will
cost $100 million or less. Although there remains a gap in the knowledge
MDA needs to make highly reliable estimates of the cost to complete ABL's
technology development, other programs have found that uncertainty
analyses help to understand the size of such a gap.
Military Utility of Initial ABL Aircraft Is Highly Uncertain
Predictions of the military utility of the initial ABL aircraft are highly
uncertain because these forecasts are based on modeling, simulations, and
analysis, rather than the demonstrated capability of the system. The
Airborne Laser program office predicts that the ABL prototype being
developed during Block 2004 will have a capability to defeat most classes
of theater and intercontinental ballistic missiles. However, program
officials base these predictions on analyses, models, and simulations that
are not yet anchored by flight test data. These assessment tools take into
consideration various factors, such as the projected power output of the
6module laser weapon, the stand-off range that ABL maintains to protect
itself from threat weapons, and the characteristics of the threat
missiles. Should any of the assumptions made in modeling these factors
prove incorrect, the output of the assessment tools might prove similarly
incorrect. According to test officials, models and simulations are
adequate for establishing system parameters, but may not be
5
sufficient for understanding the effectiveness of a fielded system. In
addition, the Department of Defense has not yet assessed the Airborne
Laser's military utility and has no fixed time frame for doing so.
4 For the last 2 fiscal years, MDA has issued a document known as a Budget
Estimate Submission that
lays out planned expenditures for each near-term block by element and the
activities planned during
those blocks.
5 We did not examine the models and simulations that MDA uses to forecast
ABL's capability or
attempt to replicate MDA's predictions of ABL's military utility.
Even if the program office's assessment proves to be correct, other
factors, such as reduced time on station and limited forward operating
capability, will limit ABL's initial military utility. For example, with
only one aircraft, 24-hour operations are not possible. In addition, in
order to operate from a forward location the aircraft will require a
forward-based chemical facility for replenishment of the chemicals that
fuel the high-energy laser. Without these facilities, the Airborne Laser
is capable of performing three constrained missions.
Support Costs Will Likely Be Higher Than Those of Other High Value Air Assets
The ABL will require unique support in addition to the already substantial
support required for all high-value air assets. For example, to remain on
station for extended periods of time, ABL will need a production facility
close to the theater of operations that can store and mix chemicals for
the high-energy laser. Likewise, ABL will require unique maintenance, such
as re-calibration and re-coating of beam control and fire control
subcomponents; chemical facilities must be secured; and a ground support
squadron and chemicals must be transported to a forward location.
The ABL will also require support systems, such as a protective escort,
that are typical of all high-value air assets. Our review of the U.S. Air
Force Total Ownership
6
Costs Database shows that operating costs for other high-value air assets,
such as the Airborne Warning and Control System or Joint Surveillance
Target Attack Radar System, range from about $24,000 per hour to $92,000
per hour. While the ABL program office has identified some of the support
systems ABL will require once it is fielded, the program does not plan to
fully address this issue, including support costs, before Block 2006.
Program Changes Likely to Result in a More Cost-Effective Strategy
MDA made program changes in February 2004 that should make its strategy
for developing ABL more cost-effective. The primary change is that the
program will no longer try to develop ABL according to a set schedule, but
will follow a more knowledge-based approach. For example, MDA no longer
plans to deliver a contingency sensor capability in the Block 2004 time
frame. Instead it plans to focus on ground- and flight-testing components
and demonstrating technologies before proceeding to the next development
phase. However, it does not intend to lose sight of its overall objective
of testing the prototype ABL aircraft against a short-range ballistic
missile in what has become known as a lethal demonstration. Program
officials also told us that they plan to defer indefinitely the purchase
of a second aircraft and other hardware, which MDA initially planned to
initiate during Block 2004.
6 The Air Force Total Ownership Costs database provides detailed cost
information on all Air Force major weapon systems.
Conclusion
We believe the ABL program made a sound and cost-effective decision to
defer the purchase of the second aircraft and additional hardware and to
pursue a more knowledge-based approach to the element's development. It
has also taken an important first step by setting a cost goal for ABL's
technology development phase. However, good investment decisions depend
upon an understanding of the total funds that will be needed to obtain an
expected benefit, and, at this time, MDA has not been able to provide
decision makers assurances that the agency's cost projections to complete
technology development can be relied upon. Decision makers could make more
informed decisions about further investments in the ABL program if they
understood the likelihood and confidence associated with MDA's cost
projections.
Recommendation
To provide a better framework for making investment decisions during the
program's research and development phase, we recommend that the Secretary
of Defense direct the Director, MDA, to complete an uncertainty analysis
on the contractor's new cost estimate that quantifies the confidence that
may be placed in the estimate.
Agency Comments and Our Evaluation
In written comments to a draft of this report (see encl. II), the DOD
concurred with our recommendation. DOD also provided separate technical
comments, which we have incorporated as appropriate.
-----
As requested by your offices, unless you publicly announce the contents of
this report
earlier, we plan no further distribution of it until 30 days from the date
of this letter.
At that point, copies of this report will be sent to the Secretary of
Defense; the
Director, Missile Defense Agency; and other interested congressional
committees.
We will also make copies available to others upon request. In addition,
the report will
be available at no charge on the GAO Web site at http://www.gao.gov.
Should you or your staff have any questions on matters discussed in this
report, please contact me on (202) 512-4841 or Barbara Haynes at (256)
922-7535. Principal contributors to this report were Beverly Breen, Alan
Frazier, LaTonya Miller, and Karen Richey.
Robert E. Levin
Director, Acquisition and Sourcing Management
Enclosure
Date Event Reason Impact to Prime Contract
program Value
Design, fabrication, Initiated
November Original integration, and program
1996 (PDRR) testing of the ABL definition $1,021,655,199
Contract Award prototype and risk
reduction
According to program
officials,
additional funds
April 1999 Contract were added for laser Increased 1,226,766,457
Restructure module tests and the contract
introduction of the costs 1-year
System Integration schedule
Laboratory growth
Unforeseen Increased
complexity in optics cost growth
April 2000 Critical and laser primarily 1,241,864,301
Design Review manufacturing for laser
technology modules
discovered
Aircraft Operating Cost per Hour Support Cost per Hour
High value air asset
AWACS E3B (ACC) $24,095 $5,418
JSTARS E8-C (ACC) 51,944 5,335
Rivet Joint RC-135W (ACC) 48,233 35,855
E4-B 747 (ACC) 92,106 61,545
Transport, fighters, &
tankers
C-17A (AMC) 11,752 4,834
F-16C (ACC) 12,340 1,941
KC-135R (AMC) 11,022 1,624
Comments From the Department of Defense
(120290)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, [email protected] (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***