Space Acquisitions: DOD Needs to Take More Action to Address	 
Unrealistic Initial Cost Estimates of Space Systems (17-NOV-06,  
GAO-07-96).							 
                                                                 
Estimated costs for the Department of Defense's (DOD) major space
acquisition programs have increased by about $12.2 billion from  
initial estimates for fiscal years 2006 through 2011. Cost growth
for ongoing Air Force programs above initial estimates accounts  
for a substantial portion of this 44 percent increase. In light  
of the role that optimistic estimating is believed to have played
in exacerbating space acquisition cost growth, you requested that
we examine (1) in what areas space system acquisition cost	 
estimates have been unrealistic and (2) what incentives and	 
pressures have contributed to the quality and usefulness of cost 
estimates for space system acquisitions.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-96						        
    ACCNO:   A63513						        
  TITLE:     Space Acquisitions: DOD Needs to Take More Action to     
Address Unrealistic Initial Cost Estimates of Space Systems	 
     DATE:   11/17/2006 
  SUBJECT:   Accountability					 
	     Cost analysis					 
	     Cost overruns					 
	     Defense cost control				 
	     Defense procurement				 
	     Procurement planning				 
	     Program evaluation 				 
	     Program management 				 
	     Strategic planning 				 
	     Aerospace research 				 
	     Cost estimates					 
	     Air Force Advanced Extremely High			 
	     Frequency Satellite Program			 
                                                                 
	     DOD Evolved Expendable Launch Vehicle		 
	     Program						 
                                                                 
	     DOT Global Positioning System			 
	     National Polar-Orbiting Operational		 
	     Environmental Satellite System			 
                                                                 
	     Space Based Infrared System-High			 
	     Wideband Gapfiller Satellites			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-96

   

     * [1]Results in Brief
     * [2]Background

          * [3]Past GAO Findings on Space Cost Growth

     * [4]Program Office Cost Estimates on Space Programs Not Realisti
     * [5]Various Incentives and Pressures within DOD Have Contributed

          * [6]Accountability Is Lacking
          * [7]Independent Estimates Not Always Relied Upon
          * [8]Independent Cost Estimates Not Updated Frequently Enough to
          * [9]Cost-Estimating Roles and Responsibilities Are Unclear
          * [10]Cost-Estimating Resources Are Considered Inadequate

     * [11]Successful Organization Approaches That Better Support Cost
     * [12]Conclusions
     * [13]Recommendations for Executive Action
     * [14]Agency Comments and Our Evaluation
     * [15]AEHF
     * [16]NPOESS
     * [17]SBIRS High
     * [18]GAO Contact
     * [19]Staff Acknowledgments
     * [20]GAO's Mission
     * [21]Obtaining Copies of GAO Reports and Testimony

          * [22]Order by Mail or Phone

     * [23]To Report Fraud, Waste, and Abuse in Federal Programs
     * [24]Congressional Relations
     * [25]Public Affairs

Report to Subcommittee on Strategic Forces, Committee on Armed Services,
House of Representatives

United States Government Accountability Office

GAO

November 2006

SPACE ACQUISITIONS

DOD Needs to Take More Action to Address Unrealistic Initial Cost
Estimates of Space Systems

GAO-07-96

Contents

Letter 1

Results in Brief 2
Background 3
Program Office Cost Estimates on Space Programs Not Realistic 7
Various Incentives and Pressures within DOD Have Contributed to Cost-
Estimating Weaknesses 13
Successful Organization Approaches That Better Support Cost Estimating 22
Conclusions 25
Recommendations for Executive Action 26
Agency Comments and Our Evaluation 27
Appendix I Scope and Methodology 29
Appendix II DOD Acquisition Categories for Major Defense Acquisition
Programs 31
Appendix III Examples of Where Program Officials Were Too Optimistic in
Their Assumptions 32
Appendix IV Examples Where Independent Cost Estimates Were Not Relied Upon
37
Appendix V Comments from the Department of Defense 45
Appendix VI GAO Contacts and Staff Acknowledgments 49
Related GAO Products 50

Tables

Table 1: Areas Where Program Officials Were Too Optimistic in Their
Assumptions 8
Table 2: Comparison of 2004 AEHF Program Office and Independent Cost
Estimates 15
Table 3: Comparison of 2003 NPOESS Program Office and Independent Cost
Estimates 16
Table 4: Comparison of 1996 SBIRS High Program Office Cost Estimate and
Independent Cost Estimate 16
Table 5: DOD Acquisition Categories and Decision Authorities 31
Table 6: Examples of Optimistic Assumptions 32
Table 7: Comparison of 2004 AEHF Program Office and Independent Cost
Estimates 37
Table 8: Historical AEHF Weight Growth 38
Table 9: Comparison of 2003 NPOESS Program Office and Independent Cost
Estimates 40
Table 10: Program Office Integration Estimates for NPOESS 41
Table 11: SBIRS High GEO 3-5 Procurement Funding Analysis 43

Figure

Figure 1: Key Events and Funding Shifts That Occurred between Estimates
for SBIRS High 18

Abbreviations

ACAT Acquisition Category AEHF Advanced Extremely High Frequency AFCAA Air
Force Cost Analysis Agency APB Acquisition Program Baseline CAIG Cost
Analysis Improvement Group DOD Department of Defense DMSP Defense
Meteorological Satellite Program EELV Evolved Expendable Launch Vehicle
GEO geosynchronous earth orbit GPS Global Positioning System

HEO highly elliptical orbit KDP key decision point NPOESS National
Polar-orbiting Operational Environmental Satellite System NRO National
Reconnaissance Office NSA National Security Agency SBIRS Space Based
Infrared System SMC Space and Missile Systems Center TRL Technology
Readiness Level TSAT Transformational Satellite Communications System TSPR
Total System Performance Responsibility WGS Wideband Gapfiller Satellites

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office

Washington, DC 20548

November 17, 2006 November 17, 2006

The Honorable Terry Everett Chairman The Honorable Silvestre Reyes Ranking
Minority Member Subcommittee on Strategic Forces Committee on Armed
Services House of Representatives The Honorable Terry Everett Chairman The
Honorable Silvestre Reyes Ranking Minority Member Subcommittee on
Strategic Forces Committee on Armed Services House of Representatives

Estimated costs for the Department of Defense's (DOD) major space
acquisition programs have increased a total of about $12.2 billion--or
nearly 44-percent--above initial estimates for fiscal years 2006 through
2011. In some cases, current estimates of costs are more than double the
original estimates. For example, the Space Based Infrared System (SBIRS)
High program was originally estimated to cost about $4 billion, but is now
estimated to cost over $10 billion. The National Polar-orbiting
Operational Satellite System (NPOESS) program was originally estimated to
cost almost $6 billion but is now over $11 billion. Such growth has had a
dramatic impact on DOD's overall space portfolio. To cover the added costs
of poorly performing programs, DOD has shifted scarce resources away from
other programs, creating a cascade of cost and schedule inefficiencies.
Estimated costs for the Department of Defense's (DOD) major space
acquisition programs have increased a total of about $12.2 billion--or
nearly 44-percent--above initial estimates for fiscal years 2006 through
2011. In some cases, current estimates of costs are more than double the
original estimates. For example, the Space Based Infrared System (SBIRS)
High program was originally estimated to cost about $4 billion, but is now
estimated to cost over $10 billion. The National Polar-orbiting
Operational Satellite System (NPOESS) program was originally estimated to
cost almost $6 billion but is now over $11 billion. Such growth has had a
dramatic impact on DOD's overall space portfolio. To cover the added costs
of poorly performing programs, DOD has shifted scarce resources away from
other programs, creating a cascade of cost and schedule inefficiencies.

Our work has identified a variety of reasons for this cost growth, most
notably that weapons programs are incentivized to produce and use
optimistic cost and schedule estimates in order to successfully compete
for funding and that DOD starts its space programs too early, that is,
before it has assurance that the capabilities it is pursuing can be
achieved within available resources and time constraints. At the same
time, however, this cost growth was partly due to the fact that DOD used
low cost estimates to establish programs' budgets and later found it was
necessary to make funding shifts that had costly, reverberating effects.
In 2003, a DOD study of space acquisition problems found that the space
acquisition system is strongly biased to produce unrealistically low cost
estimates throughout the process. The study found that most programs at
the time of contract initiation had a predictable cost growth of 50 to 100
percent. The study also found that the unrealistically low projections of
program cost and lack of provisions for management reserve seriously
distorted management decisions and program content, increased risks to Our
work has identified a variety of reasons for this cost growth, most
notably that weapons programs are incentivized to produce and use
optimistic cost and schedule estimates in order to successfully compete
for funding and that DOD starts its space programs too early, that is,
before it has assurance that the capabilities it is pursuing can be
achieved within available resources and time constraints. At the same
time, however, this cost growth was partly due to the fact that DOD used
low cost estimates to establish programs' budgets and later found it was
necessary to make funding shifts that had costly, reverberating effects.
In 2003, a DOD study of space acquisition problems found that the space
acquisition system is strongly biased to produce unrealistically low cost
estimates throughout the process. The study found that most programs at
the time of contract initiation had a predictable cost growth of 50 to 100
percent. The study also found that the unrealistically low projections of
program cost and lack of provisions for management reserve seriously
distorted management decisions and program content, increased risks to
mission success, and virtually guaranteed program delays. We have found
that most of these conditions exist in many DOD programs.

Given concerns about the role optimistic cost estimating has played in
exacerbating space acquisition problems, you requested that we examine (1)
in what areas space system acquisitions cost estimates have been
unrealistic and (2) what incentives and pressures have contributed to the
quality and usefulness of cost estimates for space system acquisitions.

In conducting our work, we developed case studies of six ongoing major
space acquisition programs that included analysis of cost and other
program documentation. These include the Advanced Extremely High Frequency
(AEHF) satellite program (communications satellites), the Evolved
Expendable Launch Vehicle (EELV) (satellite launch systems), the Global
Positioning System (GPS) IIF (navigational satellites), the National
Polar-orbiting Operational Environmental Satellite System (weather and
environmental monitoring satellites), the Space Based Infrared System High
(missile detection satellites), and the Wideband Gapfiller Satellites
(WGS) (communication satellites). We also spoke with officials from DOD,
the Air Force, and contractor offices and analyzed DOD and Air Force
acquisition and cost-estimating policies. In addition, we obtained input
on our findings from a panel of cost-estimating experts who work within
the Office of the Secretary of Defense as well as the Air Force.
Additional information on our scope and methodology is in appendix I. We
conducted our work from August 2005 to October 2006 in accordance with
generally accepted government auditing standards.

Results in Brief

Our analyses of six ongoing space programs found that original cost
estimates were unrealistic in a number of areas, specifically, savings
from increased contractor program management responsibilities, the
constancy and availability of the industrial base, savings that could be
accrued from heritage systems, the amount of weight growth that would
occur during a program, the availability of mature technology, the
stability of funding, the stability of requirements, and the achievability
of planned schedules. At times, estimates that were more realistic in
these areas were available to the Air Force, but they were not used so
that programs could sustain support amid competition for funding.

Cost-estimating and program officials we spoke with identified a number of
factors that have contributed to low estimates in addition to the larger
pressures to win support for funding. For example, although the National
Security Space Acquisition policy requires independent cost estimates that
are prepared by bodies outside the acquisition chain of command, such
estimates have not always been relied upon for program decisions or to
develop program budgets. In addition, while the policy requires that
independent cost estimates be prepared or updated at major acquisition
milestones, significant events, such as changes in the industrial base or
funding, have occurred between milestones. Moreover, within space system
acquisitions, cost-estimating officials believe that their roles and
responsibilities are not clear, and the cost-estimating function is
fragmented. Finally, according to Air Force officials, cost-estimating
resources have atrophied over the years because of the previous downsizing
of the workforce, making resources such as staff and data inadequate and
the Air Force more dependent on support contractors for the estimating
function.

While the Air Force has taken steps recently to emphasize the use of
independent cost estimates, it has not made additional changes needed to
enhance the quality of cost estimates. We are making recommendations aimed
at instituting these actions. DOD agreed with most of our recommendations,
and is taking a number of actions to improve the Air Force's
cost-estimating capability for space programs. DOD expressed concern that
requiring officials involved in milestone decisions to document and
justify their choice of cost estimates would reduce the milestone decision
authority's future decision-making flexibility. While we recognize the
importance of decision-making flexibility, we believe that more
transparency in DOD's decision making is needed given the poor foundation
of choices made in the past on space programs.

Background

Estimates of the total cost of a program are critical components in the
acquisition process because they help decision makers decide among
competing options and evaluate resource requirements at key decision
points. All military services prepare life-cycle cost estimates in support
of their acquisition programs that attempt to identify all costs of an
acquisition program, from initiation through development, production, and
disposal of the resulting system at the end of its useful life. These
estimates serve two primary purposes. First, they are used at acquisition
program milestone and decision reviews to assess whether the acquisition
is affordable or consistent with the military services' and DOD's overall
long-range funding, investment, and force structure plans. Second, they
form the basis for budget requests to Congress. A realistic estimate of
projected costs makes for effective resource allocation, and it increases
the probability of a project's success.

The requirements and guidance for cost estimating are specified in statute
and in DOD policies. By law, there is a requirement that an independent
life-cycle cost estimate be considered by the milestone decision authority
before approving system development and demonstration, or production and
deployment, of a major defense acquisition program.^1 The statute requires
DOD to prescribe regulations governing the content and submission of such
estimates and that the estimate be prepared by (1) an office or other
entity that is not under the supervision, direction, or control of the
military department, DOD agency, or other DOD component directly
responsible for carrying out the development or acquisition of the
program, or (2) by an office or other entity that is not directly
responsible for carrying out the development or acquisition of the program
if the decision authority for the program has been delegated to an
official of a military department, DOD agency, or other DOD component.^2
The statute specifies that the independent estimate is to include all
costs of development, procurement, military construction, and operations
and support, without regard to funding source or management control.^3 DOD
policy assigns specific responsibility for fulfilling the requirement of
an independent cost estimate to the Office of the Secretary of Defense
Cost Analysis Improvement Group (CAIG) for any major defense acquisition
program and major system that are subject to review by the Defense
Acquisition Board of the Defense Space Acquisition Board.^4 These board
reviews address major defense acquisition programs (including space
programs) that are designated as acquisition category (ACAT) ID, pre-major
defense acquisition programs, or ACAT IC programs (see app. II for a
description of acquisition categories ID and IC). The CAIG independent
cost estimate is prepared for milestone (known as key decision point in
space programs) B (program start, or preliminary design for space
programs), and C (low-rate initial production or build approval for space
programs). In addition, the milestone decision authority may request the
CAIG to prepare other independent cost estimates, or conduct other ad hoc
cost assessments for programs subject to its review and oversight. The
CAIG serves as the principal advisory body to the milestone decision
authority on all matters concerning an acquisition program's life-cycle
cost, and is given general responsibilities for establishing DOD policy
guidance on a number of matters relating to cost estimating.

^1 10 U.S.C. S 2434 (2000).

^2 10 U.S.C. S 2434(b)(1)(A).

^3 10 U.S.C. S 2434(b)(1)(B).

^4 DOD Directive 5000.04, Cost Analysis Improvement Group at P 2 (Aug.
2006); DOD Instruction 5000.2, Enclosure 6, Resource Estimation (May
2003).

Since 2003, cost estimating for major space system acquisitions has been
governed by the National Security Space Acquisition Policy.^5 Under this
policy, the CAIG is responsible for and leads the development of
independent cost analyses of major space acquisition programs.^6
Fulfilling the requirement that an independent cost estimate be developed
by an organization independent of the program office and the acquisition
chain of command, the CAIG does so in support of a distinct Defense Space
Acquisition Board, with the Under Secretary of the Air Force as the
milestone decision authority.^7 The CAIG is to prepare independent cost
analyses for space acquisition programs by augmenting its own staff with
an independent team of qualified personnel from across the space
community, including the Air Force Cost Analysis Agency (AFCAA) and the
cost estimating organizations of the Air Force Space Command and the Air
Force Space and Missile Systems Center. In addition to the independent
cost estimates, individual program offices also prepare cost estimates for
their acquisition programs. The independent CAIG cost estimate is designed
to assess the program office estimate and ensure realistic cost estimates
are considered. In addition, although not required in the space
acquisition policy, in some cases a cost analysis is prepared by an Air
Force service organization, such as the Air Force Cost Analysis Agency.

Past GAO Findings on Space Cost Growth

For fiscal years 2006 through 2011, estimated costs for DOD's major space
acquisition programs have increased a total of about $12.2 billion above
initial estimates. For example, the cost estimate for the SBIRS High
program rose from about $4 billion at the start of development in October
1996 to over $10 billion in September 2005, and costs are expected to rise
further. In addition, the cost estimate for the NPOESS program grew from
about $5.9 billion at program start in 2002 to nearly $11.4 billion
currently, according to the CAIG's latest estimate.

^5 National Security Space Acquisition Policy 03-01 (revised December
2004).

^6 National Security Space Acquisition Policy at Appendix 3.2.

^7 Recently, the Under Secretary of Defense for Acquisition, Technology
and Logistics withdrew its delegation of milestone decision authority from
the Air Force. As a result, although some acquisition authority was
returned to the Air Force, the Under Secretary of Defense for Acquisition,
Technology and Logistics is the current milestone decision authority for
major space system acquisitions. It is not known when or if this role will
be placed back within the Air Force.

Our past work has identified a number of causes behind the cost growth and
related problems, but several consistently stand out. First, on a broad
scale, DOD starts more weapon programs than it can afford, creating a
competition for funding that encourages low cost estimating, optimistic
scheduling, overpromising, suppressing of bad news, and, for space
programs, forsaking the opportunity to identify and assess potentially
better alternatives. Programs focus on advocacy at the expense of realism
and sound management. Invariably, with too many programs in its portfolio,
DOD is forced to continually shift funds to and from
programs--particularly as programs experience problems that require more
time and money to address. Such shifts, in turn, have had costly,
reverberating effects.

Second, as we have previously testified and reported, DOD starts its space
programs too early, that is, before it has the assurance that the
capabilities it is pursuing can be achieved within available resources and
time constraints. This tendency is caused largely by the funding process,
since acquisition programs attract more dollars than efforts concentrating
solely on proving technologies. Nevertheless, when DOD chooses to extend
technology invention into acquisition, programs experience technical
problems that require large amounts of time and money to fix. Moreover,
when this approach is followed, cost estimators are not well positioned to
develop accurate cost estimates because there are too many unknowns. Put
more simply, there is no way to estimate how long it would take to design,
develop, and build a satellite system when critical technologies planned
for that system are still in relatively early stages of discovery and
invention.

A companion problem for space systems is that programs have historically
attempted to satisfy all requirements in a single step, regardless of the
design challenge or the maturity of the technologies necessary to achieve
the full capability. Increasingly, DOD has preferred to make fewer, but
heavier, large and complex satellites that perform a multitude of missions
rather than larger constellations of smaller, less complex satellites that
gradually increase in sophistication. This has stretched technology
challenges beyond current capabilities in some cases and vastly increased
the complexities related to software--a problem that affected SBIRS High
and AEHF, for example.

In addition, several of the space programs included in our case studies,
began in the late 1990s, when DOD structured contracts in a way that
reduced oversight and shifted key decision-making responsibility onto
contractors. This approach--known as Total System Performance
Responsibility, or TSPR--was intended to facilitate acquisition reform and
enable DOD to streamline a cumbersome acquisition process and leverage
innovation and management expertise from the private sector. However, DOD
later found that this approach magnified problems related to requirements
creep and poor contractor performance. In addition, under TSPR, the
government decided not to obtain certain cost data, a decision that
resulted in the government having even less oversight of the programs and
limited information from which to manage the programs. Further, the
reduction in government oversight and involvement led to major reductions
in various government capabilities, including cost-estimating and
systems-engineering staff. The loss of cost-estimating and
systems-engineering staff in turn led to a lack of technical data needed
to develop sound cost estimates.

Our reviews have identified additional factors that have contributed to
space cost growth, though less directly. These include consolidations
within the defense supplier base for space programs, the diverse array of
officials and organizations involved with space programs, short tenures
for top leadership and program managers, as well as capacity shortfalls
that have constrained DOD's ability to optimize and oversee its space
programs. A section at the end of this report lists prior relevant GAO
reports.

Program Office Cost Estimates on Space Programs Not Realistic

Our case study analyses found that program office cost estimates--and more
specifically, the assumptions upon which those estimates were based--have
been unrealistic in eight areas, many of which are interrelated. In some
cases, such as assumptions regarding weight growth and the ability to gain
leverage from heritage, or legacy, systems, past experiences or contrary
data were ignored. In other cases, such as when contractors were given
more program management responsibility, as with TSPR, or when growth in
the commercial market was predicted, estimators assumed that promises of
reduced cost and schedule would be borne out and did not have the benefit
of experience to factor into their work. We also identified flawed
assumptions that reflected deeper flaws in acquisition strategies or
development approaches. For example, five of six programs we reviewed
assumed technology would be sufficiently mature when needed, even though
the programs began without a complete understanding of how long it would
take or how much it would cost to ensure technologies could work as
intended. In four programs, estimators assumed there would be few delays,
even though programs were adopting highly aggressive schedules while
simultaneously attempting to make ambitious leaps in capability. In four
programs, estimators assumed funding would stay constant, even though
space and weapon programs frequently experience funding shifts and the Air
Force was in the midst of starting a number of costly new space programs
to replenish older constellations.

Table 1 highlights major areas where program officials were too optimistic
in their assumptions for the six space system acquisitions we examined or
where additional evidence showed the estimate was unrealistic. In some
cases, programs may have experienced problems related to one of the
categories, but we did not have evidence to show the original assumptions
were optimistic.

Table 1: Areas Where Program Officials Were Too Optimistic in Their
Assumptions

                                               Space programs affected
                                                   GPS            SBIRS     
Optimistic assumptions               AEHF EELV  IIF   NPOESS   High    WGS
Industrial base would remain                         X        X    X   X X 
constant and available                                                     
Technology would be mature enough        X                    X    X   X X 
when needed                                                                
TSPR would reduce costs and schedule                 X        X    X   X   
Savings would occur from experience      X                         X   X X 
on heritage systems                                                        
No weight growth would occur             X                         X   X X 
Funding stream would be stable           X                    X    X   X   
An aggressive schedule                   X                         X   X X 
No growth in requirements                X                    X        X   

Source: This table is based on conversations with program and contracting
officials and analysis of data they provided. In some cases, we made our
own designations based on our prior findings.

           o Assumptions about the space industrial base: Five programs
           experienced challenges due to assumptions that were made about the
           availability and constancy of the industrial base. When cost
           estimates for some of these programs were developed, cost
           estimators assumed the programs would gain leverage from the
           commercial satellite market, which, at the time the programs were
           initiated, was widely expected to continue to grow. In the EELV
           program, for instance, the original contracting concept was for
           the Air Force to piggyback on the anticipated launch demand of the
           commercial sector. Furthermore, the Air Force assumed that it
           would benefit financially from competition among commercial
           vendors. However, the commercial demand never materialized, and
           the government was forced to bear the cost burden of maintaining
           the industrial base in order to maintain launch capability, and
           assumed savings from competition were never realized. In other
           cases, programs experienced unanticipated problems resulting from
           consolidations in the supplier base. For example, contractors took
           cost-cutting measures that reduced the quality of parts.
           Contractors also lost key technical personnel as they consolidated
           development and manufacturing facilities.

           o Assumptions about technology maturity: In five of the six space
           system acquisition programs, when cost estimates were developed,
           program officials and cost estimators assumed that technologies
           critical to the programs would be mature and available--even
           though the programs began without a complete understanding of how
           long or how much it would cost to ensure technologies could work
           as intended. Invariably, after the programs began and as their
           development continued, the technology issues ended up being more
           complex than initially believed. For example, on the NPOESS
           program, DOD and the Department of Commerce committed funds for
           the development and production of satellites before the technology
           was mature--only 1 of 14 critical technologies was mature at
           program initiation and 1 technology was determined to be less
           mature after the contractor conducted more verification testing.
           The program has since been beset by significant cost increases and
           schedule delays due in part to technical problems, such as the
           development of key sensors. On the GPS IIF program, the cost
           estimate was built on the assumption that the military code signal
           being developed would fit on a single microchip. However, once
           development started, interface issues arose and the subcontractor
           had to move to a two-microchip design, which took 8 months to
           resolve and increased cost to the program.

           o Assumptions about TSPR savings: Four programs we examined
           assumed that there would be significant savings associated with
           adopting the TSPR policy. For example, while TSPR was supposed to
           relieve contractors of unnecessary oversight, the government
           assumed that the contractors would still maintain sufficient
           systems engineering and program management levels by following
           standard practices to provide oversight of their subcontractors
           and vendors. However, for a variety of reasons, the savings never
           materialized. For instance, it was believed that by giving more
           program management responsibility to contractors and increasing
           use of commercial equipment, the government could reduce the
           number of in-house systems engineers--who normally help the
           government define its requirements by analyzing differences
           between customer needs and technical possibilities and analyze
           progress in development. Ultimately, the reduction in systems
           engineering staff resulted in cost growth as the programs
           experienced technical and quality problems that the government was
           no longer in a position to detect and prevent. Programs also came
           to realize that commercial parts being relied on were not always
           suitable for their efforts, and had to resort to costly measures
           to address this problem. In addition, in implementing TSPR, the
           government initially entered into contracts that did not allow it
           to obtain certain cost data from the contractors (e.g., contractor
           cost data reports and contractor performance reports), even though
           such data are critical for cost estimators to develop sound cost
           estimates and important for the government to maintain adequate
           insight. This was the case for EELV and GPS IIF--both of which
           have either been restructured or are now planning to issue
           follow-on contracts that will require cost and pricing data and
           earned value management data. It should be noted that the Air
           Force has since recognized problems related to its implementation
           of TSPR and rejected it as a recommended approach.

           o Assumptions about savings from heritage systems: Four programs
           assumed that they would be able to gain leverage from legacy
           satellite systems and save costs, but as the programs continued
           and more knowledge was gained about the requirements and the
           technologies needed to meet the requirements, DOD discovered that
           the legacy systems could not be relied on, as initially believed,
           and the savings were not realized. In addition, SBIRS High and
           WGS, for example, had all planned to gain leverage from commercial
           satellite development efforts because the government had planned
           to use portions of these satellites as lessons already learned in
           order to obtain design savings. However, when hardware and
           software development advances were slowed as a result of the
           Internet sector economic downturn, the government had to carry
           more design and development costs than anticipated.

           o Assumptions about weight growth: Four case study programs
           assumed no weight growth, which is among the highest drivers of
           cost growth for space systems, would occur despite leaps hoped for
           in technology and experiences in past programs. For example, the
           SBIRS High program assumed little to no weight growth, but the
           weight of the satellite spacecraft eventually grew by more than 59
           percent, while payload aboard the spacecraft grew by 44 percent.
           Moreover, with such considerable weight growth, the program could
           no longer rely on the commercial bus it had originally selected
           for this acquisition, and instead had to develop a custom
           satellite bus--a more expensive endeavor.
           o Assumptions about funding: Space programs frequently experienced
           funding shifts. Moreover, at the time the Air Force undertook the
           programs included in our case studies, it was attempting to
           replenish several older satellite constellations, which put
           further stress on its total investment in space. Despite this
           condition, when making estimates on four programs we reviewed,
           cost estimators assumed that program budgets would remain stable.
           As the programs progressed through the acquisition cycle, they
           experienced changes to their funding stream, which created program
           instability and cost growth due to the stopping and starting of
           activities. Cost estimators and program officials we interviewed
           generally agreed that space programs are not often fully funded
           and that their programs have experienced shifts in funding.
           However, they could not separate the ultimate effects of funding
           shifts, since the programs were concurrently experiencing other
           problems, such as technical or design problems, which were also
           adding costs, and these funding cuts led to other decisions that
           had reverberating consequences. For example, in some cases,
           programs abandoned their original plans to purchase satellites in
           one procurement in favor of individual orders in an effort to
           address a funding cut. While this decision enabled the programs to
           continue in the short term, it had significant long-term
           consequences on program costs since the price of each satellite
           substantially increased with the change to individual orders. In
           previous testimony and reports, we have stressed that DOD could
           avoid the need to make costly funding shifts by developing an
           overall investment strategy that would prioritize systems in its
           space portfolio with an eye toward balancing investments between
           legacy systems and new programs as well as between science and
           technology programs and acquisition investments. Such prioritizing
           would also reduce incentives to produce low estimates.

           o Assumptions about schedules: Four case study programs assumed
           that compressed schedules being proposed could be achieved--even
           though the programs were pursuing ambitious leaps in capability or
           attempting new approaches, such as using commercial equipment for
           military purposes. Moreover, in some cases, DOD had data available
           demonstrating such schedules were not realistic. In one case study
           program, WGS, the request for proposals specified that the budget
           available was $750 million for three satellites plus ground
           control with a schedule constraint of 36 months. On the basis of
           these requirements, competing contractors were asked to offer
           maximum capacity, coverage, and connectivity via a contract that
           would make use of existing commercial practices and technologies.
           This aggressive schedule was never achieved. Instead, problems due
           to higher design complexity and supplier quality issues have
           caused the WGS schedule to stretch to 78 months for the first
           expected launch. Historically, the Air Force has required between
           55 and 79 months to build satellites similar to WGS, so while the
           schedule slip is within the expected range, the original 36-month
           schedule was optimistic and not based on realistic data. For AEHF,
           the program accelerated its schedule in response to a potential
           gap in satellite coverage due to the launch failure of the third
           Milstar satellite. However, when the funding needed to achieve the
           acceleration was not delivered, the program experienced cost and
           schedule delays. Again, because these assumptions were made before
           enough information about the development was available, the
           assumptions did not hold up, and the programs experienced cost and
           schedule growth as a result.

           o Assumptions about requirements growth: Three programs--AEHF, GPS
           IIF, and SBIRS High--did not assume any requirements growth, even
           though there was a risk of growth because of the variety of
           stakeholders involved. High-level requirements for the SBIRS High
           program--which is being developed to improve missile warning,
           missile defense, technical intelligence, and battle space
           characterization--have remained stable since the program began,
           but prior DOD studies have found that lower-level requirements
           were in flux and mismanaged until the program was restructured in
           1999. According to DOD studies, this was partially due to the TSPR
           approach, which placed too much responsibility on contractors to
           negotiate these requirements; the broad customer base for SBIRS;
           and the ambitious nature of the program to begin with. To
           illustrate, the SBIRS High program has 19 key performance
           parameters to satisfy-- nearly five times more than the typical
           DOD space program. In addition, there are over 12,600 requirements
           that the program must address, and to date, requirements for
           external users have not been fully defined. DOD has since realized
           that responsibility for setting lower-level requirements should
           rest with the government and has taken actions to add more
           discipline to the requirements-setting process. In another
           example, GPS IIF was intended to follow on to the GPS II program,
           yet shortly after the contract was awarded, the government added
           the requirement for an additional auxiliary payload. This
           requirement caused the satellite design to be larger than
           originally planned, and this, in turn, required a larger launch
           vehicle. Requirements for more robust jamming capability to secure
           satellite transmissions were also added. Changes from a two-panel
           to a three-panel solar array design and flexible power were
           necessary to allow for more power and thermal capability
           requirements.

Appendix III contains additional detailed examples of instances where
program officials were too optimistic in their assumptions for the six
space system acquisitions we examined.

Various Incentives and Pressures within DOD Have Contributed to Cost- Estimating
Weaknesses

Various incentives and pressures within DOD have contributed to optimistic
program office cost estimates for space system acquisitions. As noted
earlier, our prior work has found that programs are incentivized to
produce optimistic estimates in order to gain approval for funding. At
present, DOD does not have a long-term investment strategy that would
prioritize its investments and, in turn, reduce pressures associated with
competition for funding. A 2003 DOD study on crosscutting problems
affecting space acquisitions, known as the Young Panel report, also found
that the space acquisition system, in particular, is strongly biased to
produce unrealistically low cost estimates throughout the process;
advocacy tends to dominate, and a strong motivation exists to minimize
program cost estimates, and proposals from competing contractors typically
reflected the minimum program content and a price to win. In responding to
the Young Panel report as well as our prior reports, DOD officials have
not disputed the need for long-term investment planning or that programs
are incentivized to produce low estimates.

In conducting this review, we asked cost estimators, program managers,
industry officials, and higher-level oversight officials what additional
impediments there were to sound cost estimating for space. Their responses
included that (1) there is little accountability for producing realistic
program office estimates--among both program managers and estimators; (2)
estimates produced within program offices are more often used to set
budgets than estimates produced by independent estimators; (3) even though
space programs experience frequent changes, independent cost estimates are
not updated for years at a time; (4) cost-estimator roles and
responsibilities are not clear and the cost-estimating function is
fragmented; and (5) there are not enough in-house government cost
estimators or sufficient data to support their work.

Accountability Is Lacking

It is difficult for cost estimators to be held accountable for the
estimates they develop because program decision makers are rarely held
accountable for the estimates they use to establish program budgets. This,
coupled with the pressure to compete for funding, invites program
officials to accept optimistic assumptions and ignore risk and reality
when developing cost estimates.

This view was also expressed by many DOD program managers we interviewed
for a 2005 review on program management best practices.^8 While many
program managers told us that they personally held themselves accountable,
many also commented that it is difficult to be accountable when so much is
outside their control. During our focus groups, program managers cited
sporadic instances when program managers were removed from their positions
or forced to retire if programs came in over cost or schedule, but they
also cited instances when a program manager was promoted even though the
program was experiencing difficulties.

Independent Estimates Not Always Relied Upon

We found examples from our closer examinations of the AEHF, NPOESS, and
SBIRS High programs where independent cost estimates were not relied upon
by program decision makers. Independent estimates for these space system
acquisitions forecasted considerably higher costs and lengthier schedules
than program office or service cost estimates. Yet the milestone decision
authorities used program office estimates or even lower estimates instead
of the independent estimates to establish budgets for their programs.
DOD's current space acquisition policy requires that independent cost
estimates be prepared by bodies outside the acquisition chain of command,
and be considered by program and DOD decision makers. However, the policy
does not require that the independent estimates be relied upon to set
budgets, only that they be considered at key acquisition decision points.

           o AEHF: In 2004, AEHF program decision makers relied upon the
           program office cost estimate rather than the independent estimate
           developed by the CAIG to support the production decision for the
           AEHF program--which was more than $2 billion higher. At that time,
           the AEHF program office estimated the system would cost $6
           billion. This was based on the assumption that AEHF would have 10
           times more capacity than the predecessor satellite--Milstar--but
           at half the cost and weight. The CAIG believed that this
           assumption was overly optimistic given that the AEHF weight had
           more than doubled since the program began in 1999 to obtain the
           desired increase in data rate. The latest program office estimate
           for AEHF is $6.1 billion.

^8 GAO, Best Practices: Better Support of Weapon System Program Managers
Needed to Improve Outcomes, [26]GAO-06-110 (Washington, D.C.: Nov. 30,
2005).

Table 2: Comparison of 2004 AEHF Program Office and Independent Cost
Estimates

                      Independent cost                     
                          estimate                         
                                                               Latest 
                                                              program 
Program office                                              office 
estimate       AFCAA                      Difference CAIG estimate 
$6 billion     AFCAA worked jointly     $8.7 billion           44%    $6.1
                  with the CAIG to develop                            billion
                  the independent estimate                               

Source: CAIG and GAO analysis.

Note: Estimates are in fiscal year 2006 dollars.

           o NPOESS: In 2003, to support the NPOESS development decision,
           government decision makers relied on the program office's $7.2
           billion cost estimate rather than the $8.8 billion independent
           cost estimate presented by the Air Force Cost Analysis Agency.
           AFCAA based its estimate on an analysis of historical data from
           satellite systems, independent software and hardware models, and a
           risk simulation model using input from 30 independent engineers.
           The program office relied largely on the contractor's proposal as
           well as on an unrealistic estimate of what it would cost to
           integrate the payloads onto the satellite bus. The program has
           encountered many problems as a result of these optimistic
           assumptions, and costs have risen to $11.4 billion, based on the
           latest program office cost estimate.

Table 3: Comparison of 2003 NPOESS Program Office and Independent Cost
Estimates

                      Independent                
                     cost estimate               
                                                         Latest 
                                                        program 
Program office                                        office 
estimate                  AFCAA Difference CAIG     estimate 
$7.2 billion       $8.8 billion                          23% $11.4 billion
(based on planned                                                (based on
purchase of six                                                    planned
satellites)                                                    purchase of
                                                                         four
                                                                  satellites)

Source: CAIG and GAO analysis.

Note: Estimates are in fiscal year 2006 dollars. The CAIG was not involved
in preparing the 2003 independent cost estimate.

SBIRS High. On the SBIRS High program, the program office and AFCAA
predicted cost growth as early as 1996, when the program was initiated.
While both estimates at that time were close, approximately $5.6 billion,
both were much higher than the contractor's estimated costs. The program
was subsequently estimated to cost $3.6 billion by the program office,
almost $2 billion less than the original AFCAA or program office estimate.
The program office and contractor ultimately assumed savings under TSPR
that did not materialize. For instance, with this approach, the SBIRS High
contractor used far fewer systems engineers than historical data show have
been used for similar programs. To achieve savings, the contractor dropped
important systems engineering tasks such as verification and cycling of
requirements. The lack of systems engineering resulted in latent design
flaws that required more integration and testing when components failed
initial testing.

Table 4: Comparison of 1996 SBIRS High Program Office Cost Estimate and
Independent Cost Estimate

                                                  Total                       
Program office        AFCAA independent      program Latest program office 
estimate                  cost estimate      funding              estimate 
$5.7 billion (based        $5.6 billion $3.6 billion  $10.2 billion (based 
on a planned purchase                                on a planned purchase 
of five satellites)                                   of three satellites) 

Source: AFCAA and Air Force documentation and GAO analysis.

Note: Estimates are in fiscal year 2006 dollars.

We were informed by the CAIG that independent cost estimates are rarely
used by the services to develop budgets for acquisition programs. Because
CAIG estimates are seldom used and the program offices know this,
officials we spoke with believe that there is no incentive on the part of
program offices to change their approach to cost estimating. According to
a senior CAIG official, program managers often promise to meet the maximum
amount of requirements for the least cost. These program officials would
rather rely on optimistic cost estimates from the contractors because
these estimates most likely align with program objectives.

Appendix IV contains detailed examples of where program and
cost-estimating officials disagreed on estimates.

Independent Cost Estimates Not Updated Frequently Enough to Account for
Significant Events and Changes

It is possible for space programs to continue for years--as many as 4
years--without updates of independent cost estimates and to see changes
within that span of time that have had a substantial impact on
cost--including changes in requirements, changes in planned quantities,
funding instability, design changes, quality variances resulting from
rework, manufacturing or engineering changes, changes in supply chain and
logistics management and support, technology-related problems, among
others. At times, the only mechanism that forced an updated estimate was
DOD policy that the CAIG support the Nunn-McCurdy certification process
for programs breaching a certain unit cost threshold.^9 Under this
policy,^10 the CAIG provides the Under Secretary with a recommendation
concerning the reasonableness of the most recent unit cost estimates by
the program.

Because space programs tend to experience such changes after program
start, some officials we spoke with in the DOD space cost-estimating
community believe that independent cost estimates should be updated more
frequently. Opinions differ as to the frequency and phasing of these non
milestone estimates, assessments, or reviews. A CAIG official suggested
updating cost estimates about every 18 to 24 months, while AFCAA officials
suggested annually to correspond with the annual budgeting cycle. The
current space acquisition policy requires only one independent cost
estimate after critical design review, but CAIG officials noted that years
can go by between critical design review and program completion, during
which time programs have historically experienced substantial changes.

^9 10 U.S.C. S 2433. This oversight mechanism originated with the
Department of Defense Authorization Act, 1982. It was made permanent in
the following year's authorization act and has been amended several times.
Generally, the law requires DOD to review programs and report (and in some
cases submit a certification) to Congress whenever cost growth reaches
specified thresholds. The statute is commonly known as Nunn-McCurdy, based
on the names of the sponsors of the original legislation.

^10 DOD Directive 5000.04 at P 4.8.

Figure 1 illustrates significant changes that took place on the SBIRS High
program both before and after critical design review.

Figure 1: Key Events and Funding Shifts That Occurred between Estimates
for SBIRS High

Cost-Estimating Roles and Responsibilities Are Unclear

Air Force cost-estimating officials believe that their roles and
responsibilities are not clear and that the cost-estimating function is
too fragmented. Some also asserted that the cost-estimating function
within the space community would be stronger if estimators themselves were
centralized outside the acquisition chain of command so that they would
not be biased or pressured by program office leadership to produce
optimistic estimates.

In an attempt to make the most efficient use of the limited cost estimate
expertise for DOD space system acquisitions, the space acquisition policy
called on the CAIG to augment its own staff with cost-estimating personnel
drawn from across the community to serve as team members when it developed
independent estimates. Members were to include the intelligence
community's cost analysis improvement group, the Air Force Cost Analysis
Agency, the National Reconnaissance Office (NRO) Cost Group, the Office of
the Deputy Assistant Secretary of the Army for Cost and Economics, the
Naval Center for Cost Analysis, the cost-estimating organizations of the
Air Force Space Command, Air Force Space and Missile Systems Center, and
the Space and Naval Warfare Systems Command.

At this time, however, there are still significant disconnects in views
about roles and responsibilities. Officials who reside in the acquisition
chain of command--the Air Force Space and Missile Systems Center--believe
that because the program executive officer and the program managers are
responsible for executing the programs, they are also solely responsible
for the cost estimates for the program. On the other hand, Air Force cost
estimators outside the acquisition chain of command--the Air Force Cost
Analysis Agency--believe they also hold some responsibility to ensure the
quality and consistency of cost estimates and to produce independent cost
estimates for consideration by Air Force decision makers. However,
according to officials within the Space and Missile Systems Center's (SMC)
cost-estimating group and AFCAA, the SMC cost-estimating group sees no
role for AFCAA in developing program or Air Force cost estimates and has
rejected assistance from AFCAA. According to Air Force officials, until a
clearer distinction of roles and responsibilities is defined by Air Force
leadership, issues of conflicting policy interpretation and implementation
will remain. It is also possible that these disconnects have been
exacerbated by the perception that these two communities are competing for
responsibility.

In addition, according to a senior CAIG official, the collaborative
process for developing independent estimates has not been achieved as
envisioned--principally because those who should be involved have not seen
their involvement as a priority, and those who have been involved have
required a lot of extra training to be able to make valuable
contributions. Moreover, because the various cost-estimating organizations
each have different customers, agendas, and approaches to developing cost
estimates, these differences have made it difficult for them to work as a
cohesive team.

Cost-Estimating Resources Are Considered Inadequate

Air Force space cost-estimating organizations and program offices believe
that cost-estimating resources are inadequate to do a good job of
accurately predicting costs. They believe that their cost-estimating
resources have atrophied over the years because of previous downsizing of
the workforce, making resources such as staff and data inadequate.

As noted earlier, there was a belief within the government that cost
savings could be achieved under acquisition reform initiatives by reducing
technical staff, including cost estimators, since the government would be
relying more on commercial-based solutions to achieve desired
capabilities. According to one Air Force cost-estimating official we spoke
with, this led to a decline in the number of Air Force cost estimators
from 680 to 280. High-grade positions and specialty cost-estimating job
codes were eliminated, abolishing an official cost-estimating career path,
and subordinating cost estimating as an additional duty. In the process,
according to this same Air Force official, many military and civilian
cost-estimating personnel left the cost-estimating field, and the Air
Force lost some of its best and brightest cost estimators.

Information we obtained from space program offices and cost-estimating
organizations is consistent with the assertion of a lack of requisite
resources. Eight of 13 cost-estimating organizations and program offices
we informally surveyed believe the number of cost estimators is
inadequate. Furthermore, some of these same organizations believe that
cost estimation is not a respected career field within the Air Force, and
more specifically, that Air Force cost estimators are not encouraged, nor
do they have opportunities for promotion or advancement. Regarding the
recognition and career paths for cost estimators, our data showed that
only 3 of 12 organizations agreed that previous cost estimators had moved
on to positions of equal or higher responsibility. Further, only 4 of 12
agreed that people ask to become cost estimators.

The belief that cost-estimating skills have been depleted has been echoed
in other DOD and GAO studies. According to the Young Panel report,
government capabilities to lead and manage the acquisition process have
seriously eroded, in part because of actions taken in the acquisition
reform environment of the 1990s. This has extended to cost estimating.
During our 2005 review of program management, we surveyed DOD's major
weapon system program managers and interviewed program executive officers
who similarly pointed to critical skill shortages for staff that support
them, including cost estimators. Other skill gaps identified included
systems engineering, program management, and software development. We
continue to observe these deficiencies in our more recent reviews of the
space acquisition workforce.^11

Because of the decline in in-house cost-estimating resources, space
program offices and Air Force cost-estimating organizations are now more
dependent on support contractors. Ten of 13 cost-estimating organizations
and program offices have more contractor personnel preparing cost
estimates than government personnel. At 11 space program offices,
contractors account for 64 percent of cost-estimating personnel. Support
contractor personnel generally prepare cost estimates, while government
personnel provide oversight, guidance, and review of the cost-estimating
work. By contrast, the CAIG had made a determination that cost estimating
is too important of a function to place in the hands of support
contractors, and assigns only government personnel to develop cost
estimates.

Reliance on support contractors raises questions from the cost-estimating
community about whether numbers and qualifications of government personnel
are sufficient to provide oversight of and insight into contractor cost
estimates. A senior CAIG official involved with estimating for space
acquisition programs, for example, suggested that reliance on support
contractors is a problem if the government cannot evaluate how good a cost
estimate is or lacks the ability to track it. Two studies have also raised
the concern that relying on support contractors makes it more difficult to
retain institutional knowledge and instill accountability. Further, in the
most recent defense authorization act, Congress is requiring DOD to make
it a goal that within 5 years certain critical acquisition functions,
including cost estimating, be performed by properly qualified DOD
employees, and that in developing a comprehensive strategy for supporting
the program manager role, DOD address improved resources and support such
as cost-estimating expertise.^12

11 GAO, Defense Space Activities: Management Actions Are Needed to Better
Identify, Track, and Train Air Force Space Personnel, [27]GAO-06-908
(Washington, D.C.: Sept. 21, 2006), and Defense Acquisitions: DOD Needs to
Establish an Implementing Directive to Publish Information and Take
Actions to Improve DOD Information on Critical Acquisition Positions,
[28]GAO-06-987R (Washington, D.C.: Sept. 8, 2006).

A second resource gap hampering cost estimating is the lack of reliable
technical source data. Officials we spoke with believe that cost
estimation data and databases from which to base cost estimates are
incomplete, insufficient, and outdated. They cite a lack of reliable
historical and current cost, technical, and programmatic data and
expressed concerns that available cost, schedule, technical, and risk data
are not similar to the systems they are developing cost estimates for. In
addition, some expressed concerns that relevant classified and proprietary
commercial data may exist but are not usually available to the
cost-estimating community working on unclassified programs. Some believe
that Air Force cost estimators need to be able to use all relevant data,
including those contained in NRO cost databases, since the agency builds
highly complex, classified satellites in comparable time and at comparable
costs per pound.

Successful Organization Approaches That Better Support Cost Estimating

Over the past decade, GAO has examined successful organizations in the
commercial sector to identify best practices that can be applied to weapon
system acquisitions. This work has identified a number of practices that
better support cost estimating than DOD does. For instance, unlike most
space programs we have reviewed, the successful organizations we have
studied extensively researched and defined requirements before program
start to ensure that they are achievable, given available resources. They
do not define requirements after starting programs. They also ensure
technologies are mature, that is, proven to work as intended, and assign
more ambitious efforts to corporate research departments until they are
ready to be added to future increments. In addition, these organizations
use systems engineering to close gaps between resources and requirements
before launching the development process. Taken together, these practices
help ensure that there is little guessing in how long or how many dollars
it will take to achieve an intended capability. Moreover, within the
organizations we studied, decisions to start programs are made through
long-term strategic planning and prioritizing. As a result, competition
for funding is minimized, and programs themselves do not have incentives
to present low estimates.

^12 John Warner National Defense Authorization Act for Fiscal Year 2007,
Pub. L. No. 109-364 SS 820, 853 (2006).

The successful organizations we have studied have taken additional steps
to ensure cost estimates are complete and accurate that DOD has not. For
instance, they hold program managers accountable for their estimates and
require program managers to stay with a project to its end. At the same
time, they develop common templates and tools to support data gathering
and analysis and maintain databases of historical cost, schedule, quality,
test, and performance data. Cost estimates themselves are continually
monitored and regularly updated through a series of numerous gates or
milestone decisions that demand programs assess readiness and remaining
risk within key sectors of the program as well as overall cost and
schedule issues.

Senior leaders within these organizations also actively encourage program
managers to share bad news about their programs and spend a great deal of
time breaking down stovepipes and other barriers to sharing information.
More important, they commit to fully funding programs and adhere to those
commitments. Commonly, the organizations we studied have centralized cost
estimators and other technical and business experts so that there is more
effective deployment of technical and business skills while at the same
time ensuring some measure of independence. Within DOD, the CAIG is a good
example of this. Its cost estimates are produced by civilian government
personnel (the sole military space cost estimating position will convert
to a civilian position later on this year when the military cost estimator
retires), to ensure long-term institutional knowledge and limit the
effects of staff turnover that commonly occur with military personnel.
Although the CAIG uses support contractors for conducting studies, it does
not allow cost estimates to be developed by contractors. The CAIG takes
this approach because it considers cost estimating to be a core function
and therefore too important to contract out. The Naval Air Systems
Command's Cost Analysis Division is also considered a model by some in the
cost-estimating community because of its organizational structure and
leadership support. It is a centralized cost department that provides
support to multiple program offices. The department is headed by a senior
executive-level manager, and various branches within the department are
headed by GS-15-level managers. Analysts are somewhat independent of the
program offices, as their supervisors are within the engineering
department. This cost department has strong support from its leadership,
and this support has helped it hire the number of analysts and receive the
resources it needs. However, another official pointed out that this cost
department is not completely independent from the acquisition chain of
command, since it receives funding from the program offices to conduct the
cost estimates.

GAO has made recommendations to DOD to adopt best practices we have
identified that would strengthen program management DOD-wide. Congress
also recently directed DOD to develop a strategy to enhance program
manager empowerment and accountability, agreeing with GAO's assessment
that DOD has consistently failed to give program managers the authority
that they need to successfully execute acquisition programs and, as a
result, is unable to hold them accountable.^13 GAO has also made
recommendations to the Air Force to better position its space programs for
success. In response, the Air Force has restructured its Transformational
Satellite Communications System (TSAT) to ensure that the program
incorporates technologies that have been proven to work as intended, and
it has deferred more ambitious efforts to the science and technology
community. It has committed to do the same on other programs. If
effectively implemented, such actions would, in turn, significantly
enhance the ability of independent estimators to forecast costs. However,
we have testified that DOD faces a number of challenges and impediments in
its effort to instill this approach. It needs significant shifts in
thinking about how space systems should be developed, changes in
incentives and perceptions; and further policy and process changes. And
such changes will need to be made within a larger acquisition environment
that still encourages a competition for funding and consequently pressures
programs to view success as the ability to secure the next installment
rather than the end goal of delivering the capabilities when and as
promised.

The Air Force has also been taking actions to make specific improvements
to cost estimating for space programs. In the case of TSAT, program
officials said they are updating the program's planning cost estimate on
an annual basis. Furthermore, according to one CAIG official, some program
offices have recently been using the CAIG's independent cost estimates.
Both the SBIRS High and NPOESS program offices are developing their
budgets based on the CAIG independent estimates that support the
certification process for the programs' most recent Nunn-McCurdy breaches.
Further, DOD and Air Force cost estimators we spoke to recognize that
amendments made to the Nunn-McCurdy law by the 2006 Defense Authorization
Act may increase realism in establishing initial cost estimates. As part
of the revisions, DOD is barred from changing its original baseline cost
estimate for a program until after it has breached certain Nunn-McCurdy
thresholds that require a certification and assessment of the program, and
DOD must report the baseline changes to Congress.

^13 John Warner National Defense Authorization Act for Fiscal Year 2007,
Pub. L. No. 109-364, S 853 (2006), and accompanying conference report,
H.R. Rep. No. 109-702, pages 784-785.

The Air Force has also committed to strengthening its cost-estimating
capabilities in terms of people, methodologies, and tools. For instance,
50 new cost estimators have recently been authorized to the AFCAA, some of
whom may be detailed to the Space and Missile Systems Center. Finally, key
players within the DOD space cost-estimating community are meeting on a
regular basis to discuss issues, review recent findings from GAO and other
groups, and explore lessons learned and potential ideas for improvement.

Conclusions

Costs for DOD space acquisitions over the past several decades have
consistently been underestimated--sometimes by billions of dollars. For
the most part, this has not been caused by poor cost estimating itself,
but rather the tendency to start programs before knowing whether
requirements can be achieved within available resources. In fact, with so
many unknowns about what could be achieved, how, and when, even the most
rigorous independent cost estimate could have been off by a significant
margin. Nevertheless, in the past, the Air Force has exacerbated
acquisition problems by not relying on independent cost estimates and
failing to encourage more realism in program planning and budgeting.
Moreover, even after the Air Force embraced independent cost estimating in
its acquisition policy for space, it did not facilitate better estimating
by according the cost-estimating community with the organizational clout,
support, and guidance the Air Force believes are needed to ensure the
community's analyses are used. On a positive note, the Air Force has
committed to addressing some of the root causes behind cost growth,
principally by accumulating more knowledge about technologies before
starting new programs. Though adopting this approach will be challenging
without larger DOD acquisition, funding, and requirement-setting reforms,
the Air Force can facilitate better planning and funding approaches by
aligning resources and policy to support improved cost-estimating
capability and by following through on its commitment to use independent
estimates.

Recommendations for Executive Action

We recommend that the Secretary of Defense direct the Under Secretary of
Defense for Acquisition, Technology and Logistics or the Secretary of the
Air Force, as appropriate, to take the following actions:

           1. To increase accountability and transparency of decisions in
           space programs where an independent estimate produced by the CAIG
           or AFCAA is not chosen, require officials involved in milestone
           decisions to document and justify the reasons for their choice and
           the differences between the program cost estimate and the
           independent cost estimate.
           2. To better ensure investment decisions for space programs are
           knowledge-based, instill processes and tools necessary to ensure
           lessons learned are incorporated into future estimates. This could
           include

                        o conducting postmortem reviews of past space program
                        cost estimates (program office and independent cost
                        estimates) to measure cost-estimating effectiveness
                        and to track and record cost-estimating mistakes;
                        o developing a centralized cost-estimating database
                        that provides realistic and credible data to cost
                        estimators;
                        o establishing protocols by which cost estimators
                        working with the National Reconnaissance Office can
                        share data with the DOD space cost-estimating
                        community while still maintaining appropriate
                        security over classified data; and
                        o ensuring estimates are updated as major events
                        occur within a program that could have a material
                        impact on cost, such as budget reductions,
                        integration problems, hardware/software quality
                        problems, and so forth.

           3. To optimize analysis and collaboration within the space
           cost-estimating community, clearly articulate the roles and
           responsibilities of the various Air Force cost-estimating
           organizations, and ensure that space system cost estimators are
           organized so that the Air Force can gain the most from their
           knowledge and expertise. In taking these actions for programs for
           which no independent estimate is developed by the CAIG, consider
           assigning AFCAA the responsibility for the development of
           independent cost estimates for space system acquisitions, since it
           is outside of the acquisition chain of command and therefore
           likely to be unbiased and not pressured to produce optimistic
           estimates.

Agency Comments and Our Evaluation

DOD provided us with written comments on a draft of this report. DOD
concurred with the overall findings in our report and provided technical
comments, which have been incorporated where appropriate. DOD also
concurred with two of our recommendations and partially concurred with
one.

DOD concurred with our recommendation to instill processes and tools
necessary to ensure lessons learned are incorporated into future
estimates. DOD stated it was already taking actions to address our
recommendations. For example, the CAIG has established a process whereby
key members of the national security space cost analysis community meet to
discuss and evaluate outcomes following ACAT I space program milestone
reviews or key decision point Defense Acquisition Board-level reviews, to
provide visibility to other members of the community on how the CAIG
approaches independent cost estimate development and to give the community
an opportunity to provide feedback to the CAIG on how to improve its
processes. DOD stated that the CAIG will work in the future to incorporate
peer reviews of the program office estimates within this existing
framework. DOD also concurred with our recommendation to develop a
centralized cost-estimating database, and stated that several groups
within the space cost-estimating community have been working to develop a
database of historical space program costs available to the community as a
whole, and has also reestablished a common space program work breakdown
structure that supports the various estimating methodologies employed by
the space cost community. Through the common database development process,
the community is working to make historical program cost data as widely
available as possible. DOD also agreed with our recommendation to update
cost estimates as major events occur within a program, as long as they are
program and program phase dependent. Finally, DOD concurred with our
recommendation to clearly articulate the roles and responsibilities of the
various cost-estimating organizations. DOD stated that the Air Force is
currently updating its policy directive to further clarify the roles and
responsibilities of the space cost analysis organizations to optimize
analysis and collaborations, thus making the best use of the limited
number of qualified and experienced space program cost analysts. We agree
that these actions are steps in the right direction and that they will
strengthen cost-estimating capabilities and improve space program cost
estimates.

DOD partially concurred with our recommendation to require officials
involved in milestone decisions to document and justify the reasons for
their cost estimate choice and the differences between the program cost
estimate and the independent cost estimate. In commenting on this
recommendation, DOD stated that the complex decision to determine which
cost figure to use as basis for funding and to evaluate future program
performance must weigh many competing factors that are often qualitative
in nature. It further stated that the decision is the milestone decision
authority's alone, and that documenting the explicit justification will
reduce the milestone decision authority's future decision-making
flexibility. We do not see how documenting the explicit justification will
significantly reduce the milestone decision authority's future
decision-making flexibility. While we recognize the value of
decision-making flexibility and the role that judgment must play in such
decisions, we also believe that the basis for the decisions should
withstand review, particularly after the person who made the decision has
left office. We also believe that the greater transparency of
cost-estimating decisions that a documented justification provides is
needed, particularly in light of the poor foundation of choices made in
the past on space programs.

We are sending copies of this report to interested congressional
committees and the Secretaries of Defense and the Air Force. We will also
provide copies to others on request. In addition, this report will be
available at no charge on the GAO Web site at [29]http://www.gao.gov .

If you have any questions about this report or need additional
information, please call me at (202) 512-4841 ( [30][email protected] ).
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who made
major contributions to this report are listed in appendix VI.

Cristina T. Chaplain
Acting Director, Acquisition and Sourcing Management

Appendix I: Scope and Methodology

The Chairman and the Ranking Member, Subcommittee on Strategic Forces,
House Committee on Armed Services, requested that we examine (1) in what
areas space system acquisitions cost estimates have been unrealistic and
(2) what incentives and pressures have contributed to the quality and
usefulness of cost estimates for space system acquisitions.

To determine whether cost estimates for space system acquisitions have
been realistic, we used a case study methodology. We selected six ongoing
Air Force space system acquisitions. We selected these acquisitions
because they were far enough along in their acquisition cycles for us to
be able to observe changes in the programs since their initial cost
estimates were developed. The six space system acquisitions are the
Advanced Extremely High Frequency Satellites, the Evolved Expendable
Launch Vehicle, the Global Positioning System IIF, the National
Polar-orbiting Operational Environmental Satellite System, the Space Based
Infrared System High, and the Wideband Gapfiller Satellites. For each of
the case studies, we met with the program office representatives at the
Air Force's Space and Missile Systems Center and at the program's prime
contractors. We also obtained program cost and other program documentation
to determine how the cost estimates were formulated and on what basis they
were formulated.

To determine what incentives and pressures contributed to the quality and
usefulness of cost estimates for space system acquisitions, we examined
Department of Defense (DOD) and Air Force policies for developing and
updating cost estimates for space programs. We also used a data collection
instrument to obtain information on cost-estimating practices and
resources within the Air Force Cost Analysis Agency, at the Space and
Missile Systems Center, and at the space program offices. We conducted
interviews with the Office of the Secretary of Defense's Cost Analysis
Improvement Group, the Air Force Cost Analysis Agency, and the Air Force
Space and Missile Systems Center's Cost Center. On the basis of the
results of the data collection instruments and interviews, we obtained
information on the organizational alignment of cost-estimating
organizations, including roles and responsibilities, as well as concerns
over the current cost-estimating policies and practices.

We also relied on our previous best practice studies, which have examined
pressures and incentives affecting space system acquisition programs, the
optimal levels of knowledge needed to successfully execute programs, and
complementary management practices and processes that have helped
commercial and DOD programs to reduce costs and cycle time. Moreover, we
reviewed studies from the Defense Science Board, the DOD Inspector
General, IBM, and others on space system acquisition and cost-estimating
issues.

Finally, we discussed the results of our work and our observations with an
expert panel made up of representatives from the DOD space cost-estimating
community.

We conducted our review between August 2005 and October 2006 in accordance
with generally accepted government auditing standards.

Appendix II: DOD Acquisition Categories for Major Defense Acquisition
Programs

An acquisition program is categorized based on dollar value and milestone
decision authority special interest. Table 5 contains the description and
decision authority for acquisition categories ID and IC.

Table 5: DOD Acquisition Categories and Decision Authorities

                                                        Milestone decision    
Acquisition category (ACAT)      Dollar value        authority             
ACAT ID                          Research,           Under Secretary of    
                                    development, test,  Defense for           
For designated major defense     and evaluation >    Acquisition,          
acquisition programs (special    $365 million        Technology and        
interest based on technological                      Logistics             
complexity, congressional        Procurement > $2.19                       
interest, large commitment of    billion                                   
resources, critical role in                                                
achieving a capability, or a                                               
joint program)                                                             
ACAT IC                          Research,           Head of DOD component 
                                    development, test,  or, if delegated, DOD 
For major defense acquisition    and evaluation >    component or service  
programs not designated as ACAT  $365 million        acquisition executive 
ID                                                                         
                                    Procurement > $2.19                       
                                    billion                                   

Source: DOD Instruction 5000.2, Enclosure 2, which also lists other
acquisition categories.

Note: Dollar values are fiscal year 2000 constant dollars.

Appendix III: Examples of Where Program Officials Were Too Optimistic in
Their Assumptions

Table 6 highlights major areas where program officials were too optimistic
in their assumptions for the six space system acquisitions we examined--
the Advanced Extremely High Frequency (AEHF) Satellites, the Evolved
Expendable Launch Vehicle (EELV), the Global Positioning System (GPS) IIF,
the National Polar-orbiting Operational Environmental Satellite System
(NPOESS), the Space Based Infrared System (SBIRS) High, and the Wideband
Gapfiller Satellites (WGS).

Table 6: Examples of Optimistic Assumptions

Space program                                                              
affected         Examples                                                  
Assumed industrial base would remain constant and available
EELV             The original contracting concept was for the Air Force to 
                    piggyback on the launch demand anticipated to be          
                    generated by the commercial sector. However, the          
                    commercial demand never materialized, and the government  
                    had to take on an additional cost burden. In addition,    
                    the cost for launch services increased because fixed      
                    infrastructure costs are being spread over 15 launches a  
                    year instead of the original expectation of 75 launches a 
                    year.                                                     
GPS IIF          A deteriorating manufacturing base of contractors and     
                    subcontractors caused the prime contractor to move the    
                    design team from Seal Beach, California, to Anaheim,      
                    California, in 2001. Additional moves occurred as the     
                    prime contractor consolidated development facilities to   
                    remain competitive. For each move, the prime contractor   
                    lost valuable workers, causing inefficiencies in the      
                    program. In addition, the contractor took additional      
                    cost-cutting measures that reduced quality.               
NPOESS           A long production phase on this program increases the     
                    probability for parts obsolescence. Over 70 percent of    
                    the value added to the program is from the supply base,   
                    and some critical parts that are unique to the program    
                    are produced by relatively small companies. In addition,  
                    workers required to have specialized skills must be       
                    United States citizens to obtain security clearances. The 
                    labor pool has to produce these specialized skills        
                    because degree programs currently do not produce them.    
SBIRS High       Consolidation within the supplier base has adversely      
                    affected the program. When suppliers merged, costs        
                    increased for supplier technical assistance, product      
                    rework, and hardware qualifications. In addition,         
                    unforeseen costs resulted when production processes and   
                    materials were changed and facilities and personnel were  
                    relocated.                                                
WGS              At the time of contract award, the satellite industry was 
                    flourishing with commercial satellite orders, and the     
                    contractor anticipated a large market. However, when the  
                    installation of optical fiber communication lines became  
                    widespread, many of the commercial initiatives involving  
                    proposed space systems did not materialize. The           
                    government had planned to gain leverage from the design   
                    work of commercial contractors but ended up having to pay 
                    for design efforts. In addition, because of the reduction 
                    of the number of contracts awarded, small subcontractors  
                    started to consolidate. Specialized parts became          
                    obsolete, and the Air Force was no longer considered a    
                    high-priority customer.                                   
Assumed technology would be mature enough when needed
AEHF             AEHF faced several technology maturity problems including 
                    developing a digital processing system that would support 
                    10 times the capacity of Milstar medium data rate without 
                    self-interference, and using phased array antennas at     
                    extremely high frequencies, which had never been done     
                    before. In addition, the change from a physical to an     
                    electronic process for crypto re-keys was not expected at 
                    the start of the AEHF. The predecessor program to AEHF    
                    was Milstar, which required approximately 2,400 crypto    
                    re-keys per month, which could be done physically.        
                    Regarding AEHF proposed capabilities, the number of       
                    crypto re-keys is approximately 100,000, which is too     
                    large for a physical process and must be done             
                    electronically. Changing the way the re-keys were done    
                    called for a revolutionary change in the process and led  
                    to unexpected cost and schedule growth.                   
GPS IIF          The cost estimate was built on the assumption that the    
                    military code being developed in the program would fit on 
                    one chip. However, once development started, there were   
                    interface issues, and the subcontractor had to move to a  
                    two-chip design, which added cost growth to the program.  
                    In addition, the problem took 8 months to solve.          
NPOESS           DOD and the Department of Commerce committed funds for    
                    the development and production of the satellites before   
                    the design was proven and before the technology was       
                    mature. At program initiation, only 1 of 14 critical      
                    technologies was mature, and some technology levels have  
                    been assessed downward. For example, the 1394 Bus         
                    Technology Readiness Level (TRL) was changed from 5 to 4  
                    after the contractor added more verification testing.     
SBIRS High       In 2003, GAO reported that three critical                 
                    technologies--the infrared sensor, thermal management,    
                    and onboard processor--were now mature. When the program  
                    began, in 1996, none of its critical technologies was     
                    mature.                                                   
WGS              The X-band phased array antennas and the array power      
                    chips were the most difficult technologies to mature,     
                    because these state-of-the-art elements generated too     
                    much heat, which is very difficult to remove in outer     
                    space, so they had to be redesigned.                      
Assumed Total System Performance Responsibility (TSPR) would reduce costs
and schedule     
EELV             The EELV program office entered into a TSPR contract that 
                    does not require the contractor to deliver cost or earned 
                    value management data. The program office stated that     
                    TSPR gave too many responsibilities to the contractor and 
                    not enough to the government.                             
GPS IIF          The contract that was awarded during acquisition reform   
                    efforts of the late 1990s adopted the TSPR approach.      
                    Under TSPR, there was limited oversight of the            
                    contractor, and this contributed to relaxed               
                    specifications and inspections on commercial practices,   
                    loss of quality in the manufacturing process, and         
                    poor-quality parts that caused test failures, unexpected  
                    redesigns, and the late delivery of parts.                
NPOESS           The NPOESS prime contractor has a Shared System           
                    Performance Responsibility (SSPR), which was an outgrowth 
                    of TSPR. The SSPR arrangement relegates the government's  
                    role as a participant in contractor Integrated Product    
                    Team meetings. In addition, the program is managed by     
                    officials from three separate government agencies. DOD    
                    and Department of Commerce share the cost of funding the  
                    development of NPOESS, while NASA provides funding for    
                    specific technologies and studies. Difficulties have      
                    arisen with the tri-service approach to managing NPOESS,  
                    including ensuring NPOESS follows DOD's acquisition       
                    process, but Commerce, which has control over the         
                    program, has no authority over the DOD process; each      
                    agency is driven by different program objectives (i.e.,   
                    military, civilian, science); and NASA shares equally in  
                    managing the program even though it provides no funding   
                    for the development.                                      
SBIRS High       When the original contract was awarded, acquisition       
                    reform efforts were being implemented and called for the  
                    use of commercial practices instead of government         
                    standards. In order to achieve cost savings, the SBIRS    
                    program office reduced critical up-front systems          
                    engineering design practices and follow-on quality        
                    assurance inspections based on the expectation that the   
                    contractor would perform these activities with no         
                    government oversight. The prime contractor also held the  
                    same requirements for its subcontractors as a way to keep 
                    costs down. This lack of oversight resulted in            
                    difficulties in determining the root causes when          
                    components began to fail during testing. For example,     
                    there have been latent defects that required extensive    
                    corrective action and associated cost growth with the     
                    software redesign, single board computer halts, payload   
                    reference bench rework, payload electromagnetic           
                    interference, software configuration issues, propulsion   
                    solder issues, and telescope foreign object damage. In    
                    addition, the contractor had responsibility to coordinate 
                    different agency needs, a responsibility that proved to   
                    be difficult when trying to resolve hardware interface    
                    issues.                                                   
Assumed savings from heritage systems
AEHF             The program office cost estimators relied on data from    
                    heritage systems to estimate AEHF nonrecurring costs. The 
                    Cost Analysis Improvement Group (CAIG) believed the       
                    estimates based on heritage data were subjectively        
                    derived and therefore susceptible to bias. For example,   
                    AEHF program officials assumed that the nulling antennas  
                    would have the same performance as those on Milstar,      
                    requiring little if any development. In fact, because of  
                    parts obsolescence, personnel turnover, and other issues, 
                    the entire antenna had to be redesigned at nearly the     
                    same cost as the first one. There were similar beliefs    
                    that legacy processing technology could be used, which    
                    turned out to not be possible. Further, almost all of the 
                    payload software had to be rewritten to support the new   
                    hardware. As a result, there was much less technology     
                    transfer from Milstar II to AEHF, even though the         
                    contractor was the same.                                  
NPOESS           NPOESS payload development proposals relied heavily on    
                    leveraging heritage satellite instrument technology       
                    development. The prime contractor and the program office  
                    agreed there was too much optimism regarding heritage     
                    sensor reuse. For example, the Visible Infrared           
                    Radiometer Suite (VIIRS) is more powerful and complex and 
                    will weigh 20 percent more than the heritage sensor that  
                    was used to base the estimate. In addition, the Conical   
                    Microwave Imager Sounder (CMIS) is much more complex than 
                    the heritage sensor, which took more than 8 years to      
                    develop. The program office estimated a 4-year            
                    development schedule for CMIS. The latest cost estimate   
                    for CMIS is now approximately five times the initial      
                    estimate.                                                 
SBIRS High       The original estimate for nonrecurring engineering was    
                    significantly underestimated based on actual experience   
                    in legacy sensor development and assumed software reuse.  
                    As a result, nonrecurring costs should have been two to   
                    three times higher according to historical data and       
                    independent cost estimators.                              
WGS              Originally, the contractor planned to gain leverage from  
                    a commercial satellite development effort--using the same 
                    bus and phased array antenna. The commercial satellite    
                    development effort did not materialize, leaving DOD to    
                    pay for infrastructure and hardware design costs. This    
                    caused WGS costs to increase and the schedule to slip.    
Assumed no weight growth would occur
AEHF             When the cost estimate was initially developed, satellite 
                    payload weight was assumed to be constant by the program  
                    office. When updating its independent cost estimate in    
                    2004, the CAIG found that the payload weight more than    
                    doubled between the start of development and critical     
                    design review. Weight increased because of the addition   
                    of phased array antennas, an antenna modification, and    
                    other requirements.                                       
NPOESS           The CMIS sensor weight has almost doubled since the       
                    preliminary design review. As a result, engineering       
                    change proposals were issued to modify the spacecraft to  
                    accept the higher payload weight.                         
SBIRS High       Weight growth has occurred in the spacecraft and payload. 
                    The spacecraft has experienced weight growth of about 59  
                    percent because of the need to lengthen and stiffen the   
                    structure, add a solar shield to block sunlight from the  
                    payload, and add missing wire and harnessing. The         
                    geosynchronous earth orbit (GEO) payload has experienced  
                    nearly a 44 percent weight growth because of integration  
                    hardware, pointing, and control assembly.                 
WGS              Problems with solar panel concentrators overheating       
                    caused a solar panel redesign that led to additional      
                    weight growth in the spacecraft bus.                      
Assumed funding stream would be sufficient and remain stable
AEHF             The AEHF program sustained a $100 million fiscal year     
                    2002 funding cut. The program office reported that the    
                    funding cut would result in a 6-month launch delay to the 
                    first three satellites and a delay in meeting initial     
                    operational capability. The program had rapidly staffed   
                    personnel to support a warfighter need. The funding cut   
                    resulted in contractor program reductions to fit within   
                    the revised fiscal year 2002 budget. In addition, DOD     
                    made a decision to shift the acquisition strategy from    
                    buying five satellites at one time to buying three        
                    satellites as individual buys, which also caused costs to 
                    rise.                                                     
GPS IIF          The Operational Control Segment portion of the GPS IIF    
                    program received a $37.7 million funding cut in fiscal    
                    year 2005. Because of the funding cut, the program        
                    delayed some of the software efforts and reduced some     
                    software requirements.                                    
NPOESS           Between fiscal years 2004 and 2005, DOD reduced funding   
                    for the program by about $65 million. However, funding    
                    was reduced $130 million since the Department of Commerce 
                    contributes no more funding towards the program than DOD. 
                    The program office determined that the funding cut        
                    resulted in satellite launch delays ranging from 5 to 26  
                    months and a cost increase of $391.2 million.             
SBIRS High       A funding cut in 1998-1999 because of higher budget       
                    priorities caused a reduction in the systems engineering  
                    staff and contributed to a 2-year delay of the            
                    geosynchronous earth orbit satellites. This cut caused    
                    work activities to continually stop and restart and drove 
                    the need for interim solutions that resulted in program   
                    instability and cost growth. It also led to a breach of   
                    the acquisition program baseline in 2001, resulting in a  
                    change in the procurement strategy from a single buy of   
                    five satellites to two separate buys--one for two         
                    satellites and the other for three satellites.            
                    Independent cost estimators calculated that costs would   
                    double as a result of the change in procurement strategy. 
Assumed an aggressive schedule
AEHF             The first launch was originally scheduled for June 2006,  
                    but in response to a potential gap in satellite coverage  
                    due to the launch failure of the third Milstar satellite, 
                    DOD accelerated the schedule by 18 months, aiming for a   
                    first launch in December 2004. An unsolicited contractor  
                    proposal stated that it could meet the accelerated date,  
                    even though all the requirements for AEHF were not fully  
                    determined. As a result, the program office knew that the 
                    proposed schedule was overly optimistic, but the decision 
                    was made at high levels in DOD to award the contract.     
                    However, DOD did not commit the funding to support the    
                    activities and manpower needed to design and build the    
                    satellites more quickly. Funding issues further hampered  
                    development efforts and increased schedule delays and     
                    contributed to cost increases.                            
NPOESS           When the estimate was developed, NPOESS was expected to   
                    be heavier, require more power, and have over twice as    
                    many sensors than heritage satellites. Yet the program    
                    office estimated that the satellites would be developed,  
                    integrated, and tested in less time than heritage         
                    satellites. Independent cost estimators highlighted to    
                    the NPOESS program office that the proposed integration   
                    schedule was unrealistic when compared to historical      
                    satellite programs. Later, the CAIG cautioned the program 
                    office that not only was the system integration assembly  
                    and test schedule unrealistic, but the assumptions used   
                    to develop the estimate were not credible.                
SBIRS High       The schedule proposed in 1996 did not allow sufficient    
                    time for geosynchronous earth orbit system integration    
                    and did not anticipate program design and workmanship     
                    flaws, which eventually cost the program considerable     
                    delays. In addition, the schedule was optimistic in       
                    regard to ground software productivity, and time needed   
                    to calibrate and assess the health of the satellite.      
                    There has been almost a 3-year delay in the delivery of   
                    the highly elliptical orbit (HEO) sensors and a 6-year    
                    delay in the launch of the first GEO satellite.           
WGS              The request for proposals specified that the budget       
                    available was $750 million for three satellites and the   
                    ground control system to be delivered within 36 months.   
                    On the basis of these requirements, competing contractors 
                    were asked to offer maximum capacity, coverage, and       
                    connectivity through a contract that would make use of    
                    existing commercial practices and technologies. However,  
                    higher design complexity and supplier quality issues      
                    caused the WGS schedule to stretch to 78 months for the   
                    first expected launch. Historically, DOD experienced      
                    between 55 and 79 months to develop satellites similar to 
                    WGS, so while DOD's experience is within the expected     
                    range, the original 36-month schedule was unrealistic.    
Assumed no growth in requirements
AEHF             DOD awarded the contract for AEHF before the requirements 
                    were fully established to fill the gap left by the        
                    Milstar launch failure. As a result, DOD frequently and   
                    substantially altered requirements in the early phases of 
                    the program and changed the system design. For example, a 
                    new requirement increased the need for anti-jamming       
                    protection, which led to a cost increase of $100 million. 
                    In addition, new requirements related to training,        
                    support, and maintainability led to a cost increase of    
                    $90 million.                                              
GPS IIF          GPS IIF was intended to follow on the GPS II program, yet 
                    shortly after the contract was awarded, the government    
                    added the requirement for an additional auxiliary         
                    payload. This requirement caused the satellite design to  
                    be larger than originally planned and, in turn, required  
                    a larger launch vehicle. Requirements for more robust     
                    jamming capability to secure satellite transmissions were 
                    also added. Changes from a two-panel to a three-panel     
                    solar array design and flexible power were necessary to   
                    allow for more power and thermal capability requirements. 
SBIRS High       DOD is developing SBIRS High to improve missile warning,  
                    missile defense, technical intelligence, and battle-space 
                    characterization. As such, SBIRS has many customers,      
                    including the Air Force, Army, missile defense, and other 
                    agencies, each of which has its own requirements. This    
                    has resulted in complications in developing SBIRS, due to 
                    the fact that there are 19 key performance parameters to  
                    satisfy, which are about five times more than the typical 
                    DOD program. In addition, there are over 12,600           
                    requirements for the program to address, and to date,     
                    requirements from external users have not been fully      
                    defined. Under the TSPR arrangement, the contractor was   
                    responsible for coordinating these requirements. This     
                    effort was challenging and, according to a DOD official,  
                    one better suited for the government because all agencies 
                    were to agree on requirements. The SBIRS contractor       
                    encountered numerous problems when trying to resolve the  
                    interface issues among the various agencies. Moreover,    
                    the development of interface control documents required   
                    different certification requirements for each agency, and 
                    the SBIRS contractor had limited systems engineers to     
                    handle the workload. This lack of staff resulted in many  
                    requirements not flowing down, which led to problems      
                    later on.                                                 

Source: This table is based on conversations with program and contracting
officials and analysis of data they provided. In some cases, we made our
own designations based on our prior findings.

Appendix IV: Examples Where Independent Cost Estimates Were Not Relied Upon

We found examples from our close examinations of the AEHF, NPOESS, and
SBIRS High programs where independent cost estimates were not relied upon
by program decision makers. Independent estimates for these space system
acquisitions forecasted higher costs and lengthier schedules than program
office or service cost estimates. This appendix provides detailed
information on the differences between the program office cost estimates
and the independent cost estimates for the AEHF, NPOESS, and SBIRS High
programs.

AEHF

In 2004, AEHF program decision makers relied upon the program office cost
estimate rather than the independent estimate developed by the CAIG to
support the production decision for the AEHF program. At that time, the
AEHF program office estimated the system would cost about $6 billion. This
was based on the assumption that AEHF would have 10 times more capacity
than the predecessor satellite--Milstar--but at half the cost and weight.
However, the CAIG concluded that the program could not deliver more data
capacity at half of the weight given the state of technology at that time.
In fact, the CAIG believed that in order to get the desired increase in
data rate, the weight would have to increase proportionally. As a result,
the CAIG estimated that AEHF would cost $8.7 billion, and predicted a $2.7
billion cost overrun for the AEHF program. Table 7 displays the
differences between the program office and CAIG cost estimates.

Table 7: Comparison of 2004 AEHF Program Office and Independent Cost
Estimates

Millions of fiscal year 2006 dollars
                      Independent cost                   
                          estimate                       
Program office                                       Latest program office
estimate               AFCAA      CAIG Difference                 estimate
$6,015                     a                   $8,688    44%     $6,132 

Source: CAIG and GAO analysis.

aAFCAA worked jointly with the CAIG to develop the independent estimate.

The CAIG relied on weight data from historical satellites to estimate the
cost of AEHF because it considers weight to be the single best cost
predictor for military satellite communications. The historical data from
the AEHF contractor showed that the weight had more than doubled since the
program began and the majority of the weight growth was in the payload.
The Air Force also used weight as a cost predictor, but attributed the
weight growth to structural components rather than the more costly payload
portion of the satellite. When the CAIG briefed the Air Force on its
estimate, the program office disagreed with the CAIG results, saying it
did not see much payload weight growth in the data it analyzed. The CAIG
reported that it used AEHF contractor cost reports to determine the amount
of weight growth for the payload, and that these data were corroborated by
AEHF monthly earned value management data, which showed cost overruns for
the payload effort. As table 8 shows, the payload weight for the AEHF
satellite increased about 116 percent.

Table 8: Historical AEHF Weight Growth

Milestone                 Date         Payload weight (lbs) Percent growth 
Milestone I (A)           January 1999                1,694            n/a 
Milestone II (B)          May 2001                    2,631             55 
Preliminary design review August 2001                 3,437            103 
Critical design review    April 2004                  3,659            116 

Source: CAIG.

The Air Force attributed AEHF cost growth to problems to the cryptographic
portion of the program, which is being developed by the National Security
Agency (NSA). AEHF program officials stated that weight growth was
consistent with that of other space programs. However, the CAIG stated
that major cost growth was inevitable from the start of the AEHF program
because historical data showed that it was possible to achieve a weight
reduction or an increase in data capacity, but not both at the same time.

In addition, the CAIG also stated that the Air Force was optimistic in
developing the AEHF schedule estimate. During the production decision
review in 2004, the CAIG estimated the first satellite launch date to be
28 months longer than the program office estimate, which the CAIG
estimated to have no more than a 1 percent chance of success. The CAIG
also stated that because of problems with cryptographic development and
reliability concerns with other technical aspects of the program, such as
the phased array antenna and digital signal processing, the ambitious AEHF
schedule was in jeopardy, and the program would not likely be implemented
as planned.

In February 2005, the CAIG reviewed the proposed revision to the AEHF
Acquisition Program Baseline (APB). In a memorandum sent to the Assistant
Secretary of Defense for Network and Information Integration, the CAIG
chairman did not concur with the AEHF draft APB. The CAIG chairman
explained that while the Air Force estimate included a 24 percent increase
to the average procurement unit cost, which was 1 percent below the
threshold for a Nunn-McCurdy certification, the CAIG's estimate prepared
in December 2004 projected an increase of over 100 percent. Further,
because of the vast differences between the Air Force and CAIG cost
estimates, the CAIG chairman expressed concern that Congress would
perceive the revised APB as an attempt to avoid a Nunn-McCurdy
certification.

There is still risk for the AEHF program costs to grow. As a result of
delays, AEHF satellites have not yet been through thermal vacuum testing.
Spacecraft must endure a wide range of temperatures associated with
liftoff and ascent through the atmosphere and exposure to the extreme
temperatures of space. The thermal environment is generally considered the
most stressful operating environment for hardware, and electronic parts
are especially sensitive to thermal conditions. Problems such as cracks,
bond defects, discoloration, performance drift, coating damage, and
solder-joint failure have typically occurred. Thermal vacuum testing is
used to screen out components with physical flaws and demonstrate that a
device can activate and operate in extreme and changing temperatures.
Because thermal vacuum testing provides the most realistic simulation of
flight conditions, problems typically occur during testing. If this occurs
on AEHF, more delays and cost overruns are likely.

NPOESS

NPOESS provides another example of where there were large differences
between program office and independent cost estimates. In 2003, government
decision makers relied on the program office's $7.2 billion cost estimate
rather than the $8.8 billion independent cost estimate presented by the
AFCAA to support the NPOESS development contract award. Program officials
and decision makers preferred the more optimistic assumptions and costs of
the program office estimate, viewing the independent estimate as too high.
The $1.65 billion difference between the estimates is shown in table 9.

Table 9: Comparison of 2003 NPOESS Program Office and Independent Cost
Estimates

Millions of fiscal year 2006 dollars
                       Independent cost               
                           estimate                   
Program office                                       Latest program office
estimate                AFCAA   CAIG    Delta                     estimate
$7,219                 $8,869          a               23%     $11,400 

Source: Air Force Cost Analysis Improvement Group briefing, April 2003.

Note: The program office and the AFCAA cost estimates were based on a
purchase of six satellites, and the latest estimate is based on a purchase
of four satellites, with less capability and a renewed reliance on a
European contribution.

aThe CAIG was not involved in preparing the 2003 independent cost
estimate.

AFCAA based its estimate on an analysis of historical data from satellite
systems (i.e., NASA's Aqua and Aura and DOD's Defense Meteorological
Satellite Program [DMSP] program)^1 independent software and hardware
models, and a risk simulation model using input from 30 independent
engineers. The differences between the two estimates revolved around three
major areas:

           o The first included a discrepancy of almost $270 million in the
           cost for ground software development. The program office estimated
           the cost at $90 million based on the contractor's proposal for
           scaling the software and productivity rates that were highly
           optimistic. AFCAA based its estimate on a commercial software cost
           model using DSMP and SBIRS High historical software lines of code
           growth and actual productivity rates from the Global Positioning
           System program.

           o The second difference was in the assembly and integration and
           testing estimates. Compared to actual integration efforts on
           historical satellites used by the AFCAA, the program office
           estimate to integrate the payloads onto the satellite bus was
           nearly $132 million less than AFCAA's estimate.

           o The third area involved the systems engineering and program
           management costs for space segment development and production.
           AFCAA used actual data from the Aqua and Aura satellites, while
           the program office relied on the contractor's proposal--resulting
           in a difference of more than $130 million. The program office's
           estimate was lower based on an assumption that the costs for
           systems engineering and program management would be reduced by
           almost 50 percent between development and production. AFCAA stated
           concern that Aqua, Aura, and DMSP did not show a significant
           decrease in these costs over time.

^1 Aqua collects information on evaporation from the oceans, water vapor
from the atmosphere, radioactive energy fluxes, land vegetation cover, and
land, air, and water temperatures, among other things. Aura's mission is
to study the Earth's ozone, air quality, and climate focusing exclusively
on the composition, chemistry, and dynamics of the Earth's upper and lower
atmospheres. The Defense Meteorological Satellite Program collects weather
data for military operations.

Because the program office's estimate was lower, AFCAA concluded that the
program office's cost and schedule estimates suffered from a lack of
realism. However, the results of AFCAA's independent cost estimate were
not used by the program office officials and decision makers.

In May 2004, the Under Secretary of the Air Force asked the CAIG to
prepare an independent cost estimate for the NPOESS program. The estimate
was completed in January 2005, following completion of the contractor's
re-evaluation of the program baseline in November 2004. The cost estimate
focused primarily on the proposed integration schedule of the NPOESS
satellites. This estimate, like AFCAA's estimate before it, was based on
historical cost data from analogous satellites and concluded that the
program office's proposed integration schedule for the program was
unrealistic. For example, the program office proposed an integration
schedule for the first NPOESS satellite that was about half the time
needed for an analogous satellite that had almost the same number of
sensors. In other words, the NPOESS program estimated that it would
integrate close to the same number of sensors in half the time. Table 10
illustrates how the program office developed its integration estimate for
NPOESS, which was based on data from Aqua satellites.

Table 10: Program Office Integration Estimates for NPOESS

                        Months to                                   Months to 
                        integrate             Deletion of Months to integrate 
                Number   based on Months to months due to integrate    sensor 
                    of historical integrate    unforeseen   without   without 
Program     sensors       data    sensor      problems  problems  problems 
Aqua              6         31       5.2           -17        14       2.3 
NPOESS            5         26       5.2           N/A        14       2.8 
(first                                                                     
satellite                                                                  
integration                                                                
)                                                                          

Source: NPOESS Executive Committee briefing, January 2005.

The program office relied on actual data for Aqua, with no unforeseen
problems as the basis for estimating the amount of time needed to
integrate NPOESS sensors on the first satellite, rather than using
historical data that would have yielded an estimate of 26 months to
integrate five sensors. The program office and the contractor contended
that a novel approach was being taken to satellite integration on the
NPOESS program. The CAIG disagreed with this contention, stating that the
proposed integration approach was not really novel because the use of a
test bed model is a common tool used by satellite programs and would not
yield the significant savings asserted by the program office. The CAIG,
instead, estimated 25 months for integrating five sensors based on Aqua,
Aura, and DMSP historical data. As a result, the CAIG's estimate was
almost double the program office's. The CAIG also expressed concern to
program officials that the integration schedule was severely
underestimated and that the difference between the program office estimate
and the CAIG's estimate was more than 6 years.

The program office's 2003 estimate of $7.2 billion has been shown to be
highly unrealistic, with significant cost overruns and schedule
delays--thus far--for sensor development only. Overall satellite
integration efforts have been delayed due to the problems experienced in
development of the sensors. In June 2006, the Office of the Secretary of
Defense completed the Nunn-McCurdy process and certified a restructured
program that reduced the number of satellites to be developed--from six to
four, with the first launch being delayed to 2013 from 2009. Cost has
grown from the original estimate of nearly $7.2 billion to over $11.4
billion--approximately a 60 percent increase.

SBIRS High

On the SBIRS High program, the program office and AFCAA predicted cost
growth as early as 1996, when the program was initiated. While both
estimates at that time were close ($5.7 billion in 2006 dollars by the
program office and $5.6 billion in 2006 dollars by AFCAA), both were much
more than the contractor's estimated costs. Nevertheless, the program was
subsequently budgeted at $3.6 billion by the program office, almost $2
billion less than the AFCAA or program office estimate. The CAIG stated
that the SBIRS program assumed savings under TSPR that simply did not
materialize. SBIRS program officials also planned on savings from simply
rehosting existing legacy software, but those savings were not realized
because the all software eventually was rewritten. Instead, it took 2
years longer than planned to complete the first increment of software.

Savings were also assumed by the contractor in the area of systems
engineering. The SBIRS High contractor initially estimated using fewer
systems engineers, even though historical data showed programs similar to
SBIRS High relied on three to almost four times the number of system
engineers. Some of the tasks dropped from the systems engineering effort
included verification and cycling of requirements because the government
assumed that the contractor would perform these activities with little or
no oversight. The contractor also held the same requirements for its
subcontractors, resulting in a program with limited systems engineering.
The lack of systems engineers has led to latent design flaws and
substantially more integration and testing than planned because no one
knew what had gone wrong when components began to fail during testing.
This large amount of rework and troubleshooting has led to substantial
cost and schedule increases.

In 2005, the CAIG reviewed the SBIRS High production program including
estimating the cost to develop geosynchronous earth orbiting (GEO)
satellites 3-5 as clones of GEOs 1 and 2 in order to determine the cost
growth incurred by the production program since 2002. The CAIG's analysis
projected a 25 percent Nunn-McCurdy breach in average procurement unit
cost as a result of contractor cost and schedule performance being
markedly worse than those experienced on historical satellite programs. In
addition, the CAIG found that government actions to date have been
ineffective in controlling cost and schedule growth. The program office,
on the other hand, showed a much lower cost estimate for the production
cost of GEO satellites 3-5, as seen in table 11.

Table 11: SBIRS High GEO 3-5 Procurement Funding Analysis

Millions of then-year dollars   
Cost baseline                   CAIG estimate Program office Delta Delta % 
Three individual satellite             $2,892         $2,027  $865     43% 
procurements                                                               

Source: CAIG and GAO analysis.

The CAIG based its estimate on contractor data for prime contractor
systems engineering and program management, and payload integration
assembly and test, which showed substantial increases in the period of
performance, staffing levels, and hourly rates over initial estimates. In
addition, the CAIG's estimate reflected a contractual change from a shared
fee pool to a traditional prime contractor/subcontractor relationship.

The CAIG expressed concern that despite restructuring and rebaselining the
program, SBIRS High has struggled unabated since contract award. The CAIG
also cautioned that rebaselining would only allow the program to hide
problems in the short term. For example, the CAIG reported that earned
value management data showed GEO costs were following the same downward
trend as the HEO portion of the program, which meant that additional cost
and schedule delays were possible.

Appendix V: Comments from the Department of Defense

Appendix VI: GAO Contacts and Staff Acknowledgments

GAO Contact

Cristina T. Chaplain (202) 512-4859 or [email protected]

Staff Acknowledgments

In addition to the contact named above, Brian Bothwell, Greg Campbell,
Joanna Chan, Jennifer Echard, Art Gallegos, Barbara Haynes, Anne Hobson,
Jason Lee, Sigrid McGinty, Karen Richey, Suzanne Sterling, Adam Vodraska,
and Peter Zwanzig made key contributions to this report.

Related GAO Products

Defense Space Activities: Management Actions Are Needed to Better
Identify, Track, and Train Air Force Space Personnel. [32]GAO-06-908 .
Washington, D.C.: September 21, 2006.

Defense Acquisitions: DOD Needs to Establish an Implementing Directive to
Publish Information and Take Actions to Improve DOD Information on
Critical Acquisition Positions. [33]GAO-06-987R . Washington, D.C.:
September 8, 2006.

Defense Acquisitions: Space System Acquisition Risks and Keys to
Addressing Them. [34]GAO-06-776R . Washington, D.C.: June 1, 2006.

Space Acquisitions: Improvements Needed in Space Systems Acquisitions and
Keys to Achieving Them. [35]GAO-06-626T . Washington, D.C.: April 6, 2006.

Best Practices: Better Support of Weapon System Program Managers Needed to
Improve Outcomes. [36]GAO-06-110 . Washington, D.C.: November 30, 2005.

Defense Acquisitions: Incentives and Pressures That Drive Problems
Affecting Satellite and Related Acquisitions. [37]GAO-05-570R .
Washington, D.C.: June 23, 2005.

Defense Acquisitions: Improved Management Practices Could Help Minimize
Cost Growth in Navy Shipbuilding Programs. [38]GAO-05-183 . Washington,
D.C.: February 28, 2005.

NASA: Lack of Disciplined Cost-Estimating Processes Hinders Effective
Program Management. [39]GAO-04-642 . Washington, D.C.: May 28, 2004.

Defense Acquisitions: Despite Restructuring, SBIRS High Program Remains at
Risk of Cost and Schedule Overruns. [40]GAO-04-48 . Washington, D.C.:
October 31, 2003.

Defense Acquisitions: Improvements Needed in Space Systems Acquisition
Management Policy. [41]GAO-03-1073 . Washington, D.C.: September 15, 2003.

Military Space Operations: Common Problems and Their Effects on Satellite
and Related Acquisitions. [42]GAO-03-825R . Washington, D.C.: June 2,
2003.

(120554)

GAO's Mission

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

To Report Fraud, Waste, and Abuse in Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or (202)
512-7470

Congressional Relations

Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548

Public Affairs

Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

www.gao.gov/cgi-bin/getrpt?GAO-07-96 .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact Cristina T. Chaplain at (202) 512-4841 or
[email protected].

Highlights of [50]GAO-07-96 , a report to Subcommittee on Strategic
Forces, Committee on Armed Services, House of Representatives

November 2006

SPACE ACQUISITIONS

DOD Needs to Take More Action to Address Unrealistic Initial Cost
Estimates of Space Systems

Estimated costs for the Department of Defense's (DOD) major space
acquisition programs have increased by about $12.2 billion from initial
estimates for fiscal years 2006 through 2011. Cost growth for ongoing Air
Force programs above initial estimates accounts for a substantial portion
of this 44 percent increase. In light of the role that optimistic
estimating is believed to have played in exacerbating space acquisition
cost growth, you requested that we examine (1) in what areas space system
acquisition cost estimates have been unrealistic and (2) what incentives
and pressures have contributed to the quality and usefulness of cost
estimates for space system acquisitions.

[51]What GAO Recommends

GAO recommends that DOD take a number of actions to increase the
likelihood that independent, more realistic cost estimates will be
developed and utilized.

DOD concurred with the overall findings of this report and provided
information on the specific actions it was already taking to improve the
Air Force's cost-estimating capability.

Costs for DOD space acquisitions over the past several decades have
consistently been underestimated--sometimes by billions of dollars. For
example, Space Based Infrared System High program costs were originally
estimated at $4 billion, but the program is now estimated to cost over $10
billion. Estimated costs for the National Polar-orbiting Operational
Satellite System program have grown from almost $6 billion at program
start to over $11 billion.

For the most part, cost growth has not been caused by poor cost
estimating, but rather the tendency to start programs before knowing
whether requirements can be achieved within available resources--largely
because of pressures to secure funding. At the same time, however,
unrealistic program office cost estimates have exacerbated space
acquisition problems. Specifically, with budgets originally set at
unrealistic amounts, DOD has had to resort to continually shifting funds
to and from programs, and such shifts have had costly, reverberating
effects.

Our analyses of six ongoing space programs found that original cost
estimates were particularly unrealistic about the promise of savings from
increased contractor program management responsibilities, the constancy
and availability of the industrial base, savings that could be accrued
from heritage systems, the amount of weight growth that would occur during
a program, the availability of mature technology, the stability of
funding, the stability of requirements, and the achievability of planned
schedules. At times, estimates that were more realistic in these areas
were available to the Air Force, but they were not used.

Cost-estimating and program officials we spoke with identified a number of
factors that have contributed to this condition, in addition to larger
pressures to produce low estimates that are more likely to win support for
funding.

           o Although the National Security Space Acquisition policy requires
           that independent cost estimates be prepared by bodies outside the
           acquisition chain of command, it does not require that they be
           relied upon to develop program budgets.
           o While the policy requires that cost estimates be updated at
           major acquisition milestones, significant events, such as changes
           in the industrial base or funding, have occurred between
           milestones.
           o Within space system acquisitions, cost-estimating officials
           believe that their roles and responsibilities are not clear and
           the cost-estimating function is fragmented.
           o Cost-estimating resources have atrophied over the years because
           of previous downsizing of the workforce, making resources such as
           staff and data inadequate and the Air Force more dependent on
           support contractors for the estimating function.

References

Visible links

  26. http://www.gao.gov/cgi-bin/getrpt?GAO-06-110
  27. http://www.gao.gov/cgi-bin/getrpt?GAO-06-908
  28. http://www.gao.gov/cgi-bin/getrpt?GAO-06-987R
  29. http://www.gao.gov/
  30. file:///home/webmaster/infomgt/d0796.htm#mailto:[email protected]
  32. http://www.gao.gov/cgi-bin/getrpt?GAO-06-908
  33. http://www.gao.gov/cgi-bin/getrpt?GAO-06-987R
  34. http://www.gao.gov/cgi-bin/getrpt?GAO-06-776R
  35. http://www.gao.gov/cgi-bin/getrpt?GAO-06-626T
  36. http://www.gao.gov/cgi-bin/getrpt?GAO-06-110
  37. http://www.gao.gov/cgi-bin/getrpt?GAO-05-570R
  38. http://www.gao.gov/cgi-bin/getrpt?GAO-05-183
  39. http://www.gao.gov/cgi-bin/getrpt?GAO-04-642
  40. http://www.gao.gov/cgi-bin/getrpt?GAO-04-48
  41. http://www.gao.gov/cgi-bin/getrpt?GAO-03-1073
  42. http://www.gao.gov/cgi-bin/getrpt?GAO-03-825R
  50. http://www.gao.gov/cgi-bin/getrpt?GAO-07-96
*** End of document. ***