Defense Acquisitions: Incentives and Pressures That Drive
Problems Affecting Satellite and Related Acquisitions (23-JUN-05,
GAO-05-570R).
In fiscal year 2006, the Department of Defense (DOD) expects to
spend more than $23 billion to develop, acquire, and operate
satellites and other space-related systems. These systems are
becoming increasingly critical to every facet of military
operations as well as the U.S. economy and homeland security.
Satellite systems collect information on the capabilities and
intentions of potential adversaries. They enable U.S. military
forces to be warned of missile attacks and to communicate and
navigate while avoiding hostile actions. They provide information
that allows forces to precisely attack targets in ways that
minimize collateral damage and loss of life. DOD's satellites
also enable global communications; television broadcasts; weather
forecasting; disaster planning; navigation of ships, planes,
trucks, and cars; and synchronization of computers,
communications, and electric power grids. DOD's introduction of
these desirable capabilities over time has not come without
difficulties. Space system acquisitions have experienced problems
over the past several decades that have driven up costs by
hundreds of millions, even billions of dollars, stretched
schedules by years, and increased performance risks. In some
cases, capabilities have not been delivered to the warfighter
after decades of development. As a result of these problems, DOD
is now contending with important trade-off decisions, such as the
following. Whether to keep striving to build its Space-Based
Infrared System (SBIRS) High as intended or cut back on
capabilities. This system is intended to replace and upgrade an
older generation of missile-warning satellites, but its cost has
already more than doubled and continues to increase, and its
schedule has stretched for years. Whether and how much to employ
lower orbiting satellites equipped with similar capabilities to
facilitate missile defense activities. DOD had spent two decades
on this effort without launching a single satellite. Cost and
schedule problems forced DOD to rebaseline the program several
times. Overall affordability of missile defense has driven DOD to
assess whether to continue with this particular effort as well as
pursue development of a newer generation of missile-tracking
satellites. Whether to limit the acquisition of new communication
satellites, known as the Advanced Extremely High Frequency (AEHF)
satellites, in favor of developing a newer generation of
laser-linked satellites, known as the Transformational Satellite
Communications System (TSAT). The AEHF program is running over
cost and schedule, but it incorporates more mature technologies.
TSAT promises dramatically greater bandwidth and processing
capabilities and is considered integral to DOD's efforts to
network all of its weapon systems, but there is much less
certainty as to how much the system will cost or when it can be
delivered because critical technologies are not mature. Whether
to pursue incremental increases in capability for the Global
Positioning System or embark on a more expensive program that
would offer more dramatic capability advances. Two years ago, we
issued a report to Congress that analyzed reports we had
previously issued on satellite and other space-related programs
over the past two decades as well as other studies. Our 2003
report identified common problems affecting those acquisitions.
Generally, the problems we identified were common to DOD weapons
acquisitions and were recognized within DOD and the space
community. In February 2005, Congress requested that we identify
underlying incentives and pressures that drive the problems we
had identified earlier. Congress also asked that we complete our
fieldwork by April 2005 to support the subcommittee's decisions
on DOD's appropriations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-570R
ACCNO: A27640
TITLE: Defense Acquisitions: Incentives and Pressures That Drive
Problems Affecting Satellite and Related Acquisitions
DATE: 06/23/2005
SUBJECT: Cost overruns
Defense capabilities
Defense cost control
Defense procurement
Internal controls
Performance measures
Procurement planning
Procurement policy
Satellites
Schedule slippages
Strategic planning
Weapons research and development
Air Force Total System Performance
Responsibility Program
DOD Advanced Extremely High Frequency
Satellite Program
DOD Space-Based Infrared System High
Component
Air Force Transformational Satellite
Communications System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-570R
Washington, DC 20548
June 23, 2005
The Honorable C.W. Bill Young
Chairman, Subcommittee on Defense
Committee on Appropriations
House of Representatives
Subject: Defense Acquisitions: Incentives and Pressures That Drive
Problems Affecting Satellite and Related Acquisitions
Dear Mr. Chairman:
In fiscal year 2006, the Department of Defense (DOD) expects to spend more
than $23 billion to develop, acquire, and operate satellites and other
space-related systems. These systems are becoming increasingly critical to
every facet of military operations as well as the U.S. economy and
homeland security. Satellite systems collect information on the
capabilities and intentions of potential adversaries. They enable U.S.
military forces to be warned of missile attacks and to communicate and
navigate while avoiding hostile actions. They provide information that
allows forces to precisely attack targets in ways that minimize collateral
damage and loss of life. DOD's satellites also enable global
communications; television broadcasts; weather forecasting; disaster
planning; navigation of ships, planes, trucks, and cars; and
synchronization of computers, communications, and electric power grids.
DOD's introduction of these desirable capabilities over time has not come
without difficulties. Space system acquisitions have experienced problems
over the past several decades that have driven up costs by hundreds of
millions, even billions of dollars, stretched schedules by years, and
increased performance risks. In some cases, capabilities have not been
delivered to the warfighter after decades of development. As a result of
these problems, DOD is now contending with important trade-off decisions,
such as the following.
o Whether to keep striving to build its Space-Based Infrared System
(SBIRS) High as intended or cut back on capabilities. This system is
intended to replace and upgrade an older generation of missile-warning
satellites, but its cost has already more than doubled and continues to
increase, and its schedule has stretched for years.
o Whether and how much to employ lower orbiting satellites equipped with
similar capabilities to facilitate missile defense activities. DOD had
spent two decades on this effort without launching a single satellite.
Cost and schedule problems forced DOD to rebaseline the program several
times. Overall affordability of missile defense has driven DOD to assess
whether to continue with this particular effort as well as pursue
development of a newer generation of missile-tracking satellites.
o Whether to limit the acquisition of new communication satellites,
known as the Advanced Extremely High Frequency (AEHF) satellites, in favor
of developing a newer generation of laser-linked satellites, known as the
Transformational Satellite Communications System (TSAT). The AEHF program
is running over cost and schedule, but it incorporates more mature
technologies. TSAT promises dramatically greater bandwidth and processing
capabilities and is considered integral to DOD's efforts to network all of
its weapon systems, but there is much less certainty as to how much the
system will cost or when it can be delivered because critical technologies
are not mature.
o Whether to pursue incremental increases in capability for the Global
Positioning System or embark on a more expensive program that would offer
more dramatic capability advances.
Two years ago, we issued a report to your subcommittee that analyzed
reports we had previously issued on satellite and other space-related
programs over the past two decades as well as other studies. Our 2003
report identified common problems affecting those acquisitions. 1
Generally, the problems we identified were common to DOD weapons
acquisitions and were recognized within DOD and the space community. In
February 2005, you requested that we identify underlying incentives and
pressures that drive the problems we had identified earlier. You also
asked that we complete our fieldwork by April 2005 to support the
subcommittee's decisions on DOD's appropriations.
To respond to your request, we analyzed a wide body of GAO, DOD, and
industry studies (see enc. III) that discuss acquisition problems and
underlying incentives and pressures, including our work on best practices
in weapon system development that we have conducted over the past
1GAO, Military Space Operations: Common Problems and Their Effects on
Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.: June 2,
2003).
Results in Brief
decade, our individual reviews of space system acquisitions and
crosscutting problems, DOD's independent study of problems affecting SBIRS
High, past DOD studies of crosscutting problems with space system
acquisitions, and a more recent DOD joint task force study on the
acquisition of national security space programs. We also conducted
interviews with more than 40 individuals (see enc. IV)-including
experienced space acquisition program managers and program executive
officials within Air Force Space Command and its Space and Missile Systems
Center, officials responsible for science and technology (S&T) activities
that support space, former and current officials within the Office of the
Secretary of Defense who have specific responsibility for space oversight
or more general weapon system acquisition policy and oversight, and
individuals representing various aspects of industry. We conducted our
review from February 2005 to April 2005 in accordance with generally
accepted government auditing standards.
The officials we spoke with for this review cited a set of incentives and
pressures underlying the space acquisition problems that are largely
reflective of a lack of an overall investment strategy and a corresponding
tendency to set start dates for programs before a sound business case for
them has been established. Specifically, they told us that DOD starts more
programs than it can afford and rarely prioritizes them for funding
purposes. Such an approach has cascading effects-from creating negative
behaviors associated with competing for funds, to increasing technology
challenges, to creating unanticipated and disruptive funding shifts, to
stretching out schedules in order to accommodate the whole portfolio of
space programs. Our previous reports have found these pressures are
long-standing and common to weapon acquisitions, not just space systems.
In addition, officials we spoke with also cited pressures resulting from
having a diverse array of officials and organizations involved with the
acquisition process, tensions between the S&T and acquisition communities
as to who is better suited to translate technology concepts into reality,
pressures resulting from short tenures among staff critical to achieving
acquisition success, and difficulties in overseeing contractors.
We are not making recommendations in this report because it was not within
the scope of our work to determine the actions needed to redirect the
complex set of incentives and pressures affecting space programs. However,
as we point out, our previous reports have already made
recommendations-some of which have been implemented-that we believe would
enable DOD to put space acquisition and other weapons programs on a
sounder footing. In commenting on our report, DOD
Background
pointed out that it has recently taken steps such as improving
requirements setting for all weapons systems and ensuring that decisions
to start space acquisition programs are based on adequate knowledge. Where
appropriate in this report, we also identify and present our views on
solutions being discussed and implemented within DOD.
The majority of satellite acquisition programs that DOD has pursued over
the past several decades cost more than expected and took longer to
develop and launch than planned. In our 2003 report, we tied these results
to four problems.
1. Requirements for what the satellite needed to do and how well it must
perform were not adequately defined at the beginning of a program or were
changed significantly once the program had begun.
2. Investment practices were weak. For example, potentially more
costeffective approaches were not examined and cost estimates were
optimistic.
3. Acquisition strategies were poorly executed. For example, competition
was reduced in order to get a program started quickly, or DOD did not
adequately oversee contractors.
4. Technologies were not mature enough to be included in product
development.
We also reported that several factors contributed to these problems.
First, DOD often set dates for delivering capabilities on the basis of
optimism rather than the knowledge that critical technologies would work
as intended by those dates. As a result, activities essential to
understanding and containing costs, maximizing competition among
contractors, and testing technologies were compressed or not done. Second,
a diverse array of organizations with competing interests have been
involved in overall satellite development-from the individual military
services to testing organizations, contractors, civilian agencies, and, in
some cases, international partners. This created challenges in making
tough trade-off decisions. Third, space acquisition programs have
historically attempted to satisfy all requirements in a single step,
regardless of the design challenge or the immaturity of technologies to
achieve the full capability. This approach made it difficult to match
requirements to available resources (in terms of time, money, and
technology). We also reported that other factors created challenges for
the satellite acquisition programs we reviewed.
These include a shrinking industrial base, a declining space workforce,
difficulties associated with testing satellites in a realistic
environment, as well as challenges associated with launching satellites.
DOD's own reviews have identified similar problems as our review and
expanded on factors that helped drive those problems. Most recently, DOD
conducted a Defense Science Board/Air Force Scientific Advisory Board
Joint Task Force study in 2003 (known as the Young Panel report) to assess
the acquisition of national security space programs and develop a road map
for reform. The Young Panel found that over time, "cost has replaced
mission success" as the primary driver in managing acquisitions, resulting
in excessive technical and schedule risk. Specifically, the Young Panel
reported that program managers face far less scrutiny on program technical
performance than they do on executing against the cost baseline. The Young
Panel said there are a number of reasons this is so detrimental-the
primary ones being that space is unforgiving, thousands of good decisions
can be undone by a single engineering flaw or workmanship error, and these
flaws and errors can result in catastrophe. The best way to avoid such
problems is an unrelenting emphasis on quality. The Young Panel noted that
in the past, space programs had embraced this approach. Our own reports
have shown that space programs have not done a good job at executing
against their cost baselines. For example, costs for one of DOD's most
important programs, SBIRS High, have more than doubled, and they continue
to grow. Our studies have also found that cost increases within DOD's
space programs are often attributable to the fact that programs were
started without sufficient knowledge as to what resources would be needed
to achieve success. The Young Panel similarly recognized that the best
cost performance is achieved when there is an emphasis on mission success,
which means taking steps to reduce technical and schedule risk and making
investments that enhance quality.
The Young Panel also found that unrealistic cost estimates had led to
unrealistic budgets and unexecutable programs. Specifically, the panel
found that the space acquisition system is strongly biased to produce
unrealistically low cost estimates throughout the process. During program
formulation, advocacy tends to dominate and a strong motivation exists to
minimize program cost estimates. Moreover, proposals from competing
contractors typically reflected the minimum program content and a "price
to win." Our own studies as well as other DOD studies have found that
unrealistic estimates are common among all weapon systems, not just space
systems, and that low estimates help ensure that the program will win
support over competing programs and be funded.
Like our study, the Young Panel also found that undisciplined definition
of and uncontrolled growth in requirements contributed to cost growth and
schedule delays and that flawed acquisition strategies did so as well. In
particular, the Young Panel, as well as a preceding review of SBIRS High,
found that the adoption of a Total System Performance Responsibility
policy in the 1990s-which lessened the government program management role
in favor of a stronger industry role-essentially eroded the government's
ability to effectively manage and oversee space programs and placed too
much responsibility on industry to define requirements and make tradeoff
decisions. Over time, this shift as well as other wellintended reforms
resulted in declines in critical capabilities within the government space
workforce, particularly for systems engineering.
DOD has recognized that problems with its space acquisitions need to be
addressed, and it has taken a range of actions, including shifting away
from Total System Performance Responsibility to stronger government
management and oversight of space programs, strengthening cost estimating
capabilities, adding independent oversight reviews to the decision-making
process, and adding discipline to requirements setting. However, our
recent reports and testimonies have recommended that DOD also focus on
ensuring that acquisition programs not begin until adequate knowledge has
been accumulated on critical technologies and suggested that DOD still
needs to guide its overall space portfolio with an investment strategy
that makes high-level trade-offs before beginning programs. Moreover, our
reports on all weapon system acquisitions have continually pointed out a
need to recognize and find ways to address the underlying incentives and
pressures that drive acquisition problems. Without doing so, the impact of
changes in policies or processes will continue to be limited.
The officials we spoke with for this review cited a set of incentives and
pressures underlying the space acquisition problems that are largely
reflective of a lack of an overall investment strategy and priority
setting and a corresponding tendency to set start dates for programs
before a sound business case for them has been established. In addition,
officials we spoke with also cited pressures resulting from having a
diverse set of individuals and organizations involved with the acquisition
process, tensions between the S&T and acquisition communities as to who is
better suited to translate technology concepts into reality, pressures
resulting from short tenures among staff key to achieving acquisition
success, and difficulties in overseeing contractors. Our own reviews have
identified
Incentives and Pressures that Drive Space System Acquisition Problems
similar incentives and pressures and found them common among weapon system
acquisitions.
Too Many Programs Competing for Funding
Many of the officials we spoke with identified pressures and incentives
that are rooted in the widely held belief that DOD starts more space
programs than it can afford and rarely prioritizes programs for funding
purposes before or after starting them. Such an approach has cascading
effects-from creating negative behaviors associated with competing for
funds, to increasing technology challenges, to creating unanticipated and
disruptive funding shifts, to stretching out schedules in order to
accommodate the whole portfolio of space programs. Figure 1 highlights the
cycle of pressures when DOD initiates too many programs with too little
funding.
Figure 1: Overview of Pressures Resulting from Beginning More Programs
than DOD Can Afford in the Long Run
Specifically, officials told us the following.
o DOD starts more programs than it can afford over the long run, forcing
programs to underestimate costs and overpromise capability. This was
attributed to both the Office of the Secretary of Defense and the Air
Force. The September 11, 2001, terror attacks on the United States
spurred DOD to attempt to pursue even more satellite programs, believing
that there was now a greater need for persistent surveillance and more
robust communication and networking capabilities.
o When faced with a lower budget, senior executives within Office of the
Secretary of Defense and the Air Force would rather make across-theboard
cuts to all space programs than hard decisions as to which ones to keep
and which ones to cancel or cut back.
o Because programs are funded annually and priorities have not been
established, competition for funding continues over time, forcing programs
to view success as the ability to secure the next installment rather than
the end goal of delivering capabilities when and as promised.
o More often than not, DOD seeks substantial leaps in capability versus
incremental leaps. While this approach helps a program to gain support, it
substantially increases the technical challenge and the level of unknowns
about a program at the time it is started.
o Having to continually "sell" a program also creates incentives to
suppress bad news about the program's status and avoid activities that
uncover bad news.
o Launching demonstrators in space is a good way to reduce risks and
learn about technologies before starting a new acquisition program. But
because of the high cost of testing technologies in space and the overall
competition for funding, programs are incentivized not to pursue this
approach. At the same time, resources outside acquisition programs devoted
to testing in an operational environment are declining.
o DOD faces resource shortages beyond funding because it starts more
programs than it can afford. Principally, it does not have a sufficient
workforce to support space acquisitions or experienced program managers to
guide them.
Our previous reports have found that these pressures are long-standing and
common to weapon acquisitions, not just space acquisitions. The
competition within DOD to win funding and get approval to start a new
program is intense, creating strong incentives to make a weapon system
stand out from existing or alternative systems. If the system does not
stand out or prevail over alternatives, the program could be terminated.
Moreover, overall DOD funding constraints put a high priority on
affordability, making it important for program sponsors to provide cost
estimates that will fit within the funding constraints. Instead of forcing
trade-offs, challenging performance requirements-when coupled with other
constraints, such as cost or the weight of the satellite-can drive product
developers to pursue exotic solutions and technologies that, in theory,
can do it all.
Moreover, in weapon acquisitions, optimistic cost estimates are encouraged
because they help gain program approval and attract budgetary resources.
The consequences of cost growth are not directly felt by an individual
program because they are "accommodated" through delivery delays and
quantity changes and by spreading the cost impact across many programs.
We have also reported that the practice of breaching cost and schedule
objectives to meet difficult requirements would not persist without a
customer's cooperation. Unlike commercial customers, DOD customers tend to
be tolerant of cost overruns and delays in order to get a highperformance
weapon system. Traditionally, customers have been willing to wait long
periods of time for a capability. They would rather wait for the most
desirable system to be developed than accept a less capable system,
thinking that they may not get the opportunity to acquire a new or
modified system in the future.
Our recent reports on space and other weapon systems have suggested that
having a departmentwide investment strategy for weapon systems or even
space systems would help reduce these pressures. Critical components of
such a strategy would include identifying overall capabilities and how to
achieve them, that is, what role space will play versus other air-, sea-,
and land-based assets; identifying priorities for funding; and
implementing mechanisms that would enforce the strategy. DOD has made
revisions to its requirements-setting and budgeting processes to
strengthen investment planning. However, it is unclear as to how these
changes will be implemented over time and whether they can serve as a
foundation for directing S&T and acquisition investments.
To help close knowledge gaps at the onset of programs and shorten
development time, DOD has adopted an evolutionary development
approach-that is, pursuing incremental increases in capability versus
significant leaps. Our examinations of best practices have found that this
approach can decrease time and cost for development because it closes gaps
in unknowns. Many of the officials we spoke with believe that evolutionary
development could be achieved in space by developing
constellations of larger numbers of smaller, more affordable satellites
instead of constellations of a few, very large and heavy, complex
satellites (commonly referred to as "Battlestar Galacticas" in the space
community). In addition to reducing cost and time associated with longer,
more challenging programs, this approach could help keep the space
industrial base more productive. Complementary alternatives include
developing common rather than unique satellite components, cheaper and
more responsive launch systems, as well as systems that enable DOD to
modify and fix satellites in orbit. DOD is pursuing a range of S&T efforts
along these lines. However, DOD's executive agent for space recently
testified that these approaches are not technically suitable for some of
the capabilities DOD is now pursuing, such as Space Radar (formerly the
Space-Based Radar program) and TSAT. We will be undertaking a review to
further assess the potential that these approaches offer for producing
better outcomes as well as potential barriers to integrating them into the
acquisition process.
Another solution that has been advocated by the Young Panel and many of
the officials we spoke with as a way of addressing gaps between resources
and requirements is management reserves. The Young Panel recommended using
reserves only to execute the approved program baseline and not for new
requirements. The officials we spoke with also said that management
reserves may not be needed as much as they currently are if programs do a
better job of matching resources to requirements before they begin. In
addition, several officials noted that broader investment strategies
should be in place so that DOD can afford management reserves.
Diverse Array of Officials and Organizations Involved with Space Systems Add
Pressure to Requirements Setting
The officials we spoke with widely agreed that the diverse array of
officials and organizations involved with a space program make it even
more difficult to pare back and control requirements. As officials we
spoke with pointed out, space systems may suffer from more requirements
pressures because there is usually a very broad constituency behind each
satellite program. The Global Positioning System, for example, not only
serves military users; it also serves civilians, supports various key
economic sectors such as transportation and communications, and is used by
allies. The National Polar-Orbiting Operational Environmental Satellite
System currently under development will serve military weather forecasters
as well as civilian forecasters and a broad community of scientists
studying environmental issues. The Space Radar system is expected to play
a major role in transforming military as well as intelligence-collecting
operations and other critical governmental
functions, such as homeland security. As a result, when starting these new
systems, space program managers can expect to be inundated with competing
demands-not just among military users-but also among civilian and industry
users.
Our prior reports have identified related pressures with all weapon
systems. More than 30 organizations within the requirements community may
have a hand in determining a weapon system's performance requirements
before a contractor with systems engineering expertise can identify the
gaps between the requirements and available resources. This process means
the "doability" of the requirements is often not known with certainty
until well into product development or until a significant percentage of
funds planned to develop the system has been invested. By this point in
time, customers' expectations have been set, making it difficult to change
requirements if gaps between requirements and available resources are
found.
In the past, DOD has not implemented effective mechanisms to help mitigate
these pressures. In fact, as DOD's own studies have shown that these
pressures were exacerbated when DOD pursued its Total System Performance
Responsibility approach because it turned over responsibilities related to
requirements definition to contractors who had less understanding and
ability to negotiate requirements, leaving program managers in the
position of having to continually address requirements growth without
additional resources. Moreover, the Young Panel observed that space
program managers have not had the authority needed to make trade-offs
between requirements and control requirements growth. The panel
recommended giving program managers this authority, accompanied by greater
accountability for requirements.
Developing Technologies within the S&T Environment versus the Acquisition
Environment
An important problem cited in our reports about space system acquisition
programs is the tendency to take on technology development that should
occur within the S&T environment. Our reports have stressed that the S&T
environment is more forgiving and less costly than the acquisition
environment, which is focused on delivery. This is because events such as
test failures, new discoveries, and time spent in attaining knowledge are
considered normal in the S&T environment rather than negative. Further,
when acquisition programs take on technology development, estimates for
cost, schedule, and performance are formally approved without the benefit
of knowing that technologies will work as intended.
Officials we spoke with for this review and previous reviews cited a
number of reasons that program managers and senior leaders choose to have
acquisition programs take on technology development activities that should
occur within the S&T environment.
o The lengthy development period required for space systems puts
pressure on program managers to continually develop technologies. There is
a fear that if these technologies do not reach maturity during this time
frame, they will be outdated by the time the satellites are ready to be
launched.
o Once a program has formally begun, it is easier to secure current and
future years' funding.
o Satellites tend to last longer than expected, and they cannot be
retrieved for upgrades, putting more pressure on programs to push for
attaining as much technological capability as possible within the
acquisition program.
o The acquisition community does not believe that labs in charge of
developing space technologies adequately understand its needs-in terms of
capabilities and time frames-and would rather pursue its own goals.
o Program managers also believe that they would have more control over
technology development if it was conducted by contractors who answered to
them rather than to DOD labs.
o DOD has not had an effective strategy for steering activities within
the S&T community to ensure that they will eventually fit in with
acquisition needs. (Note: DOD has recently developed a space S&T strategy.
We reported on this effort in January 2005.)
Our previous reports have found that many of these views tend to work
against, rather than for, DOD's ability to achieve timely technology
advances. When acquisition programs seek to translate advanced concepts
into reality, they invariably run into problems that require time and
money to fix. The effects of these problems are often revealed in a later
stage of development, where they have reverberating effects on other
aspects of the acquisition program and often require reworking design. For
example, early technological problems as well as more recent system
integration issues have severely affected the SBIRS High program, among
others.
We also previously found that DOD's new space acquisition policy increased
acquisition risks by allowing programs to begin without having
technologies demonstrated in an operational or simulated environment or
even begun with technologies in even lower stages of maturity.
Many officials believed this policy was necessary because of the unique
aspects of space acquisition programs, that is, their long length, their
complexity, and the high cost of operational testing associated with space
systems. As we have reported in the past, however, DOD has found ways to
test sensors and other critical technologies on experimental satellites in
the past, and it has built and launched technology demonstrator satellites
before starting acquisition programs. Moreover, as noted earlier, the
length of space and other weapon system development can be reduced by
pursuing evolutionary development. This approach does not prevent DOD from
concurrently seeking technological advances, but such activity should
occur outside an acquisition program, rather than inside, to minimize
disruptions. Last, officials within the Office of the Secretary of Defense
did not believe that space programs warranted a separate approach than
other weapon systems. They noted that ships, for example, have unique
aspects but still fall under the same acquisition policy as other weapon
systems, which encourages programs to test technologies in an operational
environment before starting. They also noted that having allowed space
programs to follow a separate acquisition process has effectively reduced
direct oversight from the Office of the Secretary of Defense.
DOD has recently revised its space acquisition policy, in part to
encourage programs to attain more knowledge about technologies before
starting. It has also taken steps to strengthen its commitment to fully
fund space programs. However, the revised policy still allows space
acquisition programs to begin before demonstrating technologies in an
operational or simulated environment.
Short Tenures and Short tenures for top leadership and program managers
within the Air Workforce Deficiencies Force and the Office of the
Secretary of Defense have lessened the sense May Disrupt Programs as of
accountability for acquisition problems and further encouraged a
short-Well as DOD's Overall term view of success, according to officials
we interviewed. Turnover
makes it difficult for upper-level managers to establish effective
workingAbility to Implement relationships with program managers, resulting
in less trust when Reform divulging problems.
These concerns have been echoed in prior GAO and DOD reports. DOD has
taken action over the past decade to lengthen the tenure of program
managers. However, the Young Panel reported that the average tenure of a
space program manager is just 2 years and recommended that tenures be
extended to a minimum of 4 years to minimize disruption to programs. Even
with this extension, it is likely that programs, which typically last
longer than 4 years, will continue to experience turnover in program
management. Many current programs, including SBIRS High, the Space
Tracking and Surveillance System, Global Positioning System II
modernization effort, and AEHF, were started in the mid-to late 1990s. We
have also reported in the past that the short tenures typical of program
managers make it difficult for them to change the system of incentives
because other participants can wait out reforms they oppose. Moreover, DOD
acquisition executives do not necessarily stay in their positions long
enough to develop the needed long-term perspective or to effectively
change traditional incentives.
Officials we spoke with frequently cited other workforce-related
deficiencies that put pressure on program managers and acquisition
executives. For example, there are not enough experienced program managers
to run space programs and or enough experts in software engineering-a
consequence of starting more programs than DOD can afford and effectively
manage. Earlier policies of having industry assume more responsibility
also contributed to this dearth of expertise within DOD. As a result, DOD
has increasingly relied on outside experts to help manage programs. At the
same time, a limited number of these outside experts are available to
provide technical support to DOD's various space programs.
Industry-Related Pressures
Officials we spoke with pointed out a number of pressures associated with
contractors who develop space systems for the government-mostly having to
do with the level of oversight and insight program managers have with
their contractors as well as pressures among contractors to produce
low-cost estimates while bidding on contracts. Specific concerns mentioned
include the following.
o Nonincumbent contractors are often able to submit a lower price than
the incumbent because they can be optimistic without being challenged by
DOD. These optimistic estimates enable them to win new contracts. At the
same time, however, nonincumbents are not necessarily the best
organizations to carry out the development program, particularly
because they do not have the technical and management experience
associated with the legacy system being replaced.
o Industry has been consolidated to a point where there may be only one
company that can develop a needed component for a satellite system. This
has enabled contractors to hold some programs hostage.
o Program managers are often not equipped to understand what is behind a
contractor's proposal, particularly because contractors are not likely to
disclose technical risks and highlight other negative aspects.
o Industry puts pressure on programs to have contractors develop
critical technologies within an acquisition environment versus having the
labs do it. When labs build technologies, the government allows the
contractors that work on the system that would ultimately use the
technologies to scrap them in favor of employing their own methods and
expertise.
o Program managers are not always experienced enough to stand up to
contractors when development is being mismanaged. Program managers also
may not understand the best ways to incentivize contractors and gain
insight into their performance.
o Contractors are facing workforce pressures similar to those
experienced by the government, that is, not enough technical expertise to
develop highly complex space systems. (Our recent report on space S&T
echoed this concern as well, pointing out that several studies have found
that both industry and the U.S. government face substantial shortages of
scientists and engineers and that recruitment of new personnel is
difficult because the space industry is one of many sectors competing for
the limited number of trained scientists and engineers.)
o Some space programs are facing pressures related to funding and
technology development because of an expectation widely held in the 1990s
that the commercial space market would experience a boom. At the time, DOD
decreased funding for some capabilities, principally space launch,
assuming the market could pay for a portion of research and development
and that economies of scale would result. It also relied on the commercial
sector to develop knowledge about production of satellites that eventually
were purchased as part of the Wideband Gapfiller Satellite program.
However, when anticipated commercial orders using the same technologies
did not pan out, the government experienced unanticipated schedule delays.
Conclusion
Agency Comments and Our Evaluation
By delving into the underlying incentives and pressures that cause space
system acquisitions to go awry, DOD will be better equipped to take the
steps needed to attain successful outcomes. The comments of the 40-plus
experts interviewed for this report reinforce our past findings that those
steps should include the development of an overall investment strategy
that prioritizes funding and the establishment of a sound business case
before starting an acquisition program.
In written comments on our draft report (see enc. I), DOD presented its
views in two primary areas. First, in responding to our conclusion that
problems in space acquisitions are largely reflective of a lack of an
investment strategy for space programs, DOD commented that it is
implementing a new requirements process-known as the Joint Capabilities
Integration and Development System (JCIDS)-designed to ensure that each
new military program is aligned with current and future joint needs. We
have acknowledged in our past work that the implementation of JCIDS is a
positive step toward realizing a DOD-wide investment strategy for weapon
systems. In using this new process to achieve better outcomes, DOD will
need to systematically prioritize its weapon system programs against
funding plans and consistently perform rigorous analysis of alternatives
that weigh the costs and benefits of achieving each desired capability via
a space platform versus an air, land, or sea platform. Until DOD's
strategic weapon system plans for the future are better linked to DOD's
budget, these programs will continue to experience funding shortfalls, the
shifting of funding from program to program, and accompanying schedule
delays.
Second, DOD commented on our conclusion that problems in space
acquisitions are also a result of DOD's tendency to begin these programs
before establishing a sound business case. DOD pointed out that criteria
in its new space acquisition policy, which was updated on December 27,
2004, are designed to ensure a program's readiness to proceed into the
development phase (or "program start"). We recognize that the new policy
should increase knowledge about space programs before investment decisions
are made but remain concerned that DOD will start acquisition programs and
commit to cost, schedule, and performance baselines before it has
established a sound business case-which we have found to be a match
between requirements and resources (time, money, and mature technologies).
DOD also provided a set of comments that it termed "corrections to errors
in fact." In enclosure II, we respond to this set of comments; changes
made
to the draft report and areas of disagreement between us and DOD are
highlighted.
We are sending copies of this report to the Secretaries of Defense and the
Air Force and interested congressional committees. We will make copies
available to others on request. In addition, the report will be available
on
the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841. Other staff making key contributions to this
report include Cristina Chaplain, Maricela Cherveny, Lily Chin, Art
Gallegos, Jean Harker, John Krump, and Nancy Rothlisberger.
Sincerely yours,
Robert E. Levin, Director
Acquisition and Sourcing Management
Page 18 GAO-05-570R Space System Acquisitions
Page 19 GAO-05-570R Space System Acquisitions
Page 20 GAO-05-570R Space System Acquisitions
Page 21 GAO-05-570R Space System Acquisitions
Page 22 GAO-05-570R Space System Acquisitions
Enclosure II: Department of Defense Comments and GAO's Responses
1. The Department of Defense (DOD) stated that we failed to acknowledge
that the SBIRS High system, as currently planned, would serve to replace
the Defense Support Program and improve on the capabilities it provides.
GAO's Response: We have changed the text in this report to reflect that
SBIRS High is intended to also upgrade existing capabilities.
2. DOD stated that it is operating its Defense Support Program satellites
longer than expected because they are "fortuitously living longer than
expected."
GAO's Response: Although DOD's original plan was to begin launching SBIRS
High satellites in 2002 regardless of the health or longevity of the
Defense Support Program satellites, we revised this point to concentrate
on the acquisition of SBIRS High.
3. DOD stated that our draft report implies that unlike the rest of DOD,
the space community has not adopted the evolutionary development
approach-that is, pursuing incremental increases in capability versus
significant leaps.
GAO's Response: While the space acquisition policy has a section on
Evolutionary Acquisition, the Air Force continues to pursue significant
leaps in technology within its acquisition programs. For example, DOD
plans to migrate from the Advance Extremely High Frequency (AEHF)
satellites with radio frequency crosslinks capable of transmitting data at
60 megabits per second to the Transformational Satellite Communication
System (TSAT) with laser crosslinks capable of supporting 20,000 megabits
per second.
4. DOD commented that technology discovery has not affected the
development of SBIRS High.
GAO's Response: Although we agree that currently the major problems on
SBIRS High are related to system integration, earlier in the program,
there were technical development problems related to the sensors and
satellites. We revised the text to show that system integration issues
have also affected the development of SBIRS High.
5. DOD stated that it is committed to testing technology in a relevant
environment no later than key decision point C. DOD added that regarding
maturity, the program office is required to conduct technology readiness
assessments. Finally, DOD stated that the integrated program summary's
detailed risk management section addresses technical maturity.
Enclosure II: Department of Defense Comments and GAO's Responses
GAO's Response: We note that the recently revised space acquisition policy
added processes to assess the maturity of critical technologies and
provided that "where feasible, critical technology should complete testing
in a relevant environment during Phase B." The policy also states that
technology that has not been tested in a relevant environment should be
moved to the next increment. However, the policy still allows programs to
continue to mature technology while they are designing the system and
undertaking other product development activities during Phase B. Our work
on best practices shows that successful acquisition programs do not start
product development unless a match between requirements and the resources
(time, technology, and money) can be made, and technologies should be
matured in an environment that is focused on technology development. Under
the DOD space acquisition policy, programs are allowed to enter Phase B
with technologies that are immature.
6. Regarding our citing some unique aspects of space acquisition
programs, DOD mentioned low quantities produced, continual development
during the acquisition cycle, long operational life, infeasibility of
conducting operational testing, and inability to repair once on orbit.
GAO's Response: The three unique aspects of space acquisition programs
cited in our report were those given by interviewed officials. Concerning
the DODmentioned criteria, our work has found that technology development
is best conducted before product development rather than continually
throughout the acquisition cycle. Also, low quantities produced and long
operational life are aspects shared by many non-space weapon systems. In
addition, we have reported that although operational testing is expensive,
it is feasible.
7. DOD took issue with a comment that the separate space acquisition
process has effectively blocked out oversight from the Office of the
Secretary of Defense (OSD). DOD clarified that its space policy is one of
inclusiveness and identified the various organizational participants in
the space acquisition process.
GAO's Response: We note that DOD is taking issue with comments we obtained
from officials within OSD. Because this information is opinion and is
qualified as such in this report, there are no errors to correct. The
space acquisition policy does provide for OSD stakeholders on the Defense
Space Acquisition Board. However, it is the Under Secretary of the Air
Force who has milestone decision authority over space programs, and not
the Under Secretary of Defense for Acquisition, Technology, and Logistics,
who has milestone decision authority for other weapon systems. We revised
our report to state that the separate space acquisition process has
effectively reduced rather than blocked direct OSD oversight.
Enclosure II: Department of Defense Comments and GAO's Responses
8. DOD commented that we had incorrectly identified the Secretary of the
Air Force's Space Plans and Policy Division as being interviewed.
GAO's Response: We changed that item in enclosure IV to read the
"Directorate of Space Acquisition."
Enclosure III: Related Reports
DOD
GAO
Report of the Defense Science Board/Air Force Scientific Advisory Board
Joint Task Force on Acquisition of National Security Space Programs, May
2003 (also referred to as the "Young Panel report") and the July 2004
update to this report.
Space-Based Infrared System Independent Review Team, Final Report,
February 2002.
Defense Acquisitions: Assessments of Selected Major Weapon Programs,
GAO-05-301 (Washington, D.C.: March 31, 2005).
Technology Development: New DOD Space Science and Technology Strategy
Provides Basis for Optimizing Investments but Future Versions Need to Be
More Robust, GAO-05-155 (Washington, D.C.: Jan. 28, 2005).
Defense Acquisitions: Space-Based Radar Effort Needs Additional Knowledge
before Starting Development, GAO-04-759 (Washington, D.C.: July 23, 2004).
Defense Acquisitions: Risks Posed by DOD's New Space Systems Acquisition
Policy, GAO-04-379R (Washington, D.C.: Jan. 29, 2004).
Space Acquisitions: Committing Prematurely to the Transformational
Satellite Program Elevates Risks for Poor Cost, Schedule, and Performance
Outcomes, GAO-04-71R (Washington, D.C.: Dec. 4, 2003).
Defense Acquisitions: Improvements Needed in Space Systems Acquisition
Policy to Optimize Growing Investment in Space, GAO-04-253T (Washington,
D.C.: Nov. 18, 2003)
Defense Acquisitions: Despite Restructuring, SBIRS High Program Remains at
Risk of Cost and Schedule Overruns, GAO-04-48 (Washington, D.C.: Oct. 31,
2003).
Defense Acquisitions: Improvements Needed in Space Systems Acquisition
Management Policy, GAO-03-1073 (Washington, D.C.: Sept. 15, 2003).
Military Space Operations: Common Problems and Their Effects on Satellite
and Related Acquisitions, GAO-03-825R (Washington, D.C.: June 2, 2003).
Enclosure III: Related Reports
Military Space Operations: Planning, Funding, and Acquisition Challenges
Facing Efforts to Strengthen Space Control, GAO-02-738 (Washington, D.C.:
Sept. 23, 2002).
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15,
2002).
Defense Acquisitions: DOD Faces Challenges in Implementing Best Practices,
GAO-02-469T (Washington, D.C.: Feb. 27, 2002).
Best Practices: Better Matching of Needs and Resources Will Lead to Better
Weapon System Outcomes, GAO-01-288 (Washington, D.C.: March 8, 2001).
Defense Acquisitions: Employing Best Practices Can Shape Better Weapon
System Decisions, GAO/T-NSIAD-00-137 (Washington, D.C.: April 26, 2000).
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30,
1999).
Best Practices: Successful Application to Weapon Acquisitions Requires
Changes in DOD's Environment, GAO/NSIAD-98-56 (Washington, D.C.: Feb. 24,
1998).
Other Booz, Allen, Hamilton, Space Systems Development Growth Analysis,
McLean, Va., October 2002.
Enclosure IV: Officials Interviewed for This Review
Office of the Secretary of Defense
o Office of the Director, Program Analysis and Evaluation and Cost
Analysis Improvement Group
o Office of Force Transformation
o Under Secretary of Defense for Acquisition, Technology, and Logistics
o Assistant Secretary of Defense for Networks and Information
Integration
Secretary of the Air Force
o National Security Space Office
o Office of Science, Technology, and Engineering
o Directorate of Space Acquisition
Air Force Space Command
o Directorate of Requirements
o Systems Engineering and Integration Office
o Space and Missile Systems Center
o Program Executive Office
o Advanced Extremely High Frequency Program
o Development and Transformation Directorate
o Evolved Expendable Launch Vehicle Program
o NAVSTAR GPS Joint Program
o Space-Based Infrared Systems Program
o Space Radar Program
Missile Defense Agency
o Space Tracking and Surveillance System Program
Naval Research Laboratory
Various industry and former high-ranking Department of Defense officials
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON RECYCLED PAPER
*** End of document. ***