Space Acquisitions: Committing Prematurely to the		 
Transformational Satellite Program Elevates Risks for Poor Cost, 
Schedule, and Performance Outcomes (04-DEC-03, GAO-04-71R).	 
                                                                 
In a multibillion-dollar effort, the Department of Defense (DOD) 
plans to build a space-based communications system that leverages
technologies never before used in space. Such a system would	 
enable DOD to transform how information is collected on potential
U.S. adversaries and how military forces are warned of hostile	 
action. The backbone of this system will be the Transformational 
Satellite (TSAT), which is expected to play a pivotal role in	 
connecting communications networks on the ground, in the air, on 
ships, and in space. TSAT represents a potential leap forward in 
communications speed, security, and availability. The Air Force, 
which heads up DOD's space programs, intends for TSAT to be	 
interoperable with similar systems being acquired for the	 
National Aeronautics and Space Administration (NASA) and the	 
intelligence agencies. The initial TSAT program is expected to	 
cost about $12 billion from 2003 to 2015 for development and	 
production. Several billions more are to be spent acquiring and  
supporting the associated ground infrastructure, including	 
thousands of user terminals. The Air Force intends to start the  
acquisition program in December 2003 and expects to launch the	 
first TSAT in 2011. To help pay for TSAT, the Air Force has	 
scaled back its acquisition of the Advanced Extremely High	 
Frequency (AEHF) satellites currently under development. However,
because of senior military commanders' concerns about TSAT's	 
risks and the potential delay in delivering improved space	 
communications, the Air Force plans to reassess the need for	 
future AEHF funding in November 2004. If TSAT is considered too  
high a risk to meet the warfighter's expectations, the		 
contingency plan is to take TSAT's funding--thereby delaying	 
TSAT's development--and use it to buy another AEHF satellite. The
Air Force has targeted November 2004 as the latest date such a	 
decision could be made and still include funds for AEHF in the	 
DOD budget submission for fiscal year 2006. We conducted this	 
assessment in response to the large investment planned and the	 
importance of the communications capabilities promised by TSAT	 
and AEHF. Specifically, we assessed the Air Force's readiness to 
(1) initiate a TSAT acquisition program in December 2003 and (2) 
make a decision in November 2004 about whether to take TSAT	 
funding and use it to buy another AEHF satellite.		 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-04-71R 					        
    ACCNO:   A08975						        
  TITLE:     Space Acquisitions: Committing Prematurely to the	      
Transformational Satellite Program Elevates Risks for Poor Cost, 
Schedule, and Performance Outcomes				 
     DATE:   12/04/2003 
  SUBJECT:   Aerospace research 				 
	     Communication					 
	     Communication satellites				 
	     Decision making					 
	     Funds management					 
	     Military procurement				 
	     Military satellites				 
	     Procurement practices				 
	     Schedule slippages 				 
	     DOD Advanced Extremely High Frequency		 
	     Satellite Program					 
                                                                 
	     DOD Transformational Satellite`			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-04-71R

United States General Accounting Office Washington, DC 20548

December 4, 2003

The Honorable Donald H. Rumsfeld The Secretary of Defense

Subject: Space Acquisitions: Committing Prematurely to the
Transformational Satellite Program Elevates Risks for Poor Cost, Schedule,
and Performance Outcomes

Dear Mr. Secretary,

In a multibillion-dollar effort, the Department of Defense (DOD) plans to
build a space-based communications system that leverages technologies
never before used in space. Such a system would enable DOD to transform
how information is collected on potential U.S. adversaries and how
military forces are warned of hostile action. The backbone of this system
will be the Transformational Satellite (TSAT),1 which is expected to play
a pivotal role in connecting communications networks on the ground, in the
air, on ships, and in space. TSAT represents a potential leap forward in
communications speed, security, and availability. The Air Force, which
heads up DOD's space programs, intends for TSAT to be interoperable with
similar systems being acquired for the National Aeronautics and Space
Administration (NASA) and the intelligence agencies.

The initial TSAT program is expected to cost about $12 billion from 2003
to 2015 for development and production. Several billions more are to be
spent acquiring and supporting the associated ground infrastructure,
including thousands of user terminals. The Air Force intends to start the
acquisition program in December 2003 and expects to launch the first TSAT
in 2011.

To help pay for TSAT, the Air Force has scaled back its acquisition of the
Advanced Extremely High Frequency (AEHF) satellites currently under
development. However, because of senior military commanders' concerns
about TSAT's risks and the potential delay in delivering improved space
communications, the Air Force plans to reassess the need for future AEHF
funding in November 2004. If TSAT is considered too high a risk to meet
the warfighter's expectations, the contingency plan is to take TSAT's
funding-thereby delaying TSAT's development-and use it to buy another AEHF
satellite. The Air Force has targeted November 2004 as the latest date
such a

1 The TSAT program also includes development of another satellite, the
Advanced Polar System (APS). Because development efforts for TSAT and APS
are similar, we are referring to both programs as TSAT in this report.
More information about APS is included in enclosure I.

decision could be made and still include funds for AEHF in the DOD budget
submission for fiscal year 2006.

We conducted this assessment in response to the large investment planned
and the importance of the communications capabilities promised by TSAT and
AEHF. Specifically, we assessed the Air Force's readiness to (1) initiate
a TSAT acquisition program in December 2003 and (2) make a decision in
November 2004 about whether to take TSAT funding and use it to buy another
AEHF satellite.

RESULTS IN BRIEF

Air Force officials have set two imminent deadlines: starting the TSAT
program in December 2003, and deciding whether to shift funding from TSAT
to AEHF in November 2004. The Air Force is currently not prepared to make
an informed decision in either case.

Air Force officials are not ready to initiate the TSAT program in December
2003 because they do not have the knowledge to reliably establish cost,
schedule, and performance goals. At program start, program managers are
required by law to establish such goals.2 Our past work on successful
acquisition programs has found that these goals cannot be set reliably
unless the critical technologies and design have been determined to meet
minimum performance requirements. Programs that do not have this knowledge
at program start have a much greater risk of resorting to costly design
changes later in the development process, asking the warfighter to
compromise on desired capabilities, or incurring schedule overruns to
correct problems. Realizing that TSAT's schedule is ambitious, the Air
Force added 2 years to the acquisition program. However, the extra time
was mostly allocated to the latter part of the development process, not to
the front end, when program managers typically need the time to become
reasonably certain that technologies and early designs will work as
envisioned.

We are concerned about the Air Force's readiness to make the planned
decision in November 2004 to take TSAT funding to buy another AEHF
satellite in case the TSAT program falters. Air Force officials have not
defined what evaluation criteria they intend to use in making this
decision. Senior military commanders want assurance that they will get at
least the level of capabilities promised by AEHF early in the next decade.
However, senior DOD and Air Force officials told us that if funds were
shifted from TSAT back to AEHF, then TSAT-the linchpin of its plan to
transform military communications-would be substantially delayed. To
promote well-informed and objective investment decisions, our past work
has found that decision makers establish and use measurable criteria for
evaluating the costs, benefits, and risks of various alternatives.

                      2 10 U.S.C. sections 2220 and 2435.

We are recommending that you direct the Secretary of Air Force to develop
critical technologies more fully and to conduct early design studies
before starting the TSAT acquisition program. We are also recommending
that you direct the Secretary to establish and use measurable evaluation
criteria for the planned November 2004 funding decision. Although DOD
agrees to adopt such criteria, it believes the acquisition program can be
started because sufficient controls are in place to allow concurrent
development of technology and product design.

BACKGROUND

DOD intends to develop a new generation of space communications systems,
taking advantage of rapidly advancing technologies. This reflects an
increasing demand and reliance on satellite communications systems to move
larger volumes of information to more users. The Air Force reports that
the demand for communications bandwidth increased 473 percent between
Operation Desert Storm in 1991 and Operation Enduring Freedom in 2001. To
help meet this demand, DOD has augmented its own satellite communications
capability with commercial satellites. However, in each major conflict in
the past decade, senior military commanders still reported shortfalls in
communications capacity, particularly for rapid transmission of large
digital files, such as those created by imagery sensors. DOD's
communications studies indicate the shortfall will continue to grow,
despite major improvements in communications satellites currently in
development.

Investment Strategy for Satellite Communications Revised

In 1996, DOD developed and began to implement a space investment strategy
that proposed a new mix of improved communication satellites for use in
2010 and beyond. Among the proposed systems were the AEHF satellite, the
Wideband Gapfiller Satellite (WGS), Advanced Polar System (APS), and
Advanced Wideband Satellite (AWS), a less capable and earlier version of
TSAT. At that time, DOD believed that AWS, AEHF, and WGS would provide a
significant increase in communications capacity and would meet the
warfighters' needs in 2010 and beyond. (More information about these
satellite systems and their associated acquisition programs is included in
enc. I.)

In 2001, DOD developed a new Transformational Communications Architecture
that uses emerging communications technologies. The architecture is
expected to transform future combat and intelligence operations, with TSAT
playing a critical role. The concept is to use laser-based and improved
radio frequency transmission systems and high-speed, Internet-like
networks that will link communications systems on the ground, in the air,
on ships, and in space. Instead of circuit-based systems, such as those
used today to link specific sending-and-receiving devices, future systems
are expected to connect multiple sending-and-receiving devices at the same
time. The ultimate goal is to remove the existing constraints to
communication and enable transmissions regardless of location, size, or
message.

Knowledge-based Acquisition Strategy Results in Better Outcomes

Historically, DOD has had difficulty meeting the cost, schedule, and
performance goals that were established at the start of its major defense
acquisition programs such as TSAT. DOD's investments in money and time
have far exceeded initial estimates for developing and acquiring
communications satellites and other weapon systems. In addition, weapon
systems have frequently been saddled with performance shortfalls.3 To
address some of these difficulties, DOD recently implemented a new space
systems acquisition policy, which intends to provide decision makers in
the Air Force with more consistent and robust information on costs,
technologies, and requirements. The new acquisition policy also promotes
rapid introduction of emerging technologies into space systems and allows
technology, design, and system development to occur concurrently in an
effort to speed the acquisition process. A recent GAO report identified
some positive aspects of the policy; however, the report stated that any
benefits will be limited because the policy permits major investments in
new programs before managers know what resources are really required to
deliver a promised capability.4

Our work on best practices in weapon system acquisitions has shown that
program managers have a much higher probability of meeting cost, schedule,
and performance objectives if the needed technologies are mature and the
developing contractor has completed early design studies before starting
the acquisition program. Having this knowledge in hand means managers can
build a strong business case and ensure their products can be successfully
developed. A business case provides the necessary structure for managers
to identify the best product solution based on knowledge of performance,
constraints and assumptions, and a risk-adjusted cost-benefit analysis. In
the past several years, GAO has developed a knowledge-based acquisition
model based on best practices by leading companies. The best practices
model has three knowledge points. Each knowledge point builds on the
preceding one. The acquired knowledge is used to identify and reduce any
risks before moving a product to the next stage of development. Figure 1
shows when the three knowledge points occur on the best practices model.

3 U.S. General Accounting Office, Military Space Operations: Common
Problems and Their Effects on
Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.: June 2,
2003).
4 U.S. General Accounting Office, Defense Acquisitions: Improvements
Needed in Space Systems
Acquisition Management Policy, GAO-03-1073 (Washington, D.C.: Sept. 15,
2003).

The first knowledge point sets the stage for the eventual outcome of an
acquisition program-desirable or problematic. When the customer's needs
match the developer's resources (which include technology, design
knowledge, time, and money) before program start, successful outcomes are
much more likely to occur. If a match occurs after program start, managers
often make additional, unanticipated investments in money and time because
gaps between requirements and resources are discovered later in the
process.

AIR FORCE IS SETTING COST, SCHEDULE, AND PERFORMANCE GOALS AND STARTING
TSAT BEFORE CRITICAL KNOWLEDGE IS ATTAINED

By December 2003, when the TSAT program is scheduled to start, Air Force
officials are required by law to establish cost, schedule, and performance
goals, but the knowledge they need to set reliable goals is still not
available. Critical technologies are underdeveloped and early design
studies have not been started. Without this essential knowledge, the Air
Force is likely to have difficulty developing a sound business case for
starting the TSAT program. If the Air Force proceeds without a sound
business case, the program is at risk of higher costs, lower performance,
and delays in providing capabilities to the warfighters. Our work has
found that successful commercial and DOD development programs insist on
having mature technologies and early design studies to support the
business case.5

Critical Technologies Are Immature

Critical technologies are necessary building blocks for a system to meet
its minimum performance requirements. If these technologies are not
available when needed, the system cannot be completed as planned. And
because technology development does

5 U.S. General Accounting Office, Best Practices: Better Management of
Technology Development Can Improve Weapon System Outcomes,
GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999); and U.S. General
Accounting Office, Best Practices: Better Matching of Needs and Resources
Will Lead to Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.:
Mar. 8, 2001).

not happen on a planned or predictable schedule, it is difficult to
predict when or if a critical technology will mature. This is why leading
commercial companies demand that critical technologies are mature before
the commitment to a new system is made. Within the federal government,
NASA leveraged this best practice by developing an analytical tool to
assess technology maturity. This tool-adopted by many DOD
programs-establishes Technology Readiness Levels (TRL) for demonstrated
performance, with a higher value indicating a greater maturity level. (The
various levels are defined in enc. II.)

According to best commercial practices and DOD guidance, the minimum
acceptable level for a technology to be included in an acquisition is TRL
6. At this level, the technology is considered sufficiently mature and has
been engineered into a subsystem or prototype that closely resembles the
final design. Also, the technology has been successfully demonstrated to
work in a relevant environment. DOD policy6 prefers the maturity to reach
TRL 7-a prototype demonstration in an operational environment. A TRL 7 for
a satellite would mean the technology prototype has achieved form, fit,
and function and has been demonstrated in space. Commercial satellite
companies frequently meet these criteria by including a new technology on
an existing satellite design for demonstration purposes. Also, NASA
usually requires all mission-critical technologies to be demonstrated in
space before being placed on a new system. In some cases, demonstrating
space technologies in an operational environment is important because
operating a system in the harsh temperatures and radiation environment of
space-where a satellite must last essentially maintenancefree for 10 to 15
years-is much more challenging than land-based operations. The new space
acquisition policy does not require a minimum threshold for including new
technology on a space acquisition program.

Critical technologies for TSAT include laser optics that can transport
information over long distances in much larger quantities than radio
waves; high-speed routers that enable multi-user networks, sophisticated
data packaging; security algorithms and management utilities; multi-beam
antennas; and software reprogrammable terminals. Table 1 shows that most
of these technologies were at a TRL 3 or 4 in October 2003. When a
technology is classified as a TRL 3, it means most of the work performed
so far has been based on analytical studies and a few laboratory tests may
have been conducted. A TRL 4 means some of the key components have been
wired and integrated and have been demonstrated to work together in a
laboratory environment. Significant effort is required to move from these
TRL levels to a TRL 6, the minimum needed to effectively begin a new
acquisition program. As shown below, the program office estimates that
most of these technologies will have reached a TRL 6 threshold by fiscal
year 2006.

6 DOD Instruction 5000.2, Operation of the Defense Acquisition System, May
12, 2003.

Table 1: Current and Expected Technical Maturity Levels of TSAT
Technologies

          Critical technology          TRL as of October        When TRL 6 is 
                                                    2003             expected 
        Information protection               3-4                FY2006        
          Laser communication                4-5                FY2006        
     Information packet processing            6                 FY2003        
Antenna for communications on the         4-5                FY2006        
                 move                                    
       Information transmission              3-4                FY2006        
              management                                 
     Protected bandwidth efficient                                            
              modulation                     3-4                FY2006

Source: MILSATCOM Joint Program Office.

If one or more of TSAT's critical technologies encounters development
problems, a backup technology should be available for insertion into the
program. The laser communications technology does not have a backup
provided by another satellite program. Typically, a backup technology does
not meet all of the user's requirements and/or can negatively affect other
design requirements of the new system, such as weight and power. For
example, the alternative for TSAT's communications antenna is the current
AEHF antenna, which does not provide the essential
communicationson-the-move capability. Reverting to alternative
technologies late in a development program results in a series of costly
design changes and a need to go back to the warfighter to determine if the
changes are acceptable.

Early Design Studies Have Not Been Started Yet

As of October 2003, 2 months before TSAT's scheduled start, the Air Force
had not awarded contracts for early design studies. In the case of
successful programs, we have found that the developing contractor
evaluates the early designs according to system engineering principles to
assure that designs are technically feasible, match the user's needs, and
can be accomplished within the time frame and funds available. Without
this disciplined engineering process, programs can learn too late that
designs needed to achieve the warfighter's requirements are not feasible.
Program managers then have little choice but to ask for more time and
money to develop better designs, or they must compromise by asking the
warfighter to accept a less capable backup design or technology. When
discovered late in a development program, these changes can be costly. Our
prior work has shown that the cost to change the design increases
significantly as a program progresses through the key decision points of
an acquisition program. For this reason, most commercial companies want
greater assurance early in a program that the design is feasible and
producible.

The Air Force plans to competitively award contracts for early design
studies in December 2003, which is when the TSAT program is scheduled to
start. These studies are to be completed in 2006, when contractors are
expected to deliver a design specification in preparation for final design
efforts. To prepare for the next step- critical design review-in 2007, the
Air Force plans to assess the preliminary designs and select one or both
contractors to continue with detailed design studies and development
activities. Figure 2 shows key dates in TSAT's acquisition schedule.

After hearing senior warfighters express concerns about the ambitious
schedule, the Air Force recently extended the launch date for the first
TSAT from 2009 to 2011. However, the additional 2 years was mostly
allocated to the build-and-test phase prior to launch. The front end of
the acquisition schedule-technology development and design-remains much as
it was before the extension. The technology development phase was not
extended and the preliminary design and critical design review dates did
not change. Based on our past reviews, the importance of technology
development and design to the success of a program is critical and TSAT's
current status shows significant immaturity to be overcome.

LACK OF EVALUATION CRITERIA RAISES CONCERNS ABOUT PENDING DECISION TO
SHIFT FUNDS FROM TSAT TO AEHF

Despite intense interest across DOD in the November 2004 decision, Air
Force officials have not defined what evaluation criteria they intend to
use to assess alternatives if the TSAT program should falter. Senior
military commanders have asked for assurances that promised communications
capabilities will be delivered early in the next decade. If TSAT is likely
to miss its promised launch date of 2011, they want funding to be
allocated to complete the AEHF constellation of satellites. However,
senior DOD and Air Force officials told us that if a fourth AEHF were
acquired and a full AEHF constellation were delivered to the warfighter as
originally planned, decision makers and funding organizations within DOD
may want to wait until AEHF has reached the end of its useful life before
replacing it with a next

generation satellite, such as TSAT. If the fourth AEHF is acquired,
officials believe TSAT will be delayed by at least a decade. To these
officials, this is not a tenable scenario because they see TSAT as the
linchpin in DOD's plan to transform military communications and related
combat systems.

To promote well-informed and objective investment decisions, our past work
has found that decision makers establish and use measurable criteria for
evaluating the costs, benefits, and risks of various alternatives.
Although senior DOD and Air Force officials told us that they expect to
have accomplished a number of tasks before making the November 2004
decision, they have not established measurable evaluation criteria for
deciding whether to shift funds from TSAT back to AEHF.

CONCLUSIONS

DOD has embarked on a new transformational communications architecture to
take advantage of emerging technologies and to remove communications
constraints from combat. The department has told the warfighter and
Congress that TSAT is a key system that is necessary to achieve this
architecture. Responding quickly, the Air Force has set an imminent
deadline of December 2003 to start the TSAT program. By starting the
program so soon, the Air Force is moving ahead without mature technologies
and early design studies-two pillars of knowledge that would help program
officials to reliably establish cost, schedule, and performance goals.
This knowledge is not expected to be available until 2006. Our work over
the years has found that when programs have been started without the
requisite knowledge, program managers and contractors are later burdened
by unreasonable expectations about cost, schedule, and performance.
Problems usually arise later that lead to cost increases, delays in
delivering needed capability to the warfighters, and performance
shortfalls.

For the planned November 2004 decision about whether to fund TSAT or AEHF,
Air Force officials would be in a better position to make a well-informed,
objective decision if they establish and use specific criteria for
evaluating alternative investments. Reporting the Air Force's
decision-making criteria and rationale to Congress would enhance
transparency and provide Congress with better information for its
oversight and funding responsibilities.

RECOMMENDATIONS FOR EXECUTIVE ACTION

To promote better cost, schedule, and performance outcomes, we recommend
that you direct the Secretary of Air Force to delay the start of the TSAT
acquisition program until technologies have been demonstrated to be at an
acceptable level of maturity (at least TRL 6) and until the developing
contractor has determined through systems engineering that the design is
feasible and producible. We also recommend that you direct the Secretary
to provide the appropriate level of funding necessary to

gain this knowledge, which is critical for building a business case to
start the TSAT program at a later time.

To promote a well-informed and objective decision-now scheduled for
November 2004-about whether to fund another AEHF satellite, we further
recommend that you direct the Secretary of Air Force to:

o  	establish measurable criteria for use when evaluating alternative
investments in TSAT and AEHF and report this criteria in the Air Force's
2005 budget submission;

o  	consider the alternative investments in TSAT and AEHF against these
measurable criteria; and

o  	provide the rationale for how these criteria were applied in the Air
Force's 2006 budget submission.

AGENCY COMMENTS AND OUR EVALUATION

In commenting on a draft of this report, the Deputy Assistant Secretary of
Defense for Networks and Information Integration disagreed with our
primary recommendation to delay the start of the TSAT acquisition program
until technologies are sufficiently matured and until the contractor
determines through systems engineering principles that the design is
feasible. DOD contends that the new Air Force National Security Space
Acquisition Policy provides sufficient controls to allow concurrent
development of technology and product design. DOD states that starting the
TSAT program enables it to establish the funding and program controls-such
as managing to the acquisition program baseline-provided by the new space
acquisition policy. DOD did, however, concur or partially concur with the
other recommendations to provide funding to mature TSAT's critical
technologies and early designs, to establish criteria for making
decisions, and to report these criteria and decisions to Congress.

We believe the new space acquisition policy does not have sufficient
controls to reverse the higher costs and longer schedules that have
plagued a number of satellite programs. The added risks of concurrent
technology and product development have not helped improve the typical
outcome for satellite programs. In a series of best practices reports
issued over the years, we have identified problems resulting in
substantially different cost and schedule outcomes when compared with
initial expectations at the outset of a new acquisition program. We have
offered improved approaches based on the best commercial and defense
practices. DOD has endorsed the practices that call for a disciplined
acquisition approach, one that separates technology from product
development and bases decisions at key junctures on a set of critical
product knowledge captured by the decision point. DOD incorporated this
knowledge-based approach in its new acquisition system policy.7

                 7 DOD Directive 5000.1 and Instruction 5000.2.

DOD's new space acquisition policy, on the other hand, is a step backward
and is similar to an older acquisition policy that contributed to many
unsuccessful acquisition programs of the past. DOD's history is filled
with examples of programs that concurrently developed technology and new
products and made decisions based on risk mitigation plans instead of
knowledge about the new products. Our June 2003 report8 on common problems
in satellite programs identified Milstar, SBIRS-Low, SBIRS-High, AEHF, and
others as suffering the consequences of this earlier acquisition strategy.
Additionally, we have found that setting an acquisition program baseline
that is not rooted in key product knowledge is unreliable and not useful
as a management tool. In fact, starting the program before technologies
are mature and a feasible design study is completed reduces accountability
and straps the program manager and the contractor with unreasonable
expectations in the baseline. Therefore, we believe that because DOD's new
space acquisition policy does not require a knowledge-based acquisition
strategy, it is destined to repeat the problems of the past.

DOD stated that extensive studies done over the last two years provide
sufficient information for the Milestone Decision Authority to determine
if the TSAT program should be initiated. However, these studies do not
provide product-specific knowledge for building a business case for TSAT.
Instead, these studies were focused on developing the overarching
communications architecture rather than detailed technology and design
information needed to build and launch TSAT.

While it is key to complete early design efforts before starting the
program, substantial investments in system design and development are at
risk if the Air Force cannot demonstrate TSAT's technologies, a number of
which were still in the early paper study phase without hardware
demonstrations to support that they would work. In its fiscal year 2004
budget submission, the Air Force had budgeted over $800 million in fiscal
years 2004 and 2005 for system design and development.

To support its case for starting the TSAT program in December 2003, DOD
states backup technologies exist and are ready to fill any technology void
that might occur. They believe this will reduce the risk. However, there
are no backup technologies that will satisfy the two most critical
warfighter requirements-laser communications (critical to transporting
intelligence, surveillance, and reconnaissance data) and communications on
the move (critical to the Future Combat System). These capabilities were
the primary basis for persuading the warfighter to favor the uncertain
future of TSAT rather than to acquire the full constellation of four AEHF
satellites, which would have provided a 500 percent increase over the
communications capability used in Operation Iraqi Freedom.

If TSAT's investments were based on knowledge captured from mature
technologies and feasible design, then these informed decisions would
reduce the potential for major and costly changes as the program enters
the build-and-demonstration phase,

8 U.S. General Accounting Office, Military Space Operations: Common
Problems and Their Effects on Satellite and Related Acquisitions,
GAO-03-825R (Washington, D.C.: June 2, 2003).

when it is too late to consider other options. We believe it is better to
keep options open now, such as AEHF, and decide at a later time when
enough knowledge has been gained to ensure TSAT is the right solution for
the 2010 time frame. Our past work shows the negative outcomes of the
concurrent and risk mitigation approach to acquisition. We also have shown
the potential for more successful outcomes if a knowledge-based approach
is applied. Therefore, we stand by our recommendation that TSAT's program
start should be delayed until technologies are mature and the developing
contractor has completed studies to demonstrate a feasible design.

To ensure that the warfighter is delivered an improved capability no later
than 2011, DOD intends to decide in November 2004, based on an assessment
of TSAT's progress, whether funding should be diverted back to the AEHF
program. In its comments, the Air Force suggests criteria for this
decision point that can only result in continuing the TSAT program. For
example, criteria for laser communications or communications on the move
do not apply to AEHF. These are capabilities promised by TSAT, not AEHF.
We believe that the criteria should be based on the maturity of critical
technologies and early design of TSAT. To ensure the transparency and
objectivity of the decision process in November 2004, these criteria
should be provided to Congress in the fiscal year 2005 budget for TSAT,
not-as DOD suggests-in the 2006 budget, when the decision will already
have been made.

In response to DOD's detailed comments, we made changes to the report
where appropriate to correct technical inaccuracies. DOD's comments are
provided in enclosure III.

SCOPE AND METHODOLOGY

In conducting our review, we analyzed the extent to which the TSAT and APS
programs have acquired the knowledge needed to set specific cost,
schedule, and performance goals. To do this, we compared the acquisition
strategy with GAO's knowledge-based acquisition model and analyzed the
differences between them. We specifically focused on the portion of
knowledge-based acquisition dealing with the necessity of matching user's
needs with developer's resources prior to making a development commitment.
We collected and analyzed information from the Office of the Assistant
Secretary of Defense for Command, Control, Communication and Intelligence
(ASDC3I), Defense Information Services Agency (DISA), the National
Security Agency (NSA), Air Force Space Command (AFSPC), U.S. Strategic
Command (USSTRATCOM), Military Satellite Communication Joint Program
Office (MJPO), Joint Forces Command (JFCOM), Aerospace Corporation and
RAND Corporation. We conducted our review from February 2003 through
November 2003 in accordance with generally accepted government auditing
standards.

As you know, 31 U.S.C. 720 requires the head of a federal agency to submit
a written
statement of actions taken on our recommendations to the Senate Committee
on
Governmental Affairs and the House Committee on Government Reform not
later
than 60 days after the date of the report and to the Senate and House
Committees on
Appropriations with the agency's first request for appropriations made
more than 60
days after the date of this report.

We are sending copies of this report to interested congressional
committees. We will
also make copies available to others upon request. In addition, the report
will be
available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff has any questions concerning this report, please
contact me at
(202) 512-4841. Key contributors to this report were Lily Chin, Mike
Hazard, Dave
Hubbell, Travis Masters, and Matt Mongin.

Sincerely yours,

Robert E. Levin
Director, Acquisition and Sourcing Management

Enclosure I: Descriptions of Communication Satellites

The Air Force is developing the following communication satellites.

Wideband Gapfiller Satellite

The Wideband Gapfiller Satellite (WGS) system is a joint Air Force and
Army program intended to provide communications to U.S. warfighters,
allies, and coalition partners during all levels of conflict, short of
nuclear war. WGS will provide essential communications services for the
commanders in chief to command and control their tactical forces. Tactical
forces will rely on WGS to provide high-capacity links to the terrestrial
portion of the Defense Information Services Network. WGS is the next
generation wide-band component in the Department of Defense's (DOD) future
Military Satellite Communications architecture. WGS is composed of three
principal segments: Space Segment (satellites), Terminal Segment (users),
and Control Segment (operators). The WGS program is leveraging commercial
methods and technological advances in the satellite industry to rapidly
design, build, launch, and support a constellation of highly capable
military communications satellites.

The WGS program is being conducted as a DOD commercial acquisition and as
such
is not subject to the same milestone and/or review processes required in
other space
acquisition programs. The Air Force reports that 95 percent of satellite
content will
be commercial off-the-shelf products. The total budget for purchasing five
WGSs is
$1.5 billion. The contract is firm fixed price over 10 years and was
awarded to Boeing
Satellite Systems in January 2001. The Air Force purchased the first two
satellites in
fiscal year 2002 and the third satellite in fiscal year 2003. It plans to
purchase
satellites four and five in fiscal years 2007 and 2008, respectively. The
first two WGS
satellites are scheduled for launch in fiscal year 2005, with the third
satellite planned
for launch in fiscal year 2006.

Upon first launch into geosynchronous orbit in 2005, WGS will be the DOD's
most
capable and powerful communications satellite. Ultimately, five WGSs will
be in
orbit, providing service in both the X- and Ka band-radio frequencies.
Each satellite is
expected to have a capacity of at least 2,100 megabits per second. WGS
will augment
X-band communications now provided by the Defense Satellite Communications
System (DSCS) and one-way Ka-band service provided by the Global Broadcast
Service (GBS). Additionally, WGS will provide new two-way Ka-band
services. These
satellites are not interconnected. They will, however, provide
communications
capacity, connectivity, and flexibility for U.S. military forces while
maintaining full
interoperability with existing and programmed DSCS and GBS terminals.

Advanced Extremely High Frequency Satellite
The Advanced Extremely High Frequency (AEHF) satellite system is to be
DOD's
next generation of high-speed, secure communication satellites. This
satellite system
is intended to replace the existing communications satellites with
improved,
survivable, jam-resistant, worldwide, secure communication capabilities at
lower

launch costs. AEHF is to support the entire range of data rates to provide
assured communications across the entire spectrum of conflict, including
nuclear war. AEHF is also designed to be "backward compatible" with
existing satellites, that is, it will support both low and medium data
rates as necessary until an AEHF constellation with higher data rates
becomes available at initial operating capability (two satellites on
orbit). The first satellite is currently planned to launch in 2006 and the
second is scheduled to launch in 2007.

The Air Force is responsible for funding, developing, and producing the
AEHF satellites and the associated ground control systems. The Air Force's
budget for developing and acquiring the first three AEHF satellites is
$4.8 billion. Each service- Army, Navy, and Air Force-is separately
responsible for funding, developing, and producing its own terminals to
communicate with AEHF.

The AEHF program began in August 1998, and the final constellation will be
composed of satellites in geosynchronous orbit that can transmit data to
each other via radio frequency cross links, and communicate with ground
stations and communication terminals carried by air, sea, and ground
forces. Each satellite will have a capacity of about 250 megabits per
second. Users communicate with the satellites through their terminals. The
mission control segment provides command and control that directs the
movements and other operations of satellites.

Transformational Satellite Communications

The Transformational Satellite (TSAT) communications system is designed to
provide improved, survivable, jam-resistant, worldwide, secure and general
purpose communications as part of an independent but interoperable set of
space-based systems that will support the National Aeronautics and Space
Administration, DOD, and the intelligence community. TSAT will replace the
current satellite system and supplement AEHF.

The TSAT architecture, requirements, and cost baselines are to be approved
in December 2003. Initial design contracts are to be awarded in December
2003; therefore, the final configuration of the TSAT system remains to be
determined. Air Force budget documentation for TSAT (funded under the
Advanced Wideband Satellite budget line) shows a total cost of $10.9
billion for purchasing the first five satellites plus a spare.

The TSAT system will be the key transport mechanism of DOD's space-based
network communications system, which has individual satellites operating
as routers in space. The TSAT constellation of five satellites will
provide continuous communication coverage from 65 degrees south latitude
to 65 degrees north. The satellites will support communications in the EHF
and Ka band radio frequency bands, in addition to passing communications
via lasers. The capacity of each satellite is expected to be at least 10
times greater than the AEHF satellites. The Air Force is currently
conducting development activities necessary in order to make a

decision to start the program in December 2003. The Air Force plans to
launch the first TSAT in fiscal year 2011. The first two satellites will
have radio frequency satellite cross links to engage the AEHF satellites
as well as having the laser cross links; the third through the fifth
satellites will have laser cross links only.

Advanced Polar Satellite The Advance Polar System (APS) is a part of the
Air Force's transformational communication architecture and is being
developed and acquired as part of the TSAT/APS acquisition program. APS
will provide the next generation protected EHF band, Ka band, and laser
satellite communications capability in the north polar region starting in
fiscal year 2012. APS will support strategic as well as tactical users who
require anti-jam and low probability of detection EHF satellite
communications. The results of the transformational communications
architecture definition will affect the APS program content. Requirements
are based on the July 1995 Polar Operational Requirements Document.
According to Air Force program officials, APS is to be a "lighter" (i.e.,
lower capacity) version of the TSAT. The current APS plan is to acquire
three satellites (two funded with development funds and one funded with
procurement dollars) and associated ground infrastructure for $1.2
billion. The three APS satellites will be placed in highly inclined orbits
and are expected to provide continuous communication services to forces
deployed from 65 degrees north to the North Pole (90 degrees north).

Enclosure II: Descriptions of Technology Readiness Levels

The Interim Defense Acquisition Guidebook (formerly DOD 5000.2-R) directs
that technology readiness assessments, using Technology Readiness Levels
(TRL) or some equivalent assessment methodology, for critical technologies
shall occur sufficiently before key decision points B and C to provide
useful technology maturity information to the acquisition review process.
TRLs, originally developed by the National Aeronautics and Space
Administration (NASA), are measured along a scale of 1 to 9, starting with
paper studies of the basic concept and ending with a technology that has
proven itself in actual usage on the intended product. As the TRL scale
increases, the risks associated with uncertain technology decrease,
because more is known about their capabilities and performance. Unexpected
problems can arise at every level, and effort must be expended to overcome
them. This effort takes time and can delay the progress to the next
readiness level. According to our previous reviews of best commercial
practices and DOD guidance, a minimum level of TRL 6 should be reached
before committing to a space acquisition program. Table 1 provides a
detailed explanation of each TRL.

Table 2: TRL Scale for Assessing Critical Technologies

Technology Readiness Levels         Technology Readiness Level Description 
                                        Lowest level of technology readiness. 
     1. Basic principles observed Scientific research begins to be translated 
                    and reported.         into technology's basic properties. 
                                  Invention begins. Once basic principles are 
                                      observed, practical applications can be 
     2. Technology concept and/or    invented. The application is speculative 
          application formulated.  and there is no proof or detailed analysis 
                                      to support the assumption. Examples are 
                                              still limited to paper studies. 
                                           Active research and development is 
3. Analytical and experimental initiated. This includes analytical studies 
critical function and/or              and laboratory studies to physically 
characteristic proof of        validate analytical predictions of separate 
concept.                              elements of the technology. Examples 
                                          include components that are not yet 
                                                integrated or representative. 
                                           Basic technological components are 
4. Component and/or breadboard     integrated to establish that the pieces 
validation in laboratory       will work together. This is relatively "low 
environment.                    fidelity" compared to the eventual system. 
                                     Examples include integration of "ad hoc" 
                                                    hardware in a laboratory. 
                                  Fidelity of breadboard technology increases 
                                       significantly. The basic technological 
5. Component and/or breadboard   components are integrated with reasonably 
           validation in relevant   realistic supporting elements so that the 
                     environment.       technology can be tested in simulated 
                                          environment. Examples include "high 
                                          fidelity" laboratory integration of 
                                                                  components. 
                                  Representative model or prototype system,   
                                  which is well beyond the breadboard tested  
6. System/subsystem model or   for level 5, is tested in a relevant        
prototype demonstration in a   environment. Represents a major step up in  
relevant environment.          a technology's demonstrated readiness.      
                                  Examples include testing a prototype in a   
                                  high fidelity laboratory environment or in  
                                  simulated operational environment.          
                                     Prototype near or at planned operational 
              7. System prototype     system. Represents a major step up from 
              demonstration in an  level 6, requiring the demonstration of an 
         operational environment.   actual system prototype in an operational 
                                    environment. Examples include testing the 
                                            prototype in a test bed aircraft. 
                                  Technology has been proven to work in its   
                                  final form and under expected conditions.   
8. Actual system completed and In almost all cases, this level represents  
       qualified through test and the end of true system development.         
                   demonstration. Examples include developmental test and     
                                  evaluation of the system in its intended    
                                  weapon system to determine if it meets      
                                  design specifications.                      
                                  Actual application of the technology in its 
          9. Actual system proven final form and under mission conditions,    
       through successful mission such as those encountered in operational    
                      operations. test and evaluation. Examples include using 
                                  the system under operational mission        
                                  conditions.                                 

                  Source: GAO based on NASA and DOD guidance.

Enclosure III: Comments From the Department of Defense

(120225)
*** End of document. ***