Defense Acquisitions: Space-Based Infrared System-low at Risk of Missing
Initial Deployment Date (Letter Report, 02/28/2001, GAO/GAO-01-6).

The Pentagon considers defenses to counter attacks from ballistic
missiles, both long-range strategic and shorter-range theater missiles,
to be critical to U.S. national security. The Air Force is developing a
new satellite system, called Space-Based Infrared System-low
(SBIRS)-low, to expand the military's infrared satellite capabilities
for supporting ballistic missile defenses. GAO reviewed the Defense
Department's (DOD) efforts to acquire SBIRS-low. Specifically, GAO (1)
evaluated the cost, schedule, and performance risks of the current
acquisition schedule; (2) evaluated the program's technical risks; and
(3) determined whether DOD has assessed alternative approaches to
SBIRS-low. GAO found that the Air Force's current SBIRS-low acquisition
schedule is as high risk of not delivering the system on time or at cost
or with expected performance. SBIRS-low has high technical risks because
some critical satellite technologies have been judged immature for the
current stage of the program. DOD acquisition policy and procedures
require that the cost and mission effectiveness of space systems be
assessed relative to alternative terrestrial systems. However, the Air
Force has not analyzed or identified terrestrial alternatives to the
SBIRS-low system because, according to Air Force Space Command
officials, terrestrial alternatives do not exist. Nevertheless, studies
on various aspects of the National Missile Defense system by the
Ballistic Missile Defense Organization and other groups have pointed out
alternatives to SBIRS-low, such as sea- or land-based radar.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GAO-01-6
     TITLE:  Defense Acquisitions: Space-Based Infrared System-low at
	     Risk of Missing Initial Deployment Date
      DATE:  02/28/2001
   SUBJECT:  Defense cost control
	     Defense capabilities
	     Weapons research and development
	     Military satellites
	     Defense procurement
	     Schedule slippages
	     Federal procurement
	     Concurrency
	     Air defense systems
IDENTIFIER:  Spaced-Based Infrared System-low

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-01-6

Report to the Chairman, Subcommittee on Defense, Committee on
Appropriations, House of Representatives

February 2001 DEFENSE ACQUISITIONS

Space- Based Infrared System- low at Risk of Missing Initial Deployment Date

GAO- 01- 6

Letter 3 Appendixes Appendix I: Space- Based Infrared System- low Program
History 24

Appendix II: Technology Readiness Levels and Their Definitions 29 Appendix
III: Comments From the Department of Defense 30 Appendix IV: GAO Contact and
Staff Acknowledgments 33

Figures Figure 1: SBIRS- low Schedule for First Launch in Fiscal Year 2006 8
Figure 2: Evolutionary Development of SBIRS- low Software 13

Figure 3: Technology Readiness Levels of Critical SBIRS- low Subsystems at
the Start of Program Definition and Risk Reduction 16 Figure 4: Original
SBIRS- low Schedule for First Launch in Fiscal

Year 2006 25 Figure 5: Revised SBIRS- low Schedule for First Launch in
Fiscal Year 2004 27

Abbreviations

DOD Department of Defense SBIRS Space- Based Infrared System

Lett er

February 28, 2001 The Honorable Jerry Lewis Chairman, Subcommittee on
Defense Committee on Appropriations House of Representatives

Dear Mr. Chairman: The Department of Defense (DOD) considers missile defense
to counter attacks from ballistic missiles, both long- range strategic and
shorter- range theater missiles, to be critical to our national security.
The Air Force is developing a new satellite system, called Space- Based
Infrared System- low (SBIRS)- low, to expand DOD's current infrared
satellite capabilities for supporting ballistic missile defense. The ability
to detect missile launches, track missiles throughout their flights, and
counter these threats is essential to ballistic missile defense. The primary
mission of SBIRS- low is to detect launches and track missile flights.
SBIRS- low accomplishes this by using infrared sensors to detect the heat
missiles radiate. DOD plans to

begin launching SBIRS- low satellites in fiscal year 2006 and estimates the
life- cycle cost 1 through fiscal year 2022 to be $11.8 billion.

Because this system is planned to support the proposed National Missile
Defense System as well as theater missile defense programs, you asked us to
evaluate DOD's plans and progress to acquire SBIRS- low. Specifically, we
(1) evaluated the cost, schedule, and performance risks of the current
acquisition schedule; (2) evaluated technical risks of the program; and (3)
determined whether DOD has assessed alternative approaches to SBIRS- low.
For each of these tasks, we discussed the program with DOD and contractor
officials and reviewed studies and reports regarding cost, schedule,
performance, and technical risks as well as possible alternatives to the
program. (See our scope and methodology section.)

Results in Brief The Air Force's current SBIRS- low acquisition schedule is
at high risk of not delivering the system on time or at cost or with
expected performance. While the Air Force's previous schedules for SBIRS-
low provided for the results of a flight test of crucial satellite functions
and capabilities to be 1 DOD defines life- cycle cost as all costs relating
to research, development, production, deployment, operations, and support
for a system.

available to support the decision to enter satellite production, the current
schedule does not provide such test results until over 5 years after
production has started. If design changes are identified as a result of the
test, these changes will have to be incorporated into satellites already
under production, and parts that have already been purchased based on the
initial design may become obsolete and replaced with new parts, increasing

program costs and causing schedule delays. In addition, DOD traditionally
schedules completion of all software 1 year before the first satellite of a
new system is to be launched. However, due to an underestimation of the
level and difficulty of this effort, completion of SBIRS- low software will
not occur until over 3 years after the first satellites are to be launched,
increasing the risk that the software may not be available when needed or
perform as required.

SBIRS- low has high technical risks because some critical satellite
technologies have been judged to be immature for the current stage of the
program. Specifically, the SBIRS- low program office rated the maturity of
five of six critical technologies at levels that constitute high risk that
the

technologies will not be available when needed. DOD acquisition policy and
procedures require that assessments be made of the cost and mission
effectiveness of space systems to alternative terrestrial systems. Such an
assessment seems prudent in light of the high risks associated with the
SBIRS- low program. However, the Air Force has not analyzed or identified
terrestrial alternatives to the SBIRS- low system because, according to Air
Force Space Command officials, terrestrial alternatives do not exist.
Nevertheless, studies on various aspects of the

National Missile Defense system by the Ballistic Missile Defense
Organization and other organizations have pointed out that alternatives to
SBIRS- low may exist, such as sea- or land- based radar. In order to reduce
the cost, schedule, performance, and technical risks in the SBIRS- low
program, we are recommending that the Secretary of Defense take actions to
(1) restructure the SBIRS- low acquisition schedule and (2) analyze and
develop, as appropriate, alternatives to satisfy critical ballistic missile
defense requirements in case SBIRS- low cannot be deployed when needed. DOD
generally agrees with the report's findings and recommendations.

Background Based on its experiences with the launching of short- range
theater missiles by Iraq during the 1991 Persian Gulf War, DOD concluded
that expanded

theater missile warning capabilities were needed and it began planning for
an improved infrared satellite sensor capability that would support both
long- range strategic and short- range theater ballistic missile warning and
defense operations. In 1994, DOD studied consolidating various infrared
space requirements, such as for ballistic missile warning and defense,

technical intelligence, and battlespace characterization, 2 and it selected
SBIRS to replace and enhance the capabilities provided by the Defense
Support Program. The Defense Support Program is a strategic surveillance and
early warning satellite system with an infrared capability to detect long-
range ballistic missile launches that has been operational for about

30 years. DOD has previously attempted to replace the Defense Support
Program with the Advanced Warning System in the early 1980s; the Boost
Surveillance and Tracking System in the late 1980s; the Follow- on Early
Warning System in the early 1990s; and the Alert, Locate, and Report
Missiles System in the mid- 1990s. These attempts failed due to immature
technology, high cost, and affordability issues. SBIRS is to use more
sophisticated infrared technologies than the Defense Support Program to
enhance the detection of strategic and theater ballistic missile launches
and the performance of the missile- tracking function.

2 Technical intelligence relates to assessments of activities such as
foreign launch technology developments and arms control compliance, and
battlespace characterization refers to activities such as battle damage
assessment.

The SBIRS development effort consists of two programs- SBIRS- high and
SBIRS- low. SBIRS- high is to consist of four satellites operating in
geosynchronous earth orbit 3 and sensors on two host satellites operating in
a highly elliptical orbit. 4 SBIRS- high will replace Defense Support
Program

satellites and is primarily to provide enhanced strategic and theater
ballistic missile warning capabilities. The SBIRS- high program includes the
consolidation of the three existing Defense Support Program ground
facilities- two overseas and one in the United States- at a single U. S.
ground station to reduce operations and maintenance costs. The program

is in the engineering and manufacturing development phase, 5 with a
scheduled launch of the first SBIRS- high satellite in fiscal year 2005. 6
The SBIRS- low program is currently in the program definition and risk
reduction acquisition phase 7 and is expected to consist of about 24
satellites in low earth orbit, but it could consist of more or less
satellites, depending on the results of contractor cost and performance
studies. The primary purpose of SBIRS- low is to support both national and
theater missile defense by tracking ballistic missiles and discriminating
between the warheads and other objects, such as decoys, that separate from
the

missile bodies throughout the middle portion of their flights. Its
deployment schedule is tied to fiscal year 2010, the date when these
capabilities are needed by the National Missile Defense System. According to
DOD, the first SBIRS- low satellites need to be launched in fiscal year 2006
if full deployment is to be accomplished by fiscal year 2010.

3 A satellite in a geosynchronous orbit has a circular period of revolution
that is equal to the period of the earth's rotation about its axis, or 24
hours. 4 A satellite in a highly elliptical orbit spends most of its time
over a designated area of the earth, known as apogee dwell. 5 This phase,
focused on finalizing the system design and ensuring it is ready for
production, consists of the steps necessary to translate the most promising
design approach into a stable, producible, supportable, and cost- effective
design; validate the manufacturing or production processes; and demonstrate
system capabilities through testing.

6 We also evaluated DOD's plans and progress to acquire SBIRS- high and we
plan to issue a separate report on this evaluation. 7 This phase consists of
steps necessary to verify preliminary design and engineering, build
prototypes, accomplish necessary planning, and fully analyze trade- off
proposals. The objective is to validate the choice of alternatives and to
provide the basis for determining whether to proceed into the next phase
(engineering and manufacturing development) of the acquisition process.

Due to the importance Congress has placed on the deployment of a National
Missile Defense System, Congress has maintained a high level of interest in
the SBIRS- low program and has included in legislative provisions dates by
which the first satellites are to be launched and initial operational
capability is to occur. 8 The National Defense Authorization Act for Fiscal
Year 2000 is the latest expression of such interest. It defines, in section
231, the SBIRS- low baseline schedule as a program schedule that includes a
first launch of a SBIRS- low satellite to be made during fiscal

year 2006. This provision also requires that before the Secretary of the Air
Force makes any changes to the SBIRS- low baseline schedule he must obtain
the approval of the Director of the Ballistic Missile Defense

Organization. 9 SBIRS- low Program The Air Force's current SBIRS- low
acquisition schedule is at high risk of Employs a High- Risk not delivering
the system on time or at cost or with expected performance. Specifically,
satellite development and production are scheduled to occur Acquisition
Strategy concurrently and the results of a 1- year flight test that is to
test and finalize the design of the satellites will not be available until
more than 5 years after

the program enters production. The software required for SBIRS- low to
perform all its missions is to be developed concurrent with the deployment
of the satellites and is not to be completed until more than 3 years after
the first SBIRS- low satellites are to be launched.

On- Orbit Test Results Will Under the Air Force's previous schedules for
SBIRS- low, the results of an

Not Be Available to Support on- orbit flight demonstration of crucial
satellite functions and capabilities

Production Decision were to be available and used to support the decision to
enter satellite

production; however, the current schedule does not provide such test results
in time to support the production decision. In February 1999, the Air Force
established the current acquisition schedule (see fig. 1) for the

SBIRS- low program, which includes a program definition and risk reduction
phase, a concurrent development and production phase, and

8 National Defense Authorization Act for Fiscal Year 1996, P. L. 104- 106,
section 216. 9 Section 231 also specifies that such approval be obtained
before the Secretary of the Air Force (1) establishes any system level
technical requirement or makes any change to any such requirement or (2)
makes any change to the budget baseline identified in the fiscal year 2000
future years defense program.

a 1- year on- orbit test with the first six SBIRS- low satellites produced
(to be launched with two launches- three satellites per launch).

Figure 1: SBIRS- low Schedule for First Launch in Fiscal Year 2006

Program Definition and Risk Reduction

Engineering and Manufacturing Development

Production 1 2 Initial launches

1- year on- orbit test 3 4 5 6 7 8910

Remaining launches 1999 2000 2001

2002 2003 2004 2005 2006 2007 2008 2009 2010 Fiscal Year Source: Air Force.

The decision to enter the engineering and manufacturing development phase
and production phases 10 is to be made in the third quarter of fiscal year
2002. The 1- year on- orbit test, which is intended to test and finalize the
design of the satellites, will not be completed until January 2008, more
than 5 years after development and production is to start. In contrast,
under previous schedules (see app. I), the Air Force had stressed the
importance of on- orbit tests, stating they were critical to support the
decision to enter production. According to the Air Force, its decision to
enter the engineering and manufacturing development and production phases
will now be based on information obtained from the ground demonstrations

performed under the program definition and risk reduction contracts and from
other completed on- orbit demonstration programs such as the Midcourse Space
Experiment and the Miniature Sensor Technology Integration Program. 11 These
program results, however, may be of limited utility to SBIRS- low. For
example, according to Air Force officials, they plan to use information on
midcourse discrimination collected by the Midcourse Space Experiment in
their decision concerning SBIRS- low development and production. However,
according to DOD's Director of Operational Test and Evaluation, the
Midcourse Space Experiment did not collect discrimination data on objects
representative of those that SBIRS- low must be able to discriminate. 10
This phase consists of the steps necessary to produce and deploy a system
for operational

use and ensure the system meets user's needs. 11 The Midcourse Space
Experiment was the first demonstration in space of the technology (long-
wave infrared) needed to identify and track ballistic missiles during the
midcourse portion (between booster burnout and missile reentry) of their
flight paths. The experiment collected data to characterize and identify
missiles against space backgrounds. The Miniature Sensor Technology
Initiative used several small satellites to demonstrate short- and mid- wave
infrared technology, which is needed to identify missile launches and to
track missiles while boosters are still burning.

According to the Air Force, launches are not to be resumed until after the
1- year on- orbit test period has been completed, test results have been
reviewed, and modifications, if required, have been made to the remaining
satellites. However, the production of satellites will not stop during the

1- year on- orbit test. As a result, by the time the test is to be completed
in fiscal year 2008, 9 satellites will have been produced (including the
first 6 used for flight- testing), an additional 21 satellites will be in
various stages

of production, and at least $1.9 billion of the $2.4 billion (then- year)
cost for these 30 satellites will have been expended or committed. 12
Because the on- orbit test results for crucial functions and capabilities is
not to be available until more than 5 years after the start of production,
there is a risk that design changes will be required for satellites in
production. For

example, if parts that have already been purchased for the SBIRS- low
operational satellites became obsolete because their acquisition was based
on the initial system design, new parts may be required, program costs will
increase, and the schedule will slip. Also, additional changes may be
necessary to the satellite configuration that could affect not just long
lead

items, but also modifications may be required to satellite components
already produced. In a July 1999 memorandum to the Under Secretary of
Defense for Acquisition and Technology, DOD's Director of Operational Test
and Evaluation expressed concern that the new (current) schedule eliminated
critical on- orbit experiments that were to be conducted under the flight
demonstration. The Director stated that while the restructured program

schedule includes ground demonstrations that were previously lacking from
the SBIRS- low program, considering the many technical challenges and high
risk in the program, DOD must seek every opportunity to obtain

early on- orbit experience. According to the Director, many of the functions
and capabilities that must be demonstrated (and would have been demonstrated
under the flight demonstration) before SBIRS- low exits the program
definition and risk reduction phase and enters the engineering and
manufacturing development phase are impossible to demonstrate with only
ground tests. For example, the Director stated that DOD has no flight
experience where two or more satellites in low earth orbit have

communicated with each other. He stated that this was challenging 12 While
the system is to consist of a constellation of about 24 satellites, the
program provides for the production of an additional 34 satellites to
maintain this constellation size through fiscal year 2022. Therefore, the 30
satellites discussed here include the 24 for the initial constellation and 6
replenishment satellites.

because of the dynamically changing positions of orbiting satellites
relative to each other and the high data rates needed to transmit data
between satellites thousands of kilometers 13 apart. Another example cited
by the Director where DOD has no flight experience is with coordinating the
operation of acquisition and tracking infrared sensors, both of which are to
be mounted on each SBIRS- low satellite. Specifically, when the acquisition

sensor detects the heat from a missile's booster motor, it must determine
and relay highly accurate information on the missile's position to the
tracking sensor. The tracking sensor must then point to the proper location
in space, find the missile, and begin tracking the missile. All of these
activities must occur within short time frames (seconds) to support missile
defense.

We have reported on numerous occasions about the risks associated with
program concurrency and of initiating production without adequate testing.
In a 1990 testimony, we cited the Navy's F/ A- 18 aircraft, the Air

Force's B- 1B Bomber, and the Navy's AEGIS Destroyer as examples where a
rush to production without adequate testing resulted in increased costs,
lower than expected performance, or both. In 1994 and 1995, we reported that
programs are often permitted to begin production with little or no scrutiny
and that the consequences have included procurement of substantial
inventories of unsatisfactory weapons requiring costly modifications to
achieve satisfactory performance, and in some cases,

deployment of substandard systems to combat forces. In 2000, we reported
that programs were allowed to begin production before the contractors and
the government had conducted enough testing to know whether the systems'
design would meet requirements. 14 13 One thousand kilometers equals 620
miles.

14 Weapon Systems: Concurrency in the Acquisition Process (GAO/ T- NSIAD-
90- 43, May 17, 1990); Weapons Acquisition: Low- Rate Initial Production
Used to Buy Weapon Systems Prematurely (GAO/ NSIAD- 95- 18, Nov. 21, 1994);
Tactical Aircraft: Concurrency in Development and Production of F- 22
Aircraft Should Be Reduced (GAO/ NSIAD- 95- 59, Apr. 19, 1995); Defense
Acquisitions: Need to Revise Acquisition Strategy to Reduce Risk for Joint
Air- to- Surface Standoff Missile (GAO/ NSIAD- 00- 75, Apr. 26, 2000); and
Missile Defense: Schedule for Navy Theater Wide Program Should Be Revised to
Reduce Risk (GAO/ NSIAD- 00- 121, May 31, 2000).

Evolutionary Software In December 1999, the SBIRS- low program office
concluded that

Development Plan Increases development of software to perform all SBIRS- low
missions, as originally Program Risk

scheduled, could not be completed 1 year before the scheduled first launch
of SBIRS- low satellites in fiscal year 2006. According to the Air Force,
this conclusion was based on lessons learned from other programs under which
software development efforts were underestimated. As a result, to maintain
the fiscal year 2006 first launch, the program office plans to use an

evolutionary software development approach under which software is to be
developed in increments. The software needed to support all SBIRS- low
missions will not be completed (ready for use for satellite operations)
until March 2010, over 3 years after the first satellites are launched.
According to Air Force officials, DOD traditionally completes software
required to support satellite systems 1 year before the scheduled first
launch of a new satellite system. DOD established this practice to reduce
risk by ensuring that all system problems have been identified and resolved,
and that the personnel operating the systems have been adequately trained.
This was the original plan for the SBIRS- low program. Under the
evolutionary approach, software will be developed to support

satellite launches, early on- orbit testing, ballistic missile defense, and
integration with SBIRS- high, followed by the software needed to support
ancillary missions, such as technical intelligence and battlespace

characterization. Figure 2 shows the schedule for the incremental
development and completion of the software relative to the launch and
testing schedule for the SBIRS- low satellites.

Figure 2: Evolutionary Development of SBIRS- low Software

1 2 Initial launches 1- year on- orbit test

3 456 7 89 10 Remaining launches

Software development increments: 1. Initial operations capability required
for launches

2. Added capability to support satellite tests

3. Full ballistic missile defense capability for entire constellation

4. Full integration of SBIRSlow with SBIRS- high

5. Ancillary missions 2005 2006 2007

2008 2009 2010 Fiscal Year Source: Air Force.

As figure 2 shows, by the time the on- orbit test period for the first six
SBIRS- low satellites is to begin in fiscal year 2007, the first two
increments of software are to be completed. According to program office
officials, these two increments of software will provide all of the
capabilities the ground control system and the satellites need to support
and perform the on- orbit test. The third increment, the ground control and
space related software required to operate the full satellite constellation
in support of ballistic missile defense, is not to be completed until fiscal
year 2008. The fourth software increment, which is to be completed in mid-
fiscal year 2009, is to integrate SBIRS- low with SBIRS- high. The fifth
increment, which is to be completed in mid- fiscal year 2010, is to add the
software required for SBIRS- low to perform ancillary missions such as
technical

intelligence, battlespace characterization, and space surveillance. Thus,
the software required to support all of SBIRS- low missions is not to be
completed until over 3 years after the first satellites are launched. While
this evolutionary approach reduces schedule pressure for completing the
ground control and space software before the first launch in fiscal year
2006, it increases the risk that software may not be available when needed
or perform as required. Under the traditional approach, all software would
have been completed in fiscal year 2005, 1 year before the launch of the
first satellites. SBIRS- low Program

The SBIRS- low program has high technical risks because some critical
Includes Immature satellite technologies have been judged to be immature for
the current stage of the program. Specifically, the SBIRS- low program
office rated the

Critical Technologies maturity of five of six critical technologies at
levels that constitute high risk the technologies will not be available when
needed.

In developing a complex system, an assessment of the maturity levels of
critical technologies can provide information on the risks those maturity
levels pose if the technologies are to be included in the development. For

example, in a previous report, 15 we discuss a tool, referred to as
Technology Readiness Levels, the National Aeronautics and Space
Administration and Air Force Research Laboratory use to determine the
readiness of technologies to be incorporated into a weapon system. 16 The
readiness levels are measured along a scale of one to nine, starting with
paper studies

of the basic concept and ending with a technology that has proven itself in
actual usage on the intended product. The Air Force Research Laboratory
considers a readiness level of six to be an acceptable risk for a program 15
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes (GAO/ NSIAD- 99- 162, July 30, 1999). We concluded
that the incorporation of advanced technologies before they are mature has
been a major source of cost increases, schedule delays, and performance
problems on weapon systems and recommended that the

Secretary of Defense adopt a disciplined and knowledge- based approach of
assessing technology maturity, such as Technology Readiness Levels, DOD-
wide, and establish the point at which a match is achieved between key
technologies and weapon system requirements as the proper point for
committing to the development and production of a weapon system. In
response, DOD agreed that Technology Readiness Levels can be used to help
guide technology maturation and transition decisions.

16 The Air Force Research Laboratory is a science and technology
organization that matures advanced technologies to the point that they can
be included in weapon system programs and be expected to perform as
required. The Laboratory uses the Technology Readiness

Levels to assess the maturity of the technologies before they are handed off
to programs.

entering the Program Definition and Risk Reduction phase- the Laboratory
considers lower readiness levels at this stage to translate to high program
cost, schedule, and performance risks. Reaching a readiness level of six
denotes a significant transition point for technology development in which
the technology moves from component testing in a laboratory environment to
demonstrating a model or prototype in a

relevant environment. At our request, the SBIRS- low program office rated
the maturity, as of the start of the Program Definition and Risk Reduction
phase, of six technologies critical to the success of the SBIRS- low
program. The program office rated five of the six technologies at levels
that, according to criteria used by the Air Force Research Laboratory,
constitute high risk in the ability of the program to meet its objectives. A
detailed description of the Technology Readiness Levels is provided in
appendix II.

As shown in figure 3, SBIRS- low entered the Program Definition and Risk
Reduction phase with a number of critical subsystem technologies with
maturities below a readiness level of six.

Figure 3: Technology Readiness Levels of Critical SBIRS- low Subsystems at
the Start of Program Definition and Risk Reduction

Scanning infrared sensor Tracking infrared sensor

Goal

Fore optics cryocooler Tracking infrared sensor cryocooler Satellite
communications crosslinks

On- board computer processor 1 2 3

4 5 6 7

Technology Readiness Levels

Specifically, the program office rated the maturity of the (1) scanning
infrared sensor, which is to acquire ballistic missiles in the early stages
of flight, at a readiness level of four; (2) tracking infrared sensor, which
is to track missiles, warheads, and other objects such as debris and decoys

during the middle and later stages of flight, at a readiness level of four;
(3) fore optics cryocooler and (4) tracking infrared sensor cryocooler,
which are needed to cool the tracking sensor optics and other sensor
components to enable the sensor to detect missile objects in space, at
readiness levels of four; (5) satellite communications crosslinks, which
enable satellites to communicate with each other, at a readiness level of
five; and (6) on- board computer processors, critical for performing

complex and autonomous satellite operations for providing missile warning
and location information within short time frames, at a level of six. So
critical are each of these subsystem technologies is that if one is not
available when needed, SBIRS- low would be unable to perform its mission.
And, in sum, five of six critical technologies are at a low maturity level,
causing high program risk.

Alternative Current DOD acquisition policy and procedures require that
assessments

be made of the cost and mission effectiveness of space systems to Approaches
to alternative terrestrial- land, sea, and air- systems. 17 Despite this
SBIRS- low Have Not requirement, DOD has not adequately analyzed or
identified cost- effective

Been Assessed alternatives to SBIRS- low that could satisfy critical missile
defense requirements such as a Navy ship- based radar capability. Compliance
with this requirement would seem especially important, given the high risks
identified with the SBIRS- low program.

Terrestrial alternatives to SBIRS- low are not being considered. While
competing SBIRS- low contractors are performing cost and trade studies on
the various options that could satisfy program requirements, none of these
studies is to consider the cost- effectiveness of terrestrial alternatives.
The most recent study assessing alternatives to SBIRS- low was performed in
1994; 18 however, according to an Air Force Space Command official, the
study's scope was focused only on options that would use space- based
infrared sensors; terrestrial options were not included. According to Air
Force Space Command officials, terrestrial alternatives to SBIRS- low do not
exist.

17 Department of Defense Directive 5000.1, Defense Acquisition, March 15,
1996; Department of Defense Regulation 5000.2- R, Mandatory Procedures for
Major Defense Acquisition Programs (MDAPs); and Major Automated Information
System (MAIS) Acquisition Programs, May 11, 1999. In addition, in commenting
on a recent report in which we evaluated the extent to which plans for
expanding military space systems conform to national and defense space
policies ( Defense Acquisitions: Improvements Needed in Military Space
Systems' Planning and Education (GAO/ NSIAD 00- 81, May 18, 2000)), DOD

stated that program assessments should highlight the relative advantages and
disadvantages of a full range of alternatives, both space- and terrestrial-
based alternatives. 18 This study was referred to as the Office of the
Secretary of Defense Space- Based Warning Summer Study. The Summer Study was
initiated to consolidate infrared space requirements and resulted in SBIRS
being selected to replace the Defense Support Program.

Studies on various aspects of the National Missile Defense System by the
Ballistic Missile Defense Organization and other organizations have pointed
out that alternatives to SBIRS- low may exist. For example, the

Ballistic Missile Defense Organization's June 1999 study, which assessed
whether and how the Navy Theater Wide program, a DOD program to develop a
ship- based theater missile defense capability, could be upgraded to provide
a limited national missile defense capability, 19 cited the potential
utility of sea- based radars to a national missile defense capability.
Specifically, the report states that properly deployed ship- based radars
can

provide a forward- based radar warning and tracking function against many of
the potential ballistic missile threats to the United States, and that
because the radars would be difficult to target due to the mobility and
unknown locations of ships, the radars would add robustness against enemy
attacks, particularly before SBIRS- low is available.

In a 1999 RAND issue paper that dealt with an assessment of the planning for
the National Missile Defense System, the authors suggest that ground- based
radars could potentially be used to provide midcourse tracking and cueing
for interceptors. 20 Specifically, they conclude that the planned initial
capability of the National Missile Defense System is inadequate and suggest
that an interim solution be considered to enhance the system's capabilities
against more sophisticated, larger, and more

geographically dispersed ballistic missile threats prior to the next planned
enhancement to the missile defense system. They suggest that one aspect of
the interim solution could include deploying additional ground- based

radars to perform ballistic missile tracking and discrimination functions,
or alternately, speeding the deployment of SBIRS- low. Conclusions The Air
Force is implementing a high- risk acquisition schedule for the

SBIRS- low program in an attempt to deploy the system starting in fiscal
year 2006 to support the National Missile Defense System. The highly
concurrent acquisition schedule has evolved because of design, development,
and technology challenges, as well as the importance Congress has placed on
the deployment of a National Missile Defense 19 Summary of Report to
Congress on Utility of Sea- Based Assets to National Missile Defense,
Ballistic Missile Defense Organization (June 1999).

20 Planning a Ballistic Missile Defense System of Systems: An Adaptive
Strategy, RAND, National Defense Research Institute (1999).

capability. Although the schedule includes on- orbit tests to finalize
satellite design and performance, the results will not be available in time
to be useful for informed decision- making related to satellite design and
production. In addition, the Air Force's evolutionary software development
approach

creates risk because it delays completion of the software needed to support
all SBIRS- low missions over 3 years after the first launch of SBIRS- low
satellites. Finally, critical satellite technologies that have been judged
to be immature

for the current phase of the program, place program success in peril. Due to
these deficiencies, the SBIRS- low program is at high risk of not delivering
the system on time or at cost or with expected performance. In spite of the
high risk that SBIRS- low will not be available to support the National
Missile Defense System when needed, DOD has not identified alternatives or
interim solutions.

Recommendations for In order to reduce the cost, schedule, performance, and
technical risks in Executive Action

the SBIRS- low program, we recommend that the Secretary of Defense direct
the Secretary of the Air Force, with the approval of the Director of the
Ballistic Missile Defense Organization, to develop a schedule that reduces
concurrency and risks, and that sets more realistic and achievable cost,
schedule, and performance goals. In addition, the Secretary of

Defense should assess the impact of the revised schedule on the National
Missile Defense program and provide the results of the assessment to
Congress. We also recommend that the Secretary of Defense direct the
Director, Ballistic Missile Defense Organization, to analyze and develop, as
appropriate, and in compliance with DOD acquisition policy and

procedures, alternative approaches to satisfy critical missile defense
midcourse tracking and discrimination requirements in case SBIRS- low cannot
be deployed when needed (based on the resulting lower risk SBIRS- low
schedule, threat analyses, and missile defense program schedules).

Agency Comments and In written comments to a draft of this report, DOD
generally agreed with

Our Evaluation our recommendations. DOD also pointed out that it is taking
actions that it believes will address our recommendations. These actions
begin to address our concerns, but they are not yet completed or approved,
and it is not

clear yet whether they will fully address the risks identified by our
review. Therefore, our recommendations are still relevant. Our first
recommendation deals with restructuring the SBIRS- low acquisition schedule
to reduce cost, schedule, performance, and technical risks; assessing the
impact of the restructured schedule on the National Missile Defense program;
and providing the results of the assessment to Congress. DOD stated it has
developed a proposed update to the SBIRS- low acquisition strategy that it
believes addresses our concerns for

concurrency in the production phase, while still retaining the fiscal year
2006 first launch date. For example, DOD's proposed strategy would delay the
full operational capability date of the first SBIRS- low constellation by 1
year and allow for additional ground demonstration program activities and
on- orbit testing, thus reducing concurrency between production and testing,
while maintaining the schedule for implementation of the full constellation.
DOD stated that the Ballistic Missile Defense Organization is assessing the
impact of this delay on the National Missile

Defense program. This proposal has not been approved and will be reviewed
for final decision by the Under Secretary of Defense for Acquisition,
Technology, and Logistics in May 2001. Since DOD's proposed update to the
SBIRS- low acquisition strategy has not been approved (due to cost concerns)
and will not be considered again for approval until May 2001, we did not
assess the proposed strategy in any detail. On the surface, the additional
on- orbit testing does somewhat reduce production concurrency. However, even
with this additional testing, the program still appears to have high
concurrency risk, for example, with substantial long lead time procurement
before testing results are complete.

Therefore, we believe our recommendation is still appropriate in relation to
the new proposal or in light of any changes to DOD's new proposal. With
regard to our second recommendation, DOD stated that it has initiated a
study to address viable alternatives to SBIRS- low capabilities and will
provide the results of the study to the Deputy Secretary of Defense on March
1, 2001. While initiation of this study is a good beginning, until it is
complete, we cannot assess the extent to which alternatives will be

identified and whether critical missile defense requirements allocated to
SBIRS- low will be satisfied. DOD's comments are reprinted in appendix III.
DOD also provided separate technical comments that we have incorporated in
this report where appropriate. Scope and To evaluate risks of the current
acquisition schedule, we had discussions Methodology

with officials of the Under Secretary of Defense for Acquisition,
Technology, and Logistics; the Under Secretary of Defense for the
Comptroller; the Assistant Secretary of Defense for Command, Control,
Communications, and Intelligence; the Office of the Director, Operational
Test and Evaluation; the Office of Program Analysis and Evaluation; and the
Assistant Secretary of the Air Force for Acquisition, all in Washington, D.
C. We also held discussions with, and reviewed documents from, officials of
the SBIRS program office in Los Angeles, California; the U. S. and Air Force
Space Commands, Colorado Springs, Colorado; the Defense Contract Management
Agency offices in Van Nuys, California, and Phoenix, Arizona; the Air Force
Operational Test and Evaluation Center, Buckley Air National Guard Base,
Aurora, Colorado; TRW, Inc., Redondo Beach, California; and Spectrum Astro,
Gilbert, Arizona.

To evaluate technical risks of the program, we had discussions with, and
reviewed documents from, officials of the program office; the Office of the
Director, Operational Test and Evaluation; and the Air Force Research
Laboratory, Albuquerque, New Mexico. We also discussed technical risks with
TRW and Spectrum Astro. To determine whether DOD has assessed alternative
approaches to SBIRS- low, we had discussions with the Ballistic Missile
Defense Organization, in Washington, D. C.; the program office; and U. S.
and Air Force Space Commands. We also reviewed two related studies by the
Ballistic Missile Defense Organization and the RAND Corporation.

We performed our work from May 1999 through December 2000 in accordance with
generally accepted government auditing standards.

If you or your staff have any questions concerning this report, please call
me on (404) 679- 1900. The GAO contact and staff acknowledgments are listed
in appendix IV. Sincerely yours,

Louis J. Rodrigues Director, Defense Acquisitions Issues

Appendi xes Space- Based Infrared System- low Program

Appendi x I

History Original Schedule The Department of Defense's (DOD) original 1995
schedule for Space- Based Infrared System- low (SBIRS)- low called for (1) a
launch of a two- satellite flight demonstration- both satellites on one
launch vehicle- in the first quarter of fiscal year 1999; (2) a deployment
decision in fiscal year 2000 after key technologies and operating concepts
were validated by the demonstration satellites; and (3) launches of SBIRS-
low satellites- 3 satellites per launch vehicle- beginning in fiscal year
2006.

According to Air Force officials, the satellite flight demonstration was
critical to validate the integration of key technologies and operational
concepts that are crucial to national missile defense and other SBIRS
missions such as technical intelligence and battlespace characterization.
The primary emphasis was to be on the ability to detect and track ballistic
missiles and their warheads throughout flight and distinguish between
missile warheads and decoys. The Air Force planned to test these satellites'
ability to perform national missile defense functions against live theater
and national missile defense targets and to use the demonstration and test
results to model and simulate the full performance capability of a

constellation of operational SBIRS- low satellites. According to the program
officials who established this acquisition strategy, performing this
function autonomously while in orbit is one of the most complex and
technologically challenging operational concepts ever attempted. They also
stressed that the two- flight demonstration satellites would have provided
an informed basis for deciding whether the program was ready to enter the
engineering and manufacturing development and

production phases of the acquisition process. They stated that a National
Missile Defense System with space- based sensors depended on a successful
flight demonstration program and that proceeding into the engineering and
manufacturing development and production phases before demonstrating this
capability would not provide an opportunity to assess

lessons learned, thus introducing unacceptable risk into the program. Figure
4 shows the original acquisition schedule for a fiscal year 2006 first
launch of SBIRS- low satellites.

Figure 4: Original SBIRS- low Schedule for First Launch in Fiscal Year 2006

Flight Demonstration

Development

Launch

Demonstration Year 1 Year 2 Pre- Engineering and Manufacturing Development

Engineering and Manufacturing Development and Production

Launch 1998 1999 2000

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Fiscal Year

Source: Air Force.

Under this schedule, the first year of the planned 2- year flight
demonstration would have been completed in the first quarter of fiscal year
2000, about the same time the program was scheduled to enter the pre-
engineering and manufacturing development phase. The first year results from
the demonstration could have influenced requirements development and system
design during this phase. The second year of the demonstration would have
been completed in the first quarter of fiscal year 2001, about the same time
the program was scheduled to enter the

engineering and manufacturing development and production phases. Thus, DOD
would have had almost 2- years of information on the demonstration
satellites' performance to consider in deciding whether the system should

enter the engineering and manufacturing development and production phases.

Accelerated Schedule DOD did not implement the original schedule because
Congress required in Established in the National Defense Authorization Act
for Fiscal Year 1996 that DOD

establish a program baseline to include a first launch of SBIRS- low
December 1996

satellites in fiscal year 2002. 1 The Defense Science Board, at DOD's
request, assessed the viability of accelerating the first launch from fiscal
year 2006 to fiscal year 2002 and found it would not be viable; however, it
did

determine that the first launch could be accelerated to fiscal year 2004. 2
Subsequently, DOD informed Congress that the first launch of SBIRS- low
satellites could not begin in fiscal year 2002 because technical, funding,
and management problems had delayed the scheduled launch of the two
demonstration satellites from the first quarter to the third quarter of
fiscal year 1999. According to Air Force officials, this delay prevented
basing a

milestone decision to enter the engineering and manufacturing development
and production phases of the SBIRS- low acquisition process, scheduled for
the first quarter of fiscal year 2000, on the results of the planned flight
demonstration. However, in December 1996, DOD committed to accelerating the
first launch of SBIRS- low satellites to fiscal year 2004. Figure 5 shows
the acquisition schedule for the flight demonstration and a fiscal year 2004
first launch of SBIRS- low. 1 P. L. 104- 106, section 216.

2 In 1996, we also assessed various SBIRS- low deployment options and
identified the cost and the risks associated with each option, including the
option DOD selected. We recommended that DOD provide Congress with complete,
consistent, and current information regarding all of the deployment options
it considered. See National Missile Defense: Risk and Funding Implications
for the Space- Based Infrared Low Component (GAO/ NSIAD- 97- 16, Feb. 25,
1997).

Figure 5: Revised SBIRS- low Schedule for First Launch in Fiscal Year 2004

Flight Demonstration

Development

Launch

Demonstration Year 1 Year 2 Pre- Engineering and Manufacturing Development

Engineering and Manufacturing Development and Production

Launch 1999 2000 2001

2002 2003 2004 2005 2006 2007 2008 2009 2010 Fiscal Year Source: Air Force.

Under this acquisition schedule, the demonstration satellites were to be
launched in the third quarter of fiscal year 1999, two quarters later than
scheduled under the original schedule. Consequently, the flight
demonstration and the pre- engineering and manufacturing development phase
would have run concurrently and the demonstration results could

not have influenced the development of requirements and the system design as
they could have under the original schedule. However, the first year of the
flight demonstration would still have been completed about 4 months before
the start of the engineering and manufacturing development and production
phases, which were still scheduled to begin in the first quarter of fiscal
year 2001 as they were under the original

schedule. As a result, DOD would have had the information from the first
year of the demonstration satellites' performance, which it considered the
most critical in deciding whether the system should enter these phases, to
support a fiscal year 2004 deployment.

Technology Readiness Levels and Their

Appendi x II

Definitions Technology readiness level Description

1. Basic principles observed and Lowest level of technology readiness.
Scientific research begins to be translated into applied reported. research
and development. Examples might include paper studies of a technology's
basic properties. 2. Technology concept and/ or Invention begins. Once basic
principles are observed, practical applications can be invented.

application formulated. The application is speculative, and there is no
proof or detailed analysis to support the assumption. Examples are still
limited to paper studies.

3. Analytical and experimental critical Active research and development is
initiated. This includes analytical studies and laboratory function and/ or
characteristic proof studies to physically validate analytical predictions
of separate elements of the technology. of concept.

Examples include components that are not yet integrated or representative.
4. Component and/ or breadboard Basic technological components are
integrated to establish that the pieces will work together. validation in
laboratory This is relatively “low fidelity” compared to the
eventual system. Examples include integration environment.

of “ad hoc” hardware in a laboratory. 5. Component and/ or
breadboard Fidelity of breadboard technology increases significantly. The
basic technological components validation in relevant environment. are
integrated with reasonably realistic supporting elements so that the
technology can be tested in a simulated environment. Examples include
“high fidelity” laboratory integration of components.

6. System/ subsystem model or Representative model or prototype system,
which is well beyond the breadboard tested for prototype demonstration in a
technology readiness level 5, is tested in a relevant environment.
Represents a major step up relevant environment. in a technology's
demonstrated readiness. Examples include testing a prototype in a high
fidelity laboratory environment or in simulated operational environment.

7. System prototype demonstration in Prototype near or at planned
operational system. Represents a major step up from technology an
operational environment. readiness level 6, requiring the demonstration of
an actual system prototype in an operational environment, such as in an
aircraft, vehicle, or space. Examples include testing the prototype in a
test bed aircraft. 8. Actual system completed and “flight Technology
has been proven to work in its final form and under expected conditions. In
almost qualified” through test and

all cases, this technology readiness level represents the end of true system
development. demonstration.

Examples include developmental test and evaluation of the system in its
intended weapon system to determine if it meets design specifications.

9. Actual system “flight proven” Actual application of the
technology in its final form and under mission conditions, such as through
successful mission those encountered in operational test and evaluation. In
almost all cases, this is the end of the operations.

last “bug fixing” aspects of true system development. Examples
include using the system under operational mission conditions. Source: Best
Practices: Better Management of Technology Development Can Improve Weapon
System Outcomes (GAO/ NSIAD- 99- 162, July 30, 1999).

Appendi x II I Comments From the Department of Defense Note: GAO comments
supplementing those in the report text appear at the end of this appendix.

See comment 1. See comments 2 and 3.

Now on p. 19. Now on p. 19.

Now on p. 19.

The following are our comments on DOD's letter dated December 14, 2000. GAO
Comments 1. We are cognizant of the fact that the National Missile Defense
program is driving the need date for SBIRS- low and not the converse. We do
not intend to suggest that the SBIRS- low acquisition schedule be a driver
for the National Missile Defense program schedule. Our primary goal in
making this recommendation is to help ensure SBIRS- low is acquired at lower
risk and will satisfy critical missile defense requirements. This is

why we are also making our second recommendation- to develop alternative
approaches to satisfy critical missile defense midcourse tracking and
discrimination requirements in case SBIRS- low (under a new lower risk
schedule) cannot be deployed when needed. 2. We disagree that we misstate
the risks of the SBIRS- low incremental

software development strategy. We recognize that an evolutionary, or
incremental, approach to software development is valid. However, an
acquisition approach such as the original SBIRS- low approach that calls for
the completion of all software prior to the first launch poses less risk
than one that does not, that is, the evolutionary or current approach. From
the perspective of meeting the schedule for a first launch in fiscal year
2006, the evolutionary software development approach may reduce schedule
risk because, according to the Air

Force, the first launch date would be unachievable under the original
strategy due to an underestimation of the software development effort.
However, from the perspective of comparing the evolutionary software
development approach with the original approach, there is increased program
risk associated with the evolutionary approach because there is less
assurance the software will be completed when needed with the mission
capabilities specified. 3. While we agree that a revised acquisition
strategy would likely increase

costs, cost increases associated with program delays or rework could also
occur under the current schedule. Due to the highly concurrent acquisition
schedule, we believe that there is substantial risk that delays and rework
resulting from the production of hardware and

software that fail to satisfy requirements may occur- resulting in cost
increases if the current schedule is strictly adhered to. We believe that
early effort to understand acquisition options and the associated costs is
important.

Appendi x I V

GAO Contact and Staff Acknowledgments GAO Contact James H. Solomon (303)
572- 7315 Staff

In addition to the person named above, Ted B. Baird, Richard Y. Horiuchi,
Acknowledgments

and Robert W. Stewart of the Denver Field Office and David G. Hubbell and
Dale M. Yuge of the Los Angeles Field Office made key contributions to this
report.

(707421) Lett er

Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to be mailed to a single
address are discounted 25 percent.

Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013

Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC

Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537

Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.

Orders by Internet: For information on how to access GAO reports on the
Internet, send an e- mail message with “info” in the body to:

info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov

To Report Fraud,

Contact one:

Waste, or Abuse in ? Web site: http:// www. gao. gov/ fraudnet/ fraudnet.
htm

Federal Programs

? e- mail: fraudnet@ gao. gov ? 1- 800- 424- 5454 (automated answering
system)

GAO United States General Accounting Office

Page 1 GAO- 01- 6 SBIRS- low

Contents

Page 2 GAO- 01- 6 SBIRS- low

Page 3 GAO- 01- 6 SBIRS- low United States General Accounting Office

Washington, D. C. 20548 Page 3 GAO- 01- 6 SBIRS- low

Page 4 GAO- 01- 6 SBIRS- low

Page 5 GAO- 01- 6 SBIRS- low

Page 6 GAO- 01- 6 SBIRS- low

Page 7 GAO- 01- 6 SBIRS- low

Page 8 GAO- 01- 6 SBIRS- low

Page 9 GAO- 01- 6 SBIRS- low

Page 10 GAO- 01- 6 SBIRS- low

Page 11 GAO- 01- 6 SBIRS- low

Page 12 GAO- 01- 6 SBIRS- low

Page 13 GAO- 01- 6 SBIRS- low

Page 14 GAO- 01- 6 SBIRS- low

Page 15 GAO- 01- 6 SBIRS- low

Page 16 GAO- 01- 6 SBIRS- low

Page 17 GAO- 01- 6 SBIRS- low

Page 18 GAO- 01- 6 SBIRS- low

Page 19 GAO- 01- 6 SBIRS- low

Page 20 GAO- 01- 6 SBIRS- low

Page 21 GAO- 01- 6 SBIRS- low

Page 22 GAO- 01- 6 SBIRS- low

Page 23 GAO- 01- 6 SBIRS- low

Page 24 GAO- 01- 6 SBIRS- low

Appendix I

Appendix I Space- Based Infrared System- low Program History

Page 25 GAO- 01- 6 SBIRS- low

Appendix I Space- Based Infrared System- low Program History

Page 26 GAO- 01- 6 SBIRS- low

Appendix I Space- Based Infrared System- low Program History

Page 27 GAO- 01- 6 SBIRS- low

Appendix I Space- Based Infrared System- low Program History

Page 28 GAO- 01- 6 SBIRS- low

Page 29 GAO- 01- 6 SBIRS- low

Appendix II

Page 30 GAO- 01- 6 SBIRS- low

Appendix III

Appendix III Comments From the Department of Defense

Page 31 GAO- 01- 6 SBIRS- low

Appendix III Comments From the Department of Defense

Page 32 GAO- 01- 6 SBIRS- low

Page 33 GAO- 01- 6 SBIRS- low

Appendix IV

United States General Accounting Office Washington, D. C. 20548- 0001

Official Business Penalty for Private Use $300

Address Correction Requested Presorted Standard

Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***