    Missile Defense: Additional Knowledge Needed in Developing System
    for Intercepting Long-Range Missiles (21-AUG-03, GAO-03-600).

    A number of countries hostile to the United States and its allies
    have or will soon have missiles capable of delivering nuclear,
    biological, or chemical weapons. To counter this threat, the
    Department of Defense's (DOD's) Missile Defense Agency (MDA) is
    developing a system to defeat ballistic missiles. MDA expects to
    spend $50 billion over the next 5 years to develop and field this
    system. A significant portion of these funds will be invested in
    the Ground-based Midcourse Defense (GMD) element. To field
    elements as soon as practicable, MDA has adopted an acquisition
    strategy whereby capabilities are upgraded as new technologies
    become available and is implementing it in 2-year blocks. Given
    the risks inherent to this strategy, GAO was asked to determine
    when MDA plans to demonstrate the maturity of technologies
    critical to the performance of GMD's Block 2004 capability and to
    identify the estimated costs to develop and field the GMD element
    and any significant risks with the estimate.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-03-600
        ACCNO:   A08177
      TITLE:     Missile Defense: Additional Knowledge Needed in
    Developing System for Intercepting Long-Range Missiles
         DATE:   08/21/2003
      SUBJECT:   Ballistic missiles
    	     Cost analysis
    	     Defense capabilities
    	     Operational testing
    	     Weapons research and development
    	     Weapons systems
    	     DOD Ground-based Midcourse Defense
    	     Element

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-03-600

    Report to the Ranking Minority Member, Subcommittee on Financial
    Management, the Budget, and International Security, Committee on
    Governmental Affairs, U. S. Senate

    United States General Accounting Office

    GAO

    August 2003 MISSILE DEFENSE Additional Knowledge Needed in Developing
    System for Intercepting Long- Range Missiles

    GAO- 03- 600

    GMD is a sophisticated weapon system being developed to protect the United
    States against limited attacks by long- range ballistic missiles. It
    consists of a collection of radars and a weapon component* a three- stage
    booster and exoatmospheric kill vehicle* integrated by a centralized
    control system that formulates battle plans and directs the operation of
    GMD components. Successful performance of these components is dependent on
    10 critical technologies.

    MDA expects to demonstrate the maturity of most of these technologies
    before fielding the GMD element, which is scheduled to begin in September
    2004. However, the agency has accepted higher cost and schedule risks by
    beginning integration of the element*s components before these
    technologies have matured. So far, MDA has matured two critical GMD
    technologies. If development and testing progress as planned, MDA expects
    to demonstrate the maturity of five other technologies by the second
    quarter of fiscal year 2004.

    The radar technologies are the least mature. MDA intends to demonstrate
    the maturity of an upgraded early warning radar in California in the first
    quarter of fiscal year 2005 and a sea- based radar in the Pacific Ocean in
    the fourth quarter of that year. Although MDA does not plan to demonstrate
    the maturity of the technology of the early warning radar in Alaska, which
    will serve as the primary fire control radar, through its own integrated
    flight tests, it may be able to do so through the anticipated launch of
    foreign test missiles.

    MDA estimates that it will spend about $21.8 billion between 1997 and 2009
    to develop the GMD element. This estimate includes $7. 8 billion to
    develop and field the GMD Block 2004 capability. For example, the funds
    will be used to install interceptors at two sites, upgrade existing radars
    and testing infrastructure, and develop the sea- based X- band radar. We
    found that MDA has incurred a greater risk of cost growth because for more
    than a year the agency was not able to rely fully on data from its primary
    tool for monitoring

    whether the GMD contractor has been performing work within cost and on
    schedule. In February 2002, MDA modified the prime contract to reflect an
    increased scope of work for developing GMD. It was not until July 2003
    that the agency completed a review to ensure that the data was fully
    reliable. A number of countries hostile to the United States and its
    allies have or will soon have missiles capable

    of delivering nuclear, biological, or chemical weapons. To counter this
    threat, the Department of Defense*s (DOD*s) Missile Defense Agency (MDA)
    is developing a system to

    defeat ballistic missiles. MDA expects to spend $50 billion over the next
    5 years to develop and field this system. A significant portion of these
    funds will be

    invested in the Ground- based Midcourse Defense (GMD) element. To field
    elements as soon

    as practicable, MDA has adopted an acquisition strategy whereby
    capabilities are upgraded as new technologies become available and is
    implementing it in 2- year blocks.

    Given the risks inherent to this strategy, GAO was asked to determine when
    MDA plans to demonstrate the maturity of technologies critical to the

    performance of GMD*s Block 2004 capability and to identify the estimated
    costs to develop and field the GMD element and any significant risks with
    the estimate.

    GAO is recommending DOD (1) explore options to demonstrate effectiveness
    of the Cobra Dane radar and (2) establish procedures to help ensure data
    are reliable from MDA*s monitoring system.

    DOD concurred with GAO*s first recommendation and partially concurred with
    GAO*s second.

    www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 600. To view the full product,
    including the scope and methodology, click on the link above. For more
    information, contact Robert E. Levin at (202) 512- 4841 or levinr@ gao.
    gov. Highlights of GAO- 03- 600, a report to the

    Ranking Minority Member, Subcommittee on Financial Management, the Budget,
    and International Security, Committee on Governmental Affairs, U. S.
    Senate

    August 2003

    MISSILE DEFENSE

    Additional Knowledge Needed in Developing System for Intercepting Long-
    Range Missiles

    Page i GAO- 03- 600 Missile Defense Letter 1 Results in Brief 3 Background
    4 MDA Expects to Demonstrate the Maturity of Most GMD

    Technologies before September 2004 8 MDA Has Risked Cost Growth Because It
    Could Not Fully Rely on Data from Its System for Monitoring Contractor
    Performance 18 Conclusions 24 Recommendations for Executive Action 25
    Agency Comments and Our Evaluation 25 Appendix I Scope and Methodology 28

    Appendix II Comments from the Department of Defense 29

    Appendix III Technology Readiness Level Assessment Matrix 32

    Appendix IV Importance of Earned Value Management 35

    Appendix V GAO Contact and Staff Acknowledgments 40

    Tables

    Table 1: Technology Readiness Levels of GMD Critical Technologies 11 Table
    2: Estimated Cost to Develop and Field GMD 19 Table 3: 32 Criteria for
    Earned Value Management Systems 35 Contents

    Page ii GAO- 03- 600 Missile Defense Figures

    Figure 1: Components of GMD 6 Figure 2: Notional GMD Concept of Operations
    7 Figure 3: Tasks GMD Plans to Accomplish for the GMD Block 2004 Project
    19 Abbreviations

    BMDO Ballistic Missile Defense Organization CPR Cost Performance Report
    DCMA Defense Contract Management Agency EVM Earned Value Management GMD
    Ground- based Midcourse Defense IBR integrated baseline review IFT
    integrated flight test

    MDA Missile Defense Agency NMD National Missile Defense TRL technology
    readiness level

    This is a work of the U. S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    Page 1 GAO- 03- 600 Missile Defense

    August 21, 2003 The Honorable Daniel K. Akaka Ranking Minority Member
    Subcommittee on Financial Management,

    the Budget, and International Security Committee on Governmental Affairs
    United States Senate

    Dear Senator Akaka: Hostile states, including those that sponsor
    terrorism, are investing significant resources to develop and deploy
    ballistic missiles of increasing range and sophistication that could be
    used against the United States, our deployed forces, and our allies. At
    least 25 countries now have, or are in

    the process of acquiring, missiles capable of delivering nuclear,
    biological, or chemical weapons. To counter this threat, the President of
    the United States in December 2002, directed the Department of Defense
    (DOD) to begin fielding a ballistic missile defense system in 2004.

    The Missile Defense Agency (MDA) within DOD is responsible for developing
    this system, including the Ground- based Midcourse Defense (GMD) element,
    1 which is being developed to protect the United States against long-
    range ballistic missiles. MDA is also building an integrated testing
    infrastructure* or *test bed** with the newly designated GMD element as
    its centerpiece. MDA expects to spend nearly $50 billion in research and
    development funds between fiscal years 2004 and 2009 to develop and field
    a ballistic missile defense system. A significant percentage of the $50
    billion will be invested in the GMD element.

    GMD is a sophisticated weapon system that will rely on state- of- the- art
    technologies that have been under development for a number of years. GMD
    will use space- based sensors to provide early warning of missile
    launches; ground- based radars to identify and refine the tracks of
    threatening warheads and associated objects; ground- based interceptors

    1 In January 2002, the Secretary of Defense created the Missile Defense
    Agency and consolidated all ballistic missile defense programs under the
    new agency. Former missile defense acquisition programs are now referred
    to as elements of a single ballistic missile defense system.

    United States General Accounting Office Washington, DC 20548

    Page 2 GAO- 03- 600 Missile Defense

    (each consisting of a three- stage booster and exoatmospheric kill
    vehicle) to destroy warheads; and a centralized control system that
    formulates battle plans and directs the operation of GMD components for
    carrying out the missile defense mission.

    To meet the technical challenge of developing both the integrated system
    and the GMD element, MDA has adopted a *capabilities- based* acquisition
    strategy and is implementing it in 2- year development blocks. This
    approach is designed to field elements as soon as practicable and to
    improve the effectiveness of fielded elements by upgrading their
    capability as new technologies become available or as the threat warrants.
    Block 2004 will be the first block fielded, followed by Blocks 2006 and
    2008. Although GMD*s Block 2004 capability is expected to be fielded
    beginning in September 2004, MDA plans to upgrade that capability through
    the end of 2005. 2 Because development and fielding of GMD involves
    substantial technical

    challenges and a major investment, you asked us to review technical and
    cost issues related to the GMD element. Specifically, we determined when
    MDA plans to demonstrate the maturity 3 of technologies critical to the
    performance of GMD*s Block 2004 capability. We also identified the
    estimated costs to develop and field the GMD element and any significant
    risks associated with the estimate.

    Our scope and methodology are included in appendix I. Although we assessed
    the maturity of specific GMD critical technologies, the scope of this
    review did not include an evaluation of MDA*s test plans for demonstrating
    GMD*s ability to operate as a system overall. Our detailed assessment of
    GMD system- level testing is included in a classified report that we
    issued in June 2003 to other congressional requesters.

    2 The intended performance of the Block 2004 capability is described in a
    classified annex to this report. 3 Technological maturity for starting
    product development or systems integration is

    achieved when prototype hardware with the desired form, fit, and function
    has been proven in a realistic operational environment. See U. S. General
    Accounting Office,

    Best Practices: Better Management of Technology Development Can Improve
    Weapon System Outcomes, GAO/ NSIAD- 99- 162 (Washington, D. C.: July
    1999).

    Page 3 GAO- 03- 600 Missile Defense

    MDA expects to demonstrate the maturity of most of the ten technologies
    critical to GMD*s initial performance before fielding of the element
    begins in September 2004. However, the agency has accepted a higher risk
    of cost growth and schedule slips by beginning the integration of the
    element*s components before these technologies have been demonstrated. So
    far, MDA has matured two critical GMD technologies* the infrared sensors
    of the kill vehicle 4 and the fire control software of the battle
    management component. 5 But if development and testing progress as
    planned, MDA expects to demonstrate the maturity of five others* resident
    in the kill vehicle, interceptor boosters, and the battle management
    component* by the second quarter of fiscal year 2004. MDA intends to
    demonstrate the maturity of an upgraded early warning radar* located at
    Beale Air Force Base, California* in the first quarter of fiscal year 2005
    and a sea- based Xband radar, located in the Pacific Ocean, in the fourth
    quarter of that year. MDA does not plan to demonstrate through its own
    integrated flight tests the maturity of a technology resident in the Cobra
    Dane radar located in Alaska, which will serve as the element*s primary
    radar when GMD is first fielded. Agency officials told us that they may be
    able to test the radar through the anticipated launch of foreign test
    missiles. However, it is not clear that testing Cobra Dane in this manner
    will provide all of the information that a dedicated test provides because
    MDA will not control the configuration of the target or the flight
    environment.

    MDA estimates that it will spend about $21.8 billion between 1997 and 2009
    to develop the GMD element. This estimate includes $7.8 billion to develop
    and field the GMD Block 2004 capability and to develop the GMD portion of
    the test bed between 2002 and 2005. For example, the funds will be used to
    install interceptors at Fort Greely, Alaska, and Vandenberg Air Force
    Base, California; upgrade existing radars and the test bed infrastructure;
    and develop the sea- based X- band radar.

    MDA has incurred a greater risk of cost growth because for more than a
    year the agency was not able to rely fully on the data from its primary
    tool for monitoring whether the GMD contractor was performing work within
    cost and on schedule* the prime contractor*s Earned Value Management

    4 The kill vehicle is the weapon component of the GMD element that
    attempts to detect and destroy threat warheads through *hit- to- kill*
    impacts. 5 The battle management component is the integrating and
    controlling component of the

    GMD element. The fire control software plans engagements and tasks GMD
    components to execute a missile defense mission. Results in Brief

    Page 4 GAO- 03- 600 Missile Defense

    (EVM) system. 6 In February 2002, MDA modified GMD*s contract to bring it
    into line with the agency*s new capabilities- based acquisition strategy.
    It took several months to establish an interim cost baseline 7 against
    which to

    measure the contractor*s performance and 13 months to complete revisions
    to the baseline. Also, MDA and the contractor did not complete a review
    until July 2003 to ensure that the revised baseline was accurate and that
    contractor personnel were correctly using it to measure performance. This
    review was of particular importance because an earlier review revealed
    significant deficiencies in the contractor*s development and use of the
    initial contract baseline. Until this review was completed, MDA did not
    know for sure whether it could rely fully on the data from its EVM system
    to recognize and correct potential problems in time to prevent significant
    cost increases and schedule delays.

    We are making recommendations that MDA (1) consider adding a test of the
    effectiveness of the radar in Alaska; and (2) ensure that procedures are
    in place that will increase MDA*s confidence in data from its EVM system.
    DOD concurred with our first recommendation and partially concurred with
    the second. In commenting on the draft report, DOD stated that the
    feasibility of these procedures will be determined and that a portion of
    the work is already being accomplished.

    The concept of using a missile to destroy another missile (hit- to- kill)
    has been explored since the mid- 1950*s, but it was not until 1984 that
    the first such intercept achieved its objective. Between the mid- 1980*s
    and late- 1990*s the United States conducted a number of experiments
    designed to demonstrate that it was possible to hit one missile with
    another. In 1997, the Ballistic Missile Defense Organization (BMDO)
    established the National Missile Defense (NMD) Joint Program Office. The
    program office was directed to demonstrate by 1999 a system that could
    protect the United States from attacks of intercontinental ballistic
    missiles and to be in a position to deploy the system if the threat
    warranted by 2003. The

    6 The EVM system is a management tool widely used by DOD to compare the
    value of contractor*s work performed to the work*s actual cost. The tool
    measures the contractor*s actual progress against its expected progress
    and enables the government and contractor to estimate the program*s
    remaining cost. 7 An interim baseline is often established by the
    contractor when the government has authorized work, but the requirements
    and terms of the work have not yet been negotiated. Until negotiations are
    completed, the contractor develops a baseline using proposed cost that has
    been divided among work packages with associated budgets and schedule.
    Background

    Page 5 GAO- 03- 600 Missile Defense

    initial system consisted of space- and ground- based sensors, early
    warning radars, interceptors, and battle management functions.

    The program underwent additional changes as the new decade began. In
    September 2000, the President decided to defer deployment of the NMD
    system, but development of the system continued with the goal of being
    ready to deploy the system when directed. This action was followed in 2001
    by BMDO*s redirection of the prime contractor*s efforts from developing
    and deploying an NMD system to developing an integrated test bed with the
    newly designated GMD system as its centerpiece. The Secretary of Defense,
    in January 2002, renamed BMDO as MDA and consolidated all ballistic
    missile defense programs under the new agency. Former missile defense
    acquisition programs became elements of a single

    ballistic missile defense system. These changes were followed in December
    2002, by the President*s directive to begin fielding in 2004 a ballistic
    missile defense system, which included components of the GMD element
    already under development. The GMD element is intended to protect the
    United States against longrange

    ballistic missiles in the midcourse phase of their flight. This is the
    point outside the atmosphere where the motors that boost an enemy missile
    into space have stopped burning and the deployed warhead follows a
    predictable path toward its target. Compared to the boost and terminal
    phases, this stage of flight offers the largest window of opportunity for
    interception and allows the GMD element a longer time to track and engage
    a target.

    As illustrated in figure 1, GMD will rely on a broad array of components
    to track and intercept missiles. Figure 2 provides a notional concept of
    how these components will operate once they are fully integrated into the
    GMD element.

    Page 6 GAO- 03- 600 Missile Defense

    Figure 1: Components of GMD

    Page 7 GAO- 03- 600 Missile Defense

    Figure 2: Notional GMD Concept of Operations

    Note: The concept of operations assumes weapons release authority has been
    previously granted by the President of the United States or the Secretary
    of Defense. Missile flight times may be too brief to ask for permission to
    launch interceptors and engage the enemy.

    Page 8 GAO- 03- 600 Missile Defense

    MDA is gaining the knowledge it needs to have confidence that technologies
    critical to the GMD Block 2004 capability will work as intended. Two of
    the ten technologies essential to the Block 2004 capability have already
    been incorporated into actual prototype hardware and have been
    demonstrated to function as expected in an operational environment. 8
    Other technologies are reaching this level of maturity. If development and
    testing proceed as planned, MDA will demonstrate the maturity of five
    additional technologies by the second quarter of fiscal year 2004 and two
    critical radar technologies during fiscal year 2005. MDA believes that its
    best opportunity to demonstrate the maturity of the tenth technology,
    technology critical to GMD*s primary radar, may come through the
    anticipated flight tests of foreign missiles.

    Our work over the years has found that making a decision to begin system
    integration of a capability before the maturity of all critical
    technologies have been demonstrated increases the program*s cost,
    schedule, and performance risks. Because the President directed DOD to
    begin fielding a ballistic missile defense system in 2004, MDA began GMD
    system integration with technologies whose maturity has not been
    demonstrated. As a result, there is a greater likelihood that critical
    technologies will not work as intended in planned flight tests. If this
    occurs, MDA may have to spend additional funds in an attempt to identify
    and correct problems by

    September 2004 or accept a less capable system. 9 Successful developers
    follow *knowledge- based acquisition* practices to get quality products to
    the customer as quickly and cost effectively as possible. As a part of
    meeting this goal, developers focus their technology programs on maturing
    technologies that have the realistic potential for being incorporated into
    the product under consideration. Accordingly, successful developers spend
    time to mature technology in a technology setting, where costs are
    typically not as great, and they do not move forward with product
    development* the initiation of a program to fully

    8 An operational environment is a real- world environment (e. g., flight
    demonstration) that addresses all of the operational requirements and
    specifications demanded of the final product. 9 U. S. General Accounting
    Office, Missile Defense: Knowledge- Based Practices Being

    Adopted, but Risks Remain, GAO- 03- 441 (Washington, D. C.: Apr. 30,
    2003). This report presents our analysis of MDA*s new approach for
    developing missile defense technology. MDA Expects to

    Demonstrate the Maturity of Most GMD Technologies before September 2004

    Importance of Maturing Technology

    Page 9 GAO- 03- 600 Missile Defense

    design, integrate, and demonstrate a product for production* until
    essential technologies are sufficiently mature.

    An analytical tool* which has been used by DOD and the National
    Aeronautics and Space Administration, called technology readiness levels
    (TRLs), 10 *can assess the maturity level of technology as well as the
    risk

    that technology poses if it is included in a product*s development. The
    nine readiness levels are associated with progressing levels of
    technological maturity and demonstrated performance relative to a
    particular

    application* starting with paper studies of applied scientific principles
    (TRL 1) and ending with a technology that has been *flight proven* on an
    actual system through successful mission operations (TRL 9). Additional

    details on TRLs are shown in appendix III. TRLs provide a gauge of how
    much knowledge the program office has on the progress or status of a
    particular technology and are based on two principal factors: (1) the
    fidelity of demonstration hardware, including design maturity and level of
    functionality achieved; and (2) the extent and realism of the environment
    in which the technology has been demonstrated.

    MDA recognizes the value of beginning system integration with mature
    technology and of using TRLs to assess the maturity of technology proposed
    for a block configuration. In particular, MDA prefers to include new
    technology in a block configuration only if the technology has reached a
    TRL 7; that is, only if prototype hardware with the desired form, fit, and
    function has been proved in an operational environment. However; MDA
    retains the flexibility to include less mature technology in a block
    configuration if that technology offers a significant benefit in
    performance

    and the risk of retaining it is acceptable and properly managed. Through
    technical discussions with the GMD joint program office and its prime
    contractor, we identified ten critical GMD technologies and jointly
    assessed the readiness level of each. The critical technologies are
    resident in the exoatmospheric kill vehicle, the boosters, the battle
    management, command, and control component, and in the element*s

    10 U. S. General Accounting Office, Best Practices: Better Management of
    Technology Development Can Improve Weapon System Outcomes, GAO/ NSIAD- 99-
    162 (Washington, D. C.: July 1999). Readiness Levels of GMD

    Element Technologies

    Page 10 GAO- 03- 600 Missile Defense

    radars. In 7 of 10 cases, we agreed with the program office and the GMD
    prime contractor on the maturity level of the element*s critical
    technologies. The differences in the remaining three cases, as discussed
    in detail below, were primarily due to interpretation of TRL definitions.
    The program office and its contractor rated the two booster technologies
    and one radar technology at higher readiness levels than, in our opinion,
    MDA had demonstrated.

    Most critical GMD technologies are currently at TRLs 5 and 6. At TRL 5,
    the technology*s development is nearing completion, but it has not been
    applied or fitted for the intended product. At this point, the technology
    has been incorporated into a high- fidelity breadboard 11 that has been
    tested in a laboratory or relevant environment 12 . Although this
    demonstrates the functionality of the technology to some extent, the
    hardware is not necessarily of the form and fit (configuration) that would
    be integrated into the final product. A new application of existing
    technology is usually assessed at a TRL 5, because the technology has not
    been demonstrated in the relevant environment for the new application. TRL
    6 begins the true *fitting* or application of the technology to the
    intended product. To reach this level, technology must be a part of a
    representative prototype that is very close to the form, fit, and function
    of that needed for the intended

    product. Reaching a TRL 6 requires a major step in a technology*s
    demonstrated readiness, that is, the prototype must be tested in a
    highfidelity laboratory environment or demonstrated in a restricted but
    relevant environment.

    Two of the ten GMD technologies were assessed at a TRL 7, the level that
    successful developers insist upon before initiating product development.
    To reach this level, a pre- production prototype of the technology must be
    demonstrated to its expected functionality in an operational environment.
    If development and testing proceed as planned by MDA, we judge that most
    of the technologies (7 of 10) will be at a TRL 7 after the completion

    11 A breadboard is a collection of integrated components that provide a
    representation of a system/ subsystem that can be used to determine
    concept feasibility and to develop technical data. A breadboard is
    typically configured for laboratory use to demonstrate the technical
    principals of immediate interest. 12 A relevant environment is defined as
    a testing environment that simulates key aspects of the operational
    environment.

    Page 11 GAO- 03- 600 Missile Defense

    of integrated flight test (IFT)- 14, 13 which is scheduled for the second
    quarter of fiscal year 2004. Table 1 summarizes our assessment of the TRL
    for each critical technology as of June 2003 and the date at which MDA
    anticipates each technology will reach TRL 7. A detailed discussion of
    each critical technology follows.

    Table 1: Technology Readiness Levels of GMD Critical Technologies Critical
    technology TRL (as of June 2003) Anticipated event/ date for achieving TRL
    7 a Exoatmospheric kill vehicle Infrared seeker 7 Achieved

    On- board discrimination 6 IFT- 14 (2nd quarter FY04) Guidance,
    navigation, and control subsystem 6 IFT- 14 (2nd quarter FY04) Boosters
    BV+ 6 IFT- 13A (1st quarter FY04)

    OSC Lite 6 IFT- 13B (1st quarter FY04) Battle management command, control,
    and communications Fire control software 7 Achieved

    In- flight interceptor communications system 6 IFT- 14 (2nd quarter FY04)
    Radars Cobra Dane radar 5 Unknown

    Beale upgraded early warning radar 5 Radar certification flight (1st
    quarter FY05) Sea- based X- band radar 5 IFT- 18 (4th quarter FY05)
    Source: GAO analysis of GMD data.

    Note: Information provided in the table* the configuration of flight test
    events and associated date* is as of June 2003 and is subject to change. a
    Assumes technology development and demonstrations will have been
    successful.

    13 Integrated flight tests of the GMD element are real- world
    demonstrations of system performance during which an interceptor is
    launched to engage and intercept a mock warhead above the atmosphere.

    Page 12 GAO- 03- 600 Missile Defense

    The exoatmospheric kill vehicle is the weapon component of the GMD
    interceptor that attempts to detect and destroy the threat reentry vehicle
    through a hit- to- kill impact. The prime contractor identified three
    critical

    technologies pertaining to the operation of the exoatmospheric kill
    vehicle. They include the following:

     Infrared seeker, which is the *eyes* of the kill vehicle. The seeker is
    designed to support kill vehicle functions like tracking and target
    discrimination. The primary subcomponents of the seeker are the infrared
    sensors, a telescope, and the cryostat that cools down the sensors.  On-
    board discrimination, which is needed to identify the true warhead

    from among decoys and associated objects. Discrimination is a critical
    function of the hit- to- kill mission that requires the successful
    execution of a sequence of functions, including target detection, target
    tracking, and the estimation of object features. As such, successful
    operation of the infrared seeker is a prerequisite for discrimination.
    Guidance, navigation, and control subsystem, which is a combination

    of hardware and software that enables the kill vehicle to track its
    position and velocity in space and to physically steer itself into the
    designated target.

    All three kill vehicle technologies have been demonstrated to some extent
    in actual integrated flight tests on near- production- representative kill
    vehicles. The infrared seeker has reached a TRL 7, because a configuration
    very much like that to be fielded has been demonstrated in previous
    integrated flight tests, and only minor design upgrades are planned to
    reach the Block 2004 configuration. The remaining two kill vehicle
    technologies are at a TRL 6, because their functionality is being upgraded
    and the technologies have yet to be incorporated into the kill vehicle and

    demonstrated in an operational environment. The on- board discrimination
    technology has not yet reached TRL 7 because MDA has not tested a
    *knowledge database* that is expected to increase the kill vehicle*s
    discrimination capability. The purpose of the database is to enable the
    kill vehicle to distinguish characteristics of threatening from non
    threatening objects. MDA expects to test the database for the first time
    in IFT- 14.

    As a software- intensive technology, on- board discrimination performance
    under all flight conditions can only be evaluated through ground testing,
    but flight- testing is needed to validate the software*s operation in a
    real world environment. The discrimination capability that will be tested
    in

    IFT- 14 is expected to be fielded as part of the Block 2004 capability.
    Exoatmospheric Kill

    Vehicle Technologies

    Page 13 GAO- 03- 600 Missile Defense

    Therefore, IFT- 14 should demonstrate the technology*s maturity if the
    test shows that the kill vehicle achieves its discrimination objective. 14
    Similarly, the guidance, navigation, and control technology will also
    increase to a TRL 7 if the technology achieves its objectives in IFT- 14.
    The inertial measurement unit, an important component of the guidance,
    navigation, and control subsystem that enables the kill vehicle to track
    its position and velocity, has not yet been tested in the severe
    environments (e. g., vibrations and accelerations) induced by the
    operational booster. This will be first attempted when one of the new
    operational boosters is used in IFT- 14. In addition to testing the
    inertial measurement unit, IFT- 14 will also test the upgraded divert
    hardware (used to actively steer the kill vehicle to its target) that is
    expected to be part of the Block 2004 configuration. The integrated
    booster stack is the part of the GMD interceptor that is

    composed of rocket motors needed to deliver and deploy the kill vehicle
    into a desired intercept trajectory. For all flight tests to date, a two-
    stage surrogate booster called the payload launch vehicle has been used.

    In July 1998, the GMD prime contractor began developing a new three- stage
    booster for the GMD program, known as the *Boost Vehicle*, from commercial
    off- the- shelf components. However, the contractor

    encountered difficulty. By the time the booster was flight tested in
    August 2001, it was already about 18 months behind schedule. The first
    booster flight test met its objectives, but the second booster tested
    drifted off course and had to be destroyed 30 seconds after launch.

    Subsequently, MDA altered its strategy for acquiring a new booster for the
    interceptor. Instead of relying on a single contractor, MDA authorized the
    GMD prime contractor to develop a second source for the booster by
    awarding a subcontract to another contractor. If development of the
    boosters proceeds as planned, both boosters will be part of the Block 2004
    capability. One booster is known as BV+ and the other as *OSC Lite.*

    The prime contractor ultimately transferred development of the boost
    vehicle to a subcontractor who is currently developing a variant* known as
    *BV+** for the GMD element. The program office and GMD

    14 See classified annex for further details. Booster Technologies

    The BV+ Booster

    Page 14 GAO- 03- 600 Missile Defense

    contractor rated the BV+ at a TRL 7. The prime contractor reasoned that
    the extent of the legacy program and its one successful flight test should
    allow for this rating. However, given the limited testing to date, we
    assessed the BV+ booster currently at a TRL 6; that is, the technology has
    been demonstrated in a restricted flight environment using hardware close
    in form, fit, and function to that which will be fielded in 2004. We
    believe the contractor*s assessment is too high at this time, because the
    step from TRL 6 to TRL 7 is significant in terms of the fidelity of the
    demonstration environment. However, the first test of a full configuration
    BV+ booster

    will occur with IFT- 13A, which is scheduled for the first quarter of
    fiscal year 2004. In our opinion, the BV+ booster will reach TRL 7 at this
    time if the booster works as planned.

    The second booster under development is referred to as *OSC Lite*. This
    booster, which is essentially the Taurus Lite missile that carries
    satellites into low- earth orbit, will be reconfigured for the GMD
    element. Despite the fact that the booster was recently tested under
    restricted flight conditions, GMD*s prime contractor believes that the
    legacy development of the Taurus Lite missile is sufficient to prove that
    the OSC Lite has reached TRL 7. However, in our opinion, because the test
    was conducted with hardware configured as it was in the Taurus missile,
    not as it will be configured for GMD*s Block 2004, the booster*s maturity
    level is comparable to that of the BV+. The first flight test of a full
    configuration OSC Lite booster is scheduled for IFT- 13B in the first
    quarter of fiscal year 2004. We believe that if the booster performs as
    intended in this test, it will reach TRL 7. The battle management
    component is the integrating and controlling

    component of the GMD element. Prime contractor officials identified and
    assessed the following sub- components as critical technologies:

     GMD fire control software, which analyzes the threat, plans engagements,
    and tasks components of the GMD element to execute a mission.  In- flight
    interceptor communications system, which enables the GMD

    fire control component to communicate with the exoatmospheric kill vehicle
    while in flight.

    The two battle management technologies have been demonstrated to some
    extent in actual integrated flight tests, and both are near their Block
    2004 design. We determined that the GMD fire control software has
    currently achieved a TRL 7 and the in- flight interceptor communications
    system has reached a TRL 6. Prime contractor officials concur with our
    assessment. The *OSC Lite* Booster

    Battle Management Command, Control, and Communications Technologies

    Page 15 GAO- 03- 600 Missile Defense

    The fire control software is nearing expected functionality and prior
    software builds have been demonstrated in GMD flight tests. Only minor
    design changes will be made to address interfacing issues (linking the
    fire control component with other GMD components) before the software
    reaches the operational configuration of Block 2004. As a software-
    intensive technology, the performance of the fire control software
    throughout the entire *flight envelope* can only be evaluated through
    ground testing. Ground testing is well underway at both the Joint National
    Integration Center at Schriever Air Force Base, Colorado, and at

    the prime contractor*s integration laboratory in Huntsville, Alabama. The
    second technology associated with the battle management component is the
    in- flight interceptor communications system. Even though the pointing
    accuracy and communications capability of this technology were
    demonstrated in previous flight tests, the operational hardware to be
    fielded by 2004 is expected to operate at a different uplink frequency
    than the legacy hardware used in these past flight tests. 15 Accordingly,
    we

    assessed the in- flight interceptor communications system at a TRL 6. The
    first integrated flight test to include an operational- like build of this
    technology is IFT- 14, and if the technology meets its objectives in this
    flight test, TRL 7 would be achieved. The GMD contractor initially
    identified the sea- based X- band radar as

    the only radar- related critical technology. Since its initial assessment
    in September 2002, the contractor has now agreed with us that the Beale
    upgraded early warning radar and the Cobra Dane radar are also critical
    technologies of the GMD element. The contractor and the GMD program office
    assessed the Beale and Cobra Dane radars at a TRL 5, because the
    technology, especially mission software, is still under development and
    has not yet been demonstrated in a relevant flight environment. 16 The
    contractor assessed the sea- based X- band radar at a TRL 6. As discussed
    below, we agree with their assessment of the Beale and Cobra Dane radars
    but rated the sea- based X- band radar as a TRL 5. 15 See classified annex
    for further details.

    16 The hardware of the Beale and Cobra Dane radars is mature since both
    are currently in operation for other missions, namely, integrated tactical
    warning and technical intelligence, respectively. Adding the ballistic
    missile defense mission to these radars requires primarily software-
    related development and testing. Radar Technologies

    Page 16 GAO- 03- 600 Missile Defense

    The early warning radar at Beale Air Force Base has participated in
    integrated flight tests in a missile- defense role using legacy hardware
    and developmental software. Design and development of operational builds
    of the software are progressing, but such builds have only been tested in
    a simulated environment. Therefore, we assessed the Beale radar

    technology at a TRL 5* an assessment driven by software considerations.
    The conversion of the early warning radar at Beale to an upgraded early
    warning radar, which consists of minor hardware and significant software
    upgrades, is planned for completion sometime during the middle of fiscal
    year 2004. After this time, the Beale radar can take part in flight-
    testing in its upgraded configuration. MDA currently plans to demonstrate
    the upgraded Beale technology in a non intercept flight test, known as a
    radar certification flight, 17 in the first quarter of fiscal year 2005.
    The Beale radar will be demonstrated at a TRL 7 if the objectives of this
    flight test

    are achieved. The Cobra Dane radar is currently being used in a
    surveillance mode to collect data on selected intercontinental ballistic
    missile test launches out of Russia and does not require real- time data
    processing and communications capabilities. To achieve a defensive
    capability by September 2004, the Cobra Dane radar is being upgraded to
    perform both of these tasks. This upgrade, which requires a number of
    software

    modifications, is designed to enable Cobra Dane to detect and track enemy
    targets much as the Beale upgraded early warning radar does. Although the
    hardware component of the Cobra Dane radar is mature and will undergo only
    minor updating, Cobra Dane*s mission software is being revised for this
    application. The revision includes reuse of existing software and
    development of new software so that the Cobra Dane radar can be integrated
    into the GMD architecture.

    Upgrades to the Cobra Dane radar are due to be completed at the beginning
    of 2004. After the software is developed and ground tested, the radar can
    reach a TRL 6, but it is uncertain when the radar will reach a TRL 7.
    Because of other funding and scheduling priorities, MDA has no plans
    through fiscal year 2007 for using this radar in integrated flight tests;
    such tests would require air- or sea- launched targets that are not
    currently part of the test program. Unless the current test program is
    modified, the only opportunities for demonstrating Cobra Dane in an
    operational environment would come from flight tests of foreign missiles.
    MDA

    17 Ground testing of interim software builds to be mounted on the Beale
    radar is ongoing.

    Page 17 GAO- 03- 600 Missile Defense

    officials anticipate that such opportunities will occur. However, it is
    not clear that testing Cobra Dane in this manner will provide all of the
    information that a dedicated test provides because MDA will not control
    the configuration of the target or the flight environment.

    The sea- based X- band radar is being built as part of the Block 2004
    capability and scheduled for completion in 2005. It will be built from
    demonstrated technologies* a sea- based platform and the prototype X- band
    radar currently being used in the GMD test program. Prime contractor
    officials told us that they consider the risk associated with the
    construction and checkout of the radar as primarily a programmatic, rather
    than technical risk, and believe that the sea- based X- band radar has
    reached a TRL 6. The contractor also stated that the initial operational
    build of the radar software is developed and currently being tested at the
    contractor*s integration laboratory. We assessed the sea- based X- band
    radar as a TRL 5 because the radar has not yet been built and because
    constructing a radar from an existing design and placing it on a sea-
    based platform is a new application of existing technology. For example,
    severe wind and sea conditions may affect the radar*s functionality*
    conditions that cannot be replicated in a laboratory. As a result,
    developers cannot be sure that the sea- based X- band radar will work as
    intended until it is

    demonstrated in this new environment. However, both we and the contractor
    agree that the maturity level of the sea- based X- band radar will
    increase to a TRL 7 if it achieves its test objectives in IFT- 18
    (scheduled for the fourth quarter of fiscal year 2005).

    Page 18 GAO- 03- 600 Missile Defense

    From the program*s inception in 1997 18 through 2009, MDA expects to spend
    about $21.8 billion to develop the GMD element. About $7.8 billion of the
    estimated cost will be needed between 2002 and 2005 to develop and field
    the Block 2004 GMD capability and to develop the GMD portion

    of the test bed. 19 However, MDA has incurred a greater risk of cost
    increases because for more than a year MDA was not sure that it could rely
    fully upon data from the prime contractor*s Earned Value Management (EVM)
    system, 20 which provides program managers and others with early warning
    of problems that could cause cost and schedule growth.

    Before the restructuring of the GMD program in 2002, about $6.2 billion
    was spent (between 1997 and 2001) to develop a ground- based defense
    capability. MDA estimates it will need an additional $7.8 billion between
    2002 and 2005 to, among other tasks, install interceptors at Fort Greely,
    Alaska, and at Vandenberg Air Force Base, California; upgrade existing
    radars and test bed infrastructure; and develop the sea- based X- band
    radar that will be added in the fourth quarter of fiscal year 2005. In
    addition, MDA will invest an additional $7.8 billion between fiscal year
    2004 and 2009 to continue efforts begun under Block 2004, such as
    enhancing capability and expanding the test bed. Table 2, below, provides
    details on the funding requirements by block and by fiscal year, and
    figure 3 provides examples of specific Block 2004 tasks.

    18 We calculated program cost from 1997 forward because the National
    Missile Defense program was established at that time. 19 The cost to
    develop and field the initial GMD capability and the ballistic missile
    defense test bed is funded in MDA*s budget within the Defense Wide
    Research, Development, Test and Evaluation appropriation. MDA is not
    requesting any procurement, military

    construction, or military personnel funds for this effort. 20 The EVM
    system is a management tool widely used by DOD to compare the value of
    contractor*s work performed to the work*s actual cost. The tool measures
    the contractor*s actual progress against its expected progress and enables
    the government and contractor to estimate the program*s remaining cost.
    MDA Has Risked

    Cost Growth Because It Could Not Fully Rely on Data from Its System for
    Monitoring Contractor Performance

    GMD Development Costs

    Page 19 GAO- 03- 600 Missile Defense

    Table 2: Estimated Cost to Develop and Field GMD

    Then- year dollars in billions Fiscal years 1997- 2001 2002 2003 2004 2005
    2006 2007 2008 2009 Subtotal

    Sunk Cost 6.2 6.2 GMD Initial Capability and Block 2004 Test Bed 3.1 2. 6
    1.2 .9 7.8

    GMD Block 2006 1.6 1.8 1.4 1. 2 6.0 GMD Block 2008 .9 .9 1.8

    Total 6.2 3.1 2. 6 2.8 2.7 1.4 1. 2 .9 .9 21.8

    Source: Ballistic Missile Defense Budget, Midcourse Defense Segment,
    February 2003. Figure 3: Tasks GMD Plans to Accomplish for the GMD Block
    2004 Project

    Page 20 GAO- 03- 600 Missile Defense

    MDA did not include the following costs is its Block 2004 estimate:  The
    cost to recruit, hire, and train military personnel to operate the initial

    defensive capability and provide site security at various locations, which
    MDA estimates to be an additional $13.4 million (half in fiscal year 2003
    and half in 2004 each), will be needed to operate GMD and provide physical
    security. Additional costs to cover these personnel throughout the life of
    the program beginning in 2005 and beyond were also omitted.  The cost to
    maintain equipment and facilities was not included.  Systems engineering
    and national team costs* which benefit all elements, including GMD and
    cannot be divided among the elements* were not

    included in MDA*s budget. Because a significant portion of MDA*s Block
    2004 GMD cost estimate is the cost of work being performed by the
    element*s prime contractor, MDA*s ability to closely monitor its
    contractor*s performance is critical to controlling costs. The tool that
    MDA, and many DOD entities, have chosen for this purpose is the EVM
    system. This system uses contractor reported

    data to provide program managers and others with timely information on a
    contractor*s ability to perform work within estimated cost and schedule.
    It does so by examining variances reported in contractor cost performance

    reports between the actual cost and time of performing work tasks and the
    budgeted or estimated cost and time. While this tool can provide
    insightful information to managers, MDA*s use of it has been hampered by
    several

    factors. Principally, although major contract modifications were made in
    February 2002, it took until July 2003 for MDA to complete a review to
    confirm the reliability of data from the EVM system. An earlier review of
    a similar nature revealed significant deficiencies in the contractor*s
    formulation and collection of EVM data. Until a new review was completed,
    MDA could not be sure about its ability to rely fully upon this data to
    identify potential problems in time to prevent significant cost

    growth and schedule delays. An accurate, valid, and current performance
    management baseline is needed to perform useful analyses using EVM. The
    baseline identifies and defines work tasks, designates and assigns
    organizational responsibility for each task, schedules the work task in
    accordance with established targets, and allocates budget to the scheduled
    work. According to DOD MDA*s Insight into

    Potential Cost Growth Was Limited by the Agency*s Inability to Rely Fully
    on Data from Earned Value Management System

    Baseline Revised over 13- Month Period

    Page 21 GAO- 03- 600 Missile Defense

    guidance, 21 a performance management baseline should be in place as early
    as possible after the contractor is authorized to proceed. Although the
    guidance does not define how quickly the contractor should establish a
    baseline, experts generally agree that it should be in place, on average,
    within 3 months after a contract is awarded or modified.

    About a year before the Secretary of Defense directed MDA to adopt an
    evolutionary acquisition strategy, the agency awarded a new contract for
    the development of a National Missile Defense system. In February 2002,
    MDA modified this contract to redirect the contractor*s efforts. Instead
    of developing a missile defense system that met all of the requirements of
    the war fighter, as the initial contract required, the modification
    directed the contractor to develop the first GMD increment, or block,
    which was to be a ballistic missile test bed with GMD as its centerpiece.

    Following the contract*s modification, the contractor in June 2002
    established an interim baseline. This baseline was developed by adding
    budgets for near- term new work to the original baseline. Because the cost
    of the work being added to the baseline had not yet been negotiated, the

    contractor based the budgets on the cost proposed to MDA, as directed by
    DOD guidelines. The contractor implemented the baseline almost within the
    3- month time frame recommended by experts. In the time between the
    modification and the development of the interim baseline, MDA authorized
    the contractor to begin work and spend a specified amount of money, and
    MDA paid the contractor about $390 million during this period.

    An option that MDA could have used to help validate the interim baseline
    was to have the Defense Contract Management Agency (DCMA) 22 verify
    contractor work packages and track the movement of funds between the
    unpriced work account and the baseline. However, neither MDA nor DCMA
    initiated these actions. In its technical comments on a draft of this
    report, DOD pointed out that during the negotiation process, MDA reviews

    prime and subcontractor proposal data that include engineering labor
    hours, material, and cost estimates. DOD further noted that these
    estimates eventually form a basis for the work packages that make up the
    data for the performance management baseline. We agree that these costs

    21 Department of Defense, Earned Value Management Implementation Guide
    (Washington, D. C.: Dec. 1996, as revised, p. 10). 22 DCMA is the agency
    that DOD has given responsibility for validating contractors* Earned Value
    data.

    Page 22 GAO- 03- 600 Missile Defense

    will eventually be associated with the work packages that make up the
    baseline. However, a joint contractor and MDA review of the initial GMD
    baseline concluded that even though these costs were otherwise fair and
    reasonable, some work packages that the contractor developed for the
    original contract*s baseline did not correctly reflect the work directed
    by MDA. An independent review of work packages included in the interim

    baseline would have increased the likelihood that the work packages were
    being properly developed and that their budget and schedule were
    appropriate. The contractor completed all revisions to the baseline for
    the prime

    contractor and all five subcontractors by March 2003, 3 months after
    negotiating the cost of the modification and 13 months after authorizing
    the work to begin. The contracting officer explained that it took until
    December 2002 to negotiate the 2002 contract change because the additional
    work was extremely complex, and, as a result, the modification needed to
    be vetted through many subcontractors that support the prime.

    The DOD guidance states that an integrated baseline review (IBR) is to be
    conducted within 6 months of award of a new contract or major change to an
    existing contract. 23 The review verifies the technical content of the
    baseline. It also ensures that contractor personnel understand and have
    been adequately trained to collect EVM data. The review also verifies the
    accuracy of the related budget and schedules, ensures that risks have been
    properly identified, assesses the contractor*s ability to implement
    properly

    EVM, and determines if the work identified by the contractor meets the
    program*s objectives. The government*s program manager and technical staff
    carry out this review with their contractor counterparts.

    Completing an IBR of the new baseline has been of particular importance
    because the July 2001 IBR for the initial contract identified more than
    300 deficiencies in the contractor*s formulation and execution of the
    baseline. For example, the contractor had not defined a critical path for
    the overall effort, many tasks did not have sufficient milestones that
    would allow the contractor to objectively measure performance, and
    contractor personnel who were responsible for reporting earned value were
    making mistakes in measuring actual performance against the baseline.

    23 Earned Value Management Implementation Guide, pp. 34 and 36. Baseline
    Review Completed in

    July 2003

    Page 23 GAO- 03- 600 Missile Defense

    MDA began a review in March 2003 of the contractor*s new baseline, which
    reflected the contract modification,. Completing this IBR took until July
    2003 because of the complexity of the program and the many subcontractors
    that were involved. Although the review team found fewer problems with the
    contractor*s formulation and execution of the new baseline, problems were
    identified. For example, the IBR showed that in some cases the baseline
    did not reflect the new statement of work. Also,

    both the prime contractor and subcontractors improperly allocated budget
    to activities that indirectly affect a work product (known as level of
    effort activities) when they could have associated these activities with a
    discrete end product. Because of the way these activities are accounted
    for, this designation could mask true cost variances.

    Before the IBR was underway, DCMA recognized another problem with the
    contractor*s EVM reports. In its December 2002 cost performance report,
    the contractor reported that it expected no cost overrun at contract
    completion. This implied that the program was not experiencing any
    problems that could result in significant cost or schedule growth.

    However, DCMA stated that October 2002 was the second month in a row that
    the contractor had used management reserve funds to offset a significant
    negative cost variance. 24 DCMA emphasized that this is not the intended
    purpose of management reserves. (Management reserves are a part of the
    total project budget intended to be used to fund work

    anticipated but not currently defined.) DCMA officials told us that while
    this is not a prohibited practice most programs wait until their work is
    almost completed, that is 80 to 90 percent complete, before making a
    judgment that the management reserve would not be needed for additional
    undefined work and could be applied to unfavorable contract cost
    variances.

    24 Defense Contract Management Agency, Ground- Based Midcourse Defense
    Monthly Assessment Report Contract No. HQ0006- 01- C- 0001 for Missile
    Defense Agency (Seal Beach, Calif.: Dec. 2002, p. 10). DCMA reported that
    cost performance reports were giving ** a misleading feeling that
    everything in the program is OK. For the 2nd month in a row, [the prime
    contractor] has covered up a significant Variance- at- Completion (-$
    107,800K) * by taking money out of Management Reserve (MR). This is not
    the intended purpose of

    using MR funds. [The prime contractor] is reporting a $0 Variance- At-
    Completion [VAC] by subtracting $107,800K from MR to reduce VAC to $0.
    Based on prior performance to date, this could be an indication of a trend
    for growth of the EAC [estimate- at- completion].* Management Reserve

    Used to Offset Expected Cost Overruns at Contract Completion

    Page 24 GAO- 03- 600 Missile Defense

    Because of the President*s direction to begin fielding a ballistic missile
    defense system in 2004, the MDA took a higher risk approach by beginning
    GMD system integration before knowing whether its critical technologies

    were mature. If development and testing progress as planned, however, MDA
    expects to have demonstrated the maturity of 7 of the 10 critical GMD
    technologies before the element is initially fielded in September 2004 and
    2 others during fiscal year 2005. If technologies do not achieve their
    objectives during testing, MDA may have to spend additional funds in an
    attempt to identify and correct problems by September 2004 or accept a
    less capable system.

    Because of other funding and scheduling priorities, MDA does not plan to
    demonstrate through integrated flight tests whether the Cobra Dane radar*s
    software can process and communicate data on the location of enemy
    missiles in *real time.* Although tests using sea- or air- launched
    targets before September 2004 would provide otherwise unavailable

    information on the software*s performance, we recognize those tests would
    be costly and funds have not been allocated for that purpose. We also
    recognize that the most cost efficient means of testing the Cobra Dane
    radar is through launches involving foreign test missiles. However, we
    believe it would be useful for MDA to consider whether the increased
    confidence provided by a planned test event outweighs other uses for those
    funds.

    MDA is investing a significant amount of money to achieve an operational
    capability during the first block of GMD*s development, and the agency
    expects to continue investing in the element*s improvement over the next
    several years. Because MDA is also developing other elements and must
    balance its investment in each, it needs an accurate GMD cost estimate. If
    it is used as intended, the EVM system can be an effective means of
    monitoring one of GMD*s largest costs, the cost of having a contractor
    develop the GMD system. It is understandable that the dynamic changes in
    MDA*s acquisition strategy led to major contract modifications, which

    made it more difficult for the contractor to establish a stable baseline.
    However, in this environment, it is even more important that MDA find ways
    to ensure the integrity of the interim baselines and to quickly determine
    that revised baselines can be fully relied on to identify potential
    problems before they significantly affect the program*s cost. Conclusions

    Page 25 GAO- 03- 600 Missile Defense

    To increase its confidence that the Ground- based Midcourse Defense
    element fielded in 2004 will operate as intended, we recommend that the
    Secretary of Defense direct the Director, Missile Defense Agency, to
    explore its options for demonstrating the upgraded Cobra Dane radar in its
    new ballistic missile defense role in a real- world environment before

    September 2004. To improve MDA*s oversight of the GMD element and to
    provide the Congress with the best available information for overseeing
    the program, we recommend that the Secretary of Defense direct the
    Director, Missile Defense Agency, to:

     ensure that when a contractor is authorized to begin new work before a
    price is negotiated that DCMA validate the performance measurement
    baseline to the extent possible by (1) tracking the movement of budget
    from the authorized, unpriced work account into the baseline, (2) verify
    that the work packages accurately reflect the new work directed, and (3)
    report the results of this effort to MDA; and  strive to initiate and
    complete an integrated baseline review (IBR) of any

    major contract modifications within 6 months. DOD*s comments on our draft
    report are reprinted in appendix II. DOD concurred with our first
    recommendation. DOD stated that MDA is exploring its options for
    demonstrating, prior to 2004, the upgraded Cobra Dane radar in a real-
    world environment. However, DOD noted that because it takes considerable
    time to develop and produce targets and to

    conduct safety and environmental assessments, completing a Cobra Dane
    radar test before September 2004 would be very challenging. DOD concluded
    that *targets of opportunity* (flight tests of foreign missiles) and
    ground testing may provide the best means to demonstrate the radar*s
    maturity in the near term.

    DOD partially concurred with our second recommendation. In responding to
    the first part of recommendation two, DOD stated that MDA and the DCMA
    will jointly determine the feasibility of tracking the budget for
    authorized, unpriced work into the baseline and will concurrently assess
    work package data while establishing the formal performance measurement
    baseline. DOD also stated that a selected portion of this work is already
    being accomplished by DCMA. We continue to believe in the feasibility of
    our recommendation. DCMA officials told us that they

    could monitor the movement of budget into the baseline and verify the work
    packages associated with the budget. In addition, the guidelines
    Recommendations for

    Executive Action Agency Comments and Our Evaluation

    Page 26 GAO- 03- 600 Missile Defense

    state that surveillance may be accomplished through sampling of internal
    and external data. We believe that if DCMA sampled the data as it is
    transferred into the baseline, the implementation of this recommendation
    should not be burdensome.

    In responding to the second part of recommendation two, DOD stated that
    MDA will continue to adhere to current DOD policy by starting an IBR of
    any major contract modification within 6 months. MDA correctly pointed

    out that DOD*s Interim Defense Acquisition Guidebook only requires a
    review be initiated within 6 months (180 days) after a contract is awarded
    or a major modification is issued. However, DOD*s Earned Value Management
    Implementation Guide states that such a review is conducted within 6
    months. Similar language is found in the applicable clause from the GMD
    contract, 25 which states that such reviews shall be

    scheduled as early as practicable and should be conducted within 180
    calendar days after the incorporation of major modifications. While we
    understand the difficulty of conducting reviews within 180 days when the
    contract is complex and many subcontractors are involved, we believe that
    it is important for the government to complete an IBR as soon as

    possible to ensure accurate measurement of progress toward the program*s
    cost, schedule, and performance goals.

    DOD also provided technical comments to this report, which we considered
    and implemented as appropriate. In its technical comments, for example,
    DOD expressed particular concern that our draft report language asserting
    MDA*s inability to rely on the EVM system was unsupported and misleading.
    DOD also stated that its prime contractor*s EVM system is reliable. It
    stated, for example, that MDA has reviewed, and continues to review on a
    monthly basis, the contractor*s cost performance reports and that the
    prime contractor*s EVM system and accounting systems have been fully
    certified and validated by DCMA. We modified our report to better
    recognize MDA*s ability to use and trust the EVM system. However, we still
    believe that MDA would benefit from taking additional measures to increase
    its confidence in the accuracy of its interim baselines. Also, when the
    revised baseline is in place, a review of its

    formulation and execution is necessary before MDA can confidently and
    fully rely on data from the EVM system. 25 Defense Federal Acquisition
    Regulation Supplement clause 252.234- 7001, EVM System (March 1998).

    Page 27 GAO- 03- 600 Missile Defense

    We conducted our review from December 2001 through August 2003 in
    accordance with generally accepted government auditing standards. As
    arranged with your staff, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days from
    its issue date. At that time, we plan to provide copies of this report to
    interested congressional committees, the Secretary of Defense, and the
    Director, Missile Defense Agency. We will make copies available to others
    upon request. In addition, the report will be available at no charge on
    the GAO Web site at http:// www. gao. gov/.

    If you or your staff have any questions concerning this report, please
    contact me on (202) 512- 4841. Major contributors to this report are
    listed in appendix V.

    Sincerely yours, Robert E. Levin Director Acquisition and Sourcing
    Management

    Appendix I: Scope and Methodology Page 28 GAO- 03- 600 Missile Defense

    To determine when MDA plans to demonstrate the maturity of technologies
    critical to the performance of GMD*s Block 2004 capability, we reviewed
    their critical technologies using technology readiness levels (TRLs)
    developed by the National Aeronautics and Space Administration

    and used by DOD. We did so by asking contractor officials at the Boeing
    System Engineering and Integration Office in Arlington, Virginia, to
    identify the most critical technologies and to assess the level of
    maturity of each technology using definitions developed by the National
    Aeronautics and Space Administration. We reviewed these assessments along
    with program documents, such as the results of recent flight tests and
    discussed the results with contractor and agency officials in order to
    reach a consensus, where appropriate, on the readiness level for each
    technology and identify the reasons for any disagreements.

    In reviewing the agency*s current cost estimate to develop the first block
    of the GMD element and its test bed, we reviewed and analyzed budget
    backup documents, cost documents, and selected acquisition reports for the
    GMD program extending over a period of several years. We also met with
    program officials responsible for managing the development and fielding of
    the GMD Block 2004 capability. For example, we met with officials from the
    GMD Joint Program Office in Arlington, Virginia, and Huntsville, Alabama;
    and the Office of the Deputy Assistant for Program Integration at the MDA,
    Arlington, Virginia.

    To determine whether there were any significant risks associated with the
    estimate, we met with agency officials responsible for determining the
    cost of the GMD element to find out if there were costs that were omitted,
    but should have been included, in the estimate. We also analyzed data from
    cost performance reports that the GMD contractor developed for the MDA. We
    reviewed data from the GMD element and contracting officials and conducted
    interviews to discuss the data. Although we did not independently verify
    the accuracy of the cost performance reports we received from MDA, the
    data were assessed independently by DCMA. Appendix I: Scope and
    Methodology

    Appendix II: Comments from the Department of Defense

    Page 29 GAO- 03- 600 Missile Defense

    Appendix II: Comments from the Department of Defense

    Appendix II: Comments from the Department of Defense

    Page 30 GAO- 03- 600 Missile Defense

    Appendix II: Comments from the Department of Defense

    Page 31 GAO- 03- 600 Missile Defense

    Appendix III: Technology Readiness Level Assessment Matrix

    Page 32 GAO- 03- 600 Missile Defense

    Technology readiness level (TRL) Description Hardware /software
    Demonstration environment

    1. Basic principles observed and reported. Lowest level of technology

    readiness. Scientific research begins to be translated into applied
    research and development. Examples might include paper studies of a
    technology*s basic properties.

    None (paper studies and analysis) None

    2. Technology concept and/ or application formulated. Invention begins.
    Once

    basic principles are observed, practical applications can be invented. The
    application is speculative, and there is no proof or detailed analysis to
    support the assumption. Examples are still limited to paper studies. None
    (paper studies and

    analysis) None 3. Analytical and experimental critical

    function and/ or characteristic proof of concept.

    Active research and development is initiated. This includes analytical
    studies and laboratory

    studies to physically validate analytical predictions of separate elements
    of the technology. Examples include components that are not yet integrated
    or representative. Analytical studies and

    demonstration of nonscale individual components (pieces of subsystem).

    Lab 4. Component and/ or breadboard.

    Validation in laboratory environment.

    Basic technological components are integrated to establish that the pieces
    will work together. This is relatively *low fidelity* compared to the
    eventual system. Examples include integration of *ad hoc* hardware in a
    laboratory.

    Low fidelity breadboard. Integration of nonscale components to show

    pieces will work together. Not fully functional or form or fit but
    representative of technically feasible approach suitable for flight
    articles.

    Lab

    Appendix III: Technology Readiness Level Assessment Matrix

    Appendix III: Technology Readiness Level Assessment Matrix

    Page 33 GAO- 03- 600 Missile Defense

    Technology readiness level (TRL) Description Hardware /software
    Demonstration environment

    5. Component and/ or breadboard validation in relevant environment.
    Fidelity of breadboard

    technology increases significantly. The basic technological components are
    integrated with reasonably realistic supporting elements so that the
    technology can be tested in a simulated environment. Examples include
    *high fidelity* laboratory integration of components.

    High fidelity breadboard. Functionally equivalent but not necessarily form
    and/ or fit (size, weight, materials, etc). Should be approaching
    appropriate

    scale. May include integration of several components with reasonably
    realistic support elements/ subsystems to demonstrate functionality.

    Lab demonstrating functionality but not form and fit. May include
    flightdemonstrating breadboard in surrogate aircraft.

    Technology ready for detailed design studies.

    6. System/ subsystem model or prototype demonstration in a relevant
    environment.

    Representative model or prototype system, which is well beyond the
    breadboard tested for TRL 5, is tested in a relevant environment.
    Represents a major step up in a technology*s

    demonstrated readiness. Examples include testing a prototype in a high
    fidelity laboratory environment or in simulated operational environment.

    Prototype. Should be very close to form, fit, and function. Probably
    includes the integration of many new components and realistic supporting
    elements/ subsystems if needed to demonstrate full functionality of the
    subsystem.

    High- fidelity lab demonstration or limited/ restricted flight
    demonstration for a relevant environment.

    Integration of technology is well defined.

    7. System prototype demonstration in an operational environment. Prototype
    near or at

    planned operational system. Represents a major step up from TRL 6,
    requiring the demonstration of an actual system prototype in an
    operational environment, such as in an aircraft, on a vehicle or in

    space. Examples include testing the prototype in a test bed aircraft.

    Prototype. Should be form, fit and function integrated with other key
    supporting elements/ subsystems to demonstrate full functionality of
    subsystem.

    Flight demonstration in representative operational environment such as
    flying test bed or demonstrator aircraft.

    Technology is well substantiated with test data.

    8. Actual system completed and *flight qualified* through test and
    demonstration. Technology has been

    proven to work in its final form and under expected conditions. In almost
    all cases, this TRL represents the end of true system development.
    Examples include developmental test and evaluation of the system in its
    intended weapon system to determine if it meets design specifications.
    Flight- qualified hardware Developmental test and evaluation

    in the actual system application

    Appendix III: Technology Readiness Level Assessment Matrix

    Page 34 GAO- 03- 600 Missile Defense

    Technology readiness level (TRL) Description Hardware /software
    Demonstration environment

    9. Actual system *flight proven* through successful mission operations.

    Actual application of the technology in its final form and under mission
    conditions, such as those encountered in operational test and evaluation.
    In almost all cases, this is the end of the last *bug fixing* aspects of
    true system development. Examples include using the system under
    operational mission conditions.

    Actual system in final form Operational test and evaluation in operational
    mission conditions Source: GAO and its analysis of National Aeronautics
    and Space Administration data.

    Note: GAO information based on U. S. General Accounting Office, Missile
    Defense: Knowledge- Based Decision Making Needed to Reduce Risks in
    Developing Airborne Laser, GAO- 02- 631 (Washington, D. C.: June 2002).

    Appendix IV: Importance of Earned Value Management

    Page 35 GAO- 03- 600 Missile Defense

    Pulling together essential cost, schedule, and technical information in a
    meaningful, coherent fashion is always a challenge for any program.
    Without this information, management of the program will be fragmented,
    presenting a distorted view of program status. For several decades, DOD

    has compared the value of work performed to the work*s actual cost. This
    measurement is referred to as Earned Value Management (EVM). Earned value
    goes beyond the two- dimensional approach of comparing budgeted costs to
    actuals. It attempts to compare the value of work accomplished during a
    given period with the work scheduled for that period. By using the value
    of completed work as a basis for estimating the cost and time needed to
    complete the program, the earned value concept should alert

    program managers to potential problems early in the program. In 1996, in
    response to acquisition reform initiatives, DOD reemphasized the
    importance of earned value in program management and adopted 32 criteria
    for evaluating the quality of management systems. These 32 criteria are
    organized into 5 basic categories: organization, planning and budgeting,
    accounting considerations, analysis and management reports, and revisions
    and data maintenance. The 32 criteria are listed in table 1. In general
    terms, the criteria require contractors to (1) define the contractual

    scope of work using a work breakdown structure; (2) identify
    organizational responsibility for the work; (3) integrate internal
    management subsystems; (4) schedule and budget authorized work; (5)
    measure the progress of work based on objective indicators; (6) collect
    the cost of labor and materials associated with the work performed;

    (7) analyze any variances from planned cost and schedules; (8) forecast
    costs at contract completion; and (9) control changes.

    Table 3: 32 Criteria for Earned Value Management Systems Categories of
    Criteria Criteria Organization 1. Define the authorized work elements for
    the program. A work breakdown structure, tailored

    for effective internal management control, is commonly used in this
    process. 2. Identify the program organizational structure, including the
    major subcontractors responsible for accomplishing the authorized work,
    and define the organizational elements in which work will be planned and
    controlled. 3. Provide for the integration of the company*s planning,
    scheduling, budgeting, work

    authorization, and cost accumulation processes with each other and, as
    appropriate, the program work breakdown structure and the program
    organizational structure.

    4. Identify the company organization or function responsible for
    controlling overhead (indirect costs).

    5. Provide for integration of the program work breakdown structure and the
    program organizational structure in a manner that permits cost and
    schedule performance measurement by elements of either or both structures
    as needed.

    Appendix IV: Importance of Earned Value Management

    Appendix IV: Importance of Earned Value Management

    Page 36 GAO- 03- 600 Missile Defense

    Categories of Criteria Criteria Planning and Budgeting 6. Schedule the
    authorized work in a manner that describes the sequence of work and

    identifies significant task interdependencies required to meet the
    requirements of the program. 7. Identify physical products, milestones,
    technical performance goals, or other indicators that will be used to
    measure progress.

    8. Establish and maintain a time- phased budget baseline, at the control
    account level, against which program performance can be measured. Budget
    for far- term efforts may be held in higher- level accounts until an
    appropriate time for allocation at the control account level. Initial
    budgets established for performance measurement will be based on either
    internal management goals or the external customer- negotiated target cost
    including estimates for authorized but undefinitized work. On government
    contracts, if an over target baseline is used for performance measurement
    reporting purposes, prior notification must be provided to the customer.

    9. Establish budgets for authorized work with identification of
    significant cost elements (labor, material, etc.) as needed for internal
    management and for control of subcontractors.

    10. To the extent it is practical to identify the authorized work in
    discrete work packages, establish budgets for this work in terms of
    dollars, hours, or other measurable units. Where the entire control
    account is not subdivided into work packages, identify the far term effort
    in larger planning packages for budget and scheduling purposes.

    11. Provide that the sum of all work package budgets plus planning package
    budgets within a control account equals the control account budget.

    12. Identify and control level of effort activity by time- phased budgets
    established for this purpose. Only that effort which is unmeasurable or
    for which measurement is impractical may be classified as level of effort.

    13. Establish overhead budgets for each significant organizational
    component of the company for expenses that will become indirect costs.
    Reflect in the program budgets, at the appropriate level, the amounts in
    overhead pools that are planned to be allocated to the program as indirect
    costs.

    14. Identify management reserves and undistributed budget. 15. Provide
    that the program target cost goal is reconciled with the sum of all
    internal program budgets and management reserves.

    Accounting Considerations 16. Record direct costs in a manner consistent
    with the budgets in a formal system controlled by the general books of
    account.

    17. When a work breakdown structure is used, summarize direct costs from
    control accounts into the work breakdown structure without allocation of a
    single control account to two or more work breakdown structure elements.

    18. Summarize direct costs from the control accounts into the contractor*s
    organizational elements without allocation of a single control account to
    two or more organizational elements.

    19. Record all indirect costs which will be allocated to the contract. 20.
    Identify unit costs, equivalent units costs, or lot costs when needed.

    Accounting Considerations 21. For EVMS, the material accounting system
    will provide for: (1) Accurate cost accumulation and assignment of costs
    to control accounts in a manner consistent with the budgets using
    recognized, acceptable, costing techniques. (2) Cost performance
    measurement at the point in time most suitable for the category of
    material involved, but no earlier than the time of progress payments or
    actual receipt of material. (3) Full accountability of all material
    purchased for the program including the residual inventory.

    Appendix IV: Importance of Earned Value Management

    Page 37 GAO- 03- 600 Missile Defense

    Categories of Criteria Criteria Analysis and Management Reports 22. At
    least on a monthly basis, generate the following information at the
    control account and

    other levels as necessary for management control using actual cost data
    from, or reconcilable with, the accounting system: (1) Comparison of the
    amount of planned budget and the amount of budget earned for work
    accomplished. This comparison provides the schedule variance. (2)
    Comparison of the amount of the budget earned and the actual (applied
    where appropriate)

    direct costs for the same work. This comparison provides the cost
    variance. 23. Identify, at least monthly, the significant differences
    between both planned and actual schedule performance and planned and
    actual cost performance, and provide the reasons for the variances in the
    detail needed by program management. 24. Identify budgeted and applied (or
    actual) indirect costs at the level and frequency needed by management for
    effective control, along with the reasons for any significant variances.
    25. Summarize the data elements and associated variances through the
    program organization

    and/ or work breakdown structure to support management needs and any
    customer reporting specified in the contract.

    26. Implement managerial actions taken as the result of earned value
    information. 27. Develop revised estimates of cost at completion based on
    performance to date, commitment values for material, and estimates of
    future conditions. Compare this information with the performance
    measurement baseline to identify variances at completion important to
    company management and any applicable customer reporting requirements
    including statements of funding requirements.

    Revisions and Data Maintenance 28. Incorporate authorized changes in a
    timely manner, recording the effects of such changes in budgets and
    schedules. In the directed effort prior to negotiation of a change, base
    such revisions on the amount estimated and budgeted to the program
    organizations.

    29. Reconcile current budgets to prior budgets in terms of changes to the
    authorized work and internal replanning in the detail needed by management
    for effective control.

    30. Control retroactive changes to records pertaining to work performed
    that would change previously reported amounts for actual costs, earned
    value, or budgets. Adjustments should be made only for correction of
    errors, routine accounting adjustments, effects of customer or management
    directed changes, or to improve the baseline integrity and accuracy of
    performance measurement data.

    31. Prevent revisions to the program budget except for authorized changes.
    32. Document changes to the performance measurement baseline. Source:
    Interim Defense Acquisition Guidebook, app. 4.

    Note: In the Interim Defense Acquisition Guidebook, DOD states that these
    guidelines are reproduced from the American National Standards (ANSI)
    Institute/ Electronic Industries Alliance (EIA) EVM System Standard (ANSI/
    EIA- 748- 98), Chapter 2 (May 19, 1998).

    Appendix IV: Importance of Earned Value Management

    Page 38 GAO- 03- 600 Missile Defense

    The criteria have become the standard for EVM and have also been adopted
    by major US government agencies, industry, and the governments of Canada
    and Australia. The full application of EVM system criteria is

    appropriate for large cost reimbursable contracts where the government
    bears the cost risk. For such contracts, the management discipline
    described by the criteria is essential. In addition, data from an EVM
    system have been proven to provide objective reports of contract status,
    allowing

    numerous indices and performance measures to be calculated. These can then
    be used to develop accurate estimates of anticipated costs at completion,
    providing early warning of impending schedule delays and cost overruns.

    The standard format for tracking earned value is through a Cost
    Performance Report (CPR). The CPR is a monthly compilation of cost,
    schedule and technical data which displays the performance measurement
    baseline, any cost and schedule variances from that baseline, the amount
    of management reserve used to date, the portion of the contract that is
    authorized unpriced work, and the contractor*s latest revised estimate to
    complete the program.

    As a result, the CPR can be used as an effective management tool because
    it provides the program manager with early warning of potential cost and
    schedule overruns. Using data from the CPR, a program manager can assess
    trends in cost and schedule performance. This information is useful
    because trends tend to continue and can be difficult to reverse. Studies
    have shown that once programs are 15 percent complete the performance
    indicators are indicative of the final outcome. For example, a CPR showing
    a negative trend for schedule status would indicate that the program is
    behind schedule. By analyzing the CPR, one could determine the cause of
    the schedule problem such as delayed flight tests, changes in
    requirements, or test problems because the CPR contains a section that
    describes the reasons for the negative status. A negative schedule
    condition is a cause for concern, because it can be a predictor of later
    cost problems since additional spending is often necessary to resolve

    problems. For instance, if a program finishes 6 months later than planned,
    additional costs will be expended to cover the salaries of personnel and
    their overhead beyond what was originally expected. CPR data provides the
    basis for independent assessments of a program*s cost and schedule status
    and can be used to project final costs at completion in addition to
    determining when a program should be completed.

    Examining a program*s management reserve is another way that a program can
    use a CPR to determine potential issues early on.

    Appendix IV: Importance of Earned Value Management

    Page 39 GAO- 03- 600 Missile Defense

    Management reserves, which are funds that may be used as needed, provide
    flexibility to cope with problems or unexpected events. EVM experts agree
    that transfers of management reserve should be tracked and reported
    because they are often problem indicators. An alarming situation arises if
    the CPR shows that the management reserve is being used at a faster pace
    than the program is progressing toward completion. For example, a problem
    would be indicated if a program has used 80 percent of its management
    reserve but only completed 40 percent of its work. A program*s management
    reserve should contain at least 10 percent of the cost to complete a
    program so that funds will always be available to cover future unexpected
    problems that are more likely to surface as the program moves into the
    testing and evaluation phase.

    Appendix V: GAO Contact and Staff Acknowledgments

    Page 40 GAO- 03- 600 Missile Defense

    Barbara Haynes (256) 922- 7500 In addition to the individual named above
    Yvette Banks, Myra Watts Butler, Cristina Chaplain, Roger Corrado,
    Jennifer Echard, Dayna Foster, Matt Lea, Karen Richey, and Randy Zounes
    made key contributions to this report. Appendix V: GAO Contact and Staff

    Acknowledgments GAO Contact Acknowledgments

    (120109)

    The General Accounting Office, the audit, evaluation and investigative arm
    of Congress, exists to support Congress in meeting its constitutional
    responsibilities and to help improve the performance and accountability of
    the federal government for the American people. GAO examines the use of
    public funds; evaluates federal programs and policies; and provides
    analyses, recommendations, and other assistance to help Congress make
    informed oversight, policy, and funding decisions. GAO*s commitment to
    good government is reflected in its core values of accountability,
    integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
    and fulltext files of current reports and testimony and an expanding
    archive of older products. The Web site features a search engine to help
    you locate documents using key words and phrases. You can print these
    documents in their entirety, including charts and other graphics.

    Each day, GAO issues a list of newly released reports, testimony, and
    correspondence. GAO posts this list, known as *Today*s Reports,* on its
    Web site daily. The list contains links to the full- text document files.
    To have GAO e- mail

    this list to you every afternoon, go to www. gao. gov and select
    *Subscribe to e- mail alerts* under the *Order GAO Products* heading.

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
    Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000

    TDD: (202) 512- 2537 Fax: (202) 512- 6061

    Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
    fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
    512- 7470 Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
    4800

    U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
    C. 20548 GAO*s Mission Obtaining Copies of

    GAO Reports and Testimony

    Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
    Programs Public Affairs
    *** End of document. ***