    Joint Strike Fighter: DOD Plans to Enter Production before
    Testing Demonstrates Acceptable Performance (15-MAR-06,
    GAO-06-356).

    The Joint Strike Fighter (JSF) is DOD's most expensive aircraft
    program. The program represents 90 percent of the remaining
    planned investment for recapitalizing DOD's aging tactical
    aircraft fleet. GAO is required by law to review the program
    annually for 5 years, beginning in fiscal year 2005. This is our
    second report and GAO assessed the program's acquisition
    approach--in terms of capturing knowledge for key investment
    decisions--and identified an alternative to improve outcomes.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-356
        ACCNO:   A49118
      TITLE:     Joint Strike Fighter: DOD Plans to Enter Production
    before Testing Demonstrates Acceptable Performance
         DATE:   03/15/2006
      SUBJECT:   Defense procurement
    	     Fighter aircraft
    	     Military aircraft
    	     Operational testing
    	     Procurement planning
    	     Product evaluation
    	     Program evaluation
    	     Tactical air forces
    	     Concurrent development and production
    	     B-2 Aircraft
    	     B-2A Aircraft
    	     Comanche Helicopter
    	     Crusader Artillery System
    	     F-16 Aircraft
    	     F-22 Aircraft
    	     F/A-18E/F Aircraft
    	     Hornet Aircraft
    	     Joint Strike Fighter
    	     Joint Strike Fighter program
    	     Osprey Aircraft
    	     Stealth Aircraft
    	     V-22 Aircraft

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-356

         * Results in Brief
         * Background
         * JSF Acquisition Strategy Will Begin Procurement before Testi
              * Key Testing Events Will Not Be Completed before Significant
                   * Initial Procurement Decisions Will Be Made before the Basic
                   * Mission Capabilities Will Not Be Tested until 4 Years after
                   * Aggressive Software Development Schedule May Cause Delays
              * The Current Acquisition Strategy Requires Prolonged Reliance
         * Evolutionary Acquisition Strategy Provides a Less Risky Alte
              * JSF Program Is Designed to Deliver Full Required Capability
              * Funding Needs Could Increase Given the Level of Risk to Comp
              * An Incremental Development Approach Provides an Alternative
         * Conclusions
         * Matters for Congressional Consideration
         * Recommendations for Executive Action
         * Agency Comments and Our Evaluation
              * Order by Mail or Phone

    Report to Congressional Committees

    United States Government Accountability Office

    GAO

    March 2006

    JOINT STRIKE FIGHTER

    DOD Plans to Enter Production before Testing Demonstrates Acceptable
    Performance

    Joint Strike Fighter Acquisition

    GAO-06-356

    Contents

    Letter 1

    Results in Brief 2
    Background 4
    JSF Acquisition Strategy Will Begin Procurement before Testing
    Demonstrates the Aircraft's Performance 6
    Evolutionary Acquisition Strategy Provides a Less Risky Alternative for
    the JSF Program 13
    Conclusions 22
    Matters for Congressional Consideration 23
    Recommendations for Executive Action 24
    Agency Comments and Our Evaluation 24
    Appendix I Scope and Methodology 29
    Appendix II Changes in JSF Program Purchase Quantities and Costs 31
    Appendix III Comments from the Department of Defense 32

    Table

    Table 1: Military Services' Planned Use for the Joint Strike Fighter 5

    Figures

    Figure 1: Overlap of Low-Rate Initial Production Investments and Testing
    Demonstrations of the JSF Variants 9
    Figure 2: JSF Program Acquisition 14
    Figure 3: JSF's Annual Funding Requirements, as of December 2004 16
    Figure 4: F-16 Incremental Development Approach 18
    Figure 5: Planned JSF Blocks 19
    Figure 6: Potential Application of an Incremental Development Approach to
    the JSF Program 21

    Abbreviations

    DOD Department of Defense

    JSF Joint Strike Fighter

    OSD Office of the Secretary of Defense

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States Government Accountability Office

    Washington, DC 20548

    March 15, 2006

    Congressional Committees

    Since the Joint Strike Fighter (JSF) program began in 1996, Congress has
    appropriated nearly $25 billion for its development. It is the Department
    of Defense's (DOD) most expensive aircraft acquisition program. As the
    program is currently planned, DOD estimates it will spend $257 billion to
    develop and procure about 2,443 aircraft and related support equipment by
    2027 and an additional $347 billion to operate and support these aircraft
    once they have been fielded.

    JSF program goals are to develop and field a family of stealthy, strike
    fighter aircraft for the Navy, Air Force, and Marine Corps, and United
    States allies. Design plans call for three variants of the aircraft, but
    the program aims to provide maximum commonality to minimize life cycle
    costs. JSF is a central part of DOD's overall recapitalization strategy
    for its tactical aircraft fleet, representing 90 percent of the remaining
    planned investment for its major tactical aircraft programs. How DOD
    manages the JSF program in the future will be critical for getting the
    most out of this remaining large investment and for success in replacing
    the aging fleet of tactical aircraft.

    The Ronald W. Reagan National Defense Authorization Act for Fiscal Year
    2005 (P.L. 108-375) requires GAO to review the JSF program annually for 5
    years.1 In March 2005, we reported the JSF's original business case was
    unexecutable and recommended that DOD establish an executable program
    consistent with best practices and DOD policy regarding evolutionary
    acquisitions.2 We also recommended that this new business case be
    accompanied by a knowledge-based acquisition approach 3-an approach that
    ensures attainment and use of demonstrated product knowledge before making
    future investments. In commenting on our report, DOD stated that JSF's
    restructured acquisition plan would incorporate a knowledge-based,
    evolutionary approach consistent with DOD policy. This is our second
    report. Specifically, we (1) determined whether the JSF program
    acquisition strategy captures critical knowledge in time to make
    production investment decisions and (2) identified an alternative to the
    current acquisition strategy to improve JSF program outcomes.

    1Section 213 of the act requires us to assess the extent to which the
    system development and demonstration program is currently meeting key
    cost, schedule, and performance goals; the likelihood that the program
    will be completed within estimated costs; and the program's current
    acquisition plan leading to production.

    2GAO, Tactical Aircraft: Opportunity to Reduce Risks in Joint Strike
    Fighter Program with Different Acquisition Strategy, GAO-05-271
    (Washington, D.C.: Mar. 15, 2005).

    The act also requires us to certify whether we had access to sufficient
    information to make informed judgments on the matters contained in our
    report. We were provided sufficient information to assess the plans to
    capture technology, design, and manufacturing knowledge for the three JSF
    variants, the evolutionary nature of the acquisition strategy, and the
    opportunities to improve outcomes. At the time of our review, the Office
    of the Secretary of Defense (OSD) was still preparing its fiscal year 2007
    budget request as well as conducting its independent assessment of the
    program cost estimate, which is not expected to be completed until
    sometime in 2006. Additionally, the Quadrennial Defense Review was ongoing
    during our review, and we did not have insights into the potential
    outcomes or how the results of the study might affect the JSF program. We
    performed our work from June 2005 through March 2006 in accordance with
    generally accepted government auditing standards. For more on our scope
    and methodology, see appendix I.

                                    Results in Brief

    Despite recent program restructuring, the JSF program continues to base
    its acquisition strategy on a highly concurrent approach that makes
    significant investments in manufacturing capabilities and production
    aircraft before flight testing demonstrates the JSF's performance. As a
    result, significant development risk remains, and it is likely that
    current cost and schedule goals will not be met. The program plans to
    proceed into low-rate initial production in 2007 with inadequate testing
    to prove a mature design for any of the three basic JSF variant airframes,
    without developing critical software, or without a fully integrated
    aircraft with advanced mission systems and prognostic maintenance
    capabilities. All three production representative variants4 will not be in
    flight testing until 2009, and a fully configured, integrated development
    aircraft will not begin flight testing until 2011-4 years after production
    begins. By this time, DOD plans to have ordered 190 aircraft at a cost of
    about $26 billion. By 2013, when initial operational testing is expected
    to be complete, the program plans to have procured 424 aircraft at an
    expected cost of $49 billion. Because it will lack necessary technology,
    design, and performance knowledge, DOD plans to procure early production
    aircraft using cost-reimbursable-type contracts. This type of contract
    places a significantly greater cost risk on DOD.

    3A business case provides demonstrated evidence that (1) the warfighter
    need exists and that it can best be met with the chosen concept and (2)
    the concept can be developed and produced within existing
    resources-including proven technologies, design knowledge, adequate
    funding, and adequate time to deliver the product when needed.
    Establishing a business case calls for a realistic assessment of risks and
    costs; doing otherwise undermines the intent of the business case and
    invites failure.

    DOD has revised its acquisition policy to embrace evolutionary
    acquisition, allowing managers to develop and evolve a product through
    small, time-phased development increments-an alternative still available
    to the JSF program. Instead of establishing time-phased requirements for
    aircraft to be delivered in sequence that could first meet DOD's need to
    recapitalize its aging fleet of aircraft and then evolve the aircraft to
    eventually achieve improved capabilities in future increments, DOD chose a
    single-step approach to develop and deliver the JSF with ultimate
    capabilities in a single 12-year system development program. The large
    amount of uncertainty in this approach has already led to poor cost and
    schedule estimates for the JSF program and a reduction in DOD's buying
    power. The length and scope of work remaining continue to make it
    difficult to predict the cost and time needed to complete the program. The
    Air Force's JSF predecessor, the F-16 Fighter program, provides a model
    for a less risky alternative JSF acquisition strategy that delivers
    weapons to the warfighter more quickly and that recapitalizes tactical
    aircraft forces sooner. The F-16 program successfully delivered 2,200
    aircraft with incremental improvements as technology became available over
    the span of about 30 years. An initial F-16 capability was delivered to
    the warfighter within about 4 years after development began and
    substantial quantities were purchased with each increment to meet
    warfighter needs, including the replacement of aging legacy tactical
    fighter aircraft.

    4A production representative aircraft is one that is built using
    production-like tooling and methods and using production workers and is of
    the design expected be built to satisfy the requirements of the
    warfighter. Flight-testing a production representative prototype that
    incorporates all critical software is the primary method to capture key
    design knowledge that indicates the system will work as planned, is
    reliable, and is ready for production. The higher the percentage of
    testing completed, the lower the risk of investing in production.

    To improve the chances for a successful outcome, we are recommending the
    JSF program delay production and investments in production capability
    until the aircraft design qualities and integrated mission capabilities of
    the fully configured and integrated JSF aircraft variants have been proven
    to work in flight testing. We are also recommending the Secretary of
    Defense report to the Congress by July 2006 on the feasibility of an
    incremental acquisition approach that follows the intent of DOD
    evolutionary acquisition policy and delivers a first increment that limits
    new content to proven technologies and design.

    DOD partially concurred with our recommendations stating that the current
    JSF acquisition strategy and management practices will be able to achieve
    the objectives of our recommendations, and therefore, further limits on
    production are unnecessary. DOD also stated that the program's strategy
    balances technical risk, financial resources, and operational needs to
    reduce program risk and is based on a knowledge-based, incremental
    approach. We continue to believe DOD's planned strategy to make
    significant investments in production concurrent with testing is very
    risky and is similar to strategies of past programs with poor outcomes. We
    believe that DOD has an opportunity to reduce risk by adopting an
    acquisition strategy based on capturing technology, design, and
    manufacturing knowledge. We also believe that smaller, more manageable
    commitments in capabilities would make JSF program cost and schedule more
    predictable and deliver needed capabilities to the warfighter sooner.
    Because DOD does not plan to make changes as a result of our
    recommendations, we believe Congress should consider delaying
    authorizations and appropriations for JSF procurement until DOD develops a
    new business case and demonstrates the aircraft design qualities and
    integrated mission capabilities of the fully configured and integrated JSF
    variants work as designed based on actual flight testing.

                                       Background

    JSF is a joint, multinational acquisition program for the Air Force, Navy,
    Marine Corps, and eight cooperative international partners. The program
    began in November 1996 with a 5-year competition between Lockheed Martin
    and Boeing to determine the most capable and affordable preliminary
    aircraft design. Lockheed Martin won the competition, and the program
    entered system development and demonstration in October 2001.

    DOD has set major expectations for the program. The program's objective is
    to develop and deploy a technically superior and affordable fleet of
    aircraft that support the warfighter in performing a wide range of
    missions in a variety of theaters. The single-seat, single-engine aircraft
    is being designed to be self-sufficient or part of a multisystem and
    multiservice operation, and to rapidly transition between air-to-surface
    and air-to-air missions while still airborne. To achieve its mission, JSF
    will incorporate low observable technologies, defensive avionics, advanced
    onboard and offboard sensor fusion,5 internal and external weapons, and
    advanced prognostic maintenance capability. According to DOD, these
    technologies represent a quantum leap over legacy tactical aircraft
    capabilities. At the same time, the JSF aircraft design includes three
    variants: a conventional takeoff and landing variant for the Air Force; an
    aircraft carrier-suitable variant for the Navy; and a short takeoff and
    vertical landing variant for the Marine Corps, the Air Force, and the
    United Kingdom. JSF is intended to replace a substantial number of aging
    fighter and attack aircraft in DOD's current inventory (see table 1).

    Table 1: Military Services' Planned Use for the Joint Strike Fighter

    Service      Planned use
    Air Force    Replacement for the F-16 and A-10; complement the F-22A
    Marine Corps Replacement for the AV-8B and F/A-18 A/C/D
    Navy         Complement the F/A-18 E/F

    Source: DOD data.

    In recent years, DOD has revised its acquisition policy to embrace an
    evolutionary, or incremental, approach to improve program outcomes. The
    acquisition policy states evolutionary acquisition is the preferred DOD
    strategy for rapid acquisition of mature technology for the user by
    delivering capability in increments, recognizing up front the need for
    future capability improvements. The objective is to balance needs with
    resources in order to put capability into the hands of the warfighter more
    quickly. The policy states that a product enters system development and
    demonstration when an affordable increment of militarily useful capability
    has been identified, technology has been demonstrated in a relevant
    environment, and a system can be developed for production in a short time
    frame (normally less than 5 years). Each increment of an evolutionary
    acquisition will have its own decision milestones and baseline-cost,
    schedule, and performance requirements. In commenting on our March 2005
    JSF report, DOD stated that JSF's restructured acquisition plan would
    balance technical, cost, and schedule risk and incorporate a
    knowledge-based, evolutionary approach to system acquisition consistent
    with DOD policy.

    5Sensor fusion is the ability to take information from both multiple
    onboard and offboard aircraft sensors and display the information in an
    easy-to-use format for the pilot. It is vitally important, because the JSF
    is a single-seat aircraft, and the pilot needs support to carry out
    multiple types of missions.

    JSF Acquisition Strategy Will Begin Procurement before Testing Demonstrates the
                                 Aircraft's Performance

    DOD's acquisition strategy for JSF has not substantially changed as a
    result of the program's restructuring last year. In May 2005, DOD
    leadership approved the program's plan, which intends to start production
    in 2007-more than 2 years before all three JSF variants have completed
    some flight testing of the aircraft's basic design, 4 years before a fully
    configured and integrated aircraft is expected to be flight tested, and 6
    years before development and initial operational testing are scheduled to
    be completed.6 Before development has ended, DOD will have potentially
    signed procurement contracts for 424 JSF aircraft at an estimated cost of
    $49 billion. Starting production before ensuring the design is mature
    through flight testing significantly increases the risk of costly design
    changes that will push the program over budget and behind schedule.
    Evidence of the risk associated with concurrently developing, testing, and
    producing the JSF aircraft can be seen in the program office strategy to
    place initial production orders on a cost reimbursement contract, placing
    an unusually high risk burden on the government during the early
    production phase.

    Key Testing Events Will Not Be Completed before Significant Procurement
    Investments Are Made

    The JSF program plans to begin low-rate initial production of the aircraft
    before many of JSF's technology advances and basic flying qualities are
    flight-tested and to produce substantial quantities of aircraft before
    initial operational testing is completed. According to JSF program plans,
    DOD's low-rate initial production quantities will increase from 5 aircraft
    a year in 2007 to 133 a year, before development and initial operational
    testing are completed in 2013.7 This production rate will require DOD to
    invest significantly in tooling, facilities, and personnel. Initial
    contractor cost estimates indicate that close to $1 billion of new tooling
    will be needed to support low-rate initial production rates. Early
    contractor data also show that the program will need to double its
    manufacturing workforce by 2008 and will need six times the number of
    manufacturing personnel before low-rate initial production ends. Total
    monthly spending for production activities in 2013 is expected to approach
    $1 billion, a significant increase from $100 million a month when
    production is scheduled to begin in 2007.

    6Initial operational testing consists of field tests intended to
    demonstrate a system's effectiveness and suitability for military use.

    7These figures do not include the potential for orders for international
    partners during low-rate initial production. Preliminary data indicate
    that these orders could significantly increase this rate.

    The cost of discovering design problems during production could be
    significant if testing shows that large, structural components of the
    aircraft require modifications. Design changes needed in one variant could
    also ripple through the other two variants, reducing efficiencies
    necessary to lower production and operational costs with common parts and
    manufacturing processes for the three variants. Some industry officials
    have indicated that the cost of design changes such as these could be 10
    to 1,000 times greater, depending on how far the product has progressed
    into production.

    When the JSF program is expected to begin low-rate initial production in
    2007,8 the program will have completed less than 1 percent of the flight
    test program and none of the three JSF variants will have a production
    representative prototype built and in flight testing. Features critical to
    JSF's operational success, such as the low-observable and highly common
    airframe and the advanced mission systems, will not have been demonstrated
    in a flight test environment. The program plans to proceed into low-rate
    initial production without demonstrating that (1) the aircraft's flying
    qualities function within the parameters of the flight envelope-that is,
    the set limits for altitude, speed, and angles of attack; (2) the aircraft
    design is reliable; or (3) a fully integrated and capable aircraft system
    can perform as intended. These are key indicators of a program's readiness
    for entering production and making significant investments in tooling,
    facilities, and materials. When the first fully integrated and capable
    development JSF is expected to fly, in 2011, DOD will have committed to
    buy 190 aircraft at an estimated cost of $26 billion under current plans.
    By the time testing is planned to be completed, in 2013, DOD will have
    procured more than double that amount-424 aircraft at an estimated cost of
    about $49 billion. Figure 1 shows the significant overlap in development
    and testing and the major investments in production capability and
    aircraft that inject additional cost and delivery risks that could delay
    delivery of a proven capability to the warfighter.

    8DOD policy contains three critical milestone decision points in its
    acquisition process. One of these, referred to as milestone C, authorizes
    entry into low-rate initial production. However, the JSF program is
    deviating from that milestone framework and has delayed this critical
    milestone decision point to the full-rate production decision point in
    2013-more than 6 years after entering low-rate initial production. This
    delay compromises the controls expected in the DOD acquisition policy for
    the start of production.

    Figure 1: Overlap of Low-Rate Initial Production Investments and Testing
    Demonstrations of the JSF Variants

    aFlight testing data reflect the percentage of the total flight tests
    completed at the time of the planned investment decision, which is
    currently planned for January of each year.

    Under the current schedule, the JSF program plans to manufacture and
    deliver 15 flight test aircraft and 7 ground test (nonflying) articles in
    5 years-an aggressive schedule when compared with schedules of other
    programs with fewer variables. Current JSF schedules are already showing
    delivery of early test aircraft could be later than the planned delivery
    date. Unplanned manufacturing and technical problems can delay the
    completion of a flight test program, increase the number of flight test
    hours needed to verify that the system will work as intended, and affect
    when key knowledge will be available for making investment decisions. For
    example, when the B-2 program began flight testing in July 1989, it
    estimated that the flight test program would last approximately 4.5 years
    and require about 3,600 flight test hours. When the test program ended in
    1997, the flight test hours had grown to 5,000 hours, or by 40 percent,
    over an 8-year period. Program officials cited several causes, including
    difficulties in manufacturing test aircraft and correcting deficiencies
    from unanticipated problems discovered during testing. The F-22A
    encountered similar delays, increasing a planned 4-year flight test
    program to about 8 years and affecting the program's ability to conduct
    operational testing and move into production on schedule.

      Initial Procurement Decisions Will Be Made before the Basic Aircraft Design Is
      Demonstrated

    While each JSF variant is similar-all are being designed to have stealth
    airframes, fly at supersonic speeds, shoot air-to-air missiles, and drop
    bombs on a target-there are subtle airframe design differences to support
    the services' different operational concepts and environments. Test
    officials acknowledge that each airframe variant will require flight
    testing to demonstrate that each will fly as intended. Yet at the time the
    JSF program expects to begin low-rate initial production, only 1 of 15
    flight test aircraft is expected to have started testing. Further, the
    aircraft's test flights are not scheduled to begin until August 2006, less
    than 6 months before the planned decision to begin low-rate initial
    production,9 and will not assess the aircraft in more demanding
    operational environments, such as high angles of attack. Moreover, the
    first flight test aircraft will not include a large number of design
    changes that resulted from an effort to significantly reduce aircraft
    weight over the past 2 years and, therefore, will not represent the
    planned production configuration. The first demonstration of a production
    representative airframe that includes the latest design changes is
    scheduled for late 2007-after production has been initiated-with first
    flight of the short takeoff and vertical landing variant.

    9During this approximately 6-month period of initial tests, which includes
    about 30 planned flights, the program intends to demonstrate the
    aircraft's flight qualities in limited flight conditions, takeoff and
    landing, and initial air refueling. However, recent program data indicate
    that the first flight may be delayed until October 2006, reducing the
    number of flights that may be completed before the production decision.

    All three variants will not be in flight testing until 2009, when the
    carrier variant is expected to have its first flight. Several key test
    events identified by the contractor to demonstrate the flight
    characteristics of these aircraft in the intended operational environment
    are not scheduled until 2009 or later. These include shipboard operations
    for the carrier and short takeoff and vertical landing variants. According
    to the contractor, the full-flight envelope for all three variants will
    not be fully demonstrated until 2011 or 2012-4 to 5 years after low-rate
    initial production is scheduled to begin.

    According to DOD, modeling and simulation will be used to evaluate the
    performance of the aircraft's flying qualities and to support decisions to
    invest in production. However, the Rand Corporation recently reported in a
    study on testing and evaluation that modeling is not a substitute for
    flight testing.10 Rand found that even in performance areas that are well
    understood, it is not unusual for flight testing to uncover problems that
    were not apparent in simulations. Examples include flight effects on the
    wing of the F/A-18 EF and buffeting of stores externally carried on
    various aircraft when flown in certain conditions. Additionally, OSD
    testing officials indicated that flight testing of each variant is
    necessary to demonstrate designed capabilities. Our past work has found
    that flying quality problems were identified during actual flight testing
    on programs like the F-22A, B-2A, and V-22.

    Finally, two major ground tests of the airframe's structural
    integrity-fatigue and static testing-will be in their very early stages or
    not have started when production begins. Fatigue testing, which measures
    the aircraft's durability over its expected life, is slated to begin in
    2008 with testing to show structural fatigue over one lifetime of flying
    to be completed in 2009 and a second lifetime completed by 2010. Fatigue
    testing uses actual JSF airframe structures that are subjected to the
    long-term stresses expected over the aircraft's life. Standards require
    this testing to be done to represent the stresses expected over two
    lifetimes. Static testing, which measures the aircraft's ability to
    withstand the stresses expected to be encountered throughout the
    aircraft's flight regime, is slated to begin in 2007 and last through
    2009. These tests are important to reduce the risk of structural problems
    emerging during production or after aircraft are fielded.

    10Rand Corporation, Test and Evaluation Trends and Costs for Aircraft and
    Guided Weapons (Santa Monica, California, 2004).

      Mission Capabilities Will Not Be Tested until 4 Years after Procurement Has
      Begun

    In addition to late design testing, many of the mission systems planned
    for JSF will not be available for initial flight testing either. Although
    laboratory tests are under way, DOD does not plan to flight-test several
    of the new technologies needed for the JSF to perform its intended
    missions until 2009 at the earliest. Defense Operational Test and
    Evaluation officials have stated that flight-testing capabilities in a
    production representative test aircraft and in the operational environment
    planned for the new system are important to reducing risk. This actual
    environment differs from what can be demonstrated in a laboratory and has
    historically identified unexpected problems. For example, the F-22A
    software worked as expected in the laboratory, but when tested in the
    aircraft, significant problems were identified. These problems delayed
    testing and the delivery of a proven capability. The different levels of
    mission capability will be tested in JSF aircraft as follows.

               o  Block 1, an initial air interdiction capability, is scheduled
               to begin testing in 2009, with initial operational testing
               scheduled for 2011-4 years after DOD plans to begin production.

               o  Block 2, an improved air interdiction and close air support
               capability, is scheduled to begin testing in 2010, with initial
               operational testing scheduled for 2012. This block will include
               several critical technologies that are not fully mature, such as
               the advanced missions systems and prognostics and health
               maintenance, but are critical to meeting requirements like sortie
               generation and mission capabilities.

               o  Block 3, the fully integrated and capable JSF, is scheduled to
               begin testing in 2011. At this time less than 50 percent of the
               planned mission capability testing will have been completed. This
               is close to the same point that the F-22A and other past programs
               experienced difficulties integrating all the complex software and
               hardware components into the aircraft. Flight testing to evaluate
               the effectiveness and suitability of the fully integrated system
               is expected to continue until the full-rate decision in late 2013.

      Aggressive Software Development Schedule May Cause Delays

    JSF's expected performance is largely dependent on demonstrating software
    that supports vehicle, mission system, and other capabilities. The program
    plans to develop over 19 million lines of code-substantially more than the
    lines of code needed for the F-22A. The software is planned to be
    developed in five blocks. The first block is scheduled for completion in
    2006, and the last block is scheduled for completion in 2011. At the time
    the program enters low-rate initial production, the program will have
    completed less than 35 percent of the software needed for the system's
    functionality. Past programs have encountered difficulties in developing
    software, delaying flight test schedules. Data provided by the program
    office indicate that the program is already showing early signs of falling
    behind its software delivery schedule. JSF program officials recognize the
    risk associated with this large software effort.

    The Current Acquisition Strategy Requires Prolonged Reliance on Cost
    Reimbursement Contracts That Place an Increased Risk on DOD

    According to program officials, the uncertainties inherent in concurrently
    developing, testing, and producing the JSF aircraft prevent the pricing of
    initial production orders on a fixed price basis. Consequently, the
    program office plans to place initial production orders on a cost
    reimbursement basis. Cost reimbursement contracts provide for payment of
    allowable incurred costs, to the extent prescribed in the contract. Such
    contracts are generally used only when the uncertainties involved in
    contract performance do not permit costs to be estimated with sufficient
    accuracy to use any type of fixed price contract. Cost reimbursement
    contracts place substantial risk on the buyer-in this case DOD-because the
    contractor's responsibility for performance costs is minimized or reduced.
    In contrast, a fixed price contract provides for a pre-established price,
    and places more risk and responsibility for costs and resulting profit or
    loss on the contractor and provides more incentive for efficient and
    economical performance.

    The program plans to transition to fixed-price-type contracts once the air
    vehicle has a mature design, has been demonstrated in flight test, and is
    producible at established cost targets. According to program officials,
    this transition will occur sometime before full-rate production begins in
    2013. The program office believes the combination of the early concept
    development work, the block development approach, and the relatively small
    numbers of aircraft in the initial production buys allow decisions to be
    made earlier than normal with an acceptable level of risk.

    Evolutionary Acquisition Strategy Provides a Less Risky Alternative for the JSF
                                        Program

    The JSF acquisition strategy currently plans a single-step approach to
    deliver a quantum leap in tactical fighter capability by 2013 and has
    already felt the negative cost and schedule impacts from the executing
    this approach. The length and scope of the remaining effort in the JSF
    program make it even more difficult to accurately estimate cost and
    delivery schedules. The JSF funding profile-which requires an average of
    $11 billion annually for the next two decades-is also at risk to increase
    if costs continue to grow or schedules are further delayed to develop the
    ultimate JSF capabilities. An alternative acquisition strategy, such as
    used by the F-16 program, that sequences capabilities over time based on
    proven technologies and design would reduce risk and deliver aircraft
    sooner. This evolutionary approach is actually the preferred approach in
    DOD's acquisition policy for acquiring new systems for more rapid delivery
    of incremental capabilities to the warfighter.

    JSF Program Is Designed to Deliver Full Required Capability in a Single-Step
    Development Program

    Instead of establishing time-phased requirements for aircraft to be
    delivered in sequence that could first meet DOD's need to recapitalize its
    aging fleet of aircraft and then evolve the aircraft to eventually achieve
    improved capabilities in future system development increments, DOD chose a
    single-step development approach to deliver the full required capability
    by the end of system development in 2013. That approach is now planned to
    last 17 years (5 years in concept development and 12 years in system
    development)-including the development of immature, undemonstrated
    technologies and will take at least two to three times longer than the
    development time of leading commercial firms or that suggested by defense
    acquisition policy. The JSF acquisition strategy is shown in figure 2.

    Figure 2: JSF Program Acquisition

    aThe bar for production is not drawn to scale. JSF production lasts for 20
    years.

    While JSF's acquisition strategy calls for delivering a small number of
    aircraft with limited capabilities, the program has committed to deliver
    the full required capability by the end of system development and
    demonstration in 2013. For JSF, this includes a set of objectives that
    exceeds those of aircraft development programs of the past. JSF will use
    cutting-edge technology to fuse data from the aircraft itself and other
    sources, from all aspects of the aircraft, and display the information to
    the pilot. The aircraft must be able to rapidly transition from ground
    attack to air-to-air missions while airborne. JSF also expects advances in
    technologies for mission systems, prognostics, and autonomic logistics
    support requiring hardware development as well as extensive and complex
    software development and integration to have lower cost of ownership than
    the legacy aircraft while being able to deploy rapidly, sustain high
    mission reliability, and maintain a high sortie generation rate.

    Past single-step development programs have been unsuccessful in predicting
    acquisition costs and delivery schedules. For example, development costs
    for the F-22A fighter, B-2 bomber, Crusader artillery vehicle, and
    Comanche helicopter skyrocketed, and production quantities were either
    substantially reduced or the program was canceled. Such outcomes are a
    poor return on investment and a failure to recapitalize aging legacy
    systems. So far the JSF program is experiencing similar results. Since the
    program's start, development cost has increased 84 percent, the
    development schedule has increased by almost 5 years, and planned delivery
    of capabilities to the warfighter has been delayed. DOD now plans to buy
    535 fewer aircraft than originally planned. As a result, DOD's buying
    power has been significantly reduced-program acquisition unit costs have
    increased by 28 percent, or $23 million, since first estimates. See
    appendix II for more details on JSF outcomes.

    Funding Needs Could Increase Given the Level of Risk to Complete the JSF Program

    Despite a lengthy restructuring effort that added over $7 billion and 18
    months to the development program, JSF officials have stated that the
    restructured program has little or no flexibility for future changes or
    unanticipated risks. Furthermore, the length and scope of the remaining
    effort make it even more difficult to accurately estimate cost and
    completion schedules. While it has been over 9 years since the program
    started, the first flight test aircraft still has not been delivered. The
    program has planned almost 8 years to complete the remaining activities of
    the system development and demonstration phase. These remaining activities
    include

               o  fully maturing 7 of the 8 critical technologies;

               o  completing the designs and releasing the engineering drawings
               for all three variants;

               o  manufacturing and delivering 15 flight test aircraft and 7
               ground test articles;

               o  developing 19 million lines of software code; and

               o  completing a 7-year, over 12,000-hour flight test program.

    The JSF program's latest planned funding profile for development and
    procurement, produced in December 2004, expects annual funding
    requirements to hover close to $13 billion in between 2012 and 2022,
    peaking at $13.8 billion in 2013. If the program fails to achieve its
    current estimated costs, funding challenges could be even greater than
    they are today. Thus, even a modest cost increase would have dramatic
    impacts on funding. For example, a 10 percent increase in production costs
    would amount to over $21 billion (see fig. 3).

    Figure 3: JSF's Annual Funding Requirements, as of December 2004

    The current cost estimate reflects the position of the JSF program office.
    The Office of Secretary of Defense Cost Analysis Improvement Group was to
    update its formal independent cost estimate in the spring of 2005, and the
    Navy and the Air Force were expected to fully fund the program consistent
    with the estimate. The group now does not expect to formally complete its
    estimate until the spring of 2006, but its preliminary estimate was
    substantially higher than the program office's. According to Cost Analysis
    Improvement Group officials, an assessment of the software and mission
    systems requirements based on more recent information could further
    increase the estimate of JSF costs.

    An Incremental Development Approach Provides an Alternative Model for Reducing
    JSF Risks

    An incremental development approach consistent with DOD's policy on
    evolutionary acquisition and best practices has potential to deliver
    warfighter capabilities in planned product increments that would increase
    JSF capabilities over time. The F-16 fighter program, the Air Force's JSF
    predecessor, successfully evolved capabilities over the span of about 30
    years, delivering increases of capabilities quickly and often, as
    technologies became available. That program may provide a model for a
    possible alternative acquisition strategy for the JSF program. Structuring
    the program into separate and manageable increments based on what is
    achievable now and in the future would allow more predictable cost and
    delivery estimates.

    Over the past three decades, the Air Force successfully procured more than
    2,200 F-16s. The F-16 acquisition approach allowed the timely and
    affordable delivery of aircraft and capability to meet the warfighter's
    needs, including the recapitalization of aging aircraft. By using an
    evolutionary approach to develop the aircraft, the program was able to
    quickly deliver new and improved capabilities to the warfighter and
    increase the aircraft's capability as new technologies were matured and
    added to the aircraft. The first increment, developed during the 1970s,
    provided a "day fighter" aircraft with basic air-to-air and air-to-ground
    capabilities. This allowed the developer to deliver new and useful
    military capability to the warfighter in less than 4 years. With each
    subsequent increment, new technology was used to improve the engine,
    radar, structure, avionics, and other systems that allow the aircraft
    today to perform close air support, ground attack, air defense, and
    suppression of enemy defense missions (see fig. 4). The evolutionary
    approach also enriched the industrial base capabilities by extending the
    life of the production over the length of this incremental approach.

    Figure 4: F-16 Incremental Development Approach

    Note: This gives the number of aircraft delivered to the U.S. Air Force
    only, no foreign military sales or other allied government sales included.

    In contrast, JSF's fully configured design represents a quantum leap in
    capability that far exceeds the capability of legacy systems that JSF is
    intended to replace. While the program is using a block structure-where
    each block adds capabilities over the proceeding block-the blocks are part
    of a single development effort, and DOD is on contract with the developer
    to deliver the warfighter the full capability (see fig. 5). The program's
    block structure provides for an escalating capability, but DOD already
    plans to buy 95 percent of JSF aircraft with the ultimate capability
    (block 3). Unlike the approach used with the F-16, this risky approach
    will likely be like past programs that have encountered significant
    increases in cost and time, not allowing DOD to quickly recapitalize the
    aging legacy aircraft.

    Figure 5: Planned JSF Blocks

    An evolutionary approach would entail delivering a first product increment
    with at least as much capability as currently operating legacy aircraft
    with sufficient quantities to allow DOD to retire older planes sooner and
    reduce operating inefficiencies. DOD has repeatedly raised concerns about
    the age of its fighter aircraft fleet, which was bought largely in the
    1970s and 1980s and will need to be replaced around 2010. Delays in
    fielding JSF aircraft may increase costs to maintain legacy aircraft to
    meet force structure requirements. Limiting development to 5-year
    increments or less, as suggested in the DOD acquisition policy, would
    force smaller, more manageable commitments in capabilities and make costs
    and schedules more predictable. Some of the more challenging JSF
    capabilities, such as the advanced mission systems or prognostics
    technologies, would be deferred, kept in the technology base (off the
    critical path of the JSF program), and added to follow-on increments once
    they are demonstrated in the more conducive technology development
    environment.11 In addition to considering available resources-including
    technology and design knowledge, budget, and time-each increment should be
    based on the warfighter's most immediate needs and the number of aircraft
    needed to maintain a viable fleet. Figure 6 shows what an incremental
    approach might look like for JSF.

    11These technologies are not expected to be fully mature until 2011.

    Figure 6: Potential Application of an Incremental Development Approach to
    the JSF Program

    aThe bar for production is not drawn to scale. JSF production lasts for 20
    years.

    Because an incremental approach would reduce the amount of risk in each
    development phase, it would make the program manager and contractor
    accountable for a lower-risk system development phase with precise cost
    and schedule targets. A shorter system development phase also makes it
    more feasible to align a program manager's tenure to the completion of the
    phase, holding the program manager accountable for decisions made. It also
    would enable the use of fixed-price-type contracts for production that
    contains a pricing structure that reduces government risk.

                                      Conclusions

    While the JSF program plays a critical role in DOD plans to recapitalize
    the services' aging tactical aircraft fleet, DOD's current acquisition
    approach for the JSF program continues to be risky and could further
    jeopardize meeting this important objective. The JSF program has already
    encountered increases to estimated development costs, delays to planned
    deliveries, and reductions in the planned number of JSF to be procured
    that have eroded DOD's buying power. We believe the current acquisition
    strategy to begin production in 2007 is too risky. By committing to
    procure large quantities of the aircraft before completing testing that
    demonstrates that the design is mature and reliable, DOD has significantly
    increased the risk of further compromising its return on investment-as
    well as adding more cost and delaying the delivery of critical
    capabilities to the warfighter. Also, making sizable investments in
    tooling and other manufacturing capability needed to produce JSFs at
    higher rates before a fully integrated aircraft is demonstrated in testing
    places the program at risk for expensive design changes as testing
    uncovers problems. Deferring production decisions until all three
    variants' performance has been demonstrated throughout their flight
    envelopes and with the full integrated capability would allow additional
    time to capture knowledge and help to significantly minimize these risks.

    To continue with the current plan to deliver the ultimate JSF capability
    will require (1) execution of a 7-year test program without further
    schedule delays, (2) development of 5 major blocks of software (19 million
    lines of code), and (3) new and yet undemonstrated technologies for
    advanced mission systems that must collect, analyze, and synthesize
    information from other platforms or sensors not all a part of the JSF
    aircraft. These are necessary to ensure the JSF has increased situational
    awareness to destroy enemy air defenses and perform deep strike missions
    with advanced weapons. The JSF program will also need, annually, an
    average of $11 billion of development and procurement funding for the next
    two decades.

    All of these factors add to the challenges faced by DOD in trying to
    manage this highly concurrent program, increasing the risk it will have
    the same poor outcomes experienced by similar major acquisition programs
    with significant cost and schedule growth and delayed modernization and
    recapitalization of aging systems. DOD has an alternative. With 90 percent
    of DOD's remaining planned investment in JSF, it can adopt a
    knowledge-driven and evolutionary acquisition approach to reduce JSF
    program risks, recapitalize its aging tactical air force sooner, and
    deliver needed capabilities to the warfighter more quickly. The
    experiences of the F-16 program-evolving and improving capabilities over
    time while providing the warfighter combat capability-provide a precedent
    for this. Requirements for the first increment of JSF under this
    evolutionary approach would match a level supported by current knowledge
    of technologies and design. This would allow the testing of those reduced
    requirements to support a knowledge-based low-rate production decision
    sooner and allow delivery of a useful product and in sufficient quantities
    to start replacing the aging legacy fighter force. Capabilities that
    demand as yet undemonstrated technologies, for example, advanced mission
    systems and software, would be included as requirements in subsequent
    future JSF aircraft increments-managed as separate development programs-as
    technology is demonstrated in the technology base and becomes available.

                        Matters for Congressional Consideration

    DOD does not plan to change its business case or acquisition plan for
    developing and buying the JSF. Without changes, the acquisition plan will
    put at risk $50 billion for procuring JSF aircraft at the same time the
    program develops and tests the aircraft's expected performance
    capabilities over a 7-year, 12,000 hour flight test program. The JSF
    program has continually missed its cost and schedule targets over the 5
    years it has been in development. If DOD were to make smaller, more
    manageable commitments in capabilities, it would make cost and schedule
    more predictable and deliver needed capabilities to the war fighter
    sooner. For these reasons, Congress may want to consider delaying
    authorizations and appropriations for procuring JSF aircraft until

               1. DOD develops a knowledge-based business case that matches
               requirements with proven technologies and design knowledge and
               available funding. Capabilities that demand technological advances
               not yet demonstrated should be part of future increments that are
               funded and managed separately once demonstrated.
               2. DOD demonstrates the aircraft design qualities and integrated
               mission capabilities of the fully configured and integrated JSF
               variants work as designed based on actual flight testing.

                          Recommendations for Executive Action

    We are making the following recommendations to reduce program risk and
    increase the likelihood of successful program outcomes by delivering
    capabilities to the warfighter when needed and within available resources.
    Specifically, we recommend that the Secretary of Defense:

               1. Delay the investment in production aircraft until sufficient
               testing has at least demonstrated the basic airframe design of
               each JSF variant in important parts of the flight envelope,
               including combinations of critical altitudes, speeds, and angles
               of attack.
               2. Once the basic airframe design has been demonstrated, limit
               production quantities to the number of aircraft needed for
               operational testing and that can be produced using the current
               production capability (based on existing tooling, materials, and
               personnel) until a fully integrated aircraft (with advanced
               mission systems and predictive maintenance systems) demonstrates
               through flight testing the required capabilities.
               3. Plan an incremental acquisition strategy that follows the
               intent of DOD evolutionary acquisition policy and delivers a first
               increment that limits new content to proven technologies and
               design. The plan should be completed and reported to the Congress
               by July 2006, and include adjustments in associated programs as
               necessary to maintain needed capabilities.

                           Agency Comments and Our Evaluation

    DOD provided us with written comments on a draft of this report. The
    comments appear in appendix III.

    DOD partially concurred with our recommendation that the Secretary of
    Defense delay the investment in production aircraft until sufficient
    testing has at least demonstrated the basic airframe design of each JSF
    variant in important parts of the flight envelope, including combinations
    of critical altitudes, speeds, and angles of attack. DOD agreed that a
    knowledge-based approach is critical to making prudent decisions and that
    this type of approach is being used by JSF. However, DOD's response did
    not provide a plan of action to show how it will ensure flight testing
    demonstrates acceptable performance before significant production
    investments are made. In the past, Congress has raised concerns about the
    costly outcomes of highly concurrent development and production efforts
    that are not "flying before buying." DOD points out that the JSF program
    is in its fifth year of a 12-year development, but starting production
    with nearly 7 years of development and testing remaining leaves a high
    probability that, because it is not flying before buying, it will incur
    costly design changes and delay getting capability to the warfighter.
    Until there has been testing to demonstrate that all three variants will
    work as intended, we believe entering production in 2007 places financial
    investments at significant risk. The risk increases as spending for
    production goes from about $100 million a month in 2007 to over $500
    million a month just 2 years later.

    DOD partially concurred with our recommendation to the Secretary of
    Defense to limit production quantities to the number of aircraft needed
    for operational testing and that can be produced using the current
    production capability until a fully integrated aircraft demonstrates the
    required capabilities through flight testing. DOD stated it is limiting
    production until aircraft capabilities are fully demonstrated and that
    further limits are not necessary. It stated that various program reviews
    to assess performance, including test objectives, are conducted to ensure
    associated program risks are understood and mitigated. We believe DOD's
    plan to invest almost $50 billion to buy over 400 aircraft concurrent with
    testing is very risky. Significant efforts remain in the JSF program to
    demonstrate the aircraft will perform as expected. A number of major DOD
    acquisition programs have employed highly concurrent acquisition
    strategies in the past and experienced significant cost increases and
    schedule delays. DOD stated its use of modeling and simulation and
    laboratory testing reduces risk. However, DOD operational test officials
    have stated that demonstrations need to occur in the actual aircraft in an
    operational environment to verify that the system works as intended. For
    example, the F-22A avionics software performed successfully in the
    laboratory but experienced significant problems that delayed the program
    and increased costs once it entered actual flight testing in the F-22A
    aircraft. JSF software and advanced electronics are more complex than the
    F-22A's. While DOD believes it can manage program risk by holding regular
    program reviews, DOD's own experience has shown this approach is not
    effective. Accepting and managing risk instead of capturing technology,
    design, and manufacturing knowledge, as suggested in DOD policy, has made
    it difficult for DOD to make informed decisions at key points.

    DOD partially concurred with our recommendation to the Secretary of
    Defense to plan an incremental acquisition strategy that follows the
    intent of DOD evolutionary acquisition policy and delivers a first
    increment that limits new content to proven technologies and design and to
    report this plan to Congress by July 2006. It stated the JSF Acquisition
    Strategy fully complies with policy and is a knowledge-based, incremental
    approach that includes three blocks of increasing degrees of warfighter
    capability. We believe DOD's strategy does not provide the benefits of an
    evolutionary approach, as suggested by DOD's policy and best practices. In
    commenting on our report, DOD did not address the salient points
    concerning the true significance of adopting an incremental acquisition
    approach-reducing program risk, delivering needed capabilities to the
    warfighter quicker, and recapitalizing the aging tactical aircraft fleet
    sooner. The JSF strategy resembles other past major acquisition programs
    that have attempted to achieve the ultimate capability in a single
    development increment, producing nearly all of the fleet with the fully
    required capability from the outset. DOD has allowed technology
    development to spill over into product development, weakening the
    foundation for program cost or schedule estimates. This has led to
    disastrous outcomes for other major acquisition programs in the past. We
    continue to believe the successful F-16 program can serve as a model for
    the JSF program. The F-16 program evolved capabilities over a 30-year
    period, buying substantial quantities of each increment in order to
    recapitalize aging tactical aircraft and provide new capabilities to the
    warfighter more quickly. If DOD were to make smaller, more manageable
    commitments in capabilities, it would make cost and schedule more
    predictable and deliver needed capabilities to the warfighter sooner.

    We are sending copies of this report to the Secretary of Defense; the
    Secretaries of the Air Force, Army, and Navy; and the Director of the
    Office of Management and Budget. We will also provide copies to others on
    request. In addition, the report will be available at no charge on the GAO
    Web site at http://www.gao.gov .

    If you or your staff have any questions concerning this report, please
    contact me at (202) 512-4841. Contact points for our offices of
    Congressional Relations and Public Affairs may be found on the last page
    of this report. Other staff making key contributions to this report were
    Michael Hazard, Assistant Director; Lily Chin; Matthew Drerup; Matthew
    Lea; Gary Middleton; Karen Sloan; and Adam Vodraska.

    Michael J. Sullivan Director Acquisition and Sourcing Management

    List of Congressional Committees

    The Honorable John Warner Chairman The Honorable Carl Levin Ranking
    Minority Member Committee on Armed Services United States Senate

    The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye Ranking
    Minority Member Subcommittee on Defense Committee on Appropriations United
    States Senate

    The Honorable Duncan L. Hunter Chairman The Honorable Ike Skelton Ranking
    Minority Member Committee on Armed Services House of Representatives

    The Honorable C.W. Bill Young Chairman The Honorable John P. Murtha
    Ranking Minority Member Subcommittee on Defense Committee on
    Appropriations House of Representatives

    Appendix I: Scope and Methodology Appendix I: Scope and Methodology

    To determine if the Joint Strike Fighter (JSF) program will capture
    critical product knowledge before making production investments, we
    compared the program's plans and results to date against best practice
    standards for applying knowledge to support major program investment
    decisions. The best practice standards are based on a GAO body of work
    that encompasses nearly 10 years and visits to over 25 major commercial
    companies. Our work has shown that valuable lessons can be learned from
    the commercial sector, as well as successful Department of Defense (DOD)
    cases, and can be applied to the development of weapon systems. We
    reviewed JSF management reports, acquisition plans, test plans, risk
    assessments, cost reports, independent program assessments, and program
    status briefings. We identified gaps in product knowledge at the
    production decision points, reasons for these gaps, and the risks to
    program cost and schedule associated with moving forward. We reviewed
    DOD's acquisition policy to determine whether JSF's approach met its
    framework and intent. We interviewed officials from the DOD acquisition
    program management office and prime contractor to gain their perspectives
    on program risks and their approaches to managing risks.

    To evaluate whether the current acquisition plan follows an evolutionary
    or incremental approach, a key best practice for meeting business case
    goals, we examined the JSF program's acquisition framework and the actions
    taken by DOD to address the recommendations made in our 2005 report. We
    compared the JSF program's approach with the approaches used by leading
    companies and successful DOD programs to evolve products to their ultimate
    capabilities. We also reviewed DOD guidance on structuring evolutionary
    acquisition programs. To determine the JSF program outcomes to date, we
    used the program estimates that justified the program when it started in
    1996. This was the point JSF transitioned from a technology development
    environment to an acquisition program environment, with the commitment to
    deliver a family of strike aircraft that meets the Air Force, Navy, and
    Marine Corps needs. At that time, total production, acquisition, and
    ownership costs had not been estimated. The total production, acquisition,
    and ownership cost estimates were first established to support the
    decision to enter the system development and demonstration phase in 2001.
    We used these estimates as the baseline for these costs. We also obtained
    information on past DOD programs from Selected Acquisition Reports and
    prior work conducted by GAO over the past 25 years.

    To identify opportunities to improve JSF program outcomes, we assessed the
    potential of applying an incremental development approach to the JSF
    acquisition program based on the commercial model and DOD guidance. We
    also examined the program history of the F-16 fighter, the JSF
    predecessor, which applied an incremental approach over its 30-year
    acquisition. The F-16 acquisition has been touted by DOD and others as a
    successful program and a model for others to follow. To examine the
    program history of the F-16, we met with F-16 program officials and
    analyzed acquisition plans, management reports, and program outcomes.

    In performing our work, we obtained information and interviewed officials
    from the JSF Joint Program Office, Arlington, Virginia; Lockheed Martin
    Aeronautical Systems, Fort Worth, Texas; Defense Contract Management
    Agency, Fort Worth, Texas; and offices of the Director, Operational Test
    and Evaluation, and Acquisition, Technology and Logistics, Program
    Analysis and Evaluation-Cost Analysis Improvement Group, which are part of
    the Office of Secretary of Defense in Washington, D.C. We also obtained
    information from the F-16 program office and Rand Corporation, Santa
    Monica, California.

    Appendix II: Changes in JSF Program Purchase Quantities and Costs Appendix
    II: Changes in JSF Program Purchase Quantities and Costs

                                                 October 2001             Changes
                                  November 1996       (system     As of     since
                                       (program   development  December   initial
                                         start)        start)      2005  estimate
    Expected quantities
    Development quantities                10            14        15        50
    Procurement quantities             2,978         2,852     2,443      (18)
    (U.S. only)
    Total quantities                   2,988         2,866     2,458      (18)
    Cost estimates (then year dollars in billions)
    Development                        $24.8         $34.4     $45.7        84
    Procurement                Not available        $196.6    $210.7         7
    Other                      Not available          $2.0       $.2      (90)
    Total program acquisition  Not available        $233.0    $256.6        10
    Unit cost estimates (then
    year dollars in millions)
    Average procurement        Not available           $69       $86        25
    Program acquisition        Not available           $81      $104        28
    Unit recurring flyaway costs (fiscal year 2002 dollars in millions)
    Conventional takeoff and           $31.5         $37.0     $44.5        41
    landing
    Short takeoff and             $33.7-39.3         $45.8     $58.7     49-74
    vertical landing
    Carrier                       $34.9-42.7         $47.8     $61.7     44-77

    Source: GAO analysis of DOD data

    Appendix III: Comments from the Department of Defense Appendix III:
    Comments from the Department of Defense

    (120450)

    GAO's Mission

    The Government Accountability Office, the audit, evaluation, and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    Obtaining Copies of GAO Reports and Testimony

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
    released reports, testimony, and correspondence on its Web site. To have
    GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    To Report Fraud, Waste, and Abuse in Federal Programs

    Contact:

    Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
    Automated answering system: (800) 424-5454 or (202) 512-7470

    Congressional Relations

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Washington,
    D.C. 20548

    Public Affairs

    Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S.
    Government Accountability Office, 441 G Street NW, Room 7149 Washington,
    D.C. 20548

    www.gao.gov/cgi-bin/getrpt? GAO-06-356 .

    To view the full product, including the scope

    and methodology, click on the link above.

    For more information, contact Michael J. Sullivan at (202) 512-4841 or
    sullivanm@gao.gov.

    Highlights of GAO-06-356 , a report to congressional committees

    March 2006

    JOINT STRIKE FIGHTER

    DOD Plans to Enter Production before Testing Demonstrates Acceptable
    Performance

    The Joint Strike Fighter (JSF) is DOD's most expensive aircraft program.
    The program represents 90 percent of the remaining planned investment for
    recapitalizing DOD's aging tactical aircraft fleet.

    GAO is required by law to review the program annually for 5 years,
    beginning in fiscal year 2005. This is our second report and GAO assessed
    the program's acquisition approach-in terms of capturing knowledge for key
    investment decisions-and identified an alternative to improve outcomes.

    What GAO Recommends

    The Congress should consider delaying authorizations and appropriations
    for JSF procurement until a new business case is developed and flight
    testing demonstrates the design and integrated mission systems work. GAO
    included this matter for consideration because DOD did not plan to make
    changes as a result of recommendations.

    GAO is recommending that DOD delay investing in production until flight
    testing shows that the JSF performs as expected, and that the program
    develop a plan, consistent with DOD's preferred policy, to adopt an
    evolutionary approach that limits new content for each increment to proven
    technologies and design. DOD partially concurred, but believes that its
    current practices achieve our recommendations' objectives.

    DOD is investing heavily in procuring JSF aircraft before flight testing
    provesit will perform as expected. For example, the JSF program plans to
    produce 424 low-rate initial production aircraft, at a total estimated
    cost of more than $49 billion, by 2013-the same time at which the program
    plans to complete initial operational testing. Producing aircraft before
    testing demonstrates the design is mature increases the likelihood of
    design changes that will lead to cost growth, schedule delays, and
    performance problems. Because the program will lack key design and testing
    knowledge, DOD plans to use cost reimbursement contracts to procure early
    production aircraft. This type of contract places a substantially greater
    cost risk on DOD and the taxpayers. Confidence that investment decisions
    will deliver expected capability within cost and schedule goals increases
    as testing proves the JSF will work as expected.

    Overlap of Production Investments and Testing

    At the same time, the JSF program has not adopted an evolutionary approach
    to acquiring the aircraft-despite DOD policy that prefers such an
    approach. Instead, the JSF program has contracted to develop and deliver
    the aircraft's full capability in a single-step, 12-year development
    program-a daunting task given the need to incorporate the technological
    advances that, according to DOD, represent a quantum leap in capability.
    DOD's buying power has already been reduced. Since initial estimates,
    program acquisition unit costs have increased by 28 percent, or $23
    million. Development costs have increased 84 percent, planned purchases
    have decreased by 535 aircraft, and the completion of development has
    slipped 5 years, delaying delivery of capabilities to the warfighter. With
    more than 90 percent of the JSF investment remaining, DOD officials have
    the opportunity to adopt a knowledge-based and evolutionary acquisition
    strategy that would maximize DOD's return on its investment. The
    acquisition approach used for the F-16 fighter, the Air Force's JSF
    predecessor, could provide a model for delivering JSF capabilities to the
    warfighter sooner and recapitalizing tactical aircraft forces more quickly
    while lowering risk. The F-16 program successfully evolved capabilities
    over the span of about 30 years, with an initial capability delivered to
    the warfighter about 4 years after development started.
    *** End of document. ***