Defense Acquisitions: Assessments of Selected Major Weapon
Programs (31-MAR-06, GAO-06-391).
In the last 5 years, the Department of Defense (DOD) has doubled
its planned investments in new weapon systems from about $700
billion in 2001 to nearly $1.4 trillion in 2006. While the
weapons that DOD develops have no rival in superiority, weapon
systems acquisition remains a long-standing high risk area. GAO's
reviews over the past 30 years have found consistent problems
with weapon acquisitions such as cost increases, schedule delays,
and performance shortfalls. In addition, DOD faces several
budgetary challenges that underscore the need to deliver its new
weapon programs within estimated costs and to obtain the most
from these investments. This report provides congressional and
DOD decision makers with an independent, knowledge-based
assessment of selected defense programs that identifies potential
risks and needed actions when a program's projected attainment of
knowledge diverges from the best practices. Programs for the
assessments were selected based on several factors including, (1)
high dollar value, (2) stage in acquisition, and (3)
congressional interest. The majority of the 52 programs covered
in this report are considered major defense acquisition programs
by DOD. This report also highlights higher level issues raised by
the cumulative experiences of individual programs. GAO updates
this report annually under the Comptroller General's authority.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-391
ACCNO: A50539
TITLE: Defense Acquisitions: Assessments of Selected Major
Weapon Programs
DATE: 03/31/2006
SUBJECT: Best practices
Best practices methodology
Cost analysis
Defense capabilities
Defense cost control
Defense procurement
Financial analysis
Military research and development
Missiles
Performance measures
Program evaluation
Program management
Risk management
Schedule slippages
Strategic planning
Technology
Weapons research and development
Weapons systems
Advanced SEAL Delivery System
Air Force B-2 Radar Modernization
Program
Air Force Evolved Expendable Launch
Vehicle Program
Air Force Small Diameter Bomb
Air Force Transformational Satellite
Communications System
Army Advanced Precision Kill Weapon
System Program
Army Excalibur Precision Guided Extended
Range Artillery Projectile
Army Future Combat Systems
Army Land Warrior System
Army Warfighter Information Network
C-130 Avionics Modernization Program
C-5 Avionics Modernization Program
C-5 Reliability Enhancement and
Reengining Program
CH-47F Helicopter
DD(X) Destroyer
DOD Airborne Laser Program
DOD Space Radar Program
F-22 Raptor Aircraft
Global Hawk Unmanned Aerial Vehicle
Joint Strike Fighter
Joint Tactical Radio System
Joint Tactical Unmanned Aerial Vehicle
Littoral Combat Ship
Longbow Apache Helicopter
Marine Corps Expeditionary Fighting
Vehicle
MDA Aegis Ballistic Missile Defense
Program
MDA Ground-Based Midcourse Defense
Program
MDA Kinetic Energy Interceptor System
MDA Terminal High Altitude Area Defense
System
MQ-9 Predator B
Multi-Mission Maritime Aircraft
National Polar-Orbiting Operational
Environmental Satellite System
NAVSTAR Global Positioning System II
Modernized Space/OCS
Navy Future Aircraft Carrier CVN-21
Navy Mobile User Objective System
Space Based Infrared System-High
V-22 Aircraft
Wideband Gapfiller Satellites
Air Force Advanced Extremely High
Frequency Satellite Program
Navy Active Electronically Scanned Array
Radar Program
Army Advanced Threat Infrared
Countermeasure/Common Missile Warning
System
E-2D Advanced Hawkeye
Atlas Missile
Delta Missile
Patriot Missile System
DOD Space Tracking and Surveillance
System
VH-71A Helicopter
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-391
* Report to Congressional Committees
* March 2006
* DEFENSE ACQUISITIONS
* Assessments of Selected Major Weapon Programs
* Contents
* Fiscal Challenges Confronting DOD Necessitate Better Acquisition
Outcomes
* DOD's Weapon Programs Portfolio Often Experiences a Reduced
Return on Investment
* A Knowledge-Based Approach Can Lead to Better Acquisition
Outcomes
* Most Programs Proceed with Lower Levels of Knowledge at Critical
Junctures
* Historically, Most Cost Growth Is Reported after the Critical
Design Review
* How to Read the Knowledge Graphic for Each Program Assessed
* Assessments of Individual Programs
* Airborne Laser (ABL)
* ABL Program
* Aerial Common Sensor (ACS)
* ACS Program
* Advanced Deployable System (ADS)
* ADS Program
* Aegis Ballistic Missile Defense (Aegis BMD)
* Aegis BMD Program
* Advanced Extremely High Frequency (AEHF) Satellites
* AEHF Program
* Active Electronically Scanned Array Radar (AESA)
* AESA Program
* Advanced Precision Kill Weapon System (APKWS)
* APKWS Program
* Advanced SEAL Delivery System (ASDS)
* ASDS Program
* Advanced Threat Infrared Countermeasure/Common Missile Warning System
* ATIRCM/CMWS Program
* B-2 Radar Modernization Program (B-2 RMP)
* B-2 RMP Program
* C-130 Avionics Modernization Program (C-130 AMP)
* C-130 AMP Program
* C-5 Avionics Modernization Program (C-5 AMP)
* C-5 AMP Program
* C-5 Reliability Enhancement and Reengining Program (C-5 RERP)
* C-5 RERP Program
* CH-47F
* CH-47F Program
* Future Aircraft Carrier CVN-21
* CVN-21 Program
* DD(X) Destroyer
* DD(X) Program
* E-2D Advanced Hawkeye (E-2D AHE)
* E-2D AHE Program
* Evolved Expendable Launch Vehicle (EELV) - Atlas V, Delta IV
* EELV Program
* Expeditionary Fighting Vehicle (EFV)
* EFV Program
* Excalibur Precision Guided Extended Range Artillery Projectile
* Excalibur Program
* F-22A Raptor
* F-22A Program
* Future Combat Systems (FCS)
* FCS Program
* Global Hawk Unmanned Aircraft System
* Global Hawk Program
* Ground-Based Midcourse Defense (GMD)
* GMD Program
* Navstar Global Positioning System (GPS) II Modernized Space/OCS
* GPS Block II Modernization Program
* Joint Land Attack Cruise Missile Defense Elevated Netted Sensor System
* JLENS Program
* Joint Strike Fighter (JSF)
* JSF Program
* Joint Tactical Radio System Airborne, Maritime, Fixed-Site (JTRS AMF)
* JTRS AMF Program
* Joint Tactical Radio System (JTRS) Cluster 1
* JTRS Cluster 1 Program
* Joint Tactical Radio System (JTRS) Cluster 5
* JTRS Cluster 5 Program
* Joint Unmanned Combat Air Systems (J-UCAS)
* J-UCAS Program
* Kinetic Energy Interceptors (KEI)
* KEI Program
* Land Warrior
* Land Warrior Program
* Littoral Combat Ship (LCS)
* LCS Program
* Longbow Apache Block III
* Longbow Apache BLIII Program
* Multi-mission Maritime Aircraft (MMA)
* MMA Program
* 21" Mission Reconfigurable Unmanned Undersea Vehicle (MRUUV)
* 21'' MRUUV Program
* Mobile User Objective System (MUOS)
* MUOS Program
* National Polar-orbiting Operational Environmental Satellite System
(NPOESS)
* NPOESS Program
* PATRIOT/MEADS Combined Aggregate Program (CAP) Fire Unit
* PATRIOT/MEADS CAP Fire Unit Program
* MQ-9 Predator B
* Predator B Program
* Space Based Infrared System (SBIRS) High
* SBIRS High Program
* Small Diameter Bomb (SDB)
* SDB Program
* Space Radar (SR)
* SR Program
* Space Tracking and Surveillance System (STSS)
* STSS Program
* Terminal High Altitude Area Defense (THAAD)
* THAAD Program
* Transformational Satellite Communications System (TSAT)
* TSAT Program
* V-22 Joint Services Advanced Vertical Lift Aircraft
* V-22 Program
* VH-71A Presidential Helicopter Replacement Program
* VH-71A Program
* Warrior Unmanned Aerial Vehicle (Warrior UAV)
* Warrior UAV Program
* Wideband Gapfiller Satellites (WGS)
* WGS Program
* Warfighter Information Network-Tactical (WIN-T)
* WIN-T Program
* Agency Comments
* Scope of Our Review
* Comments from the Department of Defense
* Scope and Methodology
* Macro Analysis
* Historical Analysis
* System Profile Data on Each Individual Two- Page Assessment
* Product Knowledge Data on Each Individual Two-Page Assessment
* Technology Readiness Levels
* GAO Contact and Acknowledgments
* GAO Contact
* Acknowledgments
* Related GAO Products
United States Government Accountability Office
GAO
March 2006
DEFENSE ACQUISITIONS
Assessments of Selected Major Weapon Programs
DEFENSE ACQUISITIONS
Assessments of Selected Major Weapon Programs
What GAO Found
GAO assessed 52 systems that represent an investment of over $850 billion,
ranging from the Missile Defense Agency's Airborne Laser to the Army's
Warfighter Information Network-Tactical. DOD often exceeds development
cost estimates by approximately 30 to 40 percent and experiences cuts in
planned quantities, missed deadlines, and performance shortfalls. Such
difficulties, absent definitive and effective reform outcomes, are likely
to cause great turmoil in a budget environment in which there are growing
fiscal imbalances as well as increasing conflict over increasingly limited
resources. While these problems are in themselves complex, they are
heightened by the fact that this current level of investment is by no
means final and unchangeable. A large number of the technologies under
development in these systems are sufficiently new and immature that it is
uncertain how long it will take or how much it will cost to make them
operational.
Most of the 52 programs GAO reviewed have proceeded with lower levels of
knowledge than suggested by best practices. Programs that start with
mature technologies do better. As shown in the figure below, programs that
began with immature technologies have experienced average research and
development cost growth of 34.9 percent; programs that began with mature
technologies have only experienced cost growth of 4.8 percent.
Average Program Research, Development, Test, and Evaluation Cost Growth
from First Full Estimate
Source: GAO analysis.
If DOD continues to move programs through development without requisite
technology, design, and production knowledge, costs and schedules will
increase, which will reduce the quantity delivered to the warfighter. This
practice will also continue to reduce DOD's buying power, as less
capability will be provided for the money invested. In the larger context,
DOD needs to make changes in its requirements and budgeting processes that
are consistent with getting the desired outcomes from the acquisition
process.
Contents
Forword
Letter 3
Fiscal Challenges Confronting DOD Necessitate Better Acquisition Outcomes
4
DOD's Weapon Programs Portfolio Often Experiences a Reduced Return on
Investment 6
A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes 8
Most Programs Proceed with Lower Levels of Knowledge at Critical Junctures
10
Historically, Most Cost Growth Is Reported after the Critical Design
Review 13
How to Read the Knowledge Graphic for Each Program Assessed 16
Assessments of Individual Programs 18
Airborne Laser (ABL) 19
Aerial Common Sensor (ACS) 21
Advanced Deployable System (ADS) 23
Aegis Ballistic Missile Defense (Aegis BMD) 25
Advanced Extremely High Frequency (AEHF) Satellites 27
Active Electronically Scanned Array Radar (AESA) 29
Advanced Precision Kill Weapon System (APKWS) 31
Advanced SEAL Delivery System (ASDS) 33
Advanced Threat Infrared Countermeasure/Common Missile Warning System 35
B-2 Radar Modernization Program (B-2 RMP) 37
C-130 Avionics Modernization Program (C-130 AMP) 39
C-5 Avionics Modernization Program (C-5 AMP) 41
C-5 Reliability Enhancement and Reengining Program (C-5 RERP) 43
CH-47F 45
Future Aircraft Carrier CVN-21 47
DD(X) Destroyer 49
E-2D Advanced Hawkeye (E-2D AHE) 51
Evolved Expendable Launch Vehicle (EELV) - Atlas V,
Delta IV 53
Expeditionary Fighting Vehicle (EFV) 55
Excalibur Precision Guided Extended Range Artillery Projectile 57
F-22A Raptor 59
Future Combat Systems (FCS) 61
Global Hawk Unmanned Aircraft System 63
Ground-Based Midcourse Defense (GMD) 65
Navstar Global Positioning System (GPS) II Modernized Space/OCS 67
Joint Land Attack Cruise Missile Defense Elevated Netted Sensor System
(JLENS) 69
Joint Strike Fighter (JSF) 71
Joint Tactical Radio System Airborne, Maritime, Fixed-Site (JTRS AMF) 73
Joint Tactical Radio System (JTRS) Cluster 1 75
Joint Tactical Radio System (JTRS) Cluster 5 77
Joint Unmanned Combat Air Systems (J-UCAS) 79
Kinetic Energy Interceptors (KEI) 81
Land Warrior 83
Littoral Combat Ship (LCS) 85
Longbow Apache Block III 87
Multi-mission Maritime Aircraft (MMA) 89
21" Mission Reconfigurable Unmanned Undersea Vehicle (MRUUV) 91
Mobile User Objective System (MUOS) 93
National Polar-orbiting Operational Environmental Satellite System
(NPOESS) 95
PATRIOT/MEADS Combined Aggregate Program (CAP) Fire Unit 97
MQ-9 Predator B 99
Space Based Infrared System (SBIRS) High 101
Small Diameter Bomb (SDB) 103
Space Radar (SR) 105
Space Tracking and Surveillance System (STSS) 107
Terminal High Altitude Area Defense (THAAD) 109
Contents
Contents
Appendixes
Appendix I: Appendix II:
Appendix III: Appendix IV:
Transformational Satellite Communications System (TSAT) 111 V-22 Joint
Services Advanced Vertical Lift Aircraft 113 VH-71A Presidential
Helicopter Replacement Program 115 Warrior Unmanned Aerial Vehicle
(Warrior UAV) 117 Wideband Gapfiller Satellites (WGS) 119 Warfighter
Information Network-Tactical (WIN-T) 121 Agency Comments 123 Scope of Our
Review 123
Comments from the Department of Defense 126
Scope and Methodology 127 Macro Analysis 127 Historical Analysis 128
System Profile Data on Each Individual Two-Page
Assessment 130 Product Knowledge Data on Each Individual Two-Page
Assessment 132
Technology Readiness Levels 134
GAO Contact and Acknowledgments 136 GAO Contact 136 Acknowledgments 136
Related GAO Products
Table 1: Total Projected Cost of DOD's Top Five Programs in Fiscal
Tables
Years 2001 and 2006 5 Table 2: Cost and Cycle Time Growth for 26 Weapon
Systems 6 Table 3: Examples of DOD Programs with Reduced Buying
Power 8 Table 4: Programs in Our Assessment Yet to Hold a Critical Design
Review 15
Figure 1: Percent of Programs That Achieved Technology Maturity
Figures
at Key Junctures
Contents Contents
Figure 2: RDT&E Percentage Increase throughout the Product
Development Cycle for 29 Programs Completed or in
Production 14
Figure 3: Depiction of a Notional Weapon System's Knowledge as
Compared with Best Practices 17
Abbreviations
AMRDEC Aviation and Missile Research Development and
Engineering Center
ARS analysis and reporting system
ATACMS BAT Army Tactical Missile System Brilliant Antiarmor
Submunition
BAMS Broad Area Maritime Surveillance
CDA Commander's Digital Assistant
CEC Cooperative Engagement Capability
CMUP Conventional Mission Upgrade Program
DARPA Defense Advanced Research Projects Agency
DBCS Dismounted Battle Command System
DOD Department of Defense
EKV exoatmospheric kill vehicle
EPLRS Enhanced Position Location Reporting System
FY fiscal year
GAO Government Accountability Office
GEO geosynchronous earth orbit
GMLRS Guided Multiple Launch Rocket System
HEO highly elliptical orbit
ICD Interface Control Design
ISR intelligence, surveillance, and reconnaissance
JASSM Joint Air-to-Surface Standoff Missile
JDAM Joint Direct Attack Munition
JLENS Joint Land Attack Cruise Missile Defense Elevated
Netted Sensor System
JPATS Joint Primary Aircraft Training System
JSOW Joint Standoff Weapon
MCS Maneuver Control System
MDA Missile Defense Agency
MDAP Major Defense Acquisition Program
MEADS Medium Extended Air Defense System
MIDS-LVT Multifunctional Information Distribution
System - Low Volume Terminal
MM III GRP Minuteman III Guidance Replacement Program
NA not applicable
NAS National Airspace System
NASA National Aeronautics and Space Administration
NOAA National Oceanic and Atmospheric Administration
OCS Operational Control System
OSD Office of the Secretary of Defense
Contents
PTIR precision track illumination radar
RDT&E Research, Development, Test and Evaluation
SAR Selected Acquisition Report
SDACS Solid Divert and Attitude Control System
SDR Systems Design Review
SOCOM Special Operations Command
SPC statistical process control
SUR surveillance radar
TBD to be determined
TBIP Tomahawk Baseline Improvement Program
TRL Technology Readiness Level
UAV Unmanned Aerial Vehicle
UHF ultra high frequency
U.S.C. United States Code
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, D.C. 20548
March 31, 2006
Congressional Committees
Current and expected fiscal imbalances demand that the Department of
Defense (DOD) maximize its return on investment and provide the warfighter
with needed capabilities at the best value for the taxpayer. Since 1990,
we have assessed weapon acquisitions as a high-risk area. Not only does it
continue to be a high risk area, but it has also taken on heightened
importance. To transform military operations, DOD has embarked on
developing multiple megasystems that are expected to be the most expensive
and complex ever. However, these costly acquisitions are running head-on
into the nation's unsustainable fiscal path. In the past 5 years, DOD has
doubled its planned investments in new weapon systems from $700 billion to
$1.4 trillion. This huge increase has not been accompanied by more
stability, better outcomes, or more buying power for the acquisition
dollar.
This is our fourth annual assessment of weapon programs. It contains our
assessment of 52 weapon programs representing a projected investment of
about $850 billion. Unfortunately, our assessments do not show appreciable
improvement in the acquisition of major weapon systems. Rather, programs
are experiencing recurring problems with cost overruns, missed deadlines,
and performance shortfalls. These cost increases mean that DOD cannot
produce as many weapons as intended nor deliver those weapons to the
warfighter when promised. These problems occur, in part, because weapon
programs do not capture the requisite knowledge when needed to efficiently
and effectively manage program risks. Programs consistently move forward
with unrealistic cost and schedule estimates, use immature technologies in
launching product development, and fail to solidify design and
manufacturing processes at appropriate points in development.
The past year has seen several major defense reviews that lay down
approaches to improve the way DOD buys weapons. These reviews contain many
constructive ideas. If they are to produce better results, however, they
must heed the lessons taught-but perhaps not learned-of acquisition
history. Specifically, policy must be manifested in decisions on
individual programs or reform will be blunted. DOD's current acquisition
policy is a case in point. The policy supports a knowledge-based,
evolutionary approach to acquiring new weapons. The practice-decisions
made on individual programs-sacrifices knowledge and executability in
favor of revolutionary solutions. It's time to challenge such solutions.
Reform will not be real unless each weapon system is shown to be both a
worthwhile investment and an executable program. Otherwise, we will
continue to start more programs than we can finish, produce less
capability for more money, and create the next set of case studies for
future defense reform reviews.
David M. Walker Comptroller General of the United States
A
United States Government Accountability Office Washington, D.C. 20548
March 31, 2006
Congressional Committees
One of the single largest investments the federal government makes is the
development and production of new weapon systems. In the last 5 years, the
Department of Defense (DOD) has doubled its planned investments in new
weapon systems from about $700 billion in 2001 to nearly $1.4 trillion in
2006. It is imperative that these investments deliver as promised not only
because of their value to the warfighter but because every dollar spent on
weapon systems means one dollar less of something else DOD or the
Government can do. There is ample basis for serious concerns on this
score. The cost of developing a weapon system continues to often exceed
estimates by approximately 30 percent to 40 percent. This in turn results
in fewer quantities, missed deadlines, and performance shortfalls. In
short, the buying power of the weapon system investment dollar is reduced;
the warfighter gets less than promised; and opportunities to make other
investments are lost. This is not to say that the nation does not get
superior weapons in the end, but that at twice the level of investment,
DOD has an obligation to get better results. In the larger context, DOD
needs to make changes in its requirements and budgeting processes that are
consistent with getting the desired outcomes from the acquisition process.
Given growing fiscal imbalances as well as competition for increasingly
scarce resources, this current level of investment is by no means final
and unchangeable. To get better results, programs need to have higher
levels of knowledge when they start, which enable better estimates of how
much they will cost to finish. Currently, a large number of the
technologies under development in major systems are sufficiently new and
immature that it is uncertain how long it will take or how much it will
cost to make them operational. Predictably, developing these systems
without sufficient knowledge will take longer and cost even more than
promised and deliver fewer quantities and other capabilities than planned.
Over the years, we have made a number of recommendations to address these
issues, both systemically and on individual programs.
In this report, we assess 52 programs that represent an investment of
approximately $858 billion.1 Our objective is twofold: to provide decision
makers with a cross-cutting analysis of DOD weapons system investment and
also to provide independent, knowledge-based assessments of individual
systems' attained knowledge and potential risks.
Fiscal Challenges Confronting DOD Necessitate Better Acquisition Outcomes
Programs were selected for individual assessment based on several factors
including, (1) high dollar value, (2) stage in acquisition, and (3)
congressional interest. The majority of the 52 programs covered in this
report are considered major defense acquisition programs by DOD.
DOD's investment in the research, development, test, and evaluation
(RDT&E) and procurement of major weapon systems is expected to rise from
$147 billion in fiscal year 2006 to $178 billion in fiscal year 2011.2
DOD's total planned investment in Major Defense Acquisition Programs is
nearly $1.4 trillion (2006 dollars) for its current portfolio, with over
$840 billion of that investment yet to be made.3
Budget simulations by GAO, the Congressional Budget Office, and others
show that, over the long term, we face a large and growing structural
deficit due primarily to known demographic trends and rising health care
costs. As the Comptroller General has noted, continuing on this
unsustainable fiscal path will gradually erode, if not suddenly damage,
our economy, our standard of living, and ultimately our national security.
Federal discretionary spending, along with other federal policies and
programs, will face serious budget pressures in the coming years stemming
from new budgetary demands and demographic trends. Defense spending falls
within the discretionary spending accounts. Further, current military
operations, such as those in Afghanistan and Iraq, consume a large share
of DOD budgets and are causing faster wear on existing weapons.
Refurbishment or replacement sooner than planned is putting further
pressure on DOD's investment accounts.
At the same time DOD is facing these problems, programs are commanding
larger budgets. DOD is undertaking new efforts that are expected to be the
1 This estimate includes total RDT&E; procurement; military construction;
and acquisition operation and maintenance appropriations to develop the
weapon systems. The macro analyses contained in this report are based on
data as of January 15, 2006, and may not reflect subsequent events. For
example, the Joint Tactical Radio System programs are currently being
restructured.
2 Estimates in then-year dollars as reported in the Fiscal Year 2006
Department of Defense Future Years Defense Program Table 1-1 for RDT&E and
Procurement.
3 This estimate is for Major Defense Acquisition Programs (MDAPs). MDAPs
are programs identified by DOD as programs that require eventual RDT&E
expenditures of more than $365 million or $2.19 billion in procurement in
fiscal year 2000 constant dollars.
most expensive and complex ever and on which DOD is heavily relying to
fundamentally transform military operations. Table 1 shows that just 5
years ago, the top five weapon systems were projected to cost about $291
billion combined; today, the top five weapon systems are projected to cost
about $550 billion.
Table 1: Total Projected Cost of DOD's Top Five Programs in Fiscal Years
2001 and 2006
Billions of constant 2006 dollars
2001 2006
Program Cost Program Cost
F-22A Raptor aircraft $65.0 Joint Strike Fighter aircraft $206.3
DDG-51 class destroyer $64.4 Future Combat Systems $127.5
ship
Virginia class $62.1 Virginia class submarine $80.4
submarine
C-17 Globemaster airlift $51.1 DDG-51 class destroyer ship $70.4
aircraft
F/A-18E/F Super Hornet $48.2 F-22A Raptor aircraft $65.4
fighter
aircraft
Total $290.8 Total $550.0
Source: GAO analysis of DOD data.
The larger scope of development associated with these megasystems produces
a much larger fiscal impact when cost and schedule estimates increase. The
top 5 programs in 2001 and the top 5 programs in 2006 have both
experienced about a 40 percent increase in projected RDT&E costs from the
first full estimate to the latest estimate. In the same base-year dollars,
the total fiscal impact was much greater for the 2006 top 5 programs,
however, as RDT&E costs increased by $33.9 billion as opposed to $16.9
billion for the top 5 from 2001 because of the larger scope of development
planned for the 2006 top 5 programs. The Joint Strike Fighter and Future
Combat Systems contribute significantly to this projected cost growth, as
their combined cost is greater than all of the top 5 programs in 2001.
DOD's Weapon Programs Portfolio Often Experiences a Reduced Return on
Investment
The way DOD develops and produces its major weapon systems has had
disappointing consequences. A large number of the programs in our
assessment are costing more and taking longer to develop than estimated.
As shown in table 2, total RDT&E costs for 26 common set4 weapon programs
increased by nearly $44.6 billion, or 37 percent, over the original
business case (the first full estimate). The same programs have also
experienced an increase in the time needed to develop capabilities with a
weighted-average schedule increase of nearly 17 percent.5
Table 2: Cost and Cycle Time Growth for 26 Weapon Systems
Billions of constant 2006 dollars
First full estimate Latest estimate Percentage change
Total cost $547.7 $627.4 14.6
RDT&E cost $120.4 $164.9 37.0
Weighted average 154.5 months 180.2 months 16.7
acquisition cycle timea
Source: GAO analysis of DOD data.
aThis is a weighted estimate of average acquisition cycle time for the 26
programs based on total program costs at the first full and latest
estimates. The simple average for these two estimates was 112.1 months for
the first full estimate and 131.3 months for the latest estimate resulting
in a 17.2 percent change.
4 The common set refers to the 26 programs that we were able to assess
since development began. The 26 programs are ACS, AEHF, AESA, APKWS, B-2
RMP, C-5 AMP, C-5 RERP, CH47F, CVN-21, E-2D AHE, EFV, Excalibur, F-22A,
FCS, Global Hawk, JSF, JTRS Cluster 5, Land Warrior, MMA, MUOS, NPOESS,
Patriot/MEADS CAP, Predator B, SDB, V-22, and WGS. We limited this
analysis to these 26 programs because all data including cost, schedule,
and quantities were available for comparison between program estimates.
The data in table 2 does not represent the same common set of 26 programs
reported in the 2005 assessment. GAO, Defense Acquisitions: Assessments of
Selected Major Weapon Programs, GAO-05-301 (Washington, D.C.: Mar. 31,
2005).
5 A weighted average gives more expensive programs a greater value.
Quantities for 9 of the common set programs have been reduced since their
first estimate.6 In addition, the weighted-average program acquisition
unit cost for 25 of the 26 programs increased by roughly 57 percent.7
The consequence of cost and cycle-time growth is manifested in a reduction
of the buying power of the defense dollar. Table 3 illustrates six
programs included in this assessment with a significant reduction in
buying power; we have reported similar outcomes in many more programs. For
example, the Air Force initially planned to buy five Spaced Based Infrared
System High satellites at a program acquisition unit cost of about $816
million (fiscal year 2006 dollars). Technology and design components
matured late in the development of the satellite, which contributed to
cost growth and four Nunn-McCurdy8 unit cost breaches. Now, the Air Force
plans to buy 3 satellites at a program acquisition unit cost of about $3.4
billion, a 315 percent increase.
6 The 9 programs are AEHF, Excalibur, APKWS, V-22, JSF, C-5 RERP, F-22A,
Global Hawk, and C-5 AMP.
7 This estimate is a weighted average based on total program cost and does
not include the Excalibur program because of its extreme unit cost growth.
The simple average program unit cost increase for the same 25 programs is
36 percent. The weighted average, including the Excalibur, is 62 percent.
8 10 U.S.C S: 2433. Requires DOD to (1) notify Congress whenever unit cost
growth is at least 15 percent, and (2) "certify" the program to Congress
when unit cost growth is at least 25 percent above the latest approved
acquisition baseline cost estimate.
Table 3: Examples of DOD Programs with Reduced Buying Power
Percent
Program Initial Initial Latest Latest of unit
estimate quantity estimate quantity cost
increase
Joint Strike $189. 2,866 $206.3billion 2,458aircraft 26.7
Fighter 8billion aircraft
Future Combat $82.6 15 systems $127.5 15 systems 54.4
Systems billion billion
F-22A Raptor $81.1 648aircraft $65.4 billion 181 aircraft 188.7
billion
Evolved
Expendable $15.4 181 vehicles $28.0 billion 138 vehicles 137.8
Launch billion
Vehicle
Space Based $4.1
Infrared billion 5 satellites $10.2 billion 3satellites 315.4
System High
Expeditionary $8.1 1,025 1,025
Fighting billion vehicles $11.1 billion vehicles 35.9
Vehicle
Source: GAO analysis of DOD data. Imagessourced in their respecitve order:
JSF Program Office; Program Manager, Unit of Action,
U.S. Army; F-22A System Program Office; (Left) (c) 2003 ILS/Lockheed
Martin, (right) (c) 2003 The Boeing Company; Lockheed Martin Space Systems
Company; General Dynamics Land Systems.
A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes
Over the last several years, we have undertaken a body of work that
examines weapon acquisition issues from a perspective that draws upon
lessons learned from best product development practices. Leading
commercial firms expect that their program managers will deliver
highquality products on time and within budget. Doing otherwise could
result in the customer walking away. Thus, those firms have created an
environment and adopted practices that put their program managers in a
good position to succeed in meeting these expectations. Collectively,
these practices comprise a process that is anchored in knowledge. It is a
process in which technology development and product development are
treated differently and managed separately. The process of developing
technology culminates in discovery-the gathering of knowledge-and must, by
its nature, allow room for unexpected results and delays. Leading firms do
not ask their product managers to develop technology. Successful programs
give responsibility for maturing technologies to science and technology
organizations, rather than the program or product development managers.
The process of developing a product culminates in delivery and, therefore,
gives great weight to design and production. The firms demand-and
receive-specific knowledge about a new product before production begins. A
program does not go forward unless a strong business case on which the
program was originally justified continues to hold true.
Successful product developers ensure a high level of knowledge is achieved
at key junctures in development. We characterize these junctures as
knowledge points. These knowledge points and associated indicators are
defined as follows:
o Knowledge point 1: Resources and needs match. This point occurs when a
sound business case is made for the product-that is, a match is made
between the customer's requirements and the product developer's
available resources in terms of knowledge, time, money, and capacity.
Achieving a high level of technology maturity at the start of system
development is an important indicator of whether this match has been
made. This means that the technologies needed to meet essential
product requirements have been demonstrated to work in their intended
environment.
o Knowledge point 2: Product design is stable. This point occurs when a
program determines that a product's design is stable-that is, it will
meet customer requirements, as well as cost, schedule, and reliability
targets. A best practice is to achieve design stability at the
system-level critical design review, usually held midway through
development. Completion of at least 90 percent of engineering drawings
at the system design review provides tangible evidence that the design
is stable.
o Knowledge point 3: Production processes are mature and the design is
reliable. This point is achieved when it has been demonstrated that
the company can manufacture the product within cost, schedule, and
quality targets. A best practice is to ensure that all key
manufacturing processes are in statistical control-that is, they are
repeatable,
Most Programs Proceed with Lower Levels of Knowledge at Critical Junctures
sustainable, and capable of consistently producing parts within the
product's quality tolerances and standards-at the start of production.
A result of this knowledge-based process is evolutionary product
development, an incremental approach that enables developers to rely more
on available resources rather than making promises about unproven
technologies. Predictability is a key to success as successful product
developers know that invention cannot be scheduled, and its cost is
difficult to estimate. They do not bring a technology into new product
development unless that technology has been demonstrated to meet the
user's requirements. Allowing technology development to spill over into
product development puts an extra burden on decision makers and provides a
weak foundation for making product development estimates. While the user
may not initially receive the ultimate capability under this approach, the
initial product is available sooner and at a lower, more predictable cost.
There is a synergy in this process, as the attainment of each successive
knowledge point builds on the preceding one. Metrics gauge when the
requisite level of knowledge has been attained. Controls are used to
ensure a high level of knowledge is attained before making additional
significant investments. Controls are considered effective if they are
backed by measurable criteria and if decision makers are required to
consider them before deciding to advance a program to the next level.
Effective controls help decision makers gauge progress in meeting cost,
schedule, and performance goals and ensure that managers will (1) conduct
activities to capture relevant product development knowledge, (2) provide
evidence that knowledge was captured, and (3) hold decision reviews to
determine that appropriate knowledge was captured to move to the next
phase. The result is a product development process that holds decision
makers accountable and delivers excellent results in a predictable manner.
To get the most out of its weapon systems investments, DOD revised its
acquisition policy in May 2003 to incorporate a knowledge-based,
evolutionary framework. The policy requires decision makers to have the
knowledge they need before moving to the next phase of development.
However, most of the programs we reviewed proceeded with lower levels of
knowledge at critical junctures and attained key elements of product
knowledge later in development than specified in DOD policy. Once a
program gets behind in demonstrated knowledge, it stays behind (see fig.
1).
Figure 1: Percent of Programs That Achieved Technology Maturity at Key Junctures
100
75
67
50
43
25
10
0 Development DOD Production
start design decision
review
Source: GAO analysis of DOD data.
Only 10 percent of the programs in our assessment demonstrated all of
their critical technologies as mature at the start of development, meaning
they fell far short of attaining knowledge point 1 when they should have.
By the time of their design review-when they should have demonstrated
knowledge point 2 (stable design)-only 43 percent had actually attained
knowledge point 1 (all critical technologies mature). By the time of the
decision to start production when the programs should have demonstrated
knowledge point 3 (production processes in control) one third still had
not attained knowledge point 1. Similarly, only 35 percent of the programs
in our assessment believed they had attained knowledge point 2 at the
design review and only 58 percent believed they had attained knowledge
point 2 by the time of the decision to start production. None of the
programs we assessed that are now in production reported using statistical
process control data to measure the maturity of production processes. This
is the data needed to demonstrate knowledge point 3. In other words, none
of the programs demonstrated knowledge point 3. This suggests that
programs that follow the policy are the exception; the predominant
practice is to still proceed with knowledge gaps.
Consequences accrue to programs that are still working to mature
technologies well into system development when they should be focused on
maturing system design and preparing for production. These consequences
involve increased risk of cost growth and schedule delays throughout the
life of the program. The cost effect of proceeding without the necessary
knowledge can be dramatic. For example, RDT&E costs for the programs that
started development with mature technologies increased by a modest average
of 4.8 percent over the first full estimate, whereas the RDT&E costs for
the programs that started development with immature technologies increased
by a much higher average of 34.9 percent over the first full estimate.
Likewise, program acquisition unit costs for the programs with mature
technology increased by less than 1 percent, whereas the programs that
started development with immature technologies experienced an average
program acquisition unit cost increase of nearly 27 percent over the first
full estimate.9
In commenting on a draft of this report, DOD stated that it is the
department's policy that technologies should be demonstrated in at least a
relevant environment before a program enters system development; whereas,
GAO utilizes the best practice standard that calls for technologies to be
assessed one step higher-demonstration in an operational environment. If
we applied the DOD's lower standard, the number of programs with mature
technologies at program start would have increased to 23 percent, compared
with 10 percent using the best practices standard. This is a higher number
but does not alter the fact that most programs begin development without
mature technology. A cost consequence for using the lower standard does
occur, however. While the RDT&E cost growth for programs that started
development with immature technologies (using the DOD standard) was about
the same at 34.6 percent, the cost growth for the programs that met DOD's
maturity standard was significantly greater at 18.8 percent than the 4.8
percent experienced by those that met the higher best practice standard.
9 These percentages are program cost weighted averages. The simple average
increase for program acquisition unit costs is 2.8 percent for the
programs that started development with mature technologies and 19.8
percent for the programs that started development with immature
technologies.
Page 12 GAO-06-391 Assessments of Selected Major Weapon Programs
Historically, Most Cost Growth Is Reported after the Critical Design Review
The order of how knowledge is built throughout product development is
important to delivering products on time and within cost. Knowledge gaps
have a cumulative effect. For example, design stability cannot be attained
if key technologies are not mature. The lack of technical maturity weakens
the knowledge available at the design review. The majority of programs in
our assessment that have held a design review did so without first
maturing critical technologies. Twenty of the 52 programs we assessed are
currently scheduled to hold their critical design reviews by the year
2011. Only 2 of those 20 programs currently expect to have their
technologies fully mature by the time of their design reviews, and only 4
of those 20 programs currently expect to have at least 90 percent design
stability by the time of their critical design reviews.
We reviewed the development cost experience of 29 programs that have
completed their product development cycle--the time between the start of
development and production.10 We found a significant portion of the
recognized total development cost increases of these programs took place
after they were approximately half way into their product development
cycle. These increases typically occurred after the time of the design
review of the programs. As shown in figure 2, the programs experienced a
cumulative increase in development costs of 28.3 percent throughout their
product development. Approximately 8.5 percent of the total development
cost growth occurred up until the time of the average critical design
review. The remaining 19.7 percent occurred after the average critical
design review.
10 The 29 programs include: ATIRCM/CMWS, AEHF, AESA Radar, AIM-9X/Air to
Air Missile, ATACMS BAT, B-1B CMUP, Bradley Fighting Vehicle A3 Upgrade,
CH-47F, CEC, EELV, F/A-18E/F, F-22A, GMLRS Tactical Rocket, JASSM, JDAM,
JPATS, JSOW, Longbow Hellfire, M1A2 Abrams, MCS, MM III GRP, MIDS-LVT,
NAS, SDB, Strategic Sealift, Stryker Family of Vehicles, Tactical
Tomahawk, Tomahawk TBIP, and V-22. The average design review is based on
21 of the 29 programs that either reported a critical design review date
in the annual Selected Acquisition Reports or was provided to us by
program officials.
Figure 2: RDT&E Percentage Increase throughout the Product Development
Cycle for 29 Programs Completed or in Production
Percentage of RDT&E cost increase over development estimate 30
Total increase 28.3%
25
20
15
10
Design review
5
0 0 10 2030 40 50 60 7080 90 100 Percentage of product development completed
Source: GAO analysis of DOD data.
This historical pattern underscores the challenges DOD faces in executing
programs currently in development. Table 4 lists the programs in our
assessment that have yet to hold their critical design review.11
11 Data as of January 15, 2006.
Table 4: Programs in Our Assessment Yet to Hold a Critical Design Review
Aerial Common Sensor
Advanced Deployable System
Advanced Precision Kill Weapon System
C-130 Avionics Modernization Program
Future Aircraft Carrier CVN-21
Future Combat Systems
Joint Land Attack Cruise Missile Defense Elevated Netted Sensor System
F-35 Joint Strike Fighter
Joint Tactical Radio System Cluster 5
Patroit/Medium Extended Air Defense System Combined Aggregated Program
Multi-mission Maritime Aircraft
Mobile User Objective System
National Polar-Orbiting Operational Environmental Satellite System
MQ-9 Predator B
Warrior UAV
Warfighter Information Network - Tactical
Source: GAO analysis of DOD data.
Note: List includes only those programs that have started development.
Four additional programs in our assessment have scheduled their critical
design review but have not yet started development.
The current planned total RDT&E investment of these 16 programs is
approximately $142 billion with a total planned investment of over $521
billion. While most of these programs have yet to experience any
significant cost increases, some have already experienced double digit
cost increases prior to their design review. Furthermore, all 16 programs
listed began development with immature technologies-10 currently still
have over half of their critical technologies immature. For these
programs, the markers for risk are present-historical experience and
technology immaturity-as are the cost, schedule, and quantity consequences
that attend that risk. If past is prologue, the decisions to continue to
move programs through development without the requisite knowledge will
continue to result in programs that are not delivered on time nor with the
quantities and capabilities promised. These consequences are exacerbated
in an environment of constrained resources as trade-offs become necessary
not only within these programs, but across the entire weapons portfolio-
resulting in a reduction of the department's buying power.
How to Read the Knowledge Graphic for Each Program Assessed
We assess each program in two pages and depict the extent of knowledge in
a stacked bar graph and provide a narrative summary at the bottom of the
first page. As illustrated in figure 3, the knowledge graph is based on
the three knowledge points and the key indicators for the attainment of
knowledge: technology maturity (depicted in orange), design stability
(depicted in green), and production maturity (depicted in blue). A "best
practice" line is drawn based on the ideal attainment of the three types
of knowledge at the three knowledge points. The closer a program's
attained knowledge is to the best practice line, the more likely the
weapon will be delivered within estimated cost and schedule. A knowledge
deficit at the start of development-indicated by a gap between the
technology knowledge attained and the best practice line-means the program
proceeded with immature technologies and faces a greater likelihood of
cost and schedule increases as technology risks are discovered and
resolved.
Figure 3: Depiction of a Notional Weapon System's Knowledge as Compared
with Best Practices
Source: GAO.
An interpretation of this notional example would be that the system
development began with key technologies immature, thereby missing
knowledge point 1. Knowledge point 2 was not attained at the design review
as some technologies were still not mature and only a small percentage of
engineering drawings had been released. Projections for the production
decision show that the program is expected to achieve greater levels of
maturity but will still fall short. It is likely that this program would
have had significant cost and schedule increases.
We conducted our review from June 2005 through March 2006 in accordance
with generally accepted government auditing standards. Appendix II
contains detailed information on our methodology.
Our assessments of the 52 weapon systems follow.
Assessments of Individual Programs
MDA's ABL element is being developed in incremental, capability-based
blocks to destroy enemy missiles during the boost phase of their flight.
Carried aboard a highly modified Boeing 747 aircraft, ABL employs a beam
control/fire control subsystem to focus the beam on a target, a highenergy
chemical laser to rupture the fuel tanks of enemy missiles, and a battle
management subsystem to plan and execute engagements. We assessed the
Block 2004 design that is under development and expected to lead to an
initial capability in a future block.
Source: Airborne Laser Program Office.
Program Transition 6-module Initial Long GAO Lethality Initial
beam/fire duration
to MDA laser control laser review demonstration capability
start test flight test
test
(11/96) (10/01) (11/04) (12/04) (12/05) (1/06) (2008 or (TBD)
later)
Although program officials expected ABL to provide an initial capability
during Block 2006, this event has been delayed and only one of its seven
critical technologies is fully mature. During Block 2004, the program
continues work on a prototype expected to provide the basic design for a
future operational capability. Program officials expect to demonstrate the
other six technologies during a prototype flight test, in late 2008, that
will assess ABL's lethality. MDA has released about 94 percent of the
engineering drawings for the prototype's design, which will be the basis
for an initial operational capability during a future block if the test is
successful. However, additional drawings may be needed if the design is
enhanced or if problems encountered during flight-testing force design
changes.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production
review start design decision
(1/06) (TBD) review (TBD)
(TBD)
Technology Maturity
Only one of ABL's seven critical technologies- managing the high power
beam-is fully mature. The program office assessed the remaining six
technologies-the six-module laser, missile tracking, atmospheric
compensation, transmissive optics, optical coatings, and jitter
control--as nearly mature. According to program officials, all of these
technologies are needed to provide the system with an initial operational
capability.
The program office assessed the six-module laser as being close to
reaching full maturity. In November 2004, the program demonstrated the
simultaneous firing of all six laser modules. However, the initial
operation of the laser was too short to make meaningful predictions of its
power, and problems experienced during recent tests limited the duration
of lasing. In December 2005, the program conducted a longer duration test
of the laser and was able to sustain the beam for more than 10 seconds.
The program also produced approximately 83 percent of the laser's design
power, which, according to program officials, is sufficient to achieve 95
percent of lethal range against all classes of ballistic missiles.
The program recently completed a series of beam control/fire control
flight tests and, as a result, has reassessed three of its critical
technologies- transmissive optics, optical coatings, and jitter
control--as nearing full maturity. The program plans to demonstrate all
technologies in an operational environment during a flight test of the
system prototype, referred to as lethal demonstration, in which ABL will
attempt to shoot down a short-range ballistic missile. Challenges with
integrating the laser and beam control/fire control subcomponents have
delayed this test into late 2008.
Design Stability
We could not assess the design stability because ABL's initial capability
will not be fully developed until the second aircraft is well underway.
While the program has released 10,280 of the 10,910 engineering drawings
for the prototype, it is unclear whether the design of the prototype
aircraft can be relied upon as a good indicator of design stability for
the second aircraft. More drawings may be needed if the design is enhanced
or if problems encountered during flight testing force design changes.
Production Maturity
The program is producing a limited quantity of hardware for the system's
prototype. However, we did not assess the production maturity of ABL
because MDA has not made a production decision.
Other Program Issues
In fiscal year 2004, MDA directed the ABL program to restructure its prime
contract, increase its cost ceiling, and refocus the contractor's efforts
on making technical progress. However, recent technical challenges
associated with the program's beam control/fire control flight test series
and long duration laser testing are causing further cost growth and
schedule slippage for the program. Since our last assessment in January
2005, ABL's planned budget through fiscal year 2009 increased by $483
million (9.4 percent), primarily in fiscal year 2009.
The program plans to award a contract for the second ABL aircraft,
initially to include only trade studies, in fiscal year 2009. MDA has
budgeted approximately $16 million for these trade study initiatives in an
effort to determine the second aircraft system performance capabilities
and to initiate the design of the second weapon system. However, program
officials stated that the commitment to purchase a second aircraft will
not be made until after the system prototype's lethal demonstration.
Agency Comments
In commenting on a draft of this assessment, MDA provided technical
comments, which were incorporated where appropriate.
The Army's ACS is an airborne reconnaissance, intelligence, surveillance,
and target acquisition system and is being designed to provide timely
intelligence data on threat forces to the land component commander. The
ACS will replace the Guardrail Common Sensor and the Airborne
Reconnaissance Low airborne systems. ACS will coexist with current systems
until it is phased in and current systems retire. The Navy will also
acquire ACS to replace its current airborne intelligence platform, the
EP-3.
Source: Lockheed Martin Corporation.
Program Development GAO Design Low-rate Full-rate Initial
start start review review decision decision capability
(7/99) (7/04) (1/06) (12/06) (9/08) (11/09) (2/10)
Due to a significant increase in the weight to integrate the prime mission
equipment on the platform, the Army terminated the development contract.
However, the ACS program will continue although development effort will be
scaled back. At development start, only one of ACS' six critical
technologies was fully mature and two more were nearing maturity.
Currently, one additional technology is nearing maturity. The Army
expected to have demonstrated the maturity of all but one critical
technology by the design review, which was scheduled for December 2006.
The program office estimated that 50 percent of drawings would have been
releasable at that time. The Army plans to reevaluate requirements,
possibly eliminating some, which will likely affect the system's
technologies, design, cost, and schedule.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (7/04) (1/06)
review (9/08) (12/06)
Technology Maturity
Only one of ACS's six critical technologies was mature when the program
started development in July 2004 and two more were nearing maturity. When
the Army terminated the development contract, one additional technology
was nearing maturity. The maturity of one of the remaining technologies
was tied to the development of the airborne version of the Joint Tactical
Radio System, which would not have been available until after ACS was
fielded. The Army expected that all of the critical technologies except
the one tied to the radios would be fully mature by 2006. It is not clear
at this time which requirements might be eliminated or the resulting
impact to the technology maturity.
Design Stability
The program office estimated that 50 percent of the drawings expected for
ACS would have been releaseable by the design review, which was scheduled
for December 2006. However, solving the problem of the increased weight to
integrate the prime mission equipment will likely affect the system's
design.
Other Program Issues
In December 2004, five months after the program began development, the
contractor informed the Army that the weight to integrate prime mission
equipment onto the selected platform had exceeded the structural limits of
the aircraft. In January 2005, a contractor team including Lockheed Martin
and the integration subcontractor initiated a risk mitigation strategy to
address the problem. At the Army's and Navy's direction, the contactor
also began to explore using a larger aircraft. In May 2005, the program
manager submitted a program deviation report notifying DOD that the issue
would likely lead to a nonrecoverable program schedule breach. At the
Army's request, the Navy convened a review team to study the problem
without advocating a particular solution. In September the review team
reported back to the Army. The team identified several factors that
contributed to the problem, including inadequate prime contractor program
management as evidenced by instability on the contractor's engineering
team, lack of design specifications for the subcontractors, and
insufficient exploration of the integration challenges during technology
development.
In September 2005, the Army ordered the contractor to stop all work under
the current contract except for work necessary to provide a written plan
with solutions and alternative strategies to maximize performance and
minimize cost and schedule impacts to the government. In November, the
contractor briefed the Army on three courses of action: refine the
configuration to reduce requirements and keep the current platform; allow
the contractor to acquire a larger platform that can accommodate the
current prime mission equipment; or decouple the platform from system
development and have the contractor deliver only the prime mission
equipment. The Army rejected all three solutions and in January 2006,
terminated the development contract for the convenience of the government.
The Army has not yet estimated the effect to the development cost and
schedule.
Recent funding cuts appear to reduce the total program cost by $43.1
million in current year dollars. Reductions were due to reprogramming and
changes in inflation indices.
Agency Comments
In commenting on a draft of this assessment, the Army provided technical
comments, which were incorporated where appropriate.
The Navy's ADS is a rapidly deployable undersea surveillance system,
scheduled for initial deployment as part of the antisubmarine warfare
mission package on the Littoral Combat Ship (LCS). ADS is designed to
detect, track, and report conventional and nuclear submarines in shallow
waters by laying sensor fields on the ocean floor that send data back to
the LCS for processing and analysis. We assessed the entire system,
including its sensors, sensor installation system, in-buoy processors, and
onboard analysis and reporting system.
Source: ADS Program Office.
Program Development GAO Design Initial Low-rate Full-rate Last
start start review review capability decision decision procurement
(12/92) (11/05) (1/06) (10/06) (9/07) (4/08) (9/09) (unknown)
The ADS program entered system development in November 2005 with none of
its four critical technologies mature. The sensors and the onboard
processing system are more mature because they leverage existing Navy
technology. Program officials identified several remaining risks for ADS,
however, such as the ability of the system to relay data from the in-buoy
processor to the on-board analysis and reporting system and the successful
deployment and installation of sensors. According to the program office,
all technologies are expected to reach maturity in 2007. ADS is expected
to be fully operational with the delivery to LCS in 2009. We were unable
to assess design stability due to a lack of design data at this time.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (11/05) (1/06)
review (4/08) (10/06)
Technology Maturity
None of ADS's four critical technologies reached maturity by the start of
system development in November 2005. Program officials stated, however,
that the maturity of all critical technologies will be demonstrated
through complete end-to-end system testing in fiscal year 2007.
Two critical technologies--the sensor subsystem, consisting of sensors and
fiber optic connecting cables, and the on-board analysis and reporting
system (ARS)--are relatively mature, in part because they leverage
existing technologies. The ARS is comprised of previously developed
software and only requires repackaging and integration into the ADS.
However, although the ARS currently meets its requirements, an ongoing
challenge is developing enhanced automation tools to reduce operator
workload, due to limited space on the LCS. The sensor system is relatively
mature because it uses sensors from previous program development.
Prototypes of both technologies have already been tested in the ocean
environment.
The remaining two critical technologies are less mature, and face several
risks and challenges. The in-buoy processor system, which compresses and
processes data from the ocean floor before sending it to the LCS, is still
in early development. According to program officials, the system's ability
to transfer data to the ARS is a high-risk area. Recent risk reduction
efforts aimed to address this issue. The system may also employ a
reduced-range radio technology as a fallback technology. Additional
development challenges for ADS include improving the overall survivability
of the buoys and increasing their endurance.
The sensor installation system, which deploys and installs sensors on the
ocean bottom, is complicated by its dependence on many smaller
technologies. Successful installation of sensors, as well as the
survivability of connector cables--from fish bites and trawling, for
example--are major development concerns. Back-up options for sensor
installation include deploying the arrays manually, as demonstrated in a
2003 test or using a deployment vehicle that was demonstrated in a fleet
exercise in 1999. Recent risk reduction efforts, however, have improved
the system's performance. In 2004 and 2005, for example, sensor deployment
and highspeed cable pullout were demonstrated successfully.
Design Stability
We were unable to assess ADS design stability due to a lack of design data
at this time.
Other Program Issues
Originally designed for deployment on another platform, the ADS program
was redirected in 2003 to focus its initial increment on deployment from
the LCS. This developmental change caused some redesign of the program,
but incorporated previously developed sensors and processing algorithms.
Moreover, although future spirals will provide the capability to deploy
ADS from an alternate platform, the first increment of ADS is wholly
focused on deployment from the LCS.
The LCS also only allows for limited manpower to support ADS processing
operations. To maximize efficiency, operators may need to be trained in
multiple systems of the LCS's antisubmarine warfare mission area. ADS
program officials are concerned that operators may not have the expertise
necessary to employ ADS effectively.
Agency Comments
In commenting on this assessment, the Navy stated that according to its
standards two ADS technologies-the sensor subsystem and the ARS- are
already mature. According to Navy officials, they evaluate ADS technology
maturity based on standards set by a Naval research group, which considers
technologies mature when they have been demonstrated in a relevant, rather
than an operational environment.
The Navy stated that it is making progress in reducing risks on key
technologies through the execution of a Technology Maturity Plan.
Specifically, Navy officials stated that they are mitigating system risks
through additional testing of the sensor installation system and risk
mitigation planning.
MDA's Aegis BMD element is a sea-based missile defense system being
developed in incremental, capability-based blocks to protect deployed U.S.
forces, allies, and friends from short- and mediumrange ballistic missile
attacks. Key components include the shipboard SPY-1 radar, hit-to-kill
missiles, and command and control systems. It will also be used as a
forward-deployed sensor for surveillance and tracking of intercontinental
ballistic missiles. We assessed only Block 2004 of the element's missile,
the Standard Missile 3 (SM-3).
Source: Aegis BMD Program Directorate.
Program Transition Design Missile Surveillance/tracking Block 2004 GAO
start to MDA review contract capability completion review
awarded
(10/95) (1/02) (5/03) (8/03) (9/04) (12/05) (1/06)
According to program officials, the Block 2004 increment of SM-3 missiles
being fielded during 2004-2005 has mature technologies and a stable
design. However, the program deferred full functionality of the missile's
Solid Divert and Altitude Control System, which maneuvers the missile's
kinetic warhead to its target, to a future upgrade. Program officials
noted that even with reduced capability, the first increment of missiles
provide a credible defense against a large population of the threat. All
drawings for the first increment of missiles have been released to
manufacturing. The program is not collecting statistical data on its
production process but is using other means to gauge production readiness.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production
start design review decision
(10/95) review (1/06) (TBD)
(5/03)
Technology Maturity
Program officials estimate that all three technologies critical to the
SM-3 missile are mature. These technologies--the missile's third stage
rocket motor and the kinetic warhead's infrared seeker and Solid Divert
and Attitude Control System (SDACS)--have been tested in flight. While the
first two technologies were fully demonstrated in flight tests, the SDACS,
which steers the kinetic warhead, was only partially demonstrated. The
SDACS operation in "pulse mode," which increases the missile's divert
capability, failed during a June 2003 flight test. According to program
officials, the test failure was likely caused by a defective subcomponent
within the SDACS, a problem that should be corrected through specific
design modifications. To implement these corrective actions, the program
is deferring full functionality of the missile's SDACS technology to the
next upgrade of the hit-to-kill missile. Program officials note that only
partial functionality of the SDACS is required for Block 2004, which has
been successfully demonstrated in flight tests.
Design Stability
Program officials reported that the design for the first 11 SM-3 missiles
being produced during Block 2004 is stable with 100 percent of its
drawings released to manufacturing. The program plans to implement design
changes in subsequent blocks (delivered during 2006-2007) to resolve the
SDACS failure witnessed in the June 2003 flight test.
Production Maturity
We did not assess the production maturity of the SM3 missiles being
procured for Block 2004. Program officials stated that given the low
quantity of missiles being produced, statistical process control data on
the production process would have no significance. The Aegis BMD program
is using other means to assess progress in production and manufacturing,
such as integrated product team reviews, risk reviews, Engineering
Manufacturing Readiness Levels, and missile metrics.
Other Program Issues
The Aegis BMD element builds upon the existing capabilities of
Aegis-equipped Navy cruisers and destroyers. Planned hardware and software
upgrades to these ships will enable them to carry out the ballistic
missile defense mission. In particular, the program is working to upgrade
Aegis destroyers for surveillance and tracking of intercontinental
ballistic missiles. Because this function is new to the element, the
program has faced a tight schedule to develop and test this added
functionality during the Block 2004 time frame. Although the program aims
to upgrade ten destroyers as part of its Block 2004 increment, this new
functionality has been exercised in a limited number of flight tests and
has never been validated in an end-to-end flight test with the GMD system,
for which it is providing long range surveillance and tracking. Since our
last assessment, Aegis BMD's planned budget through fiscal year 2009
increased by $453.5 million (5.6 percent), primarily in fiscal years 2008
and 2009.
Agency Comments
The Program Office provided technical comments to a draft of this
assessment, which were incorporated as appropriate.
The Air Force's AEHF satellite system will replenish the existing Milstar
system with higher capacity, survivable, jam-resistant, worldwide, secure
communication capabilities for strategic and tactical warfighters. The
program includes satellites and a mission control segment. Terminals used
to transmit and receive communications are acquired separately by each
service. AEHF is an international partnership program that includes
Canada, United Kingdom, and the Netherlands. We assessed the satellite and
mission control segments.
Source: Advanced EHF Program Office.
Program Development Design Production GAO First Initial
start start review decision review launch capability
(4/99) (9/01) (4/04) (6/04) (1/06) (4/08) (6/10)
According to the program office, the AEHF program's technologies are
mature and the design is stable. However, in late 2004 the program was
delayed 12 months because key cryptographic equipment would not be
delivered in time and to allow the program time to replace some critical
electronic components and add testing. Program officials stated the
12-month slip should allow ample time to resolve the issues, but added
significant cost. Total program cost increased about $1 billion. The
program still faces schedule risk due to the continued concurrent
development of two critical path items managed and developed outside the
program. Current plans are to meet full operational capability with three
AEHF satellites and the first Transformational Satellite Communications
System (TSAT) satellite, but additional AEHF satellites may be acquired if
there are deployment delays with TSAT.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO
start design decision review
(9/01) review (6/04) (1/06)
(4/04)
Technology Maturity
According to the program office, all of the 14 critical technologies are
mature, having been demonstrated in a relevant environment and most
progressing into environmental and functional performance testing.
Design Stability
AEHF's design is stable. Virtually all of the expected design drawings
have been released. The program completed its system level critical design
review in April 2004.
Production Maturity
Production maturity could not be assessed as the program office does not
collect statistical process control data.
Other Program Issues
In late 2004, the concurrent development of two critical path items led to
schedule delays and cost increases. The program was restructured in
October 2004, when the National Security Agency did not deliver key
cryptographic equipment to the payload contractor in time to meet the
launch schedule. The restructuring added 12 months to the program to allow
time to resolve the cryptographic delivery issues and resolve other
program problems including replacement of critical electronic components
and additional payload testing. Delaying the launches and resolving these
issues added about $800 million to the program. Earlier cost increases
brought the total increase to about $1 billion, incurring a Nunn-McCurdy
breach in December 2004 (10 U.S.C. 2433) at the 15 percent threshold.
The program still faces schedule risk due to the continued concurrent
development of two critical path items developed and managed outside the
program; the cryptographic components developed and produced by the
National Security Agency and the Command Post Terminals managed by another
Air Force Program Office. During 2005, the program developed emulators to
simulate key cryptographic equipment to allow payload testing and
integration to continue, and National Security Agency began delivery of
some actual components, meeting its revised delivery dates.
Program officials told us the mission control segment continues to meet or
exceed its schedule and performance milestones. In addition, the program
made progress in several areas including: completion of end-to-end testing
for the payload and terminal communications utilizing test terminals,
completion of static load testing on the satellite structure, and delivery
of the flight cryptographic hardware, which has been installed and tested
on the first satellite.
Three AEHF satellite launches are scheduled for 2008, 2009, and 2010. In
December 2002, satellites four and five were deleted from the program with
the intention of using TSAT to achieve full operational capability.
However, the AEHF contract contains options to buy additional satellites
if there are deployment problems with TSAT.
Agency Comments
The Air Force provided technical comments, which were incorporated as
appropriate.
The Navy's AESA radar is one of the top upgrades for the F/A-18E/F
aircraft. It is to be the aircraft's primary search/track and weapon
control radar and is designed to correct deficiencies in the current
radar. According to the Navy, the AESA radar is key to maintaining the
Navy's air-to-air fighting advantage and will improve the effectiveness of
the air-to-ground weapons. When completed, the radar will be inserted in
new production aircraft and retrofitted into lot 26 and above aircraft.
Source: U.S. Navy.
Program/ Design Low-rate GAO Initial Full-rate First
development start review decision review capability decision deployment
(2/01) (8/01) (6/03) (1/06) (10/06) (1/07) (11/07)
The AESA radar's critical technologies appear to be mature and the design
appears stable. However, radar development continues during production.
The program is tracking a number of risks with the technical performance
of the radar. If problems are discovered, design changes could be required
while the radar is in production. Software development continues to be the
program's top challenge. Problems in developing radar software have
resulted in deferring several advanced capabilities until future software
configurations. Radar production faces a high risk in 2006 because a
material for one of the radar's critical technologies is expected to go
out of production. Several other development and production risks have not
been resolved.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Attainment of Product Knowledge
vel of knowledge
Desired le
Datanot available
Development DOD Production GAO start design decision review (2/01) review
(6/03) (1/06) (8/01)
Technology Maturity
A technology readiness assessment for the radar in fiscal year 2004
determined that the four critical technologies were mature. To further
ensure technology maturity, a final technology assessment was held in
November 2005.
Design Stability
Although the AESA design appears to be stable, development of the radar
has continued during production. According to program officials, radar
software continues to be the top program challenge. Several advanced radar
capabilities have been deferred to future software configurations. The
radar schedule could not be extended because it is directly tied to the
F/A-18E/F schedule. According to the program office, these capabilities
will not be deferred beyond the first deployment, and no key performance
parameters will be affected by the deferral. Since the start of
development, the number of lines of software code has increased by 17
percent, and software development costs have increased by over 40 percent.
According to a program office risk assessment, other development risks
could result in design changes: the radar may not be able to track
sufficient targets simultaneously or detect tail targets at low altitude;
radiation emissions may interfere with F/A-18E/F weapon systems; and the
radar power supply may not prevent voltage modulation on the aircraft
power system. Also, the radar simulation model integrated into the F/A-18
training simulator may not accurately represent radar operation and
performance. Mitigation plans are in place to address the design risks
and, according to the program office, the likelihood of a design change is
minimal due to over 500 flights with the AESA radar.
Production Maturity
We could not assess production maturity because statistical process
control data are not being collected. Instead, manufacturing processes
continue to be monitored and controlled at each manufacturing center and
laboratory. Twenty percent of the 415 radars are to be procured during 4
low-rate production runs. The radar's third production run has been
approved. Nine radars had been delivered as of August 2005. Most radars
will be installed in F/A-18E/Fs on the aircraft production line, but 135
radars are to be retrofitted into already produced aircraft.
Radar production continues to face a number of risks. A high risk involves
a foam material for the radar's wideband radome, a critical technology.
The manufacturer plans to stop producing the material in the 2006 time
frame, which would affect future radar production. The program office
plans to mitigate this risk by making a lifetime buy of the foam material.
According to the program risk assessment, other risks include whether:
radar manufacturing capacity can ramp up enough to meet production and
reliability problems with a radar critical technology will allow initial
radars to meet a specification. Also, low-rate production is exceeding
design-to-cost and firm, fixed-price costs. For example, the estimate at
completion for the radar contract is projected to overrun the target cost
by up to 34 percent.
Other Program Issues
In response to a 1999 DOD directive, a requirement was added to the radar
for antitamper protection to guard against exploitation of critical U.S.
technologies. According to the program office, a successful critical
design review for this requirement was completed in November 2005. While
officials said there is a requirement for this protection to have no
effect on radar performance, operational tests of antitamper models may
identify problems that require design changes to the protection package.
By then, 84 radars are expected to have been produced.
Agency Comments
In commenting on a draft of this assessment, the Navy provided technical
comments, which were incorporated as appropriate.
The Army's APKWS is a precision-guided, air-to-surface missile designed to
engage soft and lightly armored targets. The system is intended to add a
new laser-based seeker to the existing Hydra 70 Rocket System and is
expected to provide a lower cost, accurate alternative to the Hellfire
missile. Future block upgrades are planned to improve system
effectiveness. We assessed the laser guidance technology used in the new
seeker.
Source: Joint Attack Munition Systems Project Office.
Program/ GAO Design Low-rate Initial Last
development start review review decision capability procurement
(12/02) (1/06) (6/06) (9/08) (6/11) (unknown)
Since our assessment of APKWS last year, the Milestone Decision Authority
curtailed the program. We reported the APKWS entered development and held
its design review before demonstrating its critical guidance technology
was fully mature and that initial system-level testing identified problems
with the design. According to program officials, placement of the laser
seeker proved to be problematic. The combination of development cost
overruns, a projected schedule slip of 1-2 years, unsatisfactory contract
performance, and environmental issues resulted in curtailment of the
initial APKWS program in January 2005. Program officials expect to award
the contract for a restructured APKWS program in the second quarter of
fiscal year 2006. Due to program uncertainty, we were unable to assess
design, technology, or production maturity.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (12/02) (1/06)
review (9/08) (6/06)
APKWS Program Agency Comments
The Army provided technical
changes, which were
Technology Maturity incorporated as appropriate.
At the time of our last report, APKWS had
one
critical technology--laser guidance. Since
the laser
technology was employed on other platforms,
program officials considered it to be
mature.
However, according to program officials,
integration
of the laser on the fins rather than in the
head of the
missile proved to be more problematic then
originally estimated. The configuration
difficulty
presented problems that the contractor could
not
overcome and keep the missile within cost
and on
schedule. The integration issue contributed
to the
cost overrun and protracted schedule, which
subsequently led to program curtailment and
restructuring. Program officials stated they
have
since identified several laser seeker and
guidance
and control systems suitable for the Guided
Rocket
requirement. Furthermore, program
representatives
feel they have sufficient information to
proceed with
the critical design review immediately after
contract
award. Because the contractor and the
specific
technical approach to be pursued are yet to
be
determined, we could not assess the maturity
of the
design, technology, or production for the
restructured program.
Other Program Issues
Although the APKWS program was scheduled to
start production of the rocket in fiscal
year 2006, a
number of program problems related to
development cost overruns, schedule
slippage, and
contract performance resulted in the Army
Program
Executive Officer for Missiles and Space
curtailing
the program in January 2005. Following
curtailment,
the Vice Chief of Staff of the Army
validated the
requirements and approved a restructured
APKWS
program and timeline. Program officials
released a
Draft Request for Proposal in June 2005 and
are
expecting to award a new contract for the
restructured APKWS program during the second
quarter of fiscal year 2006. According to
program
officials, the current fiscal year 2006
President's
budget was prepared and submitted prior to
the
Milestone Decision Authority's decision to
curtail
the initial APKWS contract and restructure
the
program. Ongoing program office efforts to
align
program funding to the new structure have
not yet
been completed.
The Special Operations Forces' ASDS is a batterypowered dry interior
hybrid combatant submersible for clandestine insertion/extraction of Navy
SEALs and their equipment. It is carried to a deployment area by specially
configured 688-class submarines. ASDS is intended to provide increased
range, payload, on-station loiter time, endurance, and
communication/sensor capacity over current submersibles. The 65-foot-long
8-foot-diameter ASDS is operated by a two-person crew and includes a lock
out/lock in diving chamber.
Source: U.S. Navy.
Program Development Initial Initial GAO Production Last
operational
start start test and capability review decision procurement
evaluation
(7/89) (9/94) (5/03) (11/03) (1/06) (TBD) (TBD)
The ASDS program is being restructured due to Attainment of Product
Knowledge reliability problems with the first boat, and the Production,
production decision for additional units has been design &
technology
cancelled. Restructuring includes developing a maturity reliability
improvement plan and conducting a critical system review to identify
issues that need to be addressed. ASDS design changes since our Design &
technologylast report include replacing the silver-zinc battery maturity
with a lithium-ion battery, replacing the aluminum tail with a titanium
tail, and several other modifications. At-sea development testing of the
lithium-ion battery has been completed. Acoustic, Technologyor noise level
problems, are being addressed; maturity however, this requirement does not
have to be met until delivery of the second ASDS boat. Until ASDS
reliability is assessed, problems are addressed, and operational testing
is completed, ASDS technology maturity and design stability
remain uncertain. Development DOD GAO Production start design review
decision (9/94) review (1/06) (TBD) (6/96)
Technology Maturity
The program office identified three ASDS critical technologies. Although
two of the three technologies were mature at the time of our last
assessment, since that time the aluminum tail (mature) has been replaced
with a titanium tail. The silver-zinc battery was replaced with a
lithium-ion battery. In an August 2005 at-sea development test of the
battery, requirements for speed, range, and endurance were exceeded.
Acoustic, or noise level problems, are being addressed. In earlier tests,
the ASDS propeller was the source of the most significant noise, and a new
composite propeller was installed before operational test and evaluation
in 2003. Although program officials believe the improved propeller will
significantly reduce the ASDS acoustic signature, precise acoustic
measurements are incomplete. Other acoustic issues will be addressed on a
time-phased basis because the acoustic requirement has been deferred until
delivery of the second boat.
Design Stability
The ASDS experienced a propulsion-related failure during Follow-on
Operational Test and Evaluation in October 2005, and the Navy decertified
ASDS from operational test readiness. The Navy is investigating the causes
of the failure and plans to complete repairs and post-repair testing in
January 2006. On November 30, 2005, the United States Special Operations
Command (SOCOM) and the Navy announced the restructuring of the ASDS
program to focus on correcting reliability deficiencies with the first
boat and to conduct verification testing of improvements before continuing
operational testing. The ASDS Reliability Action Panel, a panel of
submarine and submersible technical experts from government and industry
chartered by SOCOM and the Navy in September 2005, noted that there were
numerous examples of unpredicted component reliability problems and
failures resulting from design issues and that operational testing should
not be resumed until completion of a detailed review of mission critical
systems.
Consequently, the production decision for additional units has been
cancelled until the first boat's reliability has been improved. Under the
ASDS restructuring plan, the critical system review is expected to
identify known problems and other potential issues and identify what
design changes are needed. A Vulnerability Assessment Report assessing
ASDS survivability design features was issued in September 2005 and a
Capabilities Production Document (to replace the June 2004 ASDS
operational requirements document) is under review. Until the program's
critical system review is completed, all requirements are addressed,
technical problems are solved, and testing is completed, we believe the
ASDS final design will remain uncertain and may have cost and schedule
implications. Because the ASDS program is being restructured, we are not
assessing the current level of ASDS design stability.
Other Program Issues
In December 2004 SOCOM reduced the ASDS program quantity to three units
due to resource constraints. However, it affirmed that the operational
requirements document remained valid at six ASDS vehicles.
Agency Comments
The Navy concurred with our assessment and provided updated costs, which
were incorporated as appropriate.
Source: BAE Systems.
Low-rate GAO Full-rate Last
decision review decision procurement
(11/03) (1/06) (6/10) (2023)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO
start design decision review
(6/95) review (11/03) (1/06)
(2/97)
The Army's and Special Operations' ATIRCM/CMWS is a component of the Suite
of Integrated Infrared Countermeasures planned to defend U.S. aircraft
from advanced infrared-guided missiles. The system will be employed on
Army and Special Operations aircraft. ATIRCM/CMWS includes an active
infrared jammer, missile warning system, and countermeasure dispenser
capable of loading and employing expendables, such as flares, chaff, and
smoke.
Program/ Design development start review (6/95) (2/97)
The ATIRCM/CMWS program entered production in November 2003 with
technologies mature and designs stable. However, one of the five critical
technologies was recently downgraded due to continued technical
difficulties. Currently, the program's production processes are at various
levels of control. The CMWS portion of the program entered limited
production in February 2002 to meet urgent deployment requirements.
However, full-rate production for both components was delayed because of
reliability problems. Over the past several years, the program has had to
overcome cost and schedule problems brought on by shortfalls in knowledge.
Key technologies were demonstrated late in development and only a small
number of design drawings were completed by design review. At the low rate
production decision point, the Army developed a new cost estimate reducing
program procurement cost substantially.
Technology Maturity
The five critical technologies were considered mature until a
government/industry team recently downgraded the maturity level of the
infrared jamming head due to technical issues. Additionally, the other
four technologies did not mature until after the design review in February
1997. Most of the early technology development effort focused on the
application to rotary wing aircraft. When system development began in
1995, requirements were expanded to include Navy and Air Force fixed wing
aircraft. This change caused problems that contributed to cost increases
of over 150 percent. The Navy and the Air Force subsequently dropped out
of the program, but the Navy and the Army are currently pursuing future
joint production planning.
Design Stability
The basic design of the system is complete with 100 percent of the
drawings released to manufacturing. The design was not stable at the time
of the design review, with only 22 percent of the drawings complete due to
the expanded requirements. Two years after the design review, 90 percent
of the drawings were released and the design was stable. This resulted in
inefficient manufacturing, rework, additional testing, and a 3-year
schedule delay.
Production Stability
Production maturity could not be assessed based on the information
provided by the program office. According to program officials, the
program has 21 key manufacturing processes in various phases of control (7
CMWS and 14 ATIRCM). The CMWS production portion of the system has
stabilized and benefited from increased production rates. Also, processes
supporting both ATIRCM and CMWS will continue to be enhanced as data is
gathered and lessons learned will be included in the processes. The Army
entered limited CMWS production in February 2002 to meet an urgent need.
Subsequently, full rate production was delayed for both components due to
reliability testing failures. The program implemented reliability fixes to
six production representative subsystems for use in initial operational
test and evaluation. These systems were delivered in March 2004. The
full-rate production decision for the complete system was recently delayed
until June 2010 due to ATIRCM performance issues.
Other Program Issues
The Army uses the airframe as the acquisition quantity unit of measure
even though it is not buying an ATIRCM/CMWS system for each aircraft. When
the program began, plans called for putting an ATIRCM/CMWS on each
aircraft. Due to funding constraints, the Army reduced the number of
systems to be procured and will rotate the systems to aircraft as needed.
The Army is buying kits for each aircraft, which include the modification
hardware, wiring harness, cables necessary to install and interface the
ATIRCM/CMWS to each platform. The Army plans to buy 1,710 ATIRCM/CMWS
systems and 3,571 kits to use for aircraft integration. As a result, the
true unit procurement cost for each ATIRCM/CMWS system is more on the
order of $2.8 million.
The current program baseline includes accelerated funding to procure
additional ATIRCM/CMWS systems additional nonrecurring engineering driven
by an increase in the number and types of platforms. The quantity of
ATIRCM/CMWS systems was increased from 1,076 to 1,710 in June 2005. A new
Army cost position has been established that reflects the impact of the
CMWS full rate production decision, the increased quantities, and the
schedule delays.
Agency Comments
The ATIRCM/CMWS program has been realigned to address Global War on
Terrorism requirements and implement improvements. In response to a
November 2003 memo from the Assistant Secretary of the Army to equip all
Army helicopters in Iraq and Afghanistan with the most effective defensive
systems, the program office proposed accelerating the CMWS portion of
ATIRCM. To date, 506 installation kits and 214 CMWS's have been fielded.
Full-rate production decision for CMWS required a separate Initial
Operational Test and Evaluation, completed November 2005. CMWS full rate
production decision is planned for February 2006.
The ATIRCM system experienced performance and reliability issues during
October 2004 testing. The program has been rebaselined, allowing for
improved performance, adding a multiband laser capability and increased
ATIRCM system reliability. Full rate production is currently planned for
fiscal year 2010. This rebaselined plan was presented and approved by the
Army Acquisition Executive in December 2005.
The Air Force's B-2 RMP is designed to modify the current radar system to
resolve potential conflicts in frequency band usage. To comply with
federal requirements, the frequency must be changed to a band where the
B-2 will be designated as a primary user. The modified radar system is
being designed to support the B-2 stealth bomber and its combination of
stealth, range, payload, and near precision weapons delivery capabilities.
Source: U.S. Air Force, U.S. Edwards Air Force Base, California.
Program Development Design GAO Low-rate Initial Full-rate Last
start start review review decision capability decision procurement
(10/02) (8/04) (5/05) (1/06) (2/07) (1/08) (2/08) (2009)
Since our assessment of the B-2 RMP last year, the Attainment of Product
Knowledge program successfully completed its design review Production, in
May 2005 with all four critical technologies design &
technology
considered mature. The program had released 85 maturity percent of its
design drawings by the design review and plans to have 100 percent
released by the start of production. Program officials told us Design &
technologyproduction maturity metrics will be formulated maturity during
development and these metrics may or may not include manufacturing process
control data. The program plans to build seven radar units during
development for pilot training with the B-2 Technologywing prior to the
planned completion of flight maturity testing. Six of these units will
later be modified and placed on B-2 aircraft. These units are necessary,
but building them in development adds to the risk of later design changes
because most of the radar flight testing will not occur until after
these units are built. Development DOD GAO Production start design review
decision (8/04) review (1/06) (2/07) (5/05)
Technology Maturity
All four B-2 RMP critical technologies were considered mature at the
design review in May 2005. While the program entered development in August
2004 with two of these four critical technologies mature and two
approaching maturity, the receiver/exciter for the electronic driver cards
and aspects of the antenna designed to help keep the B-2's radar signature
low, all four are now considered mature. The program expects these
technologies to reach a slightly higher level of maturity at the start of
production in 2007.
Design Stability
The program office completed its design readiness review in May 2005 and
at that time had 85 percent of its drawings released to manufacturing. The
program plans to have 100 percent of its drawings released by the start of
production in 2007. The program, however, does not use the release of
design drawings as a measure of design maturity but instead uses the
successful completion of design events, such as subsystem design reviews,
as its primary measure of design maturity.
Production Maturity
Production maturity metrics are planned to be formulated during
development. These metrics, which may or may not include manufacturing
process control data, are planned to be used as measures of progress
toward production maturity during a production readiness review prior to
the start of production in February 2007. The program is also involved in
a proof-of-manufacturing effort to demonstrate that the transmit/receive
modules can be built to specifications.
Other Program Issues
The program plans to build seven radar units during development and later
modify six of these units for placement on operational B-2 aircraft. The
Air Force needs these radar units for air crew training and proficiency
operations. Even though these units are necessary, building them early in
development adds risk because most of the radar flight-test activity will
not occur until after these units are built.
Agency Comments
The Air Force concurred with this assessment.
Source: C-130 Avionics Modernization Program, System Program Office.
Low-rate Production Last
decision readiness review procurement
(6/06) (11/09) (2016)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (7/01) (1/06)
review (6/06) (TBD)
The Air Force's C-130 AMP standardizes the cockpit configurations and
avionics for 14 different mission designs of the C-130 fleet. It
consolidates and installs the mandated DOD Navigation/Safety
modifications, the Global Air Traffic Management systems, and the C-130
broad area review requirements. It also incorporates other reliability,
maintainability, and sustainability upgrades and provides increased
situational awareness capabilities and reduces susceptibility of Special
Operations aircraft to detection/interception.
Development GAO Design
start review review
(7/01) (1/06) (TBD)
The C-130 AMP is utilizing commercial and modified off-the-shelf
technologies, and it entered system development with five of its six
critical technologies mature. The final technology reached maturity in
2005 through a series of demonstration flights. Program officials plan to
release 90 percent of engineering drawings by the design review and have
made progress toward that goal. As of December 2005, 100 percent of
required drawings for Combat Delivery First Flight had been released.
Program delays have resulted from funding cuts, and sustained development
contract protests required a portion of the contract to be recompeted. The
August 2005 design review has been postponed indefinitely, and the low
rate initial production decision has been delayed until June 2006. These
dates may change again after program restructuring is completed.
Technology Maturity
All of the C-130 AMP's six critical technologies are fully mature, as the
program is primarily utilizing proven commercial and modified
off-the-shelf technology for all AMP capabilities. A program official
stated that the last immature critical technology, Terrain Following and
Terrain Avoidance radar, reached full maturity in 2005 by meeting the key
requirement of operability at 250 feet during demonstration flights.
Design Stability
As of December 2005, the program office had released 100 percent of
required drawings for Combat Delivery First Flight. According to the Air
Force, due to program restructuring, the Combat Talon critical design
review was postponed indefinitely, and a new review date will be
established under the current replan effort.
The modernization effort is divided into a number of capability spirals
due to the various aircraft designs. The first spiral will outfit C-130
aircraft with core capabilities and an integrated defensive system.
Special Operations C-130 aircraft will be outfitted first, and future
spirals are planned for these aircraft because they require additional,
and unique, defensive systems integration and enhanced situational
awareness.
Other Program Issues
Since GAO's last review of the C-130 AMP, the program office has postponed
the design readiness review indefinitely, pushed back the low-rate initial
production 4 months, and delayed the production readiness review 18
months.
Funding reductions in fiscal years 2003 and 2004 delayed the development
program and contributed to the rescheduling of program milestones and the
rebaselining of the program. In addition, sustained protests associated
with the C-130 AMP development contract awarded in 2001 required that a
portion of the contract be recompeted.
Agency Comments
The Air Force provided technical comments to a draft of this assessment,
which were incorporated where appropriate.
Source: Lockheed-Martin Aeronautics Company.
Production GAO Initial Last
decision review capability procurement
(2/03) (1/06) (5/06) (TBD)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO
start design decision review
(11/98) review (2/03) (1/06)
(5/01)
The Air Force's C-5 AMP is the first of two major upgrades for the C-5 to
improve the mission capability rate, transport capabilities and reduce
ownership costs. The AMP implements Global Air Traffic Management,
navigation and safety equipment, modern digital equipment, and an
allweather flight control system. The second major upgrade, the C-5
Reliability Enhancement and Reengining Program (RERP), replaces the
engines and modifies the electrical, fuel, and hydraulic systems. We
assessed the C-5 AMP.
Development Design start review (11/98) (5/01)
Since our assessment of the C-5 AMP last year, the program completed
developmental test and evaluation in August 2005, 10 months later than
planned. The program's technologies and design are considered mature as
they are relying on commercial-off-the-shelf technologies that are
installed in other commercial and military aircraft. The main challenge to
the program has been the development and integration of software--to which
the schedule delay as well as a $23 million cost overrun has been
attributed. The Air Force plans to modify 59 of the 112 C-5 aircraft. The
Air Force is also seeking funding to modify the remaining 53 C-5s;
however, that decision will not be made until the Air Force determines the
correct mix of C-5 and C-17 aircraft needed to meet DOD's airlift needs.
If the Air Force decides to use the C17s, it may not upgrade some, or all,
of the remaining 53 C-5s.
Technology Maturity
We did not assess the C-5 AMP's critical technologies because the program
used commercial technologies that are considered mature. Program officials
stated that those technologies are in use on other aircraft and that they
have not significantly changed in form, fit, or function. For example, the
new computer processors are being used in the Boeing 777, 717, other
commercial aircraft, the KC10, and a Navy reconnaissance aircraft.
Design Stability
Last year we reported that the C-5 AMP had released 100 percent of their
drawings; however, due to modifications in the design an additional 270
drawings were added. As a result, the program had completed only 54
percent of the total number of drawings for the system by the time of the
production decision. The program now reports that the contractor has
released all of the drawings for the AMP. In addition, seven major
subsystem-level design reviews were completed along with integration
activities. Demonstration of these activities were completed during
developmental test and evaluation, which started in December 2002 and was
completed in August 2005.
Production Maturity
We could not assess the production maturity because most components are
readily available as commercial-off-the-shelf items. This equipment is
being used on other military and commercial aircraft. In addition, the C-5
AMP is incorporating many other off-the-shelf systems and equipment, such
as the embedded global positioning system, the inertial navigation system,
and the multifunction control and display units. To ensure production
maturity, the program office is collecting data regarding modification kit
availability and the installation schedules.
Other Program Issues
Over the past year, the AMP program ran into significant problems while
trying to complete software development that have impacted the cost and
schedule of the program. Most notably, a software build was added to fix
problems with AMP integration, flight management system stability and
system diagnostics. The added build caused a $23 million cost overrun,
which was paid for by shifting funds from the RERP program, and extended
developmental testing to 10 months. The program office acknowledged that
an another software build may be added, depending on the results of
operational testing that is now scheduled to be completed in July 2006.
Last year we reported that the Air Force was conducting mobility studies
to determine the correct mix of C-5 and C-17 aircraft it would need in the
future. This decision has not been made yet. In the meantime, the program
office is continuing its plan to provide AMP modifications for 59 of the
aircraft while all 112 aircraft are projected to go through the RERP
program. If all 112 aircraft are needed and do go through the RERP
program, then the Air Force will need to request additional money to fund
AMP modifications for the remaining 53 aircraft.
Agency Comments
The Air Force provided technical comments, which were incorporated as
appropriate.
The Air Force's C-5 RERP is one of two major upgrades for the C-5. RERP is
designed to enhance the reliability, maintainability, and availability of
the C-5 through engine replacement and modifications to subsystems, i.e.
electrical and fuel, while the C-5 Avionics Modernization Program (AMP) is
designed to enhance the avionics. The upgrades are part of a two-phased
modernization effort to improve the mission capability rate, performance,
and transport throughput capabilities and reduce total ownership costs. We
assessed the C-5 RERP.
Source: Lockheed-Martin Aeronautics Company.
Program Development Design GAO Low-rate Full-rate Full-rate Last
decision
start start review review decision decision A-Model procurement
B-Model
(2/00) (11/01) (12/03) (1/06) (12/06) (11/08) (11/12) (FY18)
The RERP is utilizing demonstrated commercial Attainment of Product Knowledge
off-the-shelf components that require little or no modification. The
program ensured that its technologies and design were stable at critical
points in development. The program, which is currently in system
development and demonstration, plans to enter low-rate production in
December 2006. However, since last year the program has experienced a
9-month schedule delay due to multiple issues, such as a pylon redesign,
and has been subject to almost $50 million in budget cuts that further
increases schedule risk. The C-5 RERP program is also dependent on the
number of aircraft approved to undergo the C-5 AMP modernization program.
Until additional aircraft are approved for the AMP, it is uncertain how
many aircraft will undergo the RERP.
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Development DOD GAO Production start design review decision (11/01) review
(1/06) (12/06) (12/03)
Technology Maturity
The C-5 RERP's technologies are mature based on an independent technology
readiness assessment conducted in October 2001. New engines account for 64
percent of the expected improvement in mission capability rate for the
aircraft. The new engines are commercial jet engines currently being used
on numerous aircraft. According to the Air Force technology assessment,
these engines have over 238 million flying hours of use.
Design Stability
The C-5 RERP's design is undergoing changes due to a necessary redesign of
the pylon/thrust reverser to address overweight conditions and safety
concerns for the engine mount area. According to program officials, the
redesign has contributed 4 months to the overall 9 month schedule delay of
the program. Prior to this redesign, 98 percent of the design drawings
were complete. It is unclear what effect the latest redesign will have on
the completed drawings. According to the program office, the seven major
subsystem level design reviews were completed before the December 2003
system-level design review.
The program is taking advantage of AMP developed products and lessons
learned in the C-5 RERP to reduce the risk of potential schedule slips
associated with software development and integration. For example,
according to program officials, some of the baseline software and systems
integration facilities that were developed for C-5 AMP can be reused for
RERP activities.
Production Maturity
We did not assess the C-5 RERP's production maturity because the Air Force
is buying commercially available items. However, we expect that production
maturity would be at a high level because the engines have been
commercially available for many years.
Other Program Issues
The program has experienced a 9-month schedule delay since last year due
to multiple issues including, pylon weight and redesign, asymmetric thrust
reverser development problems, C-5 AMP delays, and wing rib web structure
design and manufacture. The 9-month delay has cost the program an
additional $45 million. In addition, recent budget reductions of almost
$50 million are increasing the schedule risk of the program. Almost half
of this money was shifted to the C-5 AMP to help that program complete
software development activities. The remaining funds were cut by OSD
because it appeared the program was under executing its funds. These cuts,
along with the pylon development problems mentioned earlier, have forced
the delay of the trainer program until fiscal year 2008. Program officials
are also considering aggressive steps, such as hiring additional workers
and using multiple shifts, to address potential schedule increases.
RERP officials are currently monitoring negotiations between DOD and
General Electric to bring General Electric into full compliance with the
Berry Amendment, which requires certain metals used in military systems to
be purchased from domestic sources. According to Air Force officials,
General Electric expects to be in full compliance with the Berry
Ammendment by January 2007, without impact to C-5 RERP.
The program is still waiting on the results of a mobility study to
determine the mix of C-5 and C-17 aircraft the Air Force plans to use in
the future. Until that decision is made, the Air Force is continuing its
plan to re-engine all 112 C-5 aircraft. Before that can be done, however,
all 112 will need to complete the AMP upgrade. Yet, the Air Force has only
provided funding for 59 of the aircraft to receive the AMP upgrade at this
time.
Agency Comments
The Air Force provided technical comments, which were incorporated as
appropriate.
The Army's CH-47F heavy lift helicopter is intended to provide
transportation for tactical vehicles, artillery, engineer equipment,
personnel, and logistical support equipment. It is also expected to
operate in both day and night. The program goal is to enhance performance
and extend the useful life of the CH-47 as well as produce new
helicopters. This effort includes installing a digitized cockpit,
rebuilding the airframe, and reducing aircraft vibration.
Source: Boeing Helicopters.
Program/ Design Full-rateLow-rate GAO First unit Last
development start review decision decision review equipment procurement
(12/97) (9/99) (12/02) (11/04) (1/06) (5/07) (2018)
The CH-47F technologies appear mature and the Attainment of Product Knowledge
design stable, with 100 percent of the engineering drawings released for
manufacturing. CH-47F production maturity could not be assessed as the
program is not collecting statistical process control data on key
manufacturing processes. Program officials believe that CH-47F production
is low risk because no new technology is being inserted into the aircraft,
two prototypes have been produced, and the production process was
demonstrated during the delivery of one low-rate initial production
aircraft. Since our last assessment, the CH-47F program entered full rate
production and increased quantities from 339 to 512 aircraft. Because the
increase in quantities includes 55 new build helicopters, program unit
cost increased approximately 12 percent over what we reported last year.
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Datanot available
Development DOD Production GAO
start design decision review
(12/97) review (12/02) (1/06)
(9/99)
Technology Maturity
We did not assess technology maturity or determine the number of critical
technologies in detail. The CH-47F is a modification of the existing
CH-47D helicopter. Program officials believe that all critical
technologies are mature and have been demonstrated prior to integration
into the CH-47F development program.
Design Stability
The Army entered full rate production in November 2004, with 100 percent
of the drawings released to manufacturing. However, the number of drawings
completed increased substantially since the start of low rate production.
As a result, the level of maturity achieved at design review was only 11
percent and at low rate production was 31 percent. The majority of the new
drawings were instituted to correct wire routing and installation on the
aircraft. Accordingly, the program office believed the total number of
drawings could not be determined until after the first prototype was
delivered.
Production Maturity
We did not assess production maturity because the CH-47F program does not
collect statistical process control data on its production of helicopters.
The program office relies on inspections as its means to ensure acceptable
production results.
According to the program office, the CH-47 production is low risk because
two prototypes have been produced during development and the Army recently
took delivery of its first low-rate initial production aircraft. Further,
the program reported that during low-rate production, it made significant
advances in the refinement of CH-47 production processes. Advances include
the implementation of the automated management execution system and the
introduction of laser tracking to identify key mounting points. These
enhancements are geared toward improving the manufacturing learning curve.
However, the program office acknowledges that the program will lose some
of the learning benefits during the anticipated break in production of the
CH-47F in favor of producing more MG-47 special operations configuration
helicopters during the next lot of production.
Other Program Issues
In November 2004, the Army Acquisition Executive approved the revised
program acquisition strategy and approved the start of full rate
production. This acquisition strategy includes service life extension
upgrades for the CH-47D fleet and a number of newbuild aircraft to meet
operational fleet requirements. Included in the new baseline is a revised
acquisition objective quantity of 512 upgraded aircraft as opposed to the
339 previously reported. Of the larger quantity, 2 are developmental; 55
will be new build CH-47Fs; 58 will be remanufactured in the special
operations configuration; and 397 remanufactured into CH-47Fs to replace
the current CH-47Ds. Because new builds, as opposed to only remanufactured
helicopters, have been included in the acquisition plan, unit cost
increased 12 percent over what we reported last year.
Agency Comments
In commenting on a draft of this assessment, the Army concurred with the
information presented in this report. One technical comment was provided,
which was incorporated as appropriate.
The Navy's CVN-21 class is the successor to the Nimitz-class aircraft
carrier and includes a number of advanced technologies in propulsion,
aircraft launch and recovery, weapons handling, and survivability. These
technologies are to allow for increased sortie rates and decreased manning
rates as compared to existing systems. Many of the technologies were
intended for the second ship in the class, but they were accelerated into
the first ship in a December 2002 restructuring of the program.
Source: CVN-21 Program Office.
Program Development GAO Design Construction Construction Initial
start start review review start-1st ship start-2nd ship capability
(6/00) (4/04) (1/06) (3/07) (1/08) (1/12) (9/16)
The CVN-21 entered system development in April 2004 with few of its
critical technologies fully mature. This is due in part to DOD's decision
to accelerate the installation of a number of technologies from the second
ship to the first. Program officials state that the extended construction
and design period allows further time for development. They have
established a risk reduction strategy that includes decision points for
each technology's inclusion based on a demonstrated maturity level.
Fallback technologies exist for 11 of 18 total critical technologies, but
their use would lead to drawbacks, such as performance shortfalls and/or
an increase in manpower requirements. The program has reported a 1-year
schedule slip based on decisions to balance ship construction in the
President's fiscal year 2006 budget. Program officials expect to meet
their design review date, currently set for March 2007.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/04) (1/06)
review (1/08) (3/07)
Technology Maturity
There are currently a total of 18 CVN-21 critical technologies, of which 3
are presently mature and 3 are approaching maturity. The remaining 12 are
at lower levels of maturity. The Navy expects that 14 of the 18 total
technologies will be mature or close to mature by the design review in
fiscal year 2007, and they expect all but 1 technology to be near maturity
by production start in 2008. Program officials originally reported 16
critical technologies at development start. However, one technology was
redefined into two, more specific technologies and another was added since
that time.
Six of the critical technologies are being developed by programs other
than CVN-21. Progress in those programs could affect the CVN-21 timeline.
Those technologies are the Advanced Arresting Gear, Evolved Sea Sparrow
Missile, Joint Precision Approach and Landing System, Multi-Function
Radar, Volume Search Radar, and the Advance Weapons Information Management
System/Aviation Data Management and Control System. This last technology
was redefined after development start. In the case of four technologies
the program has mature alternate systems as backup technologies. Program
officials stated that no backup is feasible for either Volume Search Radar
or Multi-Function Radar without major ship redesign.
Two technologies modified since development start are also not mature. The
Shipboard Weapons Loader is a self-propelled unit to decrease the time
required to load weapons onto aircraft. The other technology is Smart
Stores, which is a software-based system to automate CVN-21's inventory
and material asset management capabilities. The Navy's primary risks
identified for this technology center on successful integration with
planned ship systems. The Navy has identified backup technologies for each
of these technologies.
Only one critical technology, the 1,100-ton air conditioning plants, is
not planned to be near maturity by construction start. Program officials
believe the plants will reach mature levels shortly after the start of
construction. Risks associated with the plants are considered low by
officials since the technology being used is derived from commercial
applications and enhancements leveraging experience from plants found on
other US Navy ships.
Design Stability
The CVN-21 program is currently planning a design review date for March
2007. Rather than measuring design stability by percentage of engineering
drawings completed, the program uses an alternative metric that measures
earned hours completed in product model development. As a result we could
not assess the ship's design stability.
Other Program Issues
The program has delayed delivery of both the first and second ship by
adding one year to the development schedule. According to program
officials, the Navy made this decision with the intent to balance ship
construction dollars in the President's fiscal year 2006 budget. Research
and development funds were added to the program to bridge the additional
year, which allows additional time and funding to mature technologies in
the program. The one year shift does create an additional gap, where the
Navy will have to operate with only 11 carriers, between de-commissioning
of the USS Enterprise aircraft carrier and delivery of the first CVN-21 to
the fleet.
Agency Comments
In commenting on a draft of this assessment, the Navy emphasized that,
based on product model development progress, the CVN-21 program's overall
design was 44 percent complete as of November 2005 and that the program is
on schedule to support the construction of the lead ship. In addition, the
department said that although there was a one year slip based on decisions
to balance ship construction in the President's fiscal year 2006 budget,
technology development efforts were unaffected and remain on track.
The Navy's DD(X) destroyer is a multimission surface ship designed to
provide advanced land attack capability in support of forces ashore and
contribute to U.S. military dominance in littoral operations. The program
recently completed the system design phase and was authorized to begin
detail design and construction of the lead ships in November 2005. The
program will continue to mature its technologies and design as it
approaches construction.
Source: DD(X) Program Office (PMS 500).
Program Development Production GAO Initial
start start decision-1st ships review capability
(1/98) (3/04) (11/05) (1/06) (1/13)
Since last year's assessment, the program completed demonstrations of a
number of its 12 critical technologies. One of the technologies was fully
mature by the November 2005 production decision. Eight technologies were
demonstrated in a relevant environment and are near full maturity. Some of
these technologies will not be fully mature until after installation on
the first ship as testing in an operational environment is not considered
feasible. The integrated deckhouse, ship computing system, and volume
search radar are at lower levels of maturity, having completed component
level demonstrations. The Navy approved the system design to proceed into
the next phase, but a number of risks remain in both design and technology
that could lead to changes.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (3/04) review
(11/05) (1/06) (9/05)
Technology Maturity
At the November 2005 production decision, one of 12 critical technologies
for DD(X) was fully mature. While completion of tests in 2005 advanced the
maturity of technologies, development continues as the program proceeds
with detail design. Eight technologies, the advanced gun system and its
projectile, autonomic fire suppression, hull form, infrared suppression,
integrated power system, multifunction radar, and peripheral vertical
launch system are short of full maturity but have been demonstrated in a
relevant environment. Program officials state that the undersea warfare
system is fully mature, based on the use of mature components. However,
the components will not be integrated and tested together until after ship
installation. Due to practical limitations some of these technologies, the
advanced gun system and its projectile, hull form, and infrared
suppression, will not be fully demonstrated until after installation on
the lead ship.
The integrated deckhouse, volume search radar, and ship computing system
are at lower levels of maturity. The program tested a physical model of
the deckhouse for stealth requirements and placement of apertures to
minimize interference, but with only a portion of the apertures expected.
Analysis of deckhouse resilience to fire and shock has been completed, and
will be tested during detail design. The volume search radar will require
additional development to increase performance, which may aggravate an
already aggressive schedule. Software development has been progressing as
planned, although about three-quarters of the effort remains.
Design Stability
The metric for design maturity used in other programs does not apply to
DD(X), and therefore the program was not assessed according to this
metric. Instead the program assesses design stability by reviewing design
artifacts, which include items like system drawings, ship specifications,
and major equipment lists. The program office states that all 2010 design
artifacts are complete, though some may be altered as systems continue to
mature or are changed to meet cost reduction goals.
On September 14, 2005 the Navy completed the critical design review of
DD(X) and approved the start of detail design. Risk remains in the system
design due to issues in the power system, deckhouse, and hull form. The
concern with the power system is ensuring the design meets limits on space
and weight. As this system is needed early in construction, it could have
an impact on schedule if not resolved quickly. A number of systems in the
deckhouse, including the volume search radar and electronic warfare
system, are still in development and design of the deckhouse could be
affected if they exceed margins for weight and space. Furthermore, due to
the hull form's unique design, it has reduced stability in very severe
weather conditions. Program officials state they can reduce this risk
through guidance that helps the crew avoid these conditions. Model testing
for heavy sea conditions also revealed some areas which may require
strengthened structure, and program officials believe this can be
corrected.
Agency Comments
The Navy stated that the design, development and testing of critical
technologies mitigated the significant technical risks prior to critical
design review. The DD(X) ship design remained stable throughout critical
technology testing, successfully incorporating all necessary component
modifications and entering detail design with adequate weight margin. A
comprehensive test program will address all remaining risk areas described
in the report.
The Navy further noted that given the unique nature of shipbuilding, with
detail design and construction spread over 5 years, comparing DD(X)
technology readiness levels to the GAO-developed best practices is not
valid. DD(X) technology readiness levels met current acquisition policy
guidance in support of the decision to proceed into system development in
November 2005.
GAO Response
Our approach is valid because our work has shown that technological
unknowns discovered late in development lead to cost increases and
schedule delays. Some of the technologies still under development for
DD(X) could have major impact on ship design and construction schedules.
The Navy's E-2D AHE is an all-weather, twin engine, carrier-based,
aircraft designed to extend early warning surveillance capabilities. It is
the next in a series of upgrades the Navy has made to the E-2C Hawkeye
platform since its first flight in 1971. The E-2D AHE is designed to
improve battle space target detection and situational awareness,
especially in littoral areas; support Theater Air and Missile Defense
operations; and improve operational availability.
Source: Program Executive Office, Tactical Aircraft Programs(PMA-231).
Program/ Design GAO Low-rate Initial Full-rate Last
development start review review decision capability decision procurement
(6/03) (10/05) (1/06) (3/09) (4/11) (12/12) (2020)
The E-2D AHE program entered system Attainment of Product Knowledge
development in June 2003 without demonstrating Production, that its four
critical technologies had reached full design &
technology
maturity. Since that time, one of the program's maturity four critical
technologies has reached full maturity. The program expects the remaining
three critical technologies to mature before the Design &
technologyproduction decision in March 2009. While more maturity mature
backup technologies exist for the three critical technologies, use of the
backup technologies would result in degraded system performance or reduced
ability to accomodate Technologyfuture system growth. The design met best
maturity practice standards at the time of design review in October 2005.
However, until all the technologies are mature, the potential for design
changes remains. We could not assess production maturity because the
program does not plan to use
statistical process controls. Development DOD GAO Production start design
review decision (6/03) review (1/06) (3/09) (10/05)
Technology Maturity
One of the E-2D AHE's four critical technologies (the space time adaptive
processing algorithms and associated processor) is mature. The program
expects the remaining technologies (the rotodome antenna, a silicon
carbide-based transistor for the power amplifier to support UHF radio
operations, and the multichannel rotary coupler for the antenna) to be
fully mature before the start of production in March 2009.
More mature backup technologies exist for the three technologies (the
rotodome antenna, the silicon carbide-based transistor, and the
multichannel rotary coupler) and were flown on a larger test platform in
2002 and 2003. However, use of the backup technologies would result in
degraded system performance or reduced ability to accomodate future system
growth due to size and weight constraints. The next AHE technology
readiness assessment is to be performed prior to the production decision
in fiscal year 2009, and the program office anticipates that the critical
technologies will be mature at that time.
Design Stability
The program had completed 90 percent of its engineering drawings at the
Critical Design Review, which was completed on October 21, 2005. Program
officials project that they will have 100 percent completed by the planned
start of production in March 2009. However, the technology maturation
process may lead to more design changes.
Production Maturity
The program expects a low-rate production decision in March 2009, but does
not require the contractor to use statistical process controls to ensure
its critical processes are producing high quality and reliable products.
According to the program, the contractor assembles the components using
manual, not automated, processes that are not conducive to statistical
process control. The program relies on post-production data, such as
defects per unit, to track variances and non-conformance. The program also
conducts production assessment reviews every 6 months to assess the
contractor's readiness for production. The program has updated the
manufacturing processes that were established and used for the E-2C over
the past 30 years. The program considers the single station joining tool;
the installation of electrical, hydraulic and pneumatic lines; and the
installation of the prime mission equipment all critical manufacturing
processes.
The program is currently building the first two development aircraft.
Accordng to the program office, there are no significant differences in
the manufacturing processes for the development aircraft and the
production aircraft.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the E-2D
AHE program successfully executed all component and subsystem design
reviews, culminating in the successful completion of the weapon system
design review in October 2005. This review included a thorough evaluation
of the four critical technologies and all program risks. According to the
Navy, critical technologies do not represent a high risk to the AHE
program at present.
Flight testing, which will include the four critical technologies, is
planned to begin in the fourth quarter of fiscal year 2007. The test
program expects to demonstrate the design maturity of all technologies and
capabilities at that time. A Technology Readiness Assessment will be
conducted prior to the low rate production decision.
According to the Navy, integration of statistical process controls would
require significant investment to update the E-2D aircraft manufacturing
process. The Navy has elected not to make this investment due to the
maturity and over 30 years of E-2 production.
The Air Force's EELV program acquires commercial satellite launch services
from two competitive families of launch vehicles-Atlas V and Delta IV.
Initiated as an industry partnership, the program's goal is to support and
sustain assured access to space and reduce the life-cycle cost of space
launches by at least 25 percent over previous systems while meeting the
government's launch requirements. A number of variants are available
depending on the lift capability necessary for each mission. We assessed
both the Atlas V and Delta IV.
Source: (Left) (c) 2005 ILS/Lockheed Martin; (right) (c) 2003 The Boeing
Company.
Program Development Production First First GAO Initial
flight- flight-
start start decision Atlas V Delta IV review capability
(12/96) (10/98) (unknown) (8/02) (11/02) (1/06) (TBD)
While the EELV program office has access to technology, design, and
production maturity information, it does not collect this information
because it is buying the launch service. To date, eleven successful
launches have occurred-three government and eight commercial. A technical
review was completed, and the program is implementing corrective actions
to eliminate the cause of an earlier-than-expected engine shutdown during
the Delta IV Heavy Lift Vehicle launch demonstration. The EELV program's
total costs have increased due to a decline in the commercial launch
market upon which the business case was based.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (10/98) review
(1/06) (unknown) (10/99)
Technology Maturity
We could not assess the technology maturity of EELV because the Air Force
has not formally contracted for information on technology maturity from
its contractors.
Design Stability
We could not assess the design stability of EELV because the Air Force has
not formally contracted for the information needed to conduct this
assessment.
Production Maturity
We could not assess the production maturity of EELV because the Air Force
has not formally contracted for information that would facilitate this
assessment.
Other Program Issues
A decline in commercial launch demand for the EELV launch vehicles
resulted in a cost increase of more than 25 percent over the program's
objective and triggered a Nunn-McCurdy breach (10 U.S.C. 2433), that
required DOD to certify in 2004 that the program is critical to national
security and its cost estimates are reasonable. In conjunction with the
certification, the Air Force revised its mission model to reflect a
reduction of launch vehicles, conducted a study on assured access to
space, and revised its acquisition strategy.
DOD continues to be the primary user of EELV launch services due to the
decrease in commercial demand for launches--which has resulted in a
reduction in the number of launch vehicles needed. An Air Force study on
assured access to space addressed concerns about retaining both EELV
launch providers given the limited number of launches. To ensure access to
space with two distinct launch vehicles and address the decline in
commercial launch demand, the government has agreed to share a level of
risk with the launch providers through a new acquisition strategy. The new
strategy provides for a contracting approach that supports each
contractor's annual infrastructure through a launch capability contract
and replaces price-based competition with an annual award of launch
service contracts. In April 2005, the Air Force released a Request for
Proposals for EELV Launch Services and EELV Launch Capabilities Contracts.
The Air Force planned to award the contracts by October 2005. However,
this has been delayed. Negotiations are currently ongoing for the launch
capability contracts, and updated proposals are being submitted for the
launch services contracts covering the anticipated fiscal year 2006 launch
awards.
In August 2005, competitors Boeing and Lockheed Martin submitted a request
to the Federal Trade Commission for an antitrust review to support a joint
venture. This was withdrawn and resubmitted in September 2005. The Federal
Trade Commission has requested additional information to support the
review. The joint venture will combine their production, engineering,
test, and launch operations for all U.S. government launch activity. Both
contractors will share equally in the profits and costs of all government
launches.
The EELV program is taking corrective action to address a problem with the
liquid oxygen feed line that falsely indicated propellant depletion and
resulted in an early engine shutdown of the first stage engine on the
Delta IV during the Heavy Lift Vehicle Operational Launch Service
Demonstration that occurred in December 2004.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that it
collects data for technology, design, and production maturity. However,
the Air Force has not contracted for delivery of this data and therefore
does not have authority to provide this information. Program officials
also provided technical comments, which were incorporated where
appropriate.
The Marine Corps' EFV is designed to transport troops from ships offshore
to their inland destinations at higher speeds and from longer distances
than the system it is designed to replace, the Assault Amphibious Vehicle
7A1 (AAV-7A1). The EFV will have two variants---a troop carrier for 17
combat equipped Marines and 3 crew members and a command vehicle to manage
combat operations in the field. We assessed both variants.
Source: General Dynamics Land Systems.
Program Development Design GAO Low-rate Full-rate Initial Last
start start review review decision decision capability procurement
(3/95) (12/00) (1/01) (1/06) (9/06) (8/10) (9/10) (2018)
The EFV's technologies are mature and the design Attainment of Product
Knowledge is stable. Early development of fully functional Production,
prototypes facilitated design stability. Technical design &
technology
problems have been encountered, and system maturity reliability
requirements have been reduced; plans are to fully demonstrate all
requirements in fiscal year 2010. Fixes for technical problems have been
Design &
technology
identified and corrective actions are in place. maturity
Production maturity remains a concern because
the contractor will not start collecting statistical
process control data until after production starts,
and the software development effort is a Technology
continuing challenge. A fourth program maturity
restructuring has resulted in a 2-year schedule
increase and about a $2-billion increase in cost.
The program office has had reduced insight into
its prime contractor's work progress since
December 2004 because it has not received
detailed earned value cost and schedule data. Development DOD GAO
Production
start design review decision (12/00) review (1/06) (9/06) (1/01)
Technology Maturity
All five of the EFV system's critical technologies are mature and have
been demonstrated in a full-up system prototype.
Design Stability
The program has now released all of its drawings for the troop carrier and
command variants, but anticipates that about 12 percent of the drawings
will require changes to address reliability issues. While reliability
requirements have been reduced, the program office expects to fully
demonstrate both reliability and interoperability--key performance
parameters--during initial operational testing and evaluation in fiscal
year 2010. The program expects to hold a final design review in 2010 that
will include any reliability design changes. Furthermore, testing in the
early system design and demonstration phase revealed problems in the hull
electronic unit, bow flap, and hydraulics. In addition, problems with the
hardware and software modules caused unsafe testing conditions---the EFV
prototypes made turns without a direct command. After about a two-month
delay to address these problems, full vehicle testing was resumed.
According to the program office, corrective actions for all these problems
have been identified.
Production Maturity
The program plans to enter low-rate initial production in September 2006.
However, the program office does not plan to require the contractor to
collect statistical process control (SPC) data until after the start of
low-rate-initial production to demonstrate that critical manufacturing
processes will produce products within cost, schedule, performance, and
quality targets. The program office is still in the planning stages for
its production readiness reviews that will assess the production
processes, identify any additional critical manufacturing processes, and
determine the benefit of using SPC. According to the program office, to
date, no suppliers have collected SPC data for EFV-unique components, but
some suppliers are collecting SPC data on high-volume commercial parts
used on the EFV. Twelve critical processes have already been identified
and more are expected.
Other Program Issues
The EFV program relies on software to provide all electronic, firepower,
and communication functions. The program is collecting metrics relating to
cost, schedule, and quality; is using an evolutionary development
approach; and has set and completed about 98 percent of the software
requirements for the pending early operational assessment. Nevertheless,
software development continues to present a risk. The program has already
experienced growth in the size and cost of the software development as
well as schedule delays. The program manager recognized the risk and has
initiated a software risk mitigation plan.
DOD's December 2004 budgetary action to reallocate funding for higher
priorities served as the basis for the EFV's fourth rebaselining. However,
according to the program manager, a testing schedule slip of about 15
months would have been needed even without DOD's budgetary action because
additional time would have been needed after the start of low-rate initial
production in September 2006, for more robust reliability testing,
production qualification testing and training. DOD's budgetary action
resulted in a 24-month schedule increase---possibly 9 months more than
would have been needed---and a cost increase of about $2 billion.
According to the program office, the rebaselining effort was required in
order to make the EFV program executable.
The program office has not collected and managed the program with detailed
earned value management data since it began restructuring the program in
December 2004. According to a program official, the earned value
management system was reestablished after a new contractor schedule was
approved in December 2005.
Agency Comments
The program office provided technical comments, which were incorporated,
as appropriate. The Navy concurred with our assessment of the EFV program.
Further, the program manager believes the EFV program is on track to begin
a comprehensive operational assessment in January 2006 and to begin its
low-rate initial production review in September 2006.
Source: PM Excalibur and Raytheon.
Low-rate GAO Full-rate Initial Last
decision review decision capability procurement
(4/05) (1/06) (9/08) (9/08) (2020)
Attainment of Product Knowledge
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Datanot available
The Army's Excalibur is a family of global positioning system-based,
fire-and-forget, 155-mm cannon artillery precision munitions. It is
intended to improve the accuracy and range of cannon artillery. Also, the
Excalibur's near vertical angle of fall is intended to reduce the
collateral damage area around the intended target, making it more
effective in urban environments than the current artillery projectiles.
The Future Combat Systems' non-line-of-sight cannon requires the Excalibur
to meet its required range.
Program/ development start (5/97)
The Excalibur program is proceeding into early production to support an
urgent early fielding requirement in Iraq for more accurate artillery that
will reduce collateral damage. This early production run of the
Excalibur's first block will involve 180 rounds, with planned fielding by
the last quarter of 2006. According to program officials, Excalibur's
critical technologies reached full maturity in May 2005, and all of its
790 drawings were completed in July 2005. The Excalibur unitary variant
will be developed in three blocks, which will incorporate increased
capabilities and accuracy over time. Since our last assessment, the
planned quantities have been cut in half. The program continues to
experience increasing unit cost as quantities are lowered.
Development DOD Production GAO start design decision review (5/97) review
(4/05) (1/06) (N/A)
Technology Maturity
The Excalibur program is developing its unitary variant in three blocks.
All three of the unitary variant's critical technologies reached full
technology maturity in May 2005 at the time of the Excalibur's design
review. These technologies were the airframe, guidance system, and
warhead.
Design Stability
Excalibur's design appears to be stable because at the time of the May
2005 design review, 750 of 790 design drawings were releasable. All the
drawings were complete for the first Excalibur block in July 2005. The
second block is expected to have a similar number of drawings, and it is
unknown how many drawings will be involved with the third block.
Production Maturity
We could not assess Excalibur's production maturity. The first block has
entered limited production, to support an urgent fielding requirement in
Iraq, without statistical control data. The program plans to collect
statistical data during production of all blocks. Production of the second
block is scheduled for fiscal year 2007 and the third block in fiscal year
2010.
Other Program Issues
The program has encountered a number of changes since development began in
1997, including a decrease in planned quantities, a relocation of the
contractor's plant, early limited funding, technical problems, and changes
in program requirements. It was almost immediately restructured due to
limited funding, and it was restructured again in 2001. The program was
again restructured in 2002 and merged with a joint Swedish/U.S. program
known as the Trajectory Correctable Munition. This merger has helped the
Excalibur deal with design challenges, including issues related to its
original folding fin design. In May 2002, due to the cancellation of the
Crusader, the Army directed the restructure of the program to include the
Future Combat Systems' Non-Line-of-Sight Cannon.
In December 2002, the Acting Under Secretary of Defense (Acquisition,
Technology, and Logistics) approved an early fielding plan for the unitary
version. The plan currently includes developing the unitary version of the
Excalibur in three blocks. In the first block, the projectile would meet
its requirements for accuracy in a non-jammed environment and lethality
and would be available for early fielding. In the second block, the
projectile would be improved to meet its requirements for accuracy in a
jammed environment and reliability and would be available for fielding to
the Future Combat System's Non-Line-of-Sight Cannon in September 2008 or
when the cannon is available. Finally, in the third block, the projectile
would be improved to meet its range requirement and would be available for
fielding to all systems in late fiscal year 2011.
The net effect of these changes has been to lengthen the program's
schedule and to substantially decrease planned procurement quantities. As
a result, the program's overall costs and unit costs have dramatically
increased.
Agency Comments
In commenting on the draft, the Army noted that Excalibur started as a
combination of three smaller artillery-related programs with the intent to
extend range capability with an integrated rocket motor. The current
Excalibur program will allow three different Army howitzers to fire
farther away and defeat threats more quickly, lowering collateral damage
while reducing the logistic support burden. Recent program achievements
include a production decision for the first block configuration to support
early fielding to the multinational combat forces in Iraq, and successful
tests demonstrated both proximity and point-detonating modes approximately
5 meters from the target.
The Air Force's F-22A, originally planned to be an air superiority
fighter, will also have air-to-ground attack capability. It is being
designed with advanced features, such as stealth characteristics, to make
it less detectable to adversaries and capable of high speeds for long
ranges. It also has integrated aviation electronics (avionics) designed to
greatly improve pilots' awareness of the situation surrounding them. It is
designed to replace the Air Force's F-15 aircraft.
Source: F-22A System Program Office.
Program Development Low-rate Full-rate Initial GAO Last
start start decision decision capability review procurement
(10/86) (6/91) (8/01) (4/05) (12/05) (1/06) (2009)
The F-22A entered production without ensuring that production processes
were in control. In December 2004, the Secretary of Defense reduced F-22A
procurement quantites from 279 to 183. Since our last assessment of the
program, the Air Force held a full rate decision in April 2005. At that
time, about 42 percent of the aircraft were already on contract.
Technology and design matured late in the program and have contributed to
numerous problems. Avionics problems were discovered late in development,
which resulted in large cost increases and caused testing delays. The Air
Force completed initial operational test and evaluation in December 2004
and identified several deficiencies that required modfications to the
aircraft's fuel system, canopy transparency, and the applications of low
observable materials.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (6/91) review
(8/01) (1/06) (2/95)
Technology Maturity
The three critical F-22A technologies (supercruise, stealth, and
integrated avionics) appear to be mature. However, two of these
technologies, the integrated avionics and stealth, did not mature until
several years after the start of development. Integrated avionics has been
a source of major problems, delaying developmental testing and the start
of initial operational testing. Since 1997 the costs of avionics has
increased by over $951 million or 24 percent and problems discovered late
in the program were the major contibutor. In April 2004, the Air Force
began initial operational test and evaluation after reporting that these
problems were corrected.
Design Stability
The F-22A design is essentially complete, but it matured slowly, taking
over 3 years beyond the critical design review to meet best practice
standards. The late drawing release contributed to parts shortages, work
performed out of sequence, delayed flight testing, and increased costs.
Design changes resulted from flight and structural tests. For example,
problems with excessive movement of the vertical tails and overheating
problems in the fuselage and engine bay required design modifications. The
Air Force completed initial operational testing in December 2004 and
development testing in December 2005. There were several design changes
required to the aircraft as a result of operational testing. These
included changes to improve the application of low observable materials,
modifications to improve the durabilty of canopy transparencies, and
implemented software improvements to the diagnostic health management
system.
Production Maturity
The program office stopped collecting process control information in
November 2000. The contractor estimated that nearly half of the key
processess had reached a marginal level of control, but not up to best
practice standards. The Air Force has 98 production aircraft on contract.
The Air Force relies on the contractor's quality system to verify
manufacturing and performance requirements are being met. However, the Air
Force has not demonstrated the F-22A can achieve its reliability goal of 3
hours mean time between maintenance. It does not expect to achieve this
goal until the end of 2009 when most of the aircraft will have already
been bought. Best practices call for meeting reliability requirements
before entering production. At the conclustion of initial operational test
and evaluation in December 2004, the Air Force had only demonstrated about
15 percent of the reliability required to meet the current operational
requirement.
Other Program Issues
The Air Force is counting on $2.2 billon in future cost reduction plans to
offset estimated cost growth and enable the program to meet the latest
production cost estimate. If these cost reduction initiatives are not
achieved as planned, production costs could increase.
In January 2005, the Air Force Operatonal Test and Evaluation reported the
F-22A was "overwhelmingly effective" as an air superiority fighter and
that its support systems were "potentially suitable." Some deficiences
were noted, particuarly in reliability and maintainability. In August
2005, the Air Force begin follow-on test and evaluation, which is designed
to demonstrate limited air-to-ground capability and correct the
deficiencies identified during initial operational test and evaluation.
The F-22A declared initial operational capability in December 2005.
Agency Comments
The Air Force provided technical comments, which were incorporated as
appropriate.
The FCS, a program that will equip the Army's new transformational modular
combat brigades, consists of a family of systems composed of advanced,
networked combat and sustainment systems, unmanned ground and air
vehicles, and unattended sensors and munitions. Within a system-of-systems
architecture, the first increment of the FCS features 18 major systems and
other enabling systems along with an overarching network for information
superiority and survivability.
Source: Program Manager, Unit of Action, U.S. Army.
Program Development GAO Design Low-rate Initial Full-rate Last
start start review review decision capability decision procurement
(5/00) (5/03) (1/06) (9/10) (9/12) (12/14) (8/16) (unknown)
The FCS program has not demonstrated high levels of knowledge.
Requirements have been sent to system developers to begin preliminary
designs, but program officials say they may change due to feasibility and
affordability constraints. Three years after system development began,
none of FCS' 49 critical technologies are fully mature. Technology
maturation will continue throughout system development, with an associated
risk of cost growth and schedule delays. Based on program office
estimates, the cost of the restructured FCS program has grown
substantially. Earlier cost estimates were based on lower levels of
program knowledge and undefined requirements. Higher levels of knowledge
have resulted in a more realistic and higher cost estimate. Since the FCS
dominates Army investment accounts over the next decade, further cost
growth and schedule delays could affect other Army acquisitions.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (5/03) (1/06)
review (9/12) (9/10)
Technology Maturity
Since our last assessment of FCS, the program assembled an independent
review team to assess critical technologies. Although a few technologies
appear to have matured, most have either shown no improvement or are now
assessed less mature. None of the FCS program's critical technologies are
fully mature and only 18 technologies are nearing full maturity.
The program is no longer reporting on 5 of the 54 technologies from a year
ago. According to FCS officials, they met last year with the users'
representative, the Army Training and Doctrine Command, to discuss FCS
requirements and critical technologies. They agreed that a few of the
listed technologies were in fact not technologies. Instead, they were
capabilities that could be satisfied by existing or planned assets.
The FCS program is not following best practices in maturing its
technologies. The program's approach involves integration phases that
allow a staggered start for technologies to be "spun out" to current
forces. However, program officials allow technologies to be included in
the integration phases before they are mature. Further, as currently
scheduled, all four of the currently-planned integration phases will have
begun before the program has a preliminary design review for the system of
systems, and each of those integration phases will likely begin with
immature technologies. As a result, the program will involve concurrent
technology and product development and face the associated risks of such
an approach. Furthermore, the individual integration phases will not be
subjected to the milestone decision process.
Other Program Issues
The cost of the restructured FCS program has increased substantially,
based on program office estimates. An independent cost estimate will not
be completed until the spring of 2006. Projected procurement costs have
increased over 50 percent. Earlier cost estimates were based on lower
levels of program knowledge and undefined requirements. Higher levels of
knowledge, such as more defined design concepts for the manned ground
vehicles and progress in requirements definition, have produced a higher
fidelity cost estimate. The Army has adopted an initiative to
substantially reduce FCS procurement acquisition costs. However,
requirements may have to be reduced accordingly.
In August 2005, the FCS program completed the System of Systems Functional
Review. This event demonstrated that the Army understands FCS system of
systems requirements and is prepared to begin preliminary individual
system designs. Although it is a significant achievement, the program
should have demonstrated this level of knowledge 3 years ago to support
the decision to start development. In addition, program officials say they
are reserving the right to reduce requirements, pending user approval, if
technologies do not mature as planned or if satisfying a particular
requirement is not affordable. The requirements uncertainty and immature
state of technologies make the FCS acquisition approach risky.
Furthermore, successful operation of FCS-equipped Units of Action depends
on the contributions of up to 170 complementary and associated programs.
FCS will utilize these systems to help satisfy FCS operational
requirements. However, according to program officials, the list of
complementary programs continues to evolve and some are unfunded.
Agency Comments
In commenting on a draft of this assessment, the Army stated that
technology maturity is a key aspect of the process of deciding when a
technology is provided to current forces. It also stated that while the
individual integration phases will not be subjected to the milestone
decision process, the program will have annual reviews with the milestone
decision authority. The Army commented that the program is primarily
focused on about 52 critical complementary programs considered essential
to meeting the top-level key performance parameters.
GAO Response
Although technology maturity may drive decisions on providing technologies
to the current forces, the Army's definition of mature technology is below
the best practices standard. Our prior work has shown that when programs
proceed into development with technologies that do not comply with best
practices, they are exposed to an increased risk of cost growth and
schedule delays. Also, the list of critical complementary programs
continues to evolve, and the program must manage the associated cost,
schedule, and performance gaps that may result.
The Air Force's Global Hawk system is a high altitude, long endurance
unmanned aerial vehicle with integrated sensors and ground stations
providing intelligence, surveillance, and reconnaissance capabilities.
After a successful technology demonstration, the system entered
development and limited production in March 2001. Considered a
transformational system, the program was restructured twice in 2002 to
acquire 7 air vehicles similar to the original demonstrators (the RQ-4A)
and 44 of a new, larger, and more capable model (the RQ-4B).
Source: Northrop Grumman Corporation.
Demonstration Development start/ Initial GAO Full-rate Last
program start low-rate decision capability review decision procurement
(2/94) (3/01) (12/05) (1/06) (5/07) (2012)
Key product knowledge on Global Hawk is lower now than in March 2001 due
to program restructurings. Under the original plan to produce aircraft
very similar to demonstrators and slowly acquire advanced systems,
technology maturity and design stability were near best practices
standards. Program restructurings, however, added the new RQ-4B aircraft
and advanced sensors, overlapped development and production schedules, and
accelerated planned deliveries. The new technologies are still maturing
and the RQ-4B design required extensive changes. Officials are
implementing statistical process controls, but data is incomplete. In
November 2004, we reported significant risks from gaps in product
knowledge and recommended reducing near-term RQ-4B buys to only those
needed for testing. The program is now experiencing development and
procurement cost increases, schedule delays, and quality problems.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Development GAO
start design start/production review
(NA) review decision (1/06)
(NA) (3/01)
Technology Maturity
Of the Global Hawk's 13 critical technologies, 6 are mature by best
practices standards; 3 are approaching maturity; and 4 are less mature.
The less mature technologies include enhanced imagery and signals
intelligence sensors and improved radar. The desire for these capabilities
drove the decision to develop and acquire the new RQ-4B aircraft, which
can carry 50 percent more payload than the RQ-4A. Integrating and testing
advanced sensors won't be completed until late in the program after most
of the fleet has already been bought. If space, weight, and power
limitations or other performance issues surface as technologies mature,
the program may experience costly rework, extended development times, or
diminished capabilities.
Design Stability
Program officials reported achieving the best practice standard for design
drawings approved for manufacturer release in October 2004, shortly after
RQ-4B production began. However, during the first year of production,
there were more than 2,000 authorized drawing changes to the total
baseline of 1,400 drawings. More than half of those changes were
considered major, requiring model changes. Substantial commonality between
the A- and Bmodels had been expected, but as designs were finalized and
production geared up, design differences were more extensive and complex
than anticipated. By the time of our review, design deficiencies,
engineering changes, and work delays had contributed to a development
contract cost overrun of $209 million. Adding to design risk, the Air
Force plans to buy almost half the RQ-4B fleet before it completes
operational tests to verify the aircraft design.
Production Maturity
The contractor has completed RQ-4A production and is fabricating the first
RQ-4Bs. Program and contractor officials are in the process of
implementing statistical process controls.They've identified critical
manufacturing processes and started to collect data for demonstrating that
new processes are capable of meeting cost, schedule, and quality targets.
Officials also collect and analyze other performance indicators such as
defects and rework rates to monitor manufacturing quality.
Technology immaturity, increased cost for sensors, and extensive design
changes contributed to higher RQ-4B production costs than forecast.
Although improving, there have been recurring concerns about the
performance and work quality of several key subcontractors. The
subcontractor building the tail scrapped seven of the first eight main box
spars due to design maturity and process issues. The wing manufacturer
terminated its subcontractor due to poor performance and quality;
subsequently completed wings passed proof load testing and were installed
onto RQ-4B aircraft.
Other Program Issues
With RQ-4B costs increasing and schedules slipping, the Global Hawk
program is rebaselining, its fourth since the March 2001 start. In April
2005, the Air Force notified the Congress of a Nunn-McCurdy breach (see
U.S.C. 2433) with an 18-percent unit procurement cost increase over the
current baseline. Further cost increases are expected. In December 2005,
we reported that the Nunn-McCurdy notice to Congress did not include
$400.6 million (in base year 2000 dollars) budgeted for retrofit
activities, including the procurement and installation of signal
intelligence sensors in already-built aircraft. Including this amount
would increase procurement unit cost growth to 31 percent and require the
Secretary of Defense to certify the program to Congress.
Agency Comments
In commenting on a draft of this product, Air Force officials partially
concurred and offered technical comments that we incorporated where
appropriate. They emphasized Global Hawk's early and continuing support to
military operations in Iraq and Afghanistan with about 5,000 combat hours
flown by demonstrator aircraft. They stated that DOD conducts
comprehensive and forward-looking oversight, understands the risks and
benefits, and implements an appropriate acquisition strategy to mitigate
risk. Software, not hardware, is the critical element to the RQ-4B
capability, drives the deployment schedule, and represents the chief
technical and management challenges. Radar and signals sensors are the two
critical technologies and portend revolutionary capability improvement.
Each payload has a dedicated program office and contractor. Payload
integration includes test and decision points to evaluate progress.
MDA's GMD element is being developed to defend the United States against
limited long-range ballistic missile attacks. The first block, Block 2004,
consists of a collection of radars and interceptors, which are integrated
by a central control system that formulates battle plans and directs the
operation of GMD components. We assessed the maturity of all technologies
critical to the Block 2004 GMD element, but we assessed design and
production maturity for the interceptors only.
Source: Department of Defense.
Program Directive to field Integrated Initial Block 2004 GAO
start initial capability design review capability completion review
(2/96) (12/02) (3/03) (8/04) (12/05) (1/06)
Even though only 6 of GMD's 10 critical technologies are fully mature, MDA
released all hardware drawings to manufacturing and had 10 Block 2004
interceptors in silos for operational use by December 2005. However,
ongoing efforts to mature technologies, along with concurrent testing and
fielding efforts may lead to additional design changes. Although MDA is
producing hardware for operational use, it has not made a formal
production decision. Additionally, we could not assess the stability of
the production processes because the program is not collecting statistical
data for them. We expect that the prime contract could overrun its target
cost by as much as $1.5 billion.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production
start design review decision
(NA) review (1/06) (TBD)
(3/03)
Technology Maturity
Program officials assessed all of GMD's 10 critical technologies as
mature. However, four have not been demonstrated in an operational
environment and we believe that they cannot be considered fully mature.
Mature technologies include the fire control software, the exoatmospheric
kill vehicle (EKV) infrared seeker, the Orbital Sciences Corporation
booster, the Cobra Dane radar, the EKV guidance, navigation, and control
subsystem; and the in-flight interceptor communications system. The
remaining four technologies are nearing maturity. These technologies are
the Beale radar; EKV discrimination; the sea-based X-band radar; and the
BV+ booster. The program expected to demonstrate all remaining
technologies in an operational environment by December 2005, but flight
test delays and failures prevented the demonstrations. The program now
plans to demonstrate the remaining four technologies by the end of 2007.
Design Stability
Technology issues aside, the design of the Block 2004 ground-based
interceptor appears stable with 100 percent of its drawings released to
manufacturing. However, ongoing efforts to mature technologies and the
concurrent testing and fielding efforts may lead to additional drawings
and design changes.
Production Maturity
Officials do not plan to make an official production decision, although
they are delivering interceptors for the Block 2004 emergency capability.
We could not assess the maturity of the production processes for these
interceptors because the program is not collecting statistical control
data. According to program officials, data are not tracked because current
and projected quantities of GMD component hardware are low. Instead, the
GMD program measures production capability and maturity with a monthly
evaluation process called a Manufacturing Capability Assessment that
assesses critical manufacturing indicators for readiness and execution.
MDA delivered 5 interceptors for the initial capability by September 2004,
and it had 10 interceptors ready for alert by December 2005. MDA planned
to have 18 interceptors fielded by this time; however, 4 interceptors
procured for fielding were later designated as test assets and production
of 4 others was delayed as quality control improvements were implemented.
Qualification of a new BV+ booster propellant subcontractor has been
completed ending a 2 year slowdown in BV+ activities. MDA plans to procure
the eight BV+ boosters currently under contract, but these interceptors
will not be fielded until Block 2006.
Other Program Issues
GMD's prime contractor, Boeing, has overrun its budget by $600 million,
primarily because of quality issues that delayed flight and ground tests.
Although Boeing expects the large unfavorable cost variances to improve as
flight testing resumes, we anticipate that the contract will overrun its
target cost by as much as $1.5 billion. Since our last assessment, GMD's
planned budget through fiscal year 2009 has increased by $2.9 billion
(11.2 percent), primarily in fiscal years 2008 and 2009.
Agency Comments
The program office provided technical comments to a draft of this
assessment, which were incorporated as appropriate.
Source: Navstar GPS Joint Program Office, Space and Missile Systems Center.
Production GAO First satellite Initial
decision review launch capability
(7/02) (1/06) (1/07) (N/A)
Attainment of Product Knowledge
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Development DOD Production GAO
start design decision review
(2/2000) review (7/02) (1/06)
(NA)
GPS is an Air Force-led joint program with the Army, Navy, Department of
Transportation, National Geo-Spatial Intelligence Agency, United Kingdom,
and Australia. This space-based radio-positioning system nominally
consists of a 24-satellite constellation providing navigation and timing
data to military and civilian users worldwide. In 2000, Congress approved
the modernization of Block IIR and Block IIF satellites. In addition to
satellites, GPS includes a control system and receiver units. We focused
our review on the Block IIF.
Program Development start start (1/99) (2/00)
According to the program office, the Block IIF technologies are mature.
Since the start of the GPS program in 1973, GPS satellites have been
modernized in blocks with the newer blocks providing additional
capabilities and benefits. The space-qualified atomic frequency standards
for the Block IIF satellites are mature but considered a critical
technology because there is no backup technology for these clocks. The
contractor was not required to provide data on design drawings, and
statistical process control techniques are not being used to monitor
production. As a result, design stability and production maturity could
not be assessed. R&D cost growth amounted to $399 million (20.4 percent)
for satellite component modernization and the control system, and
procurement cost growth amounted to $717 million (20.1 percent) to procure
seven additional IIF satellites.
Technology Maturity
The only critical technology on the Block IIF satellites is the
space-qualified atomic frequency standards and is considered mature.
However, maintaining an industrial base capable of manufacturing frequency
standards for GPS appears to be an issue.
Design Stability
We could not assess design stability because the Block IIF contract does
not require that design drawings be delivered to the program. However, the
program office assesses design maturity by reviewing contractor
development testing, participating in technical interchange meetings and
periodic program reviews, and conducting contractor development process
and configuration audits. The contractor for the Block IIF satellites
faced significant challenges designing and implementing the programming
logic for the Application Specific Integrated Circuit microcircuit chips.
Failure to recognize and understand the complexity of the Application
Specific Integrated Circuit design and delays in security clearances
resulted in $46 million in cost overruns. To offset the overruns, the
program office reallocated $22 million, and the Congress approved $24
million to be reprogrammed from other space programs. According to program
officials, additional Block IIF satellites are not expected to experience
cost increases because the microcircuit chip problem has been resolved
with a dual design.
Production Maturity
We could not assess production maturity because the contractor does not
collect statistical process control data. However, the program office
reviews earned value management reports, integrated master schedules, and
test dates as a means of monitoring the contractors' production efforts.
When monthly earned value management reports and schedule reviews show
cost overruns and/or schedule slips, the program office may choose to
request additional information from the contractor.
Other Program Issues
The GPS Operational Control System consists of monitor stations that track
the navigation signals of all the satellites, remote ground antennas that
actively send commands to the satellite constellation, and two master
control stations (primary and backup) that update the satellites'
navigation messages. Software for the control system, referred to as
Version 6, is needed to support the operational capability of the
satellites with new military code signals. The first satellite with the
new military code was launched in September 2005 and a total of 18
satellites with this code need to be on orbit to provide initial
operational capability to military users. The program office estimates
that 18 satellites will be on orbit in fiscal year 2011, but that Version
6 will not be operational until fiscal year 2012. Thus the satellites on
orbit with the new military code, while supporting constellation
sustainment, will not be fully utilized.
Under the current schedule the initial operational capability for Version
6 had already slipped from 2008 to 2010, because funding was reallocated
to complete development of Block IIF satellites to sustain the GPS
constellation. During 2005, the program office reorganized and stopped
work on Version 6 due to the reduced funding and concerns about parallel
development of two different control systems, Block II and Block III (the
next generation of satellites and a new control system), by potentially
two different contractors. The program office plans to award a single
competitive contract for Version 6 and the Block III control system with a
first increment that will enable full military code capability in 2012.
R&D cost growth amounted to $399 million (20.4 percent) for satellite
component modernization and the control system, and procurement cost
growth amounted to $717 million (20.1 percent) to procure seven additional
IIF satellites.
Agency Comments
The Air Force generally concurred with this assessment and provided
technical comments, which were incorporated as appropriate.
The Army's JLENS is designed to provide over-the-horizon detection and
tracking of land attack cruise missiles and other targets. The Army is
developing JLENS in two spirals. Spiral 1 is completed and served as a
test bed to demonstrate initial capability. Spiral 2 will utilize two
aerostats with advanced sensors for surveillance and tracking as well as
mobile mooring stations, communication payloads, and processing stations.
JLENS provides surveillance and engagement support to other systems, such
as PAC-3 and MEADS. We assessed Spiral 2.
Source: Cruise Missile Defense Systems Project Office, JLENS Product Office.
Development GAO Design Low-rate Full-rate Initial Last
start review review decision decision capability procurement
(8/05) (1/06) (9/08) (9/10) (6/13) (9/13) (2019)
The program began development in August 2005 with one of its five critical
technologies mature. The Army determined that JLENS is primarily an
integration effort based on relatively mature technologies from other
programs and concluded that none of JLENS technologies meet the definition
of a critical technology. However, we identified five technologies in its
technology assessment that could be defined as critical because they are
essential to JLENS capabilities and integrating them will involve changes
in size, the arrangement and interconnections of subcomponents, and
software development challenges. The program plans to release 90 percent
of the engineering drawings by the design review; however, the program
faces risk of redesign until technologies demonstrate full maturity.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (8/05) (1/06)
review (9/10) (9/08)
Technology Maturity
JLENS entered system development in August 2005 with one of its five
critical technologies mature. The communications payload technology
consisting of radios and fiber optic equipment is fully mature and the
processing station technology--which serves as the JLENS operations
center--is approaching full maturity. Both sensors--the precision track
illumination radar (PTIR) and the surveillance radar (SUR) along with its
platform--are not yet mature.
In June 2005, the Office of the Deputy Assistant Secretary of the Army for
Research and Technology determined that JLENS is primarily an integration
effort based on relatively mature technologies from other programs and
therefore concluded that none of the JLENS technologies meet the
definition of a critical technology. According to the project office, many
of the JLENS technologies have legacy components that have either been
tested or fielded in an environment similar to the expected JLENS
deployment environment. However, we identified five critical technologies
based on review of the program's technology maturity assessment, an
independent assessment by the Army's Aviation and Missile Research
Development and Engineering Center (AMRDEC), and through discussions with
program officials. We determined that these technologies are critical
because they are essential to the attainment of JLENS' required
capabilities, and some will require physical modification and
demonstration of subcomponents for use in the JLENS operational
environment.
The JLENS sensors support its primary mission to acquire, track, classify,
and discriminate targets and are being developed using components from
other programs such as MDA's THAAD and the Navy's SPY-3 Radar used on the
DD(X) and the Marine Corp's Affordable Ground Based Radar. Although the
PTIR is similar to the design of the existing SPY-3 radar and the program
has developed prototypes of PTIR components, the radar is not yet mature
because only a partial structure of the antenna has been built in
prototype form. The antenna structure is a key component for maintaining
the weight requirements of the PTIR and has yet to be demonstrated for the
JLENS application. Furthermore, the antenna patch assemblies, used to
transmit and receive radio frequency energy, will require unique circuitry
and design changes to meet form and fit requirements. According to program
officials, tests to integrate the PTIR prototype components will occur
sometime in fiscal year 2006.
While approximately 80 percent of the software used by the PTIR is from
the SPY-3 radar, nearly twothirds of the software used by the SUR sensor
will need to be developed or modified. Also, the SUR uses a different
processor than legacy software, and some new modules have yet to be
tested. According to program officials, software items for the SUR are the
primary challenges for achieving technology maturity because they are
still being developed and designed. The program expects to fully
demonstrate full maturity of these items in 2009.
The JLENS platform consists of the aerostat, mobile mooring station, power
and fiber optic data transfer tethers, and ground support equipment. The
mobile mooring station, used to anchor the aerostat during operations, is
based on a fixed mooring station design. However, it is the least
well-defined component of the JLENS system because a mobile mooring
station for large aerostats has never been developed. As a result, the
current mooring station will need modifications in order to meet JLENS
mobility requirements.
Design Stability
Program officials estimate that 90 percent of its 6,230 drawings will be
released by the design review scheduled for September 2008. However, until
the maturity of the JLENS' critical technologies has been demonstrated,
the potential for design changes remains.
Agency Comments
In commenting on a draft of this assessment, the Army stated that the
JLENS technology maturity and technology readiness assessments were
reviewed by the Department of the Army and the Office of the Secretary of
Defense prior to the JLENS Defense Acquisition Board review that was held
in June 2005. Based upon these and the independent assessment conducted by
AMRDEC, the review board concluded that the JLENS technologies were at an
appropriate maturity level to proceed into the development phase of the
program. The Army also stated that as the program moves further into
development, it is anticipated that these technologies will prove out in
the integration process.
Source: JSF Program Office.
Low-rate Initial Initial Initial Last
decision capability capability capability procurement
USMC USAF USN
(1/07) (3/12) (3/13) (3/13) (2027)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (10/01) (1/06)
review (1/07) (2/06)
The JSF program goals are to develop and field a family of stealthy,
strike fighter aircraft for the Navy, Air Force, Marine Corps, and U.S.
allies, with maximum commonality to minimize costs. The carrier suitable
version will complement the Navy's F/A-18 E/F. The conventional take-off
and landing version will primarily be an air-to-ground replacement for the
Air Force's F-16 and the A-10 aircraft, and will complement the F-22A. The
short take-off and vertical landing version will replace the Marine Corps'
F/A-18 and AV-8B aircraft.
Program Development GAO
start start review
(11/96) (10/01) (1/06)
JSF program data indicates that 7 of the system's 8 critical technologies
will not be fully mature until after the first design reviews in 2006. Not
only is design stability not projected by the time of those reviews, one
of the two variants to be reviewed in 2006 is expected to have released
significantly fewer drawings than suggested by best practices.
Furthermore, the demonstration of a production representative aircraft
that includes design changes to reduce weight will not occur until late
2007, after the start of production. Less than a year after the design
review, the program plans to enter production with little demonstrated
knowledge about performance and producibility. Software also poses a risk
as the program plans to develop nearly 19 million lines of code. At the
production decision, the program will have released about 35 percent of
the software needed for the system.
Technology Maturity
The JSF entered development without its eight critical technologies being
mature. Recent data provided by the program office indicates that maturity
has progressed; however, seven technologies are still not fully mature and
are not expected to be until after the design review.
Design Stability
Currently, 26 percent of the short take-off and vertical landing variant
and less than 3 percent of the conventional variant drawings have been
released. Design reviews for these variants are scheduled for February
2006. Program data indicates that 75 percent of the drawings for the short
take-off and vertical landing variant and 18 percent of the conventional
variant are expected to be released by that time. Program officials state
that these represent the most critical drawings. The program has not yet
prototyped any of the expected designs. An early prototype is expected to
have its first flight in August 2006, but does not include many of the
design changes that resulted from an effort to reduce airframe weight. The
first demonstration of a prototype that incorporates the design changes is
scheduled for late 2007. The carrier version design review is not
scheduled until late 2006. It will not be until 2009 that all three
variants will be undergoing flight testing.
Production Maturity
The program plans to enter low rate production in early 2007 without
demonstrating production maturity. The program is taking steps to collect
key information on the maturity of manufacturing processes but will not
demonstrate that the aircraft can be produced efficiently by the
production decision. If schedules are met, the program will deliver only
one nonproduction representative aircraft before the production decision.
This aircraft, while not yet complete, has experienced labor
inefficiences, part shortages, and major work performed out of sequence.
The program will also not demonstrate that the aircraft works as intended.
At the production decision, it will (1) have completed less than 1 percent
of the planned flight test program, (2) not have flight tested a fully
configured and integrated JSF, (3) have released only 35 percent of the
software needed for the system, and (4) have little or no data from full
scale structural testing. Before development is complete in 2013, DOD
plans to buy 424 low rate production aircraft at an estimated cost of
about $49 billion. DOD plans to use cost reimbursement-type contracts for
its initial production orders, meaning that the government will pay any
cost overruns.
Other Program Issues
The program plans to develop about 19 million lines of software code.
Officials consider software a high risk item. The first of five major
software blocks is scheduled to be released in June 2006 to support first
flight. However, the Defense Contract Management Agency projects that this
release could be delayed 1 to 3 months. Subsequent blocks are showing
early indications of falling behind as well.
At this point the cost estimate represents the program office's position.
The OSD Cost Analysis Improvement Group was to update its formal
independent cost estimate in the spring of 2005, but now does not expect
to formally complete its estimate until after the 2006 design review.
However, a preliminary estimate was higher than the program office's with
large projected funding shortfalls in the 2007 to 2011 time frame.
Agency Comments
The JSF Program Executive Officer continues to nonconcur with GAO's
methodology and conclusions on technology maturity. Hardware and software
integration for multiple subsystems is ongoing in labs, years sooner than
in legacy programs. Critical design reviews were completed in March 2004
for all design areas except the airframe. The air system design review in
early 2006 will evaluate design maturity and performance against
requirements. Manufacturing of the first test aircraft is well underway
with much shorter assembly times than planned and exceptional quality
demonstrated in fabrication, assembly, and mate. As of November 2005 the
actual weight of 7,600 delivered components is within 1 percent of
predictions. While the first aircraft lacks some design improvements,
demonstrated processes and outcomes justify high confidence in design and
weight predictions for all variants due to commonality of design, tools,
and manufacturing methods. JSF acquisition strategy, including software
development, reflects a block approach. Development is on track.
Source: JTRS AMF Program Office.
Production decision
(10/09)
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production
review start design decision
(1/06) (6/06) review (10/09)
(9/08)
The JTRS program is developing software-defined radios that will
interoperate with existing radios and also increase communications and
networking capabilities. A Joint Program Executive Office provides a
central acquisition authority and balances acquisition actions across the
services. Program/product offices are developing radio hardware and
software for users with similar requirements. The Air Force/Navy-led AMF
program is developing radios that will be integrated into over 160
different types of aircraft, ships, and fixed stations.
Pre-SDD GAO Development Design
competitive review start review
contract award
(9/04) (1/06) (6/06) (9/08)
JTRS AMF has taken steps to develop knowledge prior to the start of system
development. As part of the program's acquisition strategy, a pre-system
development phase started in September 2004 with the award of competitive
system design contracts to two industry teams led by Boeing and Lockheed
Martin. Through this acquisition strategy, program officials expect
competitive designs that will help mitigate costs and other risks. While
challenges remain, program officials noted that significant progress has
been made by both industry teams in demonstrating technology and design
maturity. The program is scheduled to enter system development in June
2006. The JTRS Joint Program Executive Office is currently conducting a
broad assessment of JTRS. The assessment may result in changes to the
current JTRS AMF acquisition strategy.
Technology Maturity
To help mitigate technical risks and address key integration challenges,
JTRS AMF awarded competitive predevelopment contracts to two industry
teams led by Boeing and Lockheed Martin. In July 2006, after a full and
open competition, a contracting team will be selected for the JTRS AMF
system development. The program office will use an Army organization to
prepare an independent Technology Readiness Assessment before entry into
the system development and demonstration acquisition phase. The
identification of critical technologies and the assessment of their
respective maturities will not be available until the conclusion of the
competitive system design contract work and the Technology Readiness
Assessment is submitted by the independent assessment team. The
competitive system design contract work will be completed in February
2006.
Although critical technologies have not been formally assessed for JTRS
AMF, both teams have demonstrated progress in developing key functions of
the radio, according to program officials. Preliminary design reviews were
held in early August 2005 for both teams, and program officials indicated
that both preliminary designs met the National Security Agency's
information security requirements. Although the program is likely to face
challenges as it proceeds through system development and demonstration,
program officials are confident that the program can enter the system
development and demonstration phase with sufficiently mature technology.
This assurance is based on technical exchange and review meetings with the
contractors, along with vigorous risk reduction programs by both the
contractors and program office established during the pre-system
development and demonstration contract.
Other Program Issues
The JTRS AMF is depending on the JTRS Cluster 1 program to develop the
necessary waveforms. However, the JTRS Cluster 1 program is currently
being restructured due to significant cost and schedule problems. As a
result, the waveforms being developed under the Cluster 1 contract may not
be developed in time or may not meet JTRS AMF user requirements which may
negatively affect hardware design and cause an increase in cost and
schedule.
Another issue the program office will need to address is the development
of technologies necessary to effectively dissipate heat in some of its
smaller radios. If cooling techniques are not improved, the performance of
these radios will be limited. In addition, integrating the radios into the
diverse platforms covered by JTRS AMF will be a challenge.
Because of the ongoing cost, schedule, and technical problems with the
Cluster 1 program, the JTRS Joint Program Executive Office has begun a
broader assessment of all JTRS clusters. At this point, it is unclear how
the JTRS AMF program will be affected by the results of the broader
assessment. Because of the progress made in the JTRS AMF program, DOD may
expand the numbers and types of platforms on which it will be based. For
example, JTRS AMF program officials noted that it is likely that the
Army's rotary wing JTRS requirements will be moved from JTRS Cluster 1 to
JTRS AMF. The JTRS Joint Program Executive Office developed several
alternative acquisition strategies which were presented to the Defense
Acquisition Board in November 2005.
Agency Comments
In commenting on our draft, the program office generally concurred with
our findings and offered technical comments for our consideration. We
incorporated the technical comments where appropriate.
In addition, the program office stated that the JTRS AMF program has
managed the identified risks with mitigation plans and monitoring of the
competing contractors' technical designs and their use of advanced
technologies. The waveform dependency risk, for example, is being
mitigated by the contractors' access to alternate waveform software that
is similar in features to the Cluster 1 waveform system. The contractors
have also focused considerable investment in addressing the heat
dissipation of their designs and projected performance limits as a
function of industry technology improvements, such as processor speeds or
device sizes.
The JTRS program is developing software-defined radios that will
interoperate with existing radios and also increase communications and
networking capabilities. A Joint Program Executive Office provides a
central acquisition authority and balances acquisition actions across the
services, while product offices are developing radio hardware and software
for users with similar requirements. The Army-led JTRS Cluster 1 product
office, within the Ground Radio Systems program office, is developing
radios for ground vehicles.
Source: PM WIN-T JTRS Cluster 1.
JTRS Development Design GAO Low-rate Full-rate Last
program start start review review decision decision procurement
(9/97) (6/02) (12/03) (1/06) (TBD) (TBD) (TBD)
The JTRS Cluster 1 program is currently being restructured due to
significant cost and schedule problems that came to light in late 2004.
Since development began in 2002, the program has struggled to mature and
integrate key technologies and been forced to make design changes. For
example, the Cluster 1 design does not meet size, weight, and power
constraints or security requirements to operate in a networked
environment. The JTRS program restructure has been approved by the Defense
Acquisition Executive and provides for a path forward to meet security
requirements. Over the next year, the program will seek full approval of
the strategy, targeted for early fiscal year 2007. Due to the program
restructuring, we did not assess the current overall attainment of product
knowledge.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (6/02) review
(1/06) (TBD) (12/03)
Technology Maturity
The maturity of Cluster 1's critical technologies is unclear. The program
reported that 13 of its 20 critical technologies were mature indicating
that progress has been made since the program entered system development
in 2002 when none of the program's critical technologies were mature.
However, this progress is based on a series of contractor demonstrations
conducted in spring 2005 that used only partially functioning prototypes.
A planned operational assessment was canceled after the Army informed the
contractor of possible contract termination. Among other things, the
demonstrations did not show extensive Wideband Networking Waveform
capabilities. The Wideband Networking Waveform represents the core of the
JTRS networking capability and its integration is the most significant
technical challenge to the radio's development, according to program
officials. In addition, critical technologies such as the network bridging
software are immature. Moreover, the program continues to be challenged by
security requirements. The program has identified an interim approach to
address security requirements that complies with National Security Agency
guidance and supports the operation of networking waveforms and
interoperability with non-JTRS networks. However, the approach utilizes
only partially functioning prototypes and is expected to provide only
limited capabilities. Program officials noted that a follow-up effort
involving actual prototypes will provide full capabilities. Until the
program demonstrates an actual prototype under realistic conditions and
completes its restructuring of the program, it is difficult to evaluate
the maturity of its critical technologies.
Design Stability
The program reports achieving design stability for the basic Cluster 1
radio design. However, the National Security Agency has determined that
the current design is not sufficient to meet newly discovered security
requirements needed to operate in an open networked environment. The
program also continues to reconcile size, weight, and power requirements.
These challenges and the uncertainty of technology maturity raise concern
about the program's design stability.
Other Program Issues
In light of the technical problems and cost growth, the Office of the
Secretary of Defense in January 2005 directed the Army to stop work on
portions of the Cluster 1 development. In April 2005, the Army notified
the prime contractor that it was considering contract termination. This
action was taken based on initial findings of an assessment of the Cluster
1 program conducted by the newly established JTRS Joint Program Executive
Office, which concluded that the current program was not executable and
the contractor's ability to develop the radio was questionable. Despite
these concerns, the partial stop work order was allowed to expire, and the
prime contractor was allowed to continue portions of the Cluster 1
contract.
The JTRS Joint Program Executive Office is now proceeding with a major
restructuring of the program. It has completed its assessments of the JTRS
clusters, revising the programs' management and financial structure and
has reviewed Cluster 1 requirements with the intent of making the program
more achievable. The JTRS Joint Program Executive Office developed several
acquisition strategies which were presented to the Defense Acquisition
Board in October and November 2005. The JTRS restructure has been approved
by the Defense Acquisition Executive, and the program will seek full
approval of the strategy over the next year. Program officials expect the
restructured program to be up and running in early fiscal year 2007. In
the meantime, the program continues to mature and support prototype
design. The restructured program will emphasize an evolutionary
acquisition of the radio in increments rather than attempting to field a
complete capability all at once. In addition, DOD officials expect that
the development of the helicopter variant will be moved to the JTRS
Airborne, Maritime and Fixed-Site program.
Agency Comments
In commenting on a draft of this assessmnet, the program office generally
agreed with the information provided in this report. Program offcials also
provided technical comments, which were incorporated where appropriate.
The JTRS program is developing software-defined radios that will
interoperate with existing radios and also increase communications and
networking capabilities. A Joint Program Executive Office provides a
central acquisition authority and balances acquisition actions across the
services, while product offices are developing radio hardware and software
for users with similar requirements. The Army-led JTRS Cluster 5 product
office, within the Ground Radio Systems program office, is developing
handheld, manpack, and small embeddable radios.
Source: PM WIN-T JTRS Cluster 5.
Program/ GAO Low-rate Initial operational
development start review decision capability
(4/04) (1/06) (3/08) (5/11)
JTRS Cluster 5 began system development with one of its six critical
technologies considered mature. The program considers the five other
technologies low risk and anticipates increased levels of maturity, though
not full maturity, by the production decision in March 2008. We did not
assess design stability because no production representative drawings had
been released at the time of our assessment. The total number of drawings
has also not been identified. The JTRS Joint Program Executive Office has
conducted a broad assessment of the entire JTRS program. A JTRS program
restructure has been approved by the Defense Acquisition Executive. The
program will seek full approval of the revised strategy by the start of
fiscal year 2007. The revised Cluster 5 program is described as a moderate
risk program.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/04) (1/06)
review (3/08) (4/06)
Technology Maturity
The JTRS Cluster 5 program has identified six critical technologies and is
focused on a common set of core circuit card assemblies for all its
handheld, manpack, and small form factor radios. The program office has
assessed one of the Cluster 5 critical technologies, termed environmental
protection, as mature for use. The program office has assessed two other
critical technologies, antenna and power management, at a high level of
readiness, although not fully mature. However, the power management
technology may not be as mature as assessed given the Cluster 5
requirement to support a JTRS Wideband Networking Waveform. This waveform
is essential to providing JTRS networking services to ensure
interoperability over a wide range of frequencies. While it is not
designated a Cluster 5 critical technology, the JTRS Operational
Requirements Document designates it as a key performance parameter.
Operation of this waveform carries with it a large power requirement.
Because of the technical challenges of meeting that power requirement in
an acceptable size and weight, the Cluster 5 program is seeking some
relief from the waveform's requirements and attempting to optimize the
software code to increase its power efficiency. It is also evaluating
alternative waveforms such as the Soldier Radio Waveform to provide in a
power efficient way the needed networked services for radios with highly
constrained power and antenna size. The remaining Cluster 5 critical
technologies--microelectronics, multichannel architecture, and
security--require additional development. According to the program office,
however, all three represent a moderate level of risk and are anticipated
to reach increased levels of maturity by the production decision.
The program continues to address size, weight, and power requirements. The
Cluster 5 two-channel manpack radios are to have a maximum weight of 9
pounds. In comparison, current single channel manpack radios weigh in
excess of 13 pounds. However, the JTRS Joint Service Capabilities Working
Group recently gave the program relief in meeting this weight requirement.
Design Stability
We did not assess the design stability of JTRS Cluster 5 because the total
number of drawings is not known and there are currently no releasable
drawings complete.
Other Program Issues
In authorizing the Cluster 5 program to begin system development in April
2004, the Army Acquisition Executive directed that the program assess the
technological maturity of its plans for acquiring Future Combat System
unique small form factor JTRS capability. This review was scheduled for
spring 2005. However, in February 2005 the newly appointed JTRS Joint
Program Executive Officer assumed responsibility for all JTRS Clusters,
including Cluster 5, and began an assessment of all JTRS Clusters. Based
on this assessment, the JTRS Joint Program Executive Office developed
several alternative acquisition strategies which were presented to the
Defense Acquisition Board in October and November 2005. The restructured
program will emphasize developing and evolving the radio products in
increments rather than attempting to field a complete capability all at
once. According to program officials, delivering a JTRS capability in
increments will make the JTRS Program executable and reduce cost,
schedule, and performance risk.
Agency Comments
In commenting on our draft, the program office generally concurred with
our findings and offered technical comments for our consideration. Many of
the technical comments involved updated information on the status of the
JTRS restructuring. We incorporated all relevant updated information into
our report.
The J-UCAS program is a joint Air Force and Navy effort to develop and
demonstrate the technical feasibility and operational value of a networked
system of high performance, weaponized unmanned aircraft. Planned missions
include suppression of enemy air defenses, precision strike, persistent
surveillance, and potentially others such as electronic attack as
resources and requirements dictate. The program consolidates two formerly
separate service efforts and is to develop and demonstrate larger, more
capable, and interoperable
Source: (left) X-45C - The Boeing Company, X-47B - Northrop Grumman
Corporation, (right) X-45C (c) 2004 The Boeing Company, X-47B (c) 2004
Northrop Grumman Corporation.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production
review start design decision
(1/06) (10/11) review (TBD)
(TBD)
aircraft.
Program GAO Development
start review start
(10/03) (1/06) (FY12)
None of the eight critical technologies for this preacquisition program
are currently mature, but J-UCAS officials project that, due primarily to
planned risk reduction efforts, three will be mature and five will be
approaching maturity to support a potential system acquisition start in
fiscal year 2012. The J-UCAS program has been buffeted by frequent changes
in leadership, funding, and priorities. Leadership recently transitioned
from the Defense Advanced Research Projects Agency to the Air Force with
Navy participation and funding was reduced. The Quadrennial Defense Review
then recommended restructuring the J-UCAS program to develop a
longer-range carrier-based unmanned combat aircraft for the Navy. The Air
Force plans to consider J-UCAS technologies and accomplishments in its
efforts to develop a new long-range persistent strike capability.
Technology Maturity
None of the eight critical technologies identified for this
pre-acquisition program are currently mature. Technologies include
adaptive autonomous operations (for controlling groups of aircraft flying
in a coordinated manner without human inputs) and force integration (for
interoperating with intelligence sources and strike and surveillance
packages). Most technologies are in a maturity range in which basic
components are integrated to establish they will work together and
components are integrated with reasonably realistic support elements for
testing in a simulated environment.
J-UCAS program officials have established a structured and disciplined
framework for maturing technologies, using both in-house efforts and
technology developments outside the program. Officials project that, at a
potential system start-up in fiscal year 2012, three technologies will be
mature at the best practices standard and the remaining five will be
approaching maturity.
As currently envisioned, the J-UCAS program will develop technologies and
operationally assess demonstrator aircraft from two prime contractors
sharing a common operating system, payloads, and subsystems. Boeing X-45,
Northrop-Grumman X-47, and common systems and techologies are on contracts
and proceeding forward. The Air Force and Navy could then use the results
of the operational assessment to decide whether to start system
development program(s).
Other Program Issues
The just-completed Quadrennial Defense Review recommended restructuring
the J-UCAS program and develop an unmanned longer-range carrierbased
aircraft to increase naval reach and persistence. The Air Force is
focusing its resources on delivering a new long-range strike capability.
Officials will consider J-UCAS technologies and accomplishments in the
analysis of alternatives for the new strike capability. Final decisions,
future plans, and funding requirements were not available to us at the
time of our review.
Prior to the Quadrennial Defense Review, the J-UCAS program had already
undergone several changes in leadership, program direction and priorities,
and funding. Recognizing the potential for synergy and cost savings, OSD
consolidated separate Air Force and Navy efforts in a joint program in
October 2003 under DARPA leadership. The previous service efforts had been
targeted to specific service needs and different missions; under the joint
program, the emphasis was on developing interoperable and networked
systems utilizing a common operating system, sensors, and weapons. A
December 2004 program budget decision by OSD reduced future budgets,
directed a restructure to emphasize development of air vehicles, and
directed that management be transitioned to the Air Force with Navy
participation; this was accomplished in November 2005.
Congress reduced funding in the 2005 and 2006 budget requests. For 2005,
Congress expressed concerns that the joint program had not properly
coordinated with the two services and directed that the technology
demonstrators be completed in support of Air Force and Navy requirements.
For 2006, Congress expressed concerns about fluctuations in the program,
including Service ownership, and apparent incompatibility of the Air Force
and Navy requirements. The Congress also directed DOD to conduct an
independent study to review technical requirements and options for cost
savings, and to provide an analysis and recommendation on whether the Air
Force and Navy are sufficiently different in their respective requirements
and level of development to merit separation into service-unique programs.
Agency Comments
In commenting on a draft of this assessment, DOD said that all critical
technologies are projected to be mature enough to support start of system
development expected in fiscal year 2012. Subsequently, J-UCAS program
officials briefed us on the process and results of a new reassessment of
technology maturity levels. We updated this product to reflect the
reassessment and incorporated current events from the Quadrennial Defense
Review.
MDA's KEI element is a missile defense system designed to destroy medium,
intermediate, and intercontinental ballistic missiles during the boost and
midcourse phases of flight. Key components include hit-to-kill
interceptors, mobile launchers, and fire control and communications units.
We assessed the proposed land-based KEI capability, which according to
program officials, will be available in 2014.
Source: Kinetic Energy Interceptors Program Office, Northrop Grumman.
Program Prime contractor GAO Booster Design First Block 2012
flight integrated
start selection review test review flight test completion
(10/02) (12/03) (1/06) (4th Q/08) (4th (2nd Q/13) (2014)
Q/11)
KEI's seven critical technologies are at a relatively Attainment of
Product Knowledge low level of maturity, with two rated as high
risk--Production, the interceptor's booster motors and the algorithm
design &
technology
that enables the kill vehicle to identify the threat maturity missile's
body from the luminous exhaust plume. According to MDA officials,
integration issues and hardware manufacturability are being addressed,
Design &
technologyand the design of the demonstration hardware maturity should
become the design for the operational KEI element. In 2008, MDA will
assess KEI's achievements and decide how the program should proceed. If a
decision is made to move forward, TechnologyMDA plans to finalize the
design during the fourth maturity quarter of fiscal year 2011. At that
time, two technologies will have been demonstrated in flight tests, and
four in ground tests. KEI underwent a replan to compensate for fiscal year
2005 funding cuts and additional requirements, causing
GAO Development DOD Productionprogram delays. review start design decision
(1/06) (TBD) review (TBD) (TBD)
Technology Maturity
All seven KEI critical technologies are at a relatively low level of
maturity. These technologies are part of the element's interceptor, the
weapon component of the element consisting of a kill vehicle mounted atop
a boost vehicle. Of the seven technologies, four pertain to the boost
vehicle that propels the kill vehicle into space. Boost vehicle
technologies include two types of booster motors, an attitude control
system, and a thrust vector control sytem. The remaining three
technologies are related to the kill vehicle--its infrared seeker, divert
system, and plume-to-hardbody algorithms. Although all technologies are
immature, three of the seven are new applications of existing technologies
developed by other missile defense programs. The infrared seeker and the
third stage rocket motor come from the Aegis BMD program, and the divert
system comes from the GMD program. Backup technologies exist for all
technologies, but the infrared seeker. However, these technologies are at
the same low level of maturity as the critical technologies.
Program officials noted that they expect the design of the demonstration
hardware to be the design of the operational hardware. Therefore,
integration and manufacturability issues are being addressed in the design
of the demonstration hardware. According to program officials, KEI's
operational design will be finalized in 2011. By that time, MDA plans to
demonstrate two critical tecnologies--the thrust vector control system and
one of the two types of boosters--in two booster flight tests. Other
technologes will have been demonstrated in ground tests, such as
hardware-in-the-loop tests and flight tests. The integration of all
critical technologies will be demonstrated in an element characterization
test early in fiscal year 2012, a sea risk reduction flight test in late
2012, followed by the first integrated flight test early in 2013.
Design Stability
The KEI program office estimates that KEI's design will incorporate about
7,500 drawings. Program officials expect 5,000 of these drawings to be
complete when it holds a critical design/production readiness review for
the land-based capability in 2009. However, it is too early to make an
accurate assessment of KEI's design because all of KEI's technologies are
not mature. In addition to using the number of drawings released as a
measure of the design's maturity, the program also plans to use
Engineering and Manufacturing Readiness Levels to determine the design's
manufacturability and Software Readiness Levels to assess the maturity of
KEI's software.
Other Program Issues
In fiscal year 2008, MDA plans to assess KEI's accomplishments and make
decisions about the program's future. If MDA decides to acquire KEI,
program officials expect to begin development of a space-based test bed.
MDA expects to expend about $673 million between fiscal years 2008 and
2011 on the test bed's development, which, when complete, is envisioned as
a limited constellation of space based interceptors capable of providing
an additional layer of defense against ICBMs. In spite of its unknown
future, program officials are working to extend KEI's contract from
January 2012 (98 months) to September 2015 (143) months. Additionally, the
program has directed its contractor to investigate the effect of making
KEI capable of defeating threat missiles during the midcourse of their
flight.
The KEI program underwent a program replan to compensate for fiscal year
2005 funding cuts and the addition of new requirements, such as a
requirement for nuclear hardening imposed by MDA. Under the replan the
Block 2010 land-based capability was combined with the Block 2012
sea-based capability, both of which utilize the same interceptor.
According to program officials, KEI is undergoing further restructuring
which has delayed the landbased capability into Block 2014 and the
sea-based capability into Block 2016. Program officials noted that if they
receive additional funding, the landbased capability could still be
delivered during Block 2012.
Agency Comments
MDA provided technical comments, which were incorporated where
appropriate.
The Army's Land Warrior program is developing modular, integrated,
soldier-worn systems intended to enhance the lethality, situational
awareness, and survivability of dismounted combat and support soldiers.
The program restructured in 2005 in an effort to field capability to the
current force, focusing on the Dismounted Battle Command System (DBCS).
DBCS comprises the Commander's Digital Assistant and the MicroLight
Enhanced Position Location and Reporting System, elements of the
previously planned Land Warrior system. We assessed DBCS.
Source: Program Executive Office Soldier.
Program/ Design GAO Low-rate Full-rate Initial Last
development start review review decision decision capability procurement
(8/94) (2/05) (1/06) (6/06) (3/07) (3/08) (2018)
As an early spiral of Land Warrior, the Dismounted Attainment of Product
Knowledge Battle Command System is technologically less Production,
ambitious than previous efforts. The system's design &
technology
three critical technologies (power, radio maturity communications, and the
personal area network) are mature; however, the personal area network that
connects the components together did not Design &
technologyreach maturity by the time of the DBCS critical maturity design
review in February 2005. The program did not achieve design stability by
this design review, but all drawings are currently releasable. We could
not assess production maturity for DBCS because Technologythe program is
not collecting statistical process maturity control data at this time.
However, the results of an early evaluation of DBCS conducted by the Army
Test and Evaluation Command in August 2005 recently led the Army to
terminate the DBCS effort and focus on developing the full Land
Warrior ensemble. Development DOD GAO Production start design review
decision (8/94) review (1/06) (6/06) (2/05)
Technology Maturity
An early spiral of Land Warrior, DBCS is intended to provide a limited,
near-term capability to the current force to improve infantry unit battle
command and situational awareness. As a partial capability, DBCS is
technologically less ambitious than the system we assessed last year. The
DBCS comprises two adapted commercially available components: the
Commander's Digital Assistant (CDA) and the MicroLight Enhanced Position
Location Reporting System (EPLRS). Running the Army's Force XXI Battle
Command Brigade and Below situational awareness software, the CDA is
intended to provide leaders at the platoon and company level with blue
(friendly) force tracking capability. The MicroLight EPLRS will provide
voice and data communications at the squad level and higher. The three
critical technologies for DBCS, radio communications (the MicroLight
EPLRS), power for the CDA (batteries), and the personal area network that
connects the components together, are mature by best practice standards.
We did not assess technology maturity for the limited number of full Land
Warrior ensembles the Army is procuring for assessment purposes in 2006.
Last year we assessed what was then Block II of the program, the Land
Warrior--Stryker Interoperable system. At that time, two of the system's
four critical technologies were not mature (the personal area network and
radio communications). The Land Warrior system will eventually use the
JTRS Cluster 5 embedded radio (assessed elsewhere in this report),
scheduled to be available in fiscal year 2011.
Design Stability
The program reported that 23 design drawings out of a total expected
number of 70 were releasable at the February 2005 critical design review
for DBCS, and that all 70 drawings are currently releasable.
Production Maturity
We could not assess the maturity of production processes for DBCS because
the program is not collecting statistical process control data at this
time. Officials told us that while General Dynamics has not fully
identified the key manufacturing processes, the company has initiated
manufacturing planning in accordance with ISO 9000 guidelines.
Other Program Issues
The Army restructured the program in 2005 in response to congressional
direction to immediately field some Land Warrior capabilities to the
current force, terminating the Block II effort that was underway. The
restructured program comprised three phases. The first phase was focused
on fielding the Dismounted Battle Command System to leaders of up to 30 of
the Army's Brigade Combat Teams. The Army conducted an early evaluation of
DBCS in August 2005, during which soldiers from the 10th Mountain Division
used the system in training for an upcoming deployment to Afghanistan. The
Army Test and Evaluation Command concluded that DBCS was not suitable for
light infantry operations and reported that the system's weight and
physical configuration reduced soldiers' mobility. In addition, the
demonstration revealed concerns about power consumption as well as an
inability to interoperate with the unit's existing radios. Noting that the
unit would not take DBCS to Afghanistan, DOD's Director of Operational
Test and Evaluation concluded that the system did not demonstrate the
necessary capabilities and that the current system was not mature.
The second phase of the program, still on track, is focused on developing
an integrated Land Warrior capability in support of the Army's Stryker
Brigades. Slightly less capable than the system we assessed last year, the
program plans to field 486 of these systems to one Stryker battalion in
fiscal year 2006 for assessment purposes. The third phase, Ground Soldier
System, is the future iteration of Land Warrior capability intended to
provide a dismounted soldier capability to the Army's Future Combat
Systems. In early 2005, the program completed a plan to consolidate the
Land Warrior program with the Army's Future Force Warrior Advanced
Technology Demonstration effort.
Agency Comments
In commenting on a draft of this assessment, the Army provided technical
comments, which were incorporated where appropriate.
The Navy's LCS is to be a surface combatant optimized for littoral warfare
with innovative hull designs and reconfigurable mission packages to
counter threats in three mission areas: mine, antisubmarine, and surface
warfare. The ship and mission packages are being developed in spirals with
the first four ships, Flight 0, produced in two designs. We assessed only
Flight 0 ships and their associated mission packages.
Source: (top) Lockheed Martin, (bottom) General Dynamics.
Program Development Production Production GAO Initial
start start decision-1st decision-2nd review capability
design design
(9/02) (6/04) (12/04) (10/05) (1/06) (10/07)
The LCS program entered system development in June 2004. The program
office identified 41 critical technologies for the mission packages and 43
technologies between the two ship designs. Since our last review, LCS has
continued to test and mature its technologies for the mission packages.
Currently 19 of the 41 mission package technologies are fully mature;
seven are near full maturity; and 15 remain in development. The
technologies that have not reached maturity affect all three of the
mission packages, each of which will go through critical design review in
2006. The majority of technologies for the ship designs are fully mature
or near maturity, except for those used for launch and recovery or command
and control of off-board vehicles. Both ship designs have begun
production.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (6/04) review
(NA) (1/06) (11/04)
Technology Maturity
Nine of the technologies under development for LCS are used in multiple
applications or mission packages. Since these technologies are used on
different platforms or in different environments, the program office chose
to assess them in each setting separately. This results in a total of 41
critical technologies, 19 of which are currently mature.
The first mine warfare mission package will align with delivery of the
first ship in January 2007. In this mission, the MH-60S helicopter is to
carry subsystems for either detection or neutralization of mines. A key
component for attaching subsystems to the helicopter is currently
undergoing flight testing to correct deficiencies. Delay of the MH-60
could leave gaps in the ability to detect some mines while increasing the
time needed to neutralize others. The unmanned surface vehicle has a
similar mine neutralization capability as the MH-60S, which also acts as
its fallback. Neither the vehicle nor its payload is currently mature, and
failure to deploy it on LCS will lead to increased use of the MH-60S. For
mine warfare the vertical tactical unmanned autonomous vehicle, an
unmanned helicopter, will employ the coastal battlefield reconnaissance
and analysis system for detection of mines on the beach. Both the vehicle
and its payload are currently immature, and no fallback is available to
LCS.
The first antisubmarine and surface warfare packages will align with
delivery of the second LCS in fiscal year 2008. The MH-60R helicopter,
fully mature in each configuration, is critical for these missions.
Antisubmarine and surface warfare will also be performed by a number of
other immature systems, including the vertical tactical unmanned
autonomous vehicle and the unmanned surface vehicle. The MH-60R is the
fallback for those systems should they fail to mature. As a consequence,
reliance on helicopters will increase should the unmanned systems fail to
deploy with LCS.
Demonstration of mission package technologies will continue through 2008
and includes experiments with surrogate platforms. All technologies are
expected to reach maturity by delivery of their respective mission
package.
While the designs of the first LCS ships are novel in the experience of
the Navy, the majority of shipspecific technologies are mature or close to
full maturity. The Lockheed Martin design, the first to be produced,
currently has 16 of 21 technologies mature or close to full maturity. The
General Dynamics design currently has 20 of 22 technologies mature or
close to full maturity. Most of the immature technologies are used to
launch and recover or control the vehicles used in mission packages.
Design Stability
Design of mission packages is tracked in a unique manner as some of the
"technologies" used are fully developed systems. These systems are being
designed and produced by other programs and, to ensure that the mission
packages will be compatible with LCS, the program has established a set of
interface specifications that each system must meet. These specifications
regulate issues like electrical, communications, and maintenance needs.
The specifications for components of mission packages will be reviewed as
part of a critical design review for each warfare package. Both designs of
the first spiral of LCS ships have begun production. Application of
commercial design specifications and standards to Navy shipbuilding have
created some challenges during the design process, as has leveraging
designs with commercial lineage for military use.
Agency Comments
In commenting on the draft of this assessment, the Navy stated that the
LCS program implements spiral development to rapidly field capabilities
that fill current operational gaps while achieving unprecedented
flexibility for the future. Efficient spiral implementation is achieved
through modular mission packages operated through a common interface
specification. Mission package systems have been selected from best "state
of the practice" technologies to satisfy requirements, ranging from mature
acquisition programs to technology demonstrators. While component systems
may be technically mature, repackaging and integration into operational
mission packages requires verification testing to validate performance.
Program test plans include specific events to rapidly demonstrate the
technical maturity of the modular systems, and the flexibility of the
modular open architecture greatly reduces the risk and impact from any
single component.
Source: Boeing; Army Systems Program Office; Huntsville, AL.
Low-rate Initial Last
decision capability procurement
(3/09) (9/11) (unknown)
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production
review start design decision
(1/06) (3/06) review (3/09)
(3/07)
The Army's AH-64D Longbow Apache can be employed day or night, in adverse
weather and obscurants, and is capable of engaging and destroying advanced
threat weapon systems. The primary targets of the aircraft are mobile
armor and air defense units, with secondary targets being threat
helicopters. Block III enhancements are to ensure the Longbow Apache is
compatible with the Future Combat System architecture, is a viable member
of the future force, and is supportable through 2030. We assessed the
Block III portion of the Apache.
GAO Development Design
review start review
(1/06) (3/06) (3/07)
The Apache Block III program plans to begin system development in March
2006 with approximately 67 percent of its critical technologies fully
mature. However, while the Army plans to develop the Block III in one
acquisition program, it also plans for the program to be comprised of two
development phases. The Army expects to approve funding for the first
phase in 2006 and the second after 2010. Overall, program officials
project that at the start of development seven of the fifteen critical
technologies will be fully mature, six approaching full maturity, and two
immature. Due to the acquisition strategy and budgetary constraints, no
further efforts to mature the less-than-fully mature technologies will
occur until fiscal years 2010 to 2015. According to the program, these
technologies are primarily software upgrades that will be easy to retrofit
into helicopters.
Technology Maturity
Program officials report that 6 of 15 critical technologies are currently
fully mature. Further, when the program enters the system development and
demonstration phase in March 2006, an additional technology--the Level 4
Unmanned Aerial Vehicle Control--will have reached full maturity. These
mature technologies are planned for insertion in the helicopter in the
initial production lots. For example, the composite main rotor blade and
the modernized signal processor units are already fully mature and will be
incorporated in the 2010 to 2012 time frame. Due to the acquisition
strategy and budgetary constraints, no further efforts to mature the 8
immature technologies will occur until fiscal years 2010 to 2015. As
aircraft come off the production line, the program will have provisions in
place that will allow for these technologies to be inserted when they are
fully mature and available in the 2015 time frame.
Technical insertions for the Apache Block III effort consist of two
general categories: processor upgrades and non-processor upgrades. The
first development phase addresses some of the processor upgrades and all
of the non-processor upgrades. The second developmental effort addresses
the remaining processor upgrades. The processor upgrades are open or
partitioned software architectures that will allow integration of most of
the improvements. Processor upgrades include changes to the Instrument
Flight Rules, the Modernized Signal Processor, the Radar Frequency
Interferometer, the Control of Unmanned Aerial Vehicles, the Cognitive
Decision Aiding System, the Fire Control Radar Range Extension, the
Multi-Mode Laser, Aided Target Detection and Classification, Maritime
Targeting Modes, and the Radar Frequency Interferometer passive ranging.
Nonprocessor upgrades include changes to the engine, an improved drive
system, and the composite main rotor blade.
The processor technologies are primarily software upgrades that are low
risk and easily fieldretrofitable into helicopters with minimal cost
without having to return to the productionprocessing facility. According
to program officials, there will be costs associated with retrofitting the
helicopters but these costs should be minimal given the ability to add
software changes in the field and because the helicopter would have to be
returned to the production plant to accomplish these upgrades. Also, given
the fact that the government will perform the software retrofits on its
own as part of the normal software update process, the financial impact
will be minimal. Further, based on the current technology readiness
levels, program officials believe the technical risk to these technologies
is low even though no back-up technologies exist. If, for some reason, the
technology is unavailable for insertion at its given time, the program
would proceed with the existing technology until the new technology can be
incorporated.
Agency Comments
The Army concurred with our assessment.
Source: The Boeing Company, (c)2005 The Boeing Company.
Low-rate Full-rate Initial Last
decision decision capability procurement
(5/10) (4/13) (7/13) (2017)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (5/04) (1/06)
review (5/10) (7/07)
The Navy's MMA is part of the Broad Area Maritime Surveillance (BAMS)
family of systems, along with the BAMS Unmanned Aerial Vehicle (UAV) and
Aerial Common Sensor (ACS). This family of systems is intended to sustain
and improve the Navy's maritime warfighting capability. The MMA is the
replacement for the P-3C Orion. Its primary roles are persistent
antisubmarine warfare; antisurface warfare; and intelligence,
surveillance, and reconnaissance capabilities.
Program Development GAO Design
start start review review
(3/00) (5/04) (1/06) (7/07)
The MMA program entered development with none of its four critical
technologies mature. According to the program office, these technologies
will be demonstrated in a relevant environment by design review and tested
in an operational environment by the production decision. The program
evaluated six other technologies but decided they were not critical
because they had already been demonstrated in a relevant or operational
environment. The system's technology maturity will be demonstrated at
least 3 years later than recommended by best practices standards. However,
if those technologies do not mature as expected, the program has
identified mature back-up technologies. In addition, during the program's
preliminary design review, a recommendation was made to reassess whether
all critical technologies in the program have been identified.
Technology Maturity
None of the four critical technologies--integrated rotary sonobuoy
launcher, electronic support measures digital receiver, data fusion, and
acoustic algorithms--are mature. These technologies have not moved beyond
the laboratory environment. For three of the technologies, the components
have not been integrated into a prototype system. The program expects the
four technologies to be demonstrated in a relevant environment by design
review in July 2007 and tested in an operational environment by the
production decision in May 2010. The system's technology maturity will be
demonstrated at least 3 years later than recommended by best practices
standards.
The program office and the contractor developed maturation plans and
identified mature backup technologies for each of the critical
technologies. According to program officials, the MMA would lose some
capabilities but still meet its minimum system requirements if it used
these backups. For example, one of the biggest technology challenges for
the MMA identified by program officials is the electronic support measures
digital receiver. This technology exists as a prototype and has been
demonstrated in a high fidelity laboratory environment. The program is
leveraging the digital receivers currently in development on the EA-18G
program. If the EA-18G digital receiver program is unsuccessful, the
program will have to use legacy analog off-the-shelf receivers, which
would prevent them from gaining an increased sensitivity for certain
signals.
The four technologies we assessed were identified in the MMA's technology
readiness assessment. The program evaluated six other technologies but
decided they were not critical because they had already been demonstrated
in a relevant or operational environment. However, during the program's
November 2005 preliminary design review, a recommendation was made to
reassess whether all critical technologies in the program have been
identified.
Design Stability
We did not assess design stability as the number of releaseable drawings
is not yet available.
Other Program Issues
As of August 2005, the MMA program is on budget and on schedule. However,
if the MMA fails to develop as expected or experiences schedule slippage,
the Navy would have to rely on its aging P3C Orion fleet, which according
to DOD is plagued by serious airframe life issues, poor mission
availability rates, high ownership costs, and limited system growth
capacity.
The MMA shares the persistent intelligence, surveillance, and
reconnaissance role with the BAMS UAV. The BAMS UAV development start and
initial operations capability have been delayed 18 months and three years
respectively. If the BAMS UAV does not develop as planned or continues to
experience schedule delays, the MMA is its fallback and according to the
Navy's most recent analysis, the overall cost of the program would
increase due to a need to procure additional MMA aircraft. In addition, a
third element planned for the BAMS family of systems is the ACS. The ACS
is intended to replace three current systems: the Army's Guardrail Common
Sensor, Airborne Reconnaissance Low, and the Navy's EP-3. However, DOD
issued a stopwork order to the ACS program prime contractor in September
2005 and terminated the contract in January 2006, because the airframe
selected for the ACS could not accommodate the intended ACS mission
equipment. Decisions concerning the ACS program that have not yet been
made may determine whether the Navy participates in a future ACS program.
One of the alternatives previously assessed by the Navy to replace the
EP-3 included incorporating the ACS equipment onto the MMA airframe.
Agency Comments
The Navy concurred with GAO's assessment of the MMA program. We
incorporated technical comments provided by the Navy as appropriate. The
Navy stated that the program continues to manage the four critical
technologies. It stated that the maturation of these technologies is on
schedule and will be demonstrated in a relevant environment prior to the
July 2007 design readiness review. It also stated that the program
continues to meet or exceed the cost, schedule, and performance parameters
defined in the program's baseline agreement and that the prime contractor
also continues to execute the contract within cost and schedule
parameters.
Source: Unmanned Undersea Vehicles Program Office.
Low-rate Initial decision capability (8/10) (9/11)
Attainment of Product Knowledge
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Projection
Launched and recovered from submarine torpedo tubes, the Navy's 21" MRUUV
will independently perform a range of information gathering activities. It
supplants two related programs now limited to prototype development, the
long-term mine reconnaissance system and the advanced development unmanned
undersea vehicle. Each MRUUV system will include the vehicle, combat and
control interfaces, and enabling equipment for either mine countermeasure
or ISR missions. This assessment is as of January 2006. The planned July
2006 decision to enter development has since been delayed.
GAO Development Design
review start review
(1/06) (7/06) (9/09)
One of the MRUUV program's six critical technologies is currently mature.
While the program expects to have fully matured four of the five remaining
critical technologies by the time of development start in July 2006, the
final technology--a rechargeable battery for the system--is not expected
to reach maturity until 2008. Given the cost growth and schedule slippage
experienced on previous unmanned undersea vehicle programs, DOD is
treating the program as if it were a larger development effort and
providing increased oversight.
GAO Development DOD Production
review start design decision
(1/06) (7/06) review (8/10)
(9/09)
Technology Maturity
One of the MRUUV's six critical technologies is currently mature and the
program expects to have at least four of the remaining five critical
technologies fully mature by the start of system development in July 2006.
The one exception, a rechargeable battery used for power supply, may
require further development to ensure proper integration.
The Littoral Precision Undersea Mapping Array is a critical sonar
technology that enables object identification and obstacle avoidance,
essentially forming the "eyes" of the vehicle. An advanced development
model of the Mapping Array already has been developed, tested, and
deployed on a 21" vehicle, thereby successfully demonstrating its mine
identification capability. A more advanced, lighterweight prototype is
scheduled to be completed and tested in an operational environment in
fiscal year 2006.
The synthetic aperture sonar takes detailed pictures of underwater
objects. A surface ship has towed a sonar model in an ocean environment to
provide preliminary engineering data. A final prototype, will be completed
in fiscal year 2006 and will be tested in open water in early fiscal year
2007.
According to the project manager, the maturity of the software that
provides MRUUV's autonomous operational capability has already been
demonstrated. This software is currently being used in operational
unmanned undersea vehicles and can be applied to the MRUUV to enable it to
perform its basic mission requirements. Nevertheless, software development
will continue, with incremental improvements added as they are developed.
This may include an enhanced ability to make autonomous decisions and
functionality that will facilitate a more efficient equipment swap
process.
Technology to manage the vehicle launch and recovery process involves
acoustic signaling and mechanical activities. A predecessor vehicle on
which MRUUV is based has demonstrated homing, docking, and replacement
into a model submarine hull. Maturity of this technology could be
demonstrated by system development start if at-sea tests with a real
submarine are successful.
Intelligence, surveillance, and reconnaissance technology will be used to
provide remote monitoring capability, which involves placing the vehicle
in a strategic location to listen for specific signals. Such technology,
essentially a sensor antenna, already exists and is operational on Navy
unmanned aerial vehicle platforms. However, it needs to be miniaturized
and adapted for an ocean environment, which should be demonstrated in May
2006 when the technology will be fit onto a small underwater vehicle shell
and used in at sea testing.
MRUUV's final critical technology is battery power. Although a stable
conventional battery has been developed, the Navy is also pursuing the
development of a rechargeable battery. While the rechargeable battery has
attained functional capability, it will require further refinement to
ensure fit into a small unmanned undersea vehicle. This is expected to
occur in 2008.
Other Program Issues
Although total investment in the MRUUV is expected to be less than $365
million in research and development funding, the Office of the Secretary
of Defense may designate the program as an Acquisition Category I.
Officials at DOD believe that the program requires enhanced oversight and
visibility into program activities because of the cost growth and schedule
slippage that plagued previous unmanned undersea vehicle programs.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the
MRUUV program expects to have demonstrated all major technology risks
through other programs or through the science and technology community by
the time it reaches system development in July 2006. According to Navy
officials, remaining risks will be the responsibility of the prime
contractor to address within the systems engineering and design
integration process. The Navy also commented that a carefully structured
acquisition strategy and risk management program will continue to mitigate
risks as the program progresses through its design phase. In subsequent
comments, Navy officials noted that, as would be expected of a pre-MDAP
program, the MRUUV effort is continuing to evolve and that since GAO
conducted its audit work the program has experienced significant changes
and is likely to experience additional changes.
The Navy's MUOS, a satellite communication system, is expected to provide
low data rate voice and data communications capable of penetrating most
weather, foliage, and manmade structures. It is designed to replace the
Ultra High Frequency (UHF) Follow-On satellite system currently in
operation and provide support to worldwide, multiservice, mobile, and
fixed-site terminal users. MUOS consists of a network of advanced UHF
satellites and multiple ground segments. We assessed both the space and
ground segments.
Program Development GAO Design Production On-orbit Full
start start review review decision capability capability
(9/02) (9/04) (1/06) (3/07) (10/07) (3/10) (3/14)
In September 2004, the MUOS program was authorized to begin development.
The program currently has 9 of 11 critical technologies mature. The
remaining technologies are projected to be mature by March 2007, in time
for the critical design review. The program intends to order long lead
items for the first two satellites before achieving a final design. This
early procurement could lead to rework causing cost increases and schedule
delays if relevant designs change prior to critical design review. In
addition, the MUOS development is schedule-driven, posing several risks to
the program.
satellite, Lockheed Martin.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (9/04) (1/06)
review (10/07) (3/07)
Technology Maturity
Eight of nine critical technologies were mature at the development start
decision in September 2004. The number of critical technologies has
increased by two since our assessment of the program last year due to
continuing program analyses resulting in increased knowledge of required
technologies. Currently, nine of eleven critical technologies are mature.
The remaining technologies, a new cryptographic chip for the ground
satellite control segment and a digital to analog converter, are expected
to be mature by the time the program reaches its critical design review in
March 2007. Mature backup technologies exist in the event that they fail
to mature in time. However, use of the backup technologies would increase
the vulnerability to attacks on signal transmissions used to ensure the
satellites remain properly placed in their orbits and increase risk of
program cost and schedule growth.
Design Stability
The MUOS program has begun procuring long lead items for the first two
satellites before achieving a final design. According to the program
office, $125.5 million (then year dollars) in long lead items are to be
ordered before critical design review in March 2007, nearly double the
amount estimated last year. Such procurement could lead to rework if
relevant designs change prior to the system-level critical design review,
causing program cost increases and schedule delays. According to the
program office, delaying long lead procurement until after critical design
review would cause the program schedule to slip. In addition, the program
office noted that the majority of the long lead procurements are planned
after respective segment-level critical design reviews (which precede the
system-level critical design review) and that most are for standard
commercial satellite bus components.
Additionally, the program office has not estimated the total number of
design drawings needed to build the satellites, but this number will
likely be known next year. The development contract requires the
completion of 90 percent of the design drawings as a condition of
conducting critical design review.
Other Program Issues
In June 2004, DOD delayed the first MUOS satellite launch by one year to
fiscal year 2010 due to a delay in awarding the development contract and
to mitigate schedule risk. While the MUOS program has stayed within its
cost and schedule estimates, its schedule remains compressed. For example,
the importance of the first MUOS launch date has increased due to an
unexpected failure of a UHF Follow-On satellite in June 2005.
Communication capabilities are now expected to degrade in 2009, one year
earlier than previously estimated. Also, operational capability from the
first satellite may be used before formal on-orbit operational testing is
to take place. Usually, such testing occurs prior to placing a satellite
into service. Finally, an independent assessment conducted for the MUOS
development start decision states that the program is schedule-driven due
to software development.
According to the program office, development of MUOS ground software
represents one of the highest risks to the program due to the size and
complexity of the contractor's design. The program office stated that the
ground software is to be developed in three builds comprised of two to
four increments each (for a total of eight increments) to mitigate
schedule risk. Additionally, the program intends to track and assess
software development using numerous metrics we have found to be useful for
program success. However, our review of the software development shows
cost and schedule growth risks remain due to the concurrent development of
the three builds. Specifically, during the approximately 4-year software
development time frame, about one-half of this period consists of
concurrent development among the software builds. Such concurrency can
increase the severity of software problems due to their cascading cost and
schedule impacts on other builds.
Agency Comments
In commenting on a draft of this assessment, the Navy provided technical
comments, which were incorporated where appropriate.
NPOESS is a triagency National Oceanic and Atmospheric Administration
(NOAA), DOD, and National Aeronautics and Space Administration (NASA)
satellite program to monitor the weather and environment through the year
2020. Current NOAA and DOD satellites will be merged into a single
national system. The program consists of five segments: space; command,
control, and communications; interface data processing; launch; and field
terminal software. We assessed all segments.
Source: NPOESS Integrated Program Office.
Development start/ GAO First satellite Initial
production decision review launch capability
(8/02) (1/06) (9/10) (3/12)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Development GAO
start design review start/production
(NA) review decision (1/06)
(NA) (8/02)
Program start (3/97)
In August 2002, the NPOESS program committed to the development of
operationally capable satellites with only 1 of 14 critical technologies
mature and about half of its drawings released to manufacturing. All but
three of these technologies are expected to be mature by design review in
February 2007. The program office is not collecting statistical process
control data to assess production maturity because of the small number of
units being produced. It considers two of the four critical sensors key
program risks because of technical development challenges. In November
2005, our analysis showed the contractor was $253.8 million over budget
and may have a potential overrun of about $1.4 billion at completion. The
program reported a Nunn-McCurdy (10 U.S.C. 2433) unit cost breach in
January 2006, at the 25 percent threshold, due to continuing technical
problems.
Technology Maturity
Only 1 of the program's 14 critical technologies was mature at the
production decision in August 2002. One critical technology was deleted
from NPOESS in 2005. The program projects that all but three of the
remaining technologies will be mature by the design review in 2007.
The program undertook the NPOESS Preparatory Project, a demonstration
satellite, to reduce risk and provide a bridging mission for NASA's Earth
Observing System. This satellite is to demonstrate three of the four
critical sensors in an operational environment and was scheduled for
launch in May 2006. However, the launch of this satellite was delayed 23
months from the contract award date to April 2008. This effort is to
provide data processing centers with an early opportunity to work with
sensors, ground controls, and data-processing systems and allow for
incorporating lessons learned into the NPOESS satellites. Since our
assessment last year, the program office reports that three sensors
continue to experience cost increases and schedule delays due to technical
challenges. Two of these are considered critical sensors.
Design Stability
In August 2002, the program committed to the fabrication and production of
two satellites with operational capability before achieving design
stability or production maturity. Program officials indicated that about
55 percent of the design drawings have been released to manufacturing, and
expect to release about 88 percent by the design review in 2007, which
represents a decline of 6 percent from last year's estimate. The design
review date and other schedule dates are subject to revision based on the
results provided by an independent program assessment, DOD review, and
Nunn-McCurdy (10 U.S.C. 2433) certification process.
Production Maturity
We could not assess production maturity because, according to the program
office, it does not collect statistical process control data due to the
small number of units to be built. However, program officials report the
contractors track and use various metrics for subcomponents, such as
rework percentages and defect containment, to track production progress.
Other Program Issues
In 2002, DOD extended the launch date of one of its legacy meteorological
satellites to 2010, delaying the need for NPOESS and reducing NPOESS
funding by about $65 million between fiscal years 2004 and 2007. Funding
reductions prompted a restructuring of the NPOESS program. In September
2005, the program office submitted a new total program cost estimate of
about $8.3 billion. In November 2005, we estimated total program cost to
increase to about $9.7 billion at completion. This represented about a 50
percent increase from the original program cost estimate of $6.5 billion.
In January 2006, the program reported a Nunn-McCurdy (10 U.S.C. 2433) unit
cost breach, at the 25 percent threshold, due to continuing technical
problems. NPOESS officials stated the most recent increase is due to
technical issues surrounding the program, including the development of key
sensors. In addition, given the challenges currently facing the program,
the scheduled first launch date slipped 17 months from the contract award
date to September 2010.
Agency Comments
In commenting on a draft of this assessment, the program office stated
that every aspect of the program is being evaluated by various internal
and external groups and noted that management changes to better align the
management structure with the program phase have recently occurred at the
program office and at the prime contractor. It stated that management,
design, and manufacturing process issues at multiple levels have
contributed to the current instrument problems and resulting cost and
schedule issues. It further stated that several options are being reviewed
for technical viability and cost effectiveness as part of the Nunn-McCurdy
(10 U.S.C. 2433) certification process. The program office noted that any
changes resulting from this process may produce substantial cost,
schedule, and technical performance changes. The program office also noted
that part of the schedule slips were due to congressional budget cuts.
Technical comments were also provided and incorporated as appropriate.
Source: MEADS NPO, Lower Tier Project Office.
Initial production Last production Initial
decision decision capability
(11/12) (3/17) (9/17)
Production, design & technology maturity
Design & technology maturity
Technology maturity
The Army's Patriot/MEADS Combined Aggregate Program is the process by
which the Patriot missile system transitions to the MEADS. The MEADS
mission is to provide low-to-medium altitude air and missile defense with
the capability to counter, defeat, or destroy tactical ballistic missiles,
cruise missiles, and other air-breathing threats. MEADS is a codevelopment
program among the United States, Germany, and Italy. We assessed the MEADS
fire unit portion of the program.
Development GAO Design
start review review
(8/04) (1/06) (9/09)
The MEADS fire unit began development in 2004 with two mature critical
technologies, three critical technologies nearing maturity, and one
immature critical technology. The technologies remain at these levels.
Program plans call for a system design review in 2009, but program
estimates project that only one of the six fire unit technologies will be
more mature at that time than at development start. The program office
anticipates that all critical technologies will be fully mature by the
start of production in the first quarter of fiscal year 2013.
Development GAO DOD Production start review design decision (8/04) (1/06)
review (11/12) (9/09)
Technology Maturity
Only two of the six critical technologies--launcher electronics and
Patriot Advanced Capability (PAC)-3 missile integration--are mature. Three
other critical technologies--low noise exciter that manages the radars'
frequencies, cooling system for the radars, and slip ring that carries
power and coolants to the radars--are nearing maturity. The remaining
critical technology--the transmit/receive module that transmits/receives
signals for the fire control radar--is immature.
The program office projects that the transmit/receive module will increase
in maturity by the time of the system design review planned for 2009. The
program office expects that the five other critical technologies will be
at the same maturity levels as they were at the start of development. The
office expects all critical technologies to be fully mature by the start
of production in late 2012. There are no backup technologies for any of
the MEADS critical technologies.
Design Stability
We could not assess the design stability of MEADS because the number of
releasable drawings and total drawings expected were not available. The
program office expects to know the total number of releasable drawings at
the design review in 2009.
Other Program Issues
MEADS is being developed to employ the current PAC-3 missile and the
future PAC-3 missile segment enhancement variant. The missile segment
enhancement is a U.S.-funded effort to improve on the current PAC-3
missile capability. Program estimates indicate that the Army plans to
develop and procure missiles at a cost of approximately $6.1 billion. We
did not assess the missile and the missile segment enhancement, and the
associated costs are not included in our funding information.
The MEADS program has adopted an incremental acquisition approach wherein
MEADS major items are incrementally inserted into the current Patriot
force. There are three increments, with the first beginning in 2008,
another in 2010, and the final in 2013. The program office plans for each
increment to introduce new or upgraded capability into the program. The
Army expects MEADS to achieve initial operating capability in 2017 with
four units.
Agency Comments
The Army generally concurred with this assessment. It indicated that we
addressed critical technologies that were already areas of intense
management focus. The Army also noted that it still expects all
technologies to be fully mature by production and further stated that
there are risk mitigation plans for the maturing technologies and
alternate back-up technologies identified for the transmit/receive module.
Additionally, the Army noted that, at the design review in 2009, the
design work in the critical technologies will be at the maturity level
required to fabricate the system prototype necessary to demonstrate
required system capabilities.
Source: General Atomics-Aeronautical Systems, Incorporated.
Full-rate Initial Last
decision capability procurement
(9/08) (12/09) (2014)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (2/04) (1/06)
review (9/08) (9/06)
The Air Force's MQ-9 Predator B is a multirole, medium-to-high altitude
endurance unmanned aerial vehicle system capable of flying at higher
speeds and higher altitudes than its predecessor, the MQ-1 Predator A. The
Predator B is designed to provide a ground attack capability to find and
track small ground mobile or fixed targets. Each Predator B system will
consist of four aircraft, a ground control station, and a satellite
communication suite. We assessed the first increment of the air vehicle.
Program Development GAO Design
start start review review
(1/02) (2/04) (1/06) (9/06)
The Predator B entered system development in February 2004 with three of
its four critical technologies mature. The Air Force expects the fourth
technology to be ready in May 2006. However, no suitable back-up
technology is available. If this technology fails to mature as expected,
the Predator B will not be able to effectively perform its primary
mission--to destroy enemy targets. In 2004, the program changed to an
incremental acquisition strategy. The Air Force appears to have made
significant progress in completing design drawings for the first increment
and projects that it will have achieved design stability by the 2006
critical design review. The program has already begun production of the
Predator B aircraft, but operational testing is not scheduled to be
complete until 2008. At that point, about one-third of the quantity will
be on contract or delivered.
Technology Maturity
Three of the Predator B's four critical technologies--the synthetic
aperture radar, the multispectral targeting system, and the air
vehicle--are fully mature. The prototype of the avionics subsystem
technology, designed to integrate and store data necessary to launch
munitions, will begin ground testing in February 2006. The Air Force
expects that the technology will be integrated and mature in May 2006.
This represents about a 10 month slip from last year's estimate. No
suitable back-up technology exists. If this critical technology fails to
mature, it will prevent the Predator B from effectively performing its
primary mission, to destroy enemy targets. Subsequent increments may
require other new technologies.
Design Stability
After its Milestone B approval in February 2004, the program office was
directed to revise its acquisition strategy to develop Predator B in three
increments. The Air Force appears to have made significant progress in
completing design drawings for the first increment of Predator B. At the
time of our assessment last year, the program indicated that just over 35
percent of the drawings for the first increment had been completed. It now
reports that over 85 percent of the drawings are complete. The program
office continues to expect that 94 percent of the drawings for the first
increment will be completed by the time of the critical design review in
September 2006. Program officials acknowledge that additional drawings
will be needed for subsequent increments. Design changes and modification
of drawings are likely to occur late in development, increasing the need
to retrofit already acquired systems.
Production Maturity
Program officials said the contractor does not plan to use statistical
process controls to ensure product quality. Instead, it plans to use other
quality control measures such as scrap, rework, and repair to track
product quality. Also, initial operational testing, which is to
demonstrate that a product is ready for production, is not scheduled to be
complete until mid-2008. By that point, about one third of the aircraft
will either be in production or already delivered.
Other Program Issues
In 2004, the Predator program office was directed to adopt an incremental
acquisition strategy and field an interim combat capability by fiscal year
2006. By adopting an incremental acquisition strategy, the program office
is using the preferred approach to weapon acquisitions. To reduce the
risks of concurrently developing and producing Predator Bs, the program
office lowered annual buy quantities and extended production 5 years
through 2014. Nevertheless, the program schedule still contains a high
degree of concurrency. Before the conclusion of initial operational
testing, the Air Force will have already contracted for about one third of
the total aircraft production quantity. The Air Force currently projects
that half of these aircraft will need to be retrofitted to bring them up
to the baseline capability. Additional changes stemming from the test
program would further perturb the aircraft's cost, schedule, and
manufacturing plan.
The Air Force is still evaluating a variety of lightweight munitions for
use on the Predator B. The Air Force is also weighing the possibility of
adding new system capabilities such as launching very small or micro
unmanned aerial vehicles from the Predator B.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that the
stores management system is making good progress and its completion is
considered a low risk activity. The hardware has been installed on a test
aircraft and will begin ground testing in February 2006. The Air Force
also noted that Congress has directed an increase to the yearly production
buys in fiscal years 2004 and 2005. Program planning is in place to
upgrade these aircraft to support initial operational testing in 2008. The
Air Force stated that the ongoing developmental and operational testing
effort and the operational assessment to be conducted in 2006 will provide
valuable feed back to the acquisition and operational commands.
The Air Force's SBIRS High program is a satellite system intended to meet
requirements in the missile warning, missile defense, technical
intelligence, and battlespace characterization missions. A replacement for
the Defense Support Program, SBIRS High consisted of four satellites (plus
a spare) in geosynchronous earth orbit (GEO), two sensors on host
satellites in highly elliptical orbit (HEO), and fixed and mobile ground
stations. In 2005, the number of GEO satellites was reduced to three. We
assessed the sensors and satellites only.
Source: Lockheed Martin Space Systems Company.
Program Development Design First Second GAO First Second
review/
start start production sensor sensor review satellite satellite
decision delivery delivery delivery delivery
(2/95) (10/96) (8/01) (8/04) (9/05) (1/06) (9/08) (9/09)
The SBIRS High program's critical technologies Attainment of Product Knowledge
and design are now mature. Production maturity Production, could not be
determined because the contractor
design & technology
does not collect production statistical process maturity control data. In
August 2004 the contractor delivered the first payload (HEO 1 sensor)
after a delay of 18 months; the second was delivered in Design &
technologySeptember 2005 after a delay of 21 months. Since maturity we
last reported, total costs have increased by more than $1 billion. The
cost growth resulted in two additional Nunn-McCurdy (10 U.S.C. 2433) unit
cost breaches and a decision not to buy two Technologysatellites. Although
program officials have maturity acknowledged that the GEO satellites are
orders-of-magnitude more complex than the HEO sensors, they now believe a
more realistic program schedule has been developed. The first GEO
satellite delivery has been delayed an
vel of knowledge
Desired le
additional 5 months to late 2008. Development Design review/ Production
GAO start production decision review (10/96) decision (NA) (1/06) (8/01)
Technology Maturity
The SBIRS High program's three critical technologies--the infrared sensor,
thermal management, and on-board processor--are mature. However, program
officials stated that flawed initial systems engineering created
first-time integration and test risk associated with the GEO staring
sensor. According to program officials, early test results of the scanning
and staring sensors are positive. The staring sensor is to have the
ability to stare at one earth location and then rapidly change its focus
area, representing a significant leap in capability over the current
system.
Design Stability and Production Maturity
In portraying the program's events in this assessment, we considered the
production decision to have occurred at the time of design review because
this is when the program office began ordering long lead parts for the
fabrication of satellites. Although the program's design is now considered
stable since almost all drawings have been released, design-related
problems are now the issue. Design problems led to delayed delivery of
both HEO sensors, which were accepted for operations without meeting all
program specifications. Given the added complexity of the GEO satellites
over the HEO sensors, the probability is high that major design flaws will
be discovered on the GEO satellites as well.
We could not assess the production maturity of SBIRS High because the
contractor does not collect production statistical process control data.
However, the program office tracks and assesses production maturity
through detailed monthly manufacturing and test data and monthly updates
on flight hardware qualifications. The program office, in late 2005,
implemented initiatives for its flight software development processes and
placed fulltime program office personnel at the contractor's facility.
According to program officials, about 95 percent of flight hardware for
the first GEO satellite and 85 percent for the second has been delivered.
Other Program Issues
Integration and testing of the first GEO payload and spacecraft has begun.
It was during this process that the design errors in the HEO sensors were
discovered. Given the high probability that major design flaws will emerge
on these satellites as well, costly redesigns that could further delay
delivery are likely. However, according to program officials, additional
engineering tests have been instituted to address design issues and reduce
the likelihood of significant schedule impacts. To accommodate these
tests, each GEO satellite's delivery was delayed by an additional five
months, bringing the delay to 19 months for each satellite.
In July 2005, the program reported its third and fourth Nunn-McCurdy unit
cost breaches (10 U.S.C. 2433). As part of the mandatory program
certification process triggered by one of the cost breaches, the program
was restructured in late 2005. The program now includes procurement of
only one GEO satellite--reduced from three--and the procurement contract
is contingent upon the performance of the first developmental GEO
satellite. Although the program has reduced the total number of satellites
it will procure, total program funding continues to increase, and revised
estimates indicate the average procurement cost per unit is now 224
percent above the 2002 approved program baseline. The Air Force was
recently directed to begin efforts to develop a viable competing
capability, in parallel with the SBIRS program, and to submit a plan for
this new program by April 2006.
Agency Comments
In commenting on a draft of this assessment, the Air Force acknowledged
that the cost of the program was significantly underestimated at inception
and the program suffered from a lack of military specifications with
proper quality controls and that past restructures and replans did not
fully recognize the extent of rework necessary to ensure mission success.
It noted that the recent comprehensive review of the program resulted in a
more realistic assessment of integration and testing timelines and a
revised funding profile that accounts for the potential rework costs and
schedule delays. In addition, the program has developed one integrated
schedule for the remaining program and created a government cost
estimating capability. The Air Force noted that technical issues will be
uncovered, but early problem identification and prompt resolution will
minimize the impacts to the integrated program activity. Technical
comments were provided and incorporated where appropriate.
The Air Force's SDB is a small autonomous, conventional, air-to-ground,
precision bomb able to strike fixed and stationary targets. The weapon
will be installed on the F-15E aircraft and is designed to work with other
aircraft, such as the F-22A. The Air Force is in the process of
implementing a competitive acquisition strategy for a second increment of
the program, which includes a precision strike against moving targets in
adverse weather capability. This analysis addresses only the first
increment of the program.
Source: The Boeing Company,St. Louis, Missouri.
Program Development Low-rate GAO Initial Last Last
start start decision review capability procurement delivery
(8/01) (10/03) (4/05) (1/06) (9/06) (11/19) (9/21)
The six critical technologies for the SDB are Attainment of Product Knowledge
mature, and the design is stable. The program office held the design
review prior to starting system development and, although data were not
collected, the program maintains that the contractor has released 100
percent of the production drawings. In 2004, the program began a test
program, which combines developmental, live fire, and operational testing
in an effort to decrease time spent in system development. Of 37
developmental tests conducted, 35 were considered successful. Causes of
failure for the other two have been identified and corrected. Some
operational tests remain to be completed. SDB was approved for low rate
production in April 2005. We could not assess production maturity as
statistical process control data were not available.
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Development DOD Production GAO start design decision review (10/03) review
(4/05) (1/06) (NA)
Technology Maturity
The program office assessed all six critical technologies for the SDB as
mature. The technologies are the airframe, the Anti-Jam Global Positioning
System, the fuze, the Inertial Navigation System, the carriage, and
warhead. Program officials stated that all technologies have been tested
in realistic environments and have achieved their final form, fit, and
function.
Design Stability
The design review was held prior to the start of system development and,
although data were not collected, the program office maintains that Boeing
has now completed 100 percent of the production drawings. According to the
program office, although the contractor has ultimate responsibility for
the weapon system and has given the government a 20year "bumper to bumper"
warranty, the program office has insight into the contractor's
configuration control board process and all changes are coordinated with
the government.
The SDB program began a program of developmental, live-fire, and
operational testing in 2004. This combined testing approach is designed to
eliminate or reduce redundant testing. As of the date of this review, all
developmental tests were complete. Of the 37 developmental tests
conducted, two were classified as failures. Program officials told us that
the causes of the two failures have been corrected and verified through
additional flight tests. However, due to the concurrency of the test
program, SDB continues to face an aggressive schedule in the coming
months. Operational testing will be conducted throughout fiscal year 2006,
to be followed by a full rate production decision at the end of fiscal
year 2006.
Production Maturity
We could not assess production maturity because statistical process
control data were not available. In developing the SDB, Boeing used many
key components that are common with the Joint Direct Attack Munition
(JDAM). The SDB production line will be colocated in the same facility
used to produce the JDAM. According to program officials, the production
line layout is very similar to the processes currently used for the JDAM.
As of the date of this review, no critical manufacturing processes that
impact the critical system characteristics had been identified. SDB was
approved for low-rate initial production in April 2005 and will begin
full-rate production in 2006.
Other Program Issues
The Air Force's 2006 budget includes $47 million to begin development of
the second increment of the SDB program. At the time the fiscal year 2006
budget was prepared, the Air Force planned to have Boeing, the prime
contractor for the first increment, add the second increment requirements
to the first increment contract. However, in late 2004, Lockheed Martin
filed a bid protest of the contract award to Boeing, after a former senior
Air Force procurement official acknowledged bias in favor of Boeing. In
February 2005, GAO sustained the protest. Responding to the GAO's decision
and recommendation, the Air Force agreed to recompete the contract for the
second increment. The Air Force is in the process of implementing a
competitive acquisition strategy for a second increment.
Agency Comments
In commenting on a draft of this assessment, the Air Force concurred with
the information presented and provided technical comments, which were
incorporated as appropriate.
Source:Space Radar Integrated Program Office.
Production First Initial Last
decision launch capability procurement
(TBD) (9/15) (9/17) (TBD)
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production
review start design decision
(1/06) (8/08) review (TBD)
(12/11)
SR is an Air Force-led, joint DOD and intelligence community program to
develop a satellite to find, identify, track, and monitor moving or
stationary targets under all-weather conditions and on a nearcontinual
basis across large swaths of the earth's surface. As envisioned, SR would
generate volumes of radar imagery data for transmission to ground-, air-,
ship-, and space-based systems. We assessed the space segment.
Program GAO Development Design
start review start review
(7/03) (1/06) (8/08) (12/11)
Five critical technologies will support the SR program, and they are still
being matured. The program office is focusing its efforts on technology
risk reduction and concept definition activities. At this point, the
program is expected to enter system development before any of the
technologies are mature. The Air Force is restructuring the program to
address concerns about the affordability of SR, which includes schedule
and cost evaluations and several changes to the acquisition strategy. In
2007, the program plans to decide whether to develop on-orbit
demonstration satellites to validate technology maturity and costs. Launch
of the first fully operational SR satellite is scheduled for fiscal year
2015. Design and production maturity could not be assessed because SR is
not yet a formal acquisition and has not begun product development.
Technology Maturity
The program office assessed the electronically steered array, on-board
processing, signalprocessing algorithms for moving target indication,
information management system, and moving-target indication exploitation
hardware and software as the critical technologies needing further
development.
The program office is focusing its efforts on technology risk reduction
and concept definition activities. For example, subcomponents for the
electronically steered array are being integrated with laboratory
components to demonstrate proper functioning. In addition, on-board
processing capabilities are being demonstrated and conceptual designs for
storing and processing data have been developed. The program is also
working to further mature the remaining critical technologies. However,
the program expects to start the product development phase before these
technologies mature.
Other Program Issues
As a result of congressional concerns about the affordability of SR, DOD
and other SR users have now agreed on a path to develop a single space
radar system to meet national needs. The Air Force is restructuring the
program to reflect this agreement and schedule and associated costs are
being evaluated. The new path includes several changes to the SR
acquisition. First, in early 2005, a new Space Radar Integrated Program
Office was established in Chantilly, Virginia, to work with the
intelligence community, DOD and other users, senior Air Force leadership,
and the Congress. Second, the new SR senior leadership established a
framework with overarching guidance for maturing the critical
technologies. Third, a team of program office personnel and mission
partners established a new plan to drive fiscal year 2006 risk reduction
activities and revised cost estimates. Finally, the SR acquisition
strategy now calls for the development of a smaller constellation of high
performance, more affordable satellites and a potential on-orbit
demonstration to validate technology maturity and costs. A final decision
on an on-orbit demonstration is not expected until fiscal year 2007.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that in
response to congressional concerns and as a result of the recent
restructure, the program has implemented a disciplined program framework
approach to mature technology and reduce risk and is working more closely
with all stakeholders. The Air Force also said that this program framework
consolidates and provides coherent big picture direction to multiple
technology-development testing and experimentation activities-such as
ground, existing space, and air components-with a focus on proving
technologies early in the concept development phase of the program to
reduce technical and schedule risk in the future. Moreover, according to
Air Force officials, a robust requirements definition process has been
implemented to provide early stakeholder input and acceptance to stabilize
requirements, further reducing future risk.
MDA's STSS element is being developed in incremental, capability-based
blocks designed to track enemy missiles throughout their flight. The
initial increment is composed of two demonstration satellites built under
the Space Based Infrared System Low program. MDA plans to launch these
satellites in 2007 to assess how well they work within the context of the
missile defense system. MDA is also studying improvements to the STSS
program, and it will be building next generation satellites. We assessed
the two demonstration satellites.
Source:STSS Program Brief.
SBIRS-Low Transition STSS program GAO Demonstrator Software
program start to MDA start review satellite launch upgrades
(1995) (10/00) (2002) (1/06) (2007) (2008)
Three of the STSS program's five critical technologies are mature, and the
remaining two technologies are expected to reach maturity in early 2006.
The STSS design appears otherwise stable, with all drawings released to
manufacturing. The program office has identified certain risk areas, such
as infrared payload completion, payload data processor software
completion, and system integration and functionality. Additionally,
quality and workmanship problems with the payload have continued and have
resulted in cost and schedule overruns with the payload contract. However,
the program office still expects early delivery and launch of the
satellites. The planned budget for STSS through fiscal year 2009 grew by
more than $1.1 billion, mainly in fiscal years 2008 and 2009, due to the
addition of funds for designing and developing the program's operational
constellation.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (NA) review
(1/06) (TBD) (11/03)
Technology Maturity
Three of the five critical technologies--satellite communication
cross-links, on-board processor, and acquisition sensor--are mature. This
is one less than reported last year as MDA corrected its assessment of one
of the technologies. The remaining two technologies--the track sensor and
the single-stage cryocooler--will be mature upon completion of the thermal
vacuum testing on the first satellite's payload, which is expected to
occur in early 2006.
Design Stability
The STSS program's design is stable, with all drawings released to
manufacturing. When the STSS program started in 2002, design drawings and
the satellite components for the partially built satellites from the Space
Based Infrared System Low effort were released to manufacturing. By the
time STSS went through its design review in November 2003, the program
office had released all subsequent design drawings. However, until the
maturity of the remaining two STSS technologies has been demonstrated, the
potential for design changes remains.
Other Program Issues
The STSS program is in the process of completing the assembly,
integration, and testing of the satellite components and software
development. Until this work is complete, certain risk areas will remain.
These include infrared payload completion, payload data processor software
completion, and system integration and functionality testing. Other risk
areas that the program office had identified previously--such as the
hardware and software status, ground software development completion, and
parts obsolescence--have been resolved.
The quality and workmanship problems with the payload subcontractor
persist. These problems have continued for the last 2 years and have
contributed to a schedule delay in delivering the payload and a likely
cost overrun of between $20 million and $30 million on the payload
subcontract. Integration issues have also been discovered as the
subcontractor continues to integrate and test the payload at successively
higher levels of integration. The cause of most of these problems is due
to the difference in configuration between the pathfinder hardware that
served as the test bed for the payload software and the actual flight
hardware. In addition, the actual payload thermal vacuum test is taking
about 30 to 45 days longer than expected to resolve hardware issues that
have emerged as a result of the payload being tested in a vacuum and at
cold temperatures--a relevant environment--for the first time. In response
to these issues, quality control efforts at the subcontractor's site have
undergone significant restructuring. In addition, the prime constractor
stepped up its inspection and supervision of all processes at the
subcontractor's site and has provided mentoring.
According to the program office, many of the quality-related variances
could have been avoided if better quality processes had been in place at
the payload subcontractor. The program office expects that the quality
improvements that the payload subcontractor has implemented will reduce
the number of quality-related problems in the future. According to the
program office, the integration issues that have been discovered are not
unusual for a first time integration effort but are taking more time than
planned to work through. Upon completion of the first satellite's payload,
the program office expects the cost and schedule variances to abate,
although they will not recover. In addition, the second satellite's
hardware has consistently moved through integration and testing much more
efficiently than the first satellite's hardware. Thus, the program office
still expects the prime contractor to deliver and launch the satellites in
February 2007, which is earlier than the contract date, and has placed an
order through NASA for the Delta II launch vehicle. Since our last
assessment, STSS' planned budget through fiscal year 2009 increased by
$1,195.9 million (35.3 percent), primarily in fiscal years 2008 and 2009,
due to the addition of funds for designing and developing the program's
operational constellation.
Agency Comments
MDA was provided an opportunity to comment on a draft of this assessment,
but it did not have any comments.
Source: Missile Defense Agency, THAAD Project Office.
1st intercept attempt, Block 2004 (2nd Q/06) Initial capability available, Block
2008(FY09)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (6/00) review
(1/06) (TBD) (12/03)
MDA's THAAD element is being developed in incremental, capability-based
blocks to provide a ground-based missile defense system able to defend
against short-and medium-range ballistic missile attacks. THAAD will
include missiles, a launcher, an X-band radar, and a fire control and
communications system. We assessed the design for the Block 2008 initial
capability of one fire unit that MDA plans to hand off to the Army in
fiscal year 2009 for limited operational use.
Program Transition GAO
start to MDA review
(1/92) (10/01) (1/06)
Program officials assessed THAAD's technologies as mature and its design
as generally stable, with 95 percent of the engineering design drawings
released. The design of Block 2008, which is expected to provide a limited
operational capability, is a further maturation of THAAD `s Block 2004
design. MDA began flight testing the design with a successful controlled
test flight on November 22, 2005-8 months later than originally planned.
According to program officials, the delay was the result of technical
problems encountered during the integration of the THAAD missile, most of
which have been solved. The current schedule is aggressive, calling for
the completion of as many as five flight tests within one fiscal year.
However, program officials expect to recover most of the flight schedule
and complete 15 flight tests before handing the first fire unit over to
the Army in fiscal year 2009.
Technology Maturity
Program officials assessed all of THAAD's critical technologies as mature.
All of these technologies are included in four major components: the fire
control and communications component; the interceptor; the launcher; and
the radar.
After experiencing early test failures, program officials made changes in
the execution of the THAAD program that allowed it to make progress in
maturing critical technologies. Officials placed more emphasis on risk
reduction efforts, including adopting technology readiness levels to
assess technological maturity.
Design Stability
Program officials reported that THAAD's basic design is nearing
completion, with approximately 95 percent of the expected 9,852
engineering drawings released. However, if problems are encountered during
future flight tests, the total number of drawings could increase.
Production Maturity
We did not assess THAAD's production maturity because MDA does not know
when it will transition THAAD to the Army for production. The one fire
unit that will be handed off to the Army in 2009 for limited operational
use is considered to be primarily a test asset. Prior to a production
decision, the program office plans to assess production maturity using
risk assessments and verification reviews for assurance of the
contractor's readiness to proceed with repeatable processes and quality.
Other Program Issues
MDA expected to begin flight tests in March 2005. However, because of
technical problems experienced during the integration of the THAAD
missile, the first test was pushed out to November 2005. The current
schedule is aggressive, calling for the completion of as many as five
flight tests within one fiscal year. However, program officials told us
that most of the technical problems have been solved and that they are
confident that they will recover most of the flight test schedule. The
program expects to complete 15 flight tests before handing the first fire
unit over to the Army in fiscal year 2009.
The problems incurred by the missile component also affected the program's
cost performance. According to program officials, for the first time since
its contract was awarded in 2000, the THAAD program is experiencing an
unfavorable cumulative cost variance. Program officials noted that as of
October 2005, the program was overunning its prime contract cost by
approximately $50 million. Also, since our last assessment THAAD's planned
budget through fiscal year 2009 has increased by $514.8 million (4.5
percent) primarily in fiscal years 2008 and 2009.
Agency Comments
MDA provided technical comments, which were incorporated where
appropriate.
The Air Force's TSAT system is the space-borne element of the Global
Information Grid that will provide high data rate military satellite
communications services to DOD users. The system is designed to provide
survivable, jam-resistant, global, secure, and general-purpose radio
frequency and laser cross-links with other air and space systems. The TSAT
system consists of a constellation of five satellites, plus a sixth
satellite to ensure mission availability. We assessed the six satellites.
Source: Northrop Grumman Corporation.
GAO Development Design Production First satellite
review start review decision launch
(1/06) (11/06) (10/09) (12/09) (10/13)
In January 2004, the TSAT program received Attainment of Product Knowledge
formal approval to begin preliminary design Production, development
activities. To date, the program has design &
technology
focused on risk reduction and systems definition maturity (e.g.,
requirements allocation and system design) leading to a planned Systems
Design Review (SDR). The first launch schedule has been delayed Design &
technology
by 24 months from its in initial approved program maturity
baseline in June 2004, as a result of 2005
congressional reductions and anticipated
reductions in 2006. TSAT plans to begin product
development activities for the first increment (two Technology
satellites) following SDR in November 2006 with maturity
all of its critical technologies mature, and at that
time, a contract will be awarded to acquire
operational satellites. According to program
officials, a new acquisition strategy is being
developed, which will result in a new program
baseline, cost estimates, and schedule. GAO Development DOD Production
review start design decision (1/06) (11/06) review (12/09) (10/09)
Technology Maturity
In January 2004, DOD approved the TSAT program to enter the preliminary
design phase. One of the program's nine technologies was fully mature. The
program has focused its efforts on maturing critical technologies and
conducting systems definition activities. System definition activities
include requirements allocation and system design activities. According to
program officials, product development activities will begin in November
2006 after a contractor is selected to conduct detailed design studies and
development efforts. At that time, the program expects all critical
technologies for the first increment to be mature.
Currently only four of its seven critical technologies are mature.
However, program officials expect all critical technologies to be mature
before initiating product development activities for the first increment
in 2006.
Of the seven technologies, only four technologies--packet processing
payload, communication-on-the-move antenna, information assurance space
for internet protocol encryption and information assurance for
transmission security--are mature. We previously identified information
assurance as a single critical technology, but obtained more detailed data
for this report. The other three--dynamic bandwidth and resource
allocation, protected bandwidth efficient modulation waveforms, and single
access laser communication--are scheduled to reach maturity in 2006, about
3 years after the approval to start preliminary design development
activities.
Other Program Issues
The TSAT program cannot currently provide data on design stability,
production maturity, or software development for satellite production
because it has not yet selected a contractor to develop, build, and field
the TSAT space segment. Contracting activities to select a single
contractor are scheduled to begin in November 2006, with final award in
early 2007.
The initial June 2004 program baseline had a first satellite launch
scheduled for October 2011. The program office now estimates a first
launch date of October 2013 and attributes the launch delay to
appropriations reductions in fiscal year 2005 as well as anticipated
reductions in fiscal year 2006. The Appropriations conferees reduced the
program by $400 million due to concerns about the state of technology
maturity and concerns that DOD may have prematurely ruled out the
possibility of evolving the Advanced Extremely High Frequency and the
Wideband Gapfiller System programs. The report also stated that transition
to a formal acquisition program should be deferred until the TSAT
technologies are mature and have been demonstrated in a relevant
environment. The report requires that DOD submit the results of an
independent review that: (1) determines whether additional Advanced
Extremely High Frequency or Wideband Gapfiller System satellites will be
required and how many; and (2) whether it is feasible to insert advanced
capabilities by evolving these programs.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that the
TSAT risk reduction and systems definition activities are on schedule.
Currently the program is conducting the third independent evaluation of
the contractor's laser communications subsystems, with the fourth and
final tests scheduled for mid 2006. To date, all test goals have been met,
according to the Air Force. Similar testing is being conducted on other
key technologies, and all are on a path to be fully matured by late 2006.
According to the Air Force, the program's first launch has been delayed
from 2011 to late 2013 due to budget reductions. These delays have
resulted in increased life cycle cost and account for the majority of the
increases shown in this draft.
GAO Comment
In subsequent discussions, TSAT program officials stated that they are
developing a new acquisition strategy, along with an updated baseline with
new milestones, reflecting the $400 million congressional budget
reduction.
Source: U.S. Navy.
Full-rate GAO Initial Last
decision review capability procurement
(9/05) (1/06) (3/07) (2016)
Attainment of Product Knowledge
Production, design & technology maturity
Design & technology maturity
Technology maturity
vel of knowledge
Desired le
Datanot available
Development DOD Production GAO start design decision review (4/86) review
(9/05) (1/06) (9/02)
The V-22 Osprey is a tilt rotor, short/vertical takeoff and landing
aircraft being developed by the Navy for Joint Service application.
Variants are designed to meet the amphibious/vertical assault needs of the
Marine Corps, the strike rescue needs of the Navy, and the special
operations needs of the Air Force and the U.S. Special Operations Command.
The MV22 version will replace the CH-46E and CH-53D helicopters of the
Marine Corps. We assessed the MV-22 Block A. The Navy completed its
operational evaluation of the aircraft in June 2005.
Program Development Development
start start restart
(12/82) (4/86) (9/94)
Operational test and evaluation of MV-22 Block A has been completed, and
the aircraft found to be operationally effective and suitable. Block B is
predicted to have a drop in performance due to increased weight. Tests of
Block A revealed deficiencies with the troop seat restraint system that
has resulted in a redesign of the seat, which may require change to the
aircraft structure to achieve desired seat crash retention capability.
Also, flight clearance restrictions limited some aspects of testing,
particularly survivability, defense maneuvers, and tactics. Deficiencies
were identified with shipboard operations, passenger capability, and
operations at altitude. In September 2005, the Department of Defense
approved V-22 for full rate production; however, production aircraft
continue to be accepted with numerous deviations and waivers.
Design Stability
The design of MV-22 Block A is considered stable. However, recent Block A
operational evaluation tests identified deficiencies that could result in
design changes to parts of the aircraft. Specifically, during operational
tests the troop seat was considered a major deficiency and has been
redesigned to address shoulder harness and comfort issues. In 2002, the
Navy established a more crashworthy configuration requirement to be
consistent with DOD and Federal Aviation Administration goals to meet
common crashworthiness standards for mission equipment. According to the
program office, a change will be made to contract specifications to
require these more stringent standards. Analysis is ongoing to determine
if installation of these newly qualified seats will require aircraft
structural changes to fully achieve their designed crash retention
capability and what impact these changes would have on the aircraft's key
performance parameters.
Production Maturity
In 2001, the V-22 was approved for a limited annual production rate. In
September 2005, DOD approved the V-22 for full rate production even though
a 2004 review found one contractor had parts production problems that
could affect its ability to support fullrate production. Produced aircraft
continue to be accepted with numerous deviations and waivers, but the
number of deviations and waivers at acceptance are not now as significant
as they have been in the past.
Other Program Issues
Based on evaluation tests, the Navy reported that the MV-22 Block A is
operationally effective and suitable. However, the Navy's test report
identified three major deficiencies that must be corrected and verified by
additional operational tests. Of the three major deficiencies, troop
seating and egress were considered the most severe. This deficiency
required redesign of the seats to address comfort and seat restraint
deficiencies. Also, while tests proved that the aircraft was capable of
carrying 24 combatequipped toops, it is anticipated that operational
commanders will prefer that only 18 troops be carried in order to make
room for their extra gear.
Operational tests also identified 38 minor and 50 other deficiencies. Of
the minor deficiencies, the need to eliminate flight clearance
restrictions and increase the defensive-maneuvering envelope of the
aircraft are a priority. Flight clearance restrictions limited some
survivability, defensive maneuvers, and tactics testing and may reduce
aircraft survivability if they are not lifted. The minor deficiencies
identified could also affect operations. They include restricted shipboard
operations, limits on operations above 10,000 feet altitude, passenger
cabin cooling effectiveness, reliability problems with aircraft
components, overheating of the drivetrain gearbox in hot weather, and the
lack of supplemental oxygen for passengers that will restrict long-range
mission profiles with troops on board.
DOD has also concluded that the V-22 Block A aircraft is operationally
effective in low and medium threat environments and is operationally
suitable. However, DOD projects that Block B will not meet the Land
Assault External-Lift and Amphibious External-Lift missions (key
performance parameters). The predicted shortfall could be mitigated by
lower aircraft weight, lower operating altitude, or lower temperatures.
DOD's report does make a number of recommendations that address
operational effectiveness and suitability as well as survivability
concerns. Operational effectiveness recommendations included the need to
conduct follow-on operational tests to assess V-22 survivability in
realistic landing zone tactical approaches. These tests and tactics
development are needed to expand the maneuvering flight envelope as much
as possible and to determine whether there is operational utility in the
use of more extreme helicopter-style maneuvering in a high-threat
environment. Operational suitability recommendations included the need to
implement upgrades to the passenger seats and harnesses. The report noted
that emergency dual engine failures in the conversion/vertical take-off
landing mode below 1,600 feet above the ground are unlikely to be
survivable. Survivability recommendations included the need to install and
test a defensive weapon.
Agency Comments
In commenting on a draft of this report, the V-22 program office provided
technical comments, which were incorporated as appropriate.
The Navy's VH-71A will be a dual-piloted, multiengine helicopter employed
by Marine Helicopter Squadron One to provide safe, reliable, and timely
transportation for the President and Vice President of the United States,
heads of state, and others in varied and at times adverse climatic and
weather conditions. When the President is aboard, the VH71A will serve as
the Commander in Chief's primary command and control platform. The system
will replace the VH-3D and VH-60N. It will be developed in two increments.
We assessed increment one.
Source: VH-71A Program Office.
Development start/ Design GAO Initial
production decision review review capability
(1/05) (12/05) (1/06) (10/09)
In January 2005, the VH-71A program began Attainment of Product Knowledge
system development and committed to production Production, without fully
maturing technologies, achieving design &
technology
design stability, or demonstrating production maturity maturity. Program
officials recognize that the VH71A is a nontraditional acquisition with
significant risks due to an aggressive schedule dictated by the Design &
technology
White House in 2002. They stated that most of the maturity
system's technologies are nondevelopmental and
are currently deployed on other platforms.
However, neither of the VH-71A's two critical
technologies were demonstrated in an operational Technology
environment at development start, and the maturity
program planned to have only 65 percent of its
drawings released by design review. Concurrency
in development, design, and production increases
the likelihood of cost growth and schedule delays
because components being procured may have to
be reworked to meet the final design. Development DOD Development GAO
start design (NA) review (NA)
start/production review decision (1/06) (1/05)
Technology Maturity
The VH-71A program's two critical technologies were nearing maturity when
the program began development and committed to production in January 2005.
Since then, one of those technologies, the 10-inch cockpit control
displays, matured. A prototype of the other critical technology, the
Communication and Subsystem Processing Embedded Resource Communications
Controller, has not been flight tested and is not projected to be
demonstrated in an operational environment until 2007. The program office
believes the risk associated with fully maturing this technology to be low
because the subsystem components that make up the technology are currently
flying on other operational aircraft. Program officials stated that most
of the VH-71A technologies were not identified as critical because they
were flying on the EH-101 helicopter, on which the VH-71A is based.
Design Stability and Production Maturity
In January 2005, the program committed to the production of five aircraft
without a final design or fully defined production processes. At that
time, 55 percent of the program's total estimated drawings were releasable
to manufacturing. Sixty-five percent were projected to be releasable by
the design review in December 2005, and 80 percent were expected to be
completed by early 2006, one year after the production decision. This
concurrency in design and production increases the likelihood of cost
growth and schedule delays because components being procured may have to
be reworked to meet the final design. According to program officials, the
drawings that have not been released are most likely related to modified
communications and navigation systems and software. The program considers
the design for the rest of the air vehicle and the production processes
for the system mature because they are based on the EH-101, which is
currently in service. However, design development will continue through
low rate initial production as the program concurrently develops its
manufacturing processes. The program will not collect statistical process
control data to demonstrate production maturity, but it will monitor
indicators, such as number of non-conforming products, quality
notifications, hours per process, and scrap and rework rates.
Other Program Issues
The VH-71A program's aggressive schedule increases risk in the test
program and negatively affects the program's ability to incorporate the
insights gained from testing in increment one. To mitigate some of the
schedule risk, the program has adopted a test philosophy that combines
contractor, development, and operational testing. The Director,
Operational Test and Evaluation, has not formally approved the program's
test plans and is working with the program to make the plans more
eventbased and to develop metrics to measure progress.
Congressional insight into the program is currently limited because the
program will not start reporting on progress against its cost, schedule,
and performance baselines until June 2006, at the earliest. This reporting
has been delayed because the program does not have an approved program
baseline, even though the decision to start development and production was
made in January 2005.
Agency Comments
In commenting on a draft of this assessment, the Navy provided technical
comments, which were incorporated as appropriate. Additionally, the Navy
stated that the program is executing an accelerated schedule driven by an
urgent White House need to replace existing assets. It believes the GAO
assessment does not emphasize the risk mitigation actions taken.
Specifically, the incremental development approach minimizes risk through
modification of a certified, fielded EH-101 to the VH71 configuration,
with high-risk items deferred to the second increment. According to the
Navy, this approach allows the program to meet schedule requirements while
mitigating acknowledged risks associated with concurrent design,
development, and procurement. Use of an existing aircraft for increment
one also takes advantage of established manufacturing, production,
logistics, and training capabilities while reducing the requirements for
flight test, and an aggressive integrated test approach maximizes early,
robust testing, including operational tests. Deferring high-risk
development work to increment two provides time to accomplish design,
development, and test activities associated with more traditional
development programs.
The Army's Extended Range Multipurpose Unmanned Aerial Vehicle, now called
Warrior, is to replace the Hunter Unmanned Aerial Vehicle. A system is
composed of 12 air vehicles, and 5 ground control stations with associated
ground data terminals and portable control stations. The system is
expected to provide reconnaissance, communications, signal intelligence,
lethal and nonlethal attack and interoperability with manned aviation
assets such as Apache and the Advanced Reconnaissance Helicopter.
Source: UAVS Project Office.
Development GAO Design Low-rate Full-rate Initial Last
start review review decision decision capability procurement
(4/05) (1/06) (6/06) (3/08) (11/09) (12/09) (TBD)
Currently two of Warrior's four critical technologies are mature. The
program expects to have matured the other two critical technologies by the
time of the program's design readiness review in June 2006. However, if
these technologies do not mature in time, the Army reports that it has two
mature back up technologies that can be used in their place. General
Atomics, which makes the Air Force Predator UAV, is the prime contractor
for the Warrior UAV. Program officials estimate that about 90 percent of
Warrior's design is nondevelopmental because it is already in use on
Predator or other systems.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/05) (1/06)
review (3/08) (6/06)
Technology Maturity
Program officials report that two of four critical technologies are
mature. These technologies are the heavy fuel engine, which is certified
by the Federal Aviation Administration, and the automatic take-off and
landing system whose technology is based on a similar system in use on the
Hunter and other unmanned aerial vehicle systems. The two remaining
critical technologies, the airborne ethernet and the multirole tactical
common data link, are expected to reach maturity before the June 2006
design readiness review.
The airborne ethernet provides communications capabilities among the
avionics, the payload data recorder, the air link data units and the
payloads and weapons systems. The technology will permit transmission of
live data rather than time-delayed data. Program officials assess risk to
the program associated with this technology as low because a mature
back-up technology exists and because the interfaces for the system--such
as payloads and the support equipment package, the payload data recorder
and the meteorological sensor--are in place. Further, officials stated
that Warrior's design is similar to that of the Air Force Predator A,
which is already fielded.
The multirole tactical common data link is being developed to support data
transmission at higher rates, provide interoperability with other systems,
such as the Apache, and provide for controlling the air vehicle itself
from other platforms. According to Army officials, the technology is based
on an Army program that is currently running 6 months ahead of the
schedule needed for introduction to the Warrior system. Similar to the
airborne ethernet, a tactical data link currently exists on other systems
and could be used for the Warrior to provide a capability but at a slower
rate and offering remote control of the payloads though not the entire
vehicle.
Design Stability
The Warrior UAV program office did not provide complete data on the number
of drawings expected or currently completed. As a result, we could not
assess current design maturity. Program officials did estimate, however,
that 90 percent of the system's design was non-developmental and is
already in use in the Predator or other systems. As a result, the Army
expects design stability by the time of the design review in June 2006.
Other Program Issues
Cost and quantity data reported in this assessment may change. The Army
has not decided how many Warrior systems it will buy. Since approving
development start in April 2005, the Army has increased the number it
plans to buy from 5 to 12 through fiscal year 2015. For this review, the
Army provided data on cost and quantities and its funding plan through
fiscal year 2015. However, program office officials stated that the Army
has not decided how many Warrior systems it will buy in total nor how long
the system will be produced.
Agency Comments
The program office provided technical comments, which we incorporated as
appropriate. Program office officials also stated that the Warrior design
utilizes basic airframe technology from the Predator A, but also borrows
from the Predator B design. Warrior's design is tied to the Apache Block
III (manned-unmanned teaming) and Future Combat Systems as a network
enabler.
WGS is a joint Air Force and Army program intended to provide essential
communications services to U.S. warfighters, allies, and coalition
partners during all levels of conflict short of nuclear war. It is the
next generation wideband component in DOD's future Military Satellite
Communications architecture and is composed of the following principal
segments: space segment (satellites), terminal segment (users), and
control segment (operators). We assessed the space segment.
Source: WGS Program Office.
Development start/ Design GAO First satellite Initial Full
production decision review review launch capability capability
(11/00) (7/02) (1/06) (6/07) (8/08) (6/13)
The WGS program's technology and design are Attainment of Product Knowledge
mature. We did not review production maturity Production, data because of
the commercial nature of the WGS
design & technology
acquisition, but unit level manufacturing is maturity essentially
complete. The contractor continues to experience problems assembling the
satellites. Improperly installed fasteners on a satellite Design &
technologysubcomponent have resulted in rework on the first maturity
satellite and extensive inspections of all three satellites currently
being fabricated. The program office estimates an increase of about $276.2
million for the program, largely due to cost growth Technologyresulting
from a production gap between satellites maturity three and four. The
launch of the first satellite has now been delayed for over 3 years and is
currently scheduled for June 2007. The delay will increase costs and add
at least 22 months to the time it takes to obtain an initial operational
capability
vel of knowledge
Desired le
from the system. Development DOD Development GAO start design
start/production review (NA) review decision (1/06)
(NA) (11/00)
Technology Maturity
WGS has two technologies that are vital to program success: the digital
channelizer and the phased array antenna. According to program officials,
both technologies were mature when the program made a production decision
in November 2000.
Design Stability
The WGS design is complete, as the program office has released all the
expected drawings to manufacturing.
Production Maturity
The commercial nature of the WGS acquisition contract precludes the
program office from having access to production process control data.
However, manufacturing for WGS is essentially complete, as all the units
have been manufactured and delivered for the first satellite.
Although the design for WGS is mature and development of the first
satellite is complete, the program continues to experience problems
assembling the satellites. For example, during the replacement of a
subcomponent on the first satellite, it was discovered that certain
fasteners had been improperly installed. Discovery of the problem resulted
in extensive inspections on all three satellites currently being
fabricated, with rework required on the first satellite. In all, 148
fasteners have been found that required rework and over 1,500 fasteners
per satellite required additional inspection or testing. The testing is
expected to be completed in the summer of 2006. According to program
officials, the contractor is considering initiatives to improve oversight
to avoid similar problems in the future.
Other Program Issues
Last year we reported a December 2005 launch date for the first WGS
satellite. This date slipped to March 31, 2006, because of a launch pad
conflict with a higher priority national security satellite. At that time,
the program office reported that the initial operational capability would
not be impacted by the schedule slip. However, the launch slipped again
when the fastener issue surfaced. The launch of the first satellite is now
scheduled for June 2007. The program office reports that the 15-month slip
in the schedule for all three satellites will add workforce and rework
costs (borne by the contractor) to the program and delay the time it takes
to obtain an initial operational capability by 22 months.
In December 2002, DOD directed the addition of WGS satellites four and
five as part of the Transformational Communications Architecture. The
purpose of these satellites will be to support increased bandwidth
required for the Airborne Intelligence, Surveillance, and Reconnaissance
mission. These satellites are to launch in fiscal years 2009 and 2010,
respectively. The current contract options must be extended and
renegotiated to cover the cost of the likely 2- to 3-year production gap
between satellites three and four. The program office has selected a
contractor and is currently negotiating the final contract price for
procuring satellites four and five. Preliminary estimates show that the
production gap is the main driver of the total overall cost increase of
about $276.2 million for the program. Because of the delays in the
schedule for the first three satellites, the program office is working
with the contractor to reassess the schedule for satellites four and five.
The results could impact the full operational capability date for the
system.
Agency Comments
In commenting on a draft of this assessment, the program office stated
that rework activities associated with the 148 improperly installed
fasteners have been completed and additional inspection and testing of the
remaining fasteners will be completed in 2006. The program office also
stated that the government and contractor are instituting increased levels
of oversight on the supplier's quality management program to avoid these
types of problems on future satellites.
WIN-T is the Army's high-speed and high-capacity backbone communications
network. It is to provide reliable, secure, and seamless video, data,
imagery, and voice services, allowing users to communicate simultaneously
at various levels of security. WIN-T is to connect Army units with higher
levels of command and provide Army's tactical portion of the Global
Information Grid. WIN-T is being fielded in blocks. We assessed the first
block.
Program/ GAO Design
development start review review
(7/03) (1/06) (TBD)
WIN-T entered system development with 3 of its 12 critical technologies
nearing maturity. While these 12 technologies will not be fully mature at
the time production begins, some were demonstrated during a recent
developmental test/operational test event; the program office expects that
all 12 will be assessed as nearing maturity based on an updated
independent technology readiness assessment that will be completed in
preparation for a milestone B "relook" scheduled for August 2006. While
design stability is evaluated during WIN-T's design reviews, it cannot be
assessed using our methodology because the program office does not track
the number of releasable drawings. However, the government will require
the contractor to deliver critical Interface Control Design documents,
which, according to the program office, will allow tracking of design
stability by an independent assessor.
Source: PM WIN-T.
Low-rate Full-rate decision decision (TBD) (TBD)
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD
start review design (7/03) (1/06) review (TBD)
Initial capability (TBD)
Production decision (TBD)
Technology Maturity
WIN-T entered system development with 3 of its 12 critical technologies
close to reaching full maturity. While program officials do not expect
these 12 technologies to reach full maturity until the network is built
and can be demonstrated in an operational environment, some of the
technologies were demonstrated in a relevant environment during a
developmental test/operational test event in November 2005; the Army Test
and Evaluation Command will complete its assessment of this event by April
2006. The WIN-T program office expects that all 12 critical technologies
will be assessed as close to fully mature following the evaluation of
results from this test event. An updated independent technology readiness
assessment will be performed in preparation for what the program office
has described as a WIN-T milestone B "re-look" currently scheduled for
August 2006. This updated assessment will include demonstration results
from the developmental test/operational test event.
Design Stability
Design stability could not be assessed using our methodology because the
program office does not plan to track the number of releasable drawings as
a design metric. According to the program, WIN-T is not a manufacturing
effort, but primarily an information technology system integration effort.
Consequently, the government does not obtain releasable design drawings
for many of WIN-T's components, particularly commercial components.
Instead, design stability is evaluated at the preliminary and critical
design reviews using the exit criteria developed by the government. For
the milestone B "re-look," the government will require the contractor to
deliver critical Interface Control Design documents which, according to
the program office, will allow tracking of design stability by an
independent assessor. According to DOD, the WIN-T design will evolve using
performance-based specifications and open systems design and is to conform
to DOD's Joint Technical Architecture, which specifies the minimun set of
standards and guidance for the acquisition of all DOD systems that
produce, use, or exchange information.
Other Program Issues
A major revision to the WIN-T acquisition strategy was completed in 2004.
In September 2004, DOD approved a decision to combine the competing
contractor teams for WIN-T's system design and development. The two
originally competing contractors are now teamed to establish a single
architecture for WIN-T that, according to the revised acquisition
strategy, will leverage each contractor's proposed architecture to provide
the Army with a superior technical solution for WIN-T. Establishing the
single WIN-T architecture a year earlier than originally planned is
expected to allow other Army programs to begin following that architecture
for the Future Force.
The global war on terrorism and the lessons learned from recent military
operations have shifted the Army's focus toward providing improved
communications and networking capabilities in the near term as well as for
the Future Force. The Army fielded a beyond-line-of-sight communications
network system in 2004 to units deployed in Iraq: the Joint Network Node.
This system is an improvement over past capabilities, but does not meet
all of WIN-T's requirements -- particularly for on-the-move communication.
Currently, the Army is assessing how best to transition the Joint Network
Node to WIN-T.
Also, in August 2005, the Department of the Army conducted a study that
explored options for better synchronizing three of its major system
development efforts--WIN-T, the Future Combat Systems, and the Joint
Tactical Radio System program. As a result of this study, the WIN-T
program will be rebaselined to meet emerging requirements; a new WIN-T
capability development document that will support the rebaselining of the
program is currently under review. A milestone B "re-look" to rebaseline
the program is planned for August 2006, and a new date for the WIN-T
production decision, originally scheduled for March 2006, will be
established then.
Agency Comments
In commenting on a draft of this assessment, the Army provided technical
comments, which were incorporated as appropriate.
DOD did not provide general comments on a draft of this report, but it did
Agency Comments
provide technical comments. These comments, along with agency comments
received on the individual assessments, were included as appropriate. (See
app. I for a copy of DOD's response.)
Scope of Our Review
For the 52 programs, each assessment provides the historical and current
program status and offers the opportunity to take early corrective action
when a program's projected attainment of knowledge diverges significantly
from the best practices. The assessments also identify programs that are
employing practices worthy of emulation by other programs. If a program is
attaining the desired levels of knowledge, it has less riskbut not zero
riskof future problems. Likewise, if a program shows a gap between
demonstrated knowledge and best practices, it indicates an increased
risknot a guarantee-of future problems. The real value of the assessments
is recognizing gaps early, which provides opportunities for constructive
intervention-such as adjustments to schedule, trade-offs in requirements,
and additional funding-before cost and schedule consequences mount.
We selected programs for the assessments based on several factors,
including (1) high dollar value, (2) stage in acquisition, and (3)
congressional interest. The majority of the 52 programs covered in this
report are considered major defense acquisition programs by DOD. A program
is defined as major if its estimated research and development costs exceed
$365 million or its procurement exceeds $2.19 billion in fiscal year 2000
constant dollars. (See app. II for details of the scope and methodology.)
We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretaries of the Army, Navy,
and Air Force; and the Director, Office of Management and Budget. We will
also make copies available to others upon request. In addition, the report
will be available at no charge on the GAO Web site at http://www.gao.gov.
If you have any questions on this report, please contact me at (202)
5124841 or Paul Francis at (202) 512-4841. Major contributors to this
report are listed in appendix IV.
Katherine V. Schinasi Managing Director Acquisition and Sourcing
Management
List of Congressional Committees
The Honorable John W. Warner Chairman The Honorable Carl Levin Ranking
Member Committee on Armed Services United States Senate
The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye Ranking
Member Subcommittee on Defense Committee on Appropriations United States
Senate
The Honorable Duncan Hunter Chairman The Honorable Ike Skelton Ranking
Member Committee on Armed Services House of Representatives
The Honorable C. W. Bill Young Chairman The Honorable John P. Murtha
Ranking Member Subcommittee on Defense Committee on Appropriations House
of Representatives
Scope and Methodology
In conducting our work, we evaluated performance and risk data from each
of the programs included in this report. We summarized our assessments of
each individual program in two components-a system profile and a product
knowledge assessment. We did not validate the data provided by the
Department of Defense (DOD). However, we took several steps to address
data quality. Specifically, we reviewed the data and performed various
quality checks, which revealed some discrepancies in the data. We
discussed the underlying data and these discrepancies with program
officials and adjusted the data accordingly. We determined that the data
provided by DOD were sufficiently reliable for our engagement purposes,
after reviewing DOD's management controls for assessing data reliability.
Data for the total planned investment of major defense acquisition
Macro Analysis
programs were obtained from funding-stream data included in DOD's selected
acquisition reports or from data obtained directly from the program
offices and then aggregated across all programs in base year 2006 dollars.
To assess the total cost, schedule, and quantity changes of the programs
included in our assessment presented in table 2 and on pages 6 and 7, it
was necessary to identify those programs with all of the requisite data
available. Of the 52 programs in our assessment, 26 constituted the common
set of programs where data were available for cost, schedule, and quantity
at the first full estimate, generally milestone B, and the latest
estimate. We excluded programs that had planning estimates as their first
full estimate and if the first full estimate and latest estimate fell
within a one year period of each other. Data utilized in this analysis
were drawn from information contained in selected acquisition reports or
data provided by program offices as of January 15, 2006. We summed the
costs associated with RDT&E and total costs consisting of research,
development, testing and evaluation, procurement, military construction,
and acquisition operation and maintenance. The schedule assessment is
based on the change in the average acquisition cycle time, defined as the
number of months between program start and the achievement of initial
operation capability or an equivalent fielding date.
The weighted calculations of acquisition cycle time and program
acquisition unit cost for the common set of programs were derived by
taking the total cost estimate for each of the 26 programs and dividing it
by the aggregate total cost of all 26 programs in the common set. The
resulting quotient for each program was then multiplied by the simple
percentage
Appendix II Scope and Methodology
change in program acquisition unit costs to obtain the weighted unit cost
change of each program. Next, the sum of this weighted cost change for all
programs was calculated to get the weighted unit cost change for the
common set as a whole. To assess the weighted-average acquisition cycle
time change, we multiplied the weight calculation by the acquisition cycle
time estimate for each corresponding program. A simple average was then
taken to calculate the change between the first full estimate and the
latest estimate. We believe these calculations best represent the overall
progress of programs by placing them within the context of the common
set's aggregate cost.
To assess the percentage of programs with technology maturity, design
stability, and production maturity at each key juncture presented in
figure 1 and on pages 11 and 12, we identified programs that had actually
proceeded through each key juncture-development start, system design
review, and production start-and obtained their assessed maturity. The
percentage in figure 1 and on pages 11 and 12 include programs in the 2006
assessment only. The population size for the technology maturity at
development start is 30 programs; design review is 21 programs; and
production start is 15 programs. The population size for the design
stability at design review is 20 programs; and 12 programs at production
start. The population size for production maturity at production start is
16 programs. This information was drawn from data provided by the program
office as of January 15, 2006. For more information, see the product
knowledge assessment section in this appendix.
For the historical RDT&E cost growth analysis in figure 2, we selected
Historical Analysis
programs that have completed 100 percent of their product development
cycle-defined as the period of time between the start of the system
development and demonstration phase and the start of production. We
identified 29 programs that are now in production or have been completed
since 1998. We reviewed information provided in DOD's Selected Acquisition
Reports (SARs) or through schedule information we obtained from program
officials via our assessments to determine which programs are complete and
which ones are in production. We also reviewed the DOD Selected
Acquisition Report Summary tables to identify completed programs. We chose
completed programs that had a final SAR report month of December 1998 or
later. We also chose programs that only had a development estimate
baseline rather than a production estimate baseline because we could then
calculate an associated product development cycle time.
Appendix II Scope and Methodology
Development start refers to the commitment to system development that
coincides with either milestone II or milestone B, which begins DOD's
system development and demonstration phase. The product development cycle
concludes with production. The production decision generally refers to the
decision to enter the production and deployment phase, typically with low
rate initial production. To identify the conclusion of the cycle, or
development end, we first attempted to establish the date of the low rate
initial production decision. If this date was not available we then used
the milestone C or III date, or the production estimate date. We
identified these dates using the latest SAR for each individual program or
through schedule information obtained from program officials via our
assessments. Once the product development cycle dates were identified, we
then converted the time between the two dates into a number of months for
each program.
For each of the 29 programs in our analysis, we identified the RDT&E
development estimate and each subsequent estimate of RDT&E costs
throughout the product development cycle by reviewing each of the
program's SAR. Each SAR report date was then used in calculating the
percentage into the product development cycle where the estimate fell.
Once these calculations were completed for each of the 29 programs, we
aggregated the RDT&E estimates at each percentage point from 1 to 100
percent. The end result was the cumulative cost change in 2006 dollars for
29 programs from the development estimate with a cost change plotted for
each point from 1 to 100 percent complete. For example, the AIM 9X Air to
Air Missile's product development cycle was 45 months. The development
estimate for RDT&E was $602.2 million in December 1996. The first SAR
after development start was the December 1997 SAR, which reported an RDT&E
estimate of $589.9 million (2006 dollars). The December 1997 SAR was 12
months into development or approximately 27 percent into the product
development cycle. Since estimates are reported on an annual basis, the
initial development estimate for the AIM-9X was carried through up to 26
percent of the cycle time, the 1997 SAR estimate was then plotted at 27
percent and carried through up to the next reporting period, December
1998, which was plotted at 53 percent and so forth until 100 percent of
the cycle time was completed. Once this was completed for all programs, we
were able to identify the RDT&E cost growth trend for all 29 programs.
To identify the average critical design review date we obtained the latest
date as reported in the program's latest SAR or as provided to us via our
program assessments. If the critical design review date was not included
in the SAR, we attempted to contact the current program manager and obtain
Appendix II Scope and Methodology
System Profile Data on Each Individual Two-Page Assessment
the date. We were able to identify 21 critical design review dates for the
29 programs. Once this date was identified, we calculated the percentage
into the development cycle the critical design review occurred. For
example, the AIM-9X SAR reported that the critical design review took
place in March of 1998, approximately 15 months, or 33 percent, into the
45 month development cycle. Next, we calculated a weighted average design
review date for the 21 programs. The weighted calculations were derived by
taking the latest RDT&E cost estimate at the completion of the product
development cycle for each of the 21 programs and dividing it by the sum
of all 21 programs. The resulting quotient for each program was then
multiplied by the percentage into the product development cycle when the
design review occurred. This resulted in a weighted calculation that was
then summed across all 21 programs. The result was the weighted average
design review percentage.
The maximum RDT&E increase for the 21 design review programs was 129.10
percent for the V-22 program. The minimum RDT&E increase for the 21
programs was -15.9 percent for the Joint Primary Aircraft Training System.
The graphic on page 14 displays the RDT&E cost trend for all 29 programs
and is not limited to the 21 programs with design review dates. We found
the same trend of RDT&E cost growth occurred for the 29 programs as for
the 21 programs.
In the past 5 years, DOD revised its policies governing weapon system
acquisitions and changed the terminology used for major acquisition
events. To make DOD's acquisition terminology more consistent across the
52 program assessments, we standardized the terminology for key program
events. In the individual program assessments, program start refers to the
initiation of a program; DOD usually refers to program start as milestone
I or milestone A, which begins the concept and technology development
phase. Similarly, development start refers to the commitment to system
development that coincides with either milestone II or milestone B, which
begins DOD's system development and demonstration phase. The production
decision generally refers to the decision to enter the production and
deployment phase, typically with low rate initial production. Initial
capability refers to the initial operational capability, sometimes also
called first unit equipped or required asset availability. For the MDA
programs that do not follow the standard DOD acquisition model, but
instead develop systems in incremental capability-based blocks, we
identified the key technology development efforts that lead to an initial
capability for the block assessed.
Appendix II Scope and Methodology
The information presented on the funding needed to complete from fiscal
2006 through completion, unless otherwise noted, draws on information from
SARs or on data from the program office. In some instances the data were
not yet available, and we annotate this by the term "to be determined"
(TBD), or not applicable, annotated (NA). The quantities listed only refer
to procurement quantities. Satellite programs, in particular, produce a
large percentage of their total operational units as development
quantities, which are not included in the quantity figure.
To assess the cost, schedule, and quantity changes of each program, we
reviewed DOD's SARs or obtained data directly from the program offices. In
general, we compared the latest available SAR information with a baseline
for each program. For programs that have started product development-those
that are beyond milestone II or B-we compared the latest available SAR to
the development estimate from the first selected acquisition report issued
after the program was approved to enter development. For systems that have
not yet started system development, we compared the latest available data
to the planning estimate issued after milestone I or A. For systems not
included in SARs, we attempted to obtain comparable baseline and current
data from the individual program offices. For MDA systems for which a
baseline was not available, we compared the latest available cost
information to the amount reported last year.
All cost information is presented in base year 2006 dollars using Office
of the Secretary of Defense approved deflators to eliminate the effects of
inflation. We have depicted only the programs' main elements of
acquisition cost-research and development and procurement; however, the
total program costs also include military construction and acquisition
operation and maintenance costs. Because of rounding and these additional
costs, in some situations the total cost may not match the exact sum of
the research and development and procurement costs. The program unit costs
are calculated by dividing the total program cost by the total quantities
planned. These costs are often referred to as program acquisition unit
costs. In some instances, the data were not applicable, and we annotate
this by using the term "NA." In other instances, the current absence of
data on procurement funding and quantities precludes calculation of a
meaningful program acquisition unit cost, and we annotate this by using
the term "TBD." The quantities listed refer to total quantities, including
both procurement and development quantities.
The schedule assessment is based on acquisition cycle time, defined as the
number of months between the program start, usually milestone I or A, and
Appendix II Scope and Methodology
Product Knowledge Data on Each Individual Two-Page Assessment
the achievement of initial operational capability or an equivalent
fielding date. In some instances, the data were not yet available, and we
annotate this by using the term TBD, or were classified.
The intent of these comparisons is to provide an aggregate or overall
picture of a program's history. These assessments represent the sum total
of the federal government's actions on a program, not just those of the
program manager and the contractor. DOD does a number of detailed analyses
of changes that attempt to link specific changes with triggering events or
causes. Our analysis does not attempt to make such detailed distinctions.
To assess the product development knowledge of each program at key points
in development, we submitted a data collection instrument to each program
office. The results are graphically depicted in each two-page assessment.
We also reviewed pertinent program documentation, such as the operational
requirements document, the acquisition program baseline, test reports, and
major program reviews.
To assess technology maturity, we asked program officials to apply a tool,
referred to as technology readiness levels, for our analysis. The National
Aeronautics and Space Administration originally developed technology
readiness levels, and the Army and Air Force Science and Technology
research organizations use them to determine when technologies are ready
to be handed off from science and technology managers to product
developers. Technology readiness levels are measured on a scale of one to
nine, beginning with paper studies of a technology's feasibility and
culminating with a technology fully integrated into a completed product.
(See app. III for the definitions of technology readiness levels.) Our
best practices work has shown that a technology readiness level of 7-
demonstration of a technology in an operational environment-is the level
of technology maturity that constitutes a low risk for starting a product
development program. In our assessment, the technologies that have reached
technology readiness level 7, a prototype demonstrated in an operational
environment, are referred to as mature or fully mature and those that have
reached technology readiness level 6, a prototype demonstrated in a
relevant environment, are referred to as approaching or nearing maturity
and are assessed as attaining 50 percent of the desired level of
knowledge. Satellite technologies that have achieved technology readiness
level 6 are assessed as fully mature due to the difficulty of
demonstrating maturity in an operational environment-space.
Appendix II Scope and Methodology
In most cases, we did not validate the program offices' selection of
critical technologies or the determination of the demonstrated level of
maturity. We sought to clarify the technology readiness levels in those
cases where information existed that raised concerns. If we were to
conduct a detailed review, we might adjust the critical technologies
assessed, the readiness level demonstrated, or both. It was not always
possible to reconstruct the technological maturity of a weapon system at
key decision points after the passage of many years.
To assess design stability, we asked program officials to provide the
percentage of engineering drawings completed or projected for completion
by the design review, the production decision, and as of our current
assessment. In most cases, we did not verify or validate the percentage of
engineering drawings provided by the program office. We sought to clarify
the percentage of drawings completed in those cases where information
existed that raised concerns. Completed engineering drawings were defined
as the number of drawings released or deemed releasable to manufacturing
that can be considered the "build-to" drawings.
To assess production maturity, we asked program officials to identify the
number of critical manufacturing processes and, where available, to
quantify the extent of statistical control achieved for those processes.
In most cases, we did not verify or validate this information provided by
the program office. We sought to clarify the number of critical
manufacturing processes and percentage of statistical process control
where information existed that raised concerns. We used a standard called
the Process Capability Index, which is a process performance measurement
that quantifies how closely a process is running to its specification
limits. The index can be translated into an expected product defect rate,
and we have found it to be a best practice. We sought other data, such as
scrap and rework trends, in those cases where quantifiable statistical
control data were unavailable.
Although the knowledge points provide excellent indicators of potential
risks, by themselves, they do not cover all elements of risk that a
program encounters during development, such as funding instability. Our
detailed reviews on individual systems normally provide for a fuller
treatment of risk elements.
Technology Readiness Levels
Hardware Demonstration Technology Readiness Level Description Software
Environment
alytical
studies and laboratory studies to physically validate analytical
predictions of separate elements of the technology. Examples include
components that are not yet integrated or representative.
Analytical studies and Lab demonstration of nonscale individual components
(pieces of subsystem).
4. Component and/or breadboard. Validation in laboratory environment.
Basic technological components are integrated to establish that the pieces
will work together. This is relatively "low fidelity" compared to the
eventual system. Examples include integration of "ad hoc" hardware in a
laboratory.
Low fidelity breadboard. Lab Integration of nonscale components to show
pieces will work together. Not fully functional or form or fit but
representative of technically feasible approach suitable for flight
articles.
5. Component and/or breadboard validation in relevant environment.
Fidelity of breadboard technology increases significantly. The basic
technological components are integrated with reasonably realistic
supporting elements so that the technology can be tested in a simulated
environment. Examples include "high fidelity" laboratory integration of
components.
High fidelity breadboard. Functionally equivalent but not necessarily form
and/or fit (size weight, materials, etc.). Should be approaching
appropriate scale. May include integration of several components with
reasonably realistic support elements/subsystems to demonstrate
functionality.
Lab demonstrating functionality but not form and fit. May include flight
demonstrating breadboard in surrogate aircraft. Technology ready for
detailed design studies.
Appendix III Technology Readiness Levels
(Continued From Previous Page)
Hardware Demonstration
Technology Readiness Level Description Software Environment
6. System/subsystemmodel or prototype demonstration in a relevant
environment.
Representative model or prototype system, which is well beyond the
breadboard tested for TRL 5, is tested in a relevant environment.
Represents a major step up in a technology's demonstrated readiness.
Examples include testing a prototype in a high fidelity laboratory
environment or in simulated operational environment.
Prototype-Should be very close to form, fit and function. Probably
includes the integration of many new components and realistic supporting
elements/subsystems if needed to demonstrate full functionality of the
subsystem.
High-fidelity lab demonstration or limited/restricted flight demonstration
for a relevant environment. Integration of technology is well defined.
7. System prototype demonstration in an operational environment.
Prototype near or at planned operational system. Represents a major step
up from TRL 6, requiring the demonstration of an actual system prototype
in an operational environment, such as in an aircraft, vehicle or space.
Examples include testing the prototype in a test bed aircraft.
Prototype. Should be form, fit and function integrated with other key
supporting elements/subsystems to demonstrate full functionality of
subsystem.
Flight demonstration in representative operational environment such as
flying test bed or demonstrator aircraft. Technology is well substantiated
with test data.
8. Actual system Technology has been Flight DT&E in the
completed and proven to work in its qualified actual system
hardware
"flight qualified" final form and under application
through test and expected conditions.
demonstration. In almost all cases, this
TRL represents
the end of true system
development.
Examples include
developmental test and
evaluation of the system
in its intended
weapon system to
determine if it meets
design specifications.
9. Actual system Actual application of the Actual system OT&E in
"flight proven" technology in its in final form operational
through successful final form and under mission
mission mission conditions, conditions
operations. such as those encountered
in operational
test and evaluation. In
almost all cases,
this is the end of the
last "bug fixing"
aspects of true system
development.
Examples include using
the system under
operational mission
conditions.
Source: GAO and its analysis of National Aeronautics and Space Administration
data.
GAO Contact and Acknowledgments
Paul L. Francis (202) 512-4841
GAO Contact
Acknowledgments
David B. Best, Alan R. Frazier, and Bruce H. Thomas made key contributions
to this report. Other key contributors included Robert L. Ackley, D.
Catherine Baltzell, Ridge C. Bowman, Maricela Cherveny, Thomas J. Denomme,
Arthur Gallegos, William R. Graveline, David J. Hand, Michael J. Hazard,
Barbara H. Haynes, LaTonya D. Miller, John E. Oppenheim, Rae Ann H. Sapp,
Wendy P. Smythe, Robert S. Swierczek, and Karen S. Zuckerstein.
The following staff were responsible for individual programs:
System Primary Staff
Airborne Laser (ABL) LaTonya D. Miller
Aerial Common Sensor (ACS) Dayna L. Foster/Michael W. Aiken
Advanced Deployable System (ADS) Cristina A. Connelly/ Diana L.
Dinkelacker
Aegis Ballistic Missile Defense (Aegis BMD) Ivy G. Hubler
Advanced Extremely High Frequency Satellites Bradley L. Terry (AEHF)
Active Electronically Scanned Array Radar Joseph E. Dewechter/Jerry W.
Clark (AESA)
Advanced Precision Kill Weapon System Michele R. Williamson/Wendy P.
Smythe (APKWS)
Advanced SEAL Delivery System (ASDS) Mary K. Quinlan
Advanced Threat Infrared Danny G. Owens/Leon S. Gill Countermeasure/Common
Missile Warning System (ATIRCM/CMWS)
B-2 Radar Modernization Program (B-2 RMP) Don M. Springman/Andrew H. Redd
C-130 Avionics Modernization Program Marvin E. Bonner/Sean D. Merrill
(C-130 AMP)
C-5 Avionics Modernization Program (C-5 Cheryl K. Andrew/Sameena N.
Ismailjee AMP)
C-5 Reliability Enhancement and Reengining Sameena N. Ismailjee/Cheryl K.
Andrew Program (C-5 RERP)
CH-47F Improved Cargo Helicopter (CH-47F) Wendy P. Smythe/ Danny G. Owens
Future Aircraft Carrier (CVN-21) Brendan S. Culley/Trevor J. Thomson
(Continued From Previous Page)
System Primary Staff
DD(X) Destroyer J. Kristopher Keener/Marc J.
Castellano/
Christopher R. Durbin
E-2 Advanced Hawkeye (E-2 AHE) Gary L. Middleton/Judy T. Lasley
Evolved Expendable Launch Vehicle Maria A. Durant
(EELV)
Expeditionary Fighting Vehicle (EFV) Leon S. Gill/Danny G. Owens/
Steven B. Stern
Excalibur Precision Guided Extended John P. Swain/Carrie R. Wilson
Range
Artillery Projectile
F-22A Raptor Marvin E. Bonner
Future Combat Systems (FCS) Marcus C. Ferguson/John P. Swain/
Guisseli Reyes
Global Hawk Unmanned Aerial Vehicle Bruce D. Fairbairn/Charlie Shivers
Ground-Based Midcourse Defense (GMD) Ivy G. Hubler
NAVSTAR Global Positioning System II Jean N. Harker/Peter J. Grana
(GPS)
II Modernized Space/OCS
Joint Land Attack Cruise Missile Alan R. Frazier/Wendy P. Smythe
Defense
Elevated Netted Sensor System (JLENS)
Joint Strike Fighter (JSF) Matthew B. Lea/Matthew T. Drerup
Joint Tactical Radio System Airborne, Paul G. Williams/Ridge C. Bowman
Maritime, Fixed-Site (JTRS AMF)
Joint Tactical Radio System (JTRS) Ridge C. Bowman/Paul G. Williams
Cluster 1
Joint Tactical Radio System (JTRS) Ridge C. Bowman/ Paul G.
Cluster 5
Williams/Tristan T. To
Joint Unmanned Combat Air Systems Bruce D. Fairbairn/Charlie Shivers
(J-UCAS)
Kinetic Energy Interceptors (KEI) Jonathan E. Watkins
Land Warrior Joel C. Christenson/Susan K.
Woodward
Littoral Combat Ship (LCS) J. Kristopher Keener/Christina A.
Connelly/Christopher R. Durbin
Longbow Apache Block III Wendy P. Smythe/Danny G. Owens
Multi-mission Maritime Aircraft (MMA) Matthew F. Ebert/Heather L. Barker
21" Mission Reconfigurable Unmanned Diana L. Dinkelacker/Marc J.
Castellano
Undersea Vehicle (MRUUV)
Mobile User Objective System (MUOS) Richard Y. Horiuchi
MQ-9 Predator B Rae Ann H. Sapp
National Polar-orbiting Operational Suzanne S. Olivieri/ Lisa P.
Environmental Satellite System (NPOESS) Gardner/Carol R. Cha
PATROIT/Medium Extended Air Defense Tana M. Davis
System (MEADS) Combined Aggregate
Program (CAP) Fire Unit
(Continued From Previous Page)
System Primary Staff
Space Based Infrared System High Maricela Cherveny/Leslie K. Pollock
(SBIRS High)
Small Diameter Bomb (SDB) Carrie R. Wilson/ Guisseli Reyes
Space Radar (SR) Tony A. Beckham
Space Tracking and Surveillance System Sigrid L. McGinty (STSS)
Terminal High Altitude Area Defense (THAAD) Jonathan E. Watkins
Transformational Satellite Communications Arturo Holguin Jr. System (TSAT)
V-22 Joint Services Advanced Vertical Lift Jerry W. Clark/Bonita P. Oden
Aircraft (V-22)
VH-71A Presidential Helicopter Replacement Ronald E. Schwenn/Joseph H.
Zamoyta/ Program Kevin J. Heinz
Warrior Unmanned Aerial Vehicle Carol T. Mebane/Michele R. Williamson
(Warrior UAV)
Wideband Gapfiller Satellites (WGS) Tony A. Beckham
Warfighter Information Network-Tactical James P. Tallon/Gwyneth M.
Blevins/ (WIN-T) Paul G. Williams/Amy L. Sweet
Source: GAO.
Related GAO Products
Best Practices: Better Support of Weapon System Program Managers Needed to
Improve Outcomes. GAO-06-110. Washington, D.C.: November 30, 2005.
DOD Acquisition Outcomes: A Case for Change. GAO-06-257T. Washington,
D.C.: November 15, 2005.
Defense Acquisitions: Stronger Management Practices Are Needed to Improve
DOD's Software-Intensive Weapon Acquisitions. GAO-04-393 . Washington,
D.C.: March 1, 2004.
Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices, but
More Controls Are Needed. GAO-04-53 . Washington, D.C.: November 10, 2003.
Best Practices: Setting Requirements Differently Could Reduce Weapon
Systems' Total Ownership Costs. GAO-03-57 . Washington, D.C.: February 11,
2003.
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. GAO-02-701 . Washington, D.C.: July 15,
2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best Practices.
GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to Better
Weapon System Outcomes. GAO-01-288 . Washington, D.C.: March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better Weapon
System Outcomes. GAO/NSIAD-00-199 . Washington, D.C.: July 31, 2000.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. GAO/T-NSIAD-00-137 . Washington, D.C.: April 26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon System Program
Implement Best Practices. GAO/NSIAD-99-206 . Washington, D.C.: August 16,
1999.
Related GAO Products
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes . GAO/NSIAD-99-162. Washington, D.C.: July 30,
1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program
Outcomes. GAO/T-NSIAD-99-116 . Washington, D.C.: March 17, 1999.
Defense Acquisition: Improved Program Outcomes Are Possible.
GAO/T-NSIAD-98-123 . Washington, D.C.: March 18, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD's Environment. GAO/NSIAD-98-56 . Washington, D.C.: February
24, 1998.
Major Acquisitions: Significant Changes Underway in DOD's Earned Value
Management Process. GAO/NSIAD-97-108 . Washington, D.C.: May 5, 1997.
Best Practices: Commercial Quality Assurance Practices Offer Improvements
for DOD. GAO/NSIAD-96-162 . Washington, D.C.: August 26, 1996.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Contact:
To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202) 512-4800
Public Affairs
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***