Defense Acquisitions: Assessments of Selected Major Weapon
Programs (31-MAR-05, GAO-05-301).
The Department of Defense (DOD) is embarking on a number of
efforts to enhance warfighting and the way the department
conducts business. Major investments are being made to develop
improved weapon systems to combat various threats to U.S.
security. While the weapons that DOD ultimately develops have no
rival in superiority, weapon systems acquisition remains a
long-standing high-risk area. GAO's reviews over the past 30
years have found consistent problems with weapon acquisitions
such as cost increases, schedule delays, and performance
shortfalls. In addition, DOD faces several budgetary challenges
that underscore the need to deliver its new major weapon programs
within estimated costs and to obtain the most from those
investments. DOD can help resolve these problems by using a more
knowledge-based approach for developing new weapons. This report
provides congressional and DOD decision makers with an
independent, knowledge-based assessment of selected defense
programs that identifies potential risks and needed actions when
a program's projected attainment of knowledge diverges from the
best practice. It can also highlight those programs that employ
practices worthy of emulation by other programs. GAO plans to
update and issue this report annually.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-301
ACCNO: A20557
TITLE: Defense Acquisitions: Assessments of Selected Major
Weapon Programs
DATE: 03/31/2005
SUBJECT: Best practices
Best practices methodology
Defense procurement
Program evaluation
Program management
Weapons research and development
Weapons systems
Cost analysis
Defense capabilities
Schedule slippages
Systems evaluation
Missiles
Defense cost control
Military research and development
Strategic planning
Advanced SEAL Delivery System
Air Force Advanced Extremely High
Frequency Satellite Program
Air Force B-2 Radar Modernization
Program
Air Force E-10A Multi-Sensor Command and
Control Aircraft
Air Force Evolved Expendable Launch
Vehicle Program
Air Force Small Diameter Bomb
Air Force Transformational Satellite
Communications System
Airborne Mine Neutralization System
Army Advanced Precision Kill Weapon
System Program
Army Advanced Threat Infrared
Countermeasure/Common Missile Warning
System
Army Compact Kinetic Energy Missile
Army Excalibur Precision Guided Extended
Range Artillery Projectile
Army Future Combat Systems
Army Land Warrior System
Army Warfighter Information Network
C-130 Avionics Modernization Program
C-5 Avionics Modernization Program
C-5 Reliability Enhancement and
Reengining Program
CH-47F Helicopter
DD(X) Destroyer
DOD Space Tracking and Surveillance
System
E-2 Aircraft
E-2C Aircraft
EA-18G Aircraft
F-22 Raptor Aircraft
F/A-22 Aircraft
Global Hawk Unmanned Aerial Vehicle
Hawkeye E-2C Aircraft
Joint Air-to-Surface Standoff Missile
Joint Common Missile
Joint Standoff Weapon
Joint Strike Fighter
Joint Tactical Radio System
Joint Unmanned Combat Air Systems
Program
Littoral Combat Ship
Marine Corps Expeditionary Fighting
Vehicle
Marine Corps Heavy Lift Replacement
System
MDA Aegis Ballistic Missile Defense
Program
MDA Airborne Laser Program
MDA Ground-Based Midcourse Defense
Program
MDA Kinetic Energy Interceptor System
MDA Terminal High Altitude Area Defense
System
Medium Extended-Range Air Defense System
MQ-9 Predator B
Multi-Mission Maritime Aircraft
National Polar-Orbiting Operational
Environmental Satellite System
NAVSTAR Global Positioning System II
Modernized Space/OCS
Navy Active Electronically Scanned Array
Radar Program
Navy Cooperative Engagement Capability
System
Navy Extended Range Guided Munition
Navy Future Aircraft Carrier CVN-21
Navy Mobile User Objective System
Navy Tactical Tomahawk Missile
Osprey Aircraft
Space Based Infrared System-High
V-22 Aircraft
V-22 Joint Services Advanced Vertical
Lift Aircraft
Wideband Gapfiller Satellites
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-301
United States Government Accountability Office
GAO Report to Congressional Committees
March 2005
DEFENSE ACQUISITIONS
Assessments of Selected Major Weapon Programs
a
GAO-05-301
[IMG]
March 2005
DEFENSE ACQUISITIONS
Assessments of Selected Major Weapon Programs
What GAO Found
GAO assessed 54 programs, which represent an investment of over $800
billion, ranging from the Missile Defense Agency's Airborne Laser to the
Army's Warfighter Information Network-Tactical. GAO's assessments are
anchored in a knowledge-based approach to product development that
reflects best practices of successful programs. This approach centers on
attaining high levels of knowledge in three elements of a new product or
weapon-technology, design, and production-at key consecutive junctures in
development. If a program is not attaining these levels of knowledge, it
incurs increased risk of technical problems, with significant potential
cost and schedule growth implications (see figure). If a program is
falling short in one element, like technology maturity, it is harder to
attain the requisite amount of knowledge to prudently proceed in
succeeding elements.
Attainment of Product Knowledge
Production,design &technology maturity
Design &technology maturity
Technology maturity
Development DOD Production
start design decision Source: GAO. review
The majority of programs GAO assessed are costing more and taking longer
to develop than planned. Most of the programs proceeded with less
knowledge at critical junctures than suggested by best practices, although
some programs came close to meeting best practice standards. For example,
technology and design for the F/A-22 matured late in the program
contributing to large cost growth and schedule delays. The JASSM program,
in contrast, has achieved a high level of knowledge at critical junctures
while experiencing minimal cost increases or schedule delays.
Managing these levels of knowledge takes on additional significance as
DOD's share of the discretionary budget faces increasing pressure from the
growth in mandatory spending and the demands of ongoing military
operations. For these reasons, if DOD approves programs with low levels of
knowledge and accepts the attendant likely adverse cost and schedule
consequences, it will probably get fewer quantities for the same
investment or face difficult choices on which investments it cannot afford
to pursue.
United States Government Accountability Office
Contents
Foreword vii
Letter 1
A Challenging Time for Weapon System Investments 1
Current Programs Are Costing More and Taking Longer to
Develop 5
AKnowledge-Based Approach Can LeadtoBetter Acquisition
Outcomes 6
Most Programs Have Proceeded with Lower Levels of
Knowledge at Critical Junctures 8
Assessments of Individual Programs 14
Airborne Laser (ABL) 15
Aegis Ballistic Missile Defense (Aegis BMD) 17
Advanced Extremely High Frequency Satellites (AEHF) 19
Active Electronically Scanned Array Radar (AESA) 21
Airborne Mine Neutralization System (AMNS) 23
Advanced Precision Kill Weapon System (APKWS) 25
Advanced SEAL Delivery System (ASDS) 27
Advanced Threat Infrared Countermeasure/Common Missile
Warning System (ATIRCM/CMWS) 29
B-2 Radar Modernization Program (B-2 RMP) 31
C-130 Avionics Modernization Program (C-130 AMP) 33
C-5 Avionics Modernization Program (C-5 AMP) 35
C-5 Reliability Enhancement and Reengining Program
(C-5 RERP) 37
Cooperative Engagement Capability (CEC) 39
CH-47F Improved Cargo Helicopter (CH-47F) 41
Compact Kinetic Energy Missile (CKEM) 43
Future Aircraft Carrier CVN-21 45
DD(X) Destroyer 47
E-10A Multi-Sensor Command and Control Aircraft (E-10A) 49
E-2 Advanced Hawkeye (E-2 AHE) 51
EA-18G 53
Evolved Expendable Launch Vehicle (EELV) - Atlas V,
Delta IV 55
Page i GAO-05-301 Assessments of Selected Major Weapon Programs
Contents
Expeditionary Fighting Vehicle (EFV)
Extended Range Guided Munition (ERGM)
Excalibur Precision Guided Extended Range Artillery
Projectile
F/A-22 Raptor
Future Combat Systems (FCS)
Global Hawk Unmanned Aerial Vehicle
Ground-Based Midcourse Defense (GMD)
Navstar Global Positioning System (GPS) II Modernized
Space/OCS
Heavy Lift Replacement (HLR)
Joint Air-to-Surface Standoff Missile (JASSM)
Joint Common Missile (JCM)
Joint Strike Fighter (JSF)
Joint Standoff Weapon (JSOW)
Joint Tactical Radio System (JTRS) Cluster 1
Joint Tactical Radio System (JTRS) Cluster 5
Joint Unmanned Combat Air Systems (J-UCAS)
Kinetic Energy Interceptors (KEI)
Land Warrior
Littoral Combat Ship (LCS)
Medium Extended Air Defense System (MEADS)
Multi-mission Maritime Aircraft (MMA)
Mobile User Objective System (MUOS)
57 59
61 63 65 67 69
71 73 75 77 79 81 83 85 87 89 91 93 95 97 99
MQ-9 Predator B 101
National Polar-orbiting Operational Environmental Satellite
System (NPOESS) 103
Space Based Infrared System (SBIRS) High 105
Small Diameter Bomb (SDB) 107
Space Tracking and Surveillance System (STSS) 109
Terminal High Altitude Area Defense (THAAD) 111
Tactical Tomahawk Missile 113
Transformational Satellite Communications System (TSAT) 115
V-22 Joint Services Advanced Vertical Lift Aircraft 117
Contents
Wideband Gapfiller Satellites (WGS) 119 Warfighter Information
Network-Tactical (WIN-T) 121
Agency Comments and Our Evaluation 123 Scope of Our Review 123
Appendixes
Appendix I: Comments from the Department of Defense 126 Appendix II: Scope
and Methodology 127 Appendix III: Technology Readiness Levels 133 Appendix
IV: GAO Contact and Acknowledgments 135
Related GAO Products
Tables Table 1: Examples of Programs with Reduced Buying Power 4
Table 2: Cost and Cycle Time Growth for 26 Weapon Systems 5
Table 3: Cost and Cycle Time for the Same Programs: 2004
Assessment and 2005 Assessment 6
Figures Figure 1: RDT&E and Procurement Funding-Major Defense
Acquisition Programs 2
Figure 2: Percent of Programs That Achieved Technology Maturity
at Key Junctures 9
Figure 3: Percent of Programs Achieving Design Stability at Key
Junctures 10
Figure 4: Depiction of a Notional Weapon System's Knowledge as
Compared with Best Practices 13
Page iii GAO-05-301 Assessments of Selected Major Weapon Programs
Contents
Abbreviations
ACTS AEHF Comsec/Transec System
BAMS Broad Area Maritime Surveillance
BTERM Ballistic Trajectory Extended Range Munition
DARPA Defense Advanced Research Projects Agency
DCMA Defense Contract Management Agency
DOD Department of Defense
EKV exoatmospheric kill vehicle
FY fiscal year
GAO Government Accountability Office
GEO geosynchronous earth orbit
GPS Global Positioning System
HEO highly elliptical orbit
HLV heavy lift vehicle
IMIS Integrated Maintenance Information System
ISR intelligence, surveillance and reconnaissance
JDAM Joint Direct Attack Munition
JSSEO Joint Single Integrated Air Picture Systems Engineering
Organization MDA Missile Defense Agency NA not applicable NASA National
Aeronautics and Space Administration NATO North Atlantic Treaty
Organization NOAA National Oceanic and Atmospheric Administration OT&E
Operational Test and Evaluation PDR Preliminary Design Review RDT&E
Research, Development, Test, and Evaluation SDACS Solid Divert and
Attitude Control System SM-3 Standard Missile 3 TBD to be determined TF/TA
Terrain Following and Terrain Avoidance TRL Technology Readiness Level UAV
Unmanned Aerial Vehicle UHF ultra high frequency U.S.C. United States Code
USMC United States Marine Corps
Contents
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
Page v GAO-05-301 Assessments of Selected Major Weapon Programs
Comptroller General of the United States
United States Government Accountability Office Washington, D.C. 20548
March 31, 2005
Congressional Committees
Fiscal realities demand that the Department of Defense (DOD) get better
outcomes from its weapon system investments. Federal discretionary
spending, along with other federal policies and programs, will face
serious budget pressures in the coming years. While providing for the
common defense is in the Constitution, defense spending is considered
"discretionary" from a budget sense. Furthermore, investments in new
capabilities such as weapon systems are more discretionary than other
aspects of defense spending, such as personnel costs and the costs of
supporting and maintaining current force operations. As a result, it is
imperative that DOD's limited resources be allocated to the most
appropriate weapon system investments based on current and reasonably
expected threats and that the investments yield the results promised (such
as performance, cost, and timing) within the constraints imposed by those
resources.
We have assessed weapon acquisitions as a high-risk area since 1990.
Although U.S. weapons are the best in the world, the programs to acquire
them often take significantly longer and cost significantly more money
than promised and often deliver fewer quantities and other capabilities
than planned. It is not unusual for estimates of time and money to be off
by 20 to 50 percent. When costs and schedules increase, quantities are
cut, and the value for the warfighter-as well as the value of the
investment dollar-is reduced. In these times of asymmetric threats and
netcentricity, individual weapon system investments are getting larger and
more complex. Just 4 years ago, the top five weapon systems cost about
$281 billion; today, in the same base year dollars, the top five weapon
systems cost about $521 billion. If these megasystems are managed with
traditional margins of error, the financial consequences can be dire,
especially in light of a constrained discretionary budget.
Our work on the development of successful commercial and defense products
has shown that it is possible to get better outcomes from investments if
decisions are based on high levels of knowledge. Defense acquisition
policies support such an approach to managing weapon system programs.
However, actual practice is not yet consistently following written policy.
As this annual assessment of major weapon acquisitions shows, most
programs are proceeding with inadequate levels of knowledge, with
attendant increased risks for traditional rates of cost
growth, along with schedule delays and performance shortfalls. On the
other hand, this assessment also includes programs that are proceeding
with high levels of knowledge, showing that practice can follow policy.
This is our third annual assessment of weapon system programs. The
experiences catalogued in this report provide insights on how programs can
be better positioned to succeed. To the extent that programs are not so
positioned, the report can be used by decision makers to take actions to
reduce risks by building higher levels of knowledge.
David M. Walker Comptroller General of the United States
Page viii GAO-05-301 Assessments of Selected Major Weapon Programs
A
United States Government Accountability Office Washington, D.C. 20548
March 31, 2005
Congressional Committees
The Department of Defense (DOD) is embarking on a number of efforts to
enhance warfighting capabilities. Primary among these efforts are the
investments being made to develop improved weapon systems with
technological superiority and enhanced lethality to combat threats to U.S.
security. Investment in programs such as the Army's Future Combat Systems
and Warfighter Information Network-Tactical, the Missile Defense Agency's
suite of land, sea, air, and space systems, the Navy's advanced ships such
as the DD(X) Destroyer, and the Air Force's space systems such as the
Transformational Satellite Communications System are likely to dominate
the budget and doctrinal debate well into the next decade. Many of these
embody the dual challenge of employing complex technology with a rapid
pace of development. Fiscal realities, coupled with the larger scope of
key acquisitions, reduce the ability of budgets to accommodate typical
margins for error in terms of cost increases and schedule delays.
Identifying risks early and addressing them before they become problems
can lessen cost increases and schedule delays and thus enable budgets to
buy what was planned.
In this report, we assess 54 programs that represent an investment of
approximately $800 billion.1 Our objective is to provide decision makers
with independent, knowledge-based assessments of individual systems'
attained knowledge and potential risks.
A Challenging Time for DOD has entered a period of high investment. A
significant portion of this
investment is for the acquisition of weapon systems that offerWeapon
System technologically advanced capabilities. The investment in the
research, Investments development, and procurement of major weapon systems
is expected to
rise from $144 billion in fiscal year 2005 to $185 billion in fiscal year
2009.
Major Defense Acquisition Programs make up about 45 percent, or
1 This estimate includes total research, development, test, and evaluation
(RDT&E); procurement; military construction; and acquisition operation and
maintenance appropriations to develop the weapon systems.
$65 billion, as shown in figure 1, of the fiscal year 2005 investment
request.2 DOD's total planned investment in these programs is
approximately $1.3 trillion, with about $812 billion of that investment
yet to be made.
Figure 1: RDT&E and Procurement Funding-Major Defense Acquisition Programs
In millions of constant 2005 dollars
80,000
70,000
60,000
50,000
40,000
30,000
20,000
10,000
0 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
Fiscal year
Source: GAO analysis of DOD data.
There are several challenges to getting the most from that investment.
First, because DOD's investment in weapon systems represents one of the
largest discretionary items in the federal budget, DOD's budget faces
2 Major Defense Acquisition Programs are programs identified by DOD as
programs that require eventual RDT&E expenditures of more than $365
million or $2.19 billion in procurement in fiscal year 2000 constant
dollars.
Page 2 GAO-05-301 Assessments of Selected Major Weapon Programs
growing pressures from increases in mandatory federal spending.3 According
to the Congressional Budget Office, federal deficits are expected to
average $250 billion through fiscal year 2009 and new budgetary demands
stemming from demographic trends lie beyond that time frame. In calendar
year 2004, discretionary spending accounted for about 39 percent of the
federal budget, and current projections show that because of increases in
mandatory spending, discretionary spending is likely to decrease to 33
percent of the federal budget by fiscal year 2009.4 It will be difficult
for DOD to increase its budget share to cover cost increases in weapon
programs in that environment.
Second, DOD faces competing demands within its own budget, such as from
operations in Afghanistan and Iraq. Since September 2001, DOD has needed
$158 billion in supplemental appropriations to support the global war on
terrorism.5 The budget implications of these operations further increase
the demand made of the defense dollar and therefore the investment in new
weapon programs. For example, current military operations are causing
faster wear on existing weapons, which will need refurbishment or
replacement sooner than planned. These needs will compete with the
investment in new weapon programs.
Third, DOD programs typically take longer to develop and cost more to buy
than planned, placing additional demands on available funding. These
programs increasingly compete for resources and are sometimes forced to
make trade-offs in quantities, resulting in a reduction of buying power.
As a result, funds are not available for other competing needs and
programs yield fewer quantities for the same, if not higher, cost. Table 1
illustrates seven programs with the greatest reduction of buying power.
Some of these programs experienced higher costs for the same initial
quantity.
3 Mandatory spending is controlled by laws other than appropriation acts.
Discretionary spending is provided in appropriations acts.
4 Congressional Budget Office, The Budget and Economic Outlook: Fiscal
Years 2006 to 2015. (Washington, D.C.: January 2005.)
5 Estimate as of May 2004. Another supplemental was expected in January
2005 to cover costs of operations in Iraq and Afghanistan.
Table 1: Examples of Programs with Reduced Buying Power
Source: GAO analysis of DOD data. Images sourced in their respecitve
order: JSF Program Office; Program Manager, Unit of Action, U.S. Army;
F/A-22 System Program Office; (Left) (c) 2003 ILS/Lockheed Martin, (right)
(c) 2003 The Boeing Company; Lockheed Martin Space Systems Company;
General Dynamics Land Systems; Naval Gunnery Project Office.
If DOD cannot deliver its major new programs within estimated costs,
difficult choices have to be made regarding which investments to pursue
and which to discontinue.
Page 4 GAO-05-301 Assessments of Selected Major Weapon Programs
Current Programs Are The majority of programs in our assessment are
costing more and taking
longer to develop than estimated. As shown in table 2, total RDT&E
costsCosting More and for 26 common set6 weapon programs increased by
nearly $42.7 billion, or Taking Longer to 42 percent, over the original
business case (the first full estimate). The Develop same programs have
also experienced an increase in the time needed to
develop capabilities with a weighted-average schedule increase of nearly
20 percent.7
Table 2: Cost and Cycle Time Growth for 26 Weapon Systems
Billions of constant 2005 dollars
First full Latest Percent
estimate estimate change
Total cost $479.6 $548.9
RDT&E cost 102.0 144.7
Weighted-average acquisition cycle 146.6 months 175.3 months
timea
Source: GAO analysis of DOD data.
aThis is a weighted estimate of average acquisition cycle time for the 26
programs based on total program costs at the first full and latest
estimates. The simple average for these two estimates was 94.9 months for
the first full estimate and 114.7 months for the latest estimate,
resulting in a 20.8 percent change.
Quantities for 10 of the common set programs have been reduced since their
first estimate.8 In addition, the weighted-average program acquisition
unit cost of 25 of the 26 programs increased by roughly 50 percent.9
6 The common set refers to the 26 weapon system programs that we were able
to assess since development began and between annual assessment periods.
The 26 programs are AESA, AEHF, APKWS, C-5 AMP, C-5 RERP, CH-47F, CEC, E-2
AHE, EA-18G, Excalibur, EFV, ERGM, F/A-22, FCS, Global Hawk, JASSM, JSOW,
JSF, JTRS Cluster 1, Land Warrior, NPOESS, Tomahawk, SDB, V-22, WIN-T, and
WGS. We limited this analysis to these 26 programs because all data
including cost, schedule, cycle time, and quantities were available for
comparison between program estimates.
7 A weighted average gives more expensive programs a greater value.
8 The 10 programs are AEHF, C-5 AMP, C-5 RERP, Excalibur, ERGM, F/A-22,
Global Hawk, JSF, JSOW, and V-22.
9 This estimate is a weighted average based on total program cost and does
not include the Excalibur program because of its extreme unit cost growth.
The simple average program unit cost increase for the same 25 programs is
40 percent. The weighted average, including the Excalibur, is 52 percent.
During the last year, cost and schedule estimates for the same 26 programs
have increased noticeably since our last assessment, as shown in table 3.
Table 3: Cost and Cycle Time for the Same Programs: 2004 Assessment and 2005
Assessment
Billions of constant 2005 dollars
2004 assessment 2005 assessment Percent
change
Total cost $480.3 $548.9
RDT&E cost 127.3 144.7
Weighted-average acquisition cycle 166.1 months 175.3 months
timea
Source: GAO analysis of DOD data.
aThis is a weighted estimate of average acquisition cycle time for the 26
programs based on total program cost estimates for the 2004 assessment and
the 2005 assessment. The simple average for these two estimates was 110.7
months for the 2004 assessment and 114.7 months for the 2005 assessment,
resulting in a 3.6 percent change.
bThese estimates also include the Land Warrior program. Although this
program was not included in the 2004 assessment, the program is included
in the common set because data were available from the December 2002
Selected Acquisition Report for inclusion in this estimate.
Some of DOD's largest programs have driven these increases. For example,
research and development costs for the Army's Future Combat Systems, a
$108 billion investment, increased by approximately 51 percent over the
past year while in the midst of a major restructuring of the program.
Likewise, the Joint Strike Fighter, a $199 billion investment, has
reported a research and development cost increase of over 19 percent in
the past year.
A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes
Over the last several years we have undertaken a body of work that
examines weapon acquisition issues from a perspective that draws upon
lessons learned from best system development practices. We found that
successful programs take steps to gather knowledge that confirms that
their technologies are mature, their designs stable, and their production
processes are in control. Separating technology development from product
development is important to this effort. Successful programs make a
science and technology organization, rather than the program or product
development manager, responsible for maturing technologies. Such steps can
help to reduce costs and deliver a product on time and within budget.
DOD's current acquisition guidance embraces the use of evolutionary,
knowledge-based acquisition practices proven to be more effective and
efficient in developing new products. By fully implementing these
practices, DOD can better leverage its investments by shortening the time
it
Page 6 GAO-05-301 Assessments of Selected Major Weapon Programs
takes to develop capabilities with more predictable costs and schedules,
thereby maintaining its buying power.
Successful product developers ensure a high level of knowledge was
achieved at key junctures in development. We characterize these junctures
as knowledge points. These knowledge points and associated indicators are
defined as follows:
o Knowledge point 1: Resources and needs match. This level of knowledge
occurs when a sound business case is made for the product-that is, a match
is made between the customer's requirements and the product developer's
available resources in terms of knowledge, time, and money. Achieving a
high level of technology maturity at the start of system development is an
important indicator of whether this match has been made. This means that
the technologies needed to meet essential product requirements have been
demonstrated to work in their intended environment.
o Knowledge point 2: Product design is stable. This level of knowledge
occurs when a program determines that a product's design is stable- that
is, it will meet customer requirements and cost and schedule targets. A
best practice is to achieve design stability at the system-level critical
design review, usually held midway through development. Completion of at
least 90 percent of engineering drawings at the system design review
provides tangible evidence that the design is stable.
o Knowledge point 3: Production processes are mature. This level of
knowledge is achieved when it has been demonstrated that the product can
be manufactured within cost, schedule, and quality targets. A best
practice is to ensure that all key manufacturing processes are in
statistical control-that is, they are repeatable, sustainable, and capable
of consistently producing parts within the product's quality tolerances
and standards-at the start of production.
The attainment of each successive knowledge point builds on the preceding
one. While the knowledge itself builds continuously without clear lines of
demarcation, the attainment of knowledge points is sequential. In other
words, production maturity cannot be attained if the design is not stable,
and design stability cannot be attained if the critical technologies are
not mature.
Seeking to improve acquisition outcomes, DOD revised its acquisition
policy in May 2003 to incorporate a knowledge-based, evolutionary
framework. The policy adopts lessons learned from successful commercial
companies. For example, the policy attempts to separate technology
development from product development and requires the demonstration of
technologies to high readiness levels. The policy also allows managers to
develop a product in increments rather than trying to incorporate all of
the desired capabilities in the first version that comes off the
production line.
Most Programs Have Most of the programs we reviewed proceeded with lower
levels of
knowledge at critical junctures and attained key elements of
productProceeded with Lower knowledge later in development than specified
in DOD policy, which Levels of Knowledge at resulted in cost increases and
schedule delays.
Critical Junctures
Development Start Our work shows that the demonstration of technology
maturity by the start of system development is the key measure for
achievement of knowledge point 1. A program that proceeds into product
development without demonstrating mature technologies does so with
increased risk of cost growth and schedule delays throughout the life of
the program.
Only 15 percent of the programs we assessed began development having
demonstrated all of their technologies mature, as illustrated in figure 2.
Page 8 GAO-05-301 Assessments of Selected Major Weapon Programs
Figure 2: Percent of Programs That Achieved Technology Maturity at Key
Junctures
100
75
50
25
0
Development DOD Production start design decision review
Source: GAO analysis.
More often than not, programs sought to mature technologies well into
system development when they should have focused on maturing system design
and preparing for production. The programs that started development with
mature technologies experienced lower development and unit cost increases
than those programs that started development with immature technologies.
For example, RDT&E costs for the programs that started development with
mature technology increased by an average of 9 percent over the first full
estimate, whereas the development costs for the programs that started
development with immature technologies increased an average of 41 percent
over the first full estimate. Likewise, program acquisition unit costs for
the programs with mature technology increased by less than 1 percent,
whereas the programs that started development with immature technologies
experienced an average program acquisition
unit cost increase of nearly 21 percent over the first full estimate.10
Finally, the programs with mature technology experienced an average
schedule delay of 7 months-a 9 percent increase-whereas the schedule for
the programs that started development with immature technology increased
an average of 13 months-a 13 percent increase.
Design Review As illustrated in figure 3, 42 percent of the programs that
held a design review achieved design stability at that key juncture.
Figure 3: Percent of Programs Achieving Design Stability at Key Junctures
100
75
50
25
0
DOD Production design decision review
Source: GAO analysis.
10 These percentages are program cost weighted averages. The simple
average increase for program acquisition unit costs is 0.68 percent for
the programs that started development with mature technologies and 25
percent for the programs that started development with immature
technologies.
Page 10 GAO-05-301 Assessments of Selected Major Weapon Programs
With the exception of the Navy's V-22, which has experienced significant
design changes since development start in 1986, these programs have
experienced a 6 percent increase in development costs and an average
schedule increase of 11 months since the first full estimate.11 Those
programs that did not achieve design stability have experienced a
combined development cost increase of 46 percent and an average
schedule increase of 29 months since the first full estimate.12
Design stability cannot be attained if key technologies are not mature.
Ten programs held design review without demonstrating mature critical
technologies.13 Out of the 10 programs, 7 had experienced a cost increase,
schedule delay, or both.14 The unit cost of 5 of these programs increased
by
at least 10 percent.15 In contrast, 3 programs entered product development
with mature technologies. These three programs kept program unit cost
increases to a minimum, with costs either falling or increasing by single
digits.16
11 This estimate does not include cost and schedule data for three
programs: the V-22, Aegis BMD, and STSS. Aegis BMD and STSS were not
included in the cost and schedule estimates because they are missile
defense elements that do not provide baseline cost and schedule estimates
against which to measure progress.
12 The cost and schedule estimates do not include the THAAD system or the
Ground-Based Midcourse Defense system because they are missile defense
elements that do not provide baseline estimates against which to measure
progress. The schedule estimate does not include the ATIRCM/CMWS because a
key date is classified.
13 The 10 programs are AESA, Aegis BMD, APKWS, ATIRCM/CMWS, EFV, ERGM,
F/A-22, GMD, JTRS 1, and STSS. The F/A-22 held its design review in 1995
and while we did not formally assess the technology maturity at that
point, the F/A-22 technologies and design matured late in the program
(e.g. the F/A-22 program had released 21 percent of drawings at design
review).
14 This estimate does not include the missile defense elements (Aegis BMD,
GMD, and STSS) because they do not provide baseline estimates against
which to measure progress.
15 The five programs are AESA, ATIRCM/CMWS, EFV, ERGM, and F/A-22.
16 The three programs are the C-5 RERP, JASSM, and the Tactical Tomahawk.
C-5 RERP and JASSM were assessed to have design stability at design
review. C-5 RERP had a program unit cost increase of 8.2 percent; JASSM
had a program unit cost of increase of 7.1 percent; and Tactical Tomahawk
had a decrease of program unit cost of -13.5 percent.
Nine programs are scheduled to hold their system design review in the next
year.17 Only two of those programs, the B-2 Radar Modernization and the
Excalibur program, expect their technologies to be mature at the time of
their design reviews. The remaining seven programs project that their
technologies will not attain maturity until after their critical design
reviews.
Production Start To determine if a product's design is reliable and
producible, successful programs use statistical process control to bring
manufacturing processes under control so they are repeatable, sustainable,
and consistently producing parts within quality standards. The collection
of process control data prior to a production decision can enable a smooth
transition from product development to the production phase. Of the 19
programs in production or approaching a production decision in the next
year, only 2 collected or plan to collect statistical process control data
to measure the maturity of production processes.18 While the absence of
the data does not mean that production processes were immature, it does
prevent an assessment against an objective standard.
How to Read the Knowledge Graphic for Each Program Assessed
We assess each program in 2 pages and depict the extent of knowledge in a
stacked bar graph and provide a narrative summary at the bottom of the
first page. As illustrated in figure 4, the knowledge graph is based on
the three knowledge points and the key indicators for the attainment of
knowledge: technology maturity (depicted in orange), design stability
(depicted in green), and production maturity (depicted in blue). A "best
practice" line is drawn based on the ideal attainment of the three types
of knowledge at the three knowledge points. In some cases, we obtained
projections from the program office of future knowledge attainment. These
projections are depicted as dashed bars. The closer a program's attained
knowledge is to the best practice line, the more likely the weapon will be
delivered within estimated cost and schedule. A knowledge deficit at the
start of development-indicated by a gap between the technology knowledge
attained and the best practice line-means the program proceeded with
immature technologies and faces a greater likelihood of
17 The nine programs are AMNS, B-2 RMP, C-130 AMP, CVN-21, DD(X), E-2 AHE,
EA-18G, Excalibur, and WIN-T.
18 The two programs are APKWS and ASDS.
Page 12 GAO-05-301 Assessments of Selected Major Weapon Programs
cost and schedule increases as technology risks are discovered and
resolved.
Figure4: Depiction of a Notional Weapon System's Knowledge as Compared
with Best Practices
Source: GAO.
An interpretation of this notional example would be that the system
development began with key technologies immature, thereby missing
knowledge point 1. Knowledge point 2 was not attained at the design review
as some technologies were still not mature and only a small percentage of
engineering drawings had been released. Projections for the production
decision show that the program is expected to achieve greater levels of
maturity but will still fall short. It is likely that this program would
have had significant cost and schedule increases.
We conducted our review from July 2004 through March 2005 in accordance
with generally accepted government auditing standards. Appendix II
contains detailed information on our methodology.
Assessments of Our assessments of the 54 weapon systems follow.
Individual Programs
Page 14 GAO-05-301 Assessments of Selected Major Weapon Programs
Common Name: ABL
MDA's ABL element is being developed in incremental, capability-based
blocks to destroy enemy missiles during the boost phase of their flight.
Carried aboard a highly modified Boeing 747 aircraft, ABL employs a beam
control/fire control subsystem to focus the beam on a target, a
high-energy chemical laser to rupture the fuel tanks of enemy missiles,
and a battle management subsystem to plan and execute engagements. We
assessed the Block 2004 design that is under development and expected to
lead to an initial capability in a future block.
Source: Airborne Laser Program Office.
Program Transition to 6-module Initial beam/fire GAO Lethality Initial
start MDA laser test control flight test review demonstration capability
(11/96) (10/01) (11/04) (12/04) (1/05) (TBD) (TBD)
Although program officials expected ABL to provide an initial capability
during Block 2006, this event has been delayed and only one of its seven
critical technologies is fully mature. During Block 2004, the program
continues work on a prototype that is expected to provide the basic design
for a future operational capability. Program officials expect to
demonstrate the other six technologies during a prototype flight test that
will assess ABL's lethality. Difficulty in integrating prototype
components could delay this effort from 2005 to 2008. MDA has released
about 94 percent of the engineering drawings for the prototype's design,
which will be the basis for an initial operational capability during a
future block if the test is successful. However, additional drawings may
be needed if the design is enhanced or if problems encountered during
flight testing force design changes.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production review start design decision (1/05) (TBD)
review (TBD)
(TBD)
Common Name: ABL
ABL Program
Technology Maturity
Only one of ABL's seven critical technologies- managing the high-power
beam-is fully mature. The program office assessed three technologies- the
six-module laser, missile tracking, and atmospheric compensation-as nearly
mature. The remaining three technologies-transmissive optics, optical
coatings, and jitter control-are the least mature. According to program
officials, all of these technologies are needed to provide the system with
an initial operational capability.
While the program office has assessed the six-module laser as being close
to reaching full maturity, the power generated by grouping six laser
modules together must be demonstrated before full maturity can be
reasonably assessed. The recent demonstration of the simultaneous firing
of all six laser modules reduces risk in this area. Additional testing,
planned over the next 6 months, must still be completed to demonstrate the
full power and duration of the laser segment prior to installation on the
aircraft.
The transmissive optics, optical coatings, and jitter control are the
least mature critical technologies and consist of prototypes that have
only been tested in the laboratory or demonstrated through analysis and
simulation. The program plans to demonstrate all technologies in an
operational environment during a flight test of the system prototype,
referred to as lethal demonstration, in which ABL will attempt to shoot
down a short-range ballistic missile. Challenges with integrating the
laser and beam control/fire control subcomponents could delay this test
into 2008, but the final schedule is to be determined. Upon successful
completion of this test, MDA expects to develop a second aircraft that
will provide an initial operational capability.
Design Stability
We could not assess the design stability because ABL's initial capability
will not be fully developed until the second aircraft-what is expected to
provide an initial capability-is well underway. While the program has
released 10,280 of the 10,910 engineering drawings for the prototype, it
is unclear whether the design of the prototype aircraft can be relied upon
as a good indicator of design stability for the second aircraft. More
drawings may be needed if the design is enhanced or if problems
encountered during flight testing force design changes.
Production Maturity
We did not assess the production maturity of ABL because MDA has not made
a production decision. The program is producing a limited quantity of
hardware for the system's prototype. Program officials explained that they
continue to experience problems maintaining a stable manufacturing base
for prototype subcomponents.
Other Program Issues
Technological challenges caused the prime contract to approach its cost
ceiling during fiscal year 2004. In early April 2004, MDA directed the ABL
program to restructure the contract, increase its cost ceiling, and
refocus the contractor's efforts on making technical progress. As a
result, the cost ceiling was increased by $1.5 billion and the period of
performance was extended to 2008 from 2005. The contract is currently
valued at approximately $3.6 billion.
The focus of current work is on two near-term events. The first event was
the six-module laser test in a ground test facility that the program
completed in November 2004. The second event was the initial Beam
Control/Fire Control flight test, which occurred in December 2004.
Agency Comments
In commenting on a draft of this assessment, MDA maintained that the
current design is stable despite the assessed technology maturity.
Officials told us that because the ABL operational environment is
impractical to duplicate on the ground, the technology maturity assessment
will understate actual maturity until after 100 percent of the drawings
are released. While the officials expect changes to future blocks as part
of capability-based spiral acquisition, they believe the basic design will
directly migrate to subsequent blocks.
CommonName: AegisBMD
MDA's Aegis BMD element is a sea-based missile defense system being
developed in incremental, capability-based blocks to protect deployed U.S.
forces and critical assets from short-and medium-range ballistic missile
attacks. Key components include the shipboard SPY-1 radar, hit-to-kill
interceptors, and command and control systems. It will also be used as a
forward-deployed sensor for surveillance and tracking of intercontinental
ballistic missiles. We assessed only Block 2004 of the element's
interceptor-the Standard Missile 3 (SM-3).
Source: Aegis BMD Program Directorate.
Program start (10/95)
Transition to
MDA
(1/02)
Design review (5/03)
Surveillance/tracking
capability
(9/04)
GAO review (1/05) Block 2004 completion (12/05)
According to program officials, the first increment of SM-3 missiles being
fielded during 2004-2005 has mature technologies and a stable design.
However, the program has been struggling with the technology that
maneuvers the missile's kinetic warhead (kill vehicle) to its target.
Partial functionality of this "divert" technology was successful in 4
flight tests, but full functionality has only been demonstrated in ground
tests-it failed during a June 2003 flight test. Design modifications were
identified but will not be implemented in the first 8 missiles being
fielded. Program officials noted that even with a reduced capability,
these missiles provide a credible defense. All drawings for the first
increment of missiles have been released to manufacturing. The program is
not collecting statistical data on its production process but is using
other means to gauge production readiness.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (10/95) review
(1/05) (TBD) (5/03)
Common Name: Aegis BMD
Aegis BMD Program
Technology Maturity
Program officials estimate that all three technologies critical to the
SM-3 are mature. These technologies-the third stage rocket motor, the
infrared seeker of the kinetic warhead, and the Solid Divert and Attitude
Control System (SDACS) of the kinetic warhead-were all tested in flight.
While the first two technologies were fully demonstrated in flight tests,
the SDACS, which generates divert pulses to steer the kinetic warhead, was
only partially demonstrated. As noted previously, full "divert" technology
succeeded in ground testing but partially failed during a June 2003 flight
test. According to program officials, the test failure was likely caused
by a defective subcomponent within the SDACS, a problem that should be
corrected through specific design modifications. Program officials note
that only partial functionality of the SDACS is required for Block 2004,
which was successfully demonstrated in flight tests. Although the kinetic
warhead of these interceptors will have reduced divert capability, they
provide a credible defense against a large population of the threat and
can be retrofitted upon the completion of design updates and testing.
Design Stability
Program officials reported that the design for the first eight
interceptors being fielded during Block 2004 is stable with 100 percent of
its drawings released to manufacturing. The program plans to implement
design changes in subsequent configurations of the SM-3 (delivered during
2006-2007) to resolve the SDACS failure witnessed in the June 2003 flight
test.
Production Maturity
We did not assess the production maturity of the missiles being procured
for Block 2004. Program officials stated that given the low quantity of
missiles being produced, statistical process control data on the
production process would have no significance. The Aegis BMD program is
using other means to assess progress in production and manufacturing-such
as integrated product teams, risk reviews, and SM-3 metrics-as part of its
overall development of the SM-3.
Other Program Issues
The Aegis BMD element builds upon the existing capabilities of
Aegis-equipped Navy cruisers and destroyers. Planned hardware and software
upgrades to these ships will enable them to carry out the ballistic
missile defense mission. In particular, the program is working to upgrade
Aegis destroyers for surveillance and tracking of intercontinental
ballistic missiles. Because this function is new to the element-allowed
only after the U.S. withdrawal from the Anti-Ballistic Missile Treaty-the
program office faced a tight schedule to fully develop and test this added
functionality, which it completed in September 2004 with the deployment of
the first destroyer for this mission.
Agency Comments
In commenting on a draft of this assessment, the program office stated
that Aegis BMD progress remains on track. For example, the program
deployed the first operational destroyers (for the long-range surveillance
and tracking mission) to the Sea of Japan, delivered 5 missiles in
November, and successfully ground tested the redesigned SDACS. It noted,
however, that our review focused on the SM-3, a junior portion of the
overall cost and development of the Aegis BMD system.
In addition, the program office reiterated that SDACS technology was
successful in four of five Aegis BMD flight tests. The current SDACS
configuration is fully capable of defeating the Block 2004 threat set, and
a design update is in progress to complete the final increment of
capability. As an application of capabilities-based acquisition, the
warfighter is provided a significant capability years earlier (albeit
using partial SDACS functionality) instead of waiting for a perfect
design.
Common Name: AEHF
The Air Force's AEHF satellite system will replenish the existing Milstar
system with higher capacity, survivable, jam-resistant, worldwide, secure
communication capabilities for strategic and tactical warfighters. The
program includes satellites and a mission control segment. Terminals used
to transmit and receive communications are acquired separately by each
service. AEHF is an international partnership program that includes
Canada, United Kingdom, and the Netherlands. We assessed the satellite and
mission control segments.
Source: Advanced EHF Program Office.
Program start (4/99)
Design review (4/04)
Production decision (6/04) GAO review (1/05) First launch (4/08) Initial
capability (6/10)
According to the program office, the AEHF program's technologies are
mature and the design is stable. However, the high risk strategy of
concurrently developing two critical path items has led to further
schedule delays and cost increases. The program is relying on the
concurrent development of the AEHF Comsec/Transec System (ACTS) suite of
cryptological equipment, which limits access to authorized users, and
terminals used for satellite command and control. Both of these items are
being developed outside the program office. Delivery delays of the ACTS
and command and control terminals resulted in an additional 12-month
launch delay and an estimated 20 percent cost increase, incurring a
Nunn-McCurdy breach (10 U.S.C. 2433) at the 15 percent threshold.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (NA) review
(6/04) (1/05)
(4/04)
Common Name: AEHF
AEHF Program
Technology Maturity
All of the 14 critical technologies are mature, according to the program
office. In addition, all 19 of the application-specific integrated
circuits critical to functioning of the communications payload have been
flight qualified through demonstration and testing.
Design Stability
AEHF's design is now stable since more than 97 percent of the design
drawings have been released. While the program completed its system level
critical design review in April 2004 with only about two-thirds of the
drawings released, the AEHF contractor has since resolved all outstanding
issues from that review.
Production maturity could not be assessed as the program office does not
collect statistical process control data. In June 2004, the formal
decision was made to acquire the third and final satellite.
Other Program Issues
The concurrent development of two critical path items-the ACTS and the
command and control terminals-has led to further schedule delays and cost
growth. The ACTS is a suite of cryptological equipment installed in both
the satellite and the terminals to limit access to authorized users and is
being developed and produced by the National Security Agency. The ACTS has
already experienced significant cost growth and schedule delays due to
production problems and changing security requirements. In September 2003,
ACTS delivery delays and development problems led the program office to
delay the launch of the first two satellites by 4 months. The second
critical path item-the command post terminals-is developed and funded by
another Air Force program office. These terminals must be in place and
tested prior to the first launch or there will be a day-for-day slip in
the satellite launch schedule.
The concurrent development of the AEHF satellites, terminals, and the ACTS
has led to further delays and cost increases. Delayed delivery of the ACTS
had resulted in an additional 12-month delay. Launches for the three AEHF
satellites are now scheduled for April 2008, April 2009, and April 2010.
The launch delays along with added payload component testing and
replacement of critical electronic parts are expected to increase the
overall program cost by about 20 percent. In December 2004, the Air Force
notified Congress of a Nunn-McCurdy breach at the 15 percent threshold.
In December 2002, satellites four and five were deleted from the AEHF
program because the new Transformational Satellite Communications System
(TSAT), assessed elsewhere in this report, is to replace these satellites
if they are sufficiently developed. The Air Force scheduled an interim
review point in November 2004 to determine whether to buy additional AEHF
satellites or rely on TSAT. However, in light of the 12-month program
slip, the decision was delayed until November 2005.
Agency Comments
In commenting on a draft of this assessment, the Air Force provided
technical comments, which were incorporated where appropriate.
Common Name: AESA
The Navy's AESA radar is one of the top upgrades for the F/A-18E/F
aircraft. It is to be the aircraft's primary search/track and weapon
control radar and is designed to correct deficiencies in the current
radar. According to the Navy, the AESA radar is key to maintaining the
Navy's air-to-air fighting advantage and will improve the effectiveness of
the air-to-ground weapons. When completed, the radar will be inserted in
new production aircraft and retrofitted into lot 26 and above aircraft.
Source: U.S. Navy.
Program/ development start (2/01)
Design review (8/01)
Low-rate decision (6/03) GAO review (1/05) Initial capability (10/06) Full-rate
decision (1/07) Last procurement (2014)
The AESA radar's critical technologies were not mature at the start of
system development or at the design review, but they now appear to be
mature. The design also appears stable. However, radar development is
continuing during production. The program is tracking a number of risks
with the technical performance of the radar. If problems are discovered,
they could require design changes while the radar is in production. For
example, the software schedule leaves little room for error or rework, and
development of the radar simulation model puts training at risk. In
addition, there are some production risks that could affect the quality of
the initial radars and the aircraft delivery schedule. Antitamper
protection for the radar is currently in design. The AESA program also has
interdependencies with other programs that could make the radar vulnerable
to delays in their progress.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (2/01) review
(6/03) (1/05) (8/01)
Common Name: AESA
AESA Program
Technology Maturity
The latest technology readiness assessment for the radar determined that
the four critical technologies were mature. To further ensure technology
maturity, a mini-technology assessment is planned prior to the full-rate
production decision. By then, the technologies should have been
demonstrated in their final form and under expected conditions.
Design Stability
As of July 2004, all engineering drawings for the radar and its subsystems
had been released. At the design review in 2001, 59 percent had been
released. Development of the radar has continued during production. The
program office has identified some development risks that could result in
design changes. According to a program office risk assessment, the top
current challenge involves the software. The lack of timely software
delivery puts the program at significant risk, and could also require
radar hardware rework due to delays in the flight test program. Another
risk is that the radar simulation model integrated into the F/A-18
training simulator may not accurately represent the operation and
performance of the radar, which could result in some training that is
unrealistic. Further, the number of flight tests that can be conducted may
not be adequate to mature radar software. Other current risks include
whether the radar: will be able to track sufficient targets
simultaneously; radiation emissions will interfere with F/A-18E/F weapon
systems; and will have the capability to detect tail aspect targets at low
altitude. Mitigation plans are in place to address all design risks.
Production Maturity
During 4 low-rate production runs, 84 radars are planned-20 percent of the
415 radars to be procured. The program is currently in the second
production run. Most radars are planned to be installed in F/A-18E/Fs on
the aircraft production line. However, 135 radars will have to be
retrofitted into already produced F/A-18E/Fs-a more costly process
upfront, that, according to the Navy, is expected to save money on support
costs later. We could not assess production maturity because statistical
process control data are not being collected. Officials said they are
comfortable with manufacturing processes based on audits and inspections
conducted at some key manufacturers. Nonetheless, radar production
currently faces a number of risks. The radar contractor may have
difficulty transitioning from development to production due to production
risks, which could cause some late aircraft deliveries. Other risks
include reliability problems with one of the radar's critical technologies
may not allow initial radars to meet a specification and qualification
tests may not be complete in time, resulting in delivering radar hardware
that is not fully qualified. Moreover, full-rate production costs could
increase significantly if the projected payoff from cost reduction
initiatives is not fully realized. However, program officials expect
significant savings from the cost reduction initiatives.
Other Program Issues
The program office is closely tracking interdependencies that could place
the radar at risk. Successful development of other Navy programs is
required for the radar to meet key performance parameters. Also, the radar
program is being developed, in part, with funding from contractors.
Changes in the flow of this funding would affect the AESA program, but
program officials stated that almost all of the contractor funding has
been provided.
In 1999, DOD directed the services to implement antitamper protection to
guard against exploitation of critical U.S. technologies. This protection
was not one of the radar's original requirements. While officials said
there is a requirement for this protection to have no effect on radar
performance, operational tests of antitamper models are not planned until
after operational tests of radars without this protection, which may
identify problems that require design changes to the protection package.
The program's strategy for a depot has changed. Plans have been canceled
to stand up a Navy depot maintenance facility for the radar in 2010 at
North Island, California. Instead, Raytheon will conduct depot maintenance
at its facility in El Segundo, California, at substantial cost savings,
according to program officials.
Agency Comments
In commenting on a draft of this assessment, the Navy provided technical
comments, which were incorporated as appropriate.
Common Name: AN/ASQ-235
The Navy's AMNS is designed to relocate, identify, and neutralize bottom
or moored sea mines. AMNS consists of an operating console and a launch
and handling system containing up to four neutralizers. When deployed, the
MH-60S helicopter hovers near the target mine and lowers AMNS via a tow
cable into the water. A neutralizer, controlled through fiber-optic cable,
exits the launch and handling system and uses sonar to find the mine and
fires a lethal charge, destroying the mine and the neutralizer.
Source: U.S. Navy.
Development
start
(4/03)
GAO review (1/05)
Design review (3/05)
Low-rate decision (2/06)
Full-rate decision
and initial operational capability
(6/07)
Last procurement (2011)
The AMNS program began system development with none of its four critical
technologies mature. While progress has been made since then, program
officials do not expect to achieve technology maturity until developmental
tests are conducted in mid-2005. The AMNS program's design is stable, with
approximately 90 percent of the drawings complete. However, since the AMNS
technologies are not expected to demonstrate maturity until developmental
testing is conducted, the program runs the risk that problems identified
during that testing will require drawings to be modified. To maintain an
initial operational capability of June 2007, the program office requested
a $13 million increase in research and development funds in order to
support alternate testing on the MH-53E helicopter and to support delayed
testing on the MH-60S helicopter.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/03) (1/05)
review (2/06) (3/05)
Common Name: AN/ASQ-235
AMNS Program
Technology Maturity
The AMNS launch and handling system, the deployment subassembly, the
warhead assembly, and the neutralizer are not fully mature. The
neutralizer, which was demonstrated in a relevant environment, is
approaching full maturity. The program office describes the neutralizer as
a nondevelopmental item because it is already operational. However, it
needs to undergo safety and performance improvements before it will be
ready for AMNS. The other three technologies have not been integrated or
demonstrated outside of a laboratory environment, but program officials
have stated that no technology hurdles remain, merely engineering
challenges. Program officials expect all four technologies to demonstrate
maturity during developmental testing that is scheduled to take place
between May and October 2005.
Among risks identified by program officials are concerns that the
neutralizer will not launch properly in an environment of strong water
currents. The program office is attempting to mitigate this risk by
establishing plans and funding for testing the neutralizer in strong water
currents, including flume tank testing. Additionally, program officials
noted concerns about the survivability of the launch and handling system
in an underwater explosives environment. The program office plans for this
risk to be mitigated through an analysis of launch and handling system
internal parts and an analysis to prove that the launch and handling
system can tolerate environments of up to 50G levels.
Design Stability
Approximately 90 percent of the AMNS drawings are currently releasable.
Moreover, the program office projects all drawings to be releasable to
manufacturing at the completion of the design readiness review in March
2005. According to program officials, top level assembly drawings will be
considered at the design readiness review. Detailed designs of AMNS
components were validated through 17 interim design reviews held by the
program office.
Because the AMNS technologies are not expected to demonstrate maturity
until developmental testing is conducted in mid-2005, the program runs the
risk that any problems identified during testing would require drawings to
be modified.
Other Program Issues
The program office has requested an approximately $13 million increase in
research and development funds for the fiscal year 2006 budget. According
to program officials, this increase is required to support alternate
testing on the MH-53E helicopter and to support a 16-month delay in
completion of testing on the MH-60S helicopter. The MH-60S helicopter will
not be available to support the current AMNS development and test
schedule. Without alternate testing on the MH-53E helicopter, the program
will not be able to make a low-rate initial production decision in
February 2006 or, more importantly, maintain an initial operational
capability of June 2007. For the MH-60S helicopter, development testing is
not scheduled to start until 6 months after a low-rate initial production
decision has been made.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the
program quantity increased from 47 to 61 as a result of a change in Navy
strategy to deploy the system from Littoral Combat Ships rather than
aircraft carriers. Regarding technology maturity, it noted that currently
the program's critical technologies, for example the warhead assembly, are
slightly more mature than indicated in the assessment. In addition to
performing an analysis to prove that the launch and handling system can
tolerate high-pressure underwater environments, the Navy intends to
conduct Underwater Explosive Testing as further risk mitigation.
Regarding other program issues, the Navy stated that while alternate
platform testing on the MH-53E helicopter would enable the program to meet
its low-rate initial production decision and initial operational
capability targets, alternate platform testing is pending approval by the
Assistant Secretary of the Navy (Research, Development, and Acquisition).
It also indicated that constraints in the availability of MH-60S test
assets have the potential to delay the program's schedule and increase its
cost beyond the projections presented in the assessment.
Common Name: APKWS
The Army's APKWS is a precision-guided, air-to-surface missile designed to
engage soft and lightly armored targets. The system will add a new
laser-based seeker to the existing Hydra 70 Rocket System and is expected
to provide a lower cost, accurate alternative to the Hellfire missile.
Future block upgrades are planned to improve system effectiveness. We
assessed the laser guidance technology used in the new seeker.
Source: (c) 2003 General Dynamics Armament and Technical Products, All
Rights Reserved.
Program/development
start
(12/02)
Design review (3/04)
GAO review (1/05) Low-rate decision (9/05) Full-rate decision (12/07) Initial
capability (8/08) Last procurement (unknown)
The APKWS entered system development and held its design review before
demonstrating that its critical guidance technology was fully mature.
While the system's design was otherwise stable at the time of the March
2004 design review, initial system-level testing identified problems with
the design. Program plans call for a production decision in September 2005
and low-rate production contract award in December 2005. We were unable to
assess the program's production maturity because program officials do not
expect to begin collecting statistical data on their key manufacturing
processes until the start of production. Remaining efforts include
completing developmental and operational testing. If subsequent testing
identifies further problems with the design, additional costs of redesign
and modification of drawings late in development could be incurred.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (12/02) review
(1/05) (9/05) (3/04)
Common Name: APKWS
APKWS Program
Technology Maturity
The APKWS program has not demonstrated full maturity of its only critical
technology-laser guidance. Although a prototype guidance system was
successfully demonstrated under the Low Cost Precision Kill Advanced
Technology Demonstration, the current design for the guidance system
includes numerous hardware changes to improve system cost, performance,
and producibility. The new guidance system will not be fully integrated
and tested from an aircraft until winter 2005. Program officials noted
that although the prototype system design exists, reverting to it would
increase cost and degrade the system's performance and producibility.
Design Stability
Program officials released 100 percent of the drawings after a
system-level design review in March 2004. Recently completed testing,
however, uncovered the need for design changes. The APKWS, to date, has
completed two test flights. The first test flight went as planned. The
second flight test missile, however, experienced a mechanical failure of
the wing lock mechanism, causing the test missile to veer off target. The
program office identified a design solution, and flight testing resumed in
September 2004.
Production Maturity
Program officials expect that there will be nine key processes associated
with manufacturing the APKWS. The program plans to collect statistical
data on these processes when production begins in fiscal year 2006.
Other Program Issues
According to program officials, the Army cut APKWS research, development,
test, and evaluation (RDT&E) funding by 22.1 percent due to other funding
priorities. These officials noted that this reduction affects planned
improvements to the warhead, fuze, seeker, and propulsion subsystems.
Furthermore, the program has experienced a 15.3 percent growth in
acquisition cycle time as the result of slower initial production of the
system than originally planned.
Agency Comments
In commenting on a draft of this assessment, the Army concurred with this
assessment.
Common Name: ASDS
The Special Operations Forces' ASDS is a battery-powered, dry interior
minisubmarine developed for clandestine insertion and extraction of Navy
SEALs and their equipment. It is carried to its deployment area by a
specially configured SSN-688 class submarine. It is intended to provide
increased range, payload, on-station loiter time, and endurance over
current submersibles. The 65-foot long, 8-foot diameter ASDS is operated
by a two-person crew and equipped with a lock out/lock in chamber to allow
divers to exit and reenter the vehicle.
Source: U.S. Navy.
Program start (7/89)
Development
start
(9/94)
Initial operational test and evaluation (5/03)
Initial capability (11/03) GAO review (1/05) Production decision (12/05) Last
procurement (2011)
One of ASDS's three critical technologies-the lithium ion battery-has not
reached maturity, and the first boat has required some design changes. The
production decision has been delayed from June 2004 until December 2005 to
allow time to produce and test a new battery and develop and test other
vehicle design changes. The Navy selected a design for the lithium ion
battery and, in May 2004, it awarded a contract to develop a full shipset
unit for ASDS. Battery production will take about 1 year, and at-sea
demonstration is expected in fiscal year 2005. Concurrent with battery
replacement, other vehicle improvements are being developed and tested and
design problems are being addressed. Acoustic signature issues are being
addressed; however, this requirement does not have to be met until
delivery of the second ASDS boat.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (9/94) review
(1/05) (12/05) (6/96)
Common Name: ASDS
ASDS Program
Technology Maturity
Of the three critical technologies identified by the ASDS program office,
one-the lithium ion battery-has not reached maturity. However, it is
expected to be mature before the December 2005 production decision for
additional boats.
Acoustic, or noise level, problems are being addressed; however, the first
boat is not quiet enough to meet acoustic stealth requirements. In earlier
tests, the ASDS propeller was the source of the most significant noise,
and a new composite propeller was installed before operational test and
evaluation in 2003. Although program officials believe it meets
requirements, precise acoustic measurements have not been made and are not
scheduled to be done before the production decision. Other acoustic issues
will be addressed on a time-phased basis because the acoustic requirement
has been deferred until delivery of the second boat.
Design Stability
Although all engineering drawings for ASDS have been released to
manufacturing, ASDS design changes have been required based on additional
improvements, test results, and other issues since ASDS reached initial
operational capability in November 2003. An assessment of ASDS
survivability design features is also underway; however, the Vulnerability
Assessment Report will not be completed until April 2005.
An updated ASDS operational requirements document was approved in June
2004. The number of key performance parameters (those elements that are so
significant that a failure to meet them could call into question a
system's ability to perform missions) were reduced from 16 to 8, and they
include one new requirement (operational availability). Other requirements
are categorized as system critical requirements.
Until all requirements are addressed, technical problems are solved, and
testing is completed, we believe ASDS's final design will remain uncertain
and may have cost and schedule implications.
Other Program Issues
The Navy completed an independent cost estimate, including life-cycle
costs, in March 2004. However, data were not released, and the estimates
are now out-of-date because they do not reflect the impact of the 2-year
delay in production of the second boat. According to the June 2004
Selected Acquisition Report, the U.S. Special Operations Command was
preparing a new proposed program plan to account for the delay in the
production decision and updated cost information was expected to be
reported in the December 2004 report. However, according to the Navy's
January 2005 update, the revised program plan and updated cost estimate
will be developed, reviewed, and approved as part of the production
decision, which has been delayed until December 2005. Since the program's
first cost estimate was originally approved in 1994, research and
development costs have more than tripled.
The Navy plans to conduct follow-on testing to verify that deficiencies
and vulnerabilities identified during the May 2003 operational evaluation
are corrected. However, not all results will be known before the scheduled
production decision.
Agency Comments
The Navy provided technical comments, which were incorporated as
appropriate.
Common Name: ATIRCM/CMWS
The Army's and the Special Operations Command's ATIRCM/CMWS is a component
of the integrated infrared countermeasures suite planned to defend U.S.
aircraft from advanced infrared guided missiles. The system will be
employed on Army and Special Operations aircraft. The system includes an
active infrared jammer, a missile warning system, and a countermeasure
dispenser capable of loading and employing expendables, such as flares,
chaff, and smoke.
Source: BAE Systems.
Program/development
start
(6/95)
Low-rate decision (11/03) GAO review (1/05) Full-rate decision (8/05) Last
procurement (2023)
The ATIRCM/CMWS program entered production in November 2003 with
technologies mature and designs stable. Currently, the program's
production processes are at various levels of control. The CMWS portion of
the program entered limited production in February 2002 to meet urgent
deployment requirements. However, full-rate production for both components
was delayed because of reliability problems. Over the past several years,
the program has had to overcome cost and schedule problems brought on by
shortfalls in knowledge: key technologies were demonstrated late in
development and only a small number of design drawings were completed by
design review. At the low-rate production decision point, the Army
developed a new cost estimate reducing program procurement cost
substantially.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (6/95) review
(11/03) (1/05) (2/97)
Common Name: ATIRCM/CMWS
ATIRCM/CMWS Program
Technology Maturity
The ATIRCM/CMWS's five critical technologies are mature. However, they did
not mature until after the design review in February 1997. Most of the
early technology development effort was focused on the application to
rotary wing aircraft. When system development began in 1995, the
requirements were expanded to include Navy and Air Force fixed wing
aircraft. This change caused problems that largely contributed to cost
increases of more than 150 percent to the development contract. The Navy
and the Air Force subsequently dropped out of the program, rendering the
extra effort needless, but the Navy and the Army are currently pursuing
future joint production planning.
Design Stability
The basic design of the system is complete with 100 percent of the
drawings released to manufacturing. The design was not stable at the time
of the design review, with only 22 percent of the drawings complete. This
was primarily due to the expanded requirements. It was not until 2 years
after the design review that 90 percent of the drawings were released and
the design was considered stable. This resulted in inefficient
manufacturing, rework, additional testing, and a 3-year schedule delay.
The system design was successfully demonstrated through engineering and
manufacturing development and transitioned to production.
Production Maturity
The production maturity could not be assessed based on the information
provided by the program office. According to program officials, the
ATIRCM/CMWS program has 16 key manufacturing processes in various phases
of control. They stated that ATIRCM statistical process controls are in
development, control plans are being enhanced and as the program continues
in production and data are gathered, lessons learned will be included in
the processes. The Army entered limited CMWS production in February 2002
to meet an urgent need of the U.S. Special Operations Command.
Subsequently, full-rate production was delayed for both components due to
reliability testing failures. The program implemented reliability fixes to
six production representative subsystems that will be used for initial
operational test and evaluation.
These systems were delivered in March 2004. The full-rate production
decision for the complete system is now scheduled for 2006.
Other Program Issues
The Army procured an initial 32 systems in fiscal year 2002 for use on the
U.S. Special Operations Command's CH-47 helicopters. The Army plans to
procure a total of 99 systems to outfit special operations aircraft
between fiscal year 2003 and 2009. Currently, program officials are
working to integrate CMWS on 16 additional platform types and models,
which will result in an increase in quantity and funding. The CMWS
low-rate initial production quantity increased by 141 systems to a total
of 200. The Army procured all 200 of these systems, and deliveries are on
schedule.
At the low-rate production decision point, the Army developed a new cost
estimate for the program that featured a variety of different program
assumptions. For example, program officials deleted 17 years of Contractor
Logistics Support, reducing potential duplication, and deleted 29 training
systems. As a result, program officials report that procurement cost was
reduced by 17 percent.
Agency Comments
The Army concurred with this assessment and provided technical comments,
which were incorporated where appropriate. Additionally, the Army
commented that in January 2004, it directed the acceleration of CMWS for
deployment on Operation Iraqi Freedom aircraft. Initial operational tests
and evaluation will be completed during fiscal year 2005 for CMWS and in
fiscal year 2006 for ATIRCM.
Common Name: B-2 RMP
The Air Force's B-2 RMP is designed to modify the current radar system to
resolve potential conflicts in frequency band usage. To comply with
federal requirements, the frequency must be changed to a band where the
B-2 will be designated as a primary user. The modified radar system is
being designed to support the B-2 stealth bomber and its combination of
stealth, range, payload, and near precision weapons delivery capabilities.
Source: U.S. Air Force, U.S. Edwards Air Force Base, California.
Program start (10/02)
Development
start
(8/04)
GAO review (1/05)
Design review (6/05)
Low-rate decision (2/07) Initial capability (10/07) Full-rate decision (2/08)
Last procurement (2009)
The B-2 RMP entered system development in August 2004 with two critical
technologies mature and two approaching maturity. All critical
technologies are planned to be mature by the June 2005 design review. The
program has released 71 percent of its design drawings and plans to have
85 to 95 percent released by the June 2005 design review. Program
officials indicated that production maturity metrics will be formulated
during development and that these metrics may or may not include
manufacturing process control data. The program plans to build six radar
units during development for pilot training with the B-2 operational wing
prior to the planned completion of flight testing. Even though these units
are necessary, building them early in development adds to the risk of
later design changes because most of the radar flight-test activity will
not occur until after these units are built.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (8/04) (1/05)
review (2/07) (6/05)
Common Name: B-2 RMP
B-2 RMP Program
Technology Maturity
The B-2 RMP entered development in August 2004 with two of four critical
technologies mature and two others approaching maturity. Last year the
program reported having two critical technologies, but a formal technology
readiness assessment conducted in February 2004 concluded that two
additional technologies should be considered critical. The additional two
technologies, the receiver/exciter for the electronic driver cards and
aspects of the antenna designed to help keep the B-2's radar signature
low, are not considered fully mature but are approaching maturity. There
are no backup technologies for two technologies approaching maturity, but
both completed their design phases in April 2004 and the program office
estimates that both will be fully mature by the final design review in
June 2005.
Design Stability
The program has completed and released 71 percent of its engineering
drawings to manufacturing. The program office has scheduled the design
readiness review for June 2005 and plans to have 85 to 95 percent of its
drawings released by that time. The program, however, does not use the
release of design drawings as a measure of design maturity but instead
uses the successful completion of design events, such as subsystem design
reviews, as its primary measure of design maturity.
Production Maturity
Production maturity metrics are planned to be formulated during
development. These metrics, which may or may not include manufacturing
process control data, are planned to be used as measures of progress
toward production maturity during a production readiness review prior to
the start of production in February 2007. The program is also involved in
a proof-of-manufacturing effort to demonstrate that the transmit/receive
modules can be built to specifications.
Other Program Issues
The program plans to build six radar units during development and later
modify these units for placement on operational B-2 aircraft. The Air
Force needs these radar units available when the current B-2 radar
frequency becomes unavailable, in order to continue air crew training and
proficiency operations. Even though these units are necessary, building
them early in development adds risk because most of the radar flight-test
activity will not occur until after these units are built.
Agency Comments
The Air Force concurred with this assessment. It commented that the
program recognizes a level of risk associated with building the six
development units prior to formal testing in order to satisfy a critical
schedule constraint. It stated that, as a result, the program office has
placed a heavy emphasis on risk reduction and that the program is
progressing well thus far in system development. It further commented that
it is important to note that these six development units are also planned
to be used for collection of field level reliability and maintainability
data. It also noted that the program has successfully completed its
proofof-manufacturing effort for the transmit/receive modules, has now
delivered over 600 modules, and has completed and released approximately
70 percent of its engineering drawings.
Common Name: C-130 AMP
The Air Force's C-130 AMP standardizes the cockpit configurations and
avionics for 14 different mission designs of the C-130 fleet. It
consolidates and installs the mandated DOD Navigation/Safety
modifications, the Global Air Traffic Management systems, and the C-130
broad area review requirements. It also incorporates other reliability,
maintainability, and sustainability upgrades and provides increased
situational awareness capabilities and reduces susceptibility of Special
Operations aircraft to detection/interception.
Source: C-130 Avionics Modernization Program, System Program Office.
Development
start
(7/01)
GAO review (1/05)
Design review (8/05)
Low-rate decision (2/06) Full-rate decision (5/08) Last procurement (unknown)
The C-130 AMP is using primarily commercial and modified off-the-shelf
technologies, and it entered system development with all but one of its
six critical technologies mature. The remaining technology is nearing full
maturity; however, there is concern that it may not meet current
performance requirements. Program officials reached agreement with the
user to field a lesser set of requirements equivalent to the current
capability in fiscal year 2008. Program officials plan to release 90
percent of engineering drawings by the design review and have made
progress toward that goal. Currently, 48 percent of the engineering
drawings are releasable compared to 14 percent a year ago. Additionally,
the program office recently modified the contract to accelerate the
installation on Special Operations aircraft by 1 year, placing additional
pressure on the already compressed schedule.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (7/01) (1/05)
review (2/06) (8/05)
Common Name: C-130 AMP
C-130 AMP Program
Technology Maturity
Five of the C-130 AMP's six critical technologies are fully mature, as the
program is primarily utilizing proven commercial and modified
off-the-shelf technology for all AMP capabilities. The remaining critical
technology, the Terrain Following and Terrain Avoidance (TF/TA)
capability, was demonstrated through the Air Force Research Lab's Quiet
Knight advanced technology demonstration program and is nearing full
maturity. There is a risk, however, that the TF/TA technology may not meet
a key requirement to operate at 250 feet. Program officials worked with
the user to agree on initially fielding TF/TA capability between 250 and
1,000 feet, which is the current capability of the technology. Program
officials plan to determine through analysis the residual capability of
the TF/TA technology to fly lower. However, if such capability cannot be
achieved, redesign may be necessary or the user will have to accept
current capability.
Design Stability
The program office has made progress toward meeting its goal of releasing
90 percent of the design drawings by design readiness review, scheduled
for August 2005. This will be 9 months sooner than anticipated last year,
due to the acceleration of key program dates to meet Special Operations
Command requirements. Currently, 48 percent of the design drawings are
complete and could be released to manufacturing. Program officials stated
they are committed to meeting the required 90 percent drawing release by
design review.
The modernization effort is divided into a number of capability spirals
due to the various aircraft designs. The first spiral will outfit C-130
aircraft with core capabilities and an integrated defensive system.
Special Operations C-130 aircraft will be outfitted first, and future
spirals are planned for these aircraft because they require additional,
and unique, defensive systems integration and enhanced situational
awareness.
Other Program Issues
Funding reductions in fiscal years 2003 and 2004 delayed the C-130 AMP's
development program, which resulted in a rescheduling of program
milestones and rebaselining of the program. The design review, low-rate
initial production, and production readiness decisions were all delayed.
While program officials stated that the delay in schedule would provide
more time to resolve issues with the TF/TA technology and software, the
delay in fielding was not acceptable to the Special Operations Command.
They added funding to mature the TF/TA technology through a series of
flight demonstrations prior to the formal developmental test and
evaluation period. The system integration schedule was compressed by 9
months by accelerating installation of core and mission-unique
capabilities on Special Operations aircraft; however, this allows less
time to reduce manufacturing risks and further compresses an already
optimistic time line.
The program is also at risk if less software is reused than originally
estimated, which may cause an increase in development costs and delay the
program's schedule. Software integration remains a risk due to its
complexity, number of suppliers, potential for developmental growth,
certification of a secure operating system, and software safety standards.
The program office is working to mitigate these risks through modeling and
simulation, utilizing the systems integration laboratory built by the
contractor, and through flight demonstrations.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that
program officials worked with the user to agree on initially fielding
TF/TA capability between 250 and 1,000 feet and that an analysis will be
accomplished to determine residual TF/TA technology capability to fly
lower. The Air Force also commented that funding reductions in fiscal
years 2003 and 2004 delayed the C-130 AMP development program. It further
stated that a delay in fielding MC-130 Combat Talon aircraft until fiscal
year 2010 was unacceptable to the Special Operations Command, which added
funding to mature TF/TA technology through flight demonstrations prior to
a formal developmental test and evaluation period. The Air Force also
commented that the special operations warfighter needs are driving an
aggressive schedule.
Common Name: C-5 AMP
The Air Force's C-5 AMP is the first of two major upgrades for the C-5 to
improve the mission capability rate, transport capabilities and reduce
ownership costs. The AMP implements Global Air Traffic Management,
navigation and safety equipment, modern digital equipment, and an
all-weather flight control system. The second major upgrade, the C-5
Reliability Enhancement and Reengining Program (RERP), replaces the
engines and modifies the electrical, fuel, and hydraulic systems. We
assessed the C-5 AMP.
Source: Lockheed-Mar tin Aeronautics Company.
Development
start
(11/98)
Design review (5/01)
Production decision (2/03) GAO review (1/05) Initial capability (10/05) Last
procurement (unknown)
The program office considers the C-5 AMP's critical technologies and
design to be mature as they are relying on commercial-off-the-shelf
technologies that are installed in other commercial and military aircraft.
The C-5 AMP plans to complete developmental test and evaluation in
December 2004, a 2 month slip from last year. The main challenge to the
program is the development and integration of software-to which this
schedule delay has been attributed. The Air Force plans to modify 55 of
the 112 C-5 aircraft. The Air Force is also seeking funding to modify the
remaining 57 C-5s, however, that decision will not be made until the Air
Force determines the correct mix of C-5 and C-17 aircraft needed to meet
DOD's airlift needs. If the Air Force decides to use the C-17s, it may not
upgrade some, or all, of the remaining 57 C-5s.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (11/98) review
(2/03) (1/05) (5/01)
Desired level of knowledge Not applicable
Data not available
Data not available
Common Name: C-5 AMP
C-5 AMP Program
Technology Maturity
We did not assess the C-5 AMP's critical technologies because the program
used commercial technologies that are considered mature. Program officials
stated that those technologies are in use on other aircraft and that they
have not significantly changed in form, fit, or function. For example, the
new computer processors are being used in the Boeing 777, 717, other
commercial aircraft, the KC-10, and a Navy reconnaissance aircraft.
Design Stability
The design appears stable as the contractor has released 100 percent of
the drawings for the AMP. In addition, seven major subsystem-level design
reviews were completed, and integration activities are currently ongoing.
Demonstration of these integration activities is scheduled during
development test and evaluation, which started in December 2002 and should
be completed in December 2004.
Production Maturity
We could not assess the production maturity because most components are
readily available as commercial-off-the-shelf items. This equipment is
being used on other military and commercial aircraft. In addition, the C-5
AMP is incorporating many other off-the-shelf systems and equipment, such
as the embedded global positioning system, the inertial navigation system,
and the multifunction control and display units. To ensure production
maturity, the program office is collecting data regarding modification kit
availability and the installation schedules.
Other Program Issues
Program officials indicated the greatest risk to the AMP is software
development and integration. Several new software programs must be
developed and integrated with several other commercial off-the-shelf
software packages. According to officials, the 2 month slip in development
test and evaluation can be attributed to software development delays as
well as overall systems integration (hardware and software) delays. More
specifically, program officials stated that the two primary causes for
delays were (1) the unavailability of systems integration facilities,
including equipment, simulation software, and engineering simulator, and
(2) less robust than expected integration test scripts and computer
software configuration item designs. Program officials stated that they
have applied lessons learned from the AMP experience to the RERP program.
The C-5 RERP is assessed elsewhere in this report.
The overall quantity of the C-5 fleet has been reduced from 126 to 112 due
to the retirement of 14 aircraft. The C-5 aircraft must undergo the AMP
modifications prior to the RERP modifications. However, only 55 aircraft
have been approved for the AMP upgrades, while 112 are awaiting the RERP
upgrades. The Air Force needs to determine how many of the remaining 57
C-5s will receive the AMP upgrades. That decision will not be made until
it determines the correct mix of C-5 and C-17 aircraft needed to meet
DOD's airlift needs. According to program officials, the Air Force is
currently performing mobility studies that will be used to make a mobility
mix decision. Until it is decided whether to use C-17s to replace some, or
all, of the earlier 57 C-5s, the number of aircraft to undergo the AMP and
RERP modernization will remain uncertain.
Agency Comments
In commenting on the draft of this assessment, the Air Force stated that
the unit cost comparison between the November 1998 and the latest AMP
position does not accurately portray the program's cost growth. The
November 1998 position represents the original 126-aircraft program. The
program has since been restructured to a 55-aircraft program. According to
the Air Force, such a change would increase unit costs by a large amount
because it would be less expensive, on a unit cost basis, to procure for a
greater number of aircraft than it would be to procure for fewer aircraft.
GAO Comments
While the program has established a new cost and performance baseline
since the November 1998 decision to begin development, the comparison
presented provides an accurate picture of change since that major
decision. Although DOD may update its baseline for management purposes,
our goal is to provide an aggregate or overall picture of the program's
history.
Common Name: C-5 RERP
The Air Force's C-5 RERP is one of two major upgrades for the C-5. The
RERP is designed to enhance the reliability of the aircraft through the
replacement of engines and modifications to subsystems such as the
electrical, fuel, hydraulic and flight controls systems, while the C-5
Avionics Modernization Program (AMP) is designed to enhance the avionics.
These upgrades are part of a two-phased modernization effort to improve
the mission capability rate, transport capabilities and reduce ownership
costs. We assessed the C-5 RERP.
Source: Lockheed-Mar tin Aeronautics Company.
Program start (2/00)
Development
start
(11/01)
Design review (12/03)
GAO review (1/05) Low-rate decision (3/07)
Full-rate decision
B-model
(1/09)
Full-rate decision
A-model
(5/11)
Initial capability (7/11)
The RERP is utilizing demonstrated commercial off-the-shelf components
that require little or no modification.The program ensured that the
technology was mature and that the design was stable at critical points in
development, closely tracking best practice standards. The program is
currently in system development and plans to enter low-rate production in
March 2007. The major challenge to the program is software development and
integration. Also, the program is dependent on the number of aircraft
approved to undergo the C-5 AMP modernization program. Until additional
aircraft are approved for the AMP, it is uncertain how many aircraft will
undergo the RERP.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (11/01) review
(1/05) (3/07) (12/03)
Desired level of knowledge Not applicable
Common Name: C-5 RERP
C-5 RERP Program
Technology Maturity
The C-5 RERP's technologies are mature based on an independent technology
readiness assessment conducted in October 2001. New engines account for 64
percent of the expected improvement in mission capability rate for the
aircraft. The new engines are commercial jet engines currently being used
on numerous aircraft. According to the Air Force technology assessment,
these engines have over 70 million flying hours of use.
Design Stability
The C-5 RERP's design is stable. As of November 2003, 98 percent of the
design drawings were complete. In addition, the seven major
subsystem-level design reviews were completed before the December 2003
system-level design review.
According to program officials, the greatest risk to the RERP is software
development and integration activities. Several new software programs must
be developed, and these programs as well as other commercial off-the-shelf
software packages must be integrated. The program has experienced software
problems in the past and has taken actions to improve software activities.
The program is taking advantage of AMP-developed products and lessons
learned in the RERP to reduce the risk of schedule slips associated with
software development and integration. For example, according to program
officials, the baseline software and systems integration facilities that
were developed for the AMP will not have to be completely redeveloped for
RERP activities.
Production Maturity
We did not assess the C-5 RERP's production maturity because the Air Force
is buying commercially available items. However, we expect that production
maturity would be at a high level because the engines have been
commercially available for many years.
Other Program Issues
The C-5 RERP is dependent on the C-5 AMP (assessed elsewhere in this
report), as the aircraft has to undergo avionics modernization prior to
other enhancements. Over the past year, software development resources
that were planned for the RERP were shifted to the AMP to ensure
completion of its software activities. According to program officials,
while shifting of resources currently has not caused a significant
schedule slip to the RERP, they do acknowledge that it will have a greater
impact on the RERP if the AMP continues to slip and resources originally
planned for use on the RERP are retained to complete the AMP work.
Due to the retirement of 14 aircraft, the quantity of C-5 RERP aircraft
was reduced from 126 to 112. Although the RERP program has been authorized
for 112 aircraft, the avionics modernization has only been authorized for
55 aircraft. Therefore, until the Air Force decides on how many C-5
aircraft will undergo avionics modernization, it is uncertain how many
aircraft will undergo the RERP. That decision is contingent upon the
results of ongoing mobility studies that are examining the appropriate mix
of C-5 and C-17 aircraft for DOD's overall airlift needs.
Agency Comments
In commenting on the draft of this assessment, the Air Force stated that
the unit cost comparison between the November 2001 and the latest RERP
position does not accurately portray the program's cost growth. The
November 2001 position represents the original 126-aircraft program. The
program has since been restructured to a 112-aircraft program. It further
stated that such a change would increase unit costs by a large amount
because it would be less expensive, on a unit cost basis, to procure for a
greater number of aircraft than it would be to procure for fewer aircraft.
GAO Comments
While the program has established a new cost and performance baseline
since the November 2001 decision to begin development, the comparison
presented provides an accurate picture of change since that major
decision. Although DOD may update its baseline for management purposes,
our goal is to provide an aggregate or overall picture of the program's
history.
Common Name: CEC
The Navy's CEC is designed to connect radar systems to enhance detection
and engagement of air targets. Ships and planes equipped with their
version of CEC hardware and software will share real-time data to create
composite radar tracks-allowing the battle group to see the same radar
picture. A CEC-equipped ship can then detect and engage targets its radar
cannot see. We assessed the current shipboard and airborne versions of the
CEC.
Source: CEC Program Office.
Program/development
start
(5/95)
Design review (12/96)
Low-rate decision-ship (2/98) Low-rate decision-air (5/99) Full-rate
decision-ship (4/02) GAO review (1/05) Last procurement (2019)
The CEC's production maturity could not be assessed because the government
does not collect the necessary data on the commercially available portions
of the ship-based and airborne versions of the CEC. However, program and
contractor officials consider the production processes capable of
producing a quality product on time and within cost. The technologies and
design of both the ship-based and airborne versions of the CEC are fully
mature. In April 2002, the shipboard version was approved for full-rate
production. The airborne version remains in low-rate production and may
proceed to full-rate production pending a full-rate production decision
anticipated in September 2005.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (5/95) review
(2/98) (1/05) (12/96)
Desired level of knowledge Not applicable
Data not available
Data not available
Common Name: CEC
CEC Program
Technology Maturity
All six of the CEC's critical technologies are mature. While the shipboard
and airborne versions have different hardware, they share the same
critical technologies.
Design Stability
The CEC's basic design appears stable, as all of the drawings needed to
build the shipboard and airborne versions have been released to
manufacturing. Additional drawings for each version continue to be
released to incorporate advances in commercially available technologies,
which comprise approximately 60 percent of CEC hardware.
Production Maturity
We could not assess production maturity as data were not available.
According to program officials, CEC production is mature and noncommercial
portions do not involve critical manufacturing processes. Officials
indicated that they do not have insight into whether the manufacturing
processes for the commercial portions are critical and are under
statistical control. However, program officials are confident that a
quality product can be delivered on time and within cost given contractor
past performance.
The program office plans to seek full-rate production approval for the
airborne version in September 2005. During operational testing, the
airborne version was determined to be operationally effective but not
operationally suitable. According to the program office, it is
implementing corrections that will be verified in time to support the
full-rate production decision.
Other Program Issues
In November 2003, the Navy announced plans to improve CEC interoperability
by pursuing open architecture and functionality changes with the Joint
Single Integrated Air Picture Systems Engineering Organization (JSSEO).
The CEC Program Office discontinued planning for a Block 2 development
effort and began working with JSSEO to jointly engineer sensor measurement
and radar tracking management solutions that will be available to all
services to ensure optimum interoperability across the battlespace. The
joint track management software being developed is intended to interface
with CEC software to improve data sharing throughout different computing
environments and to facilitate component upgrades without redesigning the
entire system.
CEC officials consider the joint track management software a technical
risk since JSSEO is using a relatively new approach for combat system
software development. The officials also consider it a schedule risk that
could impact timely delivery of Navy platforms, including the DD(X) and
the Littoral Combat Ship, which are to be equipped with CEC. To mitigate
risks, the CEC program manager is closely monitoring joint track manager
progress to determine whether the software can be incorporated into the
CEC on schedule. If JSSEO does not deliver an acceptable product by
September 2005, the Navy plans to continue using current CEC software and
explore alternatives.
With discontinuation of a Block 2 effort, the program also initiated a
preplanned product improvement effort for CEC hardware. This effort takes
advantage of advances in technology to reduce size, weight, and cost
without adding new critical technologies. Improved hardware will operate
with current CEC software and joint track manager software, once ready.
The program began testing of the improved hardware in August 2004 and
plans to obtain Office of the Secretary of Defense approval for
incorporating improvements by October 2005. The program is also developing
a miniterminal land version for the Marine Corps.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that it
generally concurred with our assessment but provided clarifying comments.
Regarding the schedule risk associated with joint track management, the
Navy stated that it, along with the other services, is working with JSSEO
to reach agreement on a joint architecture for track management, combat
identification, and tactical data link integration. It explained that the
joint architectural agreement will allow appropriate existing solutions to
be integrated into the joint track manager and will be extensible to
multiple networks and different communication devices. The Navy stated
that this will reduce the risk of providing joint track management
capability in fiscal year 2008.
Common Name: CH-47F
The Army's CH-47F heavy lift helicopter is intended to provide
transportation for tactical vehicles, artillery, engineer equipment,
personnel, and logistical support equipment. It is also expected to
operate in both day and night. The program is to enhance performance and
extend the useful life of the CH-47. This effort includes installing a
digitized cockpit, rebuilding the airframe, and reducing aircraft
vibration.
Source: Boeing Helicopters.
Program/development
start
(12/97)
Design review (9/99)
Low-rate decision (12/02) Full-rate decision (11/04) GAO review (1/05) Initial
capability (5/07) Last procurement (2017)
CH-47F production maturity could not be assessed as the program is not
collecting statistical process control data on key manufacturing
processes. Program officials believe that CH-47F production is low risk
because no new technology is being inserted into the aircraft, two
prototypes have been produced, and the production process was demonstrated
during the delivery of one low-rate initial production aircraft. The
CH-47F technologies appear mature and the design stable, with 100 percent
of the engineering drawings released for manufacturing. The Army has
regained 6 months of a schedule delay anticipated when it was directed to
produce additional MH-47s for special operations.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (12/97) review
(12/02) (1/05) (9/99)
Common Name: CH-47F
CH-47F Program
Technology Maturity
We did not assess technology maturity or determine
the number of critical technologies in detail. The
CH-47F is a modification of the existing CH-47D
helicopter. Program officials believe that all
critical technologies are mature and have been
demonstrated prior to integration into the CH-47F
development program.
Design Stability
The Army completed the CH-47F engineering development and manufacturing
phase, with 100 percent of the drawings released to manufacturing.
However, at the design review, only 37 percent of the system's engineering
drawings were complete. Since that time, the number of drawings completed
increased substantially. The majority of the new drawings were instituted
to correct wire routing and installation on the aircraft; changes the
program office believed could not be determined until after the first
prototype was delivered.
Production Maturity
We did not assess production maturity because the CH-47F program does not
collect statistical process control data on its production of helicopters.
The program office relies on inspections as its means to ensure acceptable
production results.
According to the program office, the CH-47 production is low risk because
two prototypes have been produced during development and the Army recently
took delivery of its first low-rate initial production aircraft. Further,
the program reported that during low-rate production, it made significant
advances in the development and refinement of the system that are designed
to increase production efficiencies. Advances include the implementation
of the automated management execution system and the introduction of laser
tracking to identify key mounting points. These enhancements are geared
toward improving the manufacturing learning curve. However, the program
office acknowledges that the program will lose some of the learning
benefits during the anticipated break in production of the CH-47F in favor
of producing more MG-47s during the next lot of production.
Other Program Issues
In 2002, DOD directed the Army to produce 16 MH-47G aircraft for the
Special Operations Command before the start of the Army's low-rate
production for the CH-47F helicopters and to deliver those aircraft as
soon as possible. The Army initially estimated that this transfer of 16
aircraft for special operations would result in a 15-month delay in its
first unit equipped date for the CH-47F. However, according to the program
office, scheduling issues between the Army and the Special Operations
Command have been resolved. The Army now estimates that the 15-month
schedule slip has been reduced by about 6 months. The program office
reported that the CH-47F and MH-47G program strategy has been approved by
the Defense Acquisition Executive.
Further, the Army has recently approved the production of additional
CH-47F aircraft in the most recentProgram Objective Memorandum submission.
Additionally, the Army included in this submission an escalation of 19
CH-47F aircraft that had previously been scheduled at the end of the
program. These quantity changes resulted from the recent Army Aviation
Transformation Group's recommendations.
Agency Comments
The Army concurred with this assessment and provided technical comments,
which were incorporated where appropriate. Additionally, it commented that
the full-rate production decision was approved on November 22, 2004, by
the Army Acquisition Executive. Further, the program was rebaselined to
include a revised Acquisition Objective of 510 aircraft. Details of this
rebaselined program will be outlined in the December 2004 Selected
Acquisition Report.
Common Name: CKEM
The Army's CKEM is a hypervelocity missile designed to provide superior
lethality against current tanks, bunkers, buildings, and future advanced
threat armor. It is designed to provide a high rate of fire and a high
probability of kill beyond the range of tank guns, and at half the size
and weight of current kinetic energy missiles. The CKEM is a potential
candidate for use on the current Stryker Brigade and Future Combat System
vehicles. The Army is currently developing the CKEM in an Advanced
Technology Demonstration program.
Source: Lockheed-Martin--Missiles and Fire Control; Dallas, Texas.
Design start (9/03)
GAO review (1/05)
Development
start
(TBD)
Design review (TBD)
Production start (TBD)
Program officials believe the CKEM technologies will be mature when the
program enters system development. The Army is using an advanced
technology demonstration to develop the CKEM technologies to satisfy
future Army missile requirements. The technologies have already been
demonstrated in a relevant environment. Work remains to reduce the
technologies to the right size and show they can withstand the high
g-force environment. Funding inconsistencies and increased costs have
hampered technology development efforts and increased program risk.
Program officials expect at least one design change iteration once the
CKEM enters system development, which could happen in 2006 after
full-scale weapon system flight testing.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production review start design decision (1/05) (TBD)
review (TBD)
(TBD)
Common Name: CKEM
CKEM Program
Technology Maturity
Although none of its critical technologies are fully mature, the CKEM is
over a year from entering system development and all four technologies
have been demonstrated in a relevant environment. Program officials
believe all CKEM critical technologies will be fully mature when the
program proceeds with system development. The missile's four critical
technologies are a solid rocket motor, an attitude control system,
penetrator/lethality mechanisms, and guidance systems. CKEM engineers are
pioneering many of the system's technologies to satisfy future missile
requirements, which include reduced infrared signatures, longer ranges,
nondetonable propellants, and smaller size and weight.
Existing missile guidance and control components will not satisfy the size
and weight requirements and will not withstand the g-forces potentially
exerted by the CKEM. As a result, CKEM developers are working to
miniaturize existing components and improve tolerances for use under
greater velocities. The program completed testing of smaller guidance and
control prototypes in a high g-force environment. Engineers are also
designing a motor with an increased burn rate, advanced materials, and
innovative structural designs. They successfully tested a new solid-fuel
rocket motor, and they plan to begin controlled flight testing in April
2005. They also demonstrated the missile's lethality against a tank target
with advanced armor. However, system officials said that additional
technology funding is needed to fully develop component technologies and
produce a missile that will meet the size and performance goals.
Program officials believe they can mature technologies to the point that
only a single design iteration will be needed to satisfy Army objective
requirements during system development. They noted that the Assistant
Secretary of the Army for Acquisition, Logistics, and Technology
instructed them to forego involvement in the development of fire control
systems and instead focus solely on missile development. This could result
in integration problems that would require future design changes.
Other Program Issues
Program officials believe that inconsistent funding has hampered
development efforts. Over the last 3 years, the budget has been reduced
over $21 million. Those reductions were offset by reprogramming $17
million back into the program. Initially, competitive contracts were
awarded to two prime contractors. Citing funding discontinuity and
higher-than-expected contractor proposals, program officials did not
exercise an option for the second contractor's continued involvement. They
also cited funding as the reason the Army suspended international
cooperative agreements for assistance in developing associated
technologies.
The Army has not included a CKEM system development program in its future
funding plans. Nonetheless, program officials hope to have the system
ready to transition to system development in late 2006. CKEM technologies
can also be used to improve existing kinetic energy missiles, namely the
Line-of-Sight Anti-Tank missile.
Agency Comments
The Army concurred with our assessment.
Common Name: CVN-21
The Navy's CVN-21 class is the successor to the Nimitz-class aircraft
carrier and includes a number of advanced technologies in propulsion,
aircraft launch and recovery, weapons handling, and survivability. These
technologies will allow for increased sortie rates and decreased manning
rates as compared to existing systems. Many of the technologies were
intended for the second ship in the class, but they were accelerated into
the first ship in a December 2002 restructuring of the program.
Source: CVN-21 Program Office.
Program start (6/00)
Development
start
(4/04)
GAO review (1/05)
Design review (11/05)
Production decision-1st ship (1/07) Production decision-2nd ship (1/11) Initial
capability (9/15)
The CVN-21 entered system development in April 2004 with very few of its
critical technologies fully mature. This is due in part to DOD's decision
to accelerate the installation of a number of technologies from the second
ship to the first. The accelerated technologies are at much lower levels
of maturity. Program officials state that the extended construction and
design period that ends in 2014 allows further time for technology
development. Program officials have established a risk reduction strategy
that includes decision points for each technology's inclusion based on a
demonstrated maturity level. These points coincide with key design
milestones and include consideration of the fallback use of mature
technologies for all but two technologies. The program office has stated
that those two technologies are already mature and operational.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/04) (1/05)
review (1/07) (11/05)
Common Name: CVN-21
CVN-21 Program
Technology Maturity
Program officials reported that 3 of the 14 critical technologies were
mature at development start and that 4 more were approaching maturity. An
additional 7 were at much lower levels of readiness. The Navy expects that
10 of the 14 technologies will be mature or close to mature by the design
review in fiscal year 2006.
Some of the CVN-21 critical technologies are being developed by other
programs, not by the CVN-21 program. As a result, events in those programs
could affect the CVN-21 development time line. Those technologies are the
Volume Search Radar, Multi-Function Radar, Advanced Arresting Gear,
Evolved Sea Sparrow Missile and Joint Precision Approach and Landing
System. CVN-21 program officials reported that they are working closely
with all critical technology leads in those offices to ensure that their
time lines are integrated with the needs of the CVN-21 program. In case
those technologies do not mature in time for insertion into the carrier,
the CVN-21 program has identified existing or fully mature alternate
systems as backup technologies.
Since entering development, the program office has added 9 1,100-ton air
conditioning plants as a critical technology, and has added them to the
baseline design for the ship. The plants are not near maturity. The Navy
added the plants because the CVN 21's requirements for chilled water are
significantly higher than existing aircraft carriers. The Navy considers
this a low-risk development effort since they are using a proven
commercial design with upgrades to meet military shock, vibration, and
noise requirements.
Two of the four remaining technologies that are not mature, the
Omni-Directional Vehicle and Automated Weapons and Materials Movement
Technologies, are primarily mobile vehicles that can be accommodated late
in the design and construction schedule because they are not installed as
part of the ship. In addition, the Advanced Arresting Gear is not near
maturity, but according to program officials, it does not pose a
significant risk to the program because it is located high in the ship and
as such will be integrated in the latter stages of construction.
Program officials stated that it is not possible to mature some systems to
the best practices standard this early in development. One such system is
the Electromagnetic Aircraft Launch System, a replacement for the current
steam catapult system used to launch aircraft off carriers. This system
has been in development since the late 1990s, but due to the size and
complexity of the system, a prototype of it cannot be tested aboard a
surrogate ship.
Other Program Issues
Program cost estimates increased by more than $18 billion over the amount
reported last year as a result of the development start decision, which
added a second follow-on ship to the program, for a total production run
of three ships. Previous estimates were based on a single follow-on ship
and were not fully developed estimates for the entire program. In
addition, the cost estimates at development start more accurately reflect
potential inflation incurred by the shipbuilder during design and
construction of the ship.
Agency Comments
The Navy generally concurred with this assessment and reiterated that the
time frames for design and construction of an aircraft carrier allow for
evolving technologies to be brought to the ship later in the construction
cycle. It stated that if a certain technology does not mature in time for
ship construction, the technology will be replaced by a fall back
technology that may not meet projected capability, but it will at least be
equal to current capability.
Common Name: DD(X)
The Navy's DD(X) destroyer is a multimission surface ship designed to
provide advanced land attack capability in support of forces ashore and
contribute to U.S. military dominance in littoral operations. The program
is currently in the system design phase, and the Navy plans to authorize
detailed design and construction of the lead ship in March 2005. The Navy
plans to demonstrate the ship's critical technologies by building and
testing 10 developmental subsystems, referred to as engineering
development models.
Source: DD(X)Program Office (PMS 500).
Program start (1/98)
Development
start
(3/04)
GAO review (1/05) Production decision-1st ship (3/05) Initial capability (1/13)
None of the DD(X) technologies included in the 10 engineering development
models were mature at the start of development, and none are expected to
be mature at the March 2005 decision to authorize detailed design and
construction of the lead ship. Current plans call for demonstrating 3 of
the 10 subsystems by the end of the program's design review in August 2005
and an additional 3 in September 2005. Backups are available for only 2 of
the 10 developmental subsystems. As most of the testing of the engineering
development models will take place in the months immediately before and
after the design review, it is not likely that design stability will be
achieved by the time of that review.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (3/04) (1/05)
review (3/05) (8/05)
Common Name: DD(X)
DD(X) Program
Technology Maturity
None of the DD(X) technologies were mature at the start of development,
and none are expected to be mature at the March 2005 decision to authorize
detailed design and construction of the lead ship. By the end of the
design review in August 2005, only three subsystems are expected to
complete testing: the autonomic fire suppression system, the hull form,
and the infrared mock-ups. The integrated power system, peripheral
vertical launch system, and total ship computing environment are expected
to complete testing in September 2005. The dual band radar and integrated
deckhouse are to complete testing well after the design review. The
advanced gun system and undersea warfare system will not be tested as
fully integrated systems until after installation on the first ship.
The current plans for the integrated undersea warfare system include
testing the functionality of components, such as the ability of one of two
sonar arrays to detect mines, but not demonstrating the system as a whole.
Component testing of the advanced gun system is ongoing and has resulted
in changes to some components. The weight of the gun system increased as a
result of an effort to improve producibility and cost efficiency.
Land-based testing of the gun system is planned for the summer of 2005,
and flight tests for the munition are to be completed in September 2005.
The two technologies will not be tested together until after ship
installation.
The dual band radar is not scheduled to complete testing until fiscal year
2008, well after the design review. Program officials have made some
assumptions about where in the deckhouse it will be placed. If its weight
increases or other technical factors cause it to be relocated, a redesign
effort may be needed. In addition, recent component testing and design
reviews of portions of the radar have revealed shortfalls in performance.
The integrated power system recently completed a change in design, which
helps mitigate previously experienced weight issues. These design changes
will not be tested until after design review. In addition, technical
issues with components of the Permanent Magnet Motor have arisen that
could affect schedule and cost. Plans for the integrated power system do
include the use of a fallback technology, but would require trade-offs in
requirements.
Design Stability
Most of the testing of the engineering development models will take place
around the time of design review. Even if tests are successful, they will
not be completed in time to achieve design stability. Problems found in
testing could result in changes in the design, delays in product delivery,
and increases in cost. Detailed knowledge about subsystems and their
component technologies is necessary for developing the system design. If
this information is not available and assumptions about operating
characteristics have to be made, redesign may be necessary when reliable
information is available.
Agency Comments
The Navy acknowledges the aggressive DD(X) schedule but maintains that the
ability to deliver revolutionary capabilities to the fleet with reduced
crew necessitates some element of risk. Congress has expressed support for
the Navy's approach, stating in the report accompanying the fiscal year
2005 national defense authorization act "the conferees believe that taking
such risks is warranted to ensure that the DD(X) technologies are not
obsolete, and that the Navy has taken adequate steps to mitigate the risks
before ship construction begins."
The Navy disagrees with the assessment that the DD(X) will not achieve
design stability prior to design review. It stated that the ship design is
stable and reflects release of the final baseline leading to design
review. It also stated that the results from continued engineering
development model testing will be incorporated in the design and that
permission to begin design review will be based on meeting specific
entrance criteria that measure the availability of the appropriate data on
technologies.
GAO Comments
Design stability requires detailed knowledge of the form, fit, and
function of all technologies as well as the integration of individual,
fully matured subsystems. As testing for DD(X) technologies continues
beyond the dates scheduled for design review, this knowledge may not be
achieved when required.
Common Name: E-10A
The Air Force's E-10A program is being designed to exploit emerging radar
sensor technologies for airborne surface surveillance and focused air
surveillance for cruise missile defense. It will consist of an active
electronically scanned array radar; a modified Boeing 767 commercial
airframe; and a battle management, command and control computer mission
subsystem. Development of the radar has already begun; and while funding
of the first airframe has begun, the overall program has not yet entered
development. We assessed the entire system.
Source: E-10A/MP-RTIP Program Office.
GAO review (1/05)
Development
start
(4/05)
Design review (7/08)
Low-rate decision (7/10) Initial capability (3/15)
We have not assessed the technology maturity of the overall E-10A program
because program officials have not yet completed their identification and
assessment of the system's critical technologies. However, they assessed
the radar's critical technologies in October 2003, prior to the radar's
Milestone B decision. At that time, officials determined that six of the
radar's nine critical technologies were mature. The remaining three radar
technologies are not expected to reach full maturity until the first E-10A
flight in 2010. Development challenges for the overall E-10A program
include the integration of the radar, airframe, and battle management
subsystems and the software development for the battle management
subsystem.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production review start design decision (1/05) (4/05)
review (7/10)
(7/08)
Desired level of knowledge
Common Name: E-10A
E-10A Program
Technology Maturity
Because program officials have not yet completed their identification and
assessment of the program's critical technologies, we were unable to
assess the technological maturity of the overall E-10A system. Program
officials are preparing a technology development strategy as well as a
technology readiness assessment in support of the upcoming development
decision for the overall weapon system.
Program officials have identified and assessed the critical technologies
associated with the radar subsystem. They determined that six of the nine
critical technologies were mature. The remaining three radar technologies
are not expected to reach full maturity until the first E-10A flight in
2010. Tests on a smaller prototype have demonstrated the functional
capabilities of the radar, but are not representative of the E-10A radar's
form or fit. The final form of the radar will be significantly larger and
will not be integrated on the airframe until flight testing in 2010.
Design Stability
We could not assess design stability for the E-10A as the overall system
has not yet entered system development. As a result, the total number of
drawings has not yet been determined. However, a final design review of
the radar subsystem was conducted in June 2004. Program officials stated
that over 90 percent of the expected drawings for the radar had been
released at that point. They do not expect the number of radar drawings to
change significantly because key subsystems for the radar are already
being produced for other weapon systems.
Other Program Issues
In fiscal years 2003 and 2005, the E-10A's proposed budget was reduced by
Congress. Both budget cuts resulted in a restructuring of the program. As
part of the last restructuring, program officials requested that the
system development milestone decision be accelerated from July to April
2005. However, in a recent budget decision, DOD reduced the program's
fiscal year 2006 and 2007 budget request by a total of $600 million. If
this reduction is sustained, the E-10A program will have to undergo yet
another restructuring.
According to program officials, the software development for the battle
management command and control subsystem is the most critical program
risk. This subsystem will provide the machine-to-machine communications
capability needed to operate with prospective and legacy command and
control systems. The development of the battle management subsystem has
lagged behind the radar and airframe; the Air Force just awarded a
development contract for the subsystem in September 2004.
The 767 airframe is a commercial derivative that will be modified to meet
the E-10A's military requirements. In addition, the integration of the
large scale radar and the battle management subsystem may necessitate
additional modifications. The Air Force has only contracted for one
aircraft, which will be used as a testbed. As a result of the budget cuts,
the delivery of this aircraft has slipped about 1 year.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that the
E-10A program has been restructured to accommodate both an Office of the
Secretary of Defense directed development decision delay and congressional
budget cuts. It further noted that the restructuring has been accomplished
with minimal impact to ongoing design activities and has retained the
radar/E-10A synchronization necessary to deliver an E-10A weapon system
that is responsive to warfighter requirements. The Air Force also provided
technical comments, which we incorporated as appropriate.
Common Name: E-2AHE
The Navy's E-2 AHE is an all-weather, twin engine,
carrier-based, aircraft designed to extend early
warning surveillance capabilities. It is the next in a
series of upgrades the Navy has made to the E-2C
Hawkeye platform since its first flight in 1971.
The E-2 AHE is designed to improve battle space
target detection and situational awareness,
especially in littoral areas; support Theater Air
and Missile Defense operations; and improve
operational availability.
Source: Program Executive Officer, Tactical Aircraft Programs (PMA-231).
Program/ development start (6/03) GAO review (1/05)
Design review (11/05)
Low-rate decision (3/09) Initial capability (4/11) Full-rate decision (12/12)
Last procurement (2019)
The E-2 AHE program entered system development in June 2003 without
demonstrating that its four critical technologies had reached full
maturity. Since that time, one of the program's four critical technologies
has reached full maturity. Program officials do not expect to achieve
maturity on the remaining three critical technologies until after the
design review. While more mature backup technologies exist for the three
critical technologies, use of the backup technologies would result in
degraded system performance or reduced ability to accommodate future
system growth. The program office has made progress on completing design
drawings and plans to have the majority of drawings completed by the time
of design review in November 2005. However, until the technologies are
mature, the potential for design changes remains.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (6/03) (1/05)
review (3/09) (11/05)
Common Name: E-2 AHE
E-2 AHE Program
Technology Maturity
One of the E-2 AHE's four critical technologies (the space time adaptive
processing algorithms and associated processor) is mature. The program
expects the remaining technologies (the rotodome antenna, a silicon
carbide-based transistor for the power amplifier to support UHF radio
operations, and the multichannel rotary coupler for the antenna) to be
fully mature after the November 2005 design review but before the start of
production in March 2009.
More mature backup technologies exist for the three technologies (the
rotodome antenna, the silicon carbide-based transistor, and the
multichannel rotary coupler) and were flown on a larger test platform in
2002 and 2003. However, use of the backup technologies would result in
degraded system performance or reduced ability to accommodate future
system growth due to size and weight constraints. The next AHE technology
readiness assessment is to be performed prior to the production decision
for the system in fiscal year 2008, and the program office anticipates
that the critical technologies will be mature at that time.
Design Stability
The program had completed almost 35 percent of its engineering drawings at
the time of our review. Program officials project that they will have 81
percent of the drawings completed by the time of design review in November
2005, and 100 percent completed by the planned start of production in
March 2009. However, the technology maturation process may lead to more
design changes.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the AHE
program successfully executed Preliminary Design Review (PDR) in October
2004. The program office also completed PDRs for each of the AHE
subsystems, to include critical technologies, and documented appropriate
risks. The Navy noted that all program risks and associated mitigation
plans, including those for critical technologies, were reviewed for PDR.
According to the Navy, critical technologies do not currently represent a
high risk to the AHE program. Navy officials stated that the program is on
schedule and meeting cost and performance objectives.
Flight tests of the critical technologies are planned during system design
and development. The Navy noted that flight tests will inherently increase
the technology readiness levels (TRLs) of the critical technologies. These
TRLs will be formally assessed before the production decision in fiscal
year 2009.
Common Name: EA-18G
The Navy's EA-18G is an electronic attack aircraft designed to jam enemy
radar and communications and conduct electronic warfare as part of a
battle group. The program was approved as a replacement for the EA-6B
aircraft, and it will integrate its electronic warfare technology and
components into the F/A-18F platform. Because of the heavy use of the
aging EA-6B aircraft, a large number are being retired due to wear. To
prevent a gap in electronic warfighting capabilities, DOD intends to begin
fielding the EA-18G in 2009.
Source: F-18 Program Office.
Program start (8/02)
Development
start
(12/03)
GAO review (1/05)
Design review (4/05)
Low-rate decision (4/07) Full-rate decision (4/09) Initial capability (9/09)
Last procurement (2013)
The EA-18G entered system development without demonstrating that its five
critical technologies had reached full maturity. Three technologies were
very close to maturity, and two technologies have not been demonstrated in
the form they will exist on the aircraft. While the EA-18G's critical
technologies are similar to mature technologies on the EA-6B and the
F/A-18F, integrating them into the EA-18G will involve form and fit
challenges. The EA-18G will rely on planned capability upgrades developed
for the EA-6B, which could increase program risk. In addition to these
challenges, the program also faces risks with software integration. The
program office could not project the number of releasable drawings until
the design review in April 2005.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (12/03) (1/05)
review (4/07) (4/05)
Common Name: EA-18G
EA-18G Program
Technology Maturity
None of the EA-18G's five critical technologies are fully mature. While
they are similar to the mature technologies found on the EA-6B and the
F/A-18F, integrating those technologies on the EA-18G will involve form
and fit challenges. Three of the critical technologies, the ALQ-99 pods,
the F/A-18F platform, and the tactical terminal system, are approaching
full maturity. The remaining two technologies, the receiver system and the
communications countermeasures set, are not mature.
The Navy is funding a study to develop a new tactical terminal system,
which it hopes to incorporate into the EA-18G to help reduce weight,
conserve power, and reduce cooling requirements. According to the program
office, similar systems are already in use in DOD. For example, the
Special Operations Forces are using a system the size of a credit card,
significantly lighter than the current 50-pound system. If the new system
is not developed in time for the start of aircraft production, the program
plans to use a modified version of the tactical terminal system currently
in use on the EA6B.
Raytheon Systems is developing the communications countermeasures set for
the EA-18G, which will be based on a similar system currently used on the
C-130J aircraft. Raytheon is expected to begin delivery of the system in
January 2005.
Design Stability
We could not assess the design stability of the EA-18G as the number of
releasable drawings is not yet available. The EA-18G Program Office does
not expect to have an estimate of the number of design drawings until the
design review, currently planned for April 2005. By not having sufficient
design drawing information, the program places itself at increased cost
and schedule risk.
Other Program Issues
The EA-18G Program Office plans to build one-third of its aircraft during
low-rate initial production due to the need to begin replacing retiring
EA-6Bs by 2009. Any problems identified in testing during production could
result in costly modifications to the already produced aircraft. The
program office has indicated it may proceed into production even if minor
known deficiencies exist.
Because the EA-18G is using the same airframe as the F/A-18F, the program
office is conducting a study to determine what impact the increased
vibration of the EA-18G will have on the life span of the airframe. The
program office also plans to certify the aircraft to land aboard ship at
47,000 pounds, which is 3,000 pounds heavier than the similar F/A-18F
aircraft.
The F/A-18E/F aircraft has experienced problems with "wing buffet," which
can affect performance. The F/A-18F Program Office has made design
changes, which it expects will resolve the issue.
The ALQ-99 pods successfully completed shake testing, which evaluated
their ability to handle the increased vibrations of the EA-18G.
The EA-18G program may experience minor cost growth if cuts are made in
the number of EA-6Bs that are upgraded because the EA-18G program plans to
procure some of the same components as those used in the EA-6B ICAP III
upgrade. Decreased purchases by the EA-6B program would increase unit
costs of these items, thereby increasing the cost to the EA-18G.
Agency Comments
The Navy provided technical comments, which were incorporated as
appropriate.
Common Name: EELV
The Air Force's EELV program acquires commercial satellite launch services
from two competitive families of launch vehicles-Atlas V and Delta IV. The
program is an industry partnership to support and sustain assured access
to space and reduce the life-cycle cost of space launches by at least 25
percent over previous systems while meeting the government's launch
requirements. Different types of lift vehicles may be used, depending on
the particular mission. We assessed both the Atlas V and Delta IV.
Source: (Left) (c) 2003 ILS/Lockheed Martin; (right) (c) 2003 The Boeing
Company.
Program start (12/96)
Development
start
(10/98)
Production decision (unknown)
First flight-Atlas V (8/02) First flight-Delta IV (11/02)
GAO review (1/05) Initial capability (TBD)
Although the EELV Program Office has access to technology, design, and
production maturity information, it has not formally contracted for this
data because it is acquiring the launch service rather than developing the
system itself. To date, seven successful EELV launches have occurred-two
government and five commercial. With a history of launch delays, the heavy
lift vehicle (HLV) had its first demonstration flight in 2004. The EELV
program's total costs have increased about 86 percent due to a decline in
the commercial launch market upon which the business case was based.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production
start design review decision
(10/98) review (1/05) (unknown)
(10/99)
Desired level of knowledge
Common Name: EELV
EELV Program
Technology Maturity
We could not assess the technology maturity of EELV because the Air Force
has not formally contracted for information on technology maturity from
its contractors.
Design Stability
We could not assess the design stability of EELV because the Air Force has
not formally contracted for the information needed to conduct this
assessment.
Production Maturity
We could not assess the production maturity of EELV because the Air Force
has not formally contracted for information that would facilitate this
assessment.
Other Program Issues
The decline in commercial satellite launch needs in the late 1990s
resulted in program cost increases and a reduction in the anticipated
number of Atlas V and Delta IV launches. Cost increases greater than 25
percent over the program's objective triggered a Nunn-McCurdy breach (see
10 U.S.C. 2433), requiring a review by the Secretary of Defense and a
report to Congress. As provided by the law, DOD certified in April 2004
that the program is critical to national security and its cost estimates
are reasonable. In conjunction with the certification, the Air Force is
updating the 1994 Space Launch Modernization Plan (which examines launch
alternatives), and it revised its mission model to reflect a reduction of
launch vehicles. Also, the Air Force is reviewing contract structures that
could include cost type provisions for the follow-on procurement of EELV
services.
The EELV program has continued to experience schedule changes to the Delta
IV heavy lift vehicles (HLV). The Delta IV heavy-lift demonstration flight
that was planned for July 2004 did not occur until December 2004 and the
HLV first operational flight was delayed by 6 months. According to the Air
Force, these delays occurred due to a number of factors, including other
launch priorities, slips in launch dates for the first three Delta IV
missions, modifications to the HLV launch pad, and design problems
encountered during launch pad testing. In addition, both contractors are
addressing technical issues related to meeting program requirements. The
Boeing Company is addressing a Delta IV issue related to the separation of
the payload fairing device (which encloses and protects the payload).
Lockheed Martin is dealing with an Atlas V intermediate class booster
issue regarding the excessive vibration caused by the noise generated at
liftoff.
According to DOD, initiatives are in place to reduce EELV risk and ensure
access to space. The initiatives are aimed at critical rocket components,
improving the producibility of the upper stage engine, systems engineering
processes, and the availability of critical staff and facilities. Related
to these initiatives, there are three technical issues that the Air Force
is addressing. Parts of the RL-10 upper stage engine are common to both
the Delta IV and the Atlas V and an engine flaw could potentially ground
both vehicles. However, the Air Force maintains that the RL-10 has flown
successfully since the 1960s. Also, the Atlas V continues to rely on the
Russian-made RD-180 propulsion technology (though the contractor plans to
start building this technology in the United States with a first military
launch by 2012). Additionally, until the West Coast launch pad becomes
operational in 2005 in time for the first U.S. government need in 2006,
the Air Force is limited to launching the Atlas V from its East Coast
launch pad.
Agency Comments
In commenting on a draft of this assessment, the Air Force acknowledged
that technology, design, and production maturity data are not required as
a deliverable, and therefore it does not have the authority to provide
this information. However, daily interaction with both contractors
provides insight into the readiness of the launchers as well as the
potential for cost increases and schedule issues.
Common Name: EFV
The Marine Corps' EFV (formerly called the Advanced Amphibious Assault
Vehicle) is designed to transport troops from ships offshore to their
inland destinations at higher speeds and from farther distances than the
existing Assault Amphibious Vehicle 7A1 (AAV-7A1). It is designed to be
more mobile, lethal, reliable, and effective in all weather conditions.
The EFV will have two variants-a troop carrier for 17 combat equipped
Marines and 3 crew and a command vehicle to manage combat operations in
the field. We assessed both variants.
Source: General Dynamics Land Systems.
Program start (3/95)
Development
start
(12/00)
GAO review (1/05) Low-rate decision (12/05) Full-rate decision (11/08) Initial
capability (12/08) Last procurement (2016)
The EFV's technology is mature and the design is stable. However, at the
start of development, only four out of five critical technologies were
mature. The demonstration of the moving map, the last of these
technologies, has led to full technology maturation. The design was close
to meeting best practice standards at the design review, signifying the
design was relatively stable. Early development of fully functional
prototypes and other design practices have facilitated design stability.
Based on the functional prototyping, the program expects changes to
roughly 12 percent of the drawings. The demonstration of production
maturity remains a concern because the contractor does not collect
statistical process control data.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (12/00) review
(1/05) (12/05) (1/01)
Common Name: EFV
EFV Program
Technology Maturity
All five of the EFV's critical technologies are mature. The moving map
navigation technology, which was not mature at the start of product
development, was recently demonstrated in an operational environment on
the full-up system prototype. The moving map technology provides
situational awareness.
Design Stability
The program has now released all of its drawings for the troop carrier
variant. However, 12 percent require design changes to address reliability
issues. At the time of critical design review in 2001, 84 percent of the
expected drawings had been released, signifying the design was approaching
stability. The program is currently seeking to reduce the threshold for
the reliability key performance parameter based on a USMC reevaluation of
concept of operations. According to program officials, reliability is a
moderate risk but may elevate to high risk if the requirement change is
not approved. Program officials expect the EFV to meet revised reliability
thresholds by initial operational testing in November 2007.
According to the program, recent tests of an improved track and wheel
design demonstrated significant improvements in reducing vibration on the
vehicle. Program officials estimate that vibration levels have been
reduced by up to 50 percent over previous measurements. The new track and
wheel design will be incorporated on the vehicles used for the operational
assessment in March 2005.
Production Maturity
The program expects to enter low-rate production in December 2005. It will
do so without requiring the contractor to use statistical process controls
to demonstrate that the 12 critical processes are producing quality and
reliable products. Instead, the contractor plans to have 95 percent of the
production tooling and manufacturing processes in place by low-rate
production start. These processes are being utilized and refined to build
the prototype vehicles. Additionally, the program and the contractor are
in planning stages for production readiness reviews that assess production
processes, identify any additional critical manufacturing processes, and
determine the benefit of using statistical process controls. Because the
final EFV production facility is not ready, the contractor is using the
planned manufacturing processes to build prototypes at the development
facility. This will provide verification of these manufacturing processes.
However, when production moves to the new facility, processes will need to
be validated again to ensure they work as expected.
Other Program Issues
The program tracks a number of entrance criteria for low-rate production
and is on track to meet most of those criteria. One key entrance criterion
is an operational assessment scheduled for March 2005. The assessment will
include the demonstration of a launch and recovery from an amphibious
ship; transportation of Marines on water and on land; and negotiation of
the vehicle in a 4-foot surf. Another key entrance criterion,
demonstration of system reliability, is a moderate risk and may delay
low-rate production.
Agency Comments
The EFV Program Office was provided an opportunity to comment on a draft
of this assessment, but it did not have any comments.
Common Name: ERGM
The Navy's ERGM is a rocket-assisted projectile that is fired from guided
missile destroyers. ERGM is one concept the Navy is considering to meet
its fire support requirement. ERGM can be guided to targets on land at
ranges of between 15 and 50 nautical miles to provide fire support for
ground troops. It is expected to offer greater range and accuracy than the
Navy's current 13 nautical mile gun range. ERGM required modifications to
the 5-inch gun, a new munitions-handling system, and a new fire control
system. We assessed the projectile.
Source: Naval Gunner y Project Office.
Program/development
start
(7/96)
Design review (5/03)
GAO review (1/05) ERGM contract ends (12/05) Low-rate decision (10/08) Full-rate
decision (1/10) Initial capability (1/10)
Since our last assessment, the ERGM program has not demonstrated
additional technology maturity or design stability. Due to problems with
the rocket motor and propelling charge, flight testing was halted, and the
program has been unable to demonstrate the maturity of 7 of its 20
critical technologies. The program plans to resume flight testing in
February 2005. If that test is successful, four technologies will
demonstrate maturity. The program also stated that ERGM's design drawings
will not be completed because of limited program funding. Therefore, ERGM
will not reach design maturity under Raytheon's current contract. Finally,
due to concerns about ERGM's inconsistent test performance and projected
unit cost, the Navy plans to recompete the 5-inch guided projectile
requirement and restart development by mid-fiscal year 2006. If ERGM is
not selected, it will cease to be a program.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (7/96) review
(1/05) (10/08) (5/03)
Common Name: ERGM
ERGM Program
Technology Maturity
Thirteen of ERGM's 20 critical technologies are mature. The program has
completed development work on six of the seven remaining technologies, but
has yet to test them in an operational environment. Program officials
currently project that four of the remaining technologies, the tactical
telemeter and the three unitary warhead-related technologies, will be
demonstrated during a February 2005 flight test. The program's fiscal year
2005 budget request was reduced from $11.3 million to $4.5 million, and
the program's funds will be exhausted in March 2005. Unless the program
receives additional funding, none of the three remaining critical
technologies- antijam electronics, safe and arm device and fuze, and data
communication interface-will achieve maturity under the current contract
since the Navy plans to recompete the 5-inch guided projectile requirement
and restart development in early to midfiscal year 2006. If the ERGM
concept is selected, the program office projects that all ERGM critical
technologies would be demonstrated in an operational environment by 2008.
Design Stability
The program has released approximately 51 percent of its 140 production
representative drawings. None of ERGM's production representative
engineering drawings were released at its May 2003 design review. Instead,
the program conducted this review with less mature drawings and used them
to validate the design of the development test rounds. In our March 2004
report, the program office stated that it would have a complete and
updated drawing package by October 2004. However, because of a lack of
funds and the 5-inch guided projectile competition that will end the
current ERGM contract, the contractor will not complete this drawing
package. If the ERGM concept is selected, the option exists to complete
this drawing package.
Production Maturity
Since the future of the ERGM concept will not be determined until January
2006, it is unclear whether and when the program will proceed to
production. If the ERGM concept is chosen, the current manufacturing plan
states the contractor will identify key product characteristics and then
determine how to implement statistical process control.
Other Program Issues
In May 2004, the Navy awarded a contract to ATK to demonstrate an
alternative precision-guided munition concept-the Ballistic Trajectory
Extended Range Munition (BTERM). BTERM will likely be one of the concepts
competing for the new development contract. In fiscal years 2004 and 2005,
the Navy budgeted $35 million for the BTERM effort. The BTERM technology
demonstration includes six guided flight tests in 2005. At this point,
none of the BTERM critical technologies have reached maturity. However,
according to the project office, the six flight tests, if successful, will
demonstrate most of BTERM's critical technologies in a relevant or
operational environment. Finally, the latest ERGM program cost and
schedule estimates do not reflect the potential cost and time needed to
complete the 5-inch guided projectile development effort. The Navy is
currently considering an acquisition strategy that would start a new
development program with a revised program baseline, which could delay
initial operational capability until 2011 depending on the maturity of the
concept selected. The procurement cost of this new program will likely be
much higher than is currently reported for ERGM because the latest cost
estimate for the ERGM program is based on the procurement funding
available in the future year defense plan, not current inventory
requirements.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that it
intends to issue a request for proposal in fiscal year 2005 and select an
Extended Range Munition (ERM) development contractor in fiscal year 2006.
It will request that the program start the system development phase due to
the maturity of guided projectile concepts that could meet ERM
requirements. The Navy also stated that research, development, test, and
evaluation (RDT&E) funds in fiscal year 2006-2011 will be used for the ERM
development effort, resulting in an initial operational capability of no
later than fiscal year 2011. Depending upon the maturity of the concept
selected, development could end as early as fiscal year 2008 with a fiscal
year 2009 initial operational capability. In this case, fiscal year
2006-2008 RDT&E funding (about $58.4 million) would be used to complete
the program and fiscal year 2009-2011 funding (about $87.3 million) would
support spiral development and/or product improvement initiatives.
Common Name: Excalibur
The Army's Excalibur is a family of global positioning system-based,
fire-and-forget, 155-mm cannon artillery precision munitions. It is
intended to improve the accuracy and range of cannon artillery. Also, the
Excalibur's near vertical angle of fall is intended to reduce the
collateral damage area around the intended target, making it more
effective in urban environments than the current artillery projectiles.
The Future Combat Systems' non-line-of-sight cannon requires the Excalibur
to meet its required range.
Source: U.S. Army.
Program/development
start
(5/97)
GAO review (1/05) Low-rate decision (7/06) Full-rate decision (6/08) Initial
capability (9/08) Last procurement (2018)
The Excalibur program's critical technologies are not fully mature, even
though product development began over 7 years ago. Program officials
expect to have technology maturity by June 2005. The program has achieved
design stability. Currently, almost all of the Excalibur drawings are
completed and could be released to manufacturing. However, the Excalibur
is undergoing testing that may lead to design changes. The program has
encountered a number of challenges since development began, including a
decrease in planned quantities, a relocation of the contractor's plant,
early limited funding, technical problems, and changes in program
requirements. It merged with the Trajectory Correctable Munition program
in 2002.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (5/97) (1/05)
review (7/06) (6/05)
Common Name: Excalibur
Excalibur Program
Technology Maturity
None of the Excalibur's three critical technologies- the guidance control
system, the airframe, or the warhead-are fully mature. According to
program officials, all three have been demonstrated in a relevant
environment, and they are expected to reach full maturity before the
design review in June 2005. The warhead was not considered a critical
technology in 1997 because the Excalibur design called for a warhead that
was under production for other munitions. At the Army's direction, the
program has undertaken development of a different warhead that is
currently undergoing testing.
Design Stability
The most recent program restructure divided the design review into two
reviews. The first, scheduled for June 2005, freezes the first article
test design and the second, scheduled for the first quarter of fiscal year
2006, freezes the production design. The program recently completed an
Early Fielding Technical Data Package review of the design drawings. The
review found that about 97 percent of the Excalibur engineering drawings
are complete and releasable to manufacturing. The program office plans to
have all of the drawings complete by the June 2005 design review. The
Excalibur has to complete safety and other testing before it is ready for
production. This testing could lead to design changes.
Other Program Issues
The program has gone through many changes since the beginning of product
development in May 1997. It was almost immediately restructured due to
limited funding, and it was restructured again in 2001. The program was
again restructured and merged with a joint Swedish/U.S. program known as
the Trajectory Correctable Munition. This merger has helped the Excalibur
deal with design challenges, including issues related to its original
folding fin design. In May 2002, due to the cancellation of the Crusader,
the Army directed the restructure of the program to include the Future
Combat Systems' non-line-of-sight cannon. In December 2002, the Acting
Under Secretary of Defense (Acquisition, Technology, and Logistics)
approved an early fielding plan for the unitary version. The plan
currently includes developing the unitary version of the Excalibur in
three spirals. In the first spiral, the projectile would meet its
requirements for accuracy in a nonjammed environment and lethality and
would be available for fielding to Joint Lightweight 155mm cannon in
September 2006. In the second spiral, the projectile would be improved to
meet its requirements for accuracy in a jammed environment and reliability
and would be available for fielding to the Future Combat Systems'
non-line-of-sight cannon in September 2008. Finally, in the third spiral,
the projectile would be improved to meet its range requirement and would
be available for fielding to all systems in late fiscal year 2011.
The net effect of these changes has been to increase the program's
schedule and to substantially decrease planned procurement quantities. As
a result, the program's overall costs and unit costs have dramatically
increased.
Agency Comments
The Army provided technical comments, which were incorporated as
appropriate.
Common Name: F/A-22Raptor
The Air Force's F/A-22, originally planned to be an air superiority
fighter, will also have air-to-ground attack capability. It is being
designed with advanced features, such as stealth characteristics, to make
it less detectable to adversaries and capable of high speeds for long
ranges. It also has integrated aviation electronics (avionics) designed to
greatly improve pilots' awareness of the situation surrounding them. It is
designed to replace the Air Force's F-15 aircraft.
Source: F/A-22 System Program Office.
Program start (10/86)
Development
start
(6/91)
Low-rate decision (8/01) GAO review (1/05) Full-rate decision (3/05) Initial
capability (12/05) Last procurement (2011)
The F/A-22 entered production without ensuring that production processes
were in control. The Air Force expects to have about 27 percent of the
aircraft on contract prior to the full-rate decision in March 2005, yet
quality issues remain. For example, the F/A-22 has not achieved important
reliability goals and some components, like the canopies, are not lasting
as long as expected. Technology and design matured late in the program and
have contributed to numerous problems. Avionics problems were discovered
late in development, which resulted in large cost increases and caused
testing delays. The potential for further cost increases and schedule
delays exists until initial operational testing and followon testing are
completed. Additionally, $7 billion in cost reductions has to be achieved
to keep cost growth within the congressionally mandated production cost
limitation.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (6/91) review
(8/01) (1/05) (2/95)
Common Name: F/A-22 Raptor
F/A-22 Raptor Program
Technology Maturity
The three critical F/A-22 technologies (supercruise, stealth, and
integrated avionics) appear to be mature. However, two of these
technologies, the integrated avionics and stealth, did not mature until
several years after the start of development. Integrated avionics have
been a source of major problems, delaying developmental testing and the
start of initial operational testing. Since 1997 the costs of avionics
have increased by over $801 million and problems discovered late in the
program were the major contributor. In April 2004, the Air Force began
initial operational test and evaluation after reporting that these
problems were corrected.
Design Stability
The F/A-22 design is essentially complete, but it matured slowly, taking
over 3 years beyond the critical design review to meet best practice
standards. The late drawing release contributed to parts shortages, work
performed out of sequence, delayed flight testing and increased costs.
Design changes resulted from flight and structural tests. For example,
problems with excessive movement of the vertical tails and overheating
problems in the fuselage and engine bay required design modifications. The
Air Force completed development testing in December 2004 and operational
testing in November 2004. The Air Force is in the process of evaluating
the results of operational testing. The results of this evaluation could
result in additional design changes.
Production Maturity
The program office stopped collecting process control information in
November 2000. The contractor estimated that nearly half of the key
processes had reached a marginal level of control, but not up to best
practice standards. The Air Force has 67 production aircraft on contract.
The Air Force relies on the contractor's quality system to verify
manufacturing and performance requirements are being met. However, the Air
Force has not demonstrated the F/A-22 can achieve its reliability goal of
3 hours mean time between maintenance. It does not expect to achieve this
goal until 2008 when most of the aircraft will have already been bought.
Best practices call for meeting reliability requirements before entering
production. As of mid-October 2004, the Air Force had only demonstrated
about 22 percent of the reliability required.
Other Program Issues
The Air Force is counting on future cost reduction plans to offset
estimated cost growth and enable the program to meet the latest production
cost estimate. If these cost reduction initiatives are not achieved as
planned, production costs could increase.
The Integrated Maintenance Information System (IMIS), a paperless
computerized maintenance system, is used by the Air Force to maintain the
F/A-22. The system collects and analyzes problem data and develops a
maintenance solution. The system has not functioned properly causing
unnecessary maintenance actions. This has affected the Air Force's ability
to fly the test aircraft on schedule. The Air Force installed new software
in February 2004 to address many of the errors generated by IMIS and
uncovered additional errors. According to the Air Force, these problems
were resolved in July 2004. In November 2004, the Air Force upgraded IMIS
to a commercially supportable operating platform and database that added
new functionality such as wireless connectivity.
Agency Comments
In commenting on a draft of this assessment, the Air Force provided
technical comments, which were incorporated as appropriate. The Air Force
also stated that, in coordination with the DCMA and contractor teammates,
the program is aggressively pursuing cost reduction initiatives to meet
cost goals. It stated that these goals represent a significant reduction
in per aircraft cost and include substantial improvements to production by
the primes and subcontractors. The Air Force disagreed, however, with the
value we reported in our draft assessment. It stated that the initiatives
total $2.5 billion. The Air Force also indicated that the reliability of
the F/A-22, while maturing, is already comparable to legacy Air Force
fighter aircraft while delivering a required combat capability that cannot
be achieved by legacy platforms.
GAO Comments
We reviewed the Air Force's comments concerning projected production cost
reduction savings and determined that the Air Force will have to reduce
the current production estimate by approximately $7 billion to execute the
program within a congressional mandated cost cap.
Common Name: FCS
The FCS, a program that will equip the Army's new transformational modular
combat brigades, consists of a family of systems composed of advanced,
networked combat and sustainment systems, unmanned ground and air
vehicles, and unattended sensors and munitions. Within a system-of-systems
architecture, the first increment of the FCS features 18 major systems and
other enabling systems along with an overarching network for information
superiority and survivability.
Source: Program Manager, Unit of Action, U.S. Army.
Program start (5/00)
Development
start
(5/03)
GAO review (1/05)
Design review (9/10)
Low-rate decision (9/12) Initial capability (12/14) Full-rate decision (8/16)
Last procurement (unknown)
The FCS program began a major restructuring in July 2004, which delays
fielding 4 years, until 2014, and spirals various FCS technologies to the
current force. The restructuring increased the priority for developing and
demonstrating the FCS network. The program also continues refining
requirements. In some cases, the Army has decided to use different
technologies, which are less mature than the original technologies. The
program expects all of its 54 critical technologies to be mature by the
end of fiscal year 2008. Technology maturation will continue throughout
system development, with an associated increase in the risk of cost growth
and schedule delays. Since the FCS will dominate Army investment accounts
over the next decade, cost growth and schedule delays could affect all
Army acquisitions.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (5/03) (1/05)
review (9/12) (9/10)
Common Name: FCS
FCS Program
Technology Maturity
One of the FCS program's 54 critical technologies is currently mature.
Overall, the program's current technology maturity is slightly less than
it was in May 2003 when the program began development.
The program is not appropriately applying best practices to maturing its
critical technologies. It considers technical risk acceptable as long as
it can estimate that the technologies will be demonstrated in a relevant
environment before design review. Also, it does not consistently include
form or fit in technology maturation because it views sizing the
technology as an integration risk, not a technology risk. In addition, the
program could assess a technology as mature simply because it is part of
another program. For example, it assesses the maturity of the technologies
enabling the Active Protection System as mature, even though the Army is
developing the system for a current combat vehicle that is much larger
than the FCS vehicles. The technologies will need to be reduced in size
before the system can be incorporated into the FCS vehicles. Overall, the
program must continue to mature its technologies while developing the FCS.
In some cases, as the FCS requirements are refined, the Army has decided
to use different technologies that are less mature than the original
technologies. For example, in February 2004, the program assessed the
maturity of ground-to-air combat identification as fully mature primarily
because similar identification systems were readily available in air
defense systems. In September 2004, however, it reduced the technology's
maturity because it refined the FCS requirements and determined that in
order to provide required interoperability with NATO systems, the program
would have to use an operating mode that required the development of a new
interrogator. As a result, it assessed the technology as very immature.
Design Stability
The program estimates that 80 percent of its 42,750 drawings will be
released by the design review scheduled for September 2010.
Other Program Issues
The FCS program began a major restructuring in July 2004, which delays
fielding an initial FCS capability until 2014, 48 months later than
planned. The revised strategy helps meet the needs of an Army at war by
making $9 billion available for investment in future capabilities for the
current force, which include FCS technologies that are expected to be
transitioned to the current force between 2008 and 2014. It also increases
the priority of development and demonstration of the FCS network and
system-of-systems architecture along with munitions, sensors, and unmanned
vehicles.
The concept of a modular FCS equipped brigade-sized combat unit, known as
a Unit of Action, represents a major departure in the way the Army has
conducted combat operations and is a major part of the Army's
transformation efforts. To successfully develop the FCS, the Army faces a
number of technological and programmatic challenges, including equipping
Units of Action with a common family of networked vehicles and other
systems. These vehicles and systems are expected to be a fraction of the
weight of existing heavy fighting vehicles in order to improve
transportability such as being airlifted by a C-130 transport.
Agency Comments
The Army provided technical comments, which were incorporated as
appropriate. In addition, it considers technical risk acceptable as long
as it can estimate that the technologies would be demonstrated in a
relevant environment before design review. The restructured FCS program
also includes a process for periodically spiraling out technologies to the
current force as they reach acceptable levels of maturity. Additional
efforts to mature these technologies will continue as needed under the
main program. The Army believes this approach will ensure that all
technologies are proven before fielding of full FCS-equipped Units of
Action. Finally, the Army noted that, in addressing transportability
challenges, the FCS program will continue to develop and analyze
alternative technical approaches to find the design solution that best
meets the broad spectrum of user needs.
GAO Comments
The Army is holding FCS technologies to a lower maturity standard than
best practices and DOD policy calls for. This increases the risk of
program cost growth and schedule delays.
Common Name: Global Hawk
The Air Force's Global Hawk system is a high altitude, long endurance
unmanned aerial vehicle with integrated sensors and ground stations
providing intelligence, surveillance, and reconnaissance capabilities.
After a successful technology demonstration, the system entered
development and limited production in March 2001. Considered a
transformational system, the program was restructured twice in 2002 to
acquire 7 air vehicles similar to the original demonstrators (the RQ-4A)
and 44 of a new, larger, and more capable model (the RQ-4B).
Source: Northrop Grumman Corporation.
Program start (2/94)
Development start/ low-rate decision (3/01) GAO review (1/05) Initial capability
(12/05) Full-rate decision (1/07) Last procurement (2011)
Key product knowledge on Global Hawk is now less than it was in March 2001
due to the 2002 program restructurings. Officials had planned to first
produce systems very similar to technology demonstrators and then slowly
develop and acquire more advanced systems. Technology maturity and design
stability approached best practice standards for this plan. However,
program restructurings accelerated deliveries, overlapped development and
production schedules, and added the new, larger air vehicle with advanced
sensors. These actions increased development and program unit costs. While
the platform design is fairly mature, production of the new air vehicle
began with advanced sensor technologies still immature and operational
tests not planned until much later. Production maturity cannot be assessed
using knowledge-based criteria because statistical process control data
are not used.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Development/ GAO
start design production
review
(NA) review decision (1/05)
(NA) (3/01)
Common Name: Global Hawk
Global Hawk Program
Technology Maturity
Five of 14 critical technologies associated with the Global Hawk system
are mature, 3 technologies are approaching maturity, and 6 are less
mature. Three of the mature technologies are uniquely associated with the
RQ-4A. Two of the 11 RQ-4B's critical technologies are mature-one more
than last year. The less mature technologies include the airborne signals
intelligence payload and the multiplatform radar technology insertion
program. These desired capabilities largely drove the decision to develop
and acquire the new RQ-4B air vehicles, which can carry 50 percent more
payload than the original model, the RQ-4A.
Production of the first RQ-4B began in July 2004. Integrating and testing
these advanced sensors on the air vehicle will not be completed until late
in the program when most of the fleet will already have been bought. There
is risk that the sensor technologies and final designs may not meet the
space, weight, and power limitations of the RQ-4B, resulting in extended
development times, costly reworks, or diminished capabilities. The
airborne signals intelligence payload currently exceeds its weight
allocation, and the power requirements for the multiplatform radar
requirements near the RQ-4B's limit.
Design Stability
The RQ-4A design is stable, and 75 percent of RQ-4B engineering drawings
were completed by the time of its design review in April 2004. By late
fiscal year 2004, over 90 percent of the engineering drawings were
completed. However, the Air Force has not built a prototype of the RQ-4B
to demonstrate a stable design and has not established a reliability
growth plan prior to initiating production-both characteristics of best
practices used to assure design maturity. Additionally, the Air Force
plans to buy almost half the fleet before it completes initial operational
test and evaluation to verify the air vehicle design works as required.
This increases the potential that testing may identify a need to redesign
and retrofit aircraft.
Production Maturity
Although production experience and lessons learned on the RQ-4A will
benefit the RQ-4B program, the new model requires different and more
complex manufacturing processes and tooling than the original model.
Officials have not implemented, and do not plan to implement, a
comprehensive statistical process control program to demonstrate that new
manufacturing processes are in control and capable of meeting cost,
schedule, and quality targets. Officials have started to identify critical
manufacturing processes and will continue to collect performance data such
as defect and rework rates to measure product quality. There are
continuing concerns about the quality and timeliness of several key
subcontractors, which negatively affect cost and schedule of both design
and production work. We note that the acceptance of the second production
RQ-4A was delayed due to defects and flight deficiencies.
Other Program Issues
Restructuring the Global Hawk program has accelerated planned deliveries
of advanced capabilities and made development, test, and production cycles
highly concurrent. Cost increases, schedule slips, and performance
trade-offs have already occurred. We recently reported that slowing down
production to enable closing the gaps in product knowledge and
operationally testing the aircraft should be considered.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that our
knowledge-based criteria do not effectively assess Global Hawk's
evolutionary acquisition strategy. It stated that Global Hawk's spiral
approach fosters efficiency, flexibility, and innovation and includes the
controls essential to manage program risk and achieve effective program
results. The Air Force further noted that the Global Hawk program is
managing development risks as it migrates from the RQ-4A to the larger,
multiple-intelligence RQ-4B configuration. It noted that the RQ-4B is an
evolutionary design change, built upon the successful RQ-4A design, years
of extensive testing, and over 5,000 RQ-4A flight hours, and also stated
that establishing accurate RQ-4B size, weight, and power constraints
provides accurate design requirements for development of advanced sensors,
further reducing future risk. The Air Force further commented that by
using concurrent development and production processes, the Global Hawk
program plans to achieve initial operational capability approximately 5
years after program initiation, fielding greater capability than initially
planned.
Common Name: GMD
MDA's GMD element is being developed in incremental, capability-based
blocks to defend the United States against limited long-range ballistic
missile attacks. The first block consists of a collection of radars and an
interceptor-a three-stage booster and an exoatmospheric kill vehicle
(EKV)-integrated by a central control system that formulates battle plans
and directs the operation of GMD components. We assessed all technologies
critical to the Block 2004 GMD element, but only the design and production
maturity of the interceptor.
Source: Department of Defense.
Program start (2/96)
Directive to field initial capability (12/02) Integrated design review (3/03)
Initial capability (8/04) GAO review (1/05) Block 2004 completion (12/05)
Three of GMD's 10 critical technologies were fully mature, and its design
seemed stable in September 2004 when MDA placed five ground-based
interceptors in silos for the initial capability. The remaining
technologies were nearing full maturity. However, there is a risk that
design changes could occur during Block 2004 because a solution to a
technical problem in the kill vehicle has not been proved in flight tests
and additional problems could be identified during the flight tests
scheduled to occur before the end of the block. Although MDA has not made
a formal production decision, it is currently producing hardware for
operational use. We could not, however, assess the stability of MDA's
production processes as the program is not collecting statistical data on
its production processes.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (NA) review
(1/05) (TBD)
(3/03)
Common Name: GMD
GMD Program
Technology Maturity
Program officials estimate that 3 of GMD's 10 critical technologies are
mature: fire control software, the EKV's infrared seeker, and the Orbital
Sciences Corporation (OSC) booster. The remaining seven technologies are
nearing maturity. These technologies are the Lockheed Martin BV+ booster;
Sea-based X-Band radar; Cobra Dane radar; Beale radar; EKV on-board
discrimination; EKV guidance, navigation, and control subsystem; and the
in-flight interceptor communications system. The program expected to
demonstrate 3 of these technologies by the end of fiscal year 2004, but
flight test delays prevented the demonstrations. However, program
officials expect that the maturity of all 7 technologies will be
demonstrated before the end of Block 2004.
Design Stability
The Block 2004 ground-based interceptor design is stable with 100 percent
of its drawings released to manufacturing. The ongoing effort to mature
critical technologies and solve an ongoing engineering problem, however,
may lead to more design changes.
Production Maturity
Officials have not made an official production decision, although they are
delivering interceptors for the Block 2004 emergency capability. We could
not assess the production maturity of these interceptors because the
program is not collecting statistical control data on the production
process. According to program officials, data are not tracked because the
current quantities of GMD component hardware are small. Instead, the GMD
element measures production capability and maturity with a monthly
evaluation process that assesses critical manufacturing indicators for
both readiness and execution.
To reduce program risk, MDA is following a dual booster strategy,
developing the BV+ and the OSC boosters, each of which has a different
design. Although this strategy offers two different capabilities and has
helped to mitigate production risks, MDA has experienced ongoing problems
with the BV+ booster. After an explosion at the facility that mixes
propellant for the BV+ booster motors, the facility's contractor ceased
operations.
A new contract has been awarded for the production of the BV+ 2nd and 3rd
stage motors. MDA hopes to restart manufacturing in fiscal year 2005.
Therefore, all Block 2004 interceptors will use the OSC booster.
EKV and booster delivery is on schedule for the December 2005 initial
capability. MDA delivered 5 interceptors for initial defensive operations
by September 2004, and it plans to have a total of 18 on alert by December
2005. MDA originally planned to have 20 interceptors by this time;
however, two of these interceptors were later designated as test assets.
Other Program Issues
Increased cost of the EKV and the explosions at the BV+ propellant-mixing
facility were leading causes of $175 million in GMD cost growth during
fiscal year 2004. To avoid a delay in fielding the initial defensive
operation on September 30, 2004, MDA funded the cost overrun by having
other groups within MDA perform some tasks that GMD was budgeted to
complete.
Agency Comments
In commenting on a draft of this assessment, MDA stated that a formal
production decision is not anticipated or planned in the GMD acquisition
approach. It emphasized that it is not feasible to collect data on most
GMD production processes due to the extremely low quantities of system
hardware being procured, but statistical data are collected and available
on those subsystems/parts produced in sufficient volume. It also pointed
out that ongoing efforts to mature critical technologies and solve
technical problems are an inherent part of the capability-based
acquisition/block development approach and that design changes are to be
expected as the system is evolved through subsequent blocks. Technical
comments were also provided and incorporated as appropriate.
Common Name: GPS Block IIModernization
GPS is an Air Force-led joint program with the Army, Navy, Department of
Transportation, National Geo-Spatial Intelligence Agency, United Kingdom,
and Australia. This space-based radio-positioning system nominally
consists of a 24-satellite constellation providing navigation and timing
data to military and civilian users worldwide. In 2000, Congress approved
the modernization of Block IIR and Block IIF satellites. In addition to
satellites, GPS includes a control system and receiver units. We focused
our review on the Block IIF.
Source: Navstar GPS Joint Program Office, Space and Missile Systems
Center.
Program start (1/99)
Development
start
(2/00)
Production decision (6/02) GAO review (1/05) First satellite launch (12/06)
Initial capability (NA)
According to the program office, the Block IIF technologies are mature.
Since the start of the GPS program in 1973, GPS satellites have been
modernized in blocks with the newer blocks providing additional
capabilities and benefits. The GPS II modernization effort required new
technology for the atomic clocks on the IIF satellites, and this
technology has been tested in space on IIR satellites. However, the
contractor was not required to provide data on design drawings and
statistical process control techniques are not being used to monitor
production. As a result, design stability and production maturity could
not be assessed.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (2/00) review
(6/02) (1/05) (NA)
Desired level of knowledge
Common Name: GPS Block II Modernization
GPS Block II Modernization Program
Technology Maturity
The only new critical technology on the Block IIF satellites, the
space-qualified atomic frequency standards, was tested in space on Block
IIR satellites, and it is considered mature.
Design Stability
We could not assess design stability because the Block IIF contract does
not require that design drawings be delivered to DOD. However, the program
office assesses design maturity by reviewing contractor development
testing, participating in technical interchange meetings and periodic
program reviews, and conducting contractor development process and
configuration audits.
Production Maturity
We could not assess production maturity because the contractor does not
collect statistical process control data. However, the program office
reviews earned valuemanagementreports, integrated master schedules, and
test dates as a means of monitoring the contractors' production efforts.
Other Program Issues
The current Block IIF contract calls for the procurement of 12 satellites.
The Air Force estimated that this number would be sufficient for
constellation sustainment until the launch of the first GPS III satellite,
scheduled for 2010. However, in fiscal year 2003, the Air Force
restructured the GPS III launch schedule and delayed the first launch to
2012. Consequently, four additional satellites will need to be acquired to
sustain the GPS constellation due to this delay. To build these additional
satellites, several subsystems would require parts that are no longer
available and must be newly manufactured. Additional funding has been
requested for fiscal years 2005 and 2006 to pay for the nonrecurring
engineering required to manufacture these parts for the additional Block
IIF satellites.
The GPS Operational Control System consists of monitor stations that
passively track the navigation signals of all the satellites and a master
control station that updates the satellites' navigation messages. Certain
components of the control system have been delayed because funds from this
development were reallocated to complete the Block IIF development in
support of constellation sustainment. Specifically, M-Code and Flex Power
capabilities, part of the control system, will be delayed 3 years, but
according to the program office, this will not result in underutilization
of the satellites on orbit.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that the
GPS constellation first achieved final operational capability of 24
healthy and operational satellites in July 1995 and since then has
consistently exceeded this requirement. It also stated that beginning in
2000, the joint program office initiated a modernization and upgrade
program to more rapidly introduce new capabilities for the warfighter and
civil users. It further stated that, as of December 2004, the joint
program office's current estimate for launch availability of the first
modernized satellite (IIR-M) will be April 2005 and that the Block IIF
will continue the modernization program with its first satellite launch
availability in September 2006.
Common Name: HLR
The Marine Corps' HLR system will perform the marine expeditionary
heavy-lift assault transport of armored vehicles, equipment, and personnel
to support distributed operations deep inland from a sea-based center of
operations. The HLR program is expected to replace the current CH-53
helicopter with a new design to improve range and payload, survivability,
reliability and maintainability, coordination with other assets, and
overall cost of ownership.
Source: PMA-261 Program Office.
Program start (11/03)
GAO review (1/05)
Development
start
(2/05)
Design review (9/08)
Low-rate decision (1/13) Initial capability (8/15) Full-rate decision (1/16)
Last procurement (2022)
The critical technologies for the HLR program are not expected to be fully
mature before the start of development in February 2005. An initial
readiness assessment for the program identified 10 critical technologies.
A subsequent assessment reduced that number to 3-the main rotor blades,
the main rotor viscoelastic lag damper, and the main gearbox. Elements of
the 7 eliminated technology areas, including the engines, may still
present challenges to the program. The gearbox and the rotor blades are
not expected to reach full maturity until 2011 and 2012, respectfully.
Currently, an aggressive acquisition strategy is being planned.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production review start design decision (1/05) (2/05)
review (1/13)
(9/08)
Common Name: HLR
HLR Program
Technology Maturity
The three critical technologies for the HLR program-the main rotor blades,
the main rotor viscoelastic lag damper, and the main gearbox-are not
expected to be fully mature before the start of development in February
2005. A lag damper similar to that planned for use is currently in
operation on another program, but it must be resized for use on the HLR
and therefore will not reach full maturity until the critical design
review in 2008. The gearbox and the rotor blades represent new technology
areas that have only been demonstrated in a low fidelity laboratory
environment and are not expected to reach full maturity until 2011 and
2012, respectively.
Other development items may present future challenges to the HLR program.
While 10 critical technologies were originally identified for the program,
an assessment conducted in September 2004 reduced those to the 3 above. Of
the 7 technologies eliminated, 2 are being developed by the HLR program
and 5 are being developed by or used on other programs and will then have
to be integrated onto the HLR platform. In either case, this integration
can represent potential risks to cost and schedule. For example, the
program is still considering five different engine design options. While
the Navy has determined that none of the engine designs are expected to
use new or novel technology or represent a new relevant environment for
use, each requires different levels of design change, developmental risk,
and qualification. For two other technologies, less desirable backup
systems will have to be used if the technologies are not developed as
planned.
Other Program Issues
In September 2003, the Navy evaluated seven existing aircraft platforms
and determined that only the CH-53E (with substantial enhancements) was
capable of meeting requirements for performance, inventory, operational
capability dates, operating and support costs, and survivability. Previous
assessments concluded that the CH-53 airframe was experiencing substantial
fatigue due to age and lack of regular upgrades and modifications. Program
officials told us that this situation is even worse now due to increased
operational use in Afghanistan and Iraq. The 2003 analysis evaluated four
alternative CH-53E designs and recommended one of these to meet range and
payload requirements and minimize effects to service capability dates,
inventory, support costs, and risk. However, after refining operational
requirements for the HLR, the Navy selected a different alternative that
offered additional performance and reliability improvements but added
additional schedule and technical risk. To address these challenges, the
Navy expects to implement an aggressive acquisition strategy for the HLR
program, including sole-source contracting to Sikorsky Aircraft
Corporation and a single-step acquisition approach. The program also
intends to manufacture 50 of the 154 total helicopters (32 percent) during
low-rate initial production and concurrent with initial operational
testing. This concurrent production may help to field the systems sooner,
but it could also result in greater retrofit costs if unexpected design
changes are required.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the HLR
program was developed to replace the aged CH-53E and support Marine Corps
Sea Basing and other 21st Century joint operations. It added that the
program balances operational and programmatic risks and that delays to the
current HLR planned schedule will result in significant additional
procurement and operation and support costs to support the CH-53E legacy
aircraft and Marine Corps Heavy Lift shortfalls. The Navy noted that the
Office of Naval Research endorsed the HLR program initiation at Milestone
B and that the approved HLR Technology Readiness Assessment and maturation
plan include the application of engineering trade and risk reduction prior
to program initiation at Milestone B. It also noted critical technology
item maturation events coincide with key system development events such as
critical design review and prototype production. As the HLR program
matures, risk reduction will continue to be abetted through sustained
selection of nondevelopmental technologies, with an emphasis on employment
of mature technologies common to Marine, Navy, and DOD weapon systems.
Common Name: JASSM
The JASSM is a joint Air Force and Navy missile system designed to provide
a new capability to attack surface targets outside of the range of area
defenses. The JASSM will be delivered by a variety of aircraft including
the F-16 C/D, the B-52H, the F/A-18E/F, the B-2, and the B-1B. The system
includes the missile, software, and software interfaces with the host
aircraft and mission planning system. We assessed all components.
Source: Joint Air-to-Surface Standoff Missile (JASSM)Program Office
(Development Test Mission - DT-5).
Program start (6/96)
Development
start
(11/98)
Low-rate decision (12/01) Initial capability (9/03) Full-rate decision (7/04)
GAO review (1/05) Last procurement (2015)
The JASSM program entered production in December 2001 without ensuring
that production processes were in control. However, program officials
indicated that they have demonstrated the production processes by sampling
statistical data at the subsystem level and that four missiles are
selected from each production lot and tested for quality. The JASSM
program used mature technology, and the missile design was stable at the
design review. Although there were some test failures in the developmental
and operational tests run from April 2002 to September 2003, program
officials incorporated fixes that subsequent tests demonstrated to be
successful. However, in recent follow-on tests, the program continued to
have test failures, and the Air Force suspended testing until the causes
of these failures can be determined. Nevertheless, the JASSM was approved
for full-rate production in July 2004.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (11/98) review
(12/01) (1/05) (9/01)
Desired level of knowledge
Common Name: JASSM
JASSM Program
Technology Maturity
The JASSM program identified three critical technologies-global
positioning system antispoofing receiver module, low observable
technology, and composite materials-and stated that all three are mature.
They are new applications of existing technologies.
Design Stability
The contractor has released 100 percent of the drawings to manufacturing.
The program office completed developmental and operational tests and
entered follow-on test and evaluation. Fourteen developmental flight tests
were performed, with three tests failing to meet the test objectives.
Program officials identified the issues involved and incorporated fixes,
which were successfully tested in later developmental tests. Fifteen
operational tests were conducted from June 2002 to September 2003.
According to the Air Force Operational Test and Evaluation Command, 7 of
these were successful, 5 were failures, and 3 were "no test." Based on the
developmental and operational tests, the Command considered the JASSM to
be capable against the required targets but not reliable. Therefore, it
rated the missile as effective and potentially suitable and recommended
approval of full-rate production. Since that time, in follow-on test and
evaluation, the missile had three successful tests and three failures. The
Air Force halted further testing and convened a failure review board to
determine the causes for the test problems. This board was to report its
findings in October 2004.
Production Maturity
Program officials do not collect production process control data at the
system level. However, they stated that all production processes had been
demonstrated and that statistical data are collected at the subsystem
level and are sampled as required. Program officials indicated that the
contractor has produced at the rates required for the low-rate initial
production buy of 176 missiles and that it will be able to produce at the
full-rate production level of 250 missiles per year. Three production lots
are on contract and deliveries are on schedule. Program officials believe
that none of the manufacturing processes that affect critical system
characteristics are a problem, although there are key production processes
that have cost implications, such as bonding for the low observable
materials and the painting/coating application. The missile was approved
for full-rate production in July 2004.
Other Program Issues
A contract for development of an extended range version of the missile was
awarded in February 2004.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that as
a result of two test failures this summer, the Air Force Program Executive
Office for Weapons convened a Reliability Enhancement Team on August 16,
2004, to investigate ways to improve reliability of the JASSM. It further
stated that the team completed its work in October and concluded the JASSM
design was sound, concurred with the joint program office return to test
plan, and recommended award of the next lot's production contract-awarded
November 2004. Also, the team recommended the Joint Program
Office/Lockheed Martin pursue a more focused effort on subtier supplier
manufacturing process quality controls and implement a robust test program
to improve missile reliability. The Air Force stated that the key
stakeholders (Air Force, Office of the Secretary of Defense, and Congress)
concurred with the team's recommendations and the joint program office's
way ahead plan and noted that the JASSM team continues to address
near-term reliability issues identified by the Reliability Enhancement
Team.
Common Name: Joint Common Missile
The Joint Common Missile is a joint Army/Navy program with Marine Corps
participation and United Kingdom involvement. It is an air-launched and
potentially ground-launched missile designed to target tanks; light
armored vehicles; missile launchers; command, control, and communications
vehicles; bunkers; and buildings. It is to provide lineof-sight and beyond
line-of-sight capabilities and can be employed in a fire-and-forget mode
or a precision attack mode. The missile will replace systems such as
Hellfire and Maverick.
Source: Joint Common Missiles Program Office.
Program start (10/01)
Development
start
(4/04)
GAO review (1/05)
Design review (3/06)
Low-rate decision (5/08) Intitial capability (9/09) Full-rate decision (10/10)
The Joint Common Missile entered system development of the air-launched
version in April 2004, before any of its critical technologies were fully
mature. At this time, program officials do not know the number of drawings
that will be released by design review in March 2006. Program officials
currently project that the critical technologies will reach maturity 3
months prior to design review, about half way through product development.
Until all technologies are demonstrated, the potential for design change
remains. Mature backup technologies are available should the new
technologies fail to mature; however, use of backup technologies could
degrade system performance or increase costs. By beginning integration
before these technologies have been demonstrated, the potential for cost
growth, schedule delay, or decreased performance exists.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/04) (1/05)
review (5/08) (3/06)
Common Name: Joint Common Missile
Joint Common Missile Program
Technology Maturity
None of the Joint Common Missile's three critical technologies have
demonstrated full maturity according to best practices. These technologies
include a multimode seeker for increased countermeasure resistance,
boost-sustain propulsion for increased standoff range, and a multipurpose
warhead for increased lethality. Program officials noted that many of the
components of these technologies are currently in production on other
missile systems, but they have not been fully integrated into a single
missile. Maturing technologies concurrently with product development
increases the potential for cost growth and schedule delays. According to
program officials, while backup technologies exist for each of the
critical technologies, substituting any of them would result in degraded
performance or increased costs.
Design Stability
Currently, about 16 percent of the drawings for the Joint Common Missile
have been released to manufacturing. Program officials project that
approximately 41 percent of the drawings will be released by May 2005, the
end of what they term a risk mitigation phase. However, program officials
have not projected the number of drawings that will be released by design
review in March 2006. Officials project full integration of the subsystems
into the Joint Common Missile will occur by April 2005, although the
system will reach technology maturity by December 2005, over a year and a
half after the start of system development.
Program officials stated that the program's modular design will reduce
life-cycle costs, including demilitarization, and will enable continuous
technology insertion to provide improved capability against advancing
threats.
Agency Comments
In commenting on a draft of this assessment, the program office stated
that during the first and second quarters of fiscal year 2004, a
comprehensive Technology Maturity and Readiness Assessment, along with a
risk assessment, was performed by subject matter experts from the Aviation
and Missile Research and Engineering Center and the Army Test and
Evaluation Command and coordinated with respective offices within the Army
and the Navy.
This assessment was reviewed by the Department of the Army, the Office of
the Secretary of Defense, and the Director of Defense Research and
Engineering and concluded that the Joint Common Missile technology was at
an appropriate maturity level to support entry into System Design and
Development. Further, it is anticipated that progress will continue. The
system technologies combined with control test vehicle firing(s) will
substantiate maturity according to best practices by April 2005.
Common Name: JSF
The JSF program goals are to develop and field a family of stealthy,
strike fighter aircraft for the Navy, Air Force, Marine Corps, and U.S.
allies, with maximum commonality to minimize costs. The carrier suitable
version will complement the Navy's F/A-18 E/F. The conventional take-off
and landing version will primarily be an air-to-ground replacement for the
Air Force's F-16 and A-10 aircraft, and will complement the F/A-22. The
short take-off and vertical landing version will replace the Marine Corps'
F/A-18 and AV-8B aircraft.
Source: JSF Program Office.
Program start (11/96)
Development
start
(10/01)
GAO review (1/05) Low-rate decision (1/07)
Initial capability
USMC
(3/12)
Initial capability
USAF
(3/13)
Initial capability
USN
(3/13)
Last procurement (2027)
The JSF entered system development in 2001 with its critical technologies
immature, and recent assessments indicate that this is still the case.
Other risks exist as well. For example, the preliminary design review
revealed a significant weight problem that led to numerous design and
requirement changes. This resulted in delays of 16-22 months for the
design reviews and increased costs. The program expects 35 percent of its
drawing packages to be completed by the design reviews. Also, the program
expects to produce a significant number of production aircraft with little
demonstrated knowledge about performance, reliability, software maturity,
and producibility. In 2004, the program reported a Nunn-McCurdy (10 U.S.C.
2433) unit cost breach largely due to design maturation efforts, schedule
extensions, and revised labor and overhead rates.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (10/01) (1/05)
review (1/07) (2/06)
Common Name: JSF
JSF Program
Technology Maturity
The JSF entered system development without demonstrating the maturity of
its 8 critical technologies. Data provided by the program office indicate
that the technology maturity has not significantly changed. In 2004, an
independent review team examined the program and identified several
technical challenges related to the critical technologies. For example, it
found that the highly integrated subsystems still have risk and that major
challenges remain with the mission systems and software integration. The
team reported that prognostics and health management technologies needed a
focused initiative to mature them.
Design Stability
When development began, the design was not well defined, leading to
changes in requirements and design. The preliminary design review held in
March 2003 revealed significant airframe weight problems-eventually
exceeding targets by as much as 25 percent-that affected the aircraft's
ability to meet key performance requirements. Actions to resolve the
problem have added 18 months and $4.9 billion to the development program.
Program officials indicated that no drawings have been completed for any
production representative variant. Critical design reviews are scheduled
for the 2006 time frame, a 16- to 22-month delay. At the time of the
design reviews, the program expects to have released about 85 percent of
the critical structural drawings but only 35 percent of the total
engineering drawing packages needed to build the aircraft. This relatively
low level of design knowledge will continue beyond the production decision
in 2007. At the time of that commitment, the JSF will (1) have done
limited flight testing on only one nonproduction representative aircraft;
(2) not have flight-tested an integrated aircraft (with critical mission
systems and prognostics technologies); (3) have less than 40 percent of
the software lines of code needed for expected system functionality
released. By 2013, when development is scheduled to be complete, DOD plans
to have bought around 500 low-rate production aircraft at an estimated
cost over $50 billion. This highly concurrent strategy of producing and
developing aircraft increases the risks of cost growth and delays in
delivering capability to the warfighter.
Production Maturity
The program office is collecting information on the JSF production
processes. The contractor is currently in the process of identifying the
key characteristics, critical manufacturing processes and capturing some
early data. At the time of the production decision, the program will not
have demonstrated that the aircraft can be produced efficiently or with
expected reliability. These uncertainties are major contributors for DOD
plans to rely on cost reimbursable type contracts for the early production
buys. Fixed price contracts, the norm for production, are not expected
until the air vehicle has a mature design, has been demonstrated in flight
tests, and is producible at established cost targets.
Other Program Issues
In 2004, the program reported a Nunn-McCurdy (10 U.S.C. 2433) program unit
cost breach. According to the program office, total program unit costs
have increased by 19.4 percent largely due to aircraft design maturation
efforts, schedule extensions, and revised labor and overhead rates.
Agency Comments
In commenting on a draft of this assessment, the Air Force provided the
following information. A 2001 DOD review concluded the JSF had
demonstrated sufficient technical maturity for entry into development.
Design reviews were completed March 2004 on all areas except the airframe.
By the airframe design review, 85 percent of the critical structural
drawings will be complete. Subsystem hardware/software integration in the
lab is ahead of schedule, occurring sooner than legacy fighter programs.
Significant progress has been made in weight and performance issues. The
short take-off and vertical landing variant includes over 2,700 pounds of
weight reductions achieved through design optimization. More weight
improvements were achieved by modest requirement changes endorsed by the
warfighters. Requirements for other variants were not changed. Manufacture
of the first test aircraft is underway, with assembly times less than
planned. Over 1,500 test hours have been achieved on seven engines. Some
replan refinements are in work. Program concurrency reflects spiral
development strategy.
Common Name: JSOWUnitary
The JSOW is a joint Air Force and Navy guided bomb to attack targets from
outside the range of most enemy air defenses. A dispenser variant (JSOW A)
carries submunitions to attack soft targets. In 2002, the Joint
Requirements Oversight Council deferred production of an antiarmor JSOW
variant (JSOW B). The unitary variant (JSOW C) uses a seeker, autonomous
targeting acquisition software, and a single warhead to attack targets.
All the variants use a common air vehicle. We assessed the unitary variant
and the common air vehicle.
Source: Raytheon Systems Company.
Program/development
start
(4/95)
Low-rate decision (6/03) Full-rate decision (11/04) GAO review (1/05) Initial
capability (1/05) Last procurement (2014)
The JSOW program began low-rate production in June 2003 without knowing
whether production processes were in control. However, the contractor has
since identified seven critical production processes and has five of the
seven under statistical process control and performing at an acceptable
quality level. The contractor is working with the remaining two processes
to collect enough data to verify that the processes are under control.
Operational evaluation was completed in September 2004, and the beyond
low-rate production and live fire test reports required to support the
full-rate production decision were received in December 2004.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (4/95) review
(6/03) (1/05) (5/02)
Common Name: JSOW Unitary
JSOW Unitary Program
Technology Maturity
The JSOW Unitary variant's technology is mature. The program office
identified the imaging infrared seeker with the autonomous acquisition
software as the only critical technology for the system. The seeker was
not mature at the start of development, but it did demonstrate maturity in
October 2001-about three-fourths through development-when it was flown
aboard an aircraft in a captive flight test. Program officials stated that
in seven developmental tests, three free-flight tests with the seeker only
and four combined seeker/warhead tests, the seeker's performance
substantially exceeded requirements. The seeker has demonstrated greater
accuracy than required during operational testing.
Design Stability
The JSOW unitary variant's basic design is complete. At the system design
review in May 2002, the program office had completed 99 percent of the
drawings. The Navy has completed 10 developmental tests (adding one
combined seeker/warhead test in 2003) in its development program-3 sled
tests with the warhead, 3 free-flights with the seeker, and 4 combined
warhead/seeker tests. After some delay in beginning operational tests due
to problems with the fuze, the Navy completed operational testing in
September 2004 and reported that the fuze reliability met requirements.
Production Maturity
Raytheon and the Navy identified seven critical processes unique to seeker
development and collected data during low-rate production to determine
that five of the seven were in control. Raytheon is working to collect
data sufficient to characterize the remaining two processes. The Navy
reports that delivery of the seekers is ahead of schedule and that there
is low risk to meeting the quantity requirements of 17 per month. Raytheon
has maintained its on-time deliveries for the common air vehicle for more
than 33 months.
Other Program Issues
The JSOW completed operational testing in September 2004. Preliminary
analysis of the data indicated that the missile, its seeker, and warhead
met performance requirements. The final report rated the weapon as
operationally effective but noted some deficiencies in training affecting
the rating for suitability. According to a program office official, the
issues have been resolved and the revised assessment rates the weapon as
operationally effective and suitable. Reports detailing the analysis of
the testing and the weapon's operational suitability and effectiveness and
its live fire test results were received in December 2004.
Agency Comments
The Navy provided technical comments to a draft of this assessment, which
were incorporated where appropriate.
Common Name: JTRS Cluster 1
The JTRS program is developing software-defined radios that will
interoperate with existing radios and significantly increase
communications capabilities. A joint service program office is responsible
for developing the JTRS architecture and waveforms, while service-led
program offices will develop and procure radio hardware for platforms with
similar requirements. This is an assessment of Cluster 1, led by the Army,
which is developing radios for ground vehicles and helicopters.
Source: PM WIN-T JTRS Cluster 1.
JTRS program start (9/97)
Development
start
(6/02)
Design review (12/03)
GAO review (1/05) Low-rate decision (4/06) Full-rate decision (6/07) Last
procurement (unknown)
The JTRS program's demonstrated knowledge continues to be difficult to
characterize. Program officials believe that the design is stable and
production processes are in control. However, design and production
knowledge are dependent on technology maturity. None of the program's 20
critical technologies are mature, and the number of drawings has nearly
tripled since last year. The program is proceeding under an accelerated
strategy that does not allow for testing the radio's full functionality
before initial low-rate production begins. Requirements changes are being
considered that could result in design changes. The Army is proposing to
restructure the program, which may add time to the development schedule.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (6/02) review
(1/05) (4/06) (12/03)
Common Name: JTRS Cluster 1
JTRS Cluster 1 Program
Technology Maturity
While the program office has made some progress in maturing critical
technologies, none of the JTRS Cluster 1 program's 20 critical
technologies are mature. Many of these critical technologies have been
used in other radio applications but cannot be assessed as mature because
they have not been integrated into a complex radio like Cluster 1. Mature
backup technologies exist for some critical technologies, but program
officials have cautioned that substituting them would complicate
integration or result in degraded performance. Program officials pointed
out several challenges in achieving technological maturity. In particular,
the program continues to reconcile size, weight, and power requirements.
Meeting the performance objectives of the Wideband Networking Waveform is
also a challenge. Program officials expect to demonstrate maturity of all
20 critical technologies during an early operational assessment scheduled
to end in April 2005.
Design Stability
The program reports achieving design stability for the basic Cluster 1
radio design. However, while all drawings have been released to
manufacturing, the total number of drawings has nearly tripled from last
year's assessment. Program officials primarily attribute the large
increase to additional drawings required for certain components as the
design matured and more specificity of the initial component drawings.
Furthermore, program officials report that the number of drawings is
likely to change again as a result of the upcoming operational assessment
and as they move toward production. Given that the critical technologies
have yet to mature, the significant changes to the number of drawings
raise concerns about the program's design stability.
Production Maturity
The program reports that all production processes to be utilized in
manufacturing the JTRS radios are mature and in control. However, as the
program office expected, the number of processes has decreased from last
year's assessment. According to the program office, the number has
decreased because of design enhancements. The program office expects the
number of processes to change again as further design requirements take
place.
Other Program Issues
The program has a software development plan with insufficient schedule
reserve to incorporate knowledge gained from initial development
increments. It also has a compressed test and evaluation phase that leaves
little room for rework. For example, the production decision is scheduled
to occur immediately upon completion of an early operational assessment
limited to pre-engineering development models that are not fully
functional. The program office also reported an increase in procurement
costs of over $600 million primarily due to an error in estimating
manufacturing costs. The JTRS Cluster 1 information security certification
approach is also unprecedented, and the radios must go though a
certification process that is outside the program office's control. In
addition, the joint program office is exploring additional requirements
including the development of additional waveforms that operate at above
2GHz-that may be tasked to the JTRS Cluster 1 program and may also
necessitate hardware modifications. Because of emerging requirements and
other technical challenges, the Army is considering restructuring the
program, which may add more time to the development schedule.
Agency Comments
In commenting on a draft of this assessment, the program office generally
agreed with the information provided in this report. Program officials
also provided technical comments, which were incorporated where
appropriate.
Common Name: JTRS Cluster 5
The JTRS program is developing software-defined radios that will
interoperate with existing radios and also increase communications and
networking capabilities. A joint service program office is developing the
architecture and waveforms, while service-led program offices are
developing radio hardware. The Army-led JTRS Cluster 5 is developing
handheld, manpack, and small embedded radios for applications such as
ground sensors. Spiral 1 will field a two-channel manpack. Spiral 2 will
develop and field all versions. We assessed Spiral 2.
Source: PM WIN-T JTRS Cluster 5.
Program/development
start
(4/04)
GAO review (1/05)
Spiral 2 low-rate
decision
(3/08)
Initial operational capability spiral 2 (5/11)
JTRS Cluster 5 began system development with one of its six critical
technologies mature for Spiral 2. The program considers the five other
technologies low risk and anticipates increased levels of maturity, though
not full maturity, by the production decision in March 2008. We did not
assess design stability because no production representative drawings had
been released at the time of our assessment for either Spiral 1 or Spiral
2. The total number of drawings has also not been identified.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (4/04) (1/05)
review (3/08) (4/06)
Common Name: JTRS Cluster 5
JTRS Cluster 5 Program
Technology Maturity
The JTRS Cluster 5 program has identified six critical
technologies-identical for both of the Cluster 5 spirals. Spiral 1 is
based on technologies that are either commercial-off-the-shelf or
nondevelopmental items, and is focused on a two-channel manpack with
narrowband capability operating seven of the designated JTRS waveforms.
Spiral 2 is to evolve and expand Spiral 1 two-channel manpack capabilities
as well as fully developing the one- and two-channel handheld and small
form fit variants meeting the wideband and networking requirements.
The program office has assessed one of Cluster 5 critical technologies,
termed environmental protection, as mature for use in Spiral 2. It has
also assessed two other critical technologies, antenna and power
management, at a high level of readiness, although not fully mature.
However, the power management technology may not be as mature as assessed
given the Cluster 5 requirement to support a JTRS Wideband Networking
Waveform. This waveform is essential to providing JTRS networking services
to ensure interoperability over a wide range of frequencies. While it is
not designated a Cluster 5 critical technology, the JTRS Operational
Requirements Document designates it as a key performance parameter.
Operation of this waveform carries with it a large power requirement.
Because of that power requirement and the technical challenges of meeting
that requirement in an acceptable size and weight, the Cluster 5 program
is seeking some relief from the waveform's requirements, and attempting to
optimize the software code to increase its power efficiency. It is also
evaluating alternative waveforms such as the Soldier Radio Waveform to
provide in a power efficient way the needed networked services for radios
with limited power and antenna size.
The remaining Cluster 5 critical technologies- antennas, microelectronics,
multichannel architecture, and security-require additional development.
According to the program office, however, all four represent a low level
of risk and are anticipated to reach increased levels of maturity by the
production decision.
Additionally, the program continues to address size, weight, and power
requirements. The Cluster 5 manpack radios to be fielded in Spiral 2 are
to have a maximum weight of 9 pounds. In comparison, Spiral 1 units weigh
up to 13 pounds. With the help of the Army's Communications-Electronics
Research, Development and Engineering Center, the program is pursuing
power trade-offs and technical solutions to achieve the Spiral 2
requirement.
Design Stability
We did not assess the design stability of JTRS Cluster 5 because the total
number of drawings is not known and there are currently no releasable
drawings complete for either spiral.
Other Program Issues
An Acquisition Decision Memorandum in May 2004 authorized the movement of
the single channel handheld radios requirement from Spiral 1 to Spiral 2.
The memorandum also expressed concern about the immaturity of the Spiral 2
definition and required the program to update the cost and affordability
assessment during the second quarter of fiscal year 2006. Furthermore, in
recognition of the criticality of JTRS, it directed the Cluster 5 program
to conduct a review in the first quarter of fiscal year 2005 to assess the
maturity of the plans for Spiral 2. The JTRS Cluster 5 development
contract was awarded in July 2004. However, immediately thereafter, the
contractor was issued a stop-work order because of a bid protest. Work was
stopped until late October 2004, when we denied the protest and work
resumed. Impact of the stop-work order is still being assessed by the
Cluster 5 product manager.
Agency Comments
In commenting on a draft of this assessment, the program office provided
some technical comments and suggested a number of editorial changes
including additional clarifying information, which we incorporated as
appropriate. The program office indicated the critical technologies will
reach an acceptable level of maturity by the production decision in 2008.
GAO Comments
While the program office commented that the critical technologies will
reach an acceptable level of maturity by the time of the production
decision, best practices call for attaining a higher level of maturity by
the start of development.
Common Name: J-UCAS
The J-UCAS program is a combined effort of the Defense Advanced Research
Projects Agency (DARPA), the Air Force, and the Navy to demonstrate the
technical feasibility and operational value of a networked system of high
performance and weaponized unmanned air vehicles. Expected missions
include the suppression of enemy air defenses, electronic attack,
precision strike, and surveillance. The program consolidates two formerly
separate service projects and is to develop larger, more capable, and
interoperable aircraft.
Source: (left) X-45C -The Boeing Company, X-47B - Northrop Grumman
Corporation, (right) X-45C (c) 2004 The Boeing Company, X-47B (c) 2004
Northrop Grumman Corporation.
Program start (10/03)
GAO review (1/05)
Development
start
(FY10)
The J-UCAS program began in October 2003 with technologies that officials
project will sufficiently mature to support a possible 2010 start of
operational system development. The program plans to develop and
demonstrate the next generations of the original Air Force and Navy
demonstrators that will have common performance objectives and utilize
common subsystems and technologies. The program expects to conduct an
early operational assessment starting in fiscal year 2007 and then provide
the Air Force and the Navy with several program options for follow-on
efforts. A December 2004 program budget decision would restructure the
program and reduce funding. At the time of our review, it was not clear
how these changes will impact the schedule for achieving key product
knowledge.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production review start design decision (1/05) (4/10)
review (TBD)
(TBD)
Desired level of knowledge
Projection
Common Name: J-UCAS
J-UCAS Program
Technology Maturity
While none of the J-UCAS' six critical technologies are currently mature,
program officials project that they will be sufficiently ready to support
the early operational assessment scheduled to begin in fiscal year 2007
and to provide options to the Air Force and the Navy for follow-on efforts
starting in fiscal year 2010. Program officials identified the following
critical technologies needed to produce a high performance and networked
system of low observable air vehicles capable of operating in highthreat
environments for extended periods of time: (1) signature reduction; (2)
advanced tactical targeting; (3) secure robust communications; (4) force
integration, interoperability, and global information grid compatibility;
(5) adaptive autonomous operations; and (6) operations in aircraft
carrier-controlled airspace. These technologies are still maturing as
would be expected at this early presystem development stage. The targeting
and autonomous operations technologies are considered the most mature and
carrier operations technology the least mature.
Other Program Issues
The previous service-specific efforts combined in the joint program had
different primary missions and operating environments. The Air Force began
developing its system to suppress and attack enemy air defenses, while the
Navy's primary interest was for a carrier-based unmanned aerial vehicle to
provide persistent armed surveillance for the battle group. The joint
program is expected to maintain a competitive environment and continue to
develop next-generation versions of both Air Force and Navy demonstrators.
Both versions will be expected to be capable of performing all required
missions of the two services. By merging the Air Force and Navy efforts,
DOD hopes for synergy and cost savings by developing interoperable and
networked systems utilizing common operating systems, sensors, and
weapons.
The program cost of over $4 billion from startup in fiscal year 2004
through fiscal year 2009 does not include the approximately $500 million
spent on the two service-specific projects prior to consolidation. The
program will compete for funding with current operational systems such as
the Predator and the Global Hawk and other unmanned and manned systems in
varying stages of development, some with similar missions. Congress
reduced J-UCAS funding in fiscal year 2005 because the program had not
properly coordinated with the two services and directed that the
technology demonstrators be completed in support of Air Force and Navy
requirements.
A December 2004 program budget decision by DOD restructured J-UCAS by
realigning adjusted resources to the Air Force to establish a joint
program with Navy representation. It reduced total funding by about $1.1
billion from fiscal year 2006 through fiscal year 2011.
Emerging challenges include adaptation for carrier operations and
development of the common operating system. The projected weight for the
new models increased from earlier estimates in order to meet range,
payload, and persistence requirements. The common operating system is
expected to integrate and provide for interoperability of J-UCAS air
vehicles and is required to control groups of vehicles flying in a
coordinated manner and functioning in the absence of human inputs. The
program director said the common operating system is the most technically
challenging aspect of the entire J-UCAS program.
Agency Comments
In commenting on a draft of this assessment, DARPA stated that the J-UCAS
program, newly established when Congress considered fiscal year 2005
funding, is run under the guidance of a high-level executive committee and
jointly manned with DARPA, Air Force, and Navy personnel. The Air Force
and the Navy have fully coordinated on the demonstration approach using
the X-45C and X-47B in support of service priorities. According to
officials, the J-UCAS concept does not compete directly to replace any
specific manned or unmanned system but will augment a transformed force
structure and provide options to better address military needs in deep,
denied adversary environments. DARPA also stated that in addition to the
capabilities identified by the services today, J-UCAS will offer insights
into new warfighting concepts. It will also preserve opportunities for
competition in follow-on and derivative programs. Finally, DARPA noted
that the common operating system, while technically challenging,
encompasses essential mission functionality and offers the greatest
potential return in flexibility and affordability.
Common Name: KEI
MDA's KEI element is a new missile defense system designed to destroy
long-range ballistic missiles during the boost phase of flight, the period
after launch during which the missile's rocket motors are thrusting. KEI
would also engage missiles in the early ascent-phase, the period
immediately after booster burnout. Key components include hit-to-kill
interceptors, launchers, and battle management units. We assessed the
proposed land-based KEI capability, which is planned to become available
during 2012-2013 (Block 2012).
Source: Artwork developed jointly by KEI contractor and MDA.
Program Prime contractor GAO Design First integrated Block 2012 start
selection review review flight test completion (10/02) (12/03) (1/05)
(TBD) (TBD) (12/13)
All 7 KEI critical technologies are at a relatively low level of maturity,
ranging from proofs of concept established through analytical or
laboratory studies to new applications of existing technologies. For
example, the program is leveraging existing interceptor technologies-
infrared seeker, third stage rocket motor, and divert system-that are
currently used in other MDA programs. The program office rates the
development of 2 critical technologies as high risk. The first involves
one of the interceptor's booster motors, which demands high performance
for KEI engagements. In addition, the program office judges the algorithm
enabling the kill vehicle to identify the missile's body from the luminous
exhaust plume as a high-risk technology. MDA expects to mature these
technologies and integrate them into a land- and sea-based capability
under the prime contract awarded in December 2003.
Production, design & technology maturity
Design & technology maturity
Technology maturity
GAO Development DOD Production review start design decision (1/05) (TBD)
review (TBD)
(TBD)
Desired level of knowledge
Common Name: KEI
KEI Program
Technology Maturity
All 7 KEI critical technologies are at a relatively low level of maturity.
These technologies are part of the element's interceptor, the weapon
component of the element consisting of a kill vehicle mounted atop a boost
vehicle. Of the 7 technologies, 4 pertain to the boost vehicle that
propels the kill vehicle into space. They are its 2 types of booster
motors, attitude control system, and thrust vector control system. The
remaining 3 technologies pertain to the kill vehicle-its infrared seeker,
divert system, and plume-to-hardbody algorithms. Although all technologies
are immature, 3 of the 7 are derived from existing components in other
missile defense programs. The infrared seeker and the third stage rocket
motor come from the Aegis BMD program, and the divert system comes from
the GMD program. Backup technologies exist for all but the infrared
seeker, however, they are at the same low level of maturity as the
critical technologies.
The program office noted that KEI critical technologies are not at a low
level of maturity in and of themselves. The program's assessment-which
rated each technology as relatively immature-was made from a systems
perspective (i.e., it characterized the risk associated with integrating
and demonstrating these technologies in the KEI environment). The 7
critical interceptor technologies will be assessed as mature if the
program successfully completes its first intercept attempt of a boosting
missile. This flight test is expected to be conducted sometime after 2010.
Design Stability
At this time, the KEI program office does not have an estimate for the
total number of drawings for any of its Block 2012 components
(interceptor, launcher, and battle management unit). In addition to the
number of drawings, the program plans to use other metrics to assess
design maturity. Those metrics will include design, manufacturing,
producibility, and quality measures for hardware and measures of maturity
of the system's software.
Other Program Issues
In fiscal year 2004, the KEI program underwent a program replan to
compensate for anticipated fiscal year 2005 funding cuts and the addition
of new requirements (e.g., nuclear hardening) imposed by MDA. The original
program called for a Block 2010 land-based capability to be available by
the end of 2011. In the replan, the land-based capability was combined
with the sea-based capability of Block 2012, both of which utilize the
same interceptor. The KEI program is undergoing further restructuring.
Based on comments received from the program office (see below),
anticipated funding cuts beyond fiscal year 2005 are delaying the
sea-based capability into Block 2014 (2014-2015 time frame) and deferring
other activities indefinitely.
Because completion of the land-based capability continues to be pushed
further in the future, the program's funding profile has changed. Under
the plan to demonstrate an initial capability in the Block 2012 time
frame, near-term funding through fiscal year 2009 was reduced by about 10
percent, with the balance shifted into later years. The latest
restructuring noted by the program office further reduced funding by over
50 percent.
Agency Comments
In commenting on a draft of this assessment, the program office provided
information on the latest restructure of the KEI program. In short,
program funding through fiscal year 2009 was reduced from $7.8 billion (as
listed) to $3.6 billion and, accordingly, program activities such as
development of the sea-based capability were delayed into future blocks.
In addition, the program office indicated that "mission assurance" is the
program's number one priority. In other words, the program's approach to
element development is knowledge-driven, which places an emphasis on
upfront systems engineering and analysis and other risk reduction
activities.
Common Name: Land Warrior
The Army's Land Warrior system is a modular, integrated, soldier-worn
system of systems intended to enhance the lethality, situational
awareness, and survivability of dismounted combat and support soldiers.
Land Warrior comprises a computer-radio, integrated helmet assembly,
weapon, software subsystem, and protective clothing. The Army terminated
Block I (Land Warrior-Initial Capability) in 2003 due to low reliability
in developmental testing and proceeded to Block II (Land Warrior-Stryker
Interoperable). We assessed Block II.
Source: Program Executive Office Soldier.
Program/development
start
(8/94)
Design review (11/04)
GAO review (1/05) Low-rate decision (6/06) Full-rate decision (11/07) Initial
capability (6/08) Last procurement (2019)
Land Warrior entered system development in 1994 and today, two of the
system's four critical technologies are mature. The program expects one of
the remaining two-the personal area network-to be mature before the June
2006 low-rate production decision. The other technology-radio
communications-is a risk area for the program because JTRS Cluster 5
embedded radios will not be available when needed. We could not assess the
design stability of Land Warrior because the program was unable to supply
complete design data. The program reported significant cost growth in
2003, due to an increase in the Army's planned procurement of Land Warrior
systems and to increased Block II software and integration costs. The Army
recently restructured the program, putting Block II on indefinite hold as
the program focuses on fielding elements of the Land Warrior system to the
current force.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (8/94) review
(1/05) (TBD) (11/04)
Common Name: Land Warrior
Land Warrior Program
Technology Maturity
Two of the Land Warrior system's four critical technologies (the
helmet-mounted display and power) are mature. Officials told us that
despite concerns about the ability of industry to produce the
helmet-mounted display in the quantities needed, the technology involved
in the unit (which provides data and video) has been demonstrated and is
mature. The commercial battery technology that will power Land Warrior is
also mature, though overall power management remains a challenge due to
irregularities in components' power consumption.
The other two critical technologies, the personal area network and radio
communications, are not mature. The personal area network includes the
connectors, cables, and interfaces that will link components of the
soldier-worn ensemble to one another. Although such connections have in
the past proven difficult, officials expect this technology to reach
maturity before the June 2006 low-rate production decision. Land Warrior
will eventually utilize the JTRS Cluster 5 embedded radio (assessed
elsewhere in this report) when it becomes available in fiscal year 2011.
Technology for this radio is not mature. In the interim, the Land Warrior
program intends to use the Raytheon MicroLight Enhanced Position Location
Reporting System (EPLRS), a single-channel, commercial-off-the-shelf
radio. Program officials characterize the MicroLight as a cost-effective,
short-term solution. Technology for the MicroLight could not be assessed
as fully mature because it has not yet been integrated into the Land
Warrior ensemble. Program officials said the MicroLight is smaller than
other EPLRS radios in use today.
Design Stability
We could not assess the design stability of the Land Warrior system
because the program was unable to supply complete data on design drawings.
The program cited changes resulting from an impending merger with the
Army's Future Force Warrior technology integration effort as the
complicating factor.
Production Maturity
We could not assess the maturity of production processes for Land Warrior
because the program is not collecting statistical process control data at
this time. Officials told us General Dynamics has not fully identified the
key manufacturing processes, but that the company will measure production
maturity in the future.
Other Program Issues
The Land Warrior program has experienced significant challenges and delays
in its 10-year history. The program restructured after contractor
prototypes failed basic certification tests in 1998. Government testing in
2002 and 2003 revealed technical and reliability problems with Block I.
The program manager terminated Block I shortly thereafter, and focused on
developing Block II.
The Army recently restructured the program again, in response to
congressional direction to immediately field some Land Warrior
capabilities to the current force. The restructured program will produce
capabilities in five spirals and has placed Block II on indefinite hold as
it moves to field the Commander's Digital Assistant and the MicroLight
EPLRS radio in "Spiral 0." The Army received a partial waiver in December
2004 to purchase a limited number of MicroLight radios, but radio
communications will remain a risk area for the program until this issue is
fully resolved. Officials said Spiral 0 is now the program's most pressing
concern, and that the schedule for future spirals is being determined at
this time. In addition, the program is planning to merge its efforts with
the Army's Future Force Warrior technology integration effort, as directed
in the Conference Report accompanying the Department of Defense
Appropriations Act for Fiscal Year 2005. Congress also reduced the
program's fiscal year 2005 budget by $15 million due to anticipated
efficiencies resulting from this merger.
The program reported significant cost growth in 2003, due mainly to an
increase of more than 40,000 units in the Army's planned procurement of
Block II Land Warrior systems to equip a broader range of soldiers than
previously envisaged. Development costs also increased nearly 30 percent
due to software development and vehicle integration requirements for Block
II.
Agency Comments
In commenting on a draft of this assessment, the Army generally concurred
with our assessment and provided technical comments, which we incorporated
as appropriate.
Common Name: LCS
The Navy's Littoral Combat Ship is to be a fast, maneuverable, shallow
draft, surface combatant optimized for littoral warfare. LCS will employ
innovative hull designs and reconfigurable mission packages to counter
antiaccess threats in three mission areas: mine, antisubmarine, and
surface warfare. This review focuses on the technology maturity of the
mission packages associated with the acquisition of the first group of
ships. Since competition for the remainder of the ships continues, we
assessed only the mission modules.
Source: Littoral Combat Ship Program Office.
Program start (9/02)
Development
start
(6/04)
Production decision-1st design (12/04) GAO review (1/05) Production decision-2nd
design (10/05) Initial capability (10/07)
The program office identified 42 critical technologies. In June 2004, the
LCS program entered system development with 14 of these 42 technologies
mature. Five of the remaining 28 technologies are close to being mature.
However, none of the 28 technologies were projected to demonstrate full
maturity until after design review in November 2004. The acquisition
schedule for LCS calls for deploying several critical technologies as
prototypes or engineering development models for the first group of ships.
The technologies that have not reached maturity affect all three of the
littoral warfare missions: mine warfare, antisubmarine warfare, and
surface warfare. The program office designated certain information
competition sensitive. As a result, we have depicted only the level of
knowledge for the LCS mission packages. The Navy has stated that the total
program level of knowledge is higher.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (6/04) review
(NA) (1/05) (11/04)
Common Name: LCS
LCS Program
Technology Maturity
Nine of the technologies under development for LCS are used in multiple
applications or mission packages. Since these technologies are used on
different platforms or in different environments, the program office chose
to assess each use as a separate technology. This resulted in a total of
42 critical technologies, 14 of which are currently mature.
The first set of the mine warfare mission package will align with the
delivery of the first ship in January 2007. As part of this mission, the
MH-60S helicopter is to carry subsystems for either the detection or
neutralization of mines. MH-60S and its technologies for mine detection
are currently expected to complete testing in fiscal year 2005, after
first ship design review for LCS. Its mine neutralization technologies
will complete testing in fiscal year 2007, after delivery of the first
ship.
The Vertical Take-Off Unmanned Aerial Vehicle is an unmanned helicopter,
and will employ the Coastal Battlefield Reconnaissance and Analysis System
for detection of mines on the beach. By delivery to LCS in 2006, the
platform will be an engineering development model and its payload will
still be in testing. The Unmanned Surface Vehicle will be used for all
three littoral warfare missions. For mine warfare, it is expected to
deploy a mine neutralization system, but neither the vehicle nor its
payload will be fully mature by the design review.
The first spirals for antisubmarine and surface warfare packages will
align with delivery of the second ship in fiscal year 2008. MH-60R will be
used for both these missions. The helicopter and its subsystems are fully
mature in the antisubmarine warfare configuration and mostly immature in
the surface warfare configuration. It will complete testing for both
missions in September 2005.
The Vertical Take-Off Unmanned Aerial Vehicle is a communications relay
station for other platforms performing antisubmarine warfare. For surface
warfare, it may use the Advanced Precision Kill Weapons System and an
Electro-Optical Infrared system. Currently, none of the technologies are
fully mature and most will remain in testing by the second ship's design
review in August 2005. In its antisubmarine warfare configuration, the
Remote Minehunting Vehicle will use subsystems that are currently immature
and will be delivered to LCS as engineering development models. As an
antisubmarine warfare platform, the Unmanned Surface Vehicle will carry
detection systems that are not yet mature. For surface warfare operations,
the program will use a gun system and a missile system. A nonlethal weapon
system is also being considered. This vehicle and its technologies are
currently immature in all of its mission configurations.
A missile and a gun system for surface warfare will also be on the ship
itself, but currently neither of these technologies is fully mature.
Design Stability
We did not assess design stability due to the competition sensitive nature
of the ship's designs.
Other Program Issues
While the MH-60R and MH-60S complete testing in fiscal years 2005 and
2007, respectively, they will be unavailable for deployment with LCS until
fiscal year 2009.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the
primary objectives of LCS Flight 0 (the first group of LCS ships) are the
harvesting of mission systems to deliver immediate warfighting capability
in critical gaps and the design and validation of the modular open system
architecture. It also stated that the key to attaining these objectives is
the creation of a common interface that enables the independent
development of sea frames and mission packages and that the use of this
interface is critical for the development and evaluation of sea frames and
mission packages to ensure effective interoperability. The result is a
total system design that is highly adaptable to changes over the life of
the program, but isolates impact to production schedules. The mission
package technology risks described in this report are well understood,
subject to rigorous risk management including appropriate backup
technologies, and generally independent from the successful achievement of
LCS Flight 0 key performance parameters.
Common Name: MEADS
The Army's MEADS is developing a mobile air defense system to protect
deployed maneuver forces and critical assets against short-and
medium-range theater ballistic missiles, cruise missiles, and
air-breathing threats. In 2004, the Army combined management, development,
and fielding of the Patriot air defense missile system and MEADS. Although
the Army combined the programs, MEADS remains an international development
effort among the United States, Germany, and Italy. We assessed the MEADS
fire unit portion of the combined program.
Source: MEADS NPO, Lower Tier Project Office.
Development
start
(7/04)
GAO review (1/05)
Design review (9/09)
Initial production
decision
(11/12)
Last production
decision
(3/17)
Initial capability (9/17)
MEADS began development start in July 2004 with two mature critical
technologies, three critical technologies nearing maturity, and one
immature critical technology. Program plans call for a system design
review in 2009, but program estimates currently project that only one of
the six technologies will be more mature at that time than at development
start. The program office anticipates that all critical technologies will
be fully mature by the start of production in the first quarter of fiscal
year 2013.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (7/04) (1/05)
review (11/12) (9/09)
Common Name: MEADS
MEADS Program
Technology Maturity
Only two of the six critical technologies-launcher electronics and PAC-3
missile integration-were mature at development start in July 2004. Three
other critical technologies-low noise exciter that manages the radars'
frequencies, cooling system for the radars, and slip ring that carries
power and coolants to the radars-were nearing maturity. The remaining
critical technology-the transmit/receive module that transmits/receives
signals for the fire control radar-was immature.
The program office noted that four of the six critical technologies have
been demonstrated or employed. According to the office, the MEADS launcher
will employ electronics already being developed for the Theater
High-Altitude Air Defense (THAAD) and Patriot launcher, and these "common
launch electronics" completed design review in May 2003. Likewise, the
integration of the Patriot Advanced Capability-3 missile into MEADS will
be similar to integrating the missile into the existing Patriot system.
Furthermore, the office indicated that a prototype of the low noise
exciter met some 90 percent of its performance specifications during the
MEADS risk reduction phase that ended in 2004. The office stated that this
prototype provided the information on exciter design necessary to take
corrective actions in the MEADS development phase. In addition, the office
stated that the technology used in the transmit/receive module has been
employed in THAAD and demonstrated that MEADS performance requirements
could be met. However, the U.S.-developed technology as demonstrated on
THAAD is not releasable to the MEADS European partners. The partners are
developing their own transmit/receive module for MEADS, but the design has
achieved only about 75-80 percent of the performance needed.
The program office projects that the transmit/receive module will increase
in maturity by the time of the system design review planned for 2009. The
program office expects that the five other critical technologies will be
at the same maturity levels as they were at development start. The office
expects all critical technologies to be fully mature by the start of
production in late 2012. There are no backup technologies for any of the
MEADS critical technologies, with the exception of the transmit/receive
module.
Design Stability
We could not assess the design stability of MEADS because the number of
releasable drawings and total drawings expected was not available. The
program office expects to know the total number of releasable drawings at
the design review in 2009.
Other Program Issues
The program has adopted an incremental acquisition approach. There are
three increments, with the first beginning in 2008, another in 2010, and
the final in 2013. The program office plans for each increment to
introduce new or upgraded capability into the program. The Army expects
MEADS to achieve initial operational capability in 2017 with four units.
The contract award for the United States, Italy, and Germany to proceed
into design and development together has been delayed by about 9 months.
The Army originally expected the contract award to occur in June/July
2004, but the award did not occur. In September 2004, the United States
and Italy signed a memorandum of understanding to proceed to design and
development, and a letter contract was awarded to initiate that phase. The
contract has a 6-month period of performance, which coincides with the
March 2005 date when the Army expects Germany to sign the memorandum.
Agency Comments
The Army generally concurred with this assessment. It indicated that we
addressed critical technologies that were already areas of intense
management focus. Additionally, it stated that the transmit/receive
module's maturity assessment changed due to international memorandum of
understanding negotiations and U.S. National Disclosure Policy that
changed the source of the modules. The Army also noted that it still
expects all technologies to be fully mature by production and further
stated that there are risk mitigation plans for the maturing technologies
as well as alternate backup technologies now identified for the
transmit/receive module. Additionally, the Army stated that, at the design
review in 2009, the design work in the critical technologies will be at
the maturity level required to fabricate system prototypes and thus
demonstrate system capabilities.
GAO Comments
The MEADS Program Office clarified that the transmit/receive module's
maturity had decreased and we revised our assessment accordingly.
Common Name: MMA
The Navy's MMA is one element of the Broad Area Maritime Surveillance
(BAMS) family of systems, along with the BAMS Unmanned Aerial Vehicle
(UAV) and Aerial Common Sensor programs. The MMA is manned, and it will
sustain and improve armed maritime and littoral intelligence surveillance
and reconnaissance capabilities of the U.S. Navy. The primary roles of the
MMA are persistent antisubmarine and antisurface warfare. It is the
replacement for the P-3C Orion. DOD is discussing international partner
participation in the program.
Source: Boeing - Integrated Defense Systems, (c) 2004 The Boeing Company.
Program start (3/00)
Development
start
(5/04)
GAO review (1/05)
Design review (7/07)
Low-rate decision (5/10) Full-rate decision (4/13) Initial capability (7/13)
Last procurement (2017)
The MMA program entered development with none of its four critical
technologies mature. According to the program office, these technologies
will be demonstrated in a relevant environment by design review and tested
in an operational environment by the production decision. The system's
technology maturity will be demonstrated at least 3 years later than
recommended by best practice standards. However, the program has
identified mature backup technologies.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (5/04) (1/05)
review (5/10) (7/07)
Common Name: MMA
MMA Program
Technology Maturity
None of the 4 critical technologies-integrated rotary sonobuoy launcher,
electronic support measures digital receiver, data fusion, and acoustic
algorithms-are mature. These technologies have not moved beyond the
laboratory environment. For three of the technologies, the components have
not been integrated into a prototype system. The program expects the four
technologies to be demonstrated in a relevant environment by design review
in July 2007 and tested in an operational environment by the production
decision in May 2010. The system's technology maturity will be
demonstrated at least 3 years later than recommended by best practice
standards.
The program office and the contractor developed maturation plans and
identified mature backup technologies for each of the critical
technologies. According to program officials, the MMA would lose some
capabilities but still meet its minimum system requirements if it used
these backups. For example, one of the biggest technology challenges for
the MMA identified by program officials is the electronic support measures
digital receiver. This technology exists as a prototype and has been
demonstrated in a high fidelity laboratory environment. The program is
leveraging the digital receivers currently in development on the EA-18G
program. If the EA-18G digital receiver program is unsuccessful, the
program will have to use legacy analog off-the-shelf receivers, which
would prevent them from gaining an increased sensitivity for certain
signals.
The four technologies we assessed were identified in the MMA's technology
readiness assessment. The program evaluated six other technologies but
decided they were not critical because they had already been demonstrated
in a relevant or operational environment.
Design Stability
We did not assess design stability as the number of releasable drawings is
not yet available.
Other Program Issues
In addition to its primary roles of antisubmarine warfare and antisurface
warfare, the MMA shares the persistent intelligence surveillance and
reconnaissance (ISR) role with the BAMS UAV. The BAMS UAV program will not
start development until fiscal year 2005, and if it does not develop as
expected, the MMA program is the fall back to perform its mission.
According to program officials, in order to fulfill this mission, the Navy
would have to procure 14 additional aircraft by 2018, increasing the
overall cost of the program. If the MMA fails to develop as expected or
experiences schedule slippage, the Navy will have to rely on its aging
P-3C Orion fleet, which, according to DOD, is plagued by serious airframe
life issues, poor mission availability rates, high ownership costs, and
limited system growth capacity.
The MMA program is discussing international participation with Australia,
Canada, and Italy for the development phase of the program. This
participation could include both the MMA and BAMS UAV programs. DOD
expects to benefit from improved interoperability, strengthened allies,
and lower production costs due to increased sales. Program officials
stated that they are incorporating lessons learned from the Joint Strike
Fighter international program, particularly in managing partner
expectations regarding technology transfer.
Agency Comments
In commenting on a draft of this assessment, the Navy generally concurred
with our characterization of the MMA program. It stated that the four
critical technologies are tracking along their current maturation plans
and that it is confident that by design readiness review, these
technologies will be demonstrated in a relevant environment. It noted that
these four technologies are being matured through the MMA risk management
process.
With regard to the BAMS mission, the Navy stated that an analysis of
alternatives conducted in May 2002 concluded that 14 additional aircraft
would have to be in place by 2018 to replace the Legacy P-3 ISR
requirements that were allocated to the BAMS UAV. It further stated that
since that time, a BAMS UAV Operational Requirements Document has been
approved that identified additional UAV specific missions and requirements
that were not considered in the May 2002 analysis of alternatives. It
noted that there is no current completed analysis that encompasses how
many aircraft, based on new approved BAMS UAV operational requirements
document, would be required if the BAMS UAV does not develop as expected.
Common Name: MUOS
The Navy's MUOS, a satellite communication system, is expected to provide
low data rate voice and data communications capable of penetrating most
weather, foliage, and manmade structures. It is designed to replace the
Ultra High Frequency (UHF) Follow-On satellite system currently in
operation and provide support to worldwide, multiservice, mobile, and
fixed-site terminal users. MUOS consists of a network of advanced UHF
satellites and multiple ground segments. We assessed both the space and
ground segments.
Source: Artist rendering of MUOS satellite, Lockheed Martin.
Program start (9/02)
Development
start
(9/04)
GAO review (1/05)
Design review (4/07)
Production decision (10/07) Initial capability (3/10) Full capability (3/14)
In September 2004 the MUOS program was authorized to begin development.
The program currently has eight of nine critical technologies mature. The
remaining technology is projected to be mature by April 2007 in time for
the critical design review. The program intends to order long lead items
for the first two satellites before achieving a stable design. This early
procurement could lead to rework causing cost increases and schedule
delays if relevant designs change prior to critical design review. In
addition, the MUOS development schedule remains compressed, posing several
risks to the program.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO Design Production start review review decision (9/04)
(1/05) (4/07) (10/07)
Common Name: MUOS
MUOS Program
Technology Maturity
Eight of nine critical technologies were mature at the development start
decision in September 2004. The remaining technology, a new cryptographic
chip, is expected to be mature by the time the program reaches its
critical design review in April 2007. A mature backup technology exists
for this chip in the event that it fails to mature in time. However, the
use of the backup technology would increase the vulnerability to attacks
on the transmissions of signals that are used to ensure the satellites
remain properly placed in their orbits around the earth.
Design Stability and Production Maturity
The MUOS program intends to procure long lead items for the first two
satellites before achieving a stable design. The September 2004
development start decision authorized the program to procure long lead
items for these satellites. According to the program office, ordering of
long lead items is to begin in 2005 after segment-level preliminary design
reviews, but well before critical design review in April 2007. This early
procurement could lead to rework if relevant designs change prior to
critical design review, causing program cost increases and schedule
delays. According to the program office, long lead procurement is
necessary to preserve the program schedule and delaying such procurement
until after critical design review would cause the program schedule to
slip. It also noted that the dollar amount of long lead procurement prior
to critical design review is not large, at $65.9 million.
In addition, the program office has yet to determine the total number of
design drawings needed to build the satellites. According to the program
office, the development contract requires completion of 90 percent of
design drawings as a condition of conducting critical design review.
Other Program Issues
DOD delayed the first MUOS satellite launch as well as its initial
operational capability by 1 year to fiscal year 2010. Despite the delays,
the MUOS schedule remains compressed and poses several risks to the
program. For example, initial operational capability is to be declared
before on-orbit operational testing is to occur. Usually, the results of
such testing are used to support decisions for declaring operational
capability and identifying problems that may necessitate design changes.
Furthermore, the time period between the critical design review and the
first satellite launch is shorter for the MUOS program, at about 2.7
years, than that of the previous UHF Follow-On program, at about 3 years.
This schedule comparison is important given the significant leap in
increased capability that MUOS is expected to provide. While the UHF
Follow-On program increased communications capability by up to a factor of
3, the MUOS program is expected to increase communications capability by a
factor of
20. The program office, however, considers the development of the
satellite to be low risk. In addition, program officials stated that the
initial operational capability was changed to mean initial MUOS on-orbit
capability, and initial operational capability would be declared after
on-orbit operational testing takes place.
In addition, an independent program assessment states that the program is
schedule-driven primarily because of the software development effort.
According to the program office, software development for the MUOS ground
segment represents one of the highest risks to the program due to the size
and complexity of the contractor's design. The program office stated that
the ground software segment is to be developed incrementally to mitigate
schedule risk.
Agency Comments
In commenting on a draft of this assessment, the Navy provided technical
comments, which were incorporated where appropriate.
Common Name: Predator B
The Air Force's MQ-9 Predator B is a multirole, medium-to-high altitude
endurance unmanned aerial vehicle system capable of flying at higher
speeds and higher altitudes than its predecessor the MQ-1 Predator A. The
Predator B is designed to provide a ground attack capability and will
employ fused multispectral sensors to find and track small ground mobile
or fixed targets. As envisioned, each Predator B system will consist of
four aircraft, a ground control station, and a satellite communication
suite. We assessed only the air vehicle.
Source: General Atomics-Aeronautical Systems, Incorporated.
Program start (1/02)
Development
start
(2/04)
GAO review (1/05)
Design review (4/06)
Full-rate decision (12/07) Initial capability (12/09) Last procurement (2014)
The Predator B entered system development in February 2004 with three of
its four critical technologies mature. The fourth, needed for weapons
launch, has not matured as expected. The Air Force expects this technology
to be ready in August 2005-a slip of 13 months. No backup technology is
available. If this technology fails to mature, it will prevent the
Predator B from performing its primary mission to destroy enemy targets.
The program recently changed to incrementally develop versions of the
Predator B. The Air Force believes most drawings for increment one will be
complete by the 2006 critical design review. The program has also
concurrently started to produce Predator B aircraft, and operational
testing is not scheduled to be complete until 2007 when one-third of them
will be on contract. Concurrency increases the risk of redesign and need
to retrofit already acquired system.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (2/04) (1/05)
review (12/07) (4/06)
Common Name: Predator B
Predator B Program
Technology Maturity
Three of the Predator B's four critical technologies, the synthetic
aperture radar, the multispectral targeting system, and the air vehicle,
are fully mature. The avionics subsystem technology designed to integrate
and store data necessary to launch munitions is still being evaluated in a
laboratory environment. It is expected to be ready by August 2005, a
13-month schedule slip. No backup technology is available. If this
critical technology fails to mature, it will prevent the Predator B from
performing its primary mission to destroy enemy targets. The Air Force
plans to retrofit these and other air vehicles that are under production
once this capability has been fully demonstrated.
Design Stability
Subsequent to Milestone B approval in February 2004, the program office
was directed by Headquarters Air Force to develop Predator B in three
increments. DOD is in the process of defining the increments. The program
office expects 94 percent of the expected increment one drawings to be
completed by the April 2006 critical design review, which has been delayed
about 7 months since our last report. Program officials acknowledge that
additional drawings will be needed for subsequent increments. Design
changes and modification of drawings are likely to occur late in
development, increasing the need to retrofit already acquired systems.
Production Maturity
Program officials said the contractor does not plan to use statistical
process controls to ensure product quality. Instead, they plan to use
other quality control measures such as scrap, rework, and repair to track
product quality. Also, initial operational testing of increment one, which
is to demonstrate a product is ready for production, is not scheduled to
be complete until September 2007. Testing for remaining increments has not
been determined.
Other Program Issues
In February 2004, Headquarters Air Force directed the program office to
quickly field an interim combat capability to the warfighter by fiscal
year 2006. This delayed the start of the system development and
demonstration phase by 9 months to November 2004. However, the Air Force
is already concurrently on contract to produce 15 Predator Bs. The
decision to make Predator B an incremental development program has also
extended the completion of development by nearly 4 years. An incremental
approach is the preferred approach to weapon acquisitions. However, the
Air Force does not plan to have formal decisions approving entry into
development for subsequent increments as required by DOD acquisition
policy. To reduce the risks of concurrently developing and producing
Predator Bs, the program office lowered annual buy quantities and extended
production 5 years. The estimated program completion date is now 2014.
The Air Force is still evaluating a variety of lightweight munitions for
use on the Predator B. The Air Force is also weighing the possibility of
adding new system capabilities such as launching very small or micro
unmanned aerial vehicles from the Predator B and equipping it with
air-to-air missiles.
Agency Comments
In commenting on a draft of this assessment, the Air Force disagreed with
our evaluation of the Predator B development risks. It stated that the
stores management system technology is mature and that the system is being
tested. It also noted that the existing weapons release system provides a
backup capability. It also disagreed with our assessment that the Predator
B development had been extended by 4 years. It stated that, as planned,
the initial operational capability will follow the completion of the first
increment in December 2009. Future increments are to be determined. Before
starting future increments, the Air Force stated that proper approval will
be obtained from the milestone decision authority. Also, its acquisition
plan has phased production rates to the development effort, and the
increased concurrent production before operational testing has been driven
by congressional actions.
GAO Comments
The program planned to deliver the full capability Predator B in 2006, but
due to acquisition approach changes the full capability Predator B is now
scheduled for delivery in 2010-a 4 year extension.
Common Name: NPOESS
NPOESS is a triagency National Oceanic and Atmospheric Administration
(NOAA), DOD, and National Aeronautics and Space Administration (NASA)
satellite program to monitor the weather and environment through the year
2020. Current NOAA and DOD satellites will be merged into a single
national system. The program consists of five segments: space; command,
control, and communications; interface data processing; launch; and field
terminal software. We assessed all segments.
Source: NPOESS Integrated Program Office.
Program start (3/97)
Development start/ production decision (8/02) GAO review (1/05) 1st satelilite
launch (11/09) Initial capability (9/11)
In August 2002, the NPOESS program committed to the development of
satellites with operational capability without having demonstrated
technology maturity or design stability. Only 1 of its 14 critical
technologies is mature. The program expects that all but 4 of these will
be mature by the design review in April 2006. The program has released
about half of its design drawings and expects to complete about 94 percent
by design review. It is not collecting statistical process control data to
assess production maturity because of the small number of units being
produced. At present, the program office considers the three critical
sensors to be key program risks because of technical challenges. Due to a
recent program restructuring, the program office estimates that the cost
of the program will increase to $8.1 billion.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Development GAO
start design start/production
review
(NA) review decision (1/05)
(NA) (8/02)
Common Name: NPOESS
NPOESS Program
Technology Maturity
Only 1 of the program's 14 critical technologies were (and currently are)
mature at the production decision in August 2002. This is less than
reported last year due to the program office's more accurate application
of the technology standards. The program projects that all but 4 of the
technologies will be mature by the design review in 2006.
The program undertook the NPOESS Preparatory Project, a demonstration
satellite, to reduce risk and provide a bridging mission for NASA's Earth
Observing System. This satellite, scheduled for launch in 2006, is planned
to demonstrate three critical sensors in an operational environment. This
will provide data processing centers with an early opportunity to work
with sensors, ground controls, and data processing systems and allow for
incorporating lessons learned into the satellites. The three critical
sensors are experiencing continued technical problems and schedule delays.
The program office considers these sensors as top program risks.
Design Stability
In August 2002, the program committed to the development of two satellites
with operational capability before achieving design stability or
production maturity. Program officials indicated that about 50 percent of
the design drawings were released to manufacturing and expects to release
about 94 percent by the design review in 2006.
Production Maturity
We could not assess production maturity because, according to the program
office, it does not collect statistical process control data due to the
small number of units to be built. However, the ground segment contractor
uses various metrics such as schedule and cost performance indices, rework
percentages, and defect containment to ensure production is proceeding as
planned. According to the program office, monthly reviews of these metrics
reveal acceptable results.
Other Program Issues
In 2002, DOD extended the launch date of one of its legacy meteorological
satellites to 2010, delaying the need for NPOESS. DOD and NOAA thus
reduced their NPOESS funding by about $144 million through fiscal year
2007 and the program delayed the launch of the first satellite 7 months,
to November 2009.
The recent funding reductions prompted a restructuring of the NPOESS
program. The program office estimates that the cost will increase to $8.1
billion. This increase reflects changes to the contract and increased
program management costs. The program office reports that the increases
include costs associated with extending the development schedule,
increased sensor costs, and additional funds needed for mitigating risks.
The program office is planning to present a new cost estimate to its
executive oversight committee in January 2005 to ensure the program is
adequately funded. Other factors could further affect the revised cost and
schedule estimates. Specifically, the contractor is not meeting expected
cost and schedule targets of the new baseline because of technical issues
in the development of key sensors.
Agency Comments
In commenting on a draft of this report, the program office stated that it
lowered its technologies' maturity levels in September 2004 at our
request. Program officials also commented that since the government can no
longer afford full-up research and development satellites, few instruments
can attain technology maturity and systems cannot achieve design stability
or production maturity prior to entering full-scale development. The
program office stated that it spent 5 years in the Preliminary Design and
Risk Reduction phase driving down sensor and system risk, thereby
significantly increasing the technology and sensor design maturity before
entering the Acquisition and Operations phase in August 2002. It also
noted that the current instrument problems highlighted above result from
design/manufacturing process issues, which are not related to the listed
critical technologies.
GAO Comments
The NPOESS program's technology maturity levels were lowered because the
program office more accurately applied the technology standards. In
addition, these standards do not require the launch into space of a
full-up research and development satellite in order to achieve full
maturity. Rather, a representative model demonstrating the full
functionality of the subsystems in a relevant environment is sufficient.
Common Name: SBIRS High
The Air Force's SBIRS High program is a satellite system intended to
provide missile warning information and to support the missile defense,
technical intelligence, and battlespace characterization missions. It also
is intended to replace the Defense Support Program and to consist of four
satellites (plus one spare) in geosynchronous earth orbit (GEO), two
sensors on host satellites in highly elliptical orbit (HEO), and
associated fixed and mobile ground stations. We assessed the sensors and
satellites only.
Source: Lockheed Mar tin Space Systems Company.
Program start (2/95)
Development
start
(10/96)
Design review (8/01)
First sensor delivery (8/04) GAO review (1/05)
Second sensor
delivery
(2/05)
First satellite delivery (4/08)
The SBIRS High program's critical technologies have demonstrated
acceptable levels of maturity after many years of difficult development.
The design is now mature since approximately 98 percent of the expected
design drawings have been released. Production maturity could not be
determined because the contractor does not collect statistical control
data. In August 2004 the contractor delivered the first payload (the HEO 1
sensor) after a delay of 18 months. This created additional delays and
cost increases. As a result, the program is again being replanned.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (10/96) review
(NA) (1/05) (8/01)
Common Name: SBIRS High
SBIRS High Program
Technology Maturity
The SBIRS High program's three critical technologies-the infrared sensor,
thermal management, and on-board processor-are mature. Program officials
indicated that the hardware was tested in a thermal vacuum chamber under
expected flight conditions. These technologies were not mature at the
start of development.
Design Stability
The design of SBIRS High was not stable at the critical design review in
August 2001 since only 30 percent of the expected design drawings had been
released at that time. The design is now stable with about 98 percent
released.
Design stability has been an issue for SBIRS High. The first HEO sensor
was delivered in August 2004 after a delay of 18 months due to excessive
electromagnetic interference (radio waves emitted by the sensor's
electronics that interfered with the host satellite). The program office
reports that it applied the knowledge gained from the design problems on
this sensor to the second HEO sensor, which is now due for delivery in
February 2005-a 13-month delay from the restructured schedule. Initial
testing of the second sensor revealed one electromagnetic interference
issue. The program office anticipates the approval of a waiver to this
deviation.
Production Maturity
We could not assess the production maturity of SBIRS High because the
contractor does not collect statistical process control data. However, the
program office tracks and assesses production maturity through detailed
monthly manufacturing and test data and monthly updates on flight hardware
qualifications. In addition, the program office recently assigned detailed
entrance criteria to all major manufacturing and test events. These
criteria must be fully satisfied prior to program office approval to enter
the specific event. According to the program office, this new
"eventdriven" philosophy will significantly improve insight into the
maturity of the production process.
Other Program Issues
The delayed delivery of the first HEO sensor affected cost and schedule
for the remainder of the program. For example, resources needed for the
second HEO sensor and GEO satellites were instead used on the first HEO
sensor. The deliveries of the first two GEO satellites have now each been
delayed by over a year (to April 2008 and April 2009).
In May 2004, the program incurred a second Nunn-McCurdy breach (10 U.S.C.
2433), this time at the 15 percent threshold. Since program delays and the
extension of the contract through 2011 yielded a substantial funding
shortfall, Congress increased the SBIRS High fiscal year 2005 budget by
$91 million. The program office reports that future risks are being
mitigated by addressing high-risk elements earlier in the development
phase as well as earlier and more robust testing. It also plans to convene
an independent review team in early 2005 to assess the program's progress
and future risks.
Because of the lag time between the procurement of the first two GEO
satellites and the last three, the Air Force is able to consider upgrading
the on-board processors for the GEO satellites 3-5. A revised acquisition
program baseline will be submitted in March 2005 after a decision on this
upgrade is finalized and the cost impact is determined.
Agency Comments
In commenting on a draft of this report, the Air Force stated that the
February 2005 delivery of the second HEO sensor is well before the need
date of mid-June 2005. It also noted that the GEO satellite's signal
processor assembly power supply and the common gyro reference assemblies
were integrated onto the payload structure (both are key steps toward the
payload's first thermal vacuum test) and that GEO spacecraft testing has
been successful in the early identification and mitigation of
hardware/software integration issues before they become schedule critical
path concerns. It also commented that the Defense Support Program-capable
Multi-Mission Mobile Processors are in test and are on track for
operational certification by December 2005 and that initial SBIRS High
support to the Missile Defense Agency mission is in place.
Common Name: SDB
The Air Force's SDB is a small autonomous, conventional, air-to-ground,
precision bomb able to strike fixed and stationary targets. The weapon
will be installed on the F-15E aircraft and is designed to work with other
aircraft, such as the F/A-22. Potential follow-on capabilities, such as
precision strike against moving targets, are being considered.
Source: The Boeing Company, St. Louis, Missouri.
Program start (8/01)
Development
start
(10/03)
GAO review (1/05) Low-rate decision (4/05) Initial capability (9/06) Last
procurement (11/19) Last delivery (9/21)
The six critical technologies for the SDB appear mature, and the design is
stable. The program office held the design review prior to starting system
development and, although data were not collected, the program maintains
that the contractor released over 90 percent of the production drawings.
In 2004, the program began a test program, which combines developmental,
live fire, and operational testing, in an effort to decrease time spent in
system development. Although the first three flight tests were successful,
this concurrent approach may increase program risks. A low-rate production
decision is expected to be made in April 2005.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (10/03) review
(1/05) (4/05) (NA)
Common Name: SDB
SDB Program
Technology Maturity
The program office assessed all six critical technologies for the SDB as
mature. The technologies are the airframe, the Anti-Jam Global Positioning
System, the fuze, the Inertial Navigation System, the carriage, and
warhead. Program officials stated that many of the program's critical
technologies were demonstrated in a free-flight environment. They also
stated that they have flight-tested the system with the properly sized
components.
Design Stability
The design review was held prior to the start of system development and,
although data were not collected, the program office maintains that Boeing
released over 90 percent of the production drawings. According to the
program office, although the contractor has ultimate responsibility for
the weapon system and has given the government a 20-year "bumper to
bumper" warranty, the program office has insight into the contractor's
configuration control board process and all changes are coordinated with
the government.
The SDB program began a program of developmental, live fire, and
operational testing in 2004. This combined testing approach is designed to
eliminate or reduce redundant testing. However, this process could expose
the program to additional risk of design changes, as there may be more
concurrency between system developmental and operational tests than there
would be under a traditional test program. As of the date of this review,
3 of 16 planned flight tests had been conducted, each meeting its
objectives. These flight tests were conducted with live fuzes but not with
live warheads. Eleven of the 16 flight tests are planned to be conducted
prior to the low-rate production decision point.
Production Maturity
We could not assess production maturity because statistical process
control data were not available. In developing the SDB, Boeing used many
key components that are common with the Joint Direct Attack Munition
(JDAM). The SDB production line will be colocated in the same facility
used to produce the JDAM. According to program officials, the production
line layout is very similar to the processes currently used for the JDAM.
As of the date of this review, no critical manufacturing processes that
impact the critical system characteristics had been identified. A low-rate
production decision is expected to be made in April 2005.
Agency Comments
In commenting on a draft of this assessment, the Air Force concurred with
the information presented and provided technical comments, which were
incorporated as appropriate.
Common Name: STSS
MDA's STSS element is being developed in incremental, capability-based
blocks designed to track enemy missiles throughout their flight. The
initial increment is composed of two demonstration satellites built under
the Space Based Infrared System Low program. MDA plans to launch these
satellites in 2007 to assess how well they work within the context of the
missile defense system. MDA is also studying improvements to the STSS
program, and it will be building next generation satellites. We assessed
the two demonstration satellites.
Source: STSS Program Brief.
SBIRS-Low program start (1995)
Transition to MDA (10/00) STSS program start (2002) GAO review (1/05)
Demonstrator satellite launch (2007) Software upgrades (2008)
Four of the STSS program's five critical technologies are mature, and the
remaining technology is expected to reach maturity in March or April 2005.
The STSS design appears stable, with all drawings released to
manufacturing. However, until all STSS technologies demonstrate maturity,
the potential for design changes remains. The program is currently in the
process of conducting system level assembly, integration, and testing
activities and software development. Until that work is complete, certain
risk areas, such as payload hardware and software integration, will
remain. Additionally, a number of systemic quality and systems engineering
problems with the payload have persisted. Despite these issues, the
program office still expects early delivery and launch of the satellites.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (NA) review
(1/05) (TBD)
(11/03)
Common Name: STSS
STSS Program
Technology Maturity
Four of five critical technologies-satellite communication cross-links,
on-board processor, acquisition sensor, and track sensor-are mature. The
acquisition sensor reached maturity in October 2004 (a month later than
reported last year) when the thermal vacuum testing was completed. The
track sensor reached maturity in December 2004 when the payload for the
first satellite completed thermal vacuum testing, which is 3 months later
than reported last year. The single-stage cryocooler will be mature when
the payload for the first satellite completes thermal vacuum testing in
March or April 2005-about 15 months earlier than reported previously. Last
year the program had a sixth technology, the two-stage cryocooler, but it
is no longer considered critical and will not be used on the first
increment of the STSS program.
Design Stability
The STSS program's design is stable, with all drawings released to
manufacturing. When the STSS program started in 2002, design drawings and
the satellite components for the partially built satellites from the Space
Based Infrared System Low effort were released to manufacturing. By the
time STSS went through its design review in November 2003, the program
office had released all subsequent design drawings. However, until the
maturity of the STSS technologies has been demonstrated, the potential for
design changes remains.
Other Program Issues
The STSS program is in the process of completing the assembly,
integration, and testing of the satellite components and software
development. Until that work is complete, certain risk areas will remain.
Some of these include complex infrared payload hardware and software
integration; completion of the ground segment and infrared sensor software
development and testing; modifications to the tracking sensor, system
integration and testing; and handling issues related to parts
obsolescence.
In addition, the payload subcontractor has had a number of systemic
quality and systems engineering problems. These problems have continued
for the last year and have contributed to some cost and schedule overruns
on the payload subcontract. The quality and engineering problems are the
result of the subcontractor's lack of experience and systems engineering
procedures that are not clearly written. In response, the prime contractor
reviewed the subcontractor's quality program. During this time, there was
a 2-month stoppage of work at the subcontractor facility and the majority
of the subcontractor's effort was concentrated on resolving failures noted
during assembly, integration, and testing of the satellite components.
When work restarted at the facility, the subcontractor continued to
encounter difficulties in assembling the sensors and preparing the
appropriate test equipment needed for sensor-level testing. Based on these
factors and the significant remaining tasks, the prime contractor stepped
up its presence at the subcontractor's facility. In addition, the
subcontractor added technicians who have more experience working with
space hardware and brought in systems engineers to work with the
technicians.
Despite these issues, the program office still expects the prime
contractor to deliver and launch the satellites earlier than the contract
date of July 2007.
Agency Comments
In commenting on a draft of this assessment, MDA generally concurred with
our assessment and provided technical comments, which were incorporated
where appropriate.
Common Name: THAAD
MDA's THAAD element is being developed in incremental, capability-based
blocks to provide a ground-based missile defense system able to defend
against short-and medium-range ballistic missile attacks. THAAD will
include missiles, a launcher, an X-band radar, and a command and
control/battle management system. We assessed the design for the Block
2006 initial capability of one fire unit that MDA plans to hand off to the
Army for concurrent operation and testing in fiscal year 2009.
Source: THAAD Project Office.
Program start (1/92)
Transfer to MDA (10/01)
GAO review (1/05) 1st intercept attempt, Block 2004 (2nd Q/06) Initial
capability available Block 2006 (FY09)
Program officials assess THAAD's technologies as mature and its design as
generally stable. The technology assessments, however, are sometimes based
on tests of earlier component designs. The design of Block 2006, which is
expected to provide a limited operational capability, is a further
maturation of THAAD's Block 2004 design. While 91 percent of the Block
2004 engineering drawings have been released, the total number of drawings
for the 2006 capability could increase if problems are identified in
flight tests scheduled to begin early next year.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (6/00) review
(1/05) (TBD) (12/03)
Common Name: THAAD
THAAD Program
Technology Maturity
Program officials assess all of THAAD's critical technologies as mature.
These technologies are included in four major components: the command,
control, and battle management component; the interceptor; the launcher;
and the radar.
After experiencing early test failures, program officials made changes in
the execution of the THAAD program that allowed it to make progress in
maturing critical technologies. Officials placed more emphasis on risk
reduction efforts, including adopting technology readiness levels to
assess technological maturity.
Design Stability
THAAD's basic design is nearing completion, with approximately 91 percent
of the expected engineering drawings released for the basic design that is
expected to provide the initial capability. However, the THAAD Program
Office reported a decrease in the percentage of drawings released this
year (91 percent) compared to the percentage reported last year (100
percent). In 2003, the program reported that it had released all of the
expected 9,852 drawings. However, as the design matured, the program
office recognized that 11,221 engineering drawings would be required and
that it had released only 10,221 of those drawings. The number of drawings
increased as information was gained from testing, the design of
experimental items was completed, existing drawings were revised, and as
new subcomponents were needed to replace obsolete ones. The program office
successfully conducted a design review in December 2003. However, if
problems are identified during flight testing, the number of drawings may
increase as the design matures during Block 2006.
Production Maturity
We did not assess THAAD's production maturity because MDA does not know
when it will transition THAAD to the Army for production. The one fire
unit that will be handed off to the Army in 2009 for limited operational
use is considered to be primarily a test asset. Prior to a production
decision, the program office plans to assess production maturity using
Baseline Manufacturing Readiness Risk Assessments and Block Process
Verification Reviews for assurance of the contractor's readiness to
proceed with repeatable processes and quality.
Other Program Issues
Although the THAAD program has implemented many procedures to reduce
program risk, it continues to encounter some problems. For example, the
program experienced a major workmanship problem in a shelter subsystem
within the command, control, and battle management component. In addition,
an explosion at the Pratt & Whitney propellant mix facility is causing the
program to seek an alternate source. The program office's risk assessment
states "source replacements have the potential for delaying booster
delivery during the flight test program and into production."
MDA officials are examining whether one THAAD component can be deployed
early. Officials are assessing whether a THAAD-like radar can serve as a
forward-deployed radar for the Ballistic Missile Defense System.
Development, customization, and testing of the radar under another MDA
program have begun in an effort to provide this capability within the next
2 years.
Agency Comments
In commenting on a draft of this report,
MDA provided technical comments, which were
incorporated as appropriate.
Common Name: Tomahawk
The Navy's Tactical Tomahawk Block IV will allow ships and submarines to
attack land targets. Program officials say it incorporates new subsystem
features like an improved antijamming global positioning system, in-flight
retargeting, and transmission of imagery prior to impact. They also said
it will have greater reliability and its average per unit cost will be
$729,000 versus the $1.4 million of its predecessor. The Block IV includes
the missile, the weapon control system, and the mission planning system.
We assessed only the missile.
Source: Tomahawk Program Office (PMA-280).
Program start (12/97)
Development
start
(6/98)
Design review (6/00)
Low-rate decision (9/02) Initial capability (5/04) Full-rate decision (7/04) GAO
review (1/05) Last procurement (FY09)
The Tactical Tomahawk Block IV program entered low-rate production and
awarded its full-rate production contract without the knowledge needed to
ensure its production processes were in control but with mature technology
and design knowledge. The program received its first low-rate production
missile in May 2004. Other missiles such as those used in operational
testing, while production representative, were mostly put together one at
a time, so their manufacture was insufficient for collecting statistical
data necessary for process control. Officials did not expect that the
program would produce and test sufficient missile quantities to have the
necessary knowledge about its production processes until sometime during
March or April of 2005. Delivery of its first full-rate production
missiles in January 2006 depends on completing substantial
testing/verification.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Production GAO start design decision review (6/98) review
(9/02) (1/05) (6/00)
Common Name: Tomahawk
Tomahawk Program
Technology Maturity
We did not assess the readiness level of the key technologies for the
Tactical Tomahawk Block IV because its subsystems were derivative from
other programs or upgrades to preexisting subsystems. Therefore, according
to program officials, the critical technologies for the missile's key
subsystems like the antijamming global positioning system, the digital
scene matching area correlator, and the cruise engine were already mature.
Design Stability
The design of the Tactical Tomahawk missile is complete. At the design
review in June 2000, about 47 percent of the drawings had been released to
manufacturing. By the end of technical evaluation in October 2003, 100
percent of the drawings had been released. Technical evaluation was
successfully completed, and the program entered operational evaluation in
December 2003. Operational evaluation was completed in 2004, and the
missile was judged operationally effective and suitable.
Production Maturity
We could not assess the production maturity of the Tactical Tomahawk Block
IV missile because program officials said statistical process data needed
for production maturity were not available. Although the Block IV uses
much existing technology to reduce costs, the technology is arranged
inside the missile in a new manner. The new layout makes the production
process sufficiently different enough that it requires development of new
production processes and statistical controls. Officials said the program
had not yet produced and tested sufficient missile quantities to attain
this statistical control information. Tomahawk officials currently project
the program will obtain production maturity prior to January 2006.
The Navy's Operational Test and Evaluation Force judged the missile
operationally suitable and effective for combat operations but also
recommended review of quality assurance processes. Prior to this
recommendation, the program had engaged outside experts to conduct a
quality audit. The audit team concluded the audited facilities would
consistently supply material to meet the program's requisite product and
process capability requirements. The team also noted opportunities for
improvement in areas like statistical process control. Officials said a
follow-up Navy/Raytheon (the prime contractor) review indicated that
progress had been made in all areas identified for improvement. They also
said Raytheon had contracted for ongoing outside support for
implementation of quality initiatives.
Other Program Issues
At the time of our review, a full-rate 5-year production contract had been
awarded, with the multiyear feature designed to provide earliest
replenishment of inventory at lowest cost. Full-rate production is planned
for fiscal year 2004 through fiscal year 2008.
Agency Comments
Commenting on a draft of this assessment, the Navy provided technical
comments, which were incorporated where appropriate. It also noted that
all Block IV production processes have been fully defined and are
maturing.
Common Name: TSAT
The Air Force's TSAT system is designed to provide survivable,
jam-resistant, global, secure, and general-purpose laser cross-links with
other air and space systems, including the planned AEHF satellite system,
reviewed elsewhere in this report. TSAT will serve as the cornerstone of a
new DOD communications infrastructure by providing high bandwidth
connectivity to the warfighter. The system consists of a constellation of
five satellites, plus a sixth satellite to ensure mission availability. We
assessed the six satellites.
Source: Northrop Grumman Corporation.
Program start (8/02)
Development
start
(1/04)
Interim review (11/04)
GAO review (1/05) Production decision (04/08) First satellite launch (11/12)
TSAT entered the risk reduction and design development phase in January
2004 with only one of its seven critical technologies mature. The program
expects to demonstrate technology maturity but not design stability or
production maturity before awarding a contract to acquire operational
satellites in 2006.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (1/04) (1/05)
review (4/08) (NA)
Common Name: TSAT
TSAT Program
Technology Maturity
The TSAT program is in the risk reduction and design development phase,
with only one of its seven critical technologies mature. The program is
being developed in two increments-six of the technologies are associated
with the first increment and all seven are associated with the second
increment.
Of the six technologies associated with the first increment, only one
technology-the packet processing payloads-is mature. The other five-
communication-on-the-move nulling antenna, dynamic bandwidth and resource
allocation technologies, protected bandwidth efficient modulation
waveforms, information assurance, and single access laser
communications-are scheduled to reach maturity in early 2006, about 2
years after the start of development. The single access laser
communications has no backup technology, and according to program
officials, any delay in maturing this technology will cause the expected
first satellite launch date to slip beyond 2012.
The seventh critical technology, the multiaccess laser communications, is
part of the second increment. It will not reach maturity until the
production decision for the last four operational satellites in 2008,
about 4 years after the planned start of development.
Other Program Issues
Unlike current communications satellites, TSAT will be equipped with
laser-optical payloads for high-capacity links to other air and space
platforms. AEHF will depend on the first TSAT satellite, now scheduled for
launch by the end of 2012, to provide full global coverage. Because
military users are concerned with the aggressive acquisition strategy, the
Air Force scheduled an interim review point for November 2004 to determine
whether the technology development had progressed sufficiently to meet the
required launch date and decided to continue with both AEHF and TSAT
development. A second interim review point is scheduled for November 2005,
at which point the Air Force must decide on alternatives, one of which is
to buy an additional AEHF satellite. Air Force officials are in the
process of defining the evaluation criteria they intend to use to assess
TSAT's progress or identify alternatives.
TSAT is currently being rebaselined as a result of a congressional
reduction totaling $300 million in research and development funding for
fiscal year 2005. The defense authorization conference report indicated
that funding was reduced because of continuing concerns related to the
risk of the current acquisition approach.
Agency Comments
In commenting on a draft of this assessment, the Air Force stated that,
based on commercial and DOD best practices, all TSAT technologies meet, or
exceed, the level of maturity appropriate for the current risk reduction
and design development phase and that this phase provides the data
(technology readiness and design maturity) necessary for a production
contract award. It also commented that all key technologies are on
schedule to achieve maturity 10 months prior to Preliminary Design Review
and that, to further reduce risk, TSAT has backup technologies in all
areas in the event that a technology is not ready. It noted that the
backup technologies would still provide a large increase in warfighter
capability and allow for technologies to be used on later TSAT satellites.
It also noted that to be effective, risk reduction and preliminary design
must be done concurrently and iteratively. If not, the program risks
maturing technology that does not support the system design, resulting in
scrap and rework. It believes that this strategy delivers the greatest
warfighter capability at minimum risk and cost.
GAO Comments
Our prior work has shown that technologies should demonstrate a high level
of maturity before starting development to reduce the risk of cost,
schedule, and performance problems. Although the program started
development a year ago, we found that several critical technologies had
demonstrated very low levels of maturity involving analytical studies and
the demonstration of nonscale individual components in a laboratory
environment.
Common Name: V-22
The V-22 Osprey is a tilt rotor, vertical takeoff and landing aircraft
being developed by the Navy for Joint Service application. It is designed
to meet the amphibious/vertical assault needs of the Marine Corps, the
strike rescue needs of the Navy, and the special operations needs of the
Air Force and the U.S. Special Operations Command. The MV-22 version will
replace the CH-46E and CH-53D helicopters of the Marine Corps. We assessed
the MV-22 Block A, which has been undergoing changes to make it safe and
operational.
Source: U.S. Navy.
Program start (12/82)
Development
start
(4/86)
Development restart (9/94) GAO review (1/05) Full-rate decision (10/05) Initial
capability (2007) Last procurement (2015)
MV-22 Block A technologies are mature and the design is considered stable.
Problems identified during recent tests are expected to be resolved prior
to the next operational test, according to program officials. Some
redesign efforts have been identified as candidates for preplanned product
improvements. Parts issues and delayed reporting of test results could
delay the operational performance certification needed to increase
production in fiscal year 2006. Decisions on whether to lift current
flight restrictions, prior to the completion of operational evaluation,
will be made on a case-by-case basis. Recent tests found interoperability
and human factors as high-risk issues that may impact this evaluation.
Also, the contractors were asked to propose cost reduction initiatives
targeted at reducing aircraft unit cost to $58 million by fiscal year
2010.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD GAO Production start design review decision (4/86) review
(1/05) (10/05) (9/02)
Desired level of knowledge
Data not available
Common Name: V-22
V-22 Program
Technology Maturity
Although we did not assess the MV-22's technology maturity, the program
office states that based on DOD criteria, the Block A technologies are
mature. During recently completed limited operational tests, technology
maturity was assessed in a range of environmental conditions. Program
officials state that problems were identified and corrective plans
implemented to insure a successful operational test and evaluation
assessment.
Design Stability
Design for Block A is considered stable. However, additional changes to
later blocks of the aircraft have been identified. These changes include
redesign of the forward cabin; redesign of the rear cabin seating, which
is considered inadequate for combat equipped troops; redesign of a
extendable tube for fuel jettison operations; and enhancements to improve
wheel brake control and effectiveness.
Production Maturity
Process management is becoming more robust at the final assembly site on
each major fixture assembly using Six Sigma. Program officials point to
the delivery of aircraft as an indication of manufacturing maturity.
An independent review assessed a V-22 parts problem at one of the
contractors' plants that could affect its ability to support full-rate
production and concluded that in the near term they believe the current
parts shortage could be addressed with heroics. However, the team and
program officials are concerned with the institutionalization of long-term
process improvements and recommended development of a plan that addresses
both short-term part shortages and implementation of a full-rate
production plan.
The Navy plans to increase annual production of the aircraft starting in
fiscal year 2006, provided the Secretary of Defense certifies to Congress
that the program successfully completed operational testing by
demonstrating several capabilities related to V-22 safety, effectiveness,
maintainability, and reliability (Section 123, Pub. Law 107-107, Dec. 28,
2001) through operational test. The certification would allow the program
to increase annual production above the current minimum sustaining rate.
Program officials are concerned that the certification cannot be done
before completion of the fiscal year 2006 budget process and, as a result,
the request to increase production may not be granted.
Other Program Issues
The V-22 is currently being tested with operating limits, such as
defensive combat maneuver capability. Decisions on whether to relax or
remove specific restrictions will be made on a case-by-case basis prior to
the completion of operational evaluation in June 2005. The decisions on
these restrictions will impact the result of the upcoming operational
assessment. A recently completed limited assessment concluded that out of
16 critical operational issues, 2 were at high risk and 6 at medium risk
of not achieving a satisfactory resolution during upcoming operational
testing. The high-risk issues are interoperability and human factors. The
medium-risk issues are reliability, availability, logistics support,
compatibility, documentation, and diagnostics. Recently, the program
requested that the contractor submit a proposal for combining cost
reduction initiatives to reduce the aircraft unit price to a target price
of $58 million in fiscal year 2010.
Agency Comments
In commenting on a draft of this assessment, the Navy stated that the V-22
Joint Program Office continues to execute a disciplined, event-driven test
and program schedule. It noted that since returning to flight in 2003, the
V-22 has flown over 4,000 hours, both in development and operational
tests. It also stated that the Block A V-22 has demonstrated reliability
and maintainability on par with fleet aircraft and that multiship sorties
and operations have been demonstrated for nearly all missions. It further
commented that the range and speed capability of the V-22 has spawned new
tactics and realized logistics efficiencies that will reduce time,
resources and save lives.
The Navy also stated that it remains committed to fielding a V-22 weapon
system when it is tested and ready and noted that a talented team of
government and industry professionals champions the transformational
capability that the V-22 brings and is committed to its success. It
further stated that the test and training programs will continue to ensure
operators and maintainers are ready and capable from day one to ensure the
warfighter has the best equipment with the best information.
Common Name: WGS
WGS is a joint Air Force and Army program intended to provide essential
communications services to U.S. warfighters, allies, and coalition
partners during all levels of conflict short of nuclear war. It is the
next generation wideband component in DOD's future Military Satellite
Communications architecture and is composed of the following principal
segments: space segment (satellites), terminal segment (users), and
control segment (operators). We assessed the space segment.
Source: WGS Program Office.
Development start/ production decision (11/00)
Design review (7/02)
GAO review (1/05) First satellite launch (12/05) Initial capability (2/07) Full
capability (12/11)
The WGS program's technology, design, and production are now mature.
Manufacturing problems did contribute to a delay in the launch of the
first WGS satellite by almost 2 years. The program office is increasing
its oversight of the contractor to help rectify these issues and believes
the problems have been resolved. A decision to procure the fourth and
fifth satellites is expected to add millions of dollars to the program's
cost, but the program office will not know the cost of these satellites
until it receives a proposal from the contractor.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development DOD Development/ GAO
start design production review
(NA) review decision (1/05)
(NA) (11/00)
Desired level of knowledge
Common Name: WGS
WGS Program
Technology Maturity
WGS has two technologies that are vital to program success: the digital
channelizer and the phased array antenna. According to program officials,
both technologies were mature when the program entered production in
November 2000.
Design Stability
The WGS design is essentially complete, as the program office has released
over 97 percent of the expected drawings to manufacturing. Last year we
reported that the contractor had problems integrating the antenna into the
satellite because experience the contractor expected to gain on commercial
satellite orders did not materialize. The integration problems have since
been resolved, and testing of the antenna engineering models demonstrated
that the design worked as required.
Production Maturity
Due to the commercial nature of the WGS acquisition contract, the program
office does not have access to production process control data. Despite
not being able to access these data to determine production maturity, unit
level manufacturing for WGS is essentially complete, as all units have
been manufactured and delivered for the first satellite. The contractor
continues to experience difficulties in manufacturing one of the
components of the phased array antenna, making the antenna production the
top risk to the program. Approximately 254 of these antenna components
were being built when thin cracks in the copper striplines were noticed
during inspection. An early analysis showed that poor handling procedures
of inexperienced personnel contributed to the cracks, and a screening test
revealed that inconsistencies in the thickness of the copper trace used to
build the striplines were also to blame. The contractor replaced all the
flawed striplines with properly manufactured parts and implemented
additional process controls. In resolving these production issues, program
officials stated that they inspected the manufacturing facilities,
reviewed test plans and procedures, started screening parts, and now hold
monthly program reviews with the contractor. Manufacturing problems with
the phased array antenna contributed to delaying the launch of the first
WGS satellite by almost 2 years. As a result of the delay, the Air Force
revised its acquisition strategy program baseline, which was approved in
February 2004.
Other Program Issues
In December 2002, DOD directed the addition of WGS satellites four and
five, with launch dates of fiscal years 2009 and 2010, respectively.
Therefore, the current contract options must be extended and renegotiated
to cover the cost of the likely 2-to 3-year production gap between
satellites three and four. The cost estimate for the additional satellites
has grown because of a greater than anticipated effect of parts
obsolescence and loss of manufacturing knowledge to be gained during the
production of the first three satellites. In addition, the production
costs of the first three satellites have been higher than expected. The
procurement of satellites four and five is expected to add millions of
dollars to the cost of the WGS program, but the exact amount will not be
known until the program office receives a proposal from the contractor.
Negotiation for the two satellites is to begin in the second half of
fiscal year 2006.
Agency Comments
In commenting on a draft of this assessment, the program office stated
that even though manufacturing process information is unavailable, it
believes the production knowledge of WGS is mature based upon similarities
to the contractor's commercial communications satellites. In addition, the
delays experienced in the delivery of the first satellite were primarily
due to inadequate adherence to manufacturing and quality assurance
standards at subcontractor facilities rather than production knowledge
immaturity.
Common Name: WIN-T
WIN-T is the Army's high-speed and high-capacity backbone communications
network. It is to provide reliable, secure, and seamless video, data,
imagery, and voice services, allowing users to communicate simultaneously
at various levels of security. The network is to have the ability to be
initialized and modified based upon unit task organization. It is to
connect Army units with higher levels of command and provide Army's
tactical portion of the Global Information Grid. WIN-T is being fielded in
blocks. We assessed the first block.
Source: PM WIN-T.
Program/ development start (7/03) GAO review (1/05)
Design review (9/05)
Low-rate decision (3/06)
Full-rate decision (5/09) Initial capability (1/10)
WIN-T entered system development with 3 of its 12 critical technologies
close to full maturity. None of the technologies will be fully mature at
the time production begins in March 2006. Eight have backup technologies
available, but only three of these are fully mature, and use of backup
technologies would degrade system overall robustness and capabilities. Due
to significant interdependencies among critical technologies, and the fact
that some determine network functionality, it may not be possible to
demonstrate that those technologies are fully mature until after
production begins. Design stability could not be assessed because the
program office does not track the number of releasable drawings. WIN-T is
primarily an information technology system integration effort rather than
a manufacturing effort.
Production, design & technology maturity
Design & technology maturity
Technology maturity
Development GAO DOD Production start review design decision (7/03) (1/05)
review (3/06) (9/05)
Common Name: WIN-T
WIN-T Program
Technology Maturity
WIN-T entered system development with 3 of its 12 critical technologies
close to reaching full maturity. While program officials do not expect
these technologies to reach full maturity until the network is built and
can be demonstrated in an operational environment, they do expect the
technologies to have been demonstrated in a simulated operational
environment by the time the critical design review is held in September
2005. An independent Army technology readiness assessment determined that
WIN-T would enter system development prior to full definition of the first
block's design and specific technology-based components, systems, or
subsystems. WIN-T will include technologies such as mobile and static
communications nodes, network operations and support centers, transmission
relays, joint gateway nodes, points of presence for future force and
command elements, vehicular wireless packages, airborne wireless
communication packages, and personal communications devices.
Design Stability
Design stability could not be assessed because the program office does not
plan to track the number of releasable drawings as a design metric.
According to the program, WIN-T is not a manufacturing effort, but
primarily an information technology system integration effort.
Consequently, the government does not obtain releasable design drawings
for many of WIN-T's components, particularly commercial components. The
WIN-T design will evolve using performance-based specifications and open
systems design, and it is to conform to DOD's Joint Technical
Architecture.
Other Program Issues
Among other issues, the program will need to pay close attention to the
interdependent nature of the WIN-T, FCS, and JTRS programs, the
interrelationship between WIN-T and FCS and Global Information Grid
requirements, the scalability of WIN-T, the challenge of linking all the
nodes and networks of the Army's system-ofsystems, and the coordination of
unmanned relay programs with FCS. The program will also have to track
external factors that will impact WIN-T such as the DOD Net-Centric Data
Strategy. WIN-T deployment will be essential for FCS deployment and as
each system evolves, integration demonstrations will need to be performed
to ensure WIN-T and FCS interoperability.
In addition, a major revision to the WIN-T acquisition strategy is
underway. WIN-T was originally envisioned to support the Army's Future
Force. However, the global war on terrorism and the lessons learned from
recent military operations have shifted the Army's focus toward providing
WIN-T capabilities sooner. To accomplish this, DOD, in September, approved
a decision to combine the competing contractor teams for WIN-T's system
design and development. The two originally competing contractors are now
teaming to establish a single architecture for WIN-T that, according to
the revised acquisition strategy, will leverage each contractor's proposed
architecture to provide the Army with a superior technical solution for
WIN-T. Establishing the single WIN-T architecture a year earlier than
originally planned is expected to allow other Army programs to begin to
follow that architecture for near-term force procurements and build on
that architecture for the Future Force.
Agency Comments
In commenting on a draft of this assessment, the Army noted that, as a
result of merging the two competing prime contractors under a new
acquisition strategy, the "best of breed" critical technologies will be
used in the updated WIN-T architecture. This new strategy is also expected
to increase the range of available technical products and developing
technologies, thereby lowering the risk of maturing critical technologies
for production and fielding. The Army also provided technical comments,
which were incorporated where appropriate.
Agency Comments and DOD did not provide general comments on a draft of
this report, but it did provide technical comments. These comments, along
with agency
Our Evaluation comments received on the individual assessments, were
included as appropriate. (See app. I for a copy of DOD's response.).
Scope of Our Review For the 54 programs, each assessment provides the
historical and current program status and offers the opportunity to take
early corrective action when a program's projected attainment of knowledge
diverges significantly from the best practices. The assessments also
identify programs that are employing practices worthy of emulation by
other programs. If a program is attaining the desired levels of knowledge,
it has less risk-but not zero risk-of future problems. Likewise, if a
program shows a gap between demonstrated knowledge and best practices, it
indicates an increased risk-not a guarantee-of future problems. The real
value of the assessments is recognizing gaps early, which provides
opportunities for constructive intervention-such as adjustments to
schedule, trade-offs in requirements, and additional funding-before cost
and schedule consequences mount.
We selected programs for the assessments based on several factors,
including (1) high dollar value, (2) stage in acquisition, and (3)
congressional interest. The majority of the 54 programs covered in this
report are considered major defense acquisition programs by DOD. A program
is defined as major if its estimated research and development costs exceed
$365 million or its procurement exceeds $2.19 billion in fiscal year 2000
constant dollars. (See app. II for details of the scope and methodology.)
We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretaries of the Army, Navy,
and Air Force; and the Director, Office of Management and Budget. We will
also make copies available to others upon request. In addition, the report
will be available at no charge on the GAO Web site at http://www.gao.gov.
If you have any questions on this report, please contact me at
(202) 512-4841 or Paul Francis at (202) 512-4841. Major contributors to
this report are listed in appendix IV.
Katherine V. Schinasi
Managing Director
Acquisition and Sourcing Management
Page 124 GAO-05-301 Assessments of Selected Major Weapon Programs
List of Congressional Committees
The Honorable John W. Warner Chairman The Honorable Carl Levin Ranking
Minority Member Committee on Armed Services United States Senate
The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye Ranking
Minority Member Subcommittee on Defense Committee on Appropriations United
States Senate
The Honorable Duncan Hunter Chairman The Honorable Ike Skelton Ranking
Minority Member Committee on Armed Services House of Representatives
The Honorable C. W. Bill Young Chairman The Honorable John P. Murtha
Ranking Minority Member Subcommittee on Defense Committee on
Appropriations House of Representatives
Appendix I
Comments from the Department of Defense
Page 126 GAO-05-301 Assessments of Selected Major Weapon Programs
Appendix II
Scope and Methodology
In conducting our work, we evaluated performance and risk data from each
of the programs included in this report. We summarized our assessments of
each individual program in two components-a system profile and a product
knowledge assessment. We did not validate the data provided by the
Department of Defense (DOD). However, we took several steps to address
data quality. Specifically, we reviewed the data and performed various
quality checks, which revealed some discrepancies in the data. We
discussed the underlying data and these discrepancies with program
officials and adjusted the data accordingly. We determined that the data
provided by DOD were sufficiently reliable for our engagement purposes,
after reviewing DOD's management controls for assessing data reliability.
Macro Analysis Data for major defense acquisition program research,
development, test, and evaluation (RDT&E) and procurement funding in
figure 1 were obtained from DOD's selected acquisition reports or from
data obtained directly from the program offices and then aggregated across
programs between fiscal year 1998 and fiscal year 2009. Data used to
assess the fiscal year 2005 RDT&E and procurement funding plan were drawn
from the 2003 selected acquisition reports or obtained directly from the
program office. For the Missile Defense Agency (MDA) programs for which a
baseline was not available, we used the latest available cost information.
To assess the total cost, schedule, and quantity changes of the programs
included in our assessment, it was necessary to identify those programs
with all of the requisite data available. Of the 54 programs in our
assessment, 26 programs constituted the common set of programs where data
were available for cost, schedule, and quantity at the first full
estimate, generally milestone B, and the latest estimate. Data utilized in
this analysis were drawn from information contained in selected
acquisition reports or data provided by program offices as of January 14,
2005. We summed the costs associated with RDT&E and total costs consisting
of research, development testing and evaluation, procurement, military
construction, and acquisition operation and maintenance. The data were
also used for a comparison between the 2004 assessment period and the 2005
assessment period. The schedule assessment is based on the change in the
average acquisition cycle time, defined as the number of months between
program start and the achievement of initial operational capability or an
equivalent fielding date.
The weighted calculations of acquisition cycle time and program
acquisition unit cost for the common set of programs were derived by
Appendix II Scope and Methodology
taking the total cost estimate for each of the 26 programs and dividing it
by the aggregate total cost of all 26 programs in the common set. The
resulting quotient for each program was then multiplied by the simple
percentage change in program acquisition unit costs to obtain the weighted
unit cost change of each program. Next, the sum of this weighted cost
change for all programs was calculated to get the weighted unit cost
change for the common set as a whole. To assess the weighted-average
acquisition cycle time change, we multiplied the weight calculation by the
acquisition cycle time estimate for each corresponding program. A simple
average was then taken to calculate the change between the first full
estimate and the latest estimate, and between the 2004 assessment period
and the 2005 assessment period. We believe these calculations best
represent the overall progress of programs by placing them within the
context of the common set's aggregate cost.
To assess the number of programs with technology maturity and design
stability at each critical juncture, we identified programs that had
actually proceeded through the start of development and the system design
review and obtained their assessed maturity. This information was drawn
from data provided by the program office as of January 14, 2005. For more
information, see the product knowledge assessment section in this
appendix.
System Profile In the past 4 years, DOD revised its policies governing
weapon system acquisitions and changed the terminology used for major
acquisition
Assessment events. To make DOD's acquisition terminology more consistent
across the 54 program assessments, we standardized the terminology for key
program events. In the individual program assessments, program start
refers to the initiation of a program; DOD usually refers to program start
as milestone I or milestone A, which begins the concept and technology
development phase. Similarly, development start refers to the commitment
to system development that coincides with either milestone II or milestone
B, which begins DOD's system development and demonstration phase. The
production decision generally refers to the decision to enter the
production and deployment phase, typically with low-rate initial
production. Initial capability refers to the initial operational
capability, sometimes also called first unit equipped or required asset
availability. For the MDA programs that do not follow the standard DOD
acquisition model, but instead develop systems in incremental
capability-based blocks, we identified the key technology development
efforts that lead to an initial capability for the block assessed.
Page 128 GAO-05-301 Assessments of Selected Major Weapon Programs
Appendix II Scope and Methodology
The information presented on the funding needed to complete from fiscal
2005 through completion, unless otherwise noted, draws on information from
selected acquisition reports or on data from the program office. In some
instances the data were not yet available, and we annotate this by the
term "to be determined" (TBD), or not applicable, annotated (NA). The
"Latest" program costs used in cost comparisons are the latest estimates
provided by the individual programs. The quantities listed only refer to
procurement quantities. Satellite programs, in particular, produce a large
percentage of their total operational units as development quantities,
which are not included in the quantity figure.
To assess the cost, schedule, and quantity changes of each program, we
reviewed DOD's selected acquisition reports or obtained data directly from
the program offices. In general, we compared the latest available selected
acquisition report information with a baseline for each program. For
systems that have started system development-those that are beyond
milestone II or B-we compared the latest available selected acquisition
report to the development estimate from the first selected acquisition
report issued after the program was approved to enter development. For
systems that have not yet started system development, we compared the
latest available data to the planning estimate issued after milestone I or
A. For systems not included in selected acquisition reports, we attempted
to obtain comparable baseline and current data from the individual program
offices. For MDA systems for which a baseline was not available we
compared the latest available cost information to the amount reported last
year.
All cost information is presented in base year 2005 dollars, unless
otherwise noted, using Office of the Secretary of Defense approved
deflators to eliminate the effects of inflation. We have depicted only the
programs' main elements of acquisition cost-research and development and
procurement; however, the total program costs also include military
construction and acquisition operation and maintenance costs. Because of
rounding and these additional costs, in some situations the total cost may
not match the exact sum of the research and development and procurement
costs. The program unit costs are calculated by dividing the total program
cost by the total quantities planned. These costs are often referred to as
program acquisition unit costs. In some instances, the data were not
applicable, and we annotate this by using the term "NA." In other
instances, the current absence of data on procurement funding and
quantities precludes calculation of a meaningful program acquisition unit
cost and we annotate this by using the term "TBD." The quantities listed
Appendix II Scope and Methodology
refer to total quantities, including both procurement and development
quantities.
The schedule assessment is based on acquisition cycle time, defined as the
number of months between the program start, usually milestone I or A, and
the achievement of initial operational capability or an equivalent
fielding date. In some instances, the data were not yet available, and we
annotate this by using the term TBD, or was classified.
The intent of these comparisons is to provide an aggregate or overall
picture of a program's history. These assessments represent the sum total
of the federal government's actions on a program, not just those of the
program manager and the contractor. DOD does a number of detailed analyses
of changes that attempt to link specific changes with triggering events or
causes. Our analysis does not attempt to make such detailed distinctions.
Product Knowledge Assessment
To assess the product development knowledge of each program at key points
in development, we submitted a data collection instrument to each program
office. The results are graphically depicted in each 2-page assessment. We
also reviewed pertinent program documentation, such as the operational
requirements document, the acquisition program baseline, test reports, and
major program reviews.
To assess technology maturity, we asked program officials to apply a tool,
referred to as technology readiness levels, for our analysis. The National
Aeronautics and Space Administration originally developed technology
readiness levels, and the Army and Air Force Science and Technology
research organizations use them to determine when technologies are ready
to be handed off from science and technology managers to product
developers. Technology readiness levels are measured on a scale of one to
nine, beginning with paper studies of a technology's feasibility and
culminating with a technology fully integrated into a completed product.
(See appendix III for the definitions of technology readiness levels.) Our
best practices work has shown that a technology readiness level of 7-
demonstration of a technology in an operational environment-is the level
of technology maturity that constitutes a low risk for starting a product
development program. In our assessment, the technologies that have reached
technology readiness level 7, a prototype demonstrated in an operational
environment, are considered mature and those that have reached technology
readiness level 6, a prototype demonstrated in a
Page 130 GAO-05-301 Assessments of Selected Major Weapon Programs
Appendix II Scope and Methodology
relevant environment, are assessed as attaining 50 percent of the desired
level of knowledge. Satellite technologies that have achieved technology
readiness level 6 are assessed as fully mature due to the difficulty of
demonstrating maturity in an operational environment-space.
In most cases, we did not validate the program offices' selection of
critical technologies or the determination of the demonstrated level of
maturity. We sought to clarify the technology readiness levels in those
cases where information existed that raised concerns. If we were to
conduct a detailed review, we might adjust the critical technologies
assessed, the readiness level demonstrated, or both. It was not always
possible to reconstruct the technological maturity of a weapon system at
key decision points after the passage of many years.
To assess design stability, we asked program officials to provide the
percentage of engineering drawings completed or projected for completion
by the design review, the production decision, and as of our current
assessment. In most cases, we did not verify or validate the percentage of
engineering drawings provided by the program office. We sought to clarify
the percentage of drawings completed in those cases where information
existed that raised concerns. Completed engineering drawings were defined
as the number of drawings released or deemed releasable to manufacturing
that can be considered the "build to" drawings.
To assess production maturity, we asked program officials to identify the
number of critical manufacturing processes and, where available, to
quantify the extent of statistical control achieved for those processes.
In most cases, we did not verify or validate this information provided by
the program office. We sought to clarify the number of critical
manufacturing processes and percentage of statistical process control
where information existed that raised concerns. We used a standard called
the Process Capability Index, which is a process performance measurement
that quantifies how closely a process is running to its specification
limits. The index can be translated into an expected product defect rate,
and we have found it to be a best practice. We sought other data, such as
scrap and rework trends, in those cases where quantifiable statistical
control data were unavailable.
Appendix II Scope and Methodology
Although the knowledge points provide excellent indicators of potential
risks, by themselves, they do not cover all elements of risk that a
program encounters during development, such as funding instability. Our
detailed reviews on individual systems normally provide for a fuller
treatment of risk elements.
Page 132 GAO-05-301 Assessments of Selected Major Weapon Programs
Appendix III
Technology Readiness Levels
Hardware Demonstration Technology Readiness Level Description Software
Environment
1. Basic principles observed and Lowest level of technology readiness.
None (Paper studies and None
reported. Scientific research begins to be translated analysis) into
applied research and development. Examples might include paper studies of
a technology's basic properties
2. Technology concept Invention begins. Once basic None (Paper None
and/or principles studies and
application are observed, practical analysis)
formulated. applications can
be invented. The application is
speculative and there is no proof
or
detailed analysis to support the
assumption. Examples are still
limited to
paper studies.
3. Analytical and experimental Active research and development is
Analytical studies and Lab critical function and/or initiated. This
includes analytical studies demonstration of nonscale characteristic proof
of concept. and laboratory studies to physically individual components
validate analytical predictions of separate (pieces of subsystem).
elements of the technology. Examples include components that are not yet
integrated or representative.
4. Component and/or breadboard. Basic technological components are Low
fidelity breadboard. Lab Validation in laboratory integrated to establish
that the pieces will Integration of nonscale environment. work together.
This is relatively "low components to show
fidelity" compared to the eventual system. pieces will work together.
Examples include integration of "ad hoc" Not fully functional or form
hardware in a laboratory. or fit but representative of
technically feasible approach suitable for flight articles.
5. Component and/or breadboard validation in relevant environment.
Fidelity of breadboard technology increases significantly. The basic
technological components are integrated with reasonably realistic
supporting elements so that the technology can be tested in a simulated
environment. Examples include "high fidelity" laboratory integration of
components.
High fidelity breadboard. Functionally equivalent but not necessarily form
and/or fit (size weight, materials, etc.). Should be approaching
appropriate scale. May include integration of several components with
reasonably realistic support elements/subsystems to demonstrate
functionality.
Lab demonstrating functionality but not form and fit. May include flight
demonstrating breadboard in surrogate aircraft. Technology ready for
detailed design studies.
Appendix III Technology Readiness Levels
(Continued From Previous Page)
Hardware Demonstration Technology Readiness Level Description Software
Environment
6. System/subsystem model or Representative model or prototype
Prototype-Should be very High-fidelity lab prototype demonstration in a
system, which is well beyond the close to form, fit and demonstration or
relevant environment. breadboard tested for TRL 5, is tested in a
function. Probably includes limited/restricted flight
relevant environment. Represents a major the integration of many
demonstration for a
step up in a technology's demonstrated new components and relevant
environment.
readiness. Examples include testing a realistic supporting Integration of
technology is
prototype in a high fidelity laboratory elements/subsystems if well
defined.
environment or in simulated operational needed to demonstrate full
environment. functionality of the
subsystem.
7. System prototype demonstration Prototype near or at planned operational
Prototype. Should be form, Flight demonstration in
in an operational environment. system. Represents a major step up from fit
and function integrated representative operational TRL 6, requiring the
demonstration of an with other key supporting environment such as flying
actual system prototype in an operational elements/subsystems to test bed
or demonstrator environment, such as in an aircraft, demonstrate full
aircraft. Technology is well vehicle or space. Examples include
functionality of subsystem. substantiated with test testing the prototype
in a test bed aircraft. data.
8. Actual system Technology has been proven to Flight DT&E in the
completed and work in its qualified actual system
hardware
"flight qualified" final form and under expected application
through test and conditions.
demonstration. In almost all cases, this TRL
represents
the end of true system
development.
Examples include developmental
test and
evaluation of the system in
its intended
weapon system to determine if
it meets
design specifications.
9. Actual system "flight proven" Actual application of the technology in
its Actual system in final form OT&E in operational through successful
mission final form and under mission conditions, mission conditions
operations. such as those encountered in operational
test and evaluation. In almost all cases, this is the end of the last "bug
fixing" aspects of true system development. Examples include using the
system under operational mission conditions.
Source: GAO and its analysis of National Aeronautics and Space
Administration data.
Page 134 GAO-05-301 Assessments of Selected Major Weapon Programs
Appendix IV
GAO Contact and Acknowledgments
GAO Contact Paul L. Francis (202) 512-4841
Acknowledgments David B. Best, Alan R. Frazier, and Bruce H. Thomas made
key contributions to this report. Other key contributors included Robert
L. Ackley, D. Catherine Baltzell, Maricela Cherveny, Tana M. Davis, Thomas
J. Denomme, Arthur Gallegos, William R. Graveline, David J. Hand, Michael
J. Hazard, Barbara H. Haynes, Leslie M. Hickey, John E. Oppenheim,
Maria-Alaina I. Rambus, Nancy Rothlisberger, Rae Ann H. Sapp, James L.
Morrison, Wendy P. Smythe, Sharon E. Sweeney, Robert S. Swierczek, and
Karen S. Zuckerstein. The following staff were responsible for individual
programs:
System Primary Staff
Airborne Laser (ABL) LaTonya D. Miller
Aegis Ballistic Missile Defense (Aegis BMD) Randolph S. Zounes
Advanced Extremely High Frequency Satellites Bradley L. Terry/Lisa P.
Gardner (AEHF)
Active Electronically Scanned Array Radar Joseph E. Dewechter/Jerry W.
Clark (AESA)
Airborne Mine Neutralization System (AMNS) Ian A. Ferguson/Brendan S.
Culley/ Angela D. Thomas
Advanced Precision Kill Weapon System John S. Warren/Thomas L. Gordon/
(APKWS) Michele R. Williamson
Advanced SEAL Delivery System (ASDS) Mary K. Quinlan
Advanced Threat Infrared Jonathan E. Watkins/ Danny G. Owens
Countermeasure/Common Missile Warning System (ATIRCM/CMWS)
B-2 Radar Modernization Program (B-2 RMP) Don M. Springman/Arthur L. Cobb
C-130 Avionics Modernization Program Dayna L. Foster/Christopher A.
Deperro (C-130 AMP)
C-5 Avionics Modernization Program (C-5 Cheryl K. Andrew/Sameena N.
Ismailjee AMP)
C-5 Reliability Enhancement and Reengining Sameena N. Ismailjee/Cheryl K.
Andrew Program (C-5 RERP)
Cooperative Engagement Capability (CEC) Johana R. Ayers/W. William Russell
CH-47F Improved Cargo Helicopter (CH-47F) Wendy P. Smythe/ Leon S. Gill
Compact Kinetic Energy Missile (CKEM) Marcus C. Ferguson/Wendy P. Smythe
Appendix IV
GAO Contact and Acknowledgments
(Continued From Previous Page)
System Primary Staff
Future Aircraft Carrier (CVN-21) Brendan S. Culley/Trevor J. Thomson
DD(X) Destroyer J. Kristopher Keener/Angela D. Thomas
E-10A Multi-Sensor Command and Control Rae Ann H. Sapp/David R. Schilling
Aircraft (E-10A)
E-2 Advanced Hawkeye (E-2 AHE) Gary L. Middleton/Bruce H. Thomas
EA-18G (EA-18G) Christopher R. Miller/Brian T. Mullins
Evolved Expendable Launch Vehicle (EELV) Maria A. Durant/Maricela Cherveny
Expeditionary Fighting Vehicle (EFV) Alan R. Frazier/Ronald E. Schwenn
Extended Range Guided Munition (ERGM) Shelby S. Oakley/Ronald E. Schwenn/
Margaret B. McDavid
Excalibur Precision Guided Extended Range Lawrence D. Gaston/John P. Swain
Artillery Projectile
F/A-22 Raptor Marvin E. Bonner/Arthur L. Cobb
Future Combat Systems (FCS) John P. Swain/ Lawrence D. Gaston/ Marcus C.
Ferguson
Global Hawk Unmanned Aerial Vehicle Bruce D. Fairbairn/Steven M. Hunter
Ground-Based Midcourse Defense (GMD) Ivy G. Hubler
Global Positioning System II (GPS II) Jean N. Harker/Michael L. Gorin
Heavy Lift Replacement (HLR) Brian T. Mullins/Wesley A. Johnson
Joint Air-to-Surface Standoff Missile (JASSM) Beverly A. Breen/Carrie R.
Wilson
Joint Common Missile Danny G. Owens/Jonathan E. Watkins
Joint Strike Fighter (JSF) Matthew B. Lea/David R. Schilling
Joint Standoff Weapon (JSOW) Carol T. Mebane/Bradley J. Trainor
Joint Tactical Radio System (JTRS) Cluster 1 Ridge C. Bowman/James P.
Tallon
Joint Tactical Radio System (JTRS) Cluster 5 Subrata Ghoshroy/ Paul G.
Williams
Joint Unmanned Combat Air Systems Bruce D. Fairbairn/Matthew T. Drerup
(J-UCAS)
Kinetic Energy Interceptors (KEI) Randolph S. Zounes
Land Warrior Joel C. Christenson/Candice N. Wright
Littoral Combat Ship (LCS) J. Kristopher Keener/Angela D. Thomas
Medium Extended Air Defense System Tana M. Davis (MEADS)
Multi-mission Maritime Aircraft (MMA) Matthew F. Ebert/Ronald E. Schwenn/
Heather L. Barker
Mobile User Objective System (MUOS) Richard Y. Horiuchi/Tony A. Beckham
MQ-9 Predator B Steven M. Hunter/Travis J. Masters
National Polar-orbiting Operational Suzanne S. Olivieri/ Carol R. Cha/
Environmental Satellite System (NPOESS) James P. Tallon
Space Based Infrared System High Nancy Rothlisberger/Maricela Cherveny
(SBIRS High)
Page 136 GAO-05-301 Assessments of Selected Major Weapon Programs
Appendix IV
GAO Contact and Acknowledgments
(Continued From Previous Page)
System Primary Staff
Small Diameter Bomb (SDB) Carrie R. Wilson/ Beverly A. Breen
Space Tracking and Surveillance System Sigrid L. McGinty/Tony A. Beckham
(STSS)
Terminal High Altitude Area Defense (THAAD) William S. Lipscomb
Tactical Tomahawk Missile Bradley J. Trainor/Carol T. Mebane
Transformational Satellite Communications Arturo Holguin Jr./Travis J.
Masters System (TSAT)
V-22 Joint Services Advanced Vertical Lift Jerry W. Clark/Bonita P. Oden
Aircraft (V-22)
Wideband Gapfiller Satellites (WGS) Tony A. Beckham/Richard Y. Horiuchi
Warfighter Information Network-Tactical James P. Tallon/Ridge C. Bowman
(WIN-T)
Source: GAO.
Related GAO Products
Defense Acquisitions: Stronger Management Practices Are Needed to
Improve DOD's Software-Intensive Weapon Acquisitions. GAO-04-393.
Washington, D.C.: March 1, 2004.
Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices,
but More Controls Are Needed. GAO-04-53. Washington, D.C.:
November 10, 2003.
Best Practices: Setting Requirements Differently Could Reduce
Weapon Systems' Total Ownership Costs. GAO-03-57. Washington, D.C.:
February 11, 2003.
Best Practices: Capturing Design and Manufacturing Knowledge
Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.:
July 15, 2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead
to Better Weapon System Outcomes. GAO-01-288. Washington, D.C.:
March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better
Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.:
July 31, 2000.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon System
Program Implement Best Practices. GAO/NSIAD-99-206. Washington, D.C.:
August 16, 1999.
Best Practices: Better Management of Technology Development Can
Improve Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.:
July 30, 1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program
Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.
Page 138 GAO-05-301 Assessments of Selected Major Weapon Programs
Related GAO Products
Defense Acquisition: Improved Program Outcomes Are Possible.
GAO/T-NSIAD-98-123. Washington, D.C.: March 18, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD's Environment. GAO/NSIAD-98-56. Washington, D.C.:
February 24, 1998.
Major Acquisitions: Significant Changes Underway in DOD's
Earned Value Management Process. GAO/NSIAD-97-108. Washington, D.C.:
May 5, 1997.
Best Practices: Commercial Quality Assurance Practices Offer
Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.:
August 26, 1996.
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
Presorted Standard
Postage & Fees Paid
GAO
Permit No. GI00
United States
Government Accountability Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Service Requested
*** End of document. ***