Defense Acquisitions: Assessments of Major Weapon Programs
(15-MAY-03, GAO-03-476).
The weapons the Department of Defense (DOD) develops have no
rival in superiority. How they are developed can be improved,
without sacrificing the superiority of the outcome. GAO's reviews
over the past 20 years have found consistent problems with weapon
investments--cost increases, schedule delays and performance
shortfalls--along with underlying causes, such as pressure on
managers to promise more than they can deliver. The best
practices of successful product developments offer a
knowledge-based approach DOD can use to improve the way it
develops new weapons. This report is new for GAO, and draws on
its work in best practices for product development. GAO's goal
for this report is to provide congressional and DOD decision
makers with an independent, knowledge-based assessment of defense
programs that identifies potential risks, and offers an
opportunity for action when a program's projected attainment of
knowledge diverges from the best practice. It can also highlight
those programs that employ practices worthy of emulation by other
programs. GAO plans to update and issue this report annually to
the congressional defense committees.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-476
ACCNO: A06848
TITLE: Defense Acquisitions: Assessments of Major Weapon
Programs
DATE: 05/15/2003
SUBJECT: Best practices
Program evaluation
Strategic planning
Weapons
Weapons research and development
Weapons systems
Defense operations
Army Future Combat Systems
F-22 Aircraft
Joint Strike Fighter
National Polar-Orbiting Operational
Environmental Satellite System
Patriot Missile Advanced
Capability-Three Upgrade
SDI Theater High Altitude Area Defense
System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-476
A
Report to Congressional Committees
May 2003 DEFENSE ACQUISITIONS Assessments of Major Weapon Programs
GAO- 03- 476
Foreword iii Letter 1
Appendixes A Knowledge- Based Approach Can Lead to Better Acquisition
Outcomes 2 Observations 5 Agency Comments 7 Scope of Our Review 7
Appendix I: Assessments of Individual Programs 9 Introduction 9
Advanced Amphibious Assault Vehicle (AAAV) 15 Airborne Laser (ABL) 17
Advanced Extremely High Frequency (AEHF) Communications Satellite 19
AN/ APG- 79 Active Electronically Scanned Array (AESA) Radar 21 AIM- 9X
Short- Range Air- to- Air Missile 23 Advanced Threat Infrared
Countermeasures/ Common Missile Warning System (ATIRCM/ CMWS) 25
Advanced Wideband Satellite (AWS) 27 Cooperative Engagement Capability
(CEC) 29 CH- 47F Improved Cargo Helicopter 31 RAH- 66 Comanche 33 EX- 171
Extended Range Guided Munition (ERGM) 35 Excalibur Artillery Round 37 F/
A- 18E/ F Super Hornet 39 F/ A- 22 Raptor 41 Joint Air- to- Surface
Standoff Missile (JASSM) 43 Joint Common Missile 45 Joint Primary Aircraft
Training System (JPATS) 47 F- 35 Joint Strike Fighter (JSF) 49 Joint
Standoff Weapon (JSOW) 51 National Polar- orbiting Operational
Environmental Satellite System (NPOESS) 53
Patriot Advanced Capability 3 (PAC- 3) Program 55 Space Based Infrared
System (SBIRS) High 57 Theater High Altitude Area Defense (THAAD) 59
Tactical Tomahawk Missile 61 V- 22 Osprey 63 Wideband Gapfiller Satellite
(WGS) Communications System 65
Appendix II: Methodology 67 System Profile Assessment 67 Product Knowledge
Assessment 68
Appendix III: GAO Contact and Acknowledgments 71 Figures Figure 1:
Knowledge Build at Key Points in Product Development
Reduces the Risk of Unknowns 4 Figure 2: Graphic Depiction of Best
Practices for Technology, Design, and Production Knowledge 10
Figure 3: Depiction of a Notional Weapon System Program*s Knowledge as
Compared with Best Practices 12
Abbreviations
DOD Department of Defense GAO General Accounting Office USAF United States
Air Force USN United States Navy USMC United States Marine Corps
This is a work of the U. S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce copyrighted
materials separately from GAO*s product.
Congressional Committees systems. May 15, 2003 Foreword
Recent military operations in Iraq have soundly demonstrated the
superiority of United States military capabilities. The Department of
Defense (DOD) develops weaponry that is unmatched in levels of
technological sophistication and lethality. Despite their superiority,
weapon systems routinely take much longer to field, cost more to buy, and
require more support than investment plans provide for. In a constrained
funding environment, unforeseen cost growth in weapon systems forecloses
other investment choices for the government, both within and outside of
DOD. DOD*s investment in major weapon systems is expected to grow
considerably in the future as DOD works to keep legacy systems while
investing in future capabilities such as unmanned aircraft, satellite
networks, and information communication systems. For example, the
investment in weapons from fiscal years 2003 through 2009 will exceed $1
trillion. Such an investment clearly requires DOD to be as efficient and
effective as possible in the development and acquisition of weapon In the
last several years, we have undertaken a body of work that examines weapon
acquisition issues from a different, more cross- cutting perspective* one
that draws lessons learned from best product development practices to see
if they apply to weapon system development. We found that programs managed
with a knowledge- based approach*
where product knowledge is demonstrated at critical points in a
development cycle* place themselves on a low- risk path to production.
These programs are more likely to be executed within cost and schedule
estimates. We believe that by employing this approach, DOD can still field
superior weapons without attendant cost and schedule growth.
This report is a new product for GAO. It provides decision makers with a
snapshot of program performance and risk and is focused on each system*s
developmental progress vis a vis best practices. Each assessment is
summarized in an easy to read, visually descriptive 2- page format that
provides a fact- based analysis of each program*s cost, schedule, and
development status. We plan to issue this report annually in early spring,
and we intend to increase the number of systems reviewed each year. We
have briefed numerous committee staff on the product and received
positive feedback regarding the report*s utility and breath of coverage.
The continuing war on terrorism, regional instability, the challenge of
transforming the military, as well as the federal government*s short- and
long- term budget pressures have created a challenging environment for
DOD. It faces a number of difficult missions that will put its strategies
and resources under enormous strain. Consequently, it is important that
weapon systems be acquired using a knowledge- based approach to ensure
that their development is within cost, schedule, and performance
parameters. We believe that this report can provide useful insights on key
risks in development, allow decision makers to take corrective actions,
and thereby place programs in a better position to succeed.
David M. Walker Comptroller General of the United States
May 15, 2003 Letter
Congressional Committees The Department of Defense (DOD) is on the
threshold of several major investments in programs that are likely to
dominate budget and doctrinal debates well into the next decade. These
programs include, among others, the Missile Defense Agency*s suite of
land, sea, air, and space defense systems; the Army*s Future Combat
System; and the Air Force and Navy*s Joint Strike Fighter. In fiscal year
2003, the Congress appropriated $127 billion to DOD for the research,
development, and procurement of
weapon systems. Funding for weapon systems is projected to continue
growing to $182 billion in fiscal year 2009* an increase of over 43
percent. In total, the investment in weapons from fiscal years 2003
through 2009 will exceed $1 trillion. Thus, it is essential that sound
foundations for these and other weapon system investments be laid now so
that the resulting
programs can be executed within estimates of available resources. The
challenge of putting new programs on a better footing than their
predecessors is a daunting one. Clearly, the acquisition process produces
superior weapons. But it does so at a high price. Weapon systems routinely
take much longer to field, cost more to buy, and require more support than
investment plans provide for. These consequences reduce the buying power
of the defense dollar, delay capabilities for the warfighter, and force
unplanned* and possibly unnecessary* trade- offs among programs.
DOD has undertaken a number of acquisition reforms over the preceding two
decades in response to those problems, but while there have been
individual successes, these reforms have not yet yielded consistent
improvements in program outcomes. More recently, DOD leadership has
embraced an evolutionary acquisition approach, coupled with time- phased
requirements. This approach supports developing weapons in smaller, more
predictable iterations of increasing capabilities, rather than the past
approach of attempting to achieve a weapon*s maximum capability in one
design leap. DOD is also striving to give programs, such as missile
defense, more flexibility to make trade- offs between cost, schedule, and
performance that can lead to better investment decisions. It is also
currently looking at how to revise its planning, programming, and
budgeting process that has been in place for over 40 years.
Key to any effort to improve weapon system outcomes is using the lessons
that can be learned from the best practices of successful commercial and
defense product development programs. We have found that these practices
can be collectively described as a knowledge- based approach whose success
depends on the timely attainment and use of a product*s technology,
design, and production maturity. In this report, we compare the knowledge
gained on 26 DOD weapon system programs with best practices. Our objective
is to provide decision makers a means to quickly gauge the progress and
potential risks* based on demonstrated knowledge* of the individual weapon
system programs. A Knowledge- Based
All product development efforts, whether for a car, a plane, a missile, or
a Approach Can Lead to
satellite, go through a process of building knowledge. Ultimately, this
process brings together and integrates all of the technologies,
components, Better Acquisition
and subsystems needed for the product to work and to be reliably Outcomes
manufactured. The product development process can be characterized as the
reduction of risk and the resolution of unknowns through the attainment of
knowledge.
About 7 years ago, at the request of the Senate Committee on Armed
Services, we began an extensive body of work identifying best practices in
product development, both in DOD and in the commercial sector. Of
particular interest were cases in which increasingly sophisticated
products were being developed in significantly less time and at lower cost
than their predecessors. A major reason for these successes was the use of
a product development process that was anchored in knowledge. Product
developers employed specific practices to ensure that a high level of
knowledge regarding critical facets of the product was achieved at key
junctures in development. We have characterized these junctures as three
knowledge points. We have also identified key indicators that can be used
to assess the attainment of each knowledge point. When tied to major
events on a program*s schedule, they can disclose whether gaps or
shortfalls exist in demonstrated knowledge, which can presage future cost,
schedule, and performance problems. These knowledge points and associated
indicators are defined as follows.
Knowledge point 1: Resources and needs are matched. This level of
knowledge is attained when a match is made between a customer*s needs and
the developer*s technical, financial, and other resources.
Technology maturity is a particularly important indicator of resource
availability. A best practice is to achieve a high level of technology
maturity at the start of product development. This means that the
technologies needed to meet essential product requirements have been
demonstrated to work in their intended environment. Knowledge point 2:
The product design is stable. This level of
knowledge is attained when the product*s design demonstrates its ability
to meet the customer*s requirements. A best practice is to achieve design
stability at the system- level critical design review, usually held midway
through development. Completion of engineering drawings at the system
design review provides tangible evidence that the design is stable.
Knowledge point 3: Production processes are mature. This level of
knowledge is attained when it is demonstrated that the product can be
manufactured within cost, schedule and quality targets. A best practice is
to achieve production maturity at the start of production. This means
that all key manufacturing processes produce output within statistically
acceptable limits for quality. As illustrated in figure 1, the process is
building block in nature as the
attainment of each successive knowledge point builds on the proceeding
one. While the knowledge itself builds continuously without clear lines of
demarcation, the attainment of knowledge points is sequential. In other
words, production maturity cannot be attained if the design is not mature,
and design maturity cannot be attained if the key technologies are not
mature.
Figure 1: Knowledge Build at Key Points in Product Development Reduces the
Risk of Unknowns
For the most part, all three knowledge points are eventually attained on a
completed product. The difference between highly successful product
developments* those that deliver superior products within cost and
schedule projections* and problematic product developments is how this
knowledge is built and how early in the development cycle each knowledge
point is attained. When knowledge is built more slowly than these points
suggest, less knowledge is on hand at key decisions or events, such as the
decisions to start a development program, hold the critical design review,
and start production. This invites greater cost, schedule, and performance
risks because (1) problems are more likely to be discovered late in the
process and will therefore be more difficult and costly to correct and (2)
a variety of pressures encourage program managers to underestimate the
difficulties.
It is important to note that successful product developers treat
technology development as a different and separate effort that precedes
product development. This treatment of technology development is key to
reaching the first knowledge point at the start of product development, as
it is a prerequisite for capturing design and production knowledge early
in product development. This approach to attaining knowledge puts program
managers* and programs* in a better position to succeed.
Observations When programs proceed with less knowledge than suggested by
best practices, cost, schedule, and performance problems often result. To
varying degrees, all the programs we assessed proceeded with lower levels
of knowledge at critical junctures and thus attained key elements of
product knowledge later in development. In some programs, the consequences
of proceeding with early knowledge deficits have already been felt. For
example:
The F- 22 Fighter began product development with key technologies
immature* deferring knowledge point 1* and subsequently had only a quarter
of the desired amount of engineering drawings completed at the critical
design review* deferring knowledge point 2. The program has experienced
substantial cost increases and schedule delays in the latter stages of
development.
The Patriot Advanced Capability missile also reached knowledge points 1
and 2 later than best practices. The seeker technology did not demonstrate
maturity until close to the production decision and the design remains
unstable. Each seeker still needs to be reworked about 3 times on average
before it passes quality inspections. The cost of the
seeker has increased by 76 percent and contributed to a 2- year delay in
the program*s schedule. The Extended Range Guided Munition program began
with only one of
its 20 critical technologies mature* deferring knowledge point 1. While
progress has been made, program officials do not expect to achieve
maturity on all technologies until after the design review. The lack of
mature technologies contributed to subsequent test failures, cost
increases, and schedule delays.
If programs attain more knowledge as suggested by best practices, they are
in a better position to succeed in meeting cost, schedule, and performance
expectations. We found some programs that did attain key product knowledge
earlier than most. For example:
The National Polar- orbiting Operational Environmental Satellite System
program ensured that its pacing technologies were demonstrated before
committing to product development. The program plans to demonstrate three
critical sensors on a demonstrator satellite prior to their inclusion on
the new satellite.
The Theater High Altitude Area Defense System made significant strides
in product development, following a problematic preliminary development
phase. In 2000, we reported that the program*s delayed demonstration of
technologies and components and reliance on fullsystem
testing to discover problems, was a very costly method to mature the
system*s design and nearly caused the cancellation of the program. The
program has since structured a product development phase that places a
much greater emphasis on early demonstration of components, a testing
program that incorporates sufficient time between tests for learning, and
a plan to achieve design stability by
releasing 90 percent of engineering drawings by the time of the critical
design review* knowledge point 2.
In general, we found that the greatest absence of knowledge was in the
area of production. Almost no programs collected statistical process
control data, the indicator for production maturity. Unlike technology
readiness levels, which can be applied at any time, and engineering
drawing release data, which is captured on all programs, few programs
collected statistical process control data. While the absence of this data
does not necessarily mean that production processes were immature,
attained knowledge could not be assessed against an objective standard.
Other indicators of production maturity, such as scrap and rework rates,
can indicate positive trends, but are not prospective* that is, they are
not useful in guiding preparations for production. To some extent,
statistical process control data is not being collected because DOD has
been delegating more responsibility to prime contractors and reducing the
amount of data requested. The lack of such data may put program offices in
a disadvantaged position to gain insights about a contractor*s production
progress. We have recently issued a report that recommends that DOD
collect statistical process control data on its weapon system programs and
DOD has agreed with this recommendation. 1
We conducted our review from September 2002 through May 2003 in accordance
with generally accepted government auditing standards.
1 U. S. General Accounting Office, Best Practices: Capturing Design and
Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO- 02- 701
(Washington, D. C.: July 15, 2002).
Agency Comments DOD did not provide general comments on a draft of this
report, but did provide technical comments on individual assessments.
These comments,
along with program office comments, are included with each individual
assessment as appropriate.
Scope of Our Review We selected programs for the assessments based on
several factors, including (1) high dollar value, (2) stage in
acquisition, and (3)
congressional interest. The majority of the 26 programs covered in this
report are considered major defense acquisition programs by DOD. A program
is defined as major if its estimated research and development costs exceed
$365 million or its procurement exceeds $2.19 billion in fiscal year 2000
constant dollars. We plan to include more programs in subsequent years,
with a greater
focus on programs early enough in development that the assessments can be
used to improve the program*s prospects for success, and issue this report
annually to the congressional defense committees. The individual
assessment of each program can be found in appendix I. Appendix II
contains detailed information on our methodology.
We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretaries of the Army, Navy,
and Air Force; and the Director, Office of Management and Budget. We will
also make copies available to others upon request. In addition, the report
will be available at no charge on the GAO Web site at http:// www. gao.
gov. If you have any questions on this report, please contact me at (202)
512- 4841 or Paul Francis at (202) 512- 2811. Major contributors to this
report are
listed in appendix III. Jack L. Brock, Managing Director Acquisition and
Sourcing Management
List of Congressional Committees
The Honorable John W. Warner Chairman The Honorable Carl Levin Ranking
Member Committee on Armed Services United States Senate The Honorable Ted
Stevens Chairman The Honorable Daniel K. Inouye Ranking Member
Subcommittee on Defense Committee on Appropriations United States Senate
The Honorable Duncan Hunter Chairman The Honorable Ike Skelton Ranking
Minority Member Committee on Armed Services House of Representatives The
Honorable Jerry Lewis Chairman The Honorable John P. Murtha Ranking
Minority Member Subcommittee on Defense Committee on Appropriations House
of Representatives
Appendi Appendi xes x I
Assessments of Individual Programs Introduction For the 26 programs, each
assessment provides the historical and current program status and offers
the opportunity to take early corrective action when a program*s projected
attainment of knowledge diverges significantly from the best practices.
The assessments also identify programs that are employing practices worthy
of emulation by other programs. If a program is attaining the desired
levels of knowledge, it has less risk* but not zero risk* of future
problems. Likewise, if a program shows a gap between demonstrated
knowledge and best practices, it indicates an increased risk* not a
guarantee* of future problems. The real value of the assessments is
recognizing gaps early, which provides opportunities for constructive
intervention* such as adjustments to schedule, trade- offs in
requirements, and additional funding* before cost and schedule
consequences mount.
Our assessment of each program is summarized in two components*( 1) a
system profile and (2) a product knowledge assessment.
The system profile presents a general description of the product in
development; a picture of the product or a key element of the product; a
schedule timeline identifying key dates in the program; a table
identifying the prime contractor; the program office location, and the
fiscal year 2004 requested funding if available; and a table summarizing
the cost, schedule and quantity changes to the program.
The rest of the assessment analyzes the extent to which product knowledge
at the three key knowledge points has been attained. We depict the extent
of knowledge in a stacked bar graph and provide a narrative summary at the
bottom of the first page. The second page is devoted to a narrative
assessment of technology, design and production maturity, as well as other
program issues identified and comments from the program office.
The product knowledge figure is based on the three knowledge points and
the key indicators for the attainment of knowledge. A *best practice* line
is drawn based on the ideal attainment of the three types of knowledge at
the three knowledge points (see fig. 2).
Figure 2: Graphic Depiction of Best Practices for Technology, Design, and
Production Knowledge
The first major point on the best practice line represents two facts: a
commitment to a new product development has been made and the key
technologies needed for the new product are mature.
When all critical technologies have reached a technology readiness level
7, technology maturity* and thus knowledge point 1* has been attained. In
our assessment, the technologies that have reached technology readiness
level 7, a prototype demonstrated in an operational environment, are
considered mature and those that reach technology readiness level 6, a
prototype demonstrated in a relevant environment, are assessed as
attaining 50 percent of the desired level of knowledge. Satellite
technologies that achieved technology readiness level 6 were assessed as
fully mature due to the difficulty of demonstrating maturity in an
operational environment* space. (Technology readiness levels are more
fully explained in appendix II.) The second major point on the best
practice line captures technology maturity plus design maturity* knowledge
point 2. A design is considered mature when 90 percent of the engineering
drawings have been released or deemed releasable to manufacturing. In the
successful programs we have studied, design maturity is attained about
halfway through the product development phase. The third major point on
the best practice line captures the sum of technology maturity, design
maturity, and production maturity. Production is considered mature when
all key production processes are in statistical control. Ideally, this
occurs before the first products for delivery to the customer are
manufactured. As can be seen, knowledge about the technology, design, and
production of a
new product builds over time. While the knowledge itself builds
continuously without clear lines of demarcation, the attainment of
knowledge points is sequential. In other words, production maturity cannot
be attained if the design is not mature, and design maturity cannot be
attained if the key technologies are not mature.
Data for a given weapon system program is then plotted against the best
practices line. In the assessments that follow, a brown bar indicates the
technology knowledge attained by a weapon system program. The actual
technology readiness levels attained for a program*s key technologies are
measured at the start of development* normally milestone II or milestone B
in the Department of Defense*s (DOD) acquisition process. The closer a
program*s attained knowledge is to the best practice line, the more likely
the weapon will be delivered within its estimated cost and schedule. A
knowledge deficit at this point* indicated by a gap between the technology
knowledge attained by the weapon system and the best practices line* means
the program proceeded with immature technologies and may face a greater
likelihood of cost and schedule increases as technology risks are
discovered and resolved. A green bar indicates the design knowledge
attained by a weapon system program. This is calculated by measuring the
percent of engineering drawings released to manufacturing. The green bar
is stacked on top of the brown bar to indicate whether any cumulative gap*
considering both technology and design* exists at the halfway point
of product development. A blue bar indicates the production knowledge
attained by a weapon system program. This is calculated by measuring the
percentage of key production processes in statistical control. The blue
bar is stacked on top of the brown and green bars to indicate whether any
cumulative technology, design, and production gaps exist at the time
production begins. In some cases, we obtained projections from the program
office of future knowledge attainment. These projections are depicted as
dashed bars.
Figure 3 depicts an example of an assessment for a notional weapon system.
Figure 3: Depiction of a Notional Weapon System Program*s Knowledge as
Compared with Best Practices
An interpretation of this notional example would be that the product
development began with key technologies immature, thereby missing
knowledge point 1. Knowledge point 2 was not attained at the design review
as some technologies were still not mature and only a small percentage of
engineering drawings had been released. Projections for the
production decision show that the program is expected to achieve a greater
level of maturity, but will still fall short. It is likely that this
program would have had significant cost and schedule increases.
We also found three situations in which programs were unable to provide
key knowledge indicators. We used three types of labels in the knowledge
figures to depict those situations. Programs with these labels are
distinguished from those that have elected not to collect data that can be
used to assess progress against best practices. First, a few programs are
planning to collect the relevant knowledge indicator, but they have not
yet begun collecting it. In these situations, we annotate the graph with
the phrase *Data unavailable.* Second, a few programs have not followed
the
traditional acquisition model. For example, one program combined the
development start decision with the production decision. Another program
used commercial off- the- shelf components, which negated the need to
monitor production processes. In these situations, we annotate the graph
with the phrase *Not applicable.* Finally, some programs were unable to
provide or reconstruct the relevant knowledge indicator because the event
happened too many years ago. In these situations, we annotate the graph
with the phrase *Not assessed.*
Our assessments of the 26 systems follow.
Advanced Amphibious Assault Vehicle (AAAV)
The Marine Corps* AAAV is designed to transport troops from ships to shore
at higher speeds and from farther distances than the existing AAV- 7. It
is designed to be more mobile, lethal, reliable, and effective in all
weather conditions. AAAV will have two variants* a troop carrier for 17
Marines and a command vehicle to manage combat operations in the field.
Prime contractor: General Dynamics
Program office: Woodbridge, Va.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $240.7 million
12/ 00 12/ 01 change
Procurement $97.9 million
Research & development cost $1, 395.7 $1,616. 4 15.8 Quantity 0 vehicles
Procurement cost $6, 256.7 $6,741. 7 7.8 Total program cost $7, 732.7
$8,440. 9 9.2
Program unit cost $7. 544 $8.235 9.2
Total quantities 1,025 1,025 0.0 Acquisition cycle time (months) 138 150
8.7
AAAV demonstrated most technology and design knowledge at critical
junctures in the program. At the start of the program, all but one of the
critical technologies were mature. The design was close to meeting best
practice standards at the design review, signifying the design was stable.
Early development of fully functional prototypes and
other design practices facilitated design stability. However, late
maturation of the remaining technology may lead to some redesign. Also,
the demonstration of production maturity remains a concern because the
program is currently uncertain about requiring the contractor to use
statistical process controls to achieve quality objectives. The AAAV
production decision is not scheduled until
September 2005. Remaining efforts include developmental, operational, live
fire, and reliability testing.
AAAV Program
Other Technology Maturity
The Marine Corps has recently restructured the AAAV program to add 12
additional months of testing before Four of the five critical technologies
had
the September 2005 production decision. This change demonstrated an
acceptable level of maturity at the more than doubles the number of
vehicle test months start of product development. The remaining
previously planned. The change also moves the initial technology, moving
map navigation, is not expected to operational capability date from
September 2007 to achieve maturity until the spring of 2003. Program
September 2008. The program estimates a $480 million officials stated that
maturing this technology is increase in acquisition costs*$ 101 million
for added contingent on developing and testing system
testing, $75 million for development, and $304 million hardware. As a
backup, program officials said they
for recurring production. could carry out the AAAV mission using existing
technology, but it would not provide full vehicle- tovehicle
Program Office Comments
situational awareness. AAAV program officials concurred with our
Design Maturity
assessment. The AAAV design is essentially complete. However, late
maturation of the new mapping system may lead to some redesign, if testing
identifies any problems. At the critical design review, AAAV had completed
77 percent of the drawings* not up to the best practice standard of 90
percent, but higher than many
DOD programs. Early engineering prototypes* fully integrated and
functional* allowed the program to demonstrate that the design worked as
required. These early prototypes have completed over 4,000 hours of
testing that resulted in design improvements for subsequent prototypes.
To complete development, program plans call for building and testing nine
development prototypes and one live fire test vehicle. These prototypes
will be production representative vehicles for developmental, operational,
live fire and reliability testing. The first
prototype is scheduled to be available by May 2003. Production Maturity
Program officials are developing a production readiness plan to ensure
vehicles will meet cost, schedule, and quality objectives. At this time,
they are uncertain whether this plan will require the contractor to use
statistical process controls, the best practice standard. As the prime
contractor currently produces the nine developmental prototype vehicles,
it is not tracking statistical process control data. Instead, it is using
postproduction inspections, considered less
efficient and effective than statistical process controls to achieve
quality.
Airborne Laser (ABL)
The Missile Defense Agency*s ABL is designed to destroy enemy ballistic
missiles almost immediately after their launch. The system, carried aboard
a highly modified Boeing 747 aircraft, uses a highenergy chemical laser to
rupture the skin of enemy missiles; a beam control/ fire control subsystem
to guide the laser beam through the aircraft, focus the beam on the
target, and maintain the beam*s quality as it travels through the
atmosphere; and a battle
management subsystem to plan and execute the engagement. We assessed all
components.
Prime contractor: Boeing
Program office: Albuquerque, NM
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $610.0 million 1/ 97 12/ 02 change
Procurement $0 million
Research & development cost $2, 400.5 $4,415. 7 84 Quantity 0 systems
Procurement cost $3, 170.6 NA NA Total program cost $5, 571.1 NA NA
Program unit cost $795.9 NA NA
Total quantities 7 NA NA Acquisition cycle time (months) 118 NA NA Note:
Latest costs only through FY 2007. Procurement funding, quantities, and
the initial capability date have yet to be determined. NA = not
applicable.
Only one of ABL*s critical subsystems has demonstrated acceptable levels
of maturity. The Missile Defense Agency is developing an initial ABL
system to demonstrate technology critical to the system*s design and plans
to begin development of a second improved demonstration aircraft in 2003.
Either of these aircraft, or later improved configurations, could be given
to the Air Force for
operational testing and production if system- level tests show that any
one of them is capable of destroying a threat missile at an operational
range. Although the agency*s development strategy incorporates some
knowledge- based practices, it is
difficult to see how the discipline of a knowledgebased approach can be
achieved when uncertainty exists about whether the effort is a technology
development or a product development.
ABL Program
Technology Maturity
Only one of ABL*s five critical subsystems, the aircraft itself,
represents mature technology. A second subsystem, which directs the laser
energy through the aircraft, consists of several technologies that have
been tested in a simulated environment. However, three other subsystems
consist of low- fidelity
prototype technologies that have only been tested in a laboratory
environment. They include the laser, the battle management subsystem, and
the ground support subsystem.
Problems associated with maturing technology have consistently been a
source of cost and schedule growth throughout the life of the program. DOD
analysts attribute this growth to the increased complexity of designing
laser subsystems, substantial increases in engineering analysis and
design, and greater than anticipated aircraft engineering complexity.
The program is managed under the Missile Defense Agency*s new
capabilities- based acquisition strategy. This approach develops an
operational system
through a series of block upgrades. The agency plans to use the first two
blocks, block 2004 and block 2008, to demonstrate critical technologies,
but if tests show either configuration has any battlefield utility, that
configuration could be deployed in the event of an emergency.
The 2004 configuration will have a 6- module laser, rather than the 14
modules planned for the production system. The optical components can
withstand the heat produced by a 6- module laser, but the agency would
have to redesign optical components for the system to withstand the heat
associated with an increase in laser power. In addition, the 2004
configuration is far too heavy to allow the addition of laser modules that
will likely be needed in an operational ABL system. To accommodate more
modules, a weight reduction
program has begun that includes redesigning many components and the
increased use of composite materials. The program is considering whether
to use a different aircraft configuration that would allow the system*s
weight to be moved forward to relieve stress on the airframe. However, its
use would require additional design changes.
The Missile Defense Agency has made changes that are expected to improve
its ability to evolve ABL*s critical technologies, including adopting a
flexible requirements setting process, providing additional time and
facilities to develop and test these technologies, and attaining the
knowledge to match the warfighters* needs with demonstrated technology. On
the other hand, it is not clear whether the start of a
block represents a technology development or a product development. This
uncertainty may hamper the application of knowledge standards and forfeit
the
discipline necessary to ensure successful product development.
Program Office Comments
In commenting on a draft of this assessment, program officials
reemphasized their commitment to spiral development and capabilities-
based acquisition. They plan to use this strategy to improve the critical
aspects of the system by allowing the pace of technological development to
dictate the introduction of improved capabilities into the system. They
believe this strategy
is not inconsistent with knowledge- based acquisition. They also mentioned
that laser power depends not only on the number of laser modules but also
on module efficiency, optics, and pointing precision. They admit that the
laser subsystem should be operated in flight before any production
decision is made. Program officials are conducting emergency operational
capability planning to support a possible emergency ABL deployment. This
decision will be based on the potential threat and an assessment of the
capabilities ABL may provide.
The program office indicated that all but one of the battle management
components have been tested in an operational environment. This component
is the active ranger system, which provides crucial angle measurements and
range data for engaging ballistic
missiles.
Advanced Extremely High Frequency (AEHF) Communications Satellite
The Air Force*s AEHF is a satellite system intended to replace the
existing Milstar system with improved, survivable, jam- resistant,
worldwide, secure communication capabilities at lower launch costs. First
launch of an AEHF satellite is expected in 2006. The system also includes
a mission control segment with service- specific terminals to process
satellite information. DOD is negotiating international partner
participation in the program. We assessed the satellites and mission
control segments.
Prime contractor: Lockheed Martin
Program office: El Segundo, Calif.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $778.1 million 10/ 01 12/ 02 change
Procurement $0 million Research & development cost $4, 339.3 $4,608. 4 6.2
Quantity 0 satellites
Procurement cost $1, 286.2 $509. 1 -60.4 Total program cost $5, 625.5
$5,117. 5 -9.0
Program unit cost $1, 125.1 $1,705. 8 51.6
Total quantities 5 3 -40.0 Acquisition cycle time (months) 111 118 6.3
The AEHF satellite program demonstrated most technology knowledge at
development start. Eleven of the 12 critical technologies were mature,
according to best practice standards. The remaining
technology is not projected to be mature prior to the critical design
review, nor does it have a backup technology. However, some elements of
this technology are mature. The program expects to complete 90 percent of
its drawings by the critical design review. The manufacture of the
communications and transmission security subsystem is a major challenge
facing the program as upgrades are being added into the new cryptological
equipment. If production of this subsystem slips, first launch could slip
correspondingly as no backup exists.
AEHF Program
security subsystem*s functions as it is integrated into the AEHF
satellite's communications subsystem.
Technology Maturity
However, continued complications in fabrication Eleven of the 12 critical
technologies have reached
could potentially place the entire program at cost, maturity according to
best practice standards. The
schedule, and performance risk. program does not project achieving
maturity on the
Other Program Issues
remaining technology* the phased array antenna* by the design review in
June of 2004, nor does it have a
In December 2002, the Deputy Secretary of Defense backup capability.
However, some elements of this decided to change the acquisition strategy
of AEHF technology have been demonstrated in an operational
from a five- satellite program to a three- satellite environment.
program. Under the revised strategy, full capability may no longer be
satisfied by an AEHF- only
Design Maturity
constellation. The program has completed 150 or more of the 6,000
Program Office Comments
total drawings for release to manufacturing. Program officials project
completing 90 percent of drawings by
In commenting on a draft of this assessment, program the system critical
design review in June 2004. The officials stated that the program is
executing very well program has completed key segment level preliminary
since contract definitization in August 2002, with cost design reviews and
is expected to complete all design and schedule variance at less than 1
percent. reviews by the second quarter of fiscal year 2004.
Currently, at approximately 33 percent complete Program officials consider
the design and
toward first launch, the total program is on track and development of the
satellite subcomponents low risk estimated to finish on time and on
budget. The system because those components have been used on other
preliminary design review has been completed. space systems. However, the
integration of these
Critical design reviews are on track for completion by subcomponents into
a subsystem, such as the phasedarray Spring 2004. Funding cuts have, in
the past, caused antenna, has yet to be successfully
schedule slips and cost increases. Given the focus on demonstrated at the
AEHF satellite frequencies.
the critical design review, the impacts of changing Program officials
assessed the software development requirement will have increasing
deleterious effects.
for the mission control system as moderate risk and The program remains
focused on addressing critical
have developed a risk mitigation strategy. This risks that threaten cost,
schedule, and performance.
strategy includes consulting with the National New system security
requirements recently received Software Engineering Institute and the
Aerospace
from the National Security Agency for the space, Corporation and
conducting a software development
mission control, and terminal segments are being capability evaluation.
Also, the program office has evaluated. After aggressive risk management,
the most
incorporated spiral development and the use of likely impacts include
additional testing, verification,
software emulators so users and developers can see and program
documentation. The program has also how the software will look and work.
Until these
begun developing engineering models for all of the actions are completed,
software may be at risk for
critical subsystems. These efforts are on track and unplanned cost and
schedule growth.
proceeding well.
Production Maturity
Any future problems with the fabrication of the communications and
transmission security microprocessor, a component designed to limit access
to satellite transmissions to authorized users, could delay the production
schedule and the launch of the first satellite planned for December 2006.
Program officials have started a number of risk reduction efforts,
including a chip emulator whose purpose is to simulate the communications
and transmission
AN/ APG- 79 Active Electronically Scanned Array (AESA) Radar
The Navy*s AESA radar is one of the top upgrades for the F/ A- 18E/ F
aircraft. It is to be the aircraft*s primary search/ track and weapon
control radar and is designed to correct deficiencies in the current
radar. According to the Navy, the AESA radar is key to maintaining the
Navy*s air- to- air fighting
advantage and will improve the effectiveness of the air- to- ground
weapons. When completed, the radar will be inserted in new production
aircraft and retrofitted into existing aircraft.
Prime contractor: Raytheon
Program office: Patuxent River, Md.
FY 2003 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $107.1 million
6/ 01 12/ 01 change
Procurement $24.06 million
Research & development cost $518.9 $494. 2 -4.7 Quantity 0 radars
Procurement cost NA NA NA Total program cost NA NA NA
Program unit cost NA NA NA
Total quantities NA NA NA Acquisition cycle time (months) 69 68 -1.4 Note:
NA = not applicable.
The AESA radar*s demonstrated knowledge is difficult to characterize. The
fact that almost all of the engineering drawings have been completed
suggests design stability. However, until the technologies are
demonstrated, the potential for design changes remains. The AESA radar is
also dependent on other programs that could pose significant risk to the
radar*s cost, schedule, and technical performance. The technology and
design risks are significant given that the AESA radar is only a few
months from a production decision. The Navy is currently reassessing the
radar*s technology maturity. Although many of the F/ A- 18E/ F aircraft
will be retrofitted with the AESA radar, full funding for the retrofitting
has not been budgeted. If the radar is not ready for production as
scheduled, more aircraft will have to be retrofitted.
AESA Radar Program
planned total buy of 548 F/ A- 18E/ F aircraft will not receive the radar
as they are being produced. Plans are
Technology Maturity
to retrofit the radar onto 136 aircraft at a projected The AESA program*s
four critical technologies were
cost of $3.14 million each. This cost does not include not mature at the
start of development in February
the cost of new APG- 73 mechanical scanned radars 2001, and they were not
mature as of May 2002. The
that will be installed in the aircraft until AESA radars Navy is currently
reassessing the maturity of these
are available for retrofit. If delays occur in the AESA technologies. At
the time of its last assessment, two of
radar deliveries, retrofit costs will increase. the technologies had been
tested using simulation and The AESA radar is projected to weigh about two
had been tested in the laboratory. Program
270 pounds more than the current radar and will officials indicated that
they have several options for
require a more capable cooling system than the one dealing with immature
technologies, including
currently on the aircraft. The Navy expects some utilizing backup
technologies. Initial flight tests of the minor degradation in aircraft
performance, such as radar in an aircraft are scheduled for June 2003*
slightly decreased range, as a result of the increased concurrent with the
production decision. All four
weight and new cooling system. technologies are not expected to be mature
until
The AESA program is linked to a number of other late 2004.
corporate and Navy programs. For example, the radar
Design Maturity
will use a 32- port fiber channel fabric module developed by Boeing as a
commercial venture. At the design review, 67 percent of the currently
Technical difficulties with the module have caused projected total
drawings were completed. In the
schedule delays and may impact cost and performance period between June
2002 and December 2002, the of the radar. Also, Raytheon is developing
some number of total expected drawings increased by 21 hardware and
software for the radar with company percent. Program officials stated that
the increase was
funds or in coordination with other programs. due to new or modified
drawings for systems Disruptions in these efforts could adversely impact
the supporting the radar such as the radome, shield, and
AESA program. aircraft airframe. Program officials indicated that they
currently have 98 percent of the drawings complete;
Program Office Comments
however, the technology maturation process may lead The AESA program did
not provide a general to more design changes.
statement in response to our review but did provide
Production Maturity
technical comments that were incorporated where appropriate. We could not
assess the AESA program*s production maturity against best practices, as
statistical control data was not available.
Other Program Issues
Program officials estimate that the first low- rate production unit will
exceed its cost target by 27 percent. Subcontractor development cost was
considered to be the biggest contributor to this increase. The effects of
the cost increase may be minimized in low- rate production lots 1- 3
because of firm fixed price contract options. Program officials stated
that cost reduction initiatives were underway to reduce the cost overruns
by half by full- rate
production. Delivery of the first production AESA radars for insertion
into F/ A- 18E/ F aircraft on the production line is scheduled for fiscal
year 2005. As a result, 254 of the
AIM- 9X Short- Range Air- to- Air Missile
The AIM- 9X is a follow- on version of the existing AIM- 9M short- range
missile for Air Force and Navy fighters. The AIM- 9X is designed to be a
highly maneuverable, launch- and- leave missile; capable of engaging
targets using passive infrared guidance to provide full day/ night
operations and improved resistance to countermeasures and expanded target
acquisition. The full capabilities of the AIM- 9X will not be achieved
without completing development of the helmet mounted cueing system* a
separate
development program that we did not assess.
Prime contractor: Raytheon Missile System Company
Program office: Patuxent River, Md.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $2.7 million 1/ 97 12/ 01 change
Procurement $104.9 million
Research & development cost $577.0 $594.6 3.0 Quantity 531 missiles
Procurement cost $2, 116.1 $2, 055.1 -2.9 Total program cost $2, 693.0 $2,
649.6 -1.6
Program unit cost $0. 268 $0. 261 -2.5
Total quantities 10, 049 10, 142 0.9 Acquisition cycle time (months) 92
105 14.1
The AIM- 9X program entered production in September 2000 without assuring
that the manufacturing processes were mature. However, because the missile
is a follow- on to the AIM- 9M
missile, program officials believe that they have significant production
knowledge. The program ensured, prior to entering low- rate initial
production, that the missile design was stable. The program did attain
knowledge early in development by using proven technologies from
predecessor systems and other programs, as well as testing numerous
prototype versions of the missile. As a result, the program released the
majority of its engineering drawings at the design review.
AIM- 9X Program
rework data. The AIM- 9X acquisition cost and schedule history shows the
program has been able to
Technology Maturity
meet its goals. All of the AIM- 9X critical technologies are mature
Program Office Comments
because they have been demonstrated in developmental tests using actual
hardware in realistic
In commenting on a draft of this assessment, program conditions.
Specifically, the program used prototypes
officials acknowledged they did not contractually to test new technologies
and existing missile
require collection of statistical process control data on components that
are being employed in a new
critical manufacturing processes. Program officials operational
environment.
stated their strategy for demonstrating manufacturing process maturity
includes building, testing, and
Design Maturity
evaluating production representative missiles; The design of the AIM- 9X
is complete, and 100 percent
conducting multiple readiness reviews; utilizing lowrate of the drawings
have been released to manufacturing. initial production to test production
processes; The AIM- 9X program built and tested 43 prototypes of
and maturing production processes before full- rate various configurations
during development to help
production. mature the missile's design. Hardware and software performance
was assessed at subsystem and system levels, and design changes were
incorporated into the prototypes until a mature and stable missile
configuration was demonstrated. The AIM- 9X program held design reviews
for the 11 subsystems between
October 1997 and March 1998. The early design reviews, prototypes, and
early testing allowed the program to achieve a stable design at the system
design review in March 1998. At that time, the contractor had released 94
percent of its engineering drawings to manufacturing.
Production Maturity
The AIM- 9X program does not contractually require collection of
statistical process control data on critical manufacturing process, but it
has undertaken an acquisition strategy to incentivize the contractor to
reach cost and quality goals. However, the contractor and program
officials believe that they have significant knowledge about producing the
missile. The AIM- 9X is a variant of the AIM- 9M missile and uses
components
produced for other weapon systems, providing the program with significant
production knowledge. In addition, to improve the production capabilities,
the contractor built developmental units on production equipment. Program
officials believe this practice has allowed them to mature the
manufacturing processes. According to program officials, most of the
critical processes on the AIM- 9X are at the subcontractor level and a
process exists to attain cost and quality goals. This is accomplished
primarily by postproduction inspections to track production yield, scrap,
and
Advanced Threat Infrared Countermeasures/ Common Missile Warning System
The Army*s and Special Operations* ATIRCM/ CMWS is a component of the
Suite of Integrated Infrared Countermeasures planned to defend U. S.
aircraft from advanced infrared- guided missiles. The system will be
employed on Army and Special Operations
aircraft. ATIRCM/ CMWS includes an active infrared jammer, a missile
warning system, and a countermeasure dispenser capable of loading and
employing expendables, such as flares, chaff, and smoke.
Prime contractor: BAE Systems
Program office: Huntsville, Ala.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $7.2 million
3/ 96 12/ 01 change
Procurement $75.7 million
Research & development cost $568.0 $540. 1 -4.9 Quantity 2 units
Procurement cost $2, 338.9 $1,971. 9 -15.7 Total program cost $2, 906.9
$2,512. 0 -13.6
Program unit cost $0. 940 $2.330 148.0
Total quantities 3,094 1,078 -65.2 Acquisition cycle time (months)
Classified Classified NA Note: NA = not available.
The ATIRCM/ CMWS is scheduled to enter production in May 2003 with no
assurance that production processes are in control. The CMWS portion of
the ATIRCM/ CMWS program entered limited production in February 2002 to
meet an urgent need. Full- rate production for ATIRCM/ CMWS was delayed
because of reliability problems, which may indicate that production
processes were not in control. These problems are, at least in part, a
consequence of design proceeding with known shortfalls in knowledge: key
technologies were demonstrated late in development and only a small number
of design drawings were completed by design review. Resolving these
knowledge shortfalls has led to cost and schedule increases. While the key
technologies appear mature, reliability and producibility issues could
necessitate design changes.
ATIRCM/ CMWS Program
The program delayed the production decision for the combined system an
additional year to the currently Technology Maturity
scheduled May 2003 date primarily due to reliability The five critical
technologies for the system are
issues. Reliability testing was halted because of mature, but they did not
mature until after the system
numerous failures with the ATIRCM subsystem. design review. Most of the
early technology
Reliability failure can be an indicator of producibility development
effort was focused on the application to
and process control problems. The program plans to rotary wing aircraft.
However, when product
build and develop six additional subsystems during development began in
1995, the requirements were 2002 and 2003. The full- rate production
decision for expanded by Office of the Secretary of Defense
the complete system is now scheduled for 2005. direction to include Navy
and Air Force fixed wing
Other Program Issues
aircraft. According to program officials, they did not fully anticipate
the additional technology needed to
The Army procured an initial 32 systems in fiscal year meet these much
more demanding requirements. This 2002 that only included the CMWS. The
Army plans to change caused problems that largely contributed to procure a
total of 99 ATIRCM/ CMWS systems to outfit cost increases of more than 150
percent to the special operations aircraft between fiscal year 2002
development contract. The Navy and the Air Force and 2009. subsequently
dropped out of the program, rendering the extra effort needless.
Program Office Comments Design Maturity
In commenting on a draft of this assessment, program officials stated that
the Army eliminated the program's The basic design of the system is
complete, with
funding for fiscal years 2002 and 2003. In fiscal year 100 percent of the
drawings released to
2003, the Special Operations Command funded the manufacturing. However,
reliability and producibility
urgent procurement of 32 CMWSs. Subsequently, the issues could require
design changes. The design was
Army reinstated the program for fiscal years 2004* particularly immature
at the critical design review, 2009. The program office stated that the
loss of with only 22 percent of the drawings complete. A
funding in fiscal year 2003 slowed the program major cause was that the
technology requirements
markedly. The program's acquisition strategy remains were not well
understood until the system design
to equip Special Operations forces before equipping review, leading to the
discovery that a major redesign the remainder of the Army. was needed to
meet requirements. It was not until
The system was modified in 2002 to address ATIRCM 2 years after the design
review that 90 percent of the reliability, producibility, and built- in-
test issues. Six drawings were released and the design was
ATIRCM systems are being manufactured and tested considered stable.
According to program officials, the
to demonstrate and verify the enhancements. ATIRCM immature design caused
inefficient manufacturing, is scheduled to begin low- rate initial
production in rework, and testing and contributed to the 3- year
May 2003, and CMWS is scheduled to begin low- rate schedule delay. initial
production in January 2004. The program office
Production Maturity
stated that low- rate production is required to maintain a production
base. The system's operational testing is The ATIRCM/ CMWS program does
not collect
planned for March 2005. According to the program statistical control data
on its critical manufacturing
office, the prime contractor indicated that statistical processes. Program
officials have identified the
process control is not within its corporate philosophy, absence of
statistical process control data as a particularly for a program with such
low production weakness and believe it should be instituted. Despite
rates and quantities. this shortfall in knowledge, the Army entered
limited CMWS subsystem production in February 2002 to
meet an urgent need of the U. S. Special Operations Command.
Advanced Wideband Satellite (AWS)
The Advanced Wideband Satellite system is designed to provide improved,
survivable, jamresistant, worldwide, secure and general purpose
communications to support the National Aeronautics and Space
Administration, DOD and the intelligence community. It will replace the
current Milstar satellite system and supplement the AEHF satellite system,
reviewed elsewhere in this report. It will be the cornerstone of a DOD
architecture that includes the multiple satellite systems.
Prime contractor: In competition
Program office: El Segundo, Calif.
FY 2004 funding request: FY 2003 dollars in millions President*s budget
Latest Percent R& D $439.3 million 02/ 03 00/ 00 change
Procurement $0.0 million Research & development cost $5, 814.5 $0.0 0.0
Quantity 0 satellites
Procurement cost $2, 343.9 $0.0 0.0 Total program cost $8, 158.4 $0.0 0. 0
Program unit cost $2, 039.6 $0.0 0. 0
Total quantities within budget 4 0 0 Acquisition cycle time (months) 75 0
0 Note: Program costs and schedule have yet to be formally approved by
DOD. Costs only through FY 2009.
The AWS/ TSAT program is scheduled to enter product development with only
one of its five critical technologies mature, according to best practices.
The initial product development period will likely require concurrent
technology and product development activities to maintain schedule.
Although the new draft space acquisition guidance allows this approach, it
is contrary to the best practice of separating technology development from
product development.
AWS/ TSAT Program
Technology Maturity
Of the five AWS/ TSAT key space segment technologies, one is mature while
the other four are scheduled to reach maturity by January 2006, more than
2 years after development start. Three of the four immature technologies
have a backup technology available in case of development difficulties.
However, use of these technologies would degrade system overall
performance. The Single Access Laser
Communications technology has no backup and, according to program
officials, any delay in maturing this technology would result in a slip in
the expected launch date.
Other Program Issues
The program plans a development cycle that is, according to DOD
documentation, aggressive. The satellite development cycle is planned to
be 75 months: 27 months for technology development; 15 months for product
development; and 33 months for satellite build, test and launch. This
period of time
is substantially shorter than the development cycle for the AEHF satellite
(118 months vs. 75 months), though the AWS/ TSAT system is expected to
provide a transformational leap in satellite communications capability.
The program is managed under the new National
Security Space Acquisition process, which makes no clear distinction
between the end of technology development and the start of product
development. Therefore, the AWS/ TSAT acquisition strategy may allow the
system's technology development and
product development to be conducted concurrently prior to the production
decision. DOD*s acquisition system policy states that one of the entrance
criteria for the system development and demonstration phase
is technology maturity. The AWS/ TSAT acquisition strategy does not ensure
that technology maturity will be achieved prior to the start of product
development consistent with best practices.
Program Office Comments
In commenting on a draft of this assessment, program officials stated that
the National Security Space Acquisition Policy was developed to streamline
the decision- making framework and to tailor it for space systems, in
order to more efficiently field systems that incorporate rapidly changing
technology advances.
Cooperative Engagement Capability (CEC)
The Navy*s CEC is designed to connect radar systems to enhance detection
and engagement of air targets. Ships and planes equipped with their
version of CEC hardware and software will share real- time data to create
composite radar tracks, essentially allowing the battle group to see the
same radar picture. A CEC- equipped ship will then be able to detect and
launch missiles against targets its radar cannot see. We assessed block 1
of the CEC. The Navy is developing a more advanced block 2 CEC.
Prime contractor: Raytheon Systems Corporation
Program office: Washington, D. C.
FY 2004 funding request FY 2003 dollars in millions Approved Latest
Percent R& D $72.5 million
5/ 95 4/ 02 change
Procurement $128.6 million
Research & development cost $1, 154.0 $2,052. 8 77.9 Quantity 21 systems
Procurement cost $1, 292.5 $2,127. 6 64.6 Total program cost $2, 493.9
$4,180. 4 67.6
Program unit cost $13.6 $15. 4 12.8
Total quantities 183 272 48.6 Acquisition cycle time (months) 16 16 0. 0
The technologies and design of the CEC program block 1 are fully mature.
The program*s production maturity could not be assessed. The program lacks
the necessary data primarily because the government does not collect it on
the commercially available portions. However, program and
contractor officials consider the processes to be capable of producing a
quality product on time and on cost. Block 1 of the CEC program was
approved
in April 2002 for full- rate production for the shipboard version and
continued low- rate initial production for the airborne version.
CEC Program
Some solutions for interoperability and integration issues will also be
assessed in follow- on testing.
Technology Maturity
However, many of these issues are expected to be In January 2002, the
Office of Naval Research assessed resolved through the introduction of
block 2. The plan CEC*s six critical technologies. Five of the
was approved in April 2002. Block 2 is expected to technologies assessed
as mature were incorporated provide cost, performance, and functional into
the shipboard version when it successfully
improvements over the current system, though its completed the operational
evaluation in May 2001. The
details are yet to be defined. Among the anticipated sixth technology, a
data processor, was not assessed
characteristics of block 2 is interoperability with as part of the
operational evaluation but was
legacy combat systems. determined to be mature.
Program Office Comments Design Maturity
In commenting on a draft of this assessment, program CEC*s basic design
appears complete, as all of the
officials stated that a production readiness review drawings needed to
build the shipboard version have
conducted in October 2001 found CEC production to been released to
manufacturing.
be mature. They evaluated all areas of production, including quality,
configuration management, CEC program officials noted that new drawings
processes and procedures, drawings, and testing. They continue to be
released. They explained that as
stated that the contractor is delivering systems on commercially available
technologies, which comprise
schedule and within cost. To date, 29 systems over 5 approximately 60
percent of CEC*s hardware, become
years have been successfully delivered, installed, more advanced, portions
of the system will need to be
tested, and many have been deployed. Following and redesigned to
incorporate those advances.
operational testing and evaluation, the Navy found CEC to be operationally
suitable and effective and the
Production Maturity
DOD Director for Operational Test and Evaluation found CEC demonstrated
the highest reliability of any We could not assess the CEC program*s
production system tested so far of comparable complexity. maturity against
the best practice as data were not
According to program officials, CEC*s use of available. According to
program officials, the
commercial off- the- shelf components enables the noncommercially
available portions of CEC do not
program to select mature cost- effective components involve any critical
manufacturing processes. Officials
from industry, instead of manufacturing them inhouse. indicated that they
do not have insight into
In recognition of the above, DOD approved the manufacturing processes for
the commercially program for full- rate production in April 2002.
available portions, including whether these processes are critical and
whether the contractor has them under statistical control.
The program officials and the contractor are confident that a quality
product can be delivered on time and within cost based on the contractor*s
adherence to industry standards and past performance on the lowrate
initial production contracts for the shipboard version.
Other Program Issues
Battle group- level interoperability, integration, and built- in- test
false alarm rates were identified as areas needing improvement following
the operational
evaluation. Program officials expect a solution for the alarm rates to be
in place for a follow- on operational test and evaluation planned for
2004.
CH- 47F Improved Cargo Helicopter
The Army*s CH- 47F heavy lift helicopter is intended to provide
transportation for tactical vehicles, artillery, engineer equipment,
personnel, and logistical support equipment. It is expected to operate in
both day and night. The purpose of the
CH- 47F program is to improve the performance and extend the useful life
of the CH- 47. This effort includes installing a digitized cockpit,
rebuilding the airframe, and reducing aircraft vibration through airframe
stiffening.
Prime contractor: Boeing
Program office: Huntsville, Ala.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $0 million 5/ 98 10/ 02 change
Procurement $516.0 million Research & development cost $148.0 $169. 6 14.6
Quantity 16 aircraft
Procurement cost $2, 613.9 $5,927. 7 126.8 Total program cost $2, 761.9
$6,097. 3 120.7
Program unit cost $9. 145 $17.986 96.7
Total quantities 302 339 12.3 Acquisition cycle time (months) 82 99 20.7
The CH- 47F helicopter began low- rate production in December 2002,
although key production processes were not in control. Program officials
believe that CH- 47F production is low risk because no new technology is
being inserted into the aircraft, two prototypes have been produced, and
the production process has been demonstrated during the
development phase. The CH- 47F technologies and design appear mature,
although a low percentage of engineering drawings were released at the
design review. Production unit costs have more than doubled due to
contractor rate increases, increases in system capabilities, and initial
underestimation of program cost.
CH- 47F Program Technology Maturity Although we did not assess technology
maturity in detail, the CH- 47F is a modification of the existing CH- 47D
helicopter. Program officials believe that all critical technologies are
mature and have been demonstrated prior to integration into the CH- 47F
development program.
Design Maturity
The CH- 47F design is complete, with 100 percent of the drawings released
to manufacturing. However, at the design review only 37 percent of the
system*s engineering drawings were complete. Since that time, the number
of drawings completed has increased substantially. The majority of the new
drawings were
instituted to correct wire routing and installation on the aircraft,
changes program officials believed could not be determined until after the
first prototype was developed.
Production Maturity
CH- 47F production maturity could not be determined because the program
does not use statistical process control to ensure that production
processes are stable. Program officials believe the production is low risk
because two prototypes have been produced and the production processes
have been demonstrated during the development phase. The Army plans to
conduct
operational testing in fiscal year 2004 to demonstrate its readiness to
proceed into full- rate production. Prior to that decision, the Army plans
to complete a risk assessment for the CH- 47F to eliminate any production
risk that remains.
Other Program Issues
Both the total cost and the program unit cost for the CH- 47F production
program have more than doubled. This growth triggered a Nunn- McCurdy
breach (see 10 U. S. C. 2433) in December 2001, requiring a review by the
Secretary of Defense and a report to Congress. As a result, the Secretary
of Defense has certified to Congress that the CH- 47F is essential for
national
security, there are no alternatives, the new cost estimates are
reasonable, and the management structure is in place to continue to keep
costs under control. According to the program office, the cost increases
were due to (1) prime contractor labor rate increases and material cost
growth, (2) additional system capabilities required by the Army, (3)
recapitalization of 36 Special Operations aircraft,
and (4) initial underestimation of program costs. According to the program
manager, the Army has fully funded the program*s cost growth of about $2.5
billion (then- year dollars). This increase in program cost necessitated
rebaselining the CH- 47F program. The Army approved the CH- 47F
acquisition program baseline.
Program Office Comments
The CH- 47F program office generally concurred with this assessment.
RAH- 66 Comanche
The Army*s Comanche is a multi- mission helicopter intended to perform
tactical armed reconnaissance. It is designed to operate in adverse
weather across a wide spectrum of threat environments and provide improved
speed, agility, reliability, maintainability, and low observability over
existing helicopters. It is also expected to lower operating costs through
the use of integrated diagnostics, a composite airframe, and a
bearingless rotor system. It will replace the AH- 1, OH- 6, and OH- 58
helicopters.
Prime contractor: Boeing- Sikorsky
Program office: Huntsville, Ala.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $1. 1 billion 7/ 00 10/ 02 change
Procurement $0 million
Research & development cost $8, 886.4 $12,556. 3 41.3 Quantity 0 aircraft
Procurement cost $30,550.9 $21,939. 5 -28.2 Total program cost $39,824.0
$34,545. 0 -13.2
Program unit cost $32.831 $53.146 61.9
Total quantities 1,213 650 -46.4 Acquisition cycle time (months) 222 250
14.9
Most of the Comanche*s critical technologies have demonstrated acceptable
levels of maturity, and the program appears very close to meeting the best
practice standard for a stable design. This level of maturity follows many
years of difficult development. Since the program*s first cost estimate
was originally approved in 1985, the research and development cost has
almost quadrupled and the time to obtain an initial capability has
increased from 9 years to over 21 years. The program has recently
undergone another major restructuring to incorporate an
evolutionary acquisition approach and reduce concurrency and lower overall
risk. This restructuring shows promise of being a knowledgebased program
that matches program resources with user requirements.
RAH- 66 Comanche
program requirements with force requirements and program risks. Weight
issues were addressed through
Technology Maturity
increased engine performance. Initial operational Seven of the Comanche*s
eight critical technologies
capability was moved from December 2008 to are considered mature. Only one
critical technology, September 2009 to reduce risk and significantly the
radar cross- section needed for low observability, increase the amount of
testing conducted. requires additional development. The Army expects
Program Office Comments
that this technology will reach maturity in fiscal year 2005, a year
before the production decision.
In commenting on a draft of this assessment, program officials generally
concurred with our assessment.
Design Maturity
They added that in October 2002, the Office of the The Comanche program
has released 73 percent of the Secretary of Defense approved the Comanche
engineering drawings to manufacturing. The program program as an
evolutionary acquisition approach. The
has improved its ability to reach design maturity by Comanche quantity was
reduced from 1213 to 650 rescheduling the design review from July 2002 to
April
based on emerging results of the Comanche*s role in 2003. The program
estimates that it will complete
the Objective Force. This reduction in quantities, 90 percent of the
drawings by the design review under
combined with the research and development cost the proposed plan, instead
of the former 59 percent
growth, resulted in a program acquisition unit cost under the previous
program.
increase of approximately 62 percent. Excluding impacts of the quantity
reduction, the average Critical technologies have not yet been integrated
and
procurement unit cost increased 18 percent and the demonstrated on the
Comanche airframe. Prior to the
program acquisition unit cost increased 23 percent. proposed program
restructure, integration of critical technologies was considered high
risk, even though most of the technologies had reached maturity on other
platforms. Program officials believe that the restructured program reduces
integration risks and that the longer development schedule will allow for
reduced concurrent development and additional integration time and
facilities, thereby reducing critical risks. The longer schedule also
provides additional time for
near- term development testing, use of a production representative
aircraft for initial operational testing, and full qualification testing.
Additionally, the phasing of development and operational tests was revised
and expanded to reduce overall program risk.
Other Program Issues
Continuing cost and schedule issues have led to the most recent
restructuring of the program. In October 2002, the Office of the Secretary
of Defense approved the Comanche program to continue under
an evolutionary acquisition approach. However, because of uncertainties
with future funding and capabilities, quantities were reduced from 1213 to
650 aircraft. This reduction in quantities, combined with the research and
development cost growth, resulted in a unit cost increase of approximately
62 percent.
Program officials stated that the restructuring added a more robust
internal review process and balanced
EX- 171 Extended Range Guided Munition (ERGM)
The Navy*s ERGM is a rocket- assisted projectile that is fired from a gun
aboard ships. It can be guided to land targets at ranges of between about
10 and 50 nautical miles to provide fire support for ground troops. ERGM
is expected to offer increased range and accuracy compared to the Navy*s
current gun range of 13 nautical miles. ERGM requires modifications to
existing 5- inch guns, a new munitions- handling system (magazine), and a
new fire control system. We assessed the projectile only.
Prime contractor: Raytheon
Program office: Washington, D. C.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $28.6 million
4/ 97 12/ 02 change
Procurement $3.8 million
Research & development cost $77.6 $326. 1 320.5 Quantity 0 rounds
Procurement cost $308.4 $159. 4 -48.3 Total program cost $386.0 $485. 5
25.8
Program unit cost $0. 045 $0.150 233.8
Total quantities 8,570 3230 -62.3 Acquisition cycle time (months) 50 121
146.9
The ERGM program began product development with very few of its critical
technologies mature according to the best practices standards. While
significant progress has been made in the past 7 years, program officials
do not expect to achieve maturity on all critical technologies until after
the design review. No production representative engineering drawings had
been released at the time of our assessment, and none are projected by the
system design review. The program office currently expects to release
these 1 year later. In June 2002, the program conducted a successful test
of a guided tactical round under realistic launch conditions. This test
did not evaluate the performance of a new
warhead design.
ERGM Program
fiscal years 2002 and 2003, stretched out program milestones and will
delay deployment of ERGM until Technology Maturity
2006. Fourteen of ERGM*s 20 critical technologies have
Other Program Issues
demonstrated technological maturity. The remaining 6 technologies are
approaching maturity, and program
Future program costs are not accurately reflected in officials expect that
all 20 critical technologies will be the latest program cost estimate and
the fiscal year demonstrated in an operational environment by the
2004/ 2005 budget request. The cost estimate is based end of 2003,
approximately 7 months after the design
on a much lower production quantity than is contained review. Three of the
technologies yet to reach maturity in either the approved or the current
draft revision of are part of the new unitary warhead design, and a
the ERGM acquisition program baseline. The budget fourth is related to
this change. Program officials request does not fully fund the 80
operational test recently identified the unitary warhead*s safe/ arm
rounds currently required. device and fuze as a critical technology, after
a Navy
Two testing issues could affect the program. The safety review concluded
that it needed to be
Director of Operational Test and Evaluation has raised redesigned to meet
applicable DOD standards.
a concern about test range restrictions that could limit The ERGM program
began development with only one
realistic operational testing. Finally, the project of its critical
technologies mature. Having only one manager stated that the availability
of a fully capable critical technology mature at the start of product ship
to support development testing could be an issue development has caused
cost and schedule problems. due to funding shortfalls for magazine
modifications For example, when the program began, none of the
on these ships. components of the rocket motor had been integrated Some of
the cost increases and schedule slippages to into an ERGM representative
design. Subsequent
date may be attributed to the fact that the contractor problems with the
performance reliability of the motor
relocated the program in 1998, resulting in a loss of resulted in cost
growth of more than $13 million.
trained personnel and development inertia.
Design Maturity Program Office Comments
None of ERGM*s approximately 127 production In commenting on a draft of
this assessment, program representative engineering drawings have been
officials stated that although production released to manufacturing. The
program office plans representative drawings will not be available at the
for all of these drawings to be released in June 2004, design review, the
entire ERGM design would be
about one year after the design review. In the under configuration
control. Design maturity at that meantime, the design review will be used
to validate
time will be sufficient to produce all- up rounds for the design of the
development test rounds. The June
land and ship- based development testing. Based on 2004 drawing release,
which will reflect knowledge data obtained from flight and qualification
tests in gained from 8 of 18 flight tests and some qualification fiscal
year 2004, minor revisions to the ERGM tests, will be used to build the 80
production
technical data package may be made. Production representative operational
test rounds. Program representative drawings will be finalized by June
2004. officials pointed out that progress has been made in
Program officials stated that they are highly maturing the design. For
example, the main elements
encouraged by the significant progress in ERGM of the design were
validated during the guided gunfire development activities over the last
18 months. They
test in June 2002. further stated that they have a high degree of In
January 2002, in order to meet lethality and safety
confidence that ERGM will meet all performance requirements, the Navy
decided to make a significant
requirements, while meeting the production cost goals change to the
warhead design, moving from a
specified in the acquisition program baseline. multiple- submunition
design to a single explosive* or unitary* warhead. This decision, coupled
with the decision to stay within planned funding levels for
Excalibur Artillery Round
The Army*s Excalibur is a family of extended range, precision, 155- mm
artillery projectiles. It is designed to increase soldier survivability by
allowing the Future Combat Systems* nonline of sight cannon to fire from
farther away and defeat threats more quickly, while reducing logistic
support. It also is intended to be more effective when fired at urban
targets, through a combination of altered trajectory and global
positioning system accuracy.
Prime contractor: Raytheon Missile Systems
Program office: Picatinny Arsenal, NJ.
FY 2004 request: FY 2003 dollars in millions Approved Latest Percent R& D
$134 million 4/ 97 2/ 03 change
Procurement $0 million
Research & development cost $59.8 $662. 8 1, 008.7 Quantity 0 projectiles
Procurement cost $676.2 $4,135. 9 511.6 Total program cost $736.0 $4,798.
7 552.0
Program unit cost $0. 004 $0.062 1, 578.8
Total quantities 200,000 77,677 -61.2 Acquisition cycle time (months) 160
136 -15.0
The Excalibur program*s three critical technologies are not fully mature,
even though product development began over 5 years ago. The technologies
appear to be approaching maturity, and program officials project
demonstrating
technology and design maturity before the design review in 2005.
Currently, 13 percent of the drawings are at the level that could be
released to
manufacturing. Program officials expect to have a stable design by the
design review. The program has undergone a major restructuring effort. It
has encountered a number of challenges since development began, including
a substantial decrease in planned quantities, a relocation of the
contractor*s plant, limited early funding, technical problems, changes in
program direction, and a merger with another program.
Excalibur Program
Program Office Comments Technology Maturity
In commenting on a draft of this assessment, program officials generally
agreed with the information in this None of the Excalibur*s three critical
technologies are report. However, they provided the following fully mature
according to best practice standards.
clarifying comments. According to program officials, all three have been
Concerning the Excalibur design maturity, program demonstrated in a
relevant environment and are
officials stated that approximately 600 drawings are expected to reach
maturity before the design review in
anticipated at the subsystem level. But because the March 2005. The
Excalibur*s design and requisite
program is still in research and development, no technologies have changed
since product drawings have been officially released to development was
started. The three critical manufacturing. The program is fabricating
hardware technologies for the current design are the guidance
in a research and development environment. control system, the airframe,
and the warhead. The
warhead was not considered a critical technology in 1997 because the
Excalibur design called for a warhead that was under production for other
munitions. Based on Army direction, the program has undertaken development
of a different warhead that is currently undergoing testing.
Design Maturity
About 13 percent of the Excalibur*s engineering drawings are at a level
that could be released to manufacturing. The program office plans to have
all of its drawings complete and released to manufacturing by the design
review in March 2005. However, program officials could not estimate the
total number
of drawings expected.
Other Program Issues
The program has gone through many changes since the beginning of product
development in May 1997. It was almost immediately restructured due to
limited funding, and it was restructured again in 2001. In response to
congressional direction, the program was restructured to merge with the
joint Swedish/ U. S. program known as Trajectory Correctable Munitions.
The merger should help the program deal with design challenges, including
issues related to its folding fin
design. Also, in May 2002, the Office of the Secretary of Defense directed
the program to develop the Excalibur for the Future Combat Systems nonline
of sight cannon and to field it in fiscal year 2008.
Although program officials have not yet released the new cost and schedule
estimates, the net effect of these changes has been to increase the
program*s
schedule and to substantially decrease planned procurement quantities. As
a result, the program*s overall costs and unit costs have dramatically
increased.
aircraft designed to meet fighter escort, Navy's F/ A- 18C/ D and F- 14
aircraft.
Prime contractor: Program office: Boeing Patuxent River, Md.
FY 2004 funding request: R& D $179.0 million
Procurement $3.0 billion
Quantity 42 aircraft F/ A- 18E/ F Super Hornet
The Navy*s F/ A- 18E/ F is a multi- mission tactical interdiction, fleet
air defense, and close air support mission requirements. The program was
approved as a major modification to earlier F/ A- 18 aircraft in
1992. It is intended to complement and replace the
FY 2003 dollars in millions Approved
Research & development cost Acquisition cycle time (months) 6/ 92 Latest
Program unit cost $67.390 102 12/ 01 Procurement cost $61,215.3 $41,368. 2
Total program cost $67,389.7 $47,549. 8 112 Percent change
$6, 174.4 $6,181. 6 0.1 -32.4 -29.4
$86.770 28.8
Total quantities 1,000 548 -45.2
The F/ A- 18E/ F went into full- rate production in June 2000. Although
the program proceeded
9.8
without obtaining full product knowledge at key decision points, it
embraced the concepts of attaining design and manufacturing knowledge
early in development. The program released just over half of its
engineering drawings by its design review. When low- rate production
began, nearly all of the drawings were released and about 75 percent of
the manufacturing processes were in control. The Navy reduced some program
risk because aviation electronics from an earlier version of the F/ A- 18
were incorporated into the baseline F/ A- 18E/ F. Furthermore, focus was
placed on commonality between the F/ A- 18 C/ D and the F/ A- 18 E/ F,
which further reduced risk.
F/ A- 18E/ F Program
Technology Maturity
We did not assess the technology maturity of the F/ A- 18E/ F program
because it is already in full- rate production. Nevertheless, we did not
identify any technical challenges during the development of this aircraft
in our previous reviews.
Design Maturity
The F/ A- 18E/ F design appears complete. The program has released 100
percent of the design drawings to manufacturing. At the time of the
critical design review in July 1994, 56 percent of the engineering
drawings were completed and released to manufacturing for aircraft
structure and systems. According to program officials, they decided to
proceed despite the low level of completed drawings because the knowledge
gathered from earlier F/ A- 18C/ D models gave them confidence that the
design was stable. By the time of the low- rate initial production
decision, 99 percent of the drawings had been released.
Production Maturity
According to program officials, they currently have 100 percent of their
critical manufacturing processes under control, according to the best
practice standard. Therefore, they are no longer tracking processes using
statistical process control. However, defects are still
monitored through inspections, failures, and age exploration testing, and
during maintenance. If production problems are identified, the program
would resume statistical process control analysis where necessary.
Program officials estimate that about 75 percent of key manufacturing
processes were in control at the low- rate production decision in March
1997. Program officials stated that they concentrated on maturing their
manufacturing processes before starting
production. As a result of these efforts, labor efficiency rates have
steadily improved.
Other Program Issues
The F/ A- 18E/ F will not reach its full potential until after the
incorporation of several preplanned upgrades* the Active Electronically
Scanned Array (AESA) radar, the Joint Mounted Helmet Cueing System coupled
with the AIM- 9X missile, and the Advanced Targeting Forward Looking
Infrared sensor.
The level of effort and timing to incorporate some of the sensors* the
AESA radar and the Advanced Targeting sensor* may prove to be a challenge.
We have assessed the AESA radar elsewhere in this
report.
Program Office Comments
In commenting on a draft of this assessment, program officials stated
initial schedule delays were due to a procurement reduction of 10 aircraft
in a 1998 Program Objective Memorandum. Since that time, the contractor
has consistently delivered aircraft ahead of
schedule. Program officials also noted that the aircraft are demonstrating
two to three times the quality of the F/ A- 18C/ D and have provided
measurable
improvements to squadron readiness. In addition, all F/ A- 18E/ F
preplanned upgrades continue to track to their program schedules. The
Joint Mounted Helmet Cueing System has completed operational evaluation,
and the system has been incorporated into lot 24 of the aircraft
(deliveries of which began in September 2001). Deliveries of the Advanced
Targeting Forward Looking Infrared Sensor production units began in
April 2002, and the units were deployed in January 2003. Finally, program
officials stated that the AESA radar program continues to execute as
planned, and the program has received the first engineering and
manufacturing development unit.
F/ A- 22 Raptor
The Air Force*s F/ A- 22, originally planned to be an air superiority
fighter, will also have air- to- ground attack capability. It is being
designed with advanced
features, such as stealth characteristics, to make it less detectable to
adversaries and capable of high speeds for long ranges. It also has
integrated aviation electronics (avionics) designed to greatly
improve pilots' awareness of the situation surrounding them. It is
designed to replace the Air Force*s F- 15 aircraft.
Prime contractor: Lockheed Martin
Program office: Dayton, Ohio
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $936.5 million
2/ 92 12/ 02 change
Procurement $4.2 billion
Research & development cost $20,938.2 $30,836. 1 47.3 Quantity 22 aircraft
Procurement cost $54,272.2 $39,049. 2 -28.0 Total program cost $75,461.2
$70,469. 4 -6.6
Program unit cost $116.5 $253. 5 117.7
Total quantities 648 278 -57.1 Acquisition cycle time (months) 203 230
13.3
Because the F/ A- 22 Program Office stopped collecting process control
data in 2000, the program began production in 2001 with no proof that
processes were in control, as defined by best practice standards.
Technology appears mature and the design appears stable; however, problems
with the vertical tail and the avionics have been discovered recently,
which require design
modifications. Delays in capturing technology, design, and production
knowledge and these latest problems contributed to cost increases and
schedule delays. The potential exists for further cost increases and
schedule delays as a significant amount of the test program remains,
including operational tests. Also, the latest production cost estimate is
likely to increase because of several factors, and the estimate assumes
over $25 billion in offsets from cost reduction plans.
F/ A- 22 Program
practice standards. In September 2001, the Air Force awarded a contract
for 10 aircraft to begin F/ A- 22 Technology Maturity
production. Although we did not assess the F/ A- 22 key
Other Program Issues
technologies using technology readiness levels, the three critical
technologies (supercruise, stealth, and
In September 2001, the Air Force acknowledged an integrated avionics)
appear mature. Two of these
estimated production cost increase of $5.4 billion technologies,
integrated avionics and stealth, were (then- year dollars) over the
congressional cost limit. late to mature. It was not until September 2000,
or
We believe conditions exist that makes it likely over 9 years into product
development, that the
production costs will increase even further. In integrated avionics
reached maturity. During
addition, the Air Force is counting on over $25 billion development, the
integrated avionics was a source of
in cost reduction plans to offset estimated cost growth schedule delays
and cost growth. Since 1997, avionics
and enable the program to meet the production cost software development
and flight- testing have been
estimate. If these cost reduction initiatives are not delayed, and the
cost of avionics development has
achieved as planned, production costs could increase. increased by over
$980 million. Moreover, the Air
Further, the contractor has yet to demonstrate it can Force did not
complete an evaluation of stealth
efficiently build the development aircraft, and technology on a full-
scale version of the aircraft until
estimates of the cost to build the production aircraft several years into
product development.
continue to increase.
Design Maturity
In December 2002, DOD estimated development costs would increase by $876
million and that the funding The basic design of the F/ A- 22 is
essentially complete,
necessary to cover this cost increase would be as engineering drawings are
complete. However, transferred from production funding. Avionics design
changes have been necessary as a result of
problems discovered in flight- testing are the primary flight tests and
structural tests. For example, problems
contributor to a six- month extension to the with excessive movement of
the vertical tails and
development program. avionics failures in flight tests were discovered,
and they will require costly design modifications. The Air
Program Office Comments
Force still has to complete a significant amount of development testing
and operational testing. Until In commenting on a draft of this
assessment, program initial operational testing is completed as planned in
officials stated that the report implies that had the F/ A22 June 2004,
the possibility of additional design changes
deferred product development until engineering remains. and testing were
accomplished, at a level providing higher product knowledge, substantial
cost increases Design knowledge for the F/ A- 22 was built slowly.
and schedule delays would have been prevented. The Only 26 percent of the
total drawings were released at
issues cited as examples do not pose a substantial risk the 1995 design
review. The program released
to either cost or schedule and have either been fixed 90 percent of the
drawings over 3 years later, after the through minor design change or are
anticipated to be first two development aircraft had been delivered.
resolved without major impact to continued testing Late drawing release
contributed to parts shortages
and production. A program of this nature is expected and work performed
out of sequence during assembly,
to have both design and technological maturities to which drove up costs
and contributed to delaying overcome and there will be some element of
risk flight tests by 83 months.
throughout its development and into the production process. Production
Maturity
The program office stopped collecting process control information in
November 2000. The contractor estimated that nearly half of the key
processes had reached a marginal level of control, but not up to best
Joint Air- to- Surface Standoff Missile (JASSM)
JASSM is a joint Air Force and Navy program designed to attack surface
targets outside of the range of area defenses. JASSM will be delivered by
a variety of aircraft, including the F- 16 C/ D, the B- 52H, the F/ A-
18E/ F, the B- 2, and the B- 1B. The
system includes the missile, software, and software interfaces with the
host aircraft and mission planning system.
Prime contractor: Lockheed Martin
Program office: Eglin, Fla.
FY 2004 funding request FY 2003 dollars in millions Approved
Latest Percent R& D $56.3 million 11/ 98 12/ 02 change
Procurement $102.5 million Research & development cost $863.6 $1,209. 5
40.1 Quantity 250 missiles
Procurement cost $1, 078.7 $2,568. 4 138.1 Total program cost $1, 962.9
$3,777. 8 92.5
Program unit cost $0. 795 $0.852 7.2
Total quantities 2,469 4,434 79.6 Acquisition cycle time (months) 75 87
16.0
The JASSM program entered production in December 2001 without ensuring
that production processes were in control, according to best practice
standards. However, program officials indicated that they have
demonstrated the production processes and that they sample statistical
data at the subsystem level. The program ensured that the technology was
mature and that
the design was stable at critical points in development, closely tracking
best practice standards. Redesign remains one area of concern because
recent test failures have led to the delay of
operational tests. The program has identified fixes to the problems, and a
retrofit plan is in progress. The contractor*s ability to attain a higher
production rate is another area of concern.
JASSM
capacity and expanding facilities to support full- rate production plus
anticipated foreign military sales.
Technology Maturity
Program officials believe that none of the The JASSM program used existing
technologies and so manufacturing processes that affect critical system
its level of technology maturity is high. Although none
characteristics are problematic, although there are of the subsystems
involve new technologies, three
key production processes that have cost implications, critical
technologies are new applications of existing
such as the bonding for the low observable materials technologies. These
three technologies are the global
and the painting/ coating application. positioning system anti- spoofing
receiver module, the
Program Office Comments
low observable technology, and the composite materials. The program office
reports these
In commenting on a draft of this assessment, program technologies to be
mature.
officials stated that JASSM has established a new benchmark for missile
development by ensuring
Design Maturity
weapon system design maturity and production The contractor has released
100 percent of the capability were demonstrated during development
drawings to manufacturing. The two remaining prior to entering low- rate
initial production. JASSM*s concerns are the software for the missile and
the
acquisition strategy incorporated existing technology status of
integration with aircraft, although program to reduce program risk and
speed up delivery of the officials believe the risks are low. weapon to
the warfighter. The officials further stated
that JASSM*s development cycle is 33 percent faster Recent failures in
development and operational tests than comparable weapon systems, with
production have led to the delay of the remaining JASSM unit prices 50
percent less than weapon systems with operational tests. During an
operational test on
less capability. The contractor was contracted to October 10, 2002, the
missile flew its planned route
produce 82 all- up production prove- out test rounds and penetrated the
target, but it failed to detonate.
during development on the production line prior to According to program
officials, this failure occurred
low- rate initial production missile delivery. Program because the
requested test methodology was
officials noted that establishment of production experimental and exceeded
original design representative hardware during development was key
requirements for the fuze. On October 24, 2002, during to the contractor*s
ability to prove out all production the last of 11 developmental tests,
the missile went out
processes. The contractor has a capitalization plan to of control and
crashed at the test site. According to
meet full- rate production quantities. program officials, this failure was
due to a failed actuator. Program officials believe they have identified
the problems in both cases and have a retrofit plan.
Retrofits will be tested in spring 2003. However, if additional problems
occur, they will have to be corrected while JASSM is in production, which
may require additional retrofitting of missiles already produced.
Production Maturity
Program officials do not collect production process control data at the
system level. However, they stated that all production processes had been
demonstrated and that statistical data is collected at the subsystem level
and is sampled as required. Program officials indicated that the
contractor will produce at the rates required for the first production lot
and 76 missiles will be delivered. A contract for the second lot, 100
missiles, has been signed. Production concerns remaining include achieving
full- rate production
Joint Common Missile
The Joint Common Missile is an air- launched and potentially a ground-
launched missile designed to target tanks; light armored vehicles; missile
launchers; command, control, and communications vehicles; bunkers; and
buildings. It is designed to provide line of sight and beyond line- of-
sight
capabilities. It can be employed in a fire- and- forget mode* providing
maximum survivability* or a precision attack mode, providing the greatest
accuracy. The Joint Common Missile will be a joint Army and Navy program
with USMC participation.
Prime contractor: In competition
Program office: Huntsville, Ala.
FY 2004 funding request: FY 2003 dollars in millions
Latest Percent R& D $183.8 million Approved 11/ 02 change
Procurement $0 million
Research & development cost NA $563.92 0 Quantity 0 missiles
Procurement cost NA $1,597. 3 0 Total program cost NA $2,161. 2 0
Program unit cost NA TBD 0
Total quantities NA 8,425 0 Acquisition cycle time (months) NA 60 0 Note:
Funding from FY 2004 President*s Budget. Total Army and Navy Acquisition
Objective is 77,400. Official cost position to be finalized by 6/ 2003. NA
= not applicable.
The Joint Common Missile is scheduled to enter product development before
any of its critical technologies are fully mature, according to best
practices. Furthermore, program officials currently project that the
critical technologies will not reach maturity until a year after the
design review. The Army will initially focus development on an airlaunched
version.
Joint Common Missile System
risk. Prototype testing of a multi- mode seeker (tower and captive
flight), a multi- purpose warhead (heavy
Technology Maturity
armor and building structures), and a rocket motor None of the Joint
Common Missile*s three critical
(high maximum to minimum thrust profiles over technologies have
demonstrated full maturity. These operational temperatures) is currently
being critical technologies include a multi- mode seeker for
conducted with results to be available in sufficient increased
countermeasure resistance, a boost- sustain
time to support the milestone decision to begin the propulsion for
increased standoff range, and a multipurpose development phase. warhead
for increased lethality capability.
Program officials noted that many of the components of these technologies
are currently in production on other missile systems, but that they have
not been fully integrated. While backup technologies exist for each of the
critical technologies, substituting any of them would result in degraded
performance or increased costs.
Design Maturity
Program officials project that full integration of the subsystems into the
Joint Common Missile will be mature one year after the system design
review, which is scheduled for July 2004.
Other Program Issues
The current cost estimates are from the fiscal year 2004 President*s
budget. This cost estimate will be updated at the conclusion of the Army's
formal estimating process. The formal estimating process began in January
2003 for presentation at the milestone decision review in September 2003.
According to program officials the Army's acquisition objective is 54, 400
missiles and the Navy's acquisition objective is 23,000. Program officials
also indicated that the modular design will reduce life- cycle costs,
including demilitarization, and will enable continuous
technology insertion to ensure improvements against advancing threats.
Program Office Comments
In commenting on a draft of this assessment, program officials stated that
they plan to demonstrate the technological maturity required by DOD
acquisition system policy before beginning the development phase in
September 2003. Program officials further stated that the technological
maturity projected represents a major achievement in the technology's
demonstrated readiness in a relevant environment and provides the critical
technologies the maturity necessary to accomplish system integration of
demonstrated subsystems, thereby reducing program
Joint Primary Aircraft Training System (JPATS)
JPATS is a joint acquisition by the Air Force and the Navy to replace the
aging primary trainer aircraft fleet. JPATS is a variant of the Beech
Pilatus PC- 9 commercial aircraft, but it has been modified significantly
to incorporate military unique requirements. The JPATS program includes
the aircraft; the ground- based training system (simulators, course
materials), and an integrated training management system. We assessed the
aircraft.
Prime contractor: Raytheon
Program office: Wright Patterson AFB, Ohio
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $0 million
8/ 95 12/ 01 change
Procurement $283.0 million Research & development cost $349.1 $294. 2
-15.7 Quantity 52 aircraft
Procurement cost $2, 720.3 $4,316. 8 58.7 Total program cost $3, 138.6
$4,674. 8 49.0
Program unit cost $4. 408 $5.970 35.4
Total quantities 712 783 10.0 Acquisition cycle time (months) 97 113 16.5
The JPATS aircraft entered full production in December 2001 without
ensuring that the manufacturing processes were mature. The aircraft
entered limited production in 1995 before achieving design stability. DOD
considered the aircraft a
mature commercial product that did not require extensive product
development. However, program officials underestimated the number of
design changes needed to accommodate the military unique requirements. The
design has subsequently changed about 70 percent from the commercial
baseline. The JPATS initial operating capability occurred in 2002, 2 years
later than originally planned.
JPATS Program
Program Office Comments Technology Maturity
In commenting on a draft of this assessment, program officials disagreed
with our analysis of production Although we did not assess the JPATS
aircraft key
maturity. They stated that statistical process control is technologies,
the aircraft is a derivative of a
not the only determinant of maturity. The production commercial aircraft
and the technologies appear
line was certified by the International Organization for mature.
Standardization in 1994 and by the Federal Aviation Administration in
1999, and is currently producing
Design Maturity
aircraft according to these guidelines. Assembly labor The basic design of
the aircraft is currently complete. hours per aircraft are on a 78 percent
learning curve, However, the military unique design was only about and
they have decreased 65 percent since the first
5 percent complete shortly after the program was operational aircraft was
delivered. The production line
approved to enter limited production in 1995. The rate increased to five
aircraft per month by the end of design has changed about 70 percent from
its
2002, and remains there still, even as design changes commercial baseline.
Testing has revealed tangible
are incorporated into the production line. After initial examples of
design immaturity. Several subsystems,
production difficulties, over the past year the including the engine, the
UHF radio, and the
contractor has been delivering aircraft ahead of environmental control
system, have required schedule while incorporating engineering changes to
extensive modification or redesign. These and other increase the
suitability of the system. Program
problems have delayed both aircraft testing and the officials also stated
that the cycle time should be
production decision. reduced by 6 months because the JPATS program was
unable to award a contract or proceed with contract In November 2001,
operational testers concluded that
performance pending the disposition of several bid JPATS was operationally
effective but not
protests. operationally suitable. They cited concerns about the aircraft's
reliability, availability, and maintainability. GAO Comments
They also reported that the full JPATS had not yet been tested due to
uncorrected deficiencies in the
Our prior work has shown that leading commercial aircraft and the
immaturity of the software- intensive
firms rely on statistical control data as the best training information
management system. The
indicator of production readiness. Despite its contractor is incorporating
changes to the aircraft as a
commercial origins, the JPATS program entered result of operational test
issues. Operational testers
limited and full production without this information. expressed concern
that some changes may adversely Subsequent testing has uncovered numerous
problems impact other critical subsystems. Despite these issues,
that require modification and retrofit. Although the the Air Force
proceeded into full- rate production the
aircraft has been production certified by the Federal following month.
Aviation Administration, its regulations merely require
the contractor to maintain a generic quality control
Production Maturity
system and do not provide assurance that the components can be built
within cost and on schedule. Production maturity remains at issue because
We used DOD official documents to determine information about the
contractor's manufacturing
acquisition cycle time. process controls is not available. The Air Force
did not require this information because the aircraft was considered a
commercial derivative.
Other factors could affect production maturity. In 2002, two key
modifications* the environmental control system and the UHF radio* began
to be incorporated on the aircraft. The program office has also identified
additional retrofit requirements and is evaluating a replacement for the
collision warning system. The rework associated with these changes may
affect aircraft production efficiencies.
F- 35 Joint Strike Fighter (JSF)
The JSF program goals are to develop and field a family of stealthy,
strike fighter aircraft for the Navy, Air Force, Marine Corps, and U. S.
allies, with maximum commonality to minimize life- cycle costs. The
carrier suitable version will complement the
Navy F/ A- 18 E/ F. The Air Force version will primarily be an air- to-
ground replacement for the F- 16 and the A- 10 and complement the F/ A-
22. The short take- off and vertical landing version will
replace the Marine Corps F/ A- 18 and AV- 8B. Significant foreign military
purchases are expected.
Prime contractor: Lockheed Martin
Program office: Arlington, Va.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent
R& D $4. 4 billion 10/ 01 12/ 01 change
Procurement $0 million
Research & development cost $32,788.6 $32,880. 8 0.3 Quantity 0 aircraft
Procurement cost $145,733.8 $147,604. 7 1.3 Total program cost $180,047.0
$180,485. 5 0.2
Program unit cost $62.8 $63. 0 0.2
Total quantities (U. S. only) 2,866 2,866 0.0 Acquisition cycle time
(months) 185 185 0.0
The JSF program entered the development phase without demonstrating that
its eight critical technologies had reached maturity according to best
practice standards. Two technologies, propulsion and critical fabrication
techniques, were very close to maturity. DOD conducted an independent
review in 2001 and concluded that the technology maturity was sufficient
to proceed into product development. The JSF program no longer focuses on
the previous 8 technology areas, instead it uses a different method of
integration and risk management that currently tracks 23 program level
risks. We were unable to assess the new risk areas, but program data
indicates that the majority are
moderate risk. The program expects to have 80 to 90 percent of its
critical build- to- packages completed by the final design review in 2005.
JSF Program
JSF preliminary design review in late March 2003. We were unable to review
the results of those meetings
Technology Maturity
prior to the release of this report, but program office During its concept
development phase, the Joint
data indicates the discovery of higher risk levels for Strike Fighter had
eight critical technologies: short
the propulsion system and overall aircraft weight. take- off vertical
landing/ integrated flight propulsion Other Program Issues
control, prognostic and health management, integrated support systems,
subsystems technology, Due to the highly complex nature of the JSF design,
integrated core processor, radar, mission systems
the Director, Operational Test and Evaluation, expects integration, and
manufacturing. We reported in
numerous test challenges for the program. These May 2000 and again in
October 2001 that low levels of
challenges include the integration of highly advanced maturity in these
technologies could increase the sensors with the avionics systems,
vertical thrust likelihood of program cost and schedule growth.
capability for the Marine Corps version, and performance and maintenance
requirements of the The program experienced cost growth and schedule low
observable capabilities. The program has received concerns during the
concept demonstration phase,
authority for its low- rate production quantity to reach prior to starting
product development in October 2001.
15 percent* 427 aircraft* of the total production run. This included
manufacturing delays for hardware used
on the propulsion system for the Marine Corps
Program Office Comments
version. To reduce cost and schedule delays, the program eliminated
planned risk- reduction efforts and
In commenting on a draft of this assessment, program delayed other
technology demonstrations until after
officials stated that, prior to the start of the product development
began.
development phase, JSF*s key technologies had gone through an extensive
series of tests and An independent review performed by DOD in 2001,
demonstrations, culminating in four experimental using a different method
than technology readiness
aircraft proving flight capabilities for each service levels, concluded
that the overall technology maturity
variant in over 200 hours of flight. An independent of the JSF program was
sufficient to enter into DOD review concluded that JSF had demonstrated
product development. Today, the program no longer
sufficient technical maturity for low risk entry into the monitors the
eight specific technologies from the
development phase. For this phase, the program previous phase. Instead,
the program is using
officials stated that JSF has adapted the contractor's Lockheed Martin*s
Key System Development
risk mitigation approach. Risk mitigation assessments Integration approach
to monitor overall technology
in February 2003 indicated that most program level development and design
integration. Further, the
risks were rated moderate using the contractor*s program tracks 23 program
level risk areas and has approach. Cost and schedule planning for the
assessed 19 as moderate and 2 as high. Five of eight
development phase has evolved as the services critical technologies from
the concept development
iterated system operational requirements with life phase are contained
within elements of these program
cycle cost. The JSF air system preliminary design level risks. We have not
evaluated the current JSF
review is scheduled in March 2003, and the first of technique for
assessing risks.
three critical design reviews is to occur in April 2004. Finally, program
officials stated that the program is
Design Maturity
being executed in accordance with its cost, schedule, The program has
committed time and funding to the and technical baselines. system
development and demonstration phase that should improve its chances for
success. Specifically, the new program structure will now include
additional test aircraft, increased software on the aircraft, and a
greater number of flight test hours. Program documents indicate that the
1996 estimated cost and schedule for JSF*s development phase have
increased by 56 percent and 40 percent, respectively, due to
changes in program scope. Meetings were held for the
Joint Standoff Weapon (JSOW)
JSOW is a joint Air Force and Navy guided bomb to attack targets from
outside of the range of most enemy air defenses. There are three JSOW
variants that use a common air vehicle. Two variants (JSOW A and B) carry
submunitions to attack soft targets or armored vehicles. The unitary
variant (JSOW C) uses a seeker, autonomous targeting acquisition software,
and a single warhead to attack targets. We assessed the unitary variant
and the common air
vehicle.
Prime contractor: Raytheon Systems Company
Program office: Patuxent River, MD
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $0.8 million
6/ 92 12/ 01 change
Procurement $65.89 million
Research & development cost $325.2 $311. 5 -4.2 Quantity 175 missiles
Procurement cost $3, 871.4 $803. 8 -79.2 Total program cost $4, 196.6
$1,115. 3 -73.4
Program unit cost $0. 538 $0.372 -30.9
Total quantities 7,800 3,000 -61.5 Acquisition cycle time (months) 89 112
25.8
The JSOW program is scheduled to begin low- rate production in March 2003
without knowing that production processes are in control, according to
best practice standards. The program instead relies on an after-
production process of inspection to discover defects. Immature technology
at the start of development at least partially delayed design maturity,
and developmental testing of the seeker is not complete.
JSOW Program Program Office Comments Technology Maturity
In commenting on a draft of this assessment, program officials stated that
the contractor has completed The JSOW unitary*s technology appears mature.
The
17 consecutive months of on- schedule deliveries, program office
identified the imaging infrared seeker increasing the inventory to over
850 combat ready with the autonomous acquisition software as the only
assets. In addition, program officials noted that the Air critical
technology for the system. The seeker was not
Force has upgraded its JSOW inventory to mission mature at the start of
development, but it did
ready as a result of a successful resolution of demonstrate maturity in
October 2001* over threefourths remaining manufacturing, navigation, and
vibration through development* when it was flown
tolerance issues. The JSOW unitary continues aboard an aircraft in a
captive flight test. Program
development and its performance is being monitored officials stated that
in three free- flight tests, the
by the program office. seeker's performance substantially exceeded
requirements.
Design Maturity
The JSOW unitary variant*s basic design appears complete. At the system
design review in May 2002, the program office had completed 99 percent of
the drawings. The Navy included nine developmental tests in its
development program* three sled tests with the warhead, three free flights
with the seeker, and three combined warhead/ seeker tests. The Navy has
completed two of the warhead sled tests and the seeker free- flight tests.
Production Maturity
JSOW production maturity could not be determined because the contractor
does not use statistical process controls to ensure that production
processes are stable and units are produced with few, if any, defects.
Rather, the contractor uses a process of postproduction inspection to
control production quality. The contractor collects this postproduction
data on a
factorywide basis that includes JSOW production but is not specific to it.
According to the program office, the contractor delivered end items in the
past that included manufacturing defects. The program office attributes
these defects at least partially to suppliers and to reorganization and
relocation of the prime contractor to Tucson, Arizona. To mitigate the
risk of further manufacturing problems, the Navy has instituted a series
of reviews of major suppliers. The Navy will conduct an additional
production readiness review after the low- rate production is approved.
Program officials report that the contractor is meeting its revised
production schedule and that the scrap and rework rates remain low.
National Polar- orbiting Operational Environmental Satellite System
(NPOESS)
The NPOESS is a joint National Oceanic and Atmospheric Administration
(NOAA), DOD, and National Aeronautics and Space Administration satellite
program to monitor the weather and
environment. Current NOAA and DOD satellites will be merged into a single
national system (NPOESS), with projected savings of at least $1.3 billion.
The program consists of five segments: space; command, control, and
communications; interface data processing; launch; and field terminals.
Prime contractor: Northrop Grumman Space Technology
Program office: Silver Spring, Md.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $544.4 million 8/ 02 9/ 02 change
Procurement $0 million
Research & development cost $4, 029.2 $4,431. 6 10.0 Quantity 0 satellites
Procurement cost $1, 155.6 $1,264. 5 9.4 Total program cost $5, 628.2
$6,183. 4 9.9
Program unit cost $938.0 $1,030. 6 9.9
Total quantities 6 6 0.0 Acquisition cycle time (months) 172 174 1.2
The NPOESS program entered product development in August 2002 with most of
its technologies mature. The program also completed a significant portion
of the engineering drawings well in advance of the design review; however,
the total number has yet to be determined. Over 5 years ago, program
officials considered the program to have
several high- risk areas. Since then, officials have implemented several
efforts, which are expected to reduce all program areas to low risk by the
first NPOESS launch, currently scheduled for the 2008- 2009 time frame.
Perhaps the most significant step taken to reduce risk was to put the
pacing space sensor technologies into full development in advance of the
satellite system itself.
NPOESS Program
radiometer suite, the cross- tracked infrared sounder, the advanced
technology microwave sounder and the
Technology Maturity
ozone mapper/ profiler suite. This satellite will provide The NPOESS
spacecraft and the sensors under
the program office and the data processing centers development consist of
14 key technologies; twelve
with an early opportunity to work with the sensors, were mature at the
start of development in August
ground control, and data processing systems, thus 2002. allowing lessons
learned to be incorporated into the NPOESS satellites. In 1997, the
program office determined that the space segment had high cost and
technical risks and that the
Program Office Comments
interface data processing segment and overall system The NPOESS integrated
program office concurred
integration effort had high cost, schedule, and with this assessment.
technical risks. To reduce the risk to the data processing segment, two
contractors selected for program definition and risk reduction each
conducted four ground- based demonstrations of the data processing
hardware and
software components. Therefore, the program office expects the data
processing segment to be relatively mature before product development.
Program officials indicated that they achieved maturity by concentrating
on the early development of key individual sensors. The acquisition
strategy focused on maturing key sensor technologies using individual
development contracts structured to demonstrate the maturity of each
sensor through a component- level design review prior to the systemlevel
design review. The two technologies that are not mature are needed for two
key sensors* the crosstrack infrared sounder and the conical microwave
imager/ sounder. However, program officials project that those two
technologies will be mature by the system design review in 2005.
Design Maturity
Although the total number of engineering drawings has yet to be
determined, program officials indicated that at least 52 percent of the
6,829 currently identified drawings were completed and released to
manufacturing by the end of January 2003. Program officials further
project that all of the currently identifiable drawings will be complete
by the system
design review in 2005. The program is taking advantage of a unique
opportunity to demonstrate design maturity. The NPOESS Preparatory
Project, a planned demonstration satellite, is to be launched in 2006,
about 2 to 3 years before the first NPOESS satellite launch. The
demonstration satellite is scheduled to carry four critical sensors* the
visible/ infrared imager
Patriot Advanced Capability 3 (PAC- 3) Program
The Army*s Patriot system is a long- range, highmedium altitude air and
missile defense system. PAC- 3 is designed to enhance the Patriot*s
ability to detect and identify missiles and other targets, increase system
computer capabilities and the number of missiles in each launcher, improve
communications, and incorporate a new hit- to- kill missile. The PAC- 3
system has two primary components, the fire unit and the missile. We
assessed both components.
Prime contractor: Raytheon (prime) Lockheed Martin (missile segment)
Program office: Huntsville, Ala.
FY 2003 dollars in millions Approved Latest Percent FY 2004 funding
request: 2/ 95 12/ 02 change
R& D $174.5 million
Research & development cost $2, 760.7 $4,476. 2 62.1 Procurement $561.6
million
Procurement cost $3, 721.9 $7904. 9 112.4 Quantity 108 missiles
Total program cost $6, 482.6 $12,381. 2 91.0
Program unit cost $5. 170 $10.326 99.8
Total quantities 1,254 1,199 -4.4 Acquisition cycle time 66 136 106.1
The PAC- 3 program currently has only about onefourth of its critical
production processes under statistical control using best practice
standards. Continuing problems with producing and testing the missiles are
partially explained by the absence of process control and partially a
consequence of maturing PAC- 3*s design late in development. Technical and
design challenges disrupted the early part of product development, causing
cost and
schedule increases and delays in attaining production knowledge. PAC- 3*s
basic design is now complete and the technology appears mature. However,
the contractor must increase production earlier than planned because DOD
decided to
accelerate deliveries. This decision may present new production challenges
because the contractor must find and train additional personnel.
Patriot PAC- 3 Program
performance was adversely affected by PAC- 3 missile reliability and
launch failures. According to program
Technology Maturity
officials, there were several anomalies caused by Although we did not
assess the PAC- 3 technologies
manufacturing practices, software, and test hardware. using technology
readiness levels, the system's critical
However, they believe there are no systemic issues technologies appear
mature. However, a key and the anomalies have been corrected. A flight
test to technology, the Ka band seeker, was particularly late validate
these corrections is scheduled for the spring
to mature. The seeker did not mature until 1999, close of 2003.
to the low- rate production decision. Problems The program has adopted an
evolutionary acquisition
experienced during development increased the approach, with production
decisions every 2 to 3
seeker's cost by 76 percent and delayed the contractor years. In October
2002, DOD decided to buy 208
in attaining design and production knowledge. missiles covering the next 2
years. DOD plans to
accelerate the production rate immediately by adding
Design Maturity
a second manufacturing shift and test equipment. PAC- 3*s basic design is
complete, with 100 percent of Because production was not expected to be
the drawings released to manufacturing. Only
accelerated to this level this early in production, the 21 percent of the
drawings were complete when the contractor must expeditiously find and
train qualified program held its design review, which led to a number
personnel. The accelerated plan requires additional of problems. For
example, the contractor attributed a
funding of $239 million for fiscal years 2003 and 2004. $101 million cost
increase to first- time manufacturing
Program Office Comments
problems, such as some subsystems not fitting together properly and some
not passing ground or In commenting on a draft of this assessment, program
environmental tests. These problems were a major
officials stated that they believe production processes contributor to a
2- year schedule delay. To reduce
are in control. Program officials stated that they have missile costs, the
contractor has identified several meticulously and methodically examined
every critical major design changes, which will be incorporated into
process from a labor and inspection standpoint to help the design in 2004.
ensure a consistent and quality product. Despite the less than fully
successful operational tests, they also
Production Maturity
believe that they have the most successful The program has 23 percent of
the key manufacturing development flight test program in the history of
processes used to assemble the missile and the seeker missile development.
They provided technical
under control. Production maturity has deteriorated comments, which were
incorporated as appropriate.
from the 35 percent that was in control at the October 1999 low- rate
production decision. A switch in the manufacturing facilities may have
played a role. According to program officials, the program entered
production before process control was emphasized to the contractor. The
contractor is still having difficulties building the missile. For example,
each seeker still needs to be reworked about three times on
average before it passes quality inspections. Program officials have added
quality tests of components, which have improved the situation, but the
contractor
has not yet demonstrated that these tests will eliminate the need for
seeker rework in the future.
Other Program Issues
The Army conducted four operational tests in 2002; none were completely
successful. The PAC- 3 system defeated half of the targets in flight-
testing. System
Space Based Infrared System (SBIRS) High
SBIRS High will consist of a constellation of four satellites plus one
spare, two sensors on a nonSBIRS satellite, and associated ground
stations. SBIRS High is to provide missile warning and missile defense
information and will be used to support the technical intelligence and
battlespace characterization missions. The first launch of SBIRS High is
scheduled for fiscal year 2007.
Prime contractor: Lockheed Martin
Program office: El Segundo, Cal.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D: $617.2 million 3/ 98 6/ 02 change
Procurement: $95.4 million Research & development cost $3, 378.4 $6,077. 3
79.9 Quantity: 1 satellite Procurement cost $558.1 $1,417. 4 154.0
Total program cost $4, 127.0 $8,241. 2 99.7
Program unit cost $825.4 $1,648. 2 99.7
Total quantities 5 5 0. 0 Acquisition cycle time (months) NA NA NA Note:
NA = not available.
The SBIRS High program*s critical technologies have demonstrated
acceptable levels of maturity. This level of maturity follows many years
of difficult
development. The level of design stability is unknown since the contractor
was unable to provide information on the total number of releasable
drawings at specific milestones.
Similarly, production maturity could not be determined because the
contractor does not collect statistical control data. The SBIRS High
program is building the first two satellites using research and
development funding with a first launch expected in fiscal year 2007. The
program also recently underwent a major restructuring to reduce program
risk.
SBIRS High Program
10 U. S. C. 2433) occurred on December 31, 2001, requiring a review by the
Secretary of Defense and a
Technology Maturity
report to Congress. As a result, DOD certified to The SBIRS High program*s
three critical
Congress in May 2002 that the SBIRS High program is technologies* the
infrared sensor, thermal essential for national security, there are no
management, and the on- board processor* are now
alternatives that provide equal or greater capability at mature. Program
officials indicated that the hardware
less cost, cost estimates are reasonable, and the was built and tested in
a thermal vacuum chamber management structure is in place to continue to
keep
under expected flight conditions. When the program costs under control.
began product development in 1996, none of its
Program Office Comments
critical technologies were mature, according to best practice standards.
Program officials generally concurred with our assessment and provided
technical comments, which
Design Maturity
we have incorporated where appropriate. Program Program officials do not
know how many total
officials added that the fiscal year 2004 budget fully drawings are
expected for SBIRS High, and thus do funds their restructured program and
directs the not track the number of releasable drawings. As a
satellite procurement to begin in fiscal year 2006. result, we could not
assess design stability relative to best practices. Program officials did
state that the current number of releasable drawings is 2,342, about twice
the number at the time of the design review. This means that at most, no
more than half of the drawings could have been releasable at the design
review. Design stability has been an issue for this program. During
development, the satellite was redesigned to
maintain key performance parameters. Redesign efforts resulted in a 6-
month slip to the spacecraft and increased the requirement for ground
processing.
On the other hand, the two sensors that will be aboard non- SBIRS
satellites are considered stable with subsystem qualification nearing
completion, and integration and delivery of the flight payload are
expected within the year. The first of these sensors is scheduled for
delivery in May 2003* three months behind schedule. This delay is
attributed to problems
with radio waves emitted by the sensor's electronics that interfere with
the host satellite. Despite these integration difficulties, data shows
that the sensors will perform much better than expected.
Production Maturity
We could not assess the SBIRS High production maturity relative to best
practice standards because the contractor does not use statistical process
control
to ensure that production processes are stable.
Other Program Issues
The total unit cost of the SBIRS High program rose more than 25 percent in
1 year. The notification to Congress of the Nunn- McCurdy breach (see
Theater High Altitude Area Defense (THAAD)
THAAD is an element of the terminal defense segment of the Ballistic
Missile Defense System. Its mission is to defend against short and medium
range ballistic missiles. THAAD*s ability to intercept
inside and outside the atmosphere makes effective countermeasures more
difficult and allows multiple intercept opportunities. The system includes
missiles, launchers, radars, command and control/ battle management (C2/
BM), and THAAD support equipment.
Prime contractor: Lockheed Martin
Project office: Huntsville, Ala.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $730.6 million 1/ 92 2/ 03 change
Procurement $0 million
Research & development cost $4, 382.7 $10,548. 0 138.0 Quantity 0 missiles
Procurement cost NA NA NA Total program cost NA NA NA
Program unit cost NA NA NA
Total quantities (U. S. only) NA NA NA Acquisition cycle time (months) 114
NA NA Note: Procurement schedule, funding, and quantities have yet to be
determined. The THAAD schedule no longer includes production milestones.
NA = not applicable.
Most of THAAD*s critical technologies have demonstrated acceptable levels
of maturity and the program appears close to meeting the best practice
standard for a stable design. The program*s launcher and radar have
essentially attained technological maturity, but the missile and the
command and control/ battle management components are somewhat less
mature. This level
of maturity follows many years of difficult development. It appears that
the THAAD program has mostly recovered from initial problems driven by an
early fielding requirement and poor quality control. The current THAAD
acquisition strategy shows a much greater emphasis on attaining knowledge.
The program expects to reach technological maturity and design stability
by February 2004.
THAAD Program
missile in the more stressing flight environment inside the atmosphere,
and (3) prepares the system for initial
Technology Maturity
operational test and evaluation. THAAD program officials assessed 47
technologies in
Program Office Comments
four major elements* command and control/ battle management; missile
interceptor; launcher; and radar. In commenting on a draft of this
assessment, program Of the four elements, the radar is currently the most
officials stated that, to ensure the highest probability mature, followed
by the launcher, command and
of success in flight- testing, a substantial amount of control/ battle
management, and the missile. The
ground testing is being conducted in the next year and program has made
progress on technology maturity
a half. This testing includes exhaustive engineering since it began
development despite early failures in
and qualification level testing on all flight components. intercept
attempts. Early flight- test failures were
Program officials further stated that the extensive caused by a
combination of the compressed test
design, fabrication, and test preparation activity has schedule and
quality control problems. The program
been very successful to date, and the program remains was restructured
twice, before the first successful
healthy with a slightly ahead- of- schedule and undercost intercept
occurred in 1999. The research and status. development cost grew from $4.4
to $10.5 billion prior to the program's transfer to the Missile Defense
Agency, partially as a result of these problems.
The current program strategy appears geared to obtaining the necessary
knowledge by providing more time for maturing the technology before flight
tests and placing greater emphasis on risk reduction efforts. This
strategy includes utilizing technology readiness levels to assess
technological maturity.
Design Maturity
The program has released about 82 percent of total drawings. Program
officials expect to release about 91 percent of the drawings by the
system- level design review in February 2004. The Missile Defense Agency
is redesigning the missile to be more reliable and testable, with
significantly fewer parts than the previous version. The first flight test
of the redesigned missile is not scheduled to occur until at least 6
months after the system design review. Depending on the outcome, flight
tests could require more design changes and delay achieving design
stability.
Other Program Issues
THAAD was recently transferred from the Army to the Missile Defense
Agency, which has restructured and modified the contract to a block
upgrade approach. Therefore, limited information is currently available on
the total projected costs of this program.
In response to the prior program setbacks, the THAAD project office is
accelerating some risk reduction activities, and it has planned a series
of flight tests that (1) tests the missile in a less stressing intercept
environment outside the atmosphere, (2) tests the
Tactical Tomahawk Missile
The Navy*s Tactical Tomahawk (block IV) is a major upgrade to the Tomahawk
Land Attack Missile (block III). The Tactical Tomahawk missile will
provide ships and submarines with enhanced capability to attack targets on
land. New features include improved antijamming global positioning system,
in- flight retargeting, and ability to transmit battle damage imagery. The
system includes the missile, the weapon control system, and the mission
planning system. We assessed only the missile.
Prime contractor: Raytheon Systems Company
Program office: Patuxent River, Md.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $71.4 million
9/ 99 12/ 01 change
Procurement $277.6 million
Research & development cost $559.5 $584. 1 4.4 Quantity 267 missiles
Procurement cost $1, 236.4 $1,546. 0 25.0 Total program cost $1, 795.9
$2,130. 0 18.6
Program unit cost $1. 316 $1.235 -6.1
Total quantities 1,365 1,725 26.4 Acquisition cycle time (months) 58 69
19.0
The Tactical Tomahawk missile entered low- rate production without
ensuring that production processes were in control. Program officials
indicated that they plan to collect production process control data over
the next year, prior to
award of the full- rate production contract in fiscal year 2004. At that
time, program officials expect over 80 percent of the low- rate production
missiles to be in various stages of assembly. The technology and design
have reached acceptable levels of maturity. While engineering drawings
have improved to 96 percent, the program only had about
half of its drawings released at the design review. Program plans call for
a full- rate production decision in May 2004.
Tactical Tomahawk Program
awarded in mid- January 2003 for 167 units. Program officials stated that
total quantities have increased to
Technology Maturity
2,396. We did not assess the technology readiness levels of
Program Office Comments
the key technologies for the Tactical Tomahawk missile. At the time of our
review, critical technologies
In commenting on a draft of this assessment, program were mature.
According to the program office, the
officials stated that two development test flights, critical technologies
for the key subsystems* antijam
conducted prior to low- rate production awards, global positioning system,
digital scene matching area demonstrated that the Tactical Tomahawk
missile correlator, and cruise engine* were modified design met or
exceeded technical and key derivatives from other programs or upgrades to
performance parameters. They also noted that, due to existing Tomahawk
subsystems and consequently
the stability of the design and successful completion already mature. To
date, subsystem and the majority
of all component and flight qualification testing, the of missile- level
qualification testing has been
Navy*s operational test agency issued a favorable completed successfully.
operational assessment, stating that the Tactical Tomahawk missile is
potentially suitable and
Design Maturity
potentially operationally effective. The basic design of the Tactical
Tomahawk missile is essentially complete. The critical design review
occurred in June 2000. At that time, approximately
47 percent of the drawings had been released to manufacturing. In October
2002, at the first low- rate initial production award, 723 of 750 total
drawings, or about 96 percent, had been released.
Production Maturity
Officials plan to collect statistical control data at the start of the
manufacturing process but do not expect to have meaningful statistical
data until sometime in 2004. Manufacture of the Tactical Tomahawk missile
is scheduled to begin at the subcontractor's facility in 2003 and missile
assembly in 2004. Although two lowrate production contracts have been
awarded, program officials stated that data regarding manufacturing
process controls currently is very limited. Program officials told us that
it is too soon to
know what percentage of critical manufacturing processes will be under
statistical control when the full- rate production contract is awarded in
mid- 2004, but that they plan to start collecting production process
control data over the next year.
Other Program Issues
The Tactical Tomahawk missile successfully completed its first
developmental flight test in August 2002, and the first low- rate
production contract for 25 units was awarded in October 2002. A second and
final low- rate production contract was
V- 22 Ospr ey
The V- 22 Osprey is a tilt- rotor, vertical takeoff and landing aircraft
designed to meet the amphibious/ vertical assault needs of the Marine
Corps, longrange missions of Special Operations forces, and combat search
and rescue needs of the Navy. The
V- 22 will replace the CH- 46E and the CH- 53A/ D in the Marine Corps; the
H- 53 and H- 60 will augment the C- 130 in the Air Force and the Special
Operations Command; and supplement the H- 60 in the Navy. We assessed the
block A version.
Prime contractor: Bell- Boeing
Program office: Patuxent River, Md.
FY 2004 funding request: FY 2003 dollars in millions Approved Latest
Percent R& D $543.9 million 2/ 87 12/ 01 change
Procurement $1.11 billion
Research & development cost $3, 568.5 $10,253. 5 187.3 Quantity 11
aircraft
Procurement cost $29,499.2 $32,312. 7 9.5 Total program cost $33,264.7
$42,617. 5 28.1
Program unit cost $36.434 $93.051 155.4
Total quantities 913 458 -49.8 Acquisition cycle time (months) 117 261
123.1
The V- 22 program plans to enter full- rate production without ensuring
that the manufacturing processes are mature. Redesign of the aircraft*s
hydraulic and electric system, and software changes have been made to
address safety, reliability, maintainability,
and logistics supportability. These design changes and others are
undergoing developmental testing to ready the aircraft for an operational
test and evaluation test period in late 2004 through early 2005 to
determine if the V- 22 is operationally suitable and effective. The design
changes, however, have not been incorporated into the lowrate
production aircraft currently being produced. The value of contract
modifications needed to address the cost of these design changes is not
yet known. Also, parts shortages and quality issues are currently
effecting low- rate production costs. Some key performance requirements
have been eliminated.
V- 22 Program
and incorporated into production aircraft. Delivery of block A aircraft is
expected to start in the fourth
Technology Maturity
quarter of fiscal year 2003. However, the cost of Although we did not
specifically assess the V- 22*s contract modifications needed to
reconfigure already technology maturity, the program office believes key
produced aircraft and aircraft still on the assembly technologies to be
mature. An operational test report, line to the bock A configuration has
not been dated November 2000, determined that the V- 22 was
negotiated. not operationally suitable because of poor reliability,
Program Office Comments
maintainability, availability, human factors, and interoperability
problems. Immature technology, in
In commenting on draft of this assessment program part, contributed to
this assessment.
officials stated that they have restructured the program to gather more
technical knowledge through
Design Maturity
a more rigorous *event- driven* flight test program. As a result of a
crash in December 2000, the V- 22 has
Program officials strongly disagreed that the program undergone several
design changes. Specifically, the
plans to enter full- rate production without ensuring aircraft*s hydraulic
and electrical lines were that manufacturing process are mature. V- 22s
are
redesigned to improve safety, reliability, currently being manufactured at
a minimum sustaining rate (11 aircraft per year). A May 20 th
maintainability, and logistics supportability. The V- 22 defense
acquisition board review is scheduled to flight control system software
was also redesigned.
consider increasing this rate. Manufacturing processes The program office
estimates that redesign of the V- 22 and tooling are in place and being
continually analyzed resulted in 1, 755 additional drawings, increasing
the
and improved. Both companies utilize statistical total number of drawings
to 7,490. To date, all of these
process control techniques and numerous metrics to drawings are complete.
assess program performance. They do not use the The success of these
design changes will be process capability index, the only metric that GAO
determined as the aircraft undergoes additional
uses as a basis for their assessment. Program officials developmental
testing through 2005. Testing will
are also undertaking an affordability review to reduce address many
issues, including high rate of descent,
the aircraft unit cost to $58 million by 2010. High unit handling
qualities, austere environment operations, costs are driven by the current
low production and ship operations. The operational assessment of
quantities and will remain the norm until production these characteristics
will not occur until late 2004 or quantities increase. early 2005. Recent
decisions to defer some V- 22 operational requirements previously
considered
GAO Comments
critical until later blocks will void the need for some design changes in
the block A.
Our prior work has shown that leading commercial firms rely on statistical
control data, specifically, the
Production Maturity
process capability index, as the best indicator of production readiness.
The V- 22 program entered lowrate Neither V- 22 contractor collects
statistical process
production without this information and has control data on its critical
manufacturing processes. A
experienced production quality problems. recent program management
assessment rated V- 22 production as cautionary. Part shortages and
quality problems caused inefficiencies in shop and assembly operations, as
well as scrap, rework, repair, and schedule delays. Other Program Issues
Low- rate production of the V- 22 continues. V- 22s are being fabricated
and partially assembled, but not delivered until the first set of
upgrades* referred to as block A* needed to bring the V- 22 to a safe
operational and suitable configuration are approved
Wideband Gapfiller Satellite (WGS) Communications System
The Wideband Gapfiller Satellite system is a joint Air Force and Army
program intended to provide communications to the U. S. warfighters,
allies, and Coalition Partners during all levels of conflict short
of nuclear war. It is the next generation wideband component in the DOD*s
future Military Satellite Communications architecture.
Prime contractor: Boeing Satellite Systems (BSS)
Program office: El Segundo, Calif.
FY 2004 funding request: FY 2003 dollars in millions Estimate Latest
Percent R& D $36.7 million 12/ 01 12/ 02 change
Procurement $34.6 million Research & development cost $181. 2 $244.8 35.1
Quantity 0 satellites
Procurement cost $831. 5 $1, 389.8 67.2 Total program cost $1,012. 7 $1,
634.6 61.4
Program unit cost $337. 6 $326.9 -3.2
Total quantities 3 5 66.7 Acquisition cycle time (months) 50 55 10.0
The WGS program*s critical technologies and design are mature, while its
production processes are nearly mature. DOD plans to rely on commercial
technologies that will not require extensive product development. However,
two of these processes use statistical control rates that are below the
level prescribed by best practice standards. The program recently added
two satellites to better support intelligence, surveillance, and
reconnaissance missions in the future.
WGS Program
The WGS program*s two critical technologies* the digital channelizer and
the phased array antenna* are mature. Most of these technologies are
commercial derivatives. For this reason, many of the satellite
technologies selected were already at high levels of maturity. In fact,
the program is leveraging commercial technology and practices by modifying
commercial satellites to better support unique military requirements.
Design Maturity
The WGS design is essentially complete, as the program has released
approximately 95 percent of the expected drawings.
Production Maturity
The contractor has six of its eight key manufacturing processes under
control, according to the best practice standards. Program officials
indicated that they are bringing the remaining processes under statistical
control.
Program Office Comments
In commenting on a draft of this assessment, program officials stated that
while critical technology areas being applied to WGS are fairly mature,
the manufacturing of the systems using these technologies is relatively
new for the contractor. Risk of production problems was to be reduced due
to other commercial
satellite system developments and production ahead of WGS in the
development and production schedule. However, due to the drastic loss of
commercial satellite orders, only one commercial satellite with
similar technologies as WGS is now leading WGS in the manufacturing
schedule. Recently identified problems found on the *leader* program will
impact the WGS manufacturing schedule, and a first launch schedule delay
of 4 to 6 months can be expected due to time needed to resolve the
*leader* program manufacturing problems. Satellites four and five have
been directed by DOD to be launched in fiscal year 2009 and fiscal year
2010, respectively. These dates are outside the allowable dates of the WGS
contract
option clauses and will require renegotiation to finalize their cost. The
cost is expected to increase to compensate for loss of learning curve from
over a 3-
year break in production, parts obsolescence, and inflation.
Appendi x II
Methodology In conducting our work, we evaluated performance and risk data
from each of the programs included in this report. We summarized our
assessments of each individual program in two components* a system profile
and a product knowledge assessment. We did not validate or verify the data
provided by DOD. However, we took several steps to address data quality.
Specifically, we reviewed the data and performed various quality checks,
which revealed some discrepancies in the data. We discussed these
discrepancies with program officials and adjusted the data accordingly.
System Profile
In the past 3 years, DOD revised its policies governing weapon system
Assessment acquisitions and changed the terminology used for major
acquisition events. In order to make DOD*s acquisition terminology more
consistent across the 26 program assessments, we standardized the
terminology for key program events. In the individual program assessments,
program start refers to the initiation of a program; DOD usually refers to
program start as milestone I or milestone A, which begins the concept and
technology development phase. Similarly, development start refers to the
commitment to product development that coincides with either milestone II
or milestone B, which begins DOD*s system development and demonstration
phase. The production decision generally refers to the decision to enter
the production and deployment phase, typically with low- rate initial
production. Initial capability refers to the initial operational
capability, sometimes also called first unit equipped or required asset
availability.
The funding request information presented refers to the President*s fiscal
year 2004 budget request, except where noted. The program cost comparisons
are the latest estimates provided by the individual programs. The
quantities listed refer to total quantities, including both procurement
and development quantities.
To assess the cost, schedule, and quantity changes of each program, we
reviewed DOD*s selected acquisition reports or obtained data directly from
the program offices. In general, we compared the latest available selected
acquisition report information with a baseline for each program. For
systems that have started product development* those that are beyond
milestone II or B* we compared the latest available Selected Acquisition
Report to the development estimate from the first Selected Acquisition
Report issued after the program was approved to enter development. For
systems that have not yet started product development, we compared the
latest available data to the planning estimate issued after milestone I or
A. For systems not included in selected acquisition reports, we attempted
to
obtain comparable baseline and current data from the individual program
offices.
All cost information is presented in base year 2003 dollars, unless
otherwise noted, using Office of the Secretary of Defense approved
deflators to eliminate the effects of inflation. We have depicted only the
programs* main elements of acquisition cost* research and development, and
procurement, however the total program costs displayed also include
military construction and acquisition operation and maintenance costs.
Because of rounding and these additional costs, in some situations the
total cost may not match the exact sum of the research and development and
procurement costs. The program unit costs are calculated by dividing the
total program cost by the total quantities planned. These costs are often
referred to as program acquisition unit costs.
The schedule assessment is based on acquisition cycle time, defined as the
number of months between the program start, usually milestone I or A, and
the achievement of initial operational capability or an equivalent
fielding date.
The intent of these comparisons is to provide an aggregate or overall
picture of a program*s history. These assessments represent the sum total
of the federal government*s actions on a program, not just those of the
program manager and the contractor. DOD does a number of detailed analyses
of changes that attempt to link specific changes with triggering events or
causes. Our analysis does not attempt to make such detailed distinctions.
Product Knowledge To assess the product development knowledge of each
program at key
Assessment points in development, we submitted a data collection
instrument to each
program office. The results are graphically depicted in each two- page
assessment. The methodology used to generate each graph is discussed at
the beginning of appendix I. We also reviewed pertinent program
documentation, such as the operational requirements document, the
acquisition program baseline, test reports, and major program reviews.
To assess technology maturity, we asked program officials to apply a tool,
referred to as technology readiness levels, for our analysis. The National
Aeronautics and Space Administration originally developed technology
readiness levels, and the Army and Air Force Science and Technology
research organizations use them to determine when technologies are ready
to be handed off from science and technology managers to product
developers. Technology readiness levels are measured on a scale of one to
nine, beginning with paper studies of a technology*s feasibility and
culminating with a technology fully integrated into a completed product.
Our best practices work has shown that a technology readiness level of 7*
demonstration of a technology in an operational environment* is the level
of technology maturity that constitutes a low risk for starting a product
development program. In most cases, we did not validate the program
offices* selection of critical technologies or the determination of the
demonstrated level of maturity. We sought to clarify the technology
readiness levels in those cases where
information existed that raised concerns. If we were to conduct a detailed
review, we might adjust the critical technologies assessed, the readiness
level demonstrated or both. It was not always possible to reconstruct the
technological maturity of a weapon system at key decision points after the
passage of many years.
To assess design maturity, we asked program officials to provide the
percentage of engineering drawings completed or projected for completion
by the design review, the production decision, and as of our current
assessment. Completed engineering drawings were defined as the number of
drawings released or deemed releasable to manufacturing that can be
considered the *build to* drawings.
To assess production maturity, we asked program officials to identify the
number of critical manufacturing processes and, where available, to
quantify the extent of statistical control achieved for those processes.
We used a standard called the Process Capability Index, which is a process
performance measurement that quantifies how closely a process is running
to its specification limits. 1 The index can be translated into an
expected product defect rate and we have previously found it to be a best
practice. We sought other data, such as scrap and rework trends in those
cases
where quantifiable statistical control data was unavailable. 1 Process
Capability Index provides assurance that production processes are under
100 percent statistical control. A high index value equates to fewer
defects per part based on statistical process control data. The general
rule of thumb used by the manufacturing industry states that if the index
value for a process is less than 1.33, then the process is not capable of
producing a part with acceptable consistency.
Although the knowledge points provide excellent indicators of potential
risks, by themselves, they do not cover all elements of risk that a
program encounters during development, such as funding instability. Our
detailed reviews on individual systems normally provide for a fuller
treatment of risk elements.
Appendi x III
GAO Contact and Acknowledgments GAO Contact Paul Francis (202) 512- 2811
Acknowledgments David B. Best, James A. Elgas, and James L. Morrison made
key
contributions to this report and were largely supported by GAO*s
Acquisition and Sourcing Management staff. The following staff were
responsible for individual programs.
Staff System
Michael Hazard Advanced Amphibious Assault Vehicle (AAAV) Steven Martinez
Advanced Extremely High Frequency
(AEHF) Communications Satellite David Hubble Advanced Wideband Satellite/
Transformational Communications
Satellite (AWS/ TSAT) Gaines Hensley AN/ APG- 79 Active Electronically
Scanned
Array (AESA) Radar Marvin Bonner AIM- 9X Short- Range Air- to- Air Missile
Thomas Gordon Airborne Laser (ABL) Dana Solomon/ Carrie
Advanced Threat Infrared Wilson/ Danny Owens
Countermeasures/ Common Missile Warning System (ATIRCM/ CMWS) Johanna
Ayers Cooperative Engagement Capability (CEC) Leon Gill CH- 47F Improved
Cargo Helicopter Wendy Smythe RAH- 66 Comanche Marti Dey/ Ron Schwenn EX-
171 Extended Range Guided Munition
(ERGM) Larry Gaston Excalibur Artillery Round Cheryl Andrew F/ A- 18E/ F
Super Hornet Donald Springman F- 22 Raptor Beverly Breen/ Lynn
Joint Air- to- Surface Standoff Missile (JASSM) Lavalle Danny Owens Joint
Common Missile Rae Ann Sapp/ Art Cobb Joint Primary Aircraft Training
System (JPATS)
Brian Mullins/ Ron F- 35 Joint Strike Fighter (JSF) Schwenn
Staff System
Carol Mebane Joint Standoff Weapon (JSOW) Bruce Thomas National Polar-
orbiting Operational
Environmental Satellite System (NPOESS) Matthew Lea Patriot Advanced
Capability 3 Program
(PAC- 3) Tana Davis Tactical Tomahawk Missile William Lipscomb/ Tana
Theater High Altitude Area Defense (THAAD) Davis Maricela Cherveny/ Nancy
Space Based Infrared Satellite- High (SBIRSHigh) Rothlisberger Jerry Clark
V- 22 Osprey Art Gallegos/ Tony
Wideband Gapfiller Satellite Communication Beckham
System
(120185)
GAO*s Mission The General Accounting Office, the audit, evaluation and
investigative arm of Obtaining Copies of GAO Reports and Testimony
To Report Fraud, Waste, and Abuse in Federal Programs
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this
list to you every afternoon, go to www. gao. gov and select *Subscribe to
GAO Mailing Lists* under *Order GAO Products* heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO
also accepts VISA and Mastercard. Orders for 100 or more copies mailed to
a single address are discounted 25 percent. Orders should be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548 To order by Phone: Voice: (202) 512- 6000 Contact:
TDD: (202) 512- 2537 Fax: (202) 512- 6061 Web site: www. gao. gov/
fraudnet/ fraudnet. htm E- mail: fraudnet@ gao. gov Automated answering
system: (800) 424- 5454 or (202) 512- 7470 Public Affairs Jeff Nelligan,
Managing Director, NelliganJ@ gao. gov (202) 512- 4800 U. S. General
Accounting Office, 441 G Street NW, Room 7149 Washington, D. C. 20548
a
GAO United States General Accounting Office
GAO assessed 26 defense programs ranging from the Marine Corps* Advanced
Amphibious Assault Vehicle to the Missile Defense Agency*s Theater High
Altitude Area Defense system. GAO*s assessments are anchored in a
knowledge- based approach to product development that reflects best
practices of successful programs. This approach centers on attaining high
levels of knowledge in three elements of a new product or weapon*
technology, design, and production. If a program is not attaining this
level of knowledge, it incurs increased risk of technical problems,
accompanied by cost and schedule growth (see figure). If a program is
falling short in one element, like technology maturity, it is harder to
attain knowledge in succeeding elements.
All of the programs GAO assessed proceeded with less knowledge at critical
junctures than suggested by best practices, although several came close to
meeting best practice standards. GAO also found that programs generally
did not track statistical process control data, a key indicator for
production maturity. Program stakeholders can use these assessments to
recognize the gaps in knowledge early and to take advantage of
opportunities for constructive intervention* such as adjustments to
schedule, trade- offs in requirements, and additional funding. GAO has
summarized the results of its assessments in an easy to read twopage
format. Each two- page assessment contains a profile of the product that
includes a description; a timeline of development; a baseline comparison
of cost, schedule, and quantity changes to the program; and a graphical
and narrative depiction of how the product development knowledge of an
individual program compared to best practices. Each program office
submitted comments and they are included with each individual assessment
as appropriate. The weapons the Department of
Defense (DOD) develops have no rival in superiority. How they are
developed can be improved, without sacrificing the superiority of the
outcome. GAO*s reviews over the past 20 years have found consistent
problems with weapon investments* cost increases,
schedule delays and performance shortfalls* along with underlying causes,
such as pressure on managers to promise more than they can deliver. The
best practices of successful product developments offer a knowledgebased
approach DOD can use to improve the way it develops new weapons.
This report is new for GAO, and draws on its work in best practices for
product development. GAO*s
goal for this report is to provide congressional and DOD decision makers
with an independent, knowledge- based assessment of
defense programs that identifies potential risks, and offers an
opportunity for action when a program*s projected attainment of knowledge
diverges from the best practice. It can also highlight those
programs that employ practices worthy of emulation by other programs. GAO
plans to update and issue this report annually to the congressional
defense committees.
GAO makes no recommendations. Program office comments are included in the
assessments of each individual program. DEFENSE ACQUISITIONS
Assessments of Major Weapon Programs
www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 476. To view the full report,
including the scope and methodology, click on the link above. For more
information, contact Paul Francis at (202) 512- 4841 or francisp@ gao.
gov. Highlights of GAO- 03- 476, a report to
Congressional Committees
May 2003
Page i GAO- 03- 476 Acquisition Trends and Risks
Contents
Contents
Page ii GAO- 03- 476 Acquisition Trends and Risks
Page iii GAO- 03- 476 Acquisition Trends and Risks United States General
Accounting Office Washington, D. C. 20548
Comptroller General of the United States A
Page iv GAO- 03- 476 Acquisition Trends and Risks
Page 1 GAO- 03- 476 Acquisition Trends and Risks United States General
Accounting Office Washington, D. C. 20548
Page 1 GAO- 03- 476 Acquisition Trends and Risks
A
Page 2 GAO- 03- 476 Acquisition Trends and Risks
Page 3 GAO- 03- 476 Acquisition Trends and Risks
Page 4 GAO- 03- 476 Acquisition Trends and Risks
Page 5 GAO- 03- 476 Acquisition Trends and Risks
Page 6 GAO- 03- 476 Acquisition Trends and Risks
Page 7 GAO- 03- 476 Acquisition Trends and Risks
Page 8 GAO- 03- 476 Acquisition Trends and Risks
Page 9 GAO- 03- 476 Acquisition Trends and Risks
Appendix I
Appendix I Assessments of Individual Programs
Page 10 GAO- 03- 476 Acquisition Trends and Risks
Appendix I Assessments of Individual Programs
Page 11 GAO- 03- 476 Acquisition Trends and Risks
Appendix I Assessments of Individual Programs
Page 12 GAO- 03- 476 Acquisition Trends and Risks
Appendix I Assessments of Individual Programs
Page 13 GAO- 03- 476 Acquisition Trends and Risks
Appendix I Assessments of Individual Programs Page 14 GAO- 03- 476
Acquisition Trends and Risks
Appendix I Common Name: AAAV
GAO- 03- 476 Acquisition Trends and Risks Page 15
Page 16 GAO- 03- 476 Acquisition Trends and Risks
Common Name: AAAV Appendix I
Appendix I Common Name: ABL
GAO- 03- 476 Acquisition Trends and Risks Page 17
Page 18 GAO- 03- 476 Acquisition Trends and Risks
Common Name: ABL Appendix I
Appendix I Common Name: AEHF
GAO- 03- 476 Acquisition Trends and Risks Page 19
Page 20 GAO- 03- 476 Acquisition Trends and Risks
Common Name: AEHF Appendix I
Appendix I Common Name: AESA
GAO- 03- 476 Acquisition Trends and Risks Page 21
Page 22 GAO- 03- 476 Acquisition Trends and Risks
Common Name: AESA Appendix I
Appendix I Common Name: AIM- 9X GAO- 03- 476 Acquisition Trends and Risks
Page 23
Page 24 GAO- 03- 476 Acquisition Trends and Risks
Common Name: AIM- 9X Appendix I
Appendix I Common Name: ATIRCM/ CMWS
GAO- 03- 476 Acquisition Trends and Risks Page 25
Page 26 GAO- 03- 476 Acquisition Trends and Risks
Common Name: ATIRCM/ CMWS Appendix I
Appendix I Common Name: AWS
GAO- 03- 476 Acquisition Trends and Risks Page 27
Page 28 GAO- 03- 476 Acquisition Trends and Risks
Common Name: AWS Appendix I
Appendix I Common Name: CEC GAO- 03- 476 Acquisition Trends and Risks Page
29
Page 30 GAO- 03- 476 Acquisition Trends and Risks
Common Name: CEC Appendix I
Appendix I Common Name: CH- 47F
GAO- 03- 476 Acquisition Trends and Risks Page 31
Page 32 GAO- 03- 476 Acquisition Trends and Risks
Common Name: CH- 47F Appendix I
Appendix I Common Name: Comanche
GAO- 03- 476 Acquisition Trends and Risks Page 33
Page 34 GAO- 03- 476 Acquisition Trends and Risks
Common Name: Comanche Appendix I
Appendix I Common Name: ERGM
GAO- 03- 476 Acquisition Trends and Risks Page 35
Page 36 GAO- 03- 476 Acquisition Trends and Risks
Common Name: ERGM Appendix I
Appendix I Common Name: Excalibur
GAO- 03- 476 Acquisition Trends and Risks Page 37
Page 38 GAO- 03- 476 Acquisition Trends and Risks
Common Name: Excalibur Appendix I
Appendix I Common Name: F/ A- 18E/ F
GAO- 03- 476 Acquisition Trends and Risks Page 39
Page 40 GAO- 03- 476 Acquisition Trends and Risks
Common Name: F/ A- 18E/ F Appendix I
Appendix I Common Name: F/ A- 22
GAO- 03- 476 Acquisition Trends and Risks Page 41
Page 42 GAO- 03- 476 Acquisition Trends and Risks
Common Name: F/ A- 22 Appendix I
Appendix I Common Name: JASSM
GAO- 03- 476 Acquisition Trends and Risks Page 43
Page 44 GAO- 03- 476 Acquisition Trends and Risks
Common Name: JASSM Appendix I
Appendix I Common Name: Joint Common Missile
GAO- 03- 476 Acquisition Trends and Risks Page 45
Page 46 GAO- 03- 476 Acquisition Trends and Risks
Common Name: Joint Common Missile Appendix I
Appendix I Common Name: JPATS
GAO- 03- 476 Acquisition Trends and Risks Page 47
Page 48 GAO- 03- 476 Acquisition Trends and Risks
Common Name: JPATS Appendix I
Appendix I Common Name: JSF
GAO- 03- 476 Acquisition Trends and Risks Page 49
Page 50 GAO- 03- 476 Acquisition Trends and Risks
Common Name: JSF Appendix I
Appendix I Common Name: JSOW
GAO- 03- 476 Acquisition Trends and Risks Page 51
Page 52 GAO- 03- 476 Acquisition Trends and Risks
Common Name: JSOW Appendix I
Appendix I Common Name: NPOESS
GAO- 03- 476 Acquisition Trends and Risks Page 53
Page 54 GAO- 03- 476 Acquisition Trends and Risks
Common Name: NPOESS Appendix I
Appendix I Common Name: PAC- 3
GAO- 03- 476 Acquisition Trends and Risks Page 55
Page 56 GAO- 03- 476 Acquisition Trends and Risks
Common Name: PAC- 3 Appendix I
Appendix I Common Name: SBIRS High
GAO- 03- 476 Acquisition Trends and Risks Page 57
Page 58 GAO- 03- 476 Acquisition Trends and Risks
Common Name: SBIRS High Appendix I
Appendix I Common Name: THAAD
GAO- 03- 476 Acquisition Trends and Risks Page 59
Page 60 GAO- 03- 476 Acquisition Trends and Risks
Common Name: THAAD Appendix I
Appendix I Common Name: Tactical Tomahawk
GAO- 03- 476 Acquisition Trends and Risks Page 61
Page 62 GAO- 03- 476 Acquisition Trends and Risks
Common Name: Tactical Tomahawk Appendix I
Appendix I Common Name: V- 22
GAO- 03- 476 Acquisition Trends and Risks Page 63
Page 64 GAO- 03- 476 Acquisition Trends and Risks
Common Name: V- 22 Appendix I
Appendix I Common Name: WGS
GAO- 03- 476 Acquisition Trends and Risks Page 65
Page 66 GAO- 03- 476 Acquisition Trends and Risks
Common Name: WGS Appendix I
Page 67 GAO- 03- 476 Acquisition Trends and Risks
Appendix II
Appendix II Methodology
Page 68 GAO- 03- 476 Acquisition Trends and Risks
Appendix II Methodology
Page 69 GAO- 03- 476 Acquisition Trends and Risks
Appendix II Methodology
Page 70 GAO- 03- 476 Acquisition Trends and Risks
Page 71 GAO- 03- 476 Acquisition Trends and Risks
Appendix III
Appendix III GAO Contact and Acknowledgments
Page 72 GAO- 03- 476 Acquisition Trends and Risks
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300 Address Service Requested
Presorted Standard Postage & Fees Paid
GAO Permit No. GI00
*** End of document. ***