Joint Strike Fighter Acquisition: Mature Critical Technologies	 
Needed to Reduce Risks (19-OCT-01, GAO-02-39).			 
								 
The Joint Strike Fighter Program (JSFP) is the most expensive	 
aircraft program in the Department of Defense (DOD). It is	 
intended to produce affordable, next-generation aircraft to	 
replace aging aircraft in the military services' inventories.	 
Although JSFP has made good progress in some technology areas,	 
the program is at risk of not meeting its affordability objective
because critical technologies are not projected to be matured to 
levels GAO believes would indicate a low risk program at the	 
planned start of engineering and manufacturing development in	 
October 2001.							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-02-39						        
    ACCNO:   A02347						        
  TITLE:     Joint Strike Fighter Acquisition: Mature Critical	      
Technologies Needed to Reduce Risks				 
     DATE:   10/19/2001 
  SUBJECT:   Fighter aircraft					 
	     Procurement planning				 
	     Defense cost control				 
	     Defense procurement				 
	     Joint Direct Attack Munition			 
	     Joint Strike Fighter				 
	     Best Practices					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-02-39
     
Report to the Chairman, Subcommittee on National Security, Veterans'
Affairs, and International Relations, Committee on Government Reform, House
of Representatives

United States General Accounting Office

GAO

October 2001 JOINT STRIKE FIGHTER ACQUISITION

Mature Critical Technologies Needed to Reduce Risks

GAO- 02- 39

Page i GAO- 02- 39 Joint Strike Fighter Acquisition Letter 1

Results in Brief 2 Background 4 Technology Readiness Assessments Provide
Opportunities to

Improve Outcomes 6 Critical Technologies Are Not Projected to Be at Low Risk
8 Conclusions 11 Recommendations for Executive Action 11 Agency Comments and
Our Evaluation 12 Scope and Methodology 13

Appendix I Technology Readiness Levels and Their Descriptions 15

Appendix II Critical Technologies and Their Descriptions 16

Appendix III Comments From the Department of Defense 17

Table

Table 1: Military Services? Planned Use for the Joint Strike Fighter 4

Figures

Figure 1: Boeing and Lockheed Martin Joint Strike Fighter Aircraft Design
Concepts 6 Figure 2: Actual and Projected Joint Strike Fighter Critical

Technology Readiness Levels 9

Abbreviations

DOD Department of Defense TRL technology readiness level Contents

Page 1 GAO- 02- 39 Joint Strike Fighter Acquisition

October 19, 2001 The Honorable Christopher Shays Chairman, Subcommittee on
National Security,

Veterans? Affairs, and International Relations Committee on Government
Reform House of Representatives

Dear Mr. Chairman: The Joint Strike Fighter Program is the most expensive
aircraft program in the Department of Defense (DOD). It is intended to
produce affordable, next- generation aircraft to replace aging aircraft in
the military services? inventories. As currently planned, the program will
cost about $200 billion to develop and procure about 3,000 aircraft and
related support equipment. Two contractor teams, led by the Boeing Company
and by the Lockheed Martin Aeronautics Company, are competing for the
engineering and manufacturing development phase. This phase of the program
is projected to last about 8 years and cost about $20 billion and typically
involves large, fixed investments in human capital, facilities, and
materials.

Last year, we testified and reported that a key objective of the program?s
acquisition strategy is affordability and that a part of that strategy-
entering into engineering and manufacturing development with low technical
risk- would not be achieved because technologies critical to meeting the
program?s cost and requirement objectives were projected to be at low levels
of technical maturity in April 2001, the date then scheduled for awarding
the engineering and manufacturing development contract. 1 We stated that the
program?s approach was not consistent with best practices in which
technologies are more fully developed before proceeding into product
development. Organizations that use best practices recognize that delaying
the resolution of technology problems until product development- analogous
to the engineering and manufacturing development phase- can result in at
least a ten- fold cost

1 Joint Strike Fighter Acquisition: Development Schedule Should Be Changed
to Reduce Risks (GAO/ T- NSIAD- 00- 132, Mar. 16, 2000); Joint Strike
Fighter Acquisition: Development Schedule Should Be Changed to Reduce Risks
(GAO/ NSIAD- 00- 74, May 9, 2000); and Defense Acquisitions: Decisions on
the Joint Strike Fighter Will Be Critical for Acquisition Reform (GAO/ T-
NSIAD- 00- 173, May 10, 2000).

United States General Accounting Office Washington, DC 20548

Page 2 GAO- 02- 39 Joint Strike Fighter Acquisition

increase; delaying the resolution until after the start of production could
increase costs by a hundred- fold.

Because of concerns about the adequacy of the Joint Strike Fighter?s short
take- off and vertical landing flight test program, the maturity of its
critical technologies, and other factors, the Fiscal Year 2001 National
Defense Authorization Act directed that the contract for the aircraft?s
engineering and manufacturing development not be awarded until certain
criteria were met. 2 For example, the act required that the program?s short
take- off and vertical landing demonstration aircraft accumulate at least 20
hours of flight test time before the program could enter the engineering and
manufacturing development phase. The engineering and manufacturing
development contract award is now scheduled for October 2001.

At your request, we assessed whether the Joint Strike Fighter?s critical
technologies will have matured to a low technical risk at the start of the
upcoming engineering and manufacturing development phase. 3 We assessed
technical maturity based on technology readiness levels- measures pioneered
by the National Aeronautics and Space Administration and adopted by the Air
Force Research Laboratory to determine the readiness of technologies to be
incorporated into a weapon or other type of system. 4 The research
laboratory considers a technology to be low risk for the engineering and
manufacturing development stage when a prototype of that technology has been
developed that includes all of its critical components in approximately the
same size, weight, and configuration of the end product and that prototype
has been demonstrated to work in an environment similar to the planned
operational system.

Although the Joint Strike Fighter program has made good progress in some
technology areas, the program is at risk of not meeting its affordability
objective because critical technologies are not projected to

2 Section 212, P. L. 106- 398, Oct. 30, 2000. 3 A technology is considered
to be mature when it has been developed to a point that it can be readily
integrated into a new product and counted on to meet product requirements. 4
The Air Force Research Laboratory is a science and technology organization
that matures advanced technologies to the point that they can be included in
weapon system programs and be expected to perform as required. Results in
Brief

Page 3 GAO- 02- 39 Joint Strike Fighter Acquisition

be matured to levels that we believe would indicate a low risk program at
the planned start of engineering and manufacturing development in October
2001. In fact, the critical technologies are projected to be at the same
level of maturity in October 2001 as they were projected to be in April
2001, the previous scheduled date for the engineering and manufacturing
development contract. Several of these technologies under development are
critical to making the long- term ownership costs of the program affordable.
These technologies remain at higher risk levels for engineering and
manufacturing development because (1) they have not been developed to
approximately the same size, weight, and configuration of the end product
and/ or (2) they have not been demonstrated to work in an environment
similar to the planned operational system. For example, the contractors have
demonstrated certain key component technologies in flight environments, but
the tested hardware was not always the same size and weight required for the
Joint Strike Fighter aircraft. In other instances, components built to the
required size and weight were only demonstrated in ground- tests. 5

Based on our work on best practices in product development, and the
importance of the Joint Strike Fighter to DOD and the industrial base, we
believe that DOD needs to ensure that the program?s critical technologies
are at demonstrated levels of maturity before making engineering and
manufacturing investments in the program. Failure to do so could result in
increases in both the production and long- term ownership costs, schedule
delays, and compromised performance as problems arise in product
development. Moreover, the impact of failing to mature one critical
technology could ripple throughout the program. Therefore, we are
recommending that DOD ensure that critical technologies are mature before
proceeding into engineering and manufacturing to improve the likelihood of
meeting program expectations or to take additional actions if it chooses to
accept the risk of immature technologies.

In commenting on our report, DOD said that it assessed the technology
maturity of the Joint Strike Fighter to be sufficient for the next phase of
the program. DOD also said that the Joint Strike Fighter Program Office has
implemented a rigorous risk management program that will continue to monitor
and address technology risks, as well as other risks, throughout the
program?s life. We disagree with DOD?s assertion that technology is

5 Due to the current Joint Strike Fighter competition, the technologies are
not specified so as to not associate them with either Boeing or Lockheed
Martin, respectively.

Page 4 GAO- 02- 39 Joint Strike Fighter Acquisition

mature enough to move forward. The technology readiness level assessment
conducted as part of our review of the Joint Strike Fighter showed that
critical technologies are not projected to be matured to levels that would
stem risks at the start of engineering and manufacturing development. Our
previous work has shown that when programs proceed in this fashion, they
experience delays, rework, and substantial cost increases that could force
the Department to divert much- needed funds from other important weapon
system programs.

The Joint Strike Fighter Program is structured to use a common production
line to produce three versions of a single aircraft. These aircraft will be
tailored to meet conventional flight requirements for the U. S. Air Force,
short take- off and vertical landing characteristics for the U. S. Marine
Corps, and carrier operation suitability needs for the U. S. Navy. The
program will also provide aircraft to the British Royal Navy and Air Force.
Table 1 shows the services? planned use for the Joint Strike Fighter.

Table 1: Military Services? Planned Use for the Joint Strike Fighter Service
Quantity Planned use

U. S. Air Force 1, 763 Replacement for F- 16 and A- 10; complement the F- 22
U. S. Marine Corps 609 Replacement for the AV- 8B and F/ A- 18 C/ D U. S.
Navy 480 Complement the F/ A- 18 E/ F Great Britain 150 Replacement for the
Sea Harrier and GR. 7 Source: Joint Strike Fighter program office.

A key objective of the Joint Strike Fighter acquisition strategy is
affordability- reducing the development, production, and ownership costs of
the program relative to prior fighter aircraft programs. To achieve its
affordability objective, the Joint Strike Fighter program has incorporated
various acquisition initiatives into the program?s acquisition strategy and
various technological advances into the fighter. Among the acquisition
initiatives planned was to develop critical technologies to a level where
they represent low technical risk before the engineering and manufacturing
contract is awarded. The expectation was that incorporating these
initiatives into the acquisition strategy would avoid cost growth, schedule
slippage, and performance shortfalls that have been experienced in other
weapon acquisition programs. Background

Page 5 GAO- 02- 39 Joint Strike Fighter Acquisition

To date, the Joint Strike Fighter Program has awarded contracts totaling
over $2 billion to Boeing and Lockheed Martin for the current concept
demonstration phase. During this phase, DOD required each contractor to
design and build two aircraft to demonstrate the following:

 commonality/ modularity to validate the contractors? ability to produce
three aircraft versions on the same production line;

 the aircraft?s ability to do a short take- off and vertical landing,
hover, and transition to forward flight; and

 satisfactory low airspeed, carrier approach flying and handling qualities.
Each contractor was required to submit a Preferred Weapon System Concept,
which outlines its final design concept for developing a Joint Strike
Fighter aircraft that is affordable and meets performance requirements. The
Preferred Weapon System Concept includes results from the flight and ground
demonstrations and is being used by DOD to select the winning aircraft
design and to award the engineering and manufacturing development contract.
During engineering and manufacturing development, the Joint Strike Fighter
will be fully developed, engineered, designed, fabricated, tested, and
evaluated to demonstrate that the production aircraft will meet stated
requirements. Critical junctures in engineering and manufacturing
development are the preliminary and critical design reviews and commitments;
testing of aircraft; and commitments to production hardware, including the
purchase of long lead production items. It is at the critical design review
that decisions are made toward finalizing the aircraft design and begin
building test aircraft. About two- thirds of engineering and manufacturing
development funding will be spent after this review. Figure 1 shows planned
Joint Strike Fighter aircraft designs by contractor.

Page 6 GAO- 02- 39 Joint Strike Fighter Acquisition

Figure 1: Boeing and Lockheed Martin Joint Strike Fighter Aircraft Design
Concepts

Source: Boeing and Lockheed Martin.

In our previous work on best business practices, commercial firms have told
us that a key part of product development is getting the technology into the
right size, weight, and configuration needed for the intended product- in
this case, the final Joint Strike Fighter design. Once this has been
demonstrated, the technology is at an acceptable level for product
development. Technology readiness levels (TRL) can be used to assess the
maturity of technology and can reveal whether a gap exists between a
technology?s maturity and the maturity demanded for successful inclusion in
the intended product. Defining this gap for the Joint Strike Fighter
technologies is important for determining whether they can be expected to
demonstrate required capabilities before being integrated into the aircraft
design. Readiness levels are measured along a scale of one to nine, starting
with paper studies of the basic concept, proceeding with laboratory
demonstrations, and ending with a technology that has proven itself on the
intended product. (See app. I for a detailed description of TRLs.) The Air
Force Research Laboratory considers TRL 7 an acceptable risk for starting
the engineering and manufacturing development phase. The readiness
Technology Readiness

Assessments Provide Opportunities to Improve Outcomes

Page 7 GAO- 02- 39 Joint Strike Fighter Acquisition

level definitions state that for a technology to be rated at TRL 7, it must
be demonstrated using prototype hardware (such as a complete radar
subsystem) that is the same size, weight, and configuration as that called
for in the final aircraft design and that prototype has to be demonstrated
to work in an environment similar to the planned operational system.

We have previously reviewed the impact of incorporating technologies into
new product and weapon system designs. 6 The results showed that programs
met product objectives when the technologies were matured to higher levels
and conversely showed that cost and schedule problems arose when programs
started when technologies were at low readiness levels. For example, the
Joint Direct Attack Munition (JDAM) used modified variants of proven
components for guidance and global positioning. It also used mature,
existing components from other proven manufacturing processes for its own
system for controlling tail fin movements. The munition was touted for its
performance in Kosovo and was purchased for less than half of its expected
unit cost. However, the Comanche helicopter program began with critical
technologies such as the engine, rotor, and integrated avionics at TRL
levels of 5 or below. That program has seen 101 percent cost growth and 120-
percent schedule slippage as a result of these low maturity levels and other
factors.

In commenting on our report concerning better management of technology
development, DOD agreed that TRLs are important and necessary in assisting
decision makers in deciding on when and where to insert new technologies
into weapons system programs and that it is desirable to mature technologies
to TRL 7 prior to entering the engineering and manufacturing development
phase of a weapon system program. 7 Since that time, DOD has adopted the
technology readiness levels as a means of assessing the technological
maturity of new major programs. In a July 5, 2001, memorandum, the Deputy
Under Secretary of Defense (Science and Technology) stated that new DOD
regulations require that the military services? science and technology
executives conduct a technology readiness level assessment for critical
technologies identified in major weapon systems programs prior to the start
of engineering and

6 Best Practices: Successful Application to Weapon Acquisitions Requires
Changes in DOD?s Environment (GAO/ NSIAD- 98- 56, Feb. 24, 1998) and Best
Practices: Better Management of Technology Development Can Improve Weapon
System Outcomes

(GAO/ NSIAD- 99- 162, July 30, 1999). 7 GAO/ NSIAD- 99- 162, July 30, 1999.

Page 8 GAO- 02- 39 Joint Strike Fighter Acquisition

manufacturing development and production. The memorandum notes that
technology readiness levels are the preferred approach for all new major
programs unless the Deputy Under Secretary approves an equivalent assessment
method.

The Joint Strike Fighter Program, like many other DOD programs, has used
risk management plans and engineering judgment as a way of assessing
technological maturity. The Principal Deputy Under Secretary of Defense
(Acquisition and Technology) has determined that these means will continue
to be used by DOD and the Joint Strike Fighter contractors to assess the
program?s technological risk. Risk management plans and judgment are
necessary to managing any major development effort like the Joint Strike
Fighter. However, without an underpinning such as technology readiness
levels that allow transparency into program decisions, these methods allow
significant technical unknowns to be judged acceptable risks because a plan
exists for resolving the unknowns in the future. Experience on previous
programs has shown that such methods have rarely assessed technical unknowns
as a high or unacceptable risk; consequently, they failed to guide programs
to meet promised outcomes. Technology readiness levels are based on actual
demonstrations of how well technologies actually perform. Their strength
lies in the fact that they characterize knowledge that exists rather than
plans to gain knowledge in the future; they are, thus, less susceptible to
optimism.

In May 2000 we reported that all of the eight technologies identified by the
Joint Strike Fighter program office as critical to the program were expected
to be at maturity levels below that considered acceptable for low risk when
entering engineering and manufacturing development (TRL 7). The eight
critical technologies are: prognostics and health management, integrated
flight propulsion control, subsystems, integrated support system, integrated
core processor, radar, manufacturing, and mission systems integration. (See
app. II for a description of these technologies.) During our review last
year, we worked with the two competing contractors and the program office to
arrive at the applicable TRLs for the critical technologies. Specifically,
on separate visits to the contractors, with program office personnel
present, we asked the contractors? relevant technology managers to score the
technologies they considered critical to enable their Joint Strike Fighter
design to meet DOD requirements for the aircraft. At that time, we also
asked them to describe their plans to mature the technologies to the planned
start of the engineering and manufacturing development phase, then scheduled
for April 2001. Upon reviewing these Critical Technologies

Are Not Projected to Be at Low Risk

Page 9 GAO- 02- 39 Joint Strike Fighter Acquisition

scores with the program office and in order to gain an overall Joint Strike
Fighter Program perspective on technical maturity, the Joint Strike Fighter
office agreed to provide us with TRL scores for the eight technologies they
considered critical for meeting program cost and performance requirements.
Figure 2 reflects the program office scores at the time of our last review.
Due to the current Joint Strike Fighter competition, the specific
technologies mentioned previously are not linked to scores so as not to
divulge competition sensitive information.

Figure 2: Actual and Projected Joint Strike Fighter Critical Technology
Readiness Levels

Source: Joint Strike Fighter program office.

As the figure shows, all eight technologies were projected to be below the
level of maturity (TRL 7) considered acceptable for low risk when entering
the engineering and manufacturing development phase and six of the

Page 10 GAO- 02- 39 Joint Strike Fighter Acquisition

technologies were projected to be below the level of maturity (TRL 6) that
is considered low risk for entering the demonstration phase, which the Joint
Strike Fighter Program began in 1996.

During our current review, we again visited the two competing contractors to
discuss the status of the eight technologies. We learned that they have
essentially accomplished, or plan to accomplish by October 2001, the
technology development and demonstrations that they planned to accomplish as
of April 2001. Thus figure 2 represents the current assessment of technical
maturity. While two of these areas are very close to appropriate maturity
levels, the Joint Strike Fighter?s critical technologies are not projected
to be matured to levels that we believe would indicate a low risk program at
the planned start of the engineering and manufacturing development phase.
Key component technologies remain at higher risk levels for engineering and
manufacturing development because (1) they have not been developed to
approximately the same size, weight, and configuration called for in the
final aircraft design and/ or (2) they have not been demonstrated to work in
an environment similar to the planned operational system.

The Joint Strike Fighter Program has made good progress in some technology
areas. For example, contractor and program officials told us that because of
concerns about propulsion technology, both contractors focused considerable
attention on that area. Both contractors flew aircraft that demonstrated the
capability for short take- off and vertical landing and accumulated at least
20 hours of flight time on those aircraft, which should satisfy the
requirement in the Fiscal Year 2001 National Defense Authorization Act. In
some other areas, the technology maturation has not been uniform across all
critical components of a technology. For example, the radar has a number of
critical components that must work together as a system. Both contractors
have made considerable progress on one or more of those components, but the
other critical components have not been matured to an acceptable level of
risk. In order for this technology to achieve a TRL level of 7, all
components had to be (1) demonstrated in the size and weight required to
meet aircraft capabilities, (2) integrated together as they would be in the
final aircraft design, and (3) flown in an environment similar to what the
Joint Strike Fighter will be subjected. To demonstrate some critical
technologies, both contractors flew key electronic and other components in
flying avionics test beds (commercial aircraft reconfigured as flying
laboratories). While these tests occurred in a relevant environment (e. g.,
in flight), the tested hardware was not always the same size and weight
required for the Joint Strike Fighter aircraft. Conversely, some components
were built to the required size and weight,

Page 11 GAO- 02- 39 Joint Strike Fighter Acquisition

but were demonstrated only in ground- testing environments. By not having
matured all critical technology areas to appropriate maturity levels, the
program remains at risk for achieving cost and performance goals upon
entering product development.

Moving into engineering and manufacturing development creates an expectation
that the Joint Strike Fighter can be delivered for a stated time and dollar
investment and with a given set of capabilities. The decisions the
Department of Defense makes now and over the next 2 years will largely
determine whether those expectations can be met.

A key component of the Joint Strike Fighter Program?s acquisition strategy
is to enter the engineering and manufacturing development phase with low
technical risk. The program will not have achieved that point by October
2001 because technologies, which the Joint Strike Fighter Program Office
identified as critical to meeting the program?s cost and requirements
objectives, will not have been matured to an acceptable risk level. By
entering the engineering and manufacturing development phase with immature
critical technologies, the program will need to continue to develop those
technologies at the same time it will be concentrating on production issues
and the integration of subsystems into a Joint Strike Fighter. This approach
would not be consistent with best practices. In fact, it would more closely
follow DOD?s traditional practices in weapon system programs that have often
resulted in cost increases, schedule delays, and compromised performance.

To eliminate one of the major sources of cost and schedule risk, we
recommend that the Secretary of Defense delay the start of engineering and
manufacturing development until critical technologies are matured to
acceptable levels.

Alternatively, if the Secretary of Defense decides to accept these risks and
move the program into engineering and manufacturing development as
scheduled, we recommend that the Secretary dedicate the resources to
ensuring that maturity of the critical technologies is demonstrated by the
critical design review or defer the inclusion of immature technologies from
the approved design. Conclusions

Recommendations for Executive Action

Page 12 GAO- 02- 39 Joint Strike Fighter Acquisition

In written comments on a draft of this report, the Director of Strategic and
Tactical Systems, within the Office of the Under Secretary of Defense for
Acquisition, Technology, and Logistics, partially concurred with our
recommendation. DOD contended that an independent technology readiness
assessment it carried out on the program showed that technology has
extensively matured and the program is now ready to enter into systems
development and demonstration. DOD also stated that the Joint Strike Fighter
Program Office has implemented a risk management program that will continue
to monitor and address technology risks, as well as other risks, throughout
the program?s life. The full text of DOD?s comments is included in appendix
III.

We disagree with the Department?s assessment of technological maturity. The
TRL assessment conducted as part of our review showed that technologies
critical to the Joint Strike Fighter Program are not projected to be matured
to levels that we believe would indicate a low risk program at the planned
start of the engineering and manufacturing development phase. Many of the
technologies have not been demonstrated in their appropriate size and
weight, nor have they been demonstated to function in an environment in
which they will be used. For example, many of the technologies are still in
the laboratory and will require considerable maturation before they can be
incorporated into the final design. By entering the engineering and
manufacturing development phase with immature critical technologies, the
program will need to continue to develop those technologies at the same time
it will be concentrating on engineering, designing, and fabricating the
product. As it has with many other DOD programs, this approach increases the
likelihood of schedule delays and program cost increases. This is primarily
why DOD?s new acquisition regulations emphasize separating technology
development from product development. In fact, experience has shown that
resolving technology problems in product development can result in at least
a tenfold cost increase.

Moreover, DOD incorrectly states that the tools it used to assess its
technology and the TRLs used for our review are equivalent methodologies for
assessing technological maturity. The Willoughby Templates used by DOD are a
risk management tool. They can be an excellent way to manage program risks,
but in practice they have not been used to identify risk. Identifying risk
is the first step to managing it. By contrast, by focusing specifically on
assessing technology maturity against objective standards, TRLs have proven
successful at identifying risks. A more appropriate approach for DOD to take
is to use technology readiness levels in conjunction with a management tool
such as the Willoughby Agency Comments

and Our Evaluation

Page 13 GAO- 02- 39 Joint Strike Fighter Acquisition

templates since this can result in more informed decision making and fewer
unanticipated problems in an acquisition program. In fact, the Joint Strike
Fighter program provides DOD with an excellent opportunity to apply these
concepts in tandem.

To assess whether the Joint Strike Fighter?s critical technologies are
projected to mature to low technical risk at the start of the engineering
and manufacturing development phase, we used the technology readiness level
tool and information provided by Joint Strike Fighter program officials and
contractor officials at the Boeing Company, Seattle, Washington; Lockheed
Martin Aeronautics Company, Fort Worth, Texas; and Pratt & Whitney, East
Hartford, Connecticut. During our previous review, we had obtained detailed
briefings from Boeing and Lockheed Martin officials on their plans to mature
critical technologies prior to the date for awarding the engineering and
manufacturing development contract, then scheduled for April 2001. We had
also obtained program office and contractor assessments of the expected
technology readiness levels for the critical technologies at April 2001.
During our current review, we obtained detailed briefings from program
office personnel on the status of critical technologies. We also obtained
detailed briefings from Boeing, Lockheed Martin, and Pratt & Whitney
officials on the contractors? progress in maturing critical technologies and
any further maturation plans through October 2001. We compared the latest
information from the program office and the contractors to the information
obtained during our prior review to determine if the critical technologies
had been matured to higher technology readiness levels and the levels
achieved.

We conducted our review from April through September 2001 in accordance with
generally accepted government auditing standards.

We are sending copies of this report to the congressional defense
committees; the Honorable Donald H. Rumsfeld, Secretary of Defense; the
Honorable James G. Roche, Secretary of the Air Force; the Honorable Gordon
R. England, Secretary of the Navy; General James L. Jones, Commandant of the
Marine Corps; and the Honorable Mitchell E. Daniels, Jr., Director, Office
of Management and Budget. We will also make copies available to other
interested parties on request. Scope and

Methodology

Page 14 GAO- 02- 39 Joint Strike Fighter Acquisition

Please contact me at (202) 512- 4841 if you or your staff have any questions
concerning this report. Key contributors to this report were Robert
Pelletier and Brian Mullins.

Sincerely yours, Katherine V. Schinasi Director, Acquisition and

Sourcing Management

Appendix I: Technology Readiness Levels and Their Descriptions

Page 15 GAO- 02- 39 Joint Strike Fighter Acquisition

Technology readiness level Description

1. Basic principles observed and reported. Lowest level of technology
readiness. Scientific research begins to be translated into

applied research and development. Examples might include paper studies of a
technology?s basic properties. 2. Technology concept and/ or application
formulated. Invention begins. Once basic principles are observed, practical
applications can be

invented. The application is speculative and there is no proof or detailed
analysis to support the assumption. Examples are still limited to paper
studies. 3. Analytical and experimental critical function and/ or
characteristic proof of concept.

Active research and development is initiated. This includes analytical
studies and laboratory studies to physically validate analytical predictions
of separate elements of the technology. Examples include components that are
not yet integrated or representative. 4. Component and/ or breadboard
validation in laboratory environment. Basic technological components are
integrated to establish that the pieces will work

together. This is relatively ?low fidelity? compared to the eventual system.
Examples include integration of ?ad hoc? hardware in a laboratory. 5.
Component and/ or breadboard validation in relevant environment. Fidelity of
breadboard technology increases significantly. The basic technological

components are integrated with reasonably realistic supporting elements so
that the technology can be tested in a simulated environment. Examples
include ?high fidelity? laboratory integration of components. 6. System/
subsystem model or prototype demonstration in a relevant environment.
Representative model or prototype system, which is well beyond the
breadboard tested

for TRL 5, is tested in a relevant environment. Represents a major step up
in a technology?s demonstrated readiness. Examples include testing a
prototype in a high fidelity laboratory environment or in simulated
operational environment. 7. System prototype demonstration in an operational
environment. Prototype near or at planned operational system. Represents a
major step up from TRL 6,

requiring the demonstration of an actual system prototype in an operational
environment, such as in an aircraft, vehicle, or space. Examples include
testing the prototype in a test bed aircraft. 8. Actual system completed and
?flight qualified? through test and demonstration.

Technology has been proven to work in its final form and under expected
conditions. In almost all cases, this TRL represents the end of true system
development. Examples include developmental test and evaluation of the
system in its intended weapon system to determine if it meets design
specifications. 9. Actual system ?flight proven? through successful mission
operations. Actual application of the technology in its final form and under
mission conditions, such as

those encountered in operational test and evaluation. In almost all cases,
this is the end of the last ?bug fixing? aspects of true system development.
Examples include using the system under operational mission conditions.

Source: Best Practices: Better Management of Technology Development Can
Improve Weapon System Outcomes (GAO/ NSIAD- 99- 162, July 30, 1999).

Appendix I: Technology Readiness Levels and Their Descriptions

Appendix II: Critical Technologies and Their Descriptions

Page 16 GAO- 02- 39 Joint Strike Fighter Acquisition

Critical technologies Description

Prognostics and health management Involves the ability to detect and isolate
the cause of aircraft problems and then predict when maintenance activity
will have to occur on systems with pending failures. Life- cycle cost
savings are dependent on prognostics and health management through improved
sortie generation rate, reduced logistics and manpower requirements, and
more efficient inventory control. Integrated flight propulsion control
Includes integration of propulsion, vehicle management system, and other
subsystems

as they affect aircraft stability, control, and flying qualities (especially
short take- off and vertical landing). Aircraft improvements are to reduce
pilot workload and increase flight safety. Subsystems Includes areas of
electrical power, electrical wiring, environmental control systems, fire

protection, fuel systems, hydraulics, landing gear systems, mechanisms, and
secondary power. Important for reducing aircraft weight, decreasing
maintenance cost, and improving reliability. Integrated support systems
Involves designing an integrated support concept that includes an aircraft
with

supportable stealth characteristics and improved logistics and maintenance
functions. Life- cycle cost savings are expected from improved low
observable maintenance techniques and streamlined logistics and inventory
systems. Integrated core processor Includes the ability to use commercial-
based processors in an open architecture design

to provide processing capability for radar, information management,
communications, etc. Use of commercial processors reduces development and
production costs and an open architecture design reduces future development
and upgrade costs. Radar Includes advanced integration of communication,
navigation, and identification functions

and electronic warfare functions through improved apertures, antennas,
modules, radomes, etc. Important for reducing avionics cost and weight, and
decreasing maintenance cost through improved reliability. Manufacturing
Involves lean, automated, highly efficient aircraft fabrication and assembly
techniques.

Manufacturing costs should be less through improved flow time, lower
manpower requirements, and reduced tooling cost. Mission systems integration
Involves decreasing pilot workload by providing information for targeting,
situational

awareness, and survivability through fusion of radar, electronic warfare,
and communication, navigation, and identification data. Improvements are
achieved through highly integrated concept of shared and managed resources,
which reduces production costs, aircraft weight, and volume requirements, in
addition to improved reliability.

Source: Joint Strike Fighter program office.

Appendix II: Critical Technologies and Their Descriptions

Appendix III: Comments From the Department of Defense

Page 17 GAO- 02- 39 Joint Strike Fighter Acquisition

Appendix III: Comments From the Department of Defense

Appendix III: Comments From the Department of Defense

Page 18 GAO- 02- 39 Joint Strike Fighter Acquisition

Appendix III: Comments From the Department of Defense

Page 19 GAO- 02- 39 Joint Strike Fighter Acquisition (120050)

The General Accounting Office, the investigative arm of Congress, exists to
support Congress in meeting its constitutional responsibilities and to help
improve the performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates federal
programs and policies; and provides analyses, recommendations, and other
assistance to help Congress make informed oversight, policy, and funding
decisions. GAO?s commitment to good government is reflected in its core
values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents is through the
Internet. GAO?s Web site (www. gao. gov) contains abstracts and full- text
files of current reports and testimony and an expanding archive of older
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ?Today?s Reports,? on its Web
site daily. The list contains links to the full- text document files. To
have GAO E- mail this list to you every afternoon, go to our home page and
complete the easy- to- use electronic order form found under ?To Order GAO
Products.?

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U. S. General Accounting Office P. O. Box 37050 Washington, D. C. 20013

To order by phone: Voice: (202) 512- 6000 TDD: (301) 413- 0006 Fax: (202)
258- 4066

GAO Building Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)
Washington, D. C. 20013

Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm, E- mail:
fraudnet@ gao. gov, or 1- 800- 424- 5454 (automated answering system).

Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512- 4800 U. S.
General Accounting Office, 441 G. Street NW, Room 7149, Washington, D. C.
20548 GAO?s Mission

Obtaining Copies of GAO Reports and Testimony

Order by Mail or Phone Visit GAO?s Document Distribution Center

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs
*** End of document. ***