Joint Strike Fighter: Recent Decisions by DOD Add to Program	 
Risks (11-MAR-08, GAO-08-388).					 
                                                                 
The Joint Strike Fighter (JSF) program seeks to produce and field
three aircraft variants for the Air Force, Navy, Marine Corps,	 
and eight international partners. The estimated total investment 
for JSF now approaches $1 trillion to acquire and maintain 2,458 
aircraft. Under congressional mandate, GAO has annually reviewed 
the JSF program since 2005. GAO's prior reviews have identified a
number of issues and recommended actions for reducing risks and  
improving the program's outcomes. This report, the fourth under  
the mandate, focuses on the program's progress in meeting cost,  
schedule, and performance goals; plans and risks in development  
and test activities; the program's cost-estimating methods; and  
future challenges facing the program. To conduct its work, GAO	 
identified changes in cost and schedule from prior years and	 
their causes, evaluated development progress and plans, assessed 
cost-estimating methodologies against best practices, and	 
analyzed future budget requirements.				 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-388 					        
    ACCNO:   A81276						        
  TITLE:     Joint Strike Fighter: Recent Decisions by DOD Add to     
Program Risks							 
     DATE:   03/11/2008 
  SUBJECT:   Aircraft						 
	     Cost analysis					 
	     Cost overruns					 
	     Defense cost control				 
	     Defense procurement				 
	     Fighter aircraft					 
	     Financial management				 
	     Military aircraft					 
	     Military research and development			 
	     Operational testing				 
	     Performance measures				 
	     Program evaluation 				 
	     Program management 				 
	     Research and development costs			 
	     Risk assessment					 
	     Risk management					 
	     Schedule slippages 				 
	     Program goals or objectives			 
	     Cost estimates					 
	     Joint Strike Fighter program			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-388

   

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office:
GAO: 

March 2008: 

Joint Strike Fighter: 

Recent Decisions by DOD Add to Program Risks: 

GAO-08-388: 

GAO Highlights: 

Highlights of GAO-08-388, a report to Congressional Committees. 

Why GAO Did This Study: 

The Joint Strike Fighter (JSF) program seeks to produce and field three 
aircraft variants for the Air Force, Navy, Marine Corps, and eight 
international partners. The estimated total investment for JSF now 
approaches $1 trillion to acquire and maintain 2,458 aircraft. 

Under congressional mandate, GAO has annually reviewed the JSF program 
since 2005. GAOï¿½s prior reviews have identified a number of issues and 
recommended actions for reducing risks and improving the programï¿½s 
outcomes. 

This report, the fourth under the mandate, focuses on the programï¿½s 
progress in meeting cost, schedule, and performance goals; plans and 
risks in development and test activities; the programï¿½s cost-estimating 
methods; and future challenges facing the program. 

To conduct its work, GAO identified changes in cost and schedule from 
prior years and their causes, evaluated development progress and plans, 
assessed cost-estimating methodologies against best practices, and 
analyzed future budget requirements. 

What GAO Found: 

Since last yearï¿½s report, the JSF program office estimates that total 
acquisition costs increased by more than $23 billion, primarily because 
of higher estimated procurement costs. The JSF development cost 
estimate stayed about the same. Development costs were held constant by 
reducing requirements, eliminating the alternate engine program, and 
spending management reserve faster than budgeted. Facing a probable 
contract cost overrun, DOD implemented a Mid-Course Risk Reduction Plan 
to replenish management reserves from about $400 million to about $1 
billion by reducing test resources. Progress has been reported in 
several important areas, including partner agreements, first flights of 
a JSF prototype and test bed, and a more realistic procurement 
schedule. 

The midcourse plan carries the risk of design and performance problems 
not being discovered until late in the operational testing and 
production phases, when it is significantly more costly to address such 
problems. The plan also fails to address the production and schedule 
concerns that depleted management reserves. Cost and schedule pressures 
are mounting. Two-thirds of budgeted funding for JSF development has 
been spent, but only about one-half of the work has been completed. The 
contractor is on its third, soon to be fourth, manufacturing schedule, 
but test aircraft in manufacturing are still behind, the continuing 
impacts of late designs, delayed delivery of parts, and manufacturing 
inefficiencies. 

We believe that JSF costs will likely be much higher than reported. The 
estimates do not include all costs, including about $6.8 billion for 
the alternate engine program. In addition, some assumptions are overly 
optimistic and not well documented. Three independent defense offices 
separately concluded that program cost estimates are understated by as 
much as $38 billion and that the development schedule is likely to slip 
from 12 to 27 months. Discrepancies in cost estimates add to program 
risks and hinder congressional oversight. Even so, DOD does not plan 
for another fully documented, independent total program life-cycle cost 
estimate until 2013. 

As JSF finalizes the three designs, matures manufacturing processes, 
conducts flight tests, and ramps up production, it faces significant 
challenges. JSFï¿½s goalï¿½to develop and field an affordable, highly 
common family of strike aircraftï¿½is threatened by rising unit 
procurement prices and lower commonality than expected. The program 
also makes unprecedented funding demandsï¿½an average of $11 billion 
annually for two decadesï¿½and must compete with other defense and 
nondefense priorities for the shrinking federal discretionary dollar. 
Further, expected cost per flight hour now exceeds that of the F-16 
legacy fighter, one of the aircraft it is intended to replace. With 
almost 90 percent (in terms of dollars) of the acquisition program 
still ahead, it is important to address these challenges, effectively 
manage future risks, and move forward with a successful program that 
meets our and our alliesï¿½ needs. 

What GAO Recommends: 

GAO recommends that DOD revisit and, if appropriate, revise the Mid-
Course Risk Reduction Plan to address concerns about testing, use of 
management reserves, and manufacturing. GAO also recommends action to 
improve the reliability and fidelity of the JSF cost estimate. DOD 
substantially agreed. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.GAO-08-388]. For more information, contact 
Michael J. Sullivan (202) 512-4841 or [email protected]. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Progress Measured against Cost, Schedule, and Performance Goals Was 
Mixed over the Last Year: 

Development Program Faces Increased Risks of Further Cost Increases and 
More Schedule Delays: 

JSF Program Cost Estimate Is Not Reliable: 

Future Challenges as Program Moves Forward: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Response: 

Appendix I: Scope and Methodology: 

Appendix II: GAO Assessment of JSF Program Cost Estimate: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Changes in Reported JSF Program Costs, Quantities, and 
Deliveries: 

Table 2: Manufacturing Performance Data, End of September 2007: 

Table 3: Costs Excluded from the Program Office Acquisition Cost 
Estimate: 

Table 4: Outside Organizations' Assessments of JSF Cost and Schedule: 

Table 5: Costs Excluded from the Program Office Acquisition Cost 
Estimate: 

Table 6: Major Assumptions in JSF Cost Estimates: 

Table 7: Outside Organizations' Assessments of JSF Cost and Schedule: 

Table 8: JSF Development Program Cost Estimate Comparison: 

Table 9: NAVAIR and DCMA Estimates at Completion for JSF Aircraft 
Development Contract: 

Table 10: JSF Complexity Compared to That of Similar Aircraft Programs: 

Figures: 

Figure 1: JSF Annual Procurement Plans for the United States and 
International Partners: 

Figure 2: JSF's Use of Management Reserves: 

Figure 3: JSF Development Flight Tests Planned: 

Figure 4: JSF Acquisition Program's Annual Funding Requirements: 

Figure 5: Changes in Airframe Commonality: 

Figure 6: Cost and Schedule Variances on the Aircraft Development 
Contract: 

Abbreviations: 

CAIG: Cost Analysis Improvement Group: 

CTOL: conventional takeoff and landing: 

CV: carrier-suitable variant: 

DCMA: Defense Contract Management Agency: 

DOD: Department of Defense: 

EVM: earned value management: 

JSF: Joint Strike Fighter: 

IOT&E: Initial Operational Test and Evaluation: 

NAVAIR: Naval Air Systems Command: 

OSD: Office of the Secretary of Defense: 

SAR: Selected Acquisition Report: 

STOVL: short takeoff and vertical landing: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

March 11, 2008: 

Congressional Committees: 

The Joint Strike Fighter (JSF) is Department of Defense's (DOD) most 
complex and ambitious aircraft acquisition, seeking to simultaneously 
produce and field three aircraft variants for the Air Force, Navy, 
Marine Corps, and eight international partners. For the United States, 
the JSF will need a joint, long-term commitment to very large annual 
funding requirements and a total investment now approaching $1 trillion 
dollars--$300 billion to acquire 2,458 aircraft and $650 billion in 
life-cycle operation and support costs, according to official program 
estimates. The JSF is critical to our nation's plans for recapitalizing 
tactical aircraft and just as important to our allies. 

The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 
2005 requires GAO to review the JSF program annually for five 
years.[Footnote 1] Previous reports identified opportunities for the 
program to reduce risks and improve the chance for more successful 
outcomes. We have expressed concern about the substantial overlap of 
development, test, and production activities and recommended a more 
evolutionary and knowledge-based acquisition strategy with limited 
investment in production aircraft until each variant demonstrates 
required capabilities in flight testing.[Footnote 2] This is the fourth 
report under the mandate in which we (1) determine the JSF program's 
progress in meeting cost, schedule, and performance goals; (2) assess 
plans and risks in development and test activities; (3) evaluate 
program office cost-estimating methodology; and (4) identify future 
challenges facing the program. 

The act also requires us to certify whether we had access to sufficient 
information to make informed judgments on the matters contained in our 
report. While we were provided sufficient information to make the 
assessments contained in this report, a continuing concern has been the 
currency of cost and schedule data. The data we are reporting here are 
from the JSF Selected Acquisition Report (SAR) dated December 2006 that 
was released to Congress and us in April 2007. That SAR reflects the 
program position at the time of the submission of the fiscal year 2008 
President's Budget. We were not able to review the program office's 
updated estimates associated with the fiscal year 2009 budget, which 
will be reported in the new December 2007 SAR to be released in April 
2008. The program office declined to provide updated costs, stating 
that those figures are sensitive because the new budget request had not 
been finalized at the time of our review. Every year, this timing 
disconnect results in us reporting soon-to-be-outdated cost and 
schedule data. For example, shortly after our last report was issued on 
March 15, 2007, DOD released new cost estimates that disclosed an 
increase of more than $23 billion in JSF program costs. On the basis of 
the evidence we do have and our analysis, we fully expect future cost 
estimates to be substantially higher than the program estimates in this 
report. 

To conduct this work, we tracked and compared current cost and schedule 
estimates with those of prior years, identified major changes, and 
determined causes. We obtained earned value data, contractor workload 
statistics, performance indicators, and manufacturing results. We 
assessed the program office's cost estimating methodologies against 
best practices prescribed in GAO's Cost Assessment Guide. We discussed 
results to date, plans, and future challenges with DOD and contractor 
officials. We conducted this performance audit from June 2007 to March 
2008 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Results in Brief: 

The JSF total acquisition cost estimate increased by more than $23 
billion since our March 2007 report due to changes in procurement 
costs. Principal driving factors were (1) increased unit costs from 
extending the procurement period seven years at lower annual rates and 
(2) increased future price estimates based on contractor proposals for 
the first production lot. The official cost estimate for development 
remained about the same in total as it has since the program was 
restructured in 2004. However, this was largely achieved by reducing 
requirements, not fully funding the alternate engine program despite 
congressional interest in the program, and spending management reserves 
much faster than budgeted. Facing a probable contract cost overrun, DOD 
officials decided not to request additional funding and time for 
development, opting instead to reduce test resources in order to 
replenish management reserves from $400 million to $1 billion. During 
the last year, DOD and the contractor also reported progress in several 
important areas, including international partner agreements, first 
flights of a JSF prototype and test bed, and a more realistic 
procurement schedule. 

The recent decision to replenish management reserves by reducing test 
resources, known as the Mid-Course Risk Reduction Plan, significantly 
increases the risks of not completing development testing on time and 
not finding and fixing design and performance problems until late into 
operational testing and production, when it is more expensive and 
disruptive to do so. The plan also does not directly address and 
correct the continuing production and schedule concerns that depleted 
management reserves. We expect program development and procurement 
costs to increase substantially and schedule pressures to worsen based 
on performance to date and the conditions that gave rise to the risk 
reduction plan. Two-thirds of budgeted funding for the JSF has been 
spent on the prime development contract, but only about one-half of the 
work has been completed. The contractor has extended manufacturing 
schedules several times, but test aircraft delivery dates continue to 
slip. The flight test program has barely begun, but faces substantial 
risks with reduced assets as design and manufacturing problems continue 
to cause delays that further compress the time available to complete 
development. The Director, Operational Test and Evaluation, and several 
other prominent defense offices objected to the midcourse plan as too 
risky because it does not provide adequate resources for development 
testing or resolve systemic problems that depleted management reserves. 
We agree. 

We do not think the program cost estimate is reliable when judged 
against cost estimate standards used throughout the federal government 
and industry. Specifically, the program cost estimate (1) is not 
comprehensive because it does not include all applicable costs, 
including $6.8 billion for the alternate engine program; (2) is not 
accurate because some of its assumptions are overly optimistic and not 
supportable--such as applying a weight growth factor only half as large 
as historical experience on similar aircraft--and because the data 
system relied upon by the prime contractor and the program office to 
report and manage JSF costs and schedule is deficient; (3) is not well 
documented in that it does not sufficiently identify to cost analysts 
the primary methods, calculations, results, rationales and assumptions, 
and data sources used to generate cost estimates; and (4) is not 
credible according to three independent defense offices who all 
conclude that program cost estimates are understated by as much as $38 
billion and that the development schedule is likely to slip from 12 to 
27 months. Despite this and all the significant events and changes that 
have occurred in the 6 years since the start of system development, DOD 
does not intend to accomplish another fully documented, independent 
total program life-cycle cost estimate for another 6 years. 

The JSF is entering its most challenging phase as it finalizes three 
designs, matures manufacturing processes, conducts flight tests, and 
ramps up production. The first and foremost challenge is affordability. 
From its outset, the JSF goal was to develop and field an affordable, 
highly common family of strike aircraft. That goal is threatened by 
rising unit procurement prices and somewhat lower commonality than 
expected, raising concerns that the United States and its allies may 
not be able to buy as many aircraft as currently planned. The program 
also makes unprecedented demands for funding from the defense budget--
an annual average of about $11 billion for the next two decades--and 
must compete with other defense and non-defense priorities for the 
shrinking federal discretionary dollar. Further, informed by more 
knowledge as the program progresses, DOD doubled its projection of JSF 
life-cycle operating and support costs compared to last year's estimate 
and its expected cost per flight hour now exceeds that of the F-16 
legacy fighter it is intended to replace. With almost 90 percent (in 
terms of dollars) of the acquisition program still ahead, it is 
important to address these challenges, effectively manage future risks, 
and move forward with a successful program that meets our and our 
allies' needs. 

Because of the elevated risks and valid objections raised by the test 
community and other DOD offices, we recommend that DOD revisit and, if 
appropriate, revise the Mid-Course Risk Reduction plan recently 
approved. DOD should specifically address concerns about constrained 
testing capacity, the integration of flight and ground tests, depletion 
of management reserves, slippage in the manufacturing schedule, and 
progress made in correcting deficiencies in the contractor's earned 
value management system, and to examine in depth the alternatives to 
the current plan that could reduce risks. To enhance congressional 
oversight and provide DOD management with a higher-fidelity and more 
comprehensive cost estimate, we also make several recommendations to 
improve cost estimates, in particular that DOD accomplish this year a 
new total program life-cycle cost estimate, validated by the Cost 
Analysis Improvement Group, that includes risk analysis and meets DOD 
policy requirements for major system cost estimates at milestones. 

DOD substantially agreed with our recommendations. DOD believes that 
the midcourse plan is a cost-effective approach with a manageable level 
of risk that will be monitored and revised if necessary. Regarding our 
three recommendations on cost estimating, DOD indicated that it will 
implement all elements except the risk and uncertainty analysis that it 
believes is not warranted. We think that risk and uncertainty analysis 
is an important tool that establishes a confidence interval for a range 
of possible costs--as opposed to a single-point estimate--and 
facilitates good management decisions and oversight. Such analysis is a 
best practice in our Cost Assessment Guide and we note that OSD's Cost 
Analysis Improvement Group supports and uses this cost-estimating tool. 

Background: 

The JSF program goals are to develop and field an affordable, highly 
common family of stealthy, next-generation strike fighter aircraft for 
the Navy, Air Force, Marine Corps, and U.S. allies. The JSF family 
consists of three variants. The conventional takeoff and landing (CTOL) 
variant will primarily be an air-to-ground replacement for the Air 
Force's F-16 Falcon and the A-10 Warthog aircraft, and will complement 
the F-22A Raptor. The short takeoff and vertical landing (STOVL) 
variant will be a multirole strike fighter to replace the Marine Corps' 
F/A-18C/D and AV-8B Harrier aircraft. The carrier-suitable (CV) variant 
will provide the Navy a multi-role, stealthy strike aircraft to 
complement the F/A-18E/F Super Hornet. DOD is planning to buy a total 
of 2,458 JSFs. The F-35 JSF was christened Lightning II in July 2006. 

Because of the program's sheer size and the numbers of aircraft it will 
replace, the JSF is the linchpin of DOD's long-term plan to modernize 
tactical air forces. It is DOD's largest acquisition program, with 
total cost currently estimated at $300 billion; the longest in planned 
duration, with procurement projected through 2034; and the largest 
cooperative international development program.[Footnote 3] Our 
international partners are providing about $4.8 billion toward 
development, and foreign firms are part of the industrial base 
producing aircraft. They are expecting to procure a minimum of 646 CTOL 
and STOVL JSFs. DOD's funding requirements for the JSF assume the 
benefits in reduced unit costs from these purchases. 

Figure 1 shows the JSF's current procurement profile for U.S. and 
international partners. Partner purchases begin in 2009 and reach a 
maximum of 95 per year in fiscal year 2016. Total expected procurement 
in that peak year, including U.S. quantities, is 225 aircraft. 

Figure 1: JSF Annual Procurement Plans for the United States and 
International Partners: 

[See PDF for image] 

This figure is a vertical bar graph depicting the JSF Annual 
Procurement Plans for the United States and International Partners. The 
vertical axis of the graph represents number of aircraft from 0 to 250. 
The horizontal axis of the graph represents fiscal years from 2007 to 
2034. There is a bar depicted for each fiscal year representing U.S. 
buys, International buys, or a stacked combination of the two. 

Source: GAO analysis of DOD data. 

[End of figure] 

The JSF is a single-seat, single-engine aircraft, designed to rapidly 
transition between air-to-ground and air-to-air missions while still 
airborne. To achieve its mission, JSF will incorporate low-observable 
technologies, defensive avionics, advanced onboard and offboard sensor 
fusion, internal and external weapons, and advanced prognostic 
maintenance capability. According to DOD, these technologies represent 
a quantum leap over legacy tactical aircraft capabilities. In several 
ways, JSF development is also more complex and challenging than the F-
22A Raptor and F/A-18E/F Super Hornet programs, the other two 
contemporary aircraft that DOD is acquiring with JSF to recapitalize 
tactical air forces. The JSF program is simultaneously developing 
several airframes and engines for multiple customers and is projected 
to have significantly more lines of operational flight plan software 
code than the other aircraft. 

The JSF program began in November 1996 with a 5-year competition 
between Lockheed Martin and Boeing to determine the most capable and 
affordable preliminary aircraft design. Lockheed Martin won the 
competition. The program entered system development and demonstration 
in October 2001. At that time, officials planned on a 10ï¿½ years 
development period costing about $34 billion (amount includes costs of 
about $4 billion incurred before system development start). By 2003, 
system integration efforts and a preliminary design review revealed 
significant airframe weight problems that affected the aircraft's 
ability to meet key performance requirements. Weight reduction efforts 
were ultimately successful but added substantially to program cost and 
schedule estimates. In March 2004, DOD rebaselined the program (2004 
Replan), extending development by 18 months and adding $7.5 billion to 
development costs. Program officials also delayed the critical design 
reviews, first flights of development aircraft, and the low-rate 
initial production decision to allow more time to mitigate risks and 
mature designs. 

Progress Measured against Cost, Schedule, and Performance Goals Was 
Mixed over the Last Year: 

The total program acquisition cost estimate by the JSF program office 
has increased since our report last year, primarily due to higher 
projected procurement unit prices. The reported schedule for major 
events showed mostly minor slips. Engineering analyses continue to show 
performance requirements are met, but flight and ground tests planned 
through 2013 will be necessary to confirm these assessments. DOD and 
the contractor reported progress in several areas, including 
international partner agreements, first flights of a JSF prototype and 
test bed, and a more realistic procurement schedule. 

The Program Cost Estimate Increased, while Schedule and Performance 
Estimates Remained about the Same: 

JSF costs increased since last year. Table 1 shows the evolution of 
cost, quantity, and delivery estimates from the initiation of system 
development, through the 2004 Replan, to the latest data available. It 
demonstrates the impacts of higher procurement costs on unit costs and 
schedule delays on the delivery of promised capabilities to the 
warfighters. 

Table 1: Changes in Reported JSF Program Costs, Quantities, and 
Deliveries: 

Expected quantities: Development quantities; 
October 2001 (system development start): 14; 
December 2003[A](2004 Replan): 14; 
December 2005[A]: 15; 
December 2006[A]: (latest available data): 15[B]. 

Expected quantities: Procurement quantities (U.S. only); 
October 2001 (system development start): 2,852; 
December 2003[A](2004 Replan): 2,443; 
December 2005[A]: 2,443; 
December 2006[A]: (latest available data): 2,443. 

Total quantities: 
October 2001 (system development start): 2,866; 
December 2003[A](2004 Replan): 2,457; 
December 2005[A]: 2,458; 
December 2006[A]: (latest available data): 2,458. 

Cost Estimates (then-year dollars in billions): Development; 
October 2001 (system development start): $34.4; 
December 2003[A](2004 Replan): $44.8; 
December 2005[A]:$44.5; 
December 2006[A]: (latest available data): $44.2. 

Cost Estimates (then-year dollars in billions): Procurement; 
October 2001 (system development start): $196.6; 
December 2003[A](2004 Replan): $199.8; 
December 2005[A]: $231.7; 
December 2006[A]: (latest available data): $255.1. 

Cost Estimates (then-year dollars in billions): Military 
construction[C]; 
October 2001 (system development start): $2.0; 
December 2003[A](2004 Replan): $0.2; 
December 2005[A]: $0.2; 
December 2006[A]: (latest available data): $0.5. 

Total program acquisition: 
October 2001 (system development start): $233.0; 
December 2003[A](2004 Replan): $244.8; 
December 2005[A]: $276.5; 
December 2006[A]: (latest available data): $299.8. 

Unit Cost Estimates (then-year dollars in millions): Program 
acquisition; 
October 2001 (system development start): $81; 
December 2003[A](2004 Replan): $100; 
December 2005[A]: $112; 
December 2006[A]: (latest available data): $122. 

Unit Cost Estimates (then-year dollars in millions): Average 
procurement; 
October 2001 (system development start):$69; 
December 2003[A](2004 Replan): $82; 
December 2005[A]: $95; 
December 2006[A]: (latest available data): $104. 

Estimated delivery dates: First operational aircraft delivery; 
October 2001 (system development start): 2008; 
December 2003[A](2004 Replan): 2009; 
December 2005[A]: 2009; 
December 2006[A]: (latest available data): 2010. 

Estimated delivery dates: Initial operational capability; 
October 2001 (system development start): 2010-2012; 
December 2003[A](2004 Replan): 2012-2013; 
December 2005[A]: 2012-2013; 
December 2006[A]: (latest available data): 2012-2015. 

Source: GAO analysis of DOD data. 

[A] Data are from the annual Selected Acquisition Reports that are 
dated in December but not officially released until March or April of 
the following year. The December 2003 data reflect the 2004 Replan. The 
December 2006 data are the latest information on total program costs 
made available to us by DOD. 

[B] A subsequent decision by DOD in September 2007 has reduced 
development test aircraft by 2 to 13. 

[C] Military construction costs have not been fully established and the 
reporting basis changed over time in these DOD reports. The amount 
shown for December 2006 represents costs currently in the 2008 future 
years defense plan. 

[End of table] 

The current estimate for procurement costs, dated December 2006, shows 
an increase of $23.4 billion (plus 10 percent) from the estimate of a 
year earlier and a total of $55.3 billion more (plus 28 percent) since 
2004.[Footnote 4] Procurement cost increases were primarily due to (1) 
extending the procurement period seven years at lower annual rates, (2) 
increased future price estimates based on contractor proposals for the 
first production lot, (3) airframe material cost increases, and (4) 
increases resulting from design maturation. Offsetting a portion of the 
procurement cost increases were lower estimates for labor rates and 
subcontractor costs. 

The official development cost estimate has remained relatively constant 
since the 2004 Replan. However, there were significant changes in scope 
and planned use of funds in order to maintain that estimate as 
officials reduced requirements, did not include full funding for the 
alternate engine program despite congressional interest in the 
program,[Footnote 5] and spent management reserves much faster than 
budgeted. Management reserves are a pool of money set aside--in this 
case about 10 percent of the development contract value remaining--to 
handle unanticipated changes and other risks encountered as a 
development program proceeds. Weight growth early in development and 
subsequent problems resulting from late aircraft design changes and 
subsequent manufacturing inefficiencies depleted reserve funds to an 
untenable level by 2007. The program faced a probable contract overrun. 
DOD officials opted not to request additional funding and time to 
complete development and instead adopted a controversial plan that 
reduced budgeted funds for development test aircraft and flight plans 
in order to replenish management reserves from $400 million to about $1 
billion, an amount deemed prudent to complete the development phase on 
time. This plan, known as the Mid-Course Risk Reduction plan, is 
discussed in more detail later in this report. 

Reported schedule slips for key events since last year's report were 
minor for the most part, but schedules could worsen considerably if the 
delays in maturing the aircraft and engine designs and manufacturing 
test aircraft continue to push work effort into later years. This would 
further compress the time available to complete development and test 
efforts, affecting the scheduled start of initial operational test and 
evaluation and the full-rate production decision, and increasing the 
risk of further delivery delays. The CV's critical design review, the 
last of three design reviews for the program, occurred in June 2007, 
seven months later than had been expected. The initial operational 
capability date for this variant was pushed out two years to March 
2015, to provide more time to mature design and test this variant in 
the demanding carrier environment. The carrier variant is the least 
developed of the three, incorporates larger wings, is heavier, and has 
different speed and range performance requirements than the other two 
variants. 

On the basis of engineering analyses and computer modeling, the JSF 
program projects that the aircraft design will meet seven of the eight 
key performance parameters by the end of development. The aircraft is 
currently not meeting the interoperability parameter, but this depends 
on capabilities being developed outside the JSF program. Key 
performance parameters will be verified during ground and flight 
testing from 2010 to 2013. 

Progress Was Made This Year in Several Important Areas: 

DOD and the contractor made solid progress this year in several areas 
that could establish a foundation to spur future successes. With almost 
90 percent (in terms of dollars) of the acquisition program still 
ahead, these and other improvements could be leveraged to help better 
meet cost, schedule and performance goals. 

* In February 2007, the United States and eight international partners 
signed the Production, Sustainment, and Follow-on Development 
Memorandum of Understanding, committing to purchase aircraft and 
continuing joint development activities. 

* DOD reduced near-term procurement quantities and the rate of ramp-up 
to full rate production. These actions somewhat lessened the 
concurrency of development and production we have previously cited and 
make for a more achievable schedule. 

* The prime contractor and major subcontractors continued to implement 
advanced design and development techniques and utilize extensive 
computer modeling and simulation in innovative ways for design, test, 
and integration activities. 

* DOD and contractor officials also made good progress toward refining 
system capabilities, including establishing mission software 
requirements, with the goal of improving future program executability 
while still meeting warfighter requirements. 

* First flights of the prototype test aircraft and a flying test bed 
occurred in fiscal year 2007. Both are viewed as important risk 
reducers in the test program and initial flights provided valuable and 
useful information, according to program and contractor officials. 

* All test aircraft were in manufacturing during 2007. Low-rate initial 
production of the first two production aircraft and advance buys for 
the second production lot also got under way. 

Development Program Faces Increased Risks of Further Cost Increases and 
More Schedule Delays: 

Late in 2007, DOD officials approved a risky and controversial plan 
that replenishes management reserves by reducing development test 
aircraft and test flights in order to stay within current cost and 
schedule estimates. Difficulties in stabilizing aircraft designs and 
inefficient production of test aircraft resulted in spending management 
reserves faster than anticipated. The flight test program has barely 
begun, but faces substantial risks with reduced assets as delays in 
design and manufacturing continue to further compress the time 
available to complete development work prior to operational testing and 
to support the full-rate production decision. The JSF program is 
halfway to its planned completion, but is behind schedule and over 
cost. On the basis of evidence we have gathered, development costs can 
be expected to increase substantially from the current reported program 
estimate, and the time needed to complete development testing and 
subsequent initial operational testing will likely need to be extended, 
delaying the full-rate production decision now planned for October 
2013. 

Plan to Address Management Reserve Depletion Adds Risk to the JSF 
Development Effort: 

The Office of the Secretary of Defense (OSD) approved the Mid-Course 
Risk Reduction plan in September 2007. The plan reduces development 
test aircraft and test flights, and accelerates the reduction of the 
contractor's development workforce in order to restore management 
reserves to the level considered prudent to complete the development 
contract as planned and within the current cost estimate. The test 
community and others within DOD believe the plan puts the development 
flight program at considerable risk and trades known cost risk today 
for unknown cost and schedule risk in the future. 

Management reserves are budgeted funds set aside for unanticipated 
development challenges and increase a program's capacity to deal with 
unknowns. At development start, JSF budgeted reserves at 10 percent of 
contract value and expected to draw on them at about the same rate as 
contract execution. However, the program has had to use these funds 
much faster than expected to pay for persistent development cost 
increases and schedule delays. A combination of factors contributed to 
this problem, such as late release of engineering drawings, production 
taking longer than planned, and late delivery of parts from suppliers. 
In turn, these contributed to continuing cost and schedule impacts in 
the manufacture of development test aircraft, including extensive and 
inefficient out-of-station work and delays in proving out the 
production schedule. Figure 2 shows how management reserves totaling 
almost $1.4 billion have been depleted since the 2004 Replan. 

Figure 2: JSF's Use of Management Reserves: 

[See PDF for image] 

This figure is a pie-chart, depicting the following data: 

$430 million for engineering drawings: 29%; 
$370 million for supplier design and performance: 25%; 
$350 million for production materials and labor: 24%
$163 million for other: 11%; 
$160 million for weight and technical changes: 11%. 

Source: GAO analysis of DOD data. 

[End of figure] 

By mid-2007, the development program had completed one-half of the 
amount of work scheduled, but had expended two-thirds of the budget. 
Management reserves had shrunk to about $400 million, less than one-
half the amount officials believed necessary to complete the final 6 
years of development. At the same time, the program faced significant 
manufacturing and software integration challenges, costly flight 
testing, and $950 million in other known cost risks. This presented the 
program with a likely untenable contract overrun sometime in 2008 if no 
action was taken. JSF program management identified a continuing 
persistent cost variance of $250 million to $300 million in the 
aircraft development contract and the associated shortfall in reserves 
that required near-term action beyond "belt tightening." 

An overarching integrated product team considered several alternative 
actions, including doing nothing and adding funds from procurement, but 
the team chair concluded that replenishment of the management reserve 
was essential to position the JSF program to successfully address its 
anticipated future development challenges. This option, dubbed the Mid-
Course Risk Reduction Plan, removed two development aircraft (one CTOL 
and one CV), eliminated approximately 850 test flights from the current 
test plan, revised the verification strategy, increased the use of 
ground test labs and the flying test bed, and maximized the number of 
test points to be accomplished during test flights. The plan also 
accelerated reductions in contractor staff and took other actions. In 
total, these planned actions are expected to add between $470 million 
and $650 million into the reserve to recapitalize it to about $1 
billion, an amount officials believe will be needed to complete 
development. Officials intend to use reserves to recover cost and 
schedule losses in manufacturing and to cover additional future needs. 

This plan was subsequently approved by OSD, although serious risks were 
acknowledged and the team was divided on whether the added risks 
outweighed the intended benefit. Those in favor of the plan believed 
that actions were urgently needed to fix the funding imbalance and 
avoid a contract overrun. In this view, the plan would serve as a 
stopgap measure to delay another program restructure until more program 
knowledge and a clearer understanding of future cost requirements were 
gained. 

Officials from several defense offices thought the risks to testing 
were too great and that the plan did not address the underlying design 
and manufacturing problems. The Director, Operational Test and 
Evaluation, identified specific risks associated with the revised test 
verification strategy and recommended against deleting the aircraft, 
citing inadequate capacity to handle the pace of mission testing, and 
for ship suitability, signature testing, and suitability evaluations. 
This increased the likelihood of not finding and resolving critical 
design deficiencies until operational testing, when it is more costly 
and disruptive to do so. OSD's Systems and Software Engineering office 
concurred, expressed concerns that the plan did not treat the root 
causes of ongoing production problems, and doubted that the contractor 
schedule was achievable. The Cost Analysis Improvement Group and others 
agreed that there was too much risk in reducing test assets at this 
time since no production representative variant had started flight 
tests and no analysis of the management reserve depletion had been 
completed. In summary, the plan trades known cost risk today for 
unknown cost and schedule risk in the future. 

Manufacturing Inefficiencies Continue to Increase Costs and Delay the 
Production of Development Aircraft: 

According to our analysis of available evidence, manufacturing test 
aircraft continue to run behind schedule. The prime contractor has 
revised the test aircraft manufacturing schedule three times, resulting 
in slips of up to 16 months in first flight dates of test aircraft. To 
date, about 3 months of progress has been made for every 4 months of 
effort. As officials for now have decided not to extend the development 
period and delay operational tests and full-rate production, this 
inefficiency increases risk and further compresses time and assets 
available to complete test activities. 

Repercussions from the late release of engineering drawings, design 
changes, and parts shortages continue to cause delays and force 
inefficient production line workarounds where unfinished work is 
completed out of station.[Footnote 6] Production data provided by the 
Defense Contract Management Agency (DCMA) show continuing critical part 
shortages, high change traffic, out-of-station work, quality issues, 
and planning rework. These conditions have also delayed efforts to 
mature and demonstrate the production process even as work begins on 
the first production lot. The contractor has not yet proven it can 
efficiently build the JSF, and test aircraft are being built 
differently from the process expected for the production aircraft. 

The first test aircraft, a non-production-representative conventional 
landing prototype completed in 2006, required 65,000 more labor hours 
(about 35 percent more) to build than planned. It encountered most of 
its inefficiencies in the wing and final assembly phases. The second 
test aircraft, a STOVL model, left the production line in December 
2007, and its first flight is expected in May 2008, 8 months later than 
originally scheduled. It cost about 25-30 percent more to build than 
planned. Contractor data show that the wings were only three-fifths 
complete when moved to final assembly. As a result, over 25,000 more 
labor hours had to be performed out of station to complete the wing 
assembly for this aircraft. 

Table 2 shows work performance on the first seven test aircraft to 
enter manufacturing. (This does not include the original prototype 
completed in December 2006.) These data show that nearly all aircraft 
are persistently behind schedule in completing work on these three 
critical components at the Fort Worth, Texas, facility. In terms of 
cost, the data show overall good performance in constructing the 
forward fuselage, but poor results for the wing and final assembly. 

Table 2: Manufacturing Performance Data, End of September 2007: 

Development aircraft: STOVL-1; 
Forward fuselage: Days behind[A]: In mate; 
Forward fuselage: Cost efficiency[B]: 119%; 
Wing: Days behind: In mate; 
Wing: Cost efficiency: 69%; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: 77%. 

Development aircraft: STOVL-2; 
Forward fuselage: Days behind[A]: -19; 
Forward fuselage: Cost efficiency[B]: 148%; 
Wing: Days behind: -129; 
Wing: Cost efficiency: 65%; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: 69%. 

Development aircraft: STOVL-3; 
Forward fuselage: Days behind[A]: -34; 
Forward fuselage: Cost efficiency[B]: 133%; 
Wing: Days behind: -134; 
Wing: Cost efficiency: 49%; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: [Empty]. 

Development aircraft: STOVL-4; 
Forward fuselage: Days behind[A]: -68; 
Forward fuselage: Cost efficiency[B]: 115%; 
Wing: Days behind: -162; 
Wing: Cost efficiency: 41%; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: [Empty]. 

Development aircraft: CTOL-1; 
Forward fuselage: Days behind[A]: -73; 
Forward fuselage: Cost efficiency[B]: 139%; 
Wing: Days behind: -279; 
Wing: Cost efficiency: 23%; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: [Empty]. 

Development aircraft: CTOL-2; 
Forward fuselage: Days behind[A]: -35; 
Forward fuselage: Cost efficiency[B]: 78%; 
Wing: Days behind: -283; 
Wing: Cost efficiency: [Empty]; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: [Empty]. 

Development aircraft: CTOL-3; 
Forward fuselage: Days behind[A]: -35; 
Forward fuselage: Cost efficiency[B]: 58%; 
Wing: Days behind: -140; 
Wing: Cost efficiency: [Empty]; 
Final assembly: Days behind: [Empty]; 
Final assembly: Cost efficiency: [Empty]. 

Source: GAO analysis of Defense Contract Management Agency data: 

[A] This column represents the number of days the aircraft were behind 
in manufacturing and assembly of these components. 

[B] The cost efficiency column is a measure of how well the component 
is actually performing financially in relation to earned budgets. Cost 
efficiency of 100 percent or higher indicates good performance; scores 
under 100 indicate production inefficiencies. 

[End of table] 

Because of production inefficiencies and delays, the contractor has had 
to lengthen the manufacturing schedule three times to provide more time 
to complete work. Production line problems have resulted in slips of 
between 11 and 16 months to first flight dates for each variant. At the 
time of our review, a fourth schedule was being prepared that would add 
another 1 to 4 months to schedules. Officials are reporting some 
improvements in parts shortages, assembly, and product quality, but 
expect the cascading effects from the design delays and manufacturing 
inefficiencies to linger for another couple of years. 

Development Flight Test Efforts Are Beginning with Fewer Assets and 
Revised Verification Strategy: 

The flight test program has just begun, with only about 25 flights 
completed as of January 2008. The program had originally planned to 
conduct development flight tests using 15 aircraft. The recent decision 
to reduce test aircraft to 13 (including the prototype), cut back the 
number of flights, and change how some capabilities are tested will 
stress resources, compress time to complete testing, and increase the 
number of development test efforts that will overlap the planned start 
of operational testing in October 2012. Test officials are concerned 
that capacity will be too constrained to meet schedules and to 
adequately test and demonstrate aircraft in time to support operational 
testing and the full-rate production decision in October 2013. The full 
extent of changes and impacts from a revised test verification strategy 
are still evolving. Program officials reported that if test assets 
become too constrained, production aircraft may eventually be used to 
complete development testing. 

The number of development flight tests had already been reduced twice 
before the Mid-Course Risk Reduction plan, as shown in figure 3. Test 
flights have now been reduced by more than 1,800 flights (26 percent) 
over the last 2 years. 

Figure 3: JSF Development Flight Tests Planned: 

[See PDF for image] 

This figure is a vertical bar graph depicting the following data: 

October 2005 Plan: 
Number of flights: 6,979. 

November 2006 Plan: 
Number of flights: 6,127. 

August 2007 Plan: 
Number of flights: 5,997. 

November 2007 Plan: 
Number of flights: 5,147. 

Source: GAO analysis of DOD data. 

[End of figure] 

Other test issues and events included the following: 

* Flight tests started with the initial development test aircraft, 
which is not considered to be a production-representative aircraft. 
According to program officials, initial flights of this aircraft 
yielded very useful information on flight characteristics. However, 
three incidents--an electrical flight control actuator malfunction in-
flight and an engine blade failure during a ground test--delayed 
further testing from May to December 2007. Another blade failure 
occurred in February 2008. 

* Initial flights of the Cooperative Airborne Test Bed aircraft in 2007 
verified its airworthiness, and it was then modified to integrate some 
JSF systems hardware and software. In December 2007, it began some 
limited mission flight tests, but is not yet fully configured. The Mid-
Course Risk Reduction plan revised the development test verification 
strategy to increase reliance on this specially configured aircraft to 
test capabilities that were going to be demonstrated on JSF aircraft. 

* An operational assessment by testers from the Navy, Air Force, and 
the United Kingdom's Royal Air Force was accomplished from March 2004 
to December 2005 to assess development progress and current JSF mission 
capability. The February 2006 report concluded that the baseline flight 
test schedule provided little capability to deal with unforeseen 
problems and still meet the scheduled start of operational test and 
evaluation in October 2012. Testing officials said the JSF flight test 
program was following the historical pattern of legacy programs in 
making overoptimistic plans and using assumptions not supported by 
historical data. In legacy aircraft, these practices resulted in 
capacity constraints, program slips, and reduced testing tasks. We note 
that these concerns about the JSF were expressed at a time when the 
test program was expected to have the full complement of 15 test 
aircraft, not the 13 now planned. 

Development Challenges Have Been Exacerbated by Inattention to Best 
Acquisition Practices: 

A program as complex and technically challenging as the JSF would be 
expected to have some setbacks, but we believe that the cause of many 
cost and schedule problems can be traced to an acquisition strategy and 
decisions at key junctures that did not adequately follow the best 
practices we have documented in successful commercial and government 
programs.[Footnote 7] The JSF started system development before 
requisite technologies were ready, started manufacturing test aircraft 
before designs were stable, and moved to production before flight tests 
have adequately demonstrated that the aircraft design meets performance 
and operational suitability requirements. We previously reported that 
the JSF acquisition strategy incorporated excessive overlap in 
development and production, posing substantial risks for cost overruns, 
schedule slips, and late delivery of promised capabilities to the 
warfighter.[Footnote 8] 

Six years after system development start, only two of the JSF's eight 
critical technologies are mature by best practice standards, three are 
approaching maturity, and three are immature. Maturing critical 
technologies during system development led to cost growth. For example, 
development costs for the electric-hydraulic actuation and power 
thermal management systems have increased by 195 and 93 percent 
respectively since 2003. 

All three variants fell significantly short of meeting the best 
practices standard of 90 percent of drawings released at the times of 
their respective critical design reviews: 46 percent for the STOVL, 43 
percent for the CV, and 3 percent for the CTOL. Design delays and 
changes to designs were cited by the Mid-Course Risk Reduction team as 
the precipitating cause leading to the depletion of management 
reserves. The late release of drawings resulted in a cascading of 
problems in establishing suppliers and manufacturing process, which led 
to late parts deliveries, delayed the program schedule, and forced 
inefficient manufacturing processes to compensate for the delay. 

Also, the program began initial low-rate production in 2007 before 
delivering an aircraft that fully represents the expected design. 
Efforts to mature production are constrained because the designs are 
not fully proven and tested, and manufacturing processes are not 
demonstrated. A fully integrated, capable production aircraft is not 
expected to enter flight testing until fiscal year 2012, increasing 
risks that problems found may require redesign, production line 
changes, and retrofit expenses for aircraft already built. 

Program Costs Expected to Increase and Schedule Worsen: 

On the basis of the evidence, we expect JSF program costs to increase 
and the schedule worsen to the point where the development period will 
likely need to be extended and Initial Operational Test and Evaluation 
(IOT&E) and full-rate production delayed. A major program restructure 
seems inevitable, unless significant elements of the program can be 
safely eliminated or deferred. The Mid-Course Risk Reduction plan does 
not directly address design and manufacturing inefficiencies that 
created the problem in the first place. If the root causes are not 
identified and fixed, the rapid depletion of management reserves can be 
expected to continue, and more funding will be needed to complete 
development. 

There is no reason to believe that these problems can be easily and 
quickly fixed. While there have been some assembly line improvements, 
program officials expect the manufacturing problems to persist for 
about 2 more years. Officials hope this plan will give them a period of 
time to better and more fully assess all the issues and reevaluate 
development costs and schedule requirements. They are depending on the 
revised test verification plans to maintain the pace and efficacy of 
development testing, but the test community is dubious. What seem more 
likely are additional costs and time to overcome inadequate capacity 
and the elimination or deferral of more test activities. Eliminating 
development test activities and deferring additional tasks to be 
completed during operational testing increase the likelihood that 
design and performance problems will not be identified and resolved 
until late in the program, when it is more costly and disruptive and 
could delay the delivery of capabilities to the warfighter. 

There are also abundant other indicators that acquisition costs will 
substantially increase from what is now being reported to Congress. 
Specifically: 

* DOD has identified billions of dollars in unfunded requirements that 
are not in the program office estimate, including additional tooling 
and procurement price hikes. 

* A new manufacturing schedule in the works indicates continued 
degradation in the schedule and further extends times for first 
flights. 

* Both the aircraft and engine development contracts have persistent, 
substantial cost variances that cost analysts believe are too large and 
too late in the program to resolve without adding to budget. 

* The prime contractor and program office are readying a new estimate 
at completion, which is expected to be much larger than what is now 
budgeted. 

* Three defense organizations independent of the JSF program office 
have all concluded that the program office's cost estimate is 
significantly understated and the current schedule unlikely to be 
achieved. 

* For these and other reasons, we believe that the current JSF cost and 
schedule reported to Congress are not reliable for decision making, as 
discussed next. 

JSF Program Cost Estimate Is Not Reliable: 

The $299.8 billion acquisition cost estimate for the JSF program is not 
reliable because it is not sufficiently comprehensive, accurate, 
documented, or credible. GAO's Cost Assessment Guide outlines best 
practices used throughout the federal government and industry for 
producing reliable and valid cost estimates. We assessed the cost-
estimating methodologies used by the JSF program office against these 
best practices and determined that certain key costs were excluded, 
assumptions used were overly optimistic, documentation was inadequate, 
and no analysis had been done to state the confidence and certainty the 
program office had in its cost estimate. As a result of these 
weaknesses, the JSF program acquisition cost estimate is not reliable 
for decision making. Appendix II contains a more detailed discussion of 
the specific shortcomings we and the other DOD organizations have found 
in the program office cost-estimating methodologies and their potential 
impacts. 

The JSF Cost Estimate Is Not Comprehensive: 

Estimates are comprehensive when they contain a level of detail that 
ensures that all pertinent costs are included and no costs are double-
counted. It is important to ensure the completeness, consistency, and 
realism of the information contained in the cost estimate. Our review 
of the JSF development cost estimate showed that there are several cost 
categories totaling more than $10 billion that are excluded or under 
reported in the program office estimate. These items are summarized in 
table 3 below. 

Table 3: Costs Excluded from the Program Office Acquisition Cost 
Estimate: 

Cost item: Alternate engine; 
Possible impact: $6.8 billion. 

Cost item: Military construction; 
Possible impact: $1.5 billion. 

Cost item: Tooling; 
Possible impact: $2.1 billion. 

Cost item: Capabilities dropped from development; 
Possible impact: Unknown. 

Cost item: Total; 
Possible impact: $10 billion plus. 

Source: GAO analysis of DOD data: 

[End of table] 

Specifically: 

* The current acquisition cost estimate includes only near-term 
development funding for the alternate engine program, excluding 
procurement-related and other development costs of about $6.8 billion. 

* The military services have not firmly established basing needs for 
the entire planned JSF force, but an earlier top-line estimate for 
military construction was at least $2 billion. The current total cost 
estimate includes only near-term budgeted costs of $533 million. 

* The JSF program recently increased its estimate of tooling costs by 
$2.1 billion due to the inclusion of additional tooling requirements 
and estimating methodology changes. 

* Cost and performance trade-offs during development deferred some 
requirements from the current program that may later require additional 
funding. The program office has not quantified these deferrals, but 
Naval Air Systems Command (NAVAIR) officials told us that the amount 
could be in the billions of dollars. 

The JSF Cost Estimate is Not Accurate: 

Estimates are accurate when they are based on an assessment of the 
costs most likely to be incurred. Therefore, when costs change, best 
practices require that the estimate be updated to reflect changes in 
technical or program assumptions and new phases or milestones. DOD's 
Cost Analysis Improvement Group (CAIG) found that the assumptions the 
JSF program office used for weight growth, staffing head counts, 
commonality savings for similar parts, and outsourced labor rate 
savings were overly optimistic and not supported by historical data. 
[Footnote 9] For example, the program office had used a 3 percent 
factor for weight growth whereas the CAIG used a 6 percent factor more 
in line with historical data from other programs. With three variants, 
a joint program with international participation, three different 
engines (cruise, second engine, and lift) in development, and more than 
double the amount of operational flight software lines of code than the 
F-22A and more than four times that of the F/A-18E/F, the JSF program 
is substantially more complex than the F-22 or F/A-18E/F, and therefore 
may not merit assumptions that are even as optimistic as the historical 
data for those programs. 

The program cost estimate is also considered inaccurate because it 
relies on data and reports found to be deficient. JSF program office 
used Lockheed Martin earned value management (EVM) data in estimating 
development costs.[Footnote 10] However, DCMA determined that the data 
as being of very poor quality and issued a report in November 2007 
stating that it is deficient to the point where the government is not 
obtaining useful program performance data to manage risks. Among other 
problem areas, DCMA found that the contractor was using management 
reserve funds to alter its own and subcontractor performance levels and 
cost overruns. DCMA officials who conducted the review told us that the 
poor quality of the data invalidated key performance metrics regarding 
cost and schedule, as well as the contractor's estimate of the cost to 
complete the contract. At the time of our review, corrective actions 
and plans were in process. 

The JSF Cost Estimate Is Not Well Documented: 

Cost estimates are well documented when they can be easily repeated or 
updated and can be traced to original sources through auditing. 
Rigorous documentation increases the credibility of an estimate and 
helps support an organization's decision-making process. The 
documentation should explicitly identify the primary methods, 
calculations, results, rationales, assumptions, and sources of the data 
used to generate each cost element. All the steps involved in 
developing the estimate should be documented so that a cost analyst 
unfamiliar with the program can recreate the estimate with the same 
result. 

We found that the JSF cost model is highly complex and the level of 
documentation is not sufficient for someone unfamiliar with the program 
to easily recreate it. Specifically, we found that the program office 
does not have formal documentation for the development, production, and 
operation and support cost models and could not provide detailed 
documentation such as quantitative analysis to support its assumptions. 
For the development cost estimate, the JSF program officials said they 
did not have a cost model that was continually updated with actual 
costs. Instead the program office relies heavily on earned value 
management data and contractor analysis to update its development cost 
estimate. 

The JSF Cost Estimate Is Not Credible: 

Estimates are credible when they have been cross-checked with an 
independent cost estimate and when a level of uncertainty associated 
with the estimate has been identified. An independent cost estimate 
provides the estimator with an unbiased test of the reasonableness of 
the estimate and reduces the cost risk associated with the project by 
demonstrating that alternative methods generate similar results. 

Several independent organizations have reviewed the JSF program and are 
predicting much higher costs than the program office. Table 4 below 
provides a summary of these assessments. 

Table 4: Outside Organizations' Assessments of JSF Cost and Schedule: 

Assessing organization: CAIG; 
Impact on cost: $5.1 billion more for development, over $33 billion 
more for procurement; 
Impact on schedule: 12 months slip. 

Assessing organization: NAVAIR; 
Impact on cost: $8 billion to $13 billion additional development costs 
or trade-offs adding to procurement costs; 
Impact on schedule: 19-27 months slip. 

Assessing organization: DCMA; 
Impact on cost: $4.9 billion additional cost to complete Lockheed 
Martin development contract; 
Impact on schedule: Up to 12 months slip. 

Source: CAIG, NAVAIR, DCMA. 

[End of table] 

CAIG estimates were prepared using different and more realistic 
assumptions and schedule projections than the program office estimate. 
NAVAIR, which provides resources to the JSF program office cost-
estimating function, derived much higher cost estimates and a longer 
development period based on historical cost performance and removing 
what it considered to be artificial and unachievable schedule 
constraints. Officials were also concerned about the amount and future 
impact of requirements potentially traded or pushed off into the 
procurement phase, which could be even more costly. DCMA projected 
higher development costs for the aircraft contract based on adjusted 
cost and schedule performance to date and assuming additional slips. 
Officials continue to examine the contractor's deficient earned value 
management system and its misreporting of cost and schedule data. 

DOD Intends to Wait to Make a New and Independent Cost Estimate: 

The JSF program has not conducted a fully documented independent cost 
estimate since system development start in 2001. Despite reliability 
concerns and all the significant events and changes in cost, schedule, 
and quantity since then--those reported by the program office as well 
as those identified by other defense organizations and us--DOD does not 
intend to accomplish another one until required to support the full-
rate production decision in 2013. If so, this will mean that the 
program--DOD's largest acquisition and vitally important to our allies-
-will have a 12-year gap between official validated cost estimates. The 
program may complete development and be 6 years into production before 
an accurate, up-to-date, and reliable official cost estimate is done. 

Despite widely held views that costs will likely be higher and the 
schedule longer than reported, the JSF program continues to be funded 
to the level of the program office estimate. DOD acquisition policy 
requires fully documented total program life-cycle cost estimates, with 
validation by the CAIG, at certain major decision points and when 
mandated by the milestone decision authority. DOD officials decided not 
to do such an estimate at the start of low-rate initial production in 
2007, which typically coincides with a major milestone. 

Future Challenges as Program Moves Forward: 

The JSF is entering its most challenging phase as it finalizes three 
designs, matures manufacturing processes, conducts flight tests, and 
ramps up production. The first and foremost challenge is maintaining 
affordability in three dimensions--reasonable procurement prices, 
stable annual funding, and economical life-cycle operating and support 
costs. If affordability is not maintained during the acquisition 
program, quantities bought by the United States and allies may either 
decrease or else consume more of the available defense budgets. Over 
the life cycle of a system, higher costs for maintaining readiness and 
maintainability drive up annual operating expenses and may limit funds 
for new investments. Other program challenges could affect future 
quantities and the mix of aircraft procured by the United States and 
our allies. 

Affordability Concerns Have Major Repercussions: 

From its outset, the JSF goal has been to develop and field an 
affordable, highly common family of strike aircraft. Rising unit 
procurement prices, and somewhat lower commonality than expected, raise 
concerns that the United States and its allies may not be able to buy 
as many aircraft as currently planned. Average unit procurement costs 
are up 27 percent since the 2004 Replan and 51 percent since the start 
of system development (see table 1). Rising prices erode buying power, 
likely resulting in reduced quantities and delays in delivering 
promised capabilities to the warfighter. 

The program also places an unprecedented demand for funding on the 
defense budget--an annual average of about $11 billion for the next two 
decades--with attendant funding risk should political, economic or 
military conditions change. The JSF will have to annually compete with 
other defense and nondefense priorities for the shrinking discretionary 
federal dollar. To complete the acquisition program as currently 
planned, JSF will require about $269 billion from 2008 through 2034. 
Annual funding requirements for procurement increase rapidly as 
production ramps up to the full-rate production decision expected in 
October 2013. During the peak years of production, JSF procurement 
funding requirements are expected to average about $12.5 billion per 
year for the 12-year period spanning fiscal years 2012-2023. 

Figure 4 illustrates the annual funding requirements as of December 
2006 and contrasts these with plans from prior years. The December 2003 
line shows the funding profile resulting from the 2004 Replan and the 
2005 line shows the jump in funding needed to accommodate program cost 
increases in the period following the Replan. The 2006 data reflect the 
impact on annual funding requirements from extending procurement 7 
years. The extension reduced annual budget amounts, but requires 
continued funding through 2034 to procure deferred quantities. DOD 
calculated that the extension added $11.2 billion to total procurement 
cost. 

Figure 4: JSF Acquisition Program's Annual Funding Requirements: 

[See PDF for image] 

This figure is a combination line and vertical bar graph. The vertical 
axis of the graph represents funding requirements (dollars in 
billions). The horizontal axis represents fiscal years from 2007 
through 2034. Lines depict the following: December 2003 plan; and: 
December 2005 plan. Vertical bars for each fiscal year depict the 
December 2006 plan. 

Source: GAO analysis of DOD data. 

[End of figure] 

A third aspect of affordability is the life-cycle cost of ownership. 
DOD is recapitalizing its tactical air forces by replacing aging legacy 
systems with new, more capable systems, like the JSF, that incorporates 
reliability and maintainability features designed to reduce future 
operating costs. Recently, DOD sharply increased its projection of JSF 
operating and support costs compared to previous estimates. The 
December 2006 SAR projected life-cycle operating and support costs for 
all three variants at $650.3 billion, almost double the $346.7 billion 
amount shown in the December 2005 SAR and similar earlier estimates. 
The operating cost per flying hour for the JSF CTOL is now estimated to 
be greater than current flying hour cost for the F-16, one of the 
legacy aircraft to be replaced. 

Officials explained that the amounts reported in 2005 and before were 
early estimates based on very little data, whereas the new estimate is 
of higher fidelity, informed by more information as JSF development 
progresses and more knowledge is obtained. Factors responsible for the 
increased cost estimate included a revised fielding and basing plan, 
changes in repair plans, revised costs for depot maintenance, increased 
fuel costs, increased fuel consumption, revised estimates for manpower 
and mission personnel, and a new estimate of the cost of the JSF's 
autonomic logistics system. 

Overall, the cost of ownership represents a very large and continuing 
requirement for the life of fielded aircraft. According to the new 
estimate, we calculate that DOD will incur about $24 billion per year 
to operate and support JSF units, assuming the quantities now planned 
and an 8,000-hour service life for each JSF aircraft fielded over time. 

Commonality Is Less than Expected: 

From the inception of the program, DOD has anticipated major cost 
savings from developing and fielding JSF variants that share many 
common components and subsystems. While a degree of commonality has 
been achieved, expectations are now lower than they were at program 
start. Substantial commonality has been maintained for the mission 
systems among all three variants and for the propulsion system of the 
conventional and carrier variants. However, commonality among airframes 
and vehicle systems has declined overall since the start of system 
development. Figure 5 shows the decline in airframe commonality, the 
most costly of the four major categories. For example, in October 2001 
DOD anticipated that the CTOL airframe would be more than 60 percent 
common with the other variants. Commonality had declined to about 40 
percent by December 2006. Lesser commonality will likely increase 
acquisition and future support costs. 

Figure 5: Changes in Airframe Commonality: 

[See PDF for image] 

This figure is a multiple bar graph depicting the following 
information: 

Aircraft variant: CTOL;
Percent in common, October 2001: approximately 62%; 
Percent in common, March 2004: approximately 58%; 
Percent in common, December 2006: approximately 40%. 

Aircraft variant: CTOL;
Percent in common, October 2001: approximately 54%; 
Percent in common, March 2004: approximately 53%; 
Percent in common, December 2006: approximately 30%. 

Aircraft variant: CTOL;
Percent in common, October 2001: approximately 40%; 
Percent in common, March 2004: approximately 35%; 
Percent in common, December 2006: approximately 30%. 

Source: GAO analysis, DOD data. 

[End of figure] 

Navy and Marine Corps Requirements and Mix Are Unsettled: 

The current JSF program shows a total quantity of 680 aircraft to be 
procured by the Department of the Navy, but the allocation between the 
CV and STOVL variants has not been officially established. We observe 
that the Navy and Marine Corps have somewhat divergent views on the 
quantities, intended employment, and basing of JSF aircraft. The Navy 
wants the Marine Corps to buy some CV variants and continue to man some 
of its carrier-based squadrons. The Marine Corps, however, wants to 
have a future strike force composed solely of the STOVL variant and has 
established a requirement for 420 aircraft. During conflicts, the 
Marines plan to forward deploy JSFs to accompany and support the 
expeditionary ground forces. 

Navy officials told us that they have some time to make decisions 
because they will be buying a mix of both CVs and STOVLs in the early 
years of production and that funding requirements are not significantly 
affected since unit prices for both variants are about the same. 
However, we believe the continuing disagreements on basing, employment, 
and force mix will have increasingly stronger impacts on JSF plans, 
costs, and international partner relations. Decreased quantities of 
STOVLs bought by the Department of the Navy would likely result in 
higher unit prices paid by the Marine Corps and two allies buying 
STOVLs. Fundamental decisions on the mix of naval aircraft also affect 
future operating and support costs, military construction, and carrier 
requirements. 

Officials also have some reservations whether they can afford the 
quantities now planned at peak production rates. Navy and Marine Corps 
officials told us last year that buying the JSF at the current planned 
rate--requiring a ramp-up to 50 CV and STOVL aircraft by fiscal year 
2015--will be difficult to achieve and to afford, particularly if costs 
increase and schedules slip. Officials told us that a maximum of 35 per 
year was probably affordable, given budget plans at that time.[Footnote 
11] 

Containing Future Weight Growth: 

Weight growth was the most significant challenge faced by the JSF 
program early in development. Redesign efforts to address weight growth 
was the single largest factor causing the $10 billion cost increase and 
18-month extension in the development schedule since the start of 
system development. 

While the weight increase has been addressed for now, projections are 
that the aircraft weight will continue to increase during the balance 
of the development period, consistent with weight increases seen on 
legacy aircraft programs. According to an OSD official with knowledge 
of legacy aircraft development efforts, half of all weight growth 
during the development effort can be typically expected after first 
flight but prior to initial operational capability, and that additional 
small but persistent weight increases can be expected during the 
aircraft's service life. First flight of a production-representative 
JSF has not yet occurred, and weight is running very close to the 
limits as evaluated by engineering analyses and trend extrapolation. As 
designs continue to mature and flight testing intensifies, maintaining 
weight within limits to meet warfighter capability requirements will be 
a continuing challenge and pose a major risk to meeting cost, schedule, 
and performance goals. 

Conclusions: 

The clear implication from performance to date and the Mid-Course Risk 
Reduction plan is that additional costs and time will be needed to 
complete JSF development. The plan to recapitalize management reserve 
at the expense of test assets is risky with potential major impacts 
down the road on costs, performance requirements, and fielding 
schedules. The remaining development effort will be less robust than 
originally planned and depends on a revised test verification strategy 
that is still evolving. As a result, the development effort has an 
increased risk of not fully measuring JSF capabilities and deficiencies 
prior to operational testing and could result, in the words of one DOD 
official, in the future operational test period being one of discovery 
rather than validation of the aircraft's capabilities and deficiencies. 
Finding and fixing deficiencies during operational testing and after 
production has ramped up is costly, disruptive, and delays getting new 
capabilities to the warfighter. 

Because the program cost estimate is not reliable when judged against 
best standards, the decision making and oversight by Congress, top 
military leaders, and our allies are diminished. The picture they do 
have is one where costs continue to rise and schedules slip. The 
situation will be considerably worsened if the cost estimates of 
defense offices outside the program are more accurate than the 
conservative, official in-house estimates. Waiting 12 years between 
fully documented and validated total program cost estimates is contrary 
to policy and good management, given all the changes in cost, quantity, 
schedules, and other events that have occurred since the 2001 estimate. 
The size of the JSF acquisition, its impact on our and allied tactical 
air forces, and the unreliability of the current estimate argue for an 
immediate new and independent cost estimate and uncertainty analysis. 
This is critical information needed by DOD management to make sound 
trade-off decisions against competing demands and by Congress to 
perform oversight and hold DOD accountable. 

Program problems and setbacks must be put into perspective: The JSF is 
DOD's largest and most complex aircraft acquisition and an integral 
component of the future force. Problems happen in such an environment. 
Progress has been made and some significant challenges overcome, but 
more await as program moves into flight testing and low-rate 
production. Maintaining affordability so the United States military and 
our allies can buy, field, and support the numbers needed by the 
warfighter remains the overarching challenge. 

Recommendations for Executive Action: 

Because of the elevated risks and the valid objections raised by the 
test community and other DOD offices, we recommend that the Secretary 
of Defense direct elements of the department to revisit and, if 
appropriate, revise the Mid-Course Risk Reduction plan recently 
approved. This should be supported by an intensive analysis that 
includes causes of management reserve depletion, an evaluation of 
progress against the baseline manufacturing schedule, and the progress 
made in correcting deficiencies in the contractor's earned value 
management system. It should also include an in-depth examination of 
alternatives to the current plan and address the specific concerns 
raised by officials regarding testing capacity, the integration of 
ground and flight tests, and backup plans should capacity become 
overloaded. 

So that DOD may have an accurate picture of JSF cost and schedule 
requirements, and that Congress may have an accurate understanding of 
future funding requirements, we recommend that the Secretary of Defense 
direct that: 

1. The JSF program office update its cost estimate using best 
practices, so that the estimate is comprehensive, accurate, well 
documented, and credible. Specifically, the JSF program office should: 

* include costs that were inappropriately omitted from the estimate; 

* identify performance requirements that have been traded off in 
development; 

* fully document assumptions, data sources and methodologies in the 
cost model; and: 

* perform a risk and uncertainty analysis to focus on key cost drivers 
and reduce the risk of cost overruns. 

2. The program conduct a full Schedule Risk Analysis to ensure that its 
schedules are fully understood, manageable, and executable; 

3. DOD conduct a full, independent cost estimate should be conducted 
according to the highest standards of any DOD cost estimating 
organization, based on a comprehensive review of program data; that 
this cost estimate be reviewed by an independent third party such as 
the CAIG; and that the results of these estimates be briefed to all 
interested parties in DOD and Congress. 

Agency Comments and Our Response: 

DOD provided us with written comments on a draft of this report. The 
comments appear in appendix III. DOD also provided several technical 
comments, which we incorporated in this report. 

DOD substantially agreed with our recommendation to revisit the Mid-
Course Risk Reduction plan. DOD stated that the plan is a cost-
effective approach with a manageable level of risk that will be 
monitored and revised if necessary. We believe the plan's reduction of 
test resources will hamper development testing and that the Department 
will eventually have to make programmatic adjustments, adding cost and 
time. 

DOD also substantially agreed with our three recommendations on cost 
estimating. DOD indicated that it will implement all elements except 
the risk and uncertainty analysis, which is unwarranted in its view. We 
believe that risk and uncertainty analysis is an important tool that 
establishes a confidence interval for a range of possible costs--as 
opposed to a single-point estimate--and facilitates good management 
decisions and oversight. Such analysis is a best practice in our Cost 
Assessment Guide and we note that OSD's Cost Analysis Improvement Group 
supports and uses this cost-estimating tool. 

We are sending copies of this report to the Secretary of Defense; the 
Secretaries of the Air Force, Army, and Navy; and the Director of the 
Office of Management and Budget. We will also provide copies to others 
on request. In addition, the report will be made available at no charge 
on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-4841. Contact points for our offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this report. Staff members making key contributions to this 
report are listed in appendix IV. 

Signed by: 

Michael J. Sullivan: 
Director Acquisition and Sourcing Management: 

List of Congressional Committees: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Daniel K. Inouye: 
Chairman: 
The Honorable Ted Stevens: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate: 

The Honorable Ike Skelton: 
Chairman: 
The Honorable Duncan L. Hunter: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Honorable John P. Murtha: 
Chairman: 
The Honorable C.W. Bill Young: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives: 

[End of section] 

Appendix I: Scope and Methodology: 

To determine the Joint Strike Fighter (JSF) program's progress in 
meeting cost, schedule, and performance goals, we received briefings by 
program and contractor officials and reviewed financial management 
reports, budget documents, annual Selected Acquisition Reports, monthly 
status reports, performance indicators, and other data. We compared 
reported progress with prior years' data, identified changes in cost 
and schedule, and obtained officials' reasons for these changes. We 
interviewed Department of Defense (DOD), JSF program, and contractor 
officials to obtain their views on progress, ongoing concerns and 
actions taken to address them, and future plans to complete JSF 
development and ramp up procurement. 

To assess plans and risks in development, manufacturing, and test 
activities, we examined program documents and interviewed DOD and 
contractor officials about changes to the test plan and actions taken 
to modify these plans to address funding and schedule challenges. This 
included reviewing and interviewing program and Office of the Secretary 
of Defense (OSD) officials about changes to development testing that 
evolved in response to a projected shortfall in management reserves and 
a goal to stay on schedule toward a full-rate production decision in 
October 2013. We reviewed information compiled by program officials to 
document options they considered viable, changes to the test plan and 
test resources that would occur under a proposed risk reduction option, 
and challenges/risks to taking this course of action and possible 
fallback plans. We also reviewed stakeholder views of options and the 
benefits and challenges of going forward with the changes made to the 
development test plan. We collected manufacturing cost and work 
performance data to assess progress against plans, determined reasons 
for manufacturing delays, discussed program and contractor plans to 
improve, and expected impacts on development and operational tests. 

In assessing program cost estimates, we also evaluated the JSF joint 
program office estimating methodologies, assumptions, and results to 
determine whether the official cost estimates were comprehensive, 
accurate, well documented, and credible. We used our draft guide on 
estimating program schedules and costs, which is based on extensive 
research of best practices. Our Cost Assessment Guide considers an 
estimate to be accurate if it is not overly conservative, is based on 
an assessment of the most likely costs, and is adjusted properly for 
inflation; comprehensive if its level of detail ensures that all 
pertinent costs are included and no costs are double-counted; well 
documented if the estimate can be easily repeated or updated and can be 
traced to original sources through auditing; and credible if the 
estimate has been cross-checked with an independent cost estimate and a 
level of uncertainty associated with the estimate has been identified. 
We also interviewed the JSF program office's cost estimating team to 
obtain a detailed understanding of the cost model and met with the 
Department of Defense Cost Analysis Improvement Group (CAIG)[Footnote 
12] to understand their methodology, data and approach in developing 
their Joint Strike Fighter independent cost estimate. We analyzed 
earned value management (EVM) reports and met with the Naval Air 
Systems Command and the Defense Contract Management Agency (DCMA) to 
discuss the EVM data and to obtain their independent cost estimates for 
JSF development efforts. To assess the validity and reliability of 
prime contractors' earned value management systems and reports, we 
analyzed the EVM reports and reviewed audit reports prepared by the 
DCMA. 

To identify future challenges, we continued discussions with DOD and 
contractor officials on forward-looking plans and areas of emphasis. We 
analyzed budget requirements from successive plans and tracked 
contributing factors to changes in budget. We collected information on 
commonality assessments among the three variants and trends. With Navy 
and Marine Corps officials, we discussed future plans on the employment 
and quantity mix of aircraft and identified differences in plans and 
perspectives. We discussed past and present weight growth issues with 
engineers and plans for controlling future growth. 

[End of section] 

In performing our work, we obtained information and interviewed 
officials from the JSF Joint Program Office, Arlington, Virginia; 
Aeronautical Systems Center, Wright-Patterson Air Force Base, Ohio; 
Naval Air Systems Command, Patuxent River, Maryland; Defense Contract 
Management Agency, Fort Worth, Texas; and Lockheed Martin Aeronautics, 
Fort Worth, Texas. We also met and obtained data from the following OSD 
offices in Washington, D.C.: Director, Operational Test and Evaluation; 
Program Analysis and Evaluation; Cost Analysis Improvement Group; 
Portfolio Systems Acquisition (Air Warfare); and Systems and Software 
Engineering. We conducted this performance audit from June 2007 to 
February 2008 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: GAO Assessment of JSF Program Cost Estimate: 

The Current JSF Program Office Acquisition Cost Estimate Is Not 
Reliable: 

The $299.8 billion acquisition cost estimate for JSF is not reliable 
because it is not sufficiently comprehensive, accurate, documented, or 
credible. Cost-estimating organizations throughout the federal 
government and industry use certain key practices to produce sound cost 
estimates that are comprehensive and accurate and that can be easily 
traced, replicated, and updated. GAO's Cost Assessment Guide outlines 
practices that, if followed correctly, should result in high-quality, 
reliable, and valid cost estimates that management can use for making 
informed decisions. We assessed the methodology used by the JSF program 
office to determine its development cost estimate against four best 
practices characteristics, which are that an estimate should be 
comprehensive, accurate, well documented, and credible.[Footnote 13] We 
found that the JSF program office has not followed best practices for 
developing a reliable and valid life cycle cost estimate because it did 
not include certain key costs, assumptions used to develop the estimate 
are overly optimistic, the estimate is not well documented, and no 
analysis has been done to state the confidence and certainty it has in 
its estimate. As a result of these weaknesses, the JSF program 
acquisition cost estimate is not reliable for decision making. 

The JSF Cost Estimate Is Not Comprehensive: 

Estimates are comprehensive when they contain a level of detail that 
ensures that all pertinent costs are included and no costs are double-
counted. It is important to ensure the completeness, consistency, and 
realism of the information contained in the cost estimate. Our review 
of the JSF development cost estimate showed that there are several cost 
categories totaling more than $10 billion that are excluded or under 
reported in the program office estimate. These items are summarized in 
table 5. 

Table 5: Costs Excluded from the Program Office Acquisition Cost 
Estimate: 

Cost item: Alternate engine; 
Possible impact: $6.8 billion. 

Cost item: Military construction; 
Possible impact: $1.5 billion plus. 

Cost item: Tooling; 
Possible impact: $2 billion-$3 billion plus. 

Cost item: Capabilities dropped from development and deferred; 
Possible impact: Unknown. 

Cost item: Total; 
Possible impact: $10 billion plus. 

Source: GAO analysis of DOD data. 

[End of table] 

* Alternate engine program. Congress has been interested in DOD 
developing a second source for the JSF engine to induce competition and 
to reduce operational risks in the future should the sole engine 
develop problems requiring the grounding of all JSFs. DOD has not 
wanted to pursue this second engine source and twice removed funding 
from the JSF program line. In 2005, DOD deleted a total of about $7.2 
billion from the JSF's development and procurement accounts for the 
alternate engine. In 2006, it reinserted $340 million for the program, 
reflecting only development funding in the future years defense 
program. This omits about $6.8 billion left out of the JSF cost 
estimate for this program. 

* Military construction. In prior years, the JSF cost estimate included 
$2 billion for military construction costs. Since the services had not 
yet fully established basing plans, this amount was a top-level 
parametric estimate not based on discrete estimates for specific sites. 
The current December 2006 cost estimate reported military construction 
costs of $533 million, reflecting only the amount budgeted in the 
fiscal year 2008 future years defense program. This means that about 
$1.5 billion in military construction--and possibly more--will 
eventually be required for specific basing needs of the JSF fleets. DOD 
will update military construction estimates as the services identify 
specific site requirements. 

* Tooling. The JSF program recently increased its estimate of tooling 
costs due to the inclusion of additional tooling requirements and 
estimating methodology changes. This change is ongoing, and has not yet 
been included in official program estimates. According to a recent 
press report, a Lockheed Martin official stated that the full 
requirement to support procurement by our allies was not adequately 
factored into prior tooling estimates. The program estimates the 
additional cost through 2015 at about $2.1 billion. 

* Deferred capabilities. Cost and performance trade-offs during 
development have resulted in some requirements being deleted from the 
program cost estimate and deferred until later years. This includes a 
number of planned capabilities dropped from the final block of 
development software. The program office has not quantified the cost of 
these deferred capabilities, and the costs are not reflected anywhere 
in the program office's life-cycle cost estimate. Naval Air Systems 
Command (NAVAIR) officials told us that the total deferred amount could 
be in the billions of dollars. We note that prior acquisitions such as 
the Global Hawk and F-22A programs also deferred requirements that 
would later need additional funding. For example, we reported in 2005 
that the Global Hawk program costs did not include $400.6 million in 
known additional procurement costs for sensors, ground station 
enhancements, and other items required to achieve the system's initial 
full-up capability.[Footnote 14] These costs had been in the program 
baseline but were later deferred and reclassified because of cost 
pressures and schedule changes. Similarly, the Air Force's $5.9 billion 
modernization and reliability improvement program includes 
capabilities deferred from the acquisition program and reliability 
enhancements needed to correct deficiencies and achieve the level of 
reliability that was supposed to be accomplished during acquisition. 

The JSF Cost Estimate Is Not Accurate: 

Estimates are accurate when they are based on an assessment of the 
costs most likely to be incurred. Therefore, when costs change, best 
practices require that the estimate be updated to reflect changes in 
technical or program assumptions and new phases or milestones. DOD's 
Cost Analysis Improvement Group found that the assumptions the JSF 
program office used for weight growth, staffing head counts, 
commonality savings for cousin (similar) parts, and outsourced labor 
rate savings could be too optimistic, given the program's complexity. 
With three variants and three engines (cruise, alternate, and lift) in 
development, multiple customers, and more than double the amount of 
operational flight software than the F-22A and four times that of the 
F/A-18E/F, the JSF acquisition program is substantially more complex 
than those contemporary systems, and therefore may not merit 
assumptions that are even as optimistic as the historical data for 
those programs. The following table shows some major differences in 
assumptions used by the program office and the CAIG in estimating JSF 
costs. 

Table 6: Major Assumptions in JSF Cost Estimates: 

Assumption: Engineering head count; 
JSF Program office: 5,000 engineers; 
CAIG: 3,000 engineers. 

Assumption: Weight growth; 
JSF Program office: 3% average growth based on Lockheed Martin 
database; 
CAIG: 6% based on historical data. 

Assumption: Cousin parts (similar, but not identical, parts among the 
variants); 
JSF Program office: 82% credit for commonality among cousin parts; 
CAIG: 25% credit for commonality among cousin parts. 

Assumption: Labor cost savings for outsourcing; 
JSF Program office: 50% cost savings; 
CAIG: None - based labor costs on forward pricing rate agreements. 

Assumption: Additional 2,000 pounds for carrier variant; 
JSF Program office: No cost increase; 
CAIG: Estimated a cost increase. 

Assumption: Fee, and "fee on fee" for Northrop Grumman and BAE Systems 
production items; 
JSF Program office: 13% fee on all development and production 
contracts; No "fee on fee" impact; 
CAIG: 15% fee on all development and production contracts; Estimated a 
"fee on fee" impact. 

Assumption: Test failure redesign effort; 
JSF Program office: No 
additional costs; 
CAIG: Additional costs for this effort. 

Source: JSF program office, CAIG. 

[End of table] 

JSF program officials told us that they use Lockheed Martin earned 
value management data in creating their estimate of JSF development 
costs. However, DCMA, which reviews contracts and industrial 
performance for DOD, identified this data as being of very poor 
quality, calling into question the accuracy of any estimate based on 
these data. In November 2007, DCMA issued a report saying that Lockheed 
Martin's tracking of cost and schedule information at its aerospace 
unit in Fort Worth, Texas--where the JSF program is managed--is 
deficient to the point where the government is not obtaining useful 
program performance data to manage risks. 

DCMA said that Lockheed's earned value data at the Fort Worth facility 
are not sufficient to manage complex, multibillion-dollar weapon 
systems acquisition programs. Among other problem areas, DCMA found 
that Lockheed had not clearly defined roles and responsibilities, and 
was using management reserve funds to alter its own and subcontractor 
performance levels and cost overruns. These issues hurt DOD's ability 
to use the Lockheed data to determine product delivery dates and 
develop accurate estimates of program costs. DCMA officials who 
conducted the review at Lockheed Martin told us that the poor quality 
of the data invalidated key performance metrics regarding cost and 
schedule, as well as the contractor's estimate of the cost to complete 
the contract. NAVAIR had also raised concerns about Lockheed Martin's 
earned value system as early as June 2005, and these officials told us 
they were in agreement with the findings in the November 2007 DCMA 
report. NAVAIR officials also said that most deficiencies identified by 
the DCMA report have the effect of underreporting costs, and that the 
official program cost estimates will increase if the deficiencies are 
corrected. 

Also in 2007, the prime contractor alerted DOD to a billing error 
involving duplicate charges for the portion of the earned award fee 
paid to subcontractors. This resulted in $266 million in overcharges. 
Government officials became concerned that such a large discrepancy 
could occur without the government's knowledge and questioned the 
adequacy of the contractor's billing system and accounting procedures. 
DCMA and the Defense Contract Audit Agency were tasked to conduct an 
investigation. Their investigation found that the overbilling resulted 
from an accounting system error in the internal handling of award fees 
on the JSF contract. According to the investigation report, the error 
that created the overbilling has been corrected, and the government has 
recouped the overbilled principal and interest. 

The JSF Cost Estimate Is Not Well Documented: 

Cost estimates are well documented when they can be easily repeated or 
updated and can be traced to original sources through auditing. 
Rigorous documentation increases the credibility of an estimate and 
helps support an organization's decision-making process. The 
documentation should explicitly identify the primary methods, 
calculations, results, rationales or assumptions, and sources of the 
data used to generate each cost element. All the steps involved in 
developing the estimate should be documented so that a cost analyst 
unfamiliar with the program can recreate the estimate with the same 
result. 

We found that the JSF cost model is highly complex and the level of 
documentation is not sufficient for someone unfamiliar with the program 
to easily recreate it. Specifically, we found that the program office 
does not have formal documentation for the development, production, and 
operating support cost models. Instead, it relies on briefing slides 
that describe the methodology and the data sources used, but did not 
provide detailed documentation such as quantitative analysis to support 
the assumptions that were involved in producing the life-cycle cost 
estimate. For the development cost estimate, the JSF program office 
admitted it did not have a cost model that continually updates with 
actual costs. Instead the program office relies heavily on earned value 
management data and analysis from Lockheed Martin to update its 
development cost estimate, but provided us no documentation to back up 
this claim. 

The JSF Cost Estimate Is Not Credible: 

Estimates are credible when they have been cross-checked with an 
independent cost estimate and when a level of uncertainty associated 
with the estimate has been identified. An independent cost estimate 
provides the estimator with an unbiased test of the reasonableness of 
the estimate and reduces the cost risk associated with the project by 
demonstrating that alternative methods generate similar results. 

Several independent organizations have reviewed the JSF program and are 
predicting much higher costs than the program office. Table 7 below 
provides a summary of these assessments. 

Table 7: Outside Organizations' Assessments of JSF Cost and Schedule: 

Assessing organization: CAIG; 
Impact on cost: $5.1 billion more for development, over $33 billion 
more for procurement; 
Impact on schedule: 12 months slip. 

Assessing organization: NAVAIR; 
Impact on cost: $8 billion to $13 billion additional development costs 
or trade-offs adding to cost in procurement; 
Impact on schedule: 19-27 months slip. 

Assessing organization: DCMA; 
Impact on cost: $4.9 billion additional cost to complete Lockheed 
Martin contract, including the cost of a 12-month schedule slip; 
Impact on schedule: Up to 12 months slip. 

Source: CAIG, NAVAIR, DCMA. 

[End of table] 

In 2005, the CAIG performed an independent estimate of JSF program 
development costs, which include the cost of the Lockheed Martin 
contract and fees as well as the government's in-house costs. The CAIG 
expected that the development phase would cost $5.1 billion more than 
expected by the program office, measured against the program office's 
most recent data available at that time, from the 2004 Selected 
Acquisition Report (SAR). The CAIG official in charge of the estimate 
told us that while it has not formally presented an updated estimate to 
the program office, the order of magnitude of difference between the 
CAIG and program office estimates remains roughly the same as at the 
time of the 2005 independent estimate. 

Table 8: JSF Development Program Cost Estimate Comparison (Then-year 
dollars in billions): 

System development start in 2001; 
JSF: $30.2; 
CAIG: $31.4; 
Delta between CAIG And JSF: $1.2. 

December 2004 SAR; 
JSF: $41.5; 
CAIG: $46.6; 
Delta between CAIG And JSF: $5.1. 

Cost growth from 2001 start to 2004 SAR; 
JSF: $11.3; 
CAIG: $15.2; 
Delta between CAIG And JSF: $3.9. 

Source: CAIG. 

[End of table] 

The variance between the CAIG and program office estimates grew 
significantly as the program encountered problems. The CAIG explained 
that some of the $15.2 billion growth in its development cost estimate 
from Milestone B in 2001 to the December 2004 SAR was due to initial 
assumptions that 5,000 engineers would be available to work on the 
three JSF variants. This assumption turned out to be too optimistic 
since only about 3,000 engineers have been working on the program. 
Because of fewer people available to support the JSF design and 
development, the CAIG shifted the program schedule to the right, 
increasing the costs. The program office, on the other hand, assumed it 
could get the same effort done with fewer people. In addition, the CAIG 
used historical data from the F-22A program, including the costs to 
design the aircraft, test it, and redesign any fixes, and adjusted 
these data to account for differences in the JSF program, including the 
three variants. The program office relies mostly on contractor data. 

When it was awarded the development contract in 2001, Lockheed Martin 
agreed to develop the JSF aircraft for $16.5 billion, excluding fee. In 
April 2005, the development program was rebaselined, adding more than 
$6 billion to reflect funds added to the program due to weight growth 
issues in 2003. This raised the JSF baseline development contract cost 
estimate to $23.2 billion, excluding fee. 

Despite the additional funding to cover preexisting cost and schedule 
overruns, Lockheed Martin's JSF development cost and schedule 
performance has continued to decline over time. As shown in figure 6, 
cost and schedule variances continued on a downward trend despite the 
April 2005 rebaseline. As of September 2007, Lockheed Martin was 
reporting cumulative cost overruns of $305.7 million and was behind 
schedule to an extent valued at $251.3 million. 

Figure 6: Cost and Schedule Variances on the Aircraft Development 
Contract: 

[See PDF for image] 

This figure is a line graph depicting lines representing Cost variance 
and Schedule variance. The vertical axis of the graph represents 
variance from plan (dollars in millions) from -$1000 to +$400. The 
horizontal axis of the graph represents fiscal years from December 2001 
to September 2007. 

Source: GAO analysis of DOD data. 

[End of figure] 

Key drivers of cost overruns to date have included unfunded 
requirements for design changes, loss of commonality savings, critical 
part shortages, high change traffic, inefficient productivity due to 
performing work out of sequence, constant rework, suppliers' 
performance, late release of engineering requirements, a greater than 
planned effort for designs of the short takeoff and landing and the 
conventional takeoff and landing variants, and additional radar 
testing. Some of this cost variance is due to optimistic assumptions at 
the beginning of the program. For example, cost estimates assumed that 
only one design iteration would be needed, whereas in reality it takes 
numerous design iterations before the final designs are determined. 

Despite its poor performance since the rebaseline, Lockheed Martin was 
predicting only a $113 million cost overrun at contract completion. 
This is unrealistic given the persistence and size of the $305.7 
million overrun reported in September 2007, at which point the contract 
was 67 percent complete. In order to achieve a $113 million overrun at 
completion, Lockheed Martin would have to not only incur no further 
cost variances from now until completion of the contract, but it would 
also have to significantly improve its performance. This is unlikely 
given that studies of more than 700 defense programs have shown limited 
opportunity for getting a program back on track once it is more than 15 
percent to 20 percent complete. The true cost to complete the contract 
may be significantly greater, as DCMA has expressed its concern to DOD 
over Lockheed Martin's failure to regularly update its estimate of the 
costs to complete the JSF contract, stating that Lockheed's infrequent 
updates are insufficient to provide the government with information 
bearing on potential cost growth and funding needs. 

Like the CAIG, both DCMA and NAVAIR believe that Lockheed Martin's 
estimate at completion is too optimistic and that the program office 
will most likely require significantly more funding to complete the 
development program. NAVAIR provides resources to the JSF program 
office cost-estimating function, and it estimated in 2006 that JSF 
development costs could be almost $8 billion to $13 billion higher than 
estimated by the program office, or else cost billions more in 
procurement due to requirements pushed off from development. NAVAIR 
officials told us they believe that the 2006 estimate continues to be 
accurate today, but explained that since the JSF program is a joint 
program they do not control JSF cost-estimating procedures, although 
their estimates are briefed to JSF program management. The estimate 
removed what NAVAIR views as artificial constraints on the JSF schedule 
and projected forward, resulting in an estimate that the schedule would 
likely slip 19 to 27 months, and combined this with trends in cost 
performance. NAVAIR officials said that their confidence in the 
achievability of the JSF program schedule is low, as the master 
schedule comprises more than 600 individual schedules, making it 
difficult to accurately assess the achievability of the overall 
schedule. 

DCMA estimates that JSF development could cost as much as $4.9 billion 
more than program office estimates, accounting for poor cost and 
schedule performance to date and assuming further schedule slips of up 
to 12 months. DCMA confirmed that a schedule risk analysis, which uses 
statistical techniques to obtain a measure of confidence in completing 
a program, has never been performed on the JSF program. Since 
historically state-of-the-art development programs have taken longer 
than planned, a schedule risk analysis should be conducted to determine 
the level of uncertainty in the schedule. Despite these outside 
organizations' predictions of significantly higher costs to complete 
the JSF contract and the lack of realism in the contractor's own 
estimate, the JSF program office continues to use the contractor's 
estimate as its own. 

Table 9: NAVAIR and DCMA Estimates at Completion for JSF Aircraft 
Development Contract (Then-year dollars in billions): 

Lockheed Martin's estimate at completion (EAC): 
Estimates: $23.4; 
Projected overrun to the estimate: [Empty]. 

NAVAIR best case EAC projection: 
Estimates: $31.3; 
Projected overrun to the estimate: $7.9. 

NAVAIR worst case EAC projection: 
Estimates: $36.8; 
Projected overrun to the estimate: $13.4. 

DCMA EAC based on earned value data: 
Estimates: $24.8; 
Projected overrun to the estimate: [Empty]. 

DCMA additional costs for schedule slip: 
Estimates: $3.6; 
Projected overrun to the estimate: [Empty]. 

Total DCMA EAC (earned value + schedule slip): 
Estimates: $28.3; 
Projected overrun to the estimate: $4.9. 

Source: GAO analysis of DOD data. 

[End of table] 

In addition to expected cost overruns for JSF development, the CAIG is 
predicting significantly higher costs for JSF for the military services 
to purchase the aircraft. Using different assumptions about weight 
growth, labor rates, avionics and propulsion costs, and contractor 
fees, the CAIG calculated significantly higher unit costs for the 
aircraft variants (see table 6 earlier in this report for comparison of 
CAIG and program office assumptions affecting both development and 
procurement costs estimates). Multiplying these higher unit costs by 
the expected procurement quantities leads to a more than $33 billion 
(in constant year 2002 dollars) difference from official program office 
estimates for procurement costs.[Footnote 15] The CAIG estimates were 
briefed in 2006 to the DOD working group that oversees the JSF program, 
and top OSD officials were aware of the discrepancy between the CAIG 
and JSF program office estimates. 

The program office has not conducted an uncertainty analysis on its 
cost estimates despite the complexity of the program and associated 
risk and uncertainty. As shown in table 10, the JSF program is 
significantly more complicated than comparable aircraft development 
programs. 

Table 10: JSF Complexity Compared to That of Similar Aircraft Programs: 

Complexity factor: Program participation by multiple military services; 
F/A-18E/F: No; 
F/A-22: No; 
JSF: Yes. 

Complexity factor: Aircraft variants; 
F/A-18E/F: One; 
F/A-22: One; 
JSF: Three. 

Complexity factor: Avionics; 
F/A-18E/F: Off-the-shelf; 
F/A-22: New; 
JSF: New. 

Complexity factor: Stealth; 
F/A-18E/F: Minimal; 
F/A-22: Yes; 
JSF: Yes. 

Complexity factor: Software (operational flight program source lines of 
code); 
F/A-18E/F: 1.1 million; 
F/A-22: 2.2 million; 
JSF: 5.0 million[A]. 

Complexity factor: Engine(s); 
F/A-18E/F: One; 
F/A-22: One; 
JSF: Two cruise, one lift. 

Complexity factor: International participation; 
F/A-18E/F: No; 
F/A-22: No; 
JSF: Yes. 

Source: CAIG: 

[A] This assumes approximately 30 percent growth in lines of code by 
completion of development (F-22 included 34 percent growth and F/A-18, 
60 percent growth). 

[End of table] 

This complexity makes it all the more necessary to fully account for 
the effect various risks can have on the overall cost estimate. An 
uncertainty analysis assesses the extent to which the variability of an 
outcome variable is caused by uncertainty in the input parameters. It 
should be performed for every cost estimate in order to inform decision 
makers about the likelihood of success. In performing uncertainty 
analysis, an organization varies the effects of multiple elements on 
costs, and as a result, can express a level of confidence in the point 
estimate. 

We found that the JSF program has not conducted an uncertainty 
analysis. Such analysis would provide a range of possible values to 
program management and an estimate of the likelihood of the various 
possibilities. Instead, the program office only offers a single point 
estimate--one dollar figure, with no associated range--and no technical 
analysis of the likelihood that this estimate is credible. The lead 
cost estimator for the program office acknowledged that such a single 
point estimate is virtually certain to be wrong, but also stated that 
the analysis used to develop a range of values is easily manipulated 
and therefore not valuable. It is GAO's view that a point estimate 
should be accompanied by an estimated confidence level to quantify the 
uncertainty surrounding the estimate in order for management to make 
good decisions. Because the JSF program office has not conducted an 
uncertainty analysis, it is unable to provide Congress with any 
confidence level for its point estimate of approximately $300 billion 
for JSF acquisition. 

[End of section] 

Appendix III: Comments from the Department of Defense: 

Office Of The Under Secretary Of Defense: 
Acquisition, Technology And Logistics: 
3000 Defense Pentagon: 
Washington, DC 20301-3000: 

March 10, 2008: 

Mr. Michael J. Sullivan: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. Sullivan: 

This is the Department of Defense (DoD) response to the GAO draft 
report, "Joint Strike Fighter: Recent Decisions by DoD Add to Program 
Risks," dated February 5, 2008 (GAO Code 120663/GAO 08-388). 

The Department partially-concurs with the report's first two 
recommendations and concurs with the final two recommendations. The 
Department appreciates the GAO's highlighting of both program successes 
and challenges. Details of the Department's responses are contained in 
the enclosure. 

The Department values the opportunity to comment on the draft report 
and looks forward to continued discussions on the Joint Strike Fighter 
program with the GAO for next year's audit. 

Sincerely, 

Signed by: 

David G. Ahern: 
Director: 
Portfolio Systems Acquisition: 

Enclosure: As stated: 

GAO Draft Report Dated February 5, 2008: 
GAO-08-388 (GAO Code 120663): 

"Joint Strike Fighter: Recent Decisions By Dod Add To Program Risks" 

Department Of Defense Comments To The GAO Recommendations: 

Recommendation 1: The GAO recommends that the Secretary of Defense 
direct elements of the Department to revisit and, if appropriate, 
revise the Mid-Course Risk Reduction plan recently approved. (Page 31 
/GAO Draft Report) 

DOD Response: Partially-Concur. The Department continues to believe the 
Mid-Course Risk Reduction (MCRR) plan is a cost effective approach with 
a manageable level of risk. USD(AT&L) approved the plan in October 2007 
after thorough review of the benefits and risks. Implementation of the 
plan is well underway. The Department developed metrics to monitor and 
evaluate contractor Management Reserve replenishment and use, 
manufacturing line progress, and the MCRR impacts on developmental 
testing. The plan has always been to revisit and, if necessary, revise 
MCRR implementation if metrics/monitoring fail to achieve expectations. 
The Department does not plan to alter its current plan of monitoring 
MCRR execution. 

Recommendation 2: The GAO recommends that the Secretary of Defense 
direct that the Joint Strike Fighter (JSF) program office update its 
cost estimate using best practices, so that the estimate is 
comprehensive, accurate, well-documented, and credible. Specifically, 
the JSF program office should: 

(a) Include costs that were inappropriately omitted from the estimate; 

(b) Identify performance requirements that have been traded off in 
development; 

(c) Fully document assumptions, data sources, and methodologies in the 
cost model; and 

(d) Perform a risk and uncertainty analysis to focus on key cost 
drivers and reduce the risk of cost overruns. (Page 31/GAO Draft 
Report) 

DOD Response: Partially-Concur. The F-35 Program Office is completing a 
comprehensive cost estimate in preparation of an independent assessment 
directed by the Department of Defense, as indicated in the response to 
Recommendation 4. The Department believes that a Risk and Uncertainty 
Analysis as defined in GAO-07-1134SP Cost Assessment Guide is 
unwarranted due to the subjective nature of the modeling inputs. 

Recommendation 3: The GAO recommends that the Secretary of Defense 
direct that the program conduct a full Schedule Risk Analysis to ensure 
that its schedules are fully understood, manageable, and executable. 
(Page 31/GAO Draft Report) 

DOD Response: Concur. A combined Navy, Air Force, and OSD team will 
consider schedule risk in a planned joint cost estimate update, 
discussed in more detail in the response to Recommendation 4. 

Recommendation 4: The GAO recommends that the Secretary of Defense 
direct that a full, independent cost estimate be conducted according to 
the highest standards of any DoD cost estimating organization, based on 
a comprehensive review of program data; that this cost estimate be 
reviewed by an independent third party such as the Cost Analysis 
Improvement Group; and that the results of these estimates be briefed 
to all interested parties in DoD and the Congress. (Page 31/GAO Draft 
Report) 

DOD Response: Concur. In January 2008, the Department directed an 
independent joint cost assessment to be performed by a team of Navy, 
Air Force, and OSD cost experts. This joint cost assessment was 
initiated at the request of the F-35 Program Executive Office (PEO) and 
will aid the Department's development of the FY 2010 President's Budget 
request. 

The Department is pleased that the GAO endorses a path the PEO and OSD 
have already developed, i.e., a joint independent F-35 cost review. The 
Air Force Cost Analysis Agency (AFCAA), NAVAIR 4.2, and OSD Cost 
Analysis Improvement Group (CAIG) are participating in the review. We 
appreciate the fact that GAO recognizes the complexities involved in 
estimating F-35 costs. 

Factual Discrepancies: There are three significant discrepancies noted 
within the report. The March 2004 re-plan added $7.5 billion to the 
development estimate, not "over 10 billion" dollars as indicated in the 
report. Additionally, the December 2006 cost estimate reflected an 
increase in the Military Construction (MILCON), not a reduction as GAO 
states. The past six annual Selected Acquisition Reports assert that 
MILCON estimates will continue to be updated as the Services identify 
specific site requirements. Lastly, GAO cites a total procurement cost 
increase of 28% since March 2004 but omits the fact that the increase 
is less than half of that (i.e., 13%) when inflation impacts are 
excluded. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Michael Sullivan (202) 512-4841 or [email protected]: 

Staff Acknowledgments: 

Bruce Fairbairn, Assistant Director; Jerry Clark; Marvin Bonner; Simon 
Hirschfeld; Matt Lea; Karen Richey; Dave Hart; and Jim York: 

[End of section] 

Related GAO Products: 

Joint Strike Fighter: Impact of Recent Decisions on Program Risks. GAO-
08-569T. Washington, D.C.: March 11, 2008: 

Tactical Aircraft: DOD Needs a Joint and Integrated Strategy. GAO-07-
415. Washington, D.C.: April 2, 2007. 

Defense Acquisitions: Assessments of Selected Major Weapon Programs. 
GAO-07-406SP. Washington D.C.: March 30, 2007. 

Best Practices: An Integrated Portfolio Management Approach to Weapon 
System Investments Could Improve DOD's Acquisition Outcomes. GAO-07-
388. Washington, D.C.: March 30, 2007. 

Defense Acquisitions: Analysis of Costs for the Joint Strike Fighter 
Engine Program. GAO-07-656T. Washington, D.C.: March 22, 2007. 

Joint Strike Fighter: Progress Made and Challenges Remain. GAO-07-360. 
Washington, D.C.: March 15, 2007. 

Systems Acquisition: Major Weapon Systems Continue to Experience Cost 
and Schedule Problems under DOD's Revised Policy. GAO-06-368. 
Washington, D.C.: April 13, 2006. 

Defense Acquisitions: Actions Needed to Get Better Results on Weapon 
Systems Investments. GAO-06-585T. Washington, D.C.: April 5, 2006. 

Tactical Aircraft: Recapitalization Goals Are Not Supported by 
Knowledge-Based F-22A and JSF Business Cases. GAO-06-487T. Washington, 
D.C.: March 16, 2006. 

Joint Strike Fighter: DOD Plans to Enter Production before Testing 
Demonstrates Acceptable Performance. GAO-06-356. Washington, D.C.: 
March 15, 2006. 

Tactical Aircraft: F/A-22 and JSF Acquisition Plans and Implications 
for Tactical Aircraft Modernization. GAO-05-519T. Washington, D.C.: 
April 6, 2005. 

Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike 
Fighter Program with Different Acquisition Strategy. GAO-05-271. 
Washington, D.C.: March 15, 2005. 

[End of section] 

Footnotes: 

[1] Pub. L. No. 108-375, ï¿½ 213 (2004). 

[2] GAO, Joint Strike Fighter: Progress Made and Challenges Remain, 
GAO-07-360 (Washington, D.C.: Mar. 15, 2007); Joint Strike Fighter: DOD 
Plans to Enter Production before Testing Demonstrates Acceptable 
Performance, GAO-06-356 (Washington, D.C.: Mar. 15, 2006); and Tactical 
Aircraft: Opportunity to Reduce Risks in the Joint Strike Fighter 
Program with Different Acquisition Strategy, GAO-05-271 (Washington, 
D.C.: Mar. 15, 2005). 

[3] The international partners are the United Kingdom, Italy, the 
Netherlands, Turkey, Canada, Australia, Denmark, and Norway. These 
nations are contributing funds for system development and have signed 
agreements to procure a minimum of 646 aircraft. Israel and Singapore 
are security cooperation participants, and several other nations have 
reportedly expressed interest in acquiring aircraft. 

[4] To eliminate the effects of inflation, these procurement cost 
increases expressed in base year fiscal year 2002 dollars are $7.8 
billion (plus 5 percent) and $19.6 billion (plus 13 percent), 
respectively. 

[5] Congress has subsequently required that DOD obligate and expend 
sufficient annual amounts for the continued development and procurement 
of the alternate engine program. National Defense Authorization Act for 
Fiscal Year 2008, Pub. L. No. 110-181, ï¿½ 213 (2008). 

[6] An efficient production line establishes an orderly flow of work as 
a product moves from workstation to workstation and on to final 
assembly. Out-of-station work, sometimes referred to as traveled work, 
refers to completing unfinished work on major components, e.g., the 
wings, after they have left the wing workstation and moved down the 
production line to another station. 

[7] For an overview of the best practices methodologies and how current 
defense programs fared, see our report of last year on major 
acquisitions, including the JSF, in GAO, Defense Acquisitions: 
Assessments of Selected Weapon Programs, GAO-07-406SP (Washington, 
D.C.: Mar. 30, 2007). 

[8] GAO-05-271. 

[9] The CAIG serves as the principal advisory body to the milestone 
decision authority on all matters concerning an acquisition program's 
life-cycle cost, and is given general responsibilities for establishing 
DOD policy guidance on a number of matters relating to cost estimating. 
The independent CAIG cost estimate is designed to assess the program 
office estimate and ensure realistic cost estimates are considered. 

[10] Earned value management is a method of tracking and measuring the 
value of work accomplished in a given period and comparing it with the 
planned value of work scheduled and the actual cost of work 
accomplished. Its use is required by federal regulations. 

[11] GAO, Tactical Aircraft: DOD Needs a Joint and Integrated 
Investment Strategy, GAO-07-415 (Washington, D.C.: Apr. 2, 2007). 

[12] The CAIG serves as the principal advisory body to the milestone 
decision authority on all matters concerning an acquisition program's 
life-cycle cost, and is given general responsibilities for establishing 
DOD policy guidance on a number of matters relating to cost estimating. 
The independent CAIG cost estimate is designed to assess the program 
office estimate and ensure realistic cost estimates are considered. 

[13] These four best practice criteria for a reliable, high-quality 
cost estimate can be mapped to 12 steps of a high-quality cost 
estimating process that have been identified by GAO in the Cost 
Assessment Guide. Chapter 1 of the guide describes the 12 steps and how 
they map to the four best practice characteristics. 

[14] GAO, Unmanned Aircraft Systems: Global Hawk Cost Increase 
Understated in Nunn-McCurdy Report, GAO-06-222R (Washington, D.C.: Dec. 
15, 2005). 

[15] We expressed the costs in constant 2002 dollars instead of then-
year dollars because the data we relied on to make this projection were 
in constant dollars, and we did not have the quantity profiles by year 
to inflate the costs. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***