Defense Acquisitions: A Knowledge-Based Funding Approach Could	 
Improve Major Weapon System Program Outcomes (02-JUL-08,	 
GAO-08-619).							 
                                                                 
The Department of Defense (DOD) expects the cost to develop and  
procure the major weapon systems in its current portfolio to	 
total $1.6 trillion. With increased competition for funding	 
within DOD and across the federal government, effectively	 
managing these acquisitions is critical. Yet DOD programs too	 
often experience poor outcomes--like increased costs and delayed 
fielding of needed capabilities to the warfighter. In 2006, this 
Committee mandated that GAO report on DOD's processes for	 
identifying needs and allocating resources for its weapon system 
programs. In 2007, GAO reported that DOD consistently commits to 
more programs than it can support. This follow-on report assesses
DOD's funding approach, identifies key factors that influence the
effectiveness of this approach, and identifies practices that	 
could help improve DOD's approach. To conduct its work, GAO	 
assessed 20 major weapon programs in DOD's current portfolio--5  
in detail--and reviewed relevant DOD policy and guidance, prior  
GAO work, and other relevant literature. GAO also reviewed the	 
practices of selected successful companies.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-619 					        
    ACCNO:   A82689						        
  TITLE:     Defense Acquisitions: A Knowledge-Based Funding Approach 
Could Improve Major Weapon System Program Outcomes		 
     DATE:   07/02/2008 
  SUBJECT:   Cost analysis					 
	     Cost overruns					 
	     Defense budgets					 
	     Defense capabilities				 
	     Defense cost control				 
	     Defense industry					 
	     Defense operations 				 
	     Defense procurement				 
	     Federal funds					 
	     Financial management				 
	     Funds management					 
	     Future budget projections				 
	     Military research and development			 
	     Military technology				 
	     Policy evaluation					 
	     Procurement planning				 
	     Procurement policy 				 
	     Program management 				 
	     Reporting requirements				 
	     Research and development costs			 
	     Strategic planning 				 
	     Systems analysis					 
	     Systems integration				 
	     Systems management 				 
	     Technology 					 
	     Weapons systems					 
	     Cost awareness					 
	     Cost estimates					 
	     Cost growth					 
	     Program costs					 
	     Future Years Defense Program			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-619

   

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to [email protected]. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Committee on Armed Services, U.S. Senate: 

United States Government Accountability Office: 
GAO: 

June 2008: 

Defense Acquisitions: 

A Knowledge-Based Funding Approach Could Improve Major Weapon System 
Program Outcomes: 

Defense Acquisitions: 

GAO-08-619: 

GAO Highlights: 

Highlights of GAO-08-619, a report to the Committee on Armed Services, 
U.S. Senate. 

Why GAO Did This Study: 

The Department of Defense (DOD) expects the cost to develop and procure 
the major weapon systems in its current portfolio to total $1.6 
trillion. With increased competition for funding within DOD and across 
the federal government, effectively managing these acquisitions is 
critical. Yet DOD programs too often experience poor outcomesï¿½like 
increased costs and delayed fielding of needed capabilities to the 
warfighter. 

In 2006, this Committee mandated that GAO report on DODï¿½s processes for 
identifying needs and allocating resources for its weapon system 
programs. In 2007, GAO reported that DOD consistently commits to more 
programs than it can support. This follow-on report assesses DODï¿½s 
funding approach, identifies key factors that influence the 
effectiveness of this approach, and identifies practices that could 
help improve DODï¿½s approach. 

To conduct its work, GAO assessed 20 major weapon programs in DODï¿½s 
current portfolioï¿½5 in detailï¿½and reviewed relevant DOD policy and 
guidance, prior GAO work, and other relevant literature. GAO also 
reviewed the practices of selected successful companies. 

What GAO Found: 

DOD often does not commit full funding to develop its major weapon 
systems when they are initiated, despite the departmentï¿½s policy to do 
so. For a majority of the weapon system programs GAO reviewed, costs 
have exceeded the funding levels initially planned for and reflected in 
the Future Years Defense Program (FYDP)ï¿½DODï¿½s investment strategy. To 
compensate for these shortfalls, DOD makes unplanned and inefficient 
funding adjustments, like moving money from one program to another, 
deferring costs into the future, or reducing procurement quantities. 

Figure: Funding Shortfalls at the Start of Development for Five Major 
Weapon System Programs: 

Program: Multi-mission Maritime Aircraft; 
Level of funding established in the FYDP in the year the program was 
initiated: 32%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 35%; 
Funding required beyond the initial FYDP to complete development: 33%; 
Total: 100%. 

Program: Warfighter Information Network-Tactical; 
Level of funding established in the FYDP in the year the program was 
initiated: 21%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 49%; 
Funding required beyond the initial FYDP to complete development: 30%; 

Total: 100%. 

Program: Future Combat System; 
Level of funding established in the FYDP in the year the program was 
initiated: 26%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 41%; 
Funding required beyond the initial FYDP to complete development: 33%; 
Total: 100%. 

Program: Joint Strike Fighter; 
Level of funding established in the FYDP in the year the program was 
initiated: 64%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 13%; 
Funding required beyond the initial FYDP to complete development: 23%; 
Total: 100%. 

Program: Global Hawk; 
Level of funding established in the FYDP in the year the program was 
initiated: 31%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 48%; 
Funding required beyond the initial FYDP to complete development: 
Total: 21%. 

Source: DOD (data). GAO (analysis and presentation). 

[End of figure] 

DODï¿½s flawed funding process is largely driven by decision makersï¿½ 
willingness to accept unrealistic cost estimates and DODï¿½s commitment 
to more programs than it can support. DOD often underestimates 
development costsï¿½due in part to a lack of knowledge and optimistic 
assumptions about requirements and critical technologies. At the same 
time, DODï¿½s continued failure to balance its needs with available 
resources promotes unhealthy competition among programs for funding. 
This creates incentives for service and program officials to establish 
requirements that make their particular weapon systems stand out, with 
less consideration of the resources needed to develop them. Ultimately, 
DOD tends to push the need for funding to the future rather than limit 
program length or adjust requirements. 

The successful commercial companies that GAO has previously reviewed 
achieve adequate and stable funding for product development programs by 
following a disciplined, knowledge-based approach to estimating program 
costs; using manageable development cycles to increase the 
predictability of funding needs and the likelihood of program success; 
and using portfolio management practices to make decisions about which 
programs to pursue. Once programs are approved, these companies firmly 
commit to fully fund them.
 
What GAO Recommends: 

GAO is making three recommendations aimed at increasing funding 
stability and improving acquisition outcomes. DOD believes that current 
policies and initiatives sufficiently address the first two 
recommendations, and did not concur with the third. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-619]. For more 
information, contact Michael J. Sullivan at (202) 512-4841 or 
[email protected]. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Failure to Commit Full Funding to Weapon Systems Contributes to Poor 
Acquisition Outcomes: 

Unrealistic Cost Estimates and a Failure to Balance Needs with 
Available Resources Underlie DOD's Flawed Funding Approach: 

Proven Practices Help Ensure Accurate Cost Estimates and Adequate 
Program Funding: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Defense: 

Related GAO Products: 

Table: 

Table 1: Development Cost Estimates and Baselines for 20 Major Weapon 
System Programs: 

Figures: 

Figure 1: Simplified View of PPBE Process: 

Figure 2: Shortfalls at the Start of Development for Five Major Weapon 
System Programs: 

Figure 3: Costs Remaining Versus Annual Appropriations for Major 
Defense Acquisitions: 

Figure 4: Notional Comparison of Cost Estimating Uncertainty and Levels 
of Knowledge at Program Start: 

Figure 5: Range of Possible Costs Narrows as Knowledge Is Gained: 

Abbreviations: 

AEHF: Advanced Extremely High Frequency Satellite: 

CAIG: Cost Analysis Improvement Group: 

CARD: Cost Analysis Requirements Description: 

DAS: Defense Acquisition System: 

DOD: Department of Defense: 

EFV: Expeditionary Fighting Vehicle: 

FCS: Future Combat System: 

FYDP: Future Years Defense Program: 

IDA: Institute for Defense Analysis: 

JCIDS: Joint Capabilities Integration and Development System: 

JSF: Joint Strike Fighter: 

JTRS: Joint Tactical Radio System: 

MDA: Milestone Decision Authority: 

MMA: Multi-mission Maritime Aircraft: 

OMB: Office of Management and Budget: 

OSD: Office of the Secretary of Defense: 

PPBE: Planning, Programming, Budgeting, and Execution: 

SAR: Selected Acquisition Reports: 

SIBRS: Space-Based Infrared System-High: 

WIN-T: Warfighter Information Network-Tactical: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

July 2, 2008: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Department of Defense (DOD) expects the cost to develop and procure 
the major weapon systems in its current portfolio to total $1.6 
trillion, $335 billion of which is expected to be spent over the next 5 
years. Effective management of the costs of these acquisitions is 
critical given the increased competition for funds within the 
department to support ongoing military operations in Afghanistan and 
Iraq, as well as growing pressures to reduce overall DOD spending due 
to the long-term fiscal imbalances facing the federal government. 
However, many of DOD's major weapon system development programs have 
experienced poor outcomes--cost increases that add up to hundreds of 
millions of dollars, schedule delays that add up to years, and 
capabilities that fall short of what was promised.

How DOD manages its weapon system investments has been a matter of 
congressional concern for many years. In fiscal year 2006, the Senate 
Armed Services Committee raised concerns about DOD's poor track record 
with acquisition programs and directed GAO to assess how DOD's 
processes and practices for identifying requirements and allocating 
resources affect the department's weapon system acquisition programs. 
In March 2007, we reported that DOD lacks an effective, integrated 
portfolio management approach that takes into account all of the 
department's major weapon system programs and that requires tough 
decisions commensurate with available resources.[Footnote 1] In short, 
we noted that DOD commits to more programs than it can support. This 
report, also done in response to the same Senate mandate, focuses on 
DOD's funding process and its impact on major acquisitions. 
Specifically, the report (1) assesses how DOD budgets for and funds its 
major weapons system acquisition programs, (2) identifies key factors 
that influence the effectiveness of this approach, and (3) identifies 
proven processes and practices that could help improve DOD's ability to 
effectively allocate resources to its acquisition programs. GAO also 
has ongoing related work specifically assessing DOD's process for 
identifying and prioritizing warfighting capability requirements. 

To assess DOD's funding process and to identify key factors that 
influence the effectiveness of that approach, we reviewed relevant DOD 
policy guidance, legislation, and academic literature, and assessed 
cost estimates and budget data for 20 of the 95 major weapons programs 
in DOD's current portfolio--which represent more than one-third of the 
total expected cost of DOD's current portfolio of major weapon system 
programs. To gain further insights into the impact of DOD's funding 
process on individual programs, we conducted more detailed analysis for 
five of these programs: Global Hawk, Joint Strike Fighter (JSF), Future 
Combat System (FCS), Warfighter Information Network-Tactical (WIN-T), 
and Multi-mission Maritime Aircraft (MMA).[Footnote 2] In addition we 
interviewed numerous officials from the Office of the Secretary of 
Defense (OSD) as well as military service cost analysis, budgeting, and 
acquisition offices. To identify proven cost estimating and budgeting 
processes and practices that could be used by DOD to improve its 
resource allocation process, we utilized information from our March 
2007 best practices report, and conducted follow-up interviews with 
officials from three of the five companies that provided input to that 
report--Eli Lilly, IBM, and Motorola. We also relied on our Cost 
Assessment Guide, which provides a cost-estimating methodology based on 
best practices.[Footnote 3] We conducted this performance audit from 
June 2007 to May 2008 in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a reasonable 
basis for our findings and conclusions based on our audit objectives. 

Results in Brief: 

DOD often does not commit full funding to its major weapon system 
acquisition programs, despite the department's policy to "fully fund" 
programs at the start of system development.[Footnote 4] Of the 
programs we reviewed, over 75 percent were not fully funded in the 
Future Years Defense Program (FYDP)--DOD's investment strategy. We 
found that because programs typically had development cycles that 
extended beyond the FYDP time frame, the FYDP did not capture their 
full funding needs. At the same time, program costs exceeded the 
funding levels initially planned for and reflected in the years covered 
by the FYDP. To compensate for funding shortfalls, DOD often makes 
unplanned and inefficient funding adjustments, such as moving money 
between programs, deferring work and associated costs into the future, 
or reducing procurement quantities. Ultimately, such reactive practices 
obscure true program costs and contribute to the instability of many 
programs and poor acquisition outcomes. 

DOD's inability to allocate funding effectively to programs is largely 
driven by the acceptance of unrealistic cost estimates and a failure to 
balance needs based on available resources. Development costs for major 
acquisition programs are often underestimated at program initiation--30 
to 40 percent in some cases--in large part because the estimates are 
based on limited knowledge and optimistic assumptions about system 
requirements and critical technologies. For example, initial 
development cost estimates for the Army's WIN-T communications system 
were understated by at least $1.3 billion, or nearly 160 percent, in 
part because the estimates assumed that commercial-off-the-shelf radio 
technology would be available. This assumption proved to be wrong. 
Similarly, JSF's development costs were underestimated by at least $7.1 
billion, or around 20 percent, because the initial estimate assumed 
certain efficiencies that never materialized. These unrealistic cost 
estimates are developed in an environment where DOD commits to more 
programs than available resources can support, which promotes unhealthy 
competition among programs for funding. This competition creates strong 
incentives for program officials to establish requirements that make 
their particular weapon systems stand out from others, with less 
consideration given to the resources that will be needed to develop 
them. Ultimately, programs tend to push the need for funding to the 
future rather than limit program length or adjust requirements. 

Our past work on commercial best practices found that successful 
companies were able to fully fund their product development programs 
because they required programs to have realistic, knowledge-based cost 
estimates; they committed to manageable product increments; and they 
prioritized programs within resource constraints. To develop realistic 
cost estimates, these companies followed a disciplined process through 
which requirements were assessed against available resources at 
multiple gated reviews prior to committing to a new product. As more 
knowledge was gained and risk was reduced, costs estimates more 
accurately reflected true costs. Before initiating product development, 
for example, Motorola and IBM expected that actual costs would not 
exceed the latest estimate by more than 5 or 10 percent. To increase 
the predictability of funding needs--and funding stability--these 
companies expected development cycles to be manageable (2 to 5 years). 
Likewise, a study commissioned by DOD in 2006 recommended that programs 
should be time-constrained with development cycles no longer than 6 
years, and DOD is taking steps to pilot this concept. Finally, 
companies used a portfolio management approach to prioritize 
investments and allocate resources. Such an approach requires tough 
decisions about approving or terminating programs. Once programs were 
approved, these companies made a firm commitment to fully fund them. 
Ultimately, these practices helped companies avoid committing to more 
programs than they could afford and allowed them to optimize the return 
on their investments. 

To more effectively fund major weapon system acquisition programs and 
achieve successful outcomes, we are recommending that the Secretary of 
Defense develop and implement a strategy to bring DOD's current weapon 
systems portfolio into balance by aligning the number of systems with 
available resources in the FYDP. In addition, the Secretary should 
require new programs to have manageable development cycles, realistic 
cost estimates, and to have planned and programmed full funding for the 
entire development cycle. Finally, the Secretary should require all 
cost estimates submitted for funding a program at milestone decisions 
to be reported as a range of likely costs and reflect the associated 
levels of risk and uncertainty. DOD partially concurred with our first 
two recommendations, but believes that they are being sufficiently 
addressed through current acquisition policies and ongoing initiatives. 
While we agree that if implemented appropriately, DOD's current 
policies and initiatives have the potential to contribute to better 
program outcomes, we have found no evidence of widespread adoption of a 
knowledge-based process to better ensure adequate funding for programs. 
DOD agreed with the spirit of our third recommendation--to consider 
risks at key milestones--however, it did not agree that all cost 
estimates submitted for funding a program at milestone decisions should 
be reported as a range of likely costs. In its written comments, DOD 
stated that a certain method used to calculate cost ranges can produce 
misleading results. While we do not advocate a specific method for 
calculating a range of potential costs, we maintain that presenting 
such a range would provide decision makers with additional knowledge 
about the level of risk in a proposed program. 

Background: 

To plan, execute, and fund its weapon system acquisition programs, DOD 
relies on three principal decision-making systems: the Joint 
Capabilities Integration and Development System (JCIDS), which is used 
to assess gaps in warfighting capabilities and recommend solutions to 
resolve those gaps; the Defense Acquisition System (DAS), which is used 
to manage the development and procurement of weapon systems and other 
equipment; and the Planning, Programming, Budgeting, and Execution 
(PPBE) process, which is used to allocate resources. While the JCIDS 
and DAS processes are driven by specific events--such as validating 
requirements or receiving approval to start development--the PPBE 
process is calendar driven, taking nearly 2 years to go from planning 
to the beginning of budget execution. 

The PPBE process is intended to provide a framework within which DOD 
can articulate its strategy; identify force size, structure, and needed 
equipment; set program priorities; allocate resources to individual 
programs; and assess program performance. Although the different phases 
of PPBE are considered sequential, because of the amount of time 
required to develop and review resource requirements, the process is 
continuous and concurrent with at least two phases ongoing at any given 
time (see fig. 1). 

Figure 1: Simplified View of PPBE Process: 

[See PDF for image] 

This figure is an illustration of a simplified view of PPBE Process, as 
follows: 

2007 PPBE: 
Planning: Late 2004 to mid-2005; 
Programming and budgeting: Mid-2005 to late 2006; 
Execution (1 to 3 years depending on funding type): Late 2006 to 2008 
and beyond. 

2008 PPBE: 
Planning: Late 2005 to mid-2006; 
Programming and budgeting: Mid-2006 to late 2007; 
Execution (1 to 3 years depending on funding type): Late 2007 to 2009 
and beyond. 

2009 PPBE: 
Planning: Late 2006 to mid-2007; 
Programming and budgeting: Mid-2007 to late 2008; 
Execution (1 to 3 years depending on funding type): Late 2008 to 2010 
and beyond. 

2010 PPBE: 
Planning: Late 2007 to mid-2008; 
Programming and budgeting: Mid-2008 to late 2009; 
Execution (1 to 3 years depending on funding type): Late 2009 to 2011 
and beyond. 

2011 PPBE: 
Planning: Late 2008 to mid-2009; 
Programming and budgeting: Mid-2009 to late 2010; 
Execution (1 to 3 years depending on funding type): Late 2010 to 2012 
and beyond. 

Source: DOD (data). GAO (analysis and presentation). 

[End of figure] 

At the front end of the PPBE process, the Secretary of Defense provides 
planning guidance to the military services and defense agencies about 
the capabilities and resources required to deter and defeat threats. 
The services and agencies, in turn, develop individual "programs" 
(budgets) based on the planning guidance as well as fiscal guidance 
provided by the Office of Management and Budget (OMB). The service and 
agency program budgets are then subjected to a series of leadership 
reviews in the department--OSD, Director of Program Analysis and 
Evaluation, Joint Chiefs of Staff, and the Comptroller--and adjustments 
are made if necessary. Responsibility for managing the PPBE process and 
ensuring the budget is prepared and submitted to Congress resides with 
the OSD Comptroller. 

The PPBE process produces the defense portion of the President's annual 
budget request to Congress as well as the FYDP--DOD's longer-term 
investment strategy--which includes the department's resource 
allocations for the 4 to 5 fiscal years beyond the budget.[Footnote 5] 
The ultimate objective of the FYDP is to manage funds in a way that 
provides combatant commanders with the best mix of forces, equipment, 
and support attainable within fiscal constraints established by OMB. 
Once complete, the FYDP is expected to provide an aggregate picture of 
the anticipated force levels and funding needs of individual programs. 
Data in the FYDP tell the Secretary of Defense, the President, 
Congress, and the American people what the department expects to invest 
in and how much it will spend over time. 

DOD policy requires that the dollars and manpower needed to carry out 
the first 5-to 6-years of a weapon system acquisition program must be 
included in the FYDP. The value of the FYDP greatly depends on the 
accuracy of the cost estimates supporting each individual program. 
Having a realistic cost estimate provides a basis for accurate 
budgeting and effective resource allocation, increasing the probability 
of a program's success in meeting its targets. Two cost estimates are 
required before a program is approved to start system development 
(Milestone B) and at the beginning of production (Milestone C)--the 
service or program office estimate and an independent cost estimate. 
The independent cost estimate for most major acquisition programs is 
developed by the Cost Analysis Improvement Group (CAIG). At Milestone 
B, the Milestone Decision Authority (MDA)[Footnote 6] uses the two 
estimates to determine the program's official cost baseline and whether 
the funding reflected in the FYDP is adequate to cover the portion of 
the estimated costs of the program that fall within the FYDP time 
period.[Footnote 7] If not, the services are expected to adjust the 
FYDP to ensure that their programs are fully funded to the approved 
baseline. To receive Milestone B approval, DOD is required to certify 
to Congress[Footnote 8] that funding is available to support the 
portion of the program that falls within the period covered by the FYDP 
that is submitted during the fiscal year in which the certification is 
made.[Footnote 9] 

Our past work has consistently shown that DOD's major weapon system 
programs do not meet their cost and schedule targets. Since the mid- 
1990s, we have studied leading commercial companies in order to 
identify best practices for developing and producing new products. 
Taking into account the differences between commercial product 
development and weapons acquisitions, we articulated a best practices 
model that relies on increasing knowledge when developing new products, 
separating technology development from product development, and 
following an evolutionary or incremental approach to product 
development.[Footnote 10] This knowledge-based approach requires 
developers to make investment decisions on the basis of specific, 
measurable levels of knowledge at critical junctures before investing 
more money and before advancing to the next phase of acquisition. An 
evolutionary product development process defines the individual 
increments on the basis of mature technologies and a feasible design 
that are matched with firm requirements. The knowledge-based, 
evolutionary approach in our model is intended to help reduce 
development risks and to achieve better program outcomes on a more 
consistent basis. 

In October 2000, DOD began to significantly revise the department's 
acquisition policy, adopting a knowledge-based, evolutionary product 
development approach. DOD's revised policy emphasizes the importance of 
and provides a good framework for capturing knowledge about critical 
technologies, product design, and manufacturing processes. In recent 
years, we have reported that if properly implemented and enforced, 
DOD's acquisition policy could reduce technical risk at the start of a 
program and make cost and delivery estimates much more predictable. 

Failure to Commit Full Funding to Weapon Systems Contributes to Poor 
Acquisition Outcomes: 

DOD often does not commit full funding to its major weapon system 
acquisitions when they are initiated, despite the department's policy 
to do so. For a majority of the programs we reviewed, costs exceeded 
the funding levels initially planned for and reflected in the FYDP. To 
make up for these funding shortfalls, DOD often shifts funds from one 
program to pay for another, reduces system capabilities, cuts 
procurement quantities, extends development and procurement schedules, 
or in rare cases terminates programs. Such actions not only create 
instability in DOD's weapon system portfolio, they also obscure the 
true future costs of current commitments, making it difficult to make 
informed investment decisions. 

At the beginning of system development, the approved cost estimates for 
most of the major acquisition programs we reviewed have exceeded the 
funding initially established in the FYDP. First, our analysis 
indicates that 14 of the 20 programs had development cycles that 
extended beyond the FYDP. As a result, the FYDP could not capture all 
of the estimated funding needed to complete development. Second, the 
funding established in the FYDP at the beginning of system development 
for 15 of the 20 programs was less than what they required for that 
time period.[Footnote 11] For example, at the start of system 
development in 2003, the initial approved cost estimate for the 
development portion of the FCS program was about $20 billion, of which 
$13 billion was needed within the 2003 FYDP time frame--that is for 
fiscal years 2003 through 2007. However, at that time the 2003 FYDP 
only contained $5 billion for FCS development. This means that the 
department had only committed to fund 25 percent of FCS's estimated 
development cost when the program started, and only 39 percent of the 
funding needed in its first 5 years. Other programs we reviewed 
experienced similar shortfalls (see fig. 2). 

Figure 2: Shortfalls at the Start of Development for Five Major Weapon 
System Programs: 

[See PDF for image] 

Program: Multi-mission Maritime Aircraft; 
Level of funding established in the FYDP in the year the program was 
initiated: 32%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 35%; 
Funding required beyond the initial FYDP to complete development: 33%; 
Total: 100%. 

Program: Warfighter Information Network-Tactical; 
Level of funding established in the FYDP in the year the program was 
initiated: 21%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 49%; 
Funding required beyond the initial FYDP to complete development: 30%; 
Total: 100%. 

Program: Future Combat System; 
Level of funding established in the FYDP in the year the program was 
initiated: 26%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 41%; 
Funding required beyond the initial FYDP to complete development: 33%; 
Total: 100%. 

Program: Joint Strike Fighter; 
Level of funding established in the FYDP in the year the program was 
initiated: 64%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 13%; 
Funding required beyond the initial FYDP to complete development: 23%; 
Total: 100%. 

Program: Global Hawk; 
Level of funding established in the FYDP in the year the program was 
initiated: 31%; 
Level of funding the program needed to be fully funded in the initial 
FYDP: 48%; 
Funding required beyond the initial FYDP to complete development: 
Total: 21%. 

Source: DOD (data). GAO (analysis and presentation). 

[End of figure] 

To bring funding commitments more in line with original program 
baselines, DOD typically makes adjustments in subsequent FYDPs. By that 
time, however, program cost baselines have often increased--as more 
knowledge about technologies, design, and manufacturing needs is 
gained--creating new funding shortfalls. For example, by the third year 
of the WIN-T program, the program's cost estimate, and therefore 
funding needs, had increased 72 percent because requirements were 
unstable and critical technologies were not ready, creating a 26 
percent shortfall in the FYDP. Similarly, a 57 percent increase in 
FCS's cost estimate created an additional 12 percent--or $2 billion-- 
funding shortfall. 

In reaction to funding shortfalls in individual programs, DOD often 
moves funds from other programs,[Footnote 12] reduces capabilities by 
scaling back requirements, cuts procurement quantities, or extends 
development schedules. Although rare, programs have also been 
terminated. Such reactive measures destabilize programs and in many 
cases increase program costs. For example, as we previously reported, 
funding for the Air Force's Space-Based Infrared System-High (SIBRS) 
satellite system was cut in 1998 and 1999 to pay for higher budget 
priorities, which contributed to a 2-year delay in the program and a 
breach of the cost baseline. As a result, the Air Force changed SIBRS's 
procurement strategy, which independent cost estimators calculated 
would double program costs.[Footnote 13] For the F-22A Raptor program, 
our past work has noted that the Air Force drastically reduced its 
planned buys from 648 to 183 as program costs escalated. Similarly, the 
number of requirements for the Joint Tactical Radio System (JTRS) was 
reduced or deferred by about one-third as the program encountered 
development problems. This change had a reverberating effect on several 
JTRS-dependent efforts--such as the Army's modernization of radios for 
its helicopters--as those programs had to make adjustments and go 
forward with alternative, less capable solutions.[Footnote 14] Making 
these types of adjustments to compensate for inadequate funding 
obscures future resource requirements and limits the Congress's 
visibility over DOD's true funding needs. 

Unrealistic Cost Estimates and a Failure to Balance Needs with 
Available Resources Underlie DOD's Flawed Funding Approach: 

For most of the 20 major acquisition programs that we reviewed, initial 
funding was based on cost estimates that proved to be too low. DOD's 
funding approach is further compromised by the department's failure to 
balance requirements with available resources. Because DOD commits to 
more programs than resources can support, programs often have to 
compete for funding by overpromising capabilities and by providing low 
cost estimates as inputs to the funding process. We have previously 
reported that when such trends go unchecked, Congress is consistently 
faced with a difficult choice: pull funds from other federal programs 
to support DOD's acquisitions or accept less warfighting capability 
than promised.[Footnote 15] 

Unrealistic Cost Estimates Drive Inaccurate Funding Commitments: 

The foundation of an accurate funding commitment for a weapon system 
program should be a realistic cost estimate that is based on a high 
degree of knowledge about requirements, technology, design, and 
manufacturing. Realistic estimates also provide a sound basis for 
setting priorities by allowing decision makers to compare the relative 
value of one program to another and to make adjustments accordingly. 
These adjustments could include a decision about whether to proceed 
with a program, reduce requirements, or defer requirements to a future 
increment. Most of the 20 programs we reviewed underestimated costs. 
Unrealistic cost estimates are largely the result of a lack of 
knowledge, failure to adequately account for risk and uncertainty, and 
overly optimistic assumptions about the time and resources needed to 
develop weapon systems. By repeatedly relying on unrealistically low 
cost estimates, DOD has initiated more programs than its budget can 
support. 

Our assessment of cost data for 20 major acquisition programs in DOD's 
current portfolio found that the majority of these programs were 
initiated with cost estimates for system development that were too low-
-a finding consistent with our prior work and with cost growth patterns 
reported by RAND, the Institute for Defense Analysis (IDA), and other 
organizations that conduct defense analyses.[Footnote 16] For 19 of the 
20 programs, the independent CAIG estimates were higher than the 
service estimates--by as much as 139 percent in one case--yet the CAIG 
estimates for 5 of those programs were still understated by billions of 
dollars (see table 1). In addition, while 5 of the 20 programs had not 
reported cost growth as of December 2007, the remaining 15 programs 
had. For example, the initial CAIG estimate for the Expeditionary 
Fighting Vehicle (EFV) program was about $1.4 billion compared to a 
service estimate of about $1.1 billion, but development costs for the 
EFV system is now expected to be close to $3.6 billion. Similarly, the 
Army initially estimated that WIN-T development would cost $338 
million, but the development program is now expected to cost over $2.0 
billion, or $1.7 billion more than initially estimated. Eight of the 20 
programs have reported development cost growth of more than 35 percent, 
resulting in the need for nearly $19 billion in additional funding. 
Estimates that are this far off the mark do not provide the necessary 
foundation for sufficient funding commitments. 

Table 1: Development Cost Estimates and Baselines for 20 Major Weapon 
System Programs: 

Program: Global Hawk: 
Development cost estimate: Service: $905; 
Development cost estimate: CAIG: $992; 
Percent difference: 10%; 
Development cost baselines: Initial: $967; 
Development cost baselines: Current: $3,515; 
Percent change: 264%. 

Program: UH-60M helicopter upgrade; 
Development cost estimate: Service: $311; 
Development cost estimate: CAIG: $379; 
Percent difference: 22%; 
Development cost baselines: Initial: $311; 
Development cost baselines: Current: $838; 
Percent change: 169%. 

Program: WIN-T; 
Development cost estimate: Service: $338[A]; 
Development cost estimate: CAIG: $807; 
Percent difference: 139%; 
Development cost baselines: Initial: $796; 
Development cost baselines: Current: $2,088;
Percent change: 162%. 

Program: C-130 Avionics Modernization; 
Development cost estimate: Service: $1,020; 
Development cost estimate: CAIG: $1,175; 
Percent difference: 15%; 
Development cost baselines: Initial: $720; 
Development cost baselines: Current: $1,844; 
Percent change: 156%. 

Program: EFV; 
Development cost estimate: Service: $1,056; 
Development cost estimate: CAIG: $1,438; 
Percent difference: 36%; 
Development cost baselines: Initial: $1,472; 
Development cost baselines: Current: $3,556; 
Percent change: 142%. 

Program: Advanced Extremely High Frequency Satellites; 
Development cost estimate: Service: $3,031; 
Development cost estimate: CAIG: $3,175; 
Percent difference: 5%; 
Development cost baselines: Initial: $2,923; 
Development cost baselines: Current: $6,008; 
Percent change: 105%. 

Program: Wideband Global SATCOM; 
Development cost estimate: Service: $296; 
Development cost estimate: CAIG: $414; 
Percent difference: 40%; 
Development cost baselines: Initial: $199; 
Development cost baselines: Current: $323; 
Percent change: 62%. 

Program: Future Combat Systems; 
Development cost estimate: Service: $20,248; 
Development cost estimate: CAIG: $27,184; 
Percent difference: 34%; 
Development cost baselines: Initial: $20,248; 
Development cost baselines: Current: $27,955; 
Percent change: 38%. 

Program: Joint Strike Fighter[B]; 
Development cost estimate: Service: $30,500; 
Development cost estimate: CAIG: $31,476; 
Percent difference: 3%; 
Development cost baselines: Initial: $33,939; 
Development cost baselines: Current: $40,210; 
Percent change: 18%. 

Program: COBRA JUDY replacement; 
Development cost estimate: Service: $1,398; 
Development cost estimate: CAIG: $1,521; 
Percent difference: 9%; 
Development cost baselines: Initial: $1,527; 
Development cost baselines: Current: $1,626; 
Percent change: 6%. 

Program: E-2 Advanced Hawkeye; 
Development cost estimate: Service: $3,495; 
Development cost estimate: CAIG: $3,720; 
Percent difference: 6%; 
Development cost baselines: Initial: $3,589; 
Development cost baselines: Current: $3,796; 
Percent change: 6%. 

Program: EA-18G; 
Development cost estimate: Service: $1,707; 
Development cost estimate: CAIG: $1,795; 
Percent difference: 5%; 
Development cost baselines: Initial: 1,797; 
Development cost baselines: Current: $1,865; 
Percent change: 4%. 

Program: VH-71 Presidential Helicopter Replacement Program; 
Development cost estimate: Service: $3,378; 
Development cost estimate: CAIG: $3,569; 
Percent difference: 6%; 
Development cost baselines: Initial: $3,771; 
Development cost baselines: Current: $3,859; 
Percent change: 2%. 

Program: Joint Land Attack Cruise Missile Defense Elevated Netted 
Sensor; 
Development cost estimate: Service: $1,781; 
Development cost estimate: CAIG: $1,926; 
Percent difference: 8%; 
Development cost baselines: Initial: $1,894; 
Development cost baselines: Current: $1,922; 
Percent change: 1%. 

Program: C-5 RERP; 
Development cost estimate: Service: $1,454; 
Development cost estimate: CAIG: $1,583; 
Percent difference: 9%; 
Development cost baselines: Initial: $1,627; 
Development cost baselines: Current: $1,630; 
Percent change: 0[C]. 

Program: CH-53K Heavy Lift Replacement; 
Development cost estimate: Service: $3,970; 
Development cost estimate: CAIG: $4,293; 
Percent difference: 8%; 
Development cost baselines: Initial: $4,149; 
Development cost baselines: Current: $4,095; 
Percent change: -1%. 

Program: Longbow Apache III; 
Development cost estimate: Service: $1,155; 
Development cost estimate: CAIG: $1,382; 
Percent difference: 20%; 
Development cost baselines: Initial: $1,095; 
Development cost baselines: Current: $1,087; 
Percent change: -1%. 

Program: MMA; 
Development cost estimate: Service: $6,100; 
Development cost estimate: CAIG: $6,970; 
Percent difference: 14%; 
Development cost baselines: Initial: $7,080; 
Development cost baselines: Current: $6,804; 
Percent change: -4%. 

Program: Small Diameter Bomb; 
Development cost estimate: Service: $416; 
Development cost estimate: CAIG:$427; 
Percent difference: 3%; 
Development cost baselines: Initial: $415; 
Development cost baselines: Current: $395; 
Percent change: -5%. 

Program: Standard Missile 6; 
Development cost estimate: Service: $1,000; 
Development cost estimate: CAIG: $992; 
Percent difference: -1%; 
Development cost baselines: Initial: $1,009; 
Development cost baselines: Current: $932; 
Percent change: -8%. 

Source: GAO analysis of DOD data. 

[A] The service's original estimate for the WIN-T program of $338 
million was revised prior to Milestone B to align with the CAIG's 
estimate. 

[B] JSF data include Air Force and Navy portions of the program only. 

[C] C-5 RERP costs have increased by almost $4 million. However, due to 
rounding, the table indicates a 0 percent increase. 

[End of table] 

DOD has found similar problems in other acquisition programs. For 
example, in 2003, a DOD study found that space programs are strongly 
biased to produce unrealistically low cost estimates throughout the 
acquisition process.[Footnote 17] The study found that most programs at 
the time of contract initiation had a predictable cost growth of 50 to 
100 percent. The study also found that the unrealistically low 
projections of program cost and lack of provisions for management 
reserve seriously distorted management decisions and program content, 
increased risks to mission success, and virtually guaranteed program 
delays. 

Inaccurate cost estimates are often the result of limited knowledge 
about requirements and technologies. Our best practices work has shown 
that conducting early disciplined analysis, such as systems 
engineering,[Footnote 18] builds knowledge that enables a developer to 
identify and resolve gaps between requirements and available resources 
before beginning product development. DOD's acquisition policy and 
guidance emphasize the importance of obtaining knowledge prior to 
Milestone B--the start of system development--through key analyses and 
activities--such as conducting an analysis of alternatives (AOA), 
[Footnote 19] refining requirements, and reducing technology risks. 
Knowledge gained from these analyses and activities should inform a 
Cost Analysis Requirements Description (CARD)--a key document that 
quantifies the program's technical, physical, programmatic, and 
performance characteristics for developing cost estimates. However, we 
have frequently reported that DOD programs do not adequately define 
requirements, mature technologies, or develop an effective acquisition 
strategy before Milestone B--the point at which cost estimates are 
approved.[Footnote 20] According to several CAIG analysts, CARDs often 
lack sufficient detail about planned program content to develop sound 
cost estimates. One analyst noted that the CARD for the FCS program had 
to be sent back to the program office because it was too vague. 

At the same time, programs are expected to deliver the CARD to the CAIG 
at least 180 days prior to Milestone B, while other supporting 
documents that contain critical program information--such as the 
program requirements document--are not required until about 90 days in 
advance of a milestone review. Many programs conduct analyses of 
alternatives and other analyses at the same time program cost estimates 
are being developed. According to a senior CAIG official, such 
concurrency limits their ability to develop a realistic independent 
estimate. One senior DOD official also noted that the department's 
acquisition policy may have encouraged programs to rush to Milestone B, 
limiting the quality of the data available to develop a program cost 
estimate. Although DOD's acquisition policy has included an early 
formal review at Milestone A, this review was not mandatory, and as we 
have reported in the past, most major acquisition programs have not 
gone through this early review.[Footnote 21] 

Cost analysts have also indicated that the lack of comparable products 
and good historical program data limits knowledge, making it difficult 
to develop realistic cost estimates. This is true for the majority of 
DOD's larger and more complex transformational programs that are 
seeking to provide capabilities that require advanced technologies and 
equipment that have no historical precedent, like JSF and FCS. 

In the absence of knowledge, cost estimators must rely heavily on 
assumptions about system requirements, technology, and design maturity 
as well as the time and funding needed. However, the level of resources 
needed for product development is often understated, as we found in 
several of the programs we reviewed. For example: 

* FCS: The estimated lines of code needed to support FCS's software 
development are almost three times original assumptions--from 32 
million to 95 million lines of code--leading to an increase in software 
development costs that now approaches $8 billion. A 2007 IDA report 
also identified significant additional unquantifiable cost risk due to 
immature technologies, dependencies on complementary programs, 
concurrent experimentation and development, and the overall complexity 
and synchronization of FCS development activities.[Footnote 22] 

* JSF: JSF assumed that the commonality between the three variants of 
aircraft and the use of a joint development program, instead of three 
separate programs, could cut development costs by about 40 percent. 
However, after development started, significant design issues forced 
the program to delay development approximately 18 months to conduct 
unexpected design work. In addition, the assumed commonality between 
the variants decreased. As we reported in 2005, these two factors 
contributed to cost increases that nearly eroded all of the assumed 
cost savings.[Footnote 23] 

* WIN-T: The Army assumed that the radios and software needed to 
support the WIN-T system would be commercially available. However, once 
system development started, the Army learned that the radios and 
software would require significantly more development and integration 
than initially anticipated. Further, the Army assumed that WIN-T would 
be able to meet its portion of the FCS program requirements. However, 
subsequent changes in the FCS requirements contributed to the need to 
restructure the WIN-T program. 

* Global Hawk: The Air Force assumed that validated warfighter 
requirements could be met with minor additional development to a 
smaller version, subsequently designated the RQ-4A. However, 1 year 
after initiating both system development and low-rate initial 
production, it was determined that a larger airframe and additional, 
unproven technologies would be needed. As a result, DOD restructured 
the acquisition strategy to include a second model--designated RQ-4B-- 
tripling development costs and extending the development cycle from 7 
years to 12 years. 

These examples are consistent with our work over the past several 
decades, which has raised concerns about the cost and schedule 
implications of DOD's cost estimating assumptions in its major 
acquisitions. Our work on DOD's space acquisition programs has found 
that these programs regularly made overly optimistic cost-estimating 
assumptions, including assumptions about their ability to define 
requirements, mature technologies, and secure the funding and other 
resources needed to develop the system within a specified time frame. 
[Footnote 24] Similarly, we recently testified that the Navy tends to 
underestimate the costs needed to construct ships, resulting in 
unrealistic budgets and large cost increases after ship construction 
has begun. [Footnote 25] We noted that for two major ship programs, the 
Navy assumed significant savings based on efficiencies that did not 
materialize. We linked these optimistic assumptions to cost growth and 
schedule delays in a number of these programs. 

Cost estimates that lack knowledge and rely heavily on assumptions have 
inherently high levels of risk and uncertainty. Conducting quantitative 
risk and uncertainty analysis provides a way to assess the variability 
in an estimate. Using this type of analysis, cost estimators can model 
such effects as a schedule slipping or a key technology failing to 
materialize, thereby identifying a range of likely costs around an 
estimate. Presenting decision makers with the range of likely costs 
around an estimate provides insight into the amount of cost, schedule, 
and technical risks they are being asked to accept and conveys a level 
of confidence associated with achieving the proposed estimate. A range 
of costs also provides a basis for deciding how much funding a program 
needs to be successful. 

In developing cost estimates, the services and the CAIG often do not 
present decision makers with the range of costs around an estimate. 
[Footnote 26] Instead, they present a single, or point, estimate as the 
most probable cost, which OSD expects to be a 50 percent chance that 
actual program costs will be at or below the estimated value. Several 
recent studies have questioned DOD's approach and recommended 
establishing estimates with higher levels of confidence under the 
assumption that the result will be more realistic estimates.[Footnote 
27] For example, the Defense Acquisition Performance Assessment Panel 
recommended adjusting program cost estimates to reflect "high 
confidence"--defined as an 80 percent chance of completing development 
at or below the estimated amount. Requiring a higher confidence level 
could provide a better basis for determining program funding, but only 
if the quality of the cost estimate is sound and the underlying risk 
and uncertainty associated with the estimate are accurately captured. 
If the quality of the estimate is poor to begin with, simply applying a 
higher confidence level to the estimate will not make it any more 
realistic. 

DOD's Failure to Balance Needs with Resources Promotes Unhealthy 
Competition for Funding: 

In prior years, we have reported that DOD commits to more programs than 
its resources can support.[Footnote 28] DOD's failure to balance 
requirements with available resources promotes unhealthy competition 
among programs for funding. Ultimately, programs tend to push the need 
for funding to the future rather than limit program length or adjust 
requirements. 

DOD's portfolio of weapon system programs has grown over the past 
several years at a pace that far exceeds available resources. From 1992 
to 2007, the estimated acquisition costs remaining for major weapons 
programs increased almost 120 percent, while the annual funding 
provided for these programs only increased 57 percent, creating a 
fiscal bow wave that may be unsustainable (see fig. 3). If this trend 
goes unchecked and fiscal pressures to reduce spending continue to grow 
as expected, Congress will be faced with a difficult choice to either 
pull funds from other federal programs to support DOD's acquisitions or 
accept less warfighting capability than promised. 

Figure 3: Costs Remaining versus Annual Appropriations for Major 
Defense Acquisitions: 

[See PDF for image] 

This figure is a multiple line graph depicting the following 
information: 

Fiscal year: 1992; 
Annual RDTE and Procurement Appropriations: $100; 
Costs Remaining for Major Defense Acquisitions: $324. 

Fiscal year: 1993; 
Annual RDTE and Procurement Appropriations: $91; 
Costs Remaining for Major Defense Acquisitions: $271. 

Fiscal year: 1994; 
Annual RDTE and Procurement Appropriations: $79; 
Costs Remaining for Major Defense Acquisitions: $253. 

Fiscal year: 1995; 
Annual RDTE and Procurement Appropriations: $78; 
Costs Remaining for Major Defense Acquisitions: $341. 

Fiscal year: 1996; 
Annual RDTE and Procurement Appropriations: $78; 
Costs Remaining for Major Defense Acquisitions: $345. 

Fiscal year: 1997; 
Annual RDTE and Procurement Appropriations: $79; 
Costs Remaining for Major Defense Acquisitions: $331. 

Fiscal year: 1998; 
Annual RDTE and Procurement Appropriations: $82; 
Costs Remaining for Major Defense Acquisitions: $280. 

Fiscal year: 1999; 
Annual RDTE and Procurement Appropriations: $89; 
Costs Remaining for Major Defense Acquisitions: $233. 

Fiscal year: 2000; 
Annual RDTE and Procurement Appropriations: $94; 
Costs Remaining for Major Defense Acquisitions: $309. 

Fiscal year: 2001; 
Annual RDTE and Procurement Appropriations: $104; 
Costs Remaining for Major Defense Acquisitions: $234. 

Fiscal year: 2002; 
Annual RDTE and Procurement Appropriations: $111; 
Costs Remaining for Major Defense Acquisitions: $586. 

Fiscal year: 2003; 
Annual RDTE and Procurement Appropriations: $137; 
Costs Remaining for Major Defense Acquisitions: $579. 

Fiscal year: 2004; 
Annual RDTE and Procurement Appropriations: $148; 
Costs Remaining for Major Defense Acquisitions: $722. 

Fiscal year: 2005; 
Annual RDTE and Procurement Appropriations: $165; 
Costs Remaining for Major Defense Acquisitions: $745. 

Fiscal year: 2006; 
Annual RDTE and Procurement Appropriations: %147; 
Costs Remaining for Major Defense Acquisitions: $841. 

Fiscal year: 2007; 
Annual RDTE and Procurement Appropriations: $160. 
Costs Remaining for Major Defense Acquisitions: $767. 

Source: DOD (data). GAO (analysis and presentation). 

[End of figure] 

DOD's ability to prioritize needs within available resources is 
hampered because its processes and management structures for 
determining capability needs and allocating resources are not 
effectively integrated. Capability needs are formally identified and 
validated through JCIDS, which does not account for the resources that 
will be needed to meet those needs. Instead, resources are assessed and 
allocated through the PPBE process, which as we have reported in the 
past, is service-centric and does not effectively link resources to 
capabilities. In addition, the PPBE and JCIDS processes are led by 
different organizations within DOD, as is the process for executing 
acquisitions, making it difficult to hold any one person or 
organization accountable for saying no to a proposed program or for 
program outcomes. 

DOD's failure to balance its needs with available resources promotes 
unhealthy competition among and within the services to get funding for 
new programs and to sustain funding for existing ones. This competition 
for funding creates strong incentives for service and program officials 
to establish requirements that make their particular weapon systems 
stand out from others, with less consideration given to the resources 
that will be needed to develop them. Because DOD's overall funding must 
fit within fiscal constraints of the FYDP that are established by OMB, 
services place a high priority on the appearance of affordability. To 
maintain a competitive edge, services develop cost estimates that will 
fit within established funding levels. Also, because of the length of 
the PPBE process, initial funding for a program is often established in 
the FYDP before the program's initial cost estimated is developed and 
approved. In addition, program officials are motivated to initiate a 
weapon system development program because once initiated a program is 
in a more competitive position to attract high levels of funding and 
management support. Many of the weapon system program managers who 
responded to a GAO survey in 2005 cited this competitive funding 
environment as a key obstacle to their ability to effectively manage 
their programs.[Footnote 29] For example, program managers provided 
comments such as the following on competition for funding: 

* "OSD has reduced funding without any understanding or appreciation 
for program impacts. Funding cuts appear arbitrary." 

* "OSD's near-term execution year focus results in great instability. 
In reality, it should provide more strategic vectors for the department 
instead of short-term adjustments to fix more tactical-level funding 
needs." 

* "The service and OSD typically cut programs to pay top down bills." 

* "There is no such thing as funding stability in DOD. Funding 
reductions and program stretch-outs are the norm due to top down fiscal 
bills that occur during the execution year." 

* "Unstable funding results in pressure to do aggressive things in 
order to minimize the impact of budget cuts on schedule and 
performance. I believe this has been a major factor in recent ï¿½ program 
execution problems." 

* "When funding gets tight, we have been considered a bill payer for 
others, even if it has "broken" our program." 

OSD reviews program budgets and makes adjustments toward the end of the 
PPBE cycle--often only 2 or 3 months before the budget is submitted to 
Congress. For example, in December 2004, OSD cut $30 billion from the 
2006 FYDP for many of its major acquisition programs, including 
prominent aircraft programs like the F-22A Raptor, the V-22 Osprey, and 
the C-130J Hercules. Attempting to balance investments this late in the 
process often leads to additional churn in programs and encumbers 
efforts to meet strategic objectives and joint needs. 

Instead of rethinking performance requirements and making needed trade- 
offs within and among programs, DOD and the services often pursue 
ambitious solutions and technologies that result in cost growth and the 
need for unplanned funding. Ultimately, DOD often defers costs into the 
future--beyond the FDYP--expecting that additional funding will become 
available when it is needed. According to DOD and service officials, 
what often ends up happening is that programs have to extend their 
schedules to spread costs out over time or reduce requirements to hold 
costs down. These strategies ultimately result in delaying the delivery 
of programs or providing the warfighter with less capability than 
promised. 

Proven Practices Help Ensure Accurate Cost Estimates and Adequate 
Program Funding: 

Successful commercial companies that we have previously reviewed 
achieved adequate and stable funding for product development programs 
by following practices that we have recognized as best practices for 
estimating and managing program costs.[Footnote 30] In contrast to DOD, 
these companies followed a disciplined, knowledge-based approach to 
estimating program costs. Prior to initiating product development, 
these companies expected cost estimates to be refined based on 
increasing product knowledge and assessed the estimates at multiple 
gated reviews. At the same time, the companies expected proposed 
programs to have a manageable development cycle, which increases the 
predictability of funding needs and the likelihood of program success. 
Understanding that resources are limited, the companies used portfolio 
management practices to make tough decisions about which programs to 
pursue. Once programs were approved, these companies firmly committed 
to fully fund them--which they often achieved by committing resources 
incrementally to successive phases of a program. By making these 
incremental commitments, these companies were able to limit the 
disruptions that can be caused by poorly performing programs, and thus 
maintained a high degree of stability within and among other programs. 

A Disciplined, Knowledge-Based Approach and Manageable Development 
Cycles Are Key to Realistic Cost Estimates: 

As part of our best practices work on successful product development, 
we have found that following a disciplined, knowledge-based approach 
that reduces risk and uncertainty over time is integral to developing 
realistic cost estimates. The successful companies we have reviewed 
conducted multiple management reviews prior to initiating product 
development to assess the business case of each proposed product, 
including its cost estimate. They typically expected cost estimates to 
be developed early on and refined over time as knowledge is gained. In 
addition, they expected estimates to be transparent because it allowed 
them to make informed decisions about the risks and uncertainty 
associated with proposed programs and whether they should pursue or 
cancel them. While these companies acknowledged that early in a 
program's life cycle less is known about the product's design and 
technologies, they still expected to receive rough cost estimates at 
that stage. They expected the estimates to be revised and more precise 
as knowledge was gained. Before committing to product development, they 
expected the level of uncertainty in the estimates to be low. 

Our Cost Assessment Guide similarly emphasizes the need to refine cost 
estimates based on knowledge gained over time and account for risk and 
uncertainty in the estimates. Early cost estimates are more uncertain 
because less is known about requirements and the opportunity for change 
is greater. As more knowledge is gained, programs can retire some risk 
and reduce the potential for unexpected cost and schedule growth. The 
best practice is not to commit to product development until the level 
of knowledge is high and the level of uncertainty is low. However, our 
past work has found that DOD typically commits to starting programs 
with low levels of knowledge and high levels of uncertainty (see fig. 
4). 

Figure 4: Notional Comparison of Cost Estimating Uncertainty and Levels 
of Knowledge at Program Start: 

[See PDF for image] 

This figure is a matrix depicting a notional comparison of cost 
estimating uncertainty and levels of knowledge at program start. 
Knowledge moves from low to high on the horizontal axis, and cost 
estimating uncertainty moves from low to high on the vertical axis. 
Noted on the matrix: 

Typical DOD program start: Occurs with high cost estimating uncertainty 
and low knowledge. 

Best practice program start: Occurs with high knowledge and low cost 
estimating uncertainty. 

Source: GAO. 

[End of figure] 

Despite having high levels of uncertainty, DOD commits to development 
programs based on point estimates that are expected to represent most 
likely costs. The Air Force's Cost Risk and Uncertainty Analysis 
Handbook notes that decision makers need point estimates when preparing 
and managing a budget because programs are funded and executed using 
discrete dollars, not ranges of dollars. However, DOD's estimates often 
prove to be understated by as much as 30 to 40 percent. To make more 
informed investment decisions, cost estimating best practices call for 
estimating a range of possible costs around a point estimate to provide 
information about the levels of uncertainty and confidence. As proposed 
programs gain more knowledge and progress through the phases leading up 
to the start of product development, these ranges should narrow (see 
fig. 5). According to cost-estimating experts and representatives of 
one of the commercial companies we spoke with, it is better to 
overestimate than underestimate costs. 

Figure 5: Range of Possible Costs Narrows as Knowledge Is Gained: 

[See PDF for image] 

This figure s an illustration of how the range of possible costs 
narrows as knowledge is gained. 

Prior to the concept refinement phase, the margin of overestimating is 
extremely high, and the margin for underestimating is at its' highest. 
During the concept refinement state and the concept definition phase, 
these margins narrow significantly. During the product development 
phase, bot margins continue to narrow and approach cost estimating 
certainty. 

Source: GAO. 

[End of figure] 

Officials in Motorola's Government and Enterprise Mobility Solutions 
business unit have established a rigorous review process, within which 
proposed products are expected to provide cost estimates that fall 
within an established range at successive review gates. At the first 
review gate, the range is generous, allowing for estimates to be as 
much as 75 percent too high and 25 percent too low. As the proposed 
program moves through review gates and more becomes known about 
requirements, technologies, and design, the established range narrows. 
At product development initiation, the cost estimate is expected to be 
no more than 10 percent higher and 5 percent lower than what actual 
costs will be. IBM similarly allows products to deviate from their 
original estimates as long as the deviation is within agreed-upon 
limits, which are established in a contract between senior management 
and project managers.[Footnote 31] Product development teams are 
expected to execute according to the contract. According to IBM 
officials, program cost growth of more than 5 or 10 percent is 
generally not acceptable. 

Successful companies have found that a relatively short, manageable 
development cycle is a hallmark of an executable program. Officials at 
Motorola told us that cycle time is one of the key metrics they use 
when making decisions about what programs to pursue. They noted that 
development programs longer than 2 or 3 years are not likely to be 
initiated because the increased uncertainty would make it too difficult 
to accurately plan for and execute the programs. This is consistent 
with what we found at other successful commercial companies as well. By 
constraining development cycles, it is easier to more accurately 
estimate costs, and with more accurate cost estimates companies are 
able to more precisely predict the future funding needs and effectively 
allocate resources. 

In 1998, the Under Secretary of Defense for Acquisition, Technology and 
Logistics stated that the department's objective must and will be to 
achieve acquisition cycle times no longer than 5 to 7 years, noting 
that long acquisition cycle times for major defense programs lead to 
higher costs and diminished military effectiveness. DOD's acquisition 
policy, revised in 2003, suggests that system development should be 
limited to a manageable time frame--about 5 years. An assessment of 
DOD's acquisition system commissioned by the Deputy Secretary of 
Defense in 2006 similarly recommended that programs should be time- 
constrained with development cycles no longer than 6 years from 
Milestone A to low-rate initial production.[Footnote 32] According to 
the assessment, a time-constrained development cycle can reduce 
pressure on investment accounts and increase funding stability for all 
programs. With development cycles of 6 years or less, programs could be 
fully funded within the FYDP time frame. In addition, constrained cycle 
times would force programs to conduct more detailed systems engineering 
analyses, increasing the likelihood that their requirements can be met 
with available resources. We recently reported that while there are 
isolated examples of DOD programs with cycle times shorter than 5 
years, the majority of programs were initiated with much longer cycle 
times. In some cases, these longer cycle times have been extended. 
[Footnote 33] For example, FCS was expected to be a 7.5-year 
development effort but is now a 12-year program and will likely be 
extended again. Unconstrained and lengthy cycle times promote program 
funding instability--especially when considering DOD's tendency to 
change requirements and funding as well as frequent changes in 
leadership. 

A Portfolio-Based Investment Approach with Incremental Commitments 
Supports Adequate and Stable Program Funding: 

To ensure that resources are available to fully support their programs, 
successful commercial companies we reviewed used an integrated 
portfolio management approach to make incremental investment decisions. 
In making these decisions, the companies looked across their entire 
product portfolio within the context of available resources to ensure 
that they were pursuing a balanced mix of products that can optimize 
the return on their investment. Once a decision was made to pursue a 
development program, the companies committed to fully funding the first 
phase of that program, and made similar commitments prior to initiating 
each successive phase. Before programs were approved to enter the next 
phase of product development, Eli Lilly and IBM required program 
officials and management to sign a contract that represented the 
program's promise to deliver a given product that would meet the 
customers' needs within established cost and time constraints. These 
contracts also represented the ongoing commitment of management to 
provide the necessary funding and other resources needed to ensure that 
the program could be successfully executed. 

By making incremental commitments companies are better positioned to 
effectively manage and maintain stability in programs. For companies 
like Eli Lilly--which tends to have development cycles of 10 to 15 
years on average--the practice of making short-term, incremental 
commitments mitigates many of the cost, schedule, and performance risks 
inherent in long development programs. When Eli Lilly decides to pursue 
a new drug development program, management and the project team enter 
into a contract that identifies deliverables, time frames, and the 
costs to get to the next milestone. The contract represents 
management's commitment to fund the entire phase. IBM similarly 
allocates funding in increments. For example, at the initial review 
gate, funding is allocated to the product development team to support 
the development of a sound business plan, which is expected to be 
presented at the next review gate before any further funding commitment 
is made. 

Making incremental commitments also allows companies the flexibility to 
effectively manage their portfolio. As they assess the value and 
progress of each investment at multiple points throughout product 
development, companies are able to make tough decisions to defer or 
terminate programs and rebalance their portfolio. Terminating low-value 
or poor-performing programs is important to the successful companies we 
spoke with, because each dollar spent on a failing program is one less 
dollar available for use elsewhere in the company. As a result, they 
emphasize the need to make these tough decisions early. For example, at 
Eli Lilly projects are terminated at early points in the review process 
when it is determined that their critical success factors cannot be 
achieved. Because Eli Lilly's projects typically have a high degree of 
technical risk, only about 1 percent of those that start early 
development actually make it to the marketplace. 

We have found that successful portfolio management requires a strong 
governance structure with committed leadership that empowers portfolio 
managers to make decisions about the best way to invest resources and 
holds those managers accountable for the outcomes they achieve. Because 
portfolio managers are on the front line, several of the companies we 
have reviewed empower these managers to make product investment 
decisions and then hold them accountable for outcomes, not just for 
individual products but also for the overall performance of the 
portfolio. These organizations underscore the importance of holding 
individuals accountable, aligning performance expectations with 
organizational goals, and cascading those expectations down to lower 
levels. 

In contrast, DOD approves proposed programs without adequately 
considering its overall portfolio and commits to programs with less 
knowledge of cost and feasibility. Moreover, DOD lacks the 
accountability that commercial companies emphasize is necessary to 
ensure successful outcomes. Consequently, DOD starts more programs than 
current and likely future resources can support and has less assurance 
that its investment decisions address the right mix of warfighting 
needs. 

Conclusions: 

At a time when the federal budget is strained by spending needs for a 
growing number of national priorities, it is imperative that DOD get 
the best value for every dollar of its significant investments. Yet DOD 
has more major weapons system programs in its portfolio than it can 
afford. All too often, these programs incur cost increases well beyond 
original funding levels and significant delays in delivering weapon 
systems to the warfighter. These outcomes are the direct result of 
DOD's failure to prioritize its needs within resource constraints and a 
funding process that allows programs to go forward with unpredictable 
cost estimates and lengthy development cycles--not a sound basis for 
allocating resources and ensuring program stability. Successful 
commercial companies we have reviewed recognize the importance of 
having adequate knowledge about requirements and available resources 
before initiating a development program. These companies have learned 
that disciplined processes with early management review points, 
knowledge-based cost estimates, manageable product development cycles, 
and a portfolio approach to funding product development efforts are key 
to achieving program success. While DOD has recognized that more 
accurate cost estimates coupled with manageable development cycles 
could improve program outcomes, it has yet to take action. Until DOD 
revises its policies and processes to address funding imbalances and 
reform its funding approach, programs will continue to experience 
instability and delayed delivery of needed capabilities to the 
warfighter. 

Recommendations for Executive Action: 

To better ensure adequate funding for DOD's major weapon system 
acquisition programs and to increase the likelihood of achieving 
successful outcomes, we recommend that the Secretary of Defense take 
the following three actions: 

* Develop and implement a strategy to bring the department's current 
portfolio into balance by aligning the number of programs and the cost 
and schedule of those programs with available resources. In developing 
and implementing a strategy, the department should determine ways to 
prioritize needs and identify whether the budget and the FYDP should be 
increased to more accurately reflect the actual costs of current 
programs or whether the portfolio of current programs should be reduced 
and lower-priority programs terminated to match available resources. 

* Require that all new programs have manageable development cycles, 
realistic cost estimates, and have planned and programmed full funding 
for the entire development cycle. 

* Require all cost estimates submitted for funding a program at 
milestone decisions to be reported as a range of likely costs and 
reflect the associated levels of risk and uncertainty. At Milestone A, 
require estimates that allow for a wide range of likely costs. At 
Milestone B, require estimates that, based on knowledge gained, are 
more precise--in line with best practice standards. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, DOD partially concurred 
with our first and second recommendations and non-concurred with the 
third. DOD's partial concurrences are rooted in the belief that its 
current policies and initiatives address our recommendations. We agree 
that aspects of DOD's current policies appear consistent with our 
recommendations, and the initiatives could contribute to better program 
outcomes if implemented appropriately. However, we have found no 
evidence of widespread adoption of these policies or any other process 
that would better ensure adequate funding for DOD's major weapon system 
acquisition programs and increase the likelihood of achieving 
successful outcomes. 

DOD partially concurred with our first recommendation--that the 
Secretary of Defense develop and implement a strategy to bring the 
department's current portfolio into balance by aligning the number of 
programs and the cost and schedule of those programs with available 
resources. In its written response, DOD identified the 2003 
restructuring of the PPBE process as one initiative it has taken to 
address the mismatch between program commitments and available 
resources. Yet, since the restructuring, more than 5 years--and several 
budget cycles--have lapsed, and the department is still committed to 
more programs than it can support. While DOD notes that seeing the 
effects of process changes will take years, the department does not 
indicate how many additional years are needed to see positive results 
or what incremental changes it should be held accountable for. In the 
meantime, the department continues to risk tax dollars and delaying the 
delivery of needed capabilities to the warfighter. The department also 
cites several other, more recent initiatives like Capability Portfolio 
Management, Capital Accounts, and Configuration Steering Boards that 
are intended to balance needs with resources; drive more realistic, 
cost-effective plans and budgets; limit requirements growth; and 
improve oversight. While we believe that these initiatives, like many 
before them, are well intentioned, DOD has not established indicators 
to measure their success, and it is unclear how the initiatives will 
help bring DOD's current portfolio into balance. Further, we are 
concerned that they do not go far enough to address the systemic 
cultural and structural problems identified in this report. 

DOD states that "external influences" can cause turbulence in program 
execution. Specifically, the department notes that it does not control 
the amount of Total Obligation Authority (TOA) available in the FYDP or 
how much funding is appropriated by Congress. While we acknowledge that 
decisions made by OMB and Congress can directly affect program funding, 
we believe that our recommendation is sound regardless of the total 
amount of funding provided to the department. As we have recently 
reported, substantially more funding has been committed to develop new 
weapon systems since 2000, yet cost overruns and schedule delays have 
increased even more. By focusing on external influences DOD misses the 
main point of our recommendation--to align the number of programs in 
the current portfolio and the cost and schedule of those programs with 
available resources. 

DOD also partially concurred with our second recommendation--to require 
all new programs to have manageable development cycles, realistic cost 
estimates, and full funding for the entire development cycle. Again, 
the department's partial concurrence is based on its belief that 
certain DOD policies and initiatives respond to this recommendation. We 
agree that some DOD policies and initiatives emphasize a knowledge- 
based approach to acquiring weapon systems, but acquisition officials 
do not effectively implement these policies. DOD programs are often 
initiated with development cycle times that are much longer than the 5 
years the department's policy suggests. As we note in this report, 
programs with development cycles of 6 years or less could be fully 
funded in a single FYDP and would be less likely to experience funding 
instability. DOD further commented that the current acquisition policy 
requires cost estimates and, where required, independent cost estimates 
to be completed at key program milestones and full funding to be 
programmed prior to Milestone B. However, we found that because DOD's 
cost estimates at key milestones are based on limited knowledge and 
optimistic assumptions about requirements and technology, they often 
significantly underestimate true costs and do not provide the necessary 
foundation for making accurate funding commitments. In addition, our 
analysis of DOD's selected acquisition reports and budget data found 
that DOD often does not commit full funding to its major weapon system 
acquisitions when they are initiated, despite the department's policy 
to do so. 

While the department generally agreed that product development cycles 
should be "manageable," it questioned the comparability of DOD to the 
private sector firms we reviewed with regard to industry goals, 
incentives, products, and services. This misses the point. We are 
recommending that development cycles for all new DOD programs be 
manageable--a guiding principle that the successful commercial 
companies we have reviewed follow to enable better program outcomes. 
The department's own acquisition policy suggests development cycles of 
about 5 years--an objective the department established a decade ago. 
Similarly, a recent study commissioned by DOD recommended that programs 
should be time-constrained with development cycles no longer than 6 
years. In general, DOD indicated an interest in obtaining more insight 
into our methodology. Much of this information, including information 
about the companies and commodities that we have reviewed in the past, 
can be found in most of the works cited in the related GAO products 
section of this report. 

DOD did not concur with our third recommendation. While DOD agreed that 
risks should be considered at key milestones, it did not agree that 
cost estimates submitted at milestone decisions should be reported as a 
range of likely costs--a wide range at Milestone A based on limited 
knowledge, narrowing to a more precise range at Milestone B as more 
knowledge is gained. The CAIG accounts for cost risk by examining 
program schedule durations, technical risks, contract vehicles, 
incentives, and management structures. However, DOD's current practice 
of presenting a single point estimate as the "most likely cost" 
provides decision makers with limited insight into these risks. DOD 
also cited concerns that a certain method for calculating cost ranges 
can produce misleading results. While we do not advocate a specific 
method for calculating a range of potential costs, we maintain that 
presenting such a range would provide decision makers with additional 
knowledge about the level of risk in a proposed program. 

Finally, DOD states that our report does not consider certain causal 
factors bearing on the estimates we cite in table 1. It is unclear how 
DOD came to this conclusion. Our report states that inaccurate cost 
estimates are often the result of limited knowledge about requirements 
and technologies. We also note that conducting early disciplined 
analyses, such as systems engineering, builds knowledge that enables a 
developer to identify and resolve gaps between requirements and 
available resources before beginning product development. As knowledge 
increases, uncertainty and associated risks in the cost estimate 
decrease. Therefore, we believe that following a disciplined knowledge- 
based process--a recommendation we have made repeatedly in our reviews 
of DOD's major weapon system programs--would obviate most of the causal 
factors DOD cites, including "incomplete, error-full, or volatile 
requirements or program specification" and "engineering change 
proposals." 

DOD's written comments are reprinted in appendix II. The Department 
also provided technical comments, which were incorporated as 
appropriate. 

We are sending copies of this report to the Secretary of Defense; the 
Secretaries of the Air Force, Army, and Navy; and the Director of the 
Office of Management and Budget. We will provide copies to others on 
request. This report will also be available at no charge on GAO's Web 
site at [hyperlink, http://www.gao.gov]. 

If you have any questions about this report or need additional 
information, please contact me at (202) 512-4841 or [email protected]. 
Key contributors to this report were John Oppenheim, Assistant 
Director; Travis Masters; Keith Hudson; Victoria Klepacz; Karen Sloan; 
Karen Richey; and John Krump. 

Signed by: 

Michael J. Sullivan: 
Director, Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

This report assesses the extent to which the Department of Defense's 
(DOD) resource allocation approach supports stability within and across 
its major weapon system acquisition programs. Specifically, our 
objectives were to (1) assess how DOD budgets for and funds its major 
weapon system acquisition programs, (2) identify key factors that 
influence the effectiveness of this approach, and (3) identify proven 
processes and practices that could help improve DOD's ability to 
effectively allocate resources to its acquisition programs. 

To assess the effectiveness of DOD process for funding for its major 
weapon system acquisition programs, we compared initial approved cost 
estimates for 20 of the 95 major weapons programs in DOD's current 
portfolio to the amount of development funding contained in each 
program's budget documentation from the year in which development 
started--which covers the initial Future Years Defense Program (FYDP) 
time frame. The 20 programs we reviewed initiated system development 
from 1999 through 2006 and represent almost 35 percent of the total 
expected cost of DOD's current portfolio of major weapon system 
programs. Due to data limitations, we were not able to assess a number 
of programs, including several major ship acquisitions, in our cost 
estimate and funding analysis. While our findings may not be 
generalized to all of DOD's acquisition programs, we believe that 
capturing more than one-third of DOD's planned investment is 
significant and provides insights into DOD's resource allocation 
approach for its major weapon system programs. To gain further insights 
into the impact of DOD's funding process on individual programs, we 
conducted more detailed analysis for 5 high-profile programs, which 
represent the nearly 75 percent of DOD's planned investment in the 20 
programs: Global Hawk, Joint Strike Fighter (JSF), Future Combat System 
(FCS), Warfighter Information Network-Tactical (WIN-T), and Multi- 
mission Maritime Aircraft (MMA).[Footnote 34] We obtained cost and 
baseline data from DOD's selected acquisition reports (SAR) submitted 
through December 2007, FYDP and program funding data from the 
President's Budget exhibits through fiscal year 2008, and program cost 
estimates from the Office of the Secretary of Defense's (OSD) Cost 
Analysis Improvement Group (CAIG) and the service cost analysis centers 
developed to support each program's Milestone B review. Our analysis 
began with cost and funding data from the fiscal year that each program 
entered system development--received Milestone B approval--and ended 
with the data reported in the December 2007 SARs. We also examined and 
compared individual program cost estimates from the CAIG, service cost 
analysis centers, and program offices to determine how they differed 
from current program costs. 

To better understand how DOD funds its weapon system acquisitions and 
to identify the key factors that influence its effectiveness, we 
reviewed DOD Acquisition Regulations, DOD Financial Management 
Regulations, Title 10 of the United States Code, Planning, Programming, 
Budgeting, and Execution guidance from Management Information Directive-
913, as well as individual military service budgeting policies and 
guidance. We also met with knowledgeable officials from the OSD CAIG; 
OSD Program Analysis and Evaluation; the Office of the Under Secretary 
of Defense for Acquisition, Technology and Logistics; the military 
service cost analysis centers; military service planning and 
programming organizations; the Defense Acquisition University; the 
Brookings Institution; the Institute for Defense Analysis; and the 
Naval Postgraduate School. 

To identify proven cost estimating and budgeting processes and 
practices that could be used by DOD to improve its resource allocation 
process, we relied on our prior work in best practices at successful 
commercial companies and reviewed documentation and interviews from 
previous GAO best practices work. We also conducted follow-up 
interviews with key officials from Eli Lilly, IBM, and Motorola 
[Footnote 35] to gain a better understanding of how they prioritize 
projects for funding and ensure that each product receives and 
maintains adequate funding. We also relied on our Cost Assessment 
Guide--which provides a cost-estimating methodology based on best 
practices--and numerous other GAO products, including our extensive 
body of best practices work. 

We conducted this performance audit from June 2007 to May 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Office Of The Under Secretary Of Defense: 
Acquisition, Technology And Logistics: 
3000 Defense Pentagon
Washington, DC 20301-3000 

June 13, 2008: 

Mr. Michael J. Sullivan: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. Sullivan: 

This is the Department of Defense (DoD) response to the GAO draft 
report GAO-08-619, "Defense Acquisitions: A Knowledge-Based Funding 
Approach Could Improve Major Weapon System Program Outcomes," dated May 
15, 2008 (GAO Code 120665). 

The Department partially concurs with two recommendations and 
nonconcurs with the other draft report recommendation. The rationale 
for DoD's position is included in the enclosure. 

The Department appreciates the opportunity to comment on the draft 
report. Technical comments were provided separately. My point of 
contact for this effort is Mr. Joseph A. Alfano, 703-697-3343. 

Sincerely, 

Signed by: 

Dr. Nancy L. Spruill: 
Director: 
Acquisition Resources & Analysis: 

Enclosure: As stated: 

GAO Draft Report Dated May 15, 2008: 
GAO-08-619 (GAO Code 120665): 

"Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve 
Major Weapon System Program Outcomes" 

Department Of Defense Comments To The GAO Recommendations: 

Recommendation 1: The GAO recommends that the Secretary of Defense 
develop and implement a strategy to bring the department's current 
portfolio into balance by aligning the number of programs and the cost 
and schedule of those programs with available resources. In developing 
and implementing a strategy, the Department should determine ways to 
prioritize needs and identify whether the budget and the Future Years 
Defense Program should be increased to more accurately reflect the 
actual costs of current programs or whether the portfolio of current 
programs should be reduced and lower priority programs terminated to 
match available resources. (p. 28/GAO Draft Report) 

DOD Response: Partially concur. The Department goes to great lengths in 
order to develop overall investment strategy and guidance, then flows 
that guidance to the appropriate levels, and then prioritizes 
requirements in order to "align the number of programs and the cost and 
schedule of those programs with available resources." The restructure 
of the Planning, Programming, and Budgeting System into the Planning, 
Programming, Budgeting, and Execution process was one of several 
initiatives the Department has taken towards this end; however, 
positive results from these types of process changes will take years 
and experience across several programs in order to validate their 
positive effects. Yet while the Department has control over its 
programming and budgeting processes (within limits set by the Office of 
Management and Budget) it does not control the amount of Total 
Obligation Authority available over the Future Years Defense Program, 
nor how much funding is appropriated by Congress. At times, these 
external influences will also cause turbulence in program execution. 

Over the last several years the Department has taken on several 
initiatives to address this GAO recommendation. A recent Defense 
initiative known as Capability Portfolio Management is intended to 
provide an enterprise-level, horizontal (cross-component) view of the 
Department to better balance and harmonize the Departments capability 
needs with existing and planned force management and development 
efforts and produce strategically aligned outcomes optimized for the 
enterprise. As of February 2008, Department leadership has established 
nine Capability Portfolio Managers aligned to the nine Tier 1 Joint 
Capability Areas (the Department's capabilities language), formalized 
four and is testing the other five. Capability Portfolio Managers are 
focused on highlighting opportunities to better integrate, coordinate 
and synchronize programs to strategic intent and capability priorities 
within resource and time constraints. 

With respect to budgeting, more realistic, cost-effective plans and 
budgets are goals for the integrated Planning, Programming, Budgeting 
and Execution process. Establishing Capital Accounts will result in 
more stable, predictable acquisition lifecycle management program 
execution, and the process would benefit from this guaranteed funding 
stream. Pilot programs provide valuable lessons learned to apply 
capital accounts to a wide variety of acquisition and sustainment 
programs. Current initiatives include coordination with the Office of 
Management and Budget Program Assessment Rating Tool and establishment 
of an authoritative financial information source by integrating 
transactional-level accounting data. Further, Configuration Steering 
Boards have been implemented across all major programs to limit cost 
and requirements growth through senior level evaluation of trade-offs 
and opportunities to descope programs and seek savings. 

In the oversight of Major Defense Acquisition Programs and the Defense 
Acquisition Board process, multiple initiatives are being implemented 
to improve and standardize the milestone decision process. The Defense 
Acquisition Executive Summaries system is being reengineered to ensure 
visibility for senior leaders. The system will help facilitate 
information sharing, accurate input of data, and tracking of negative 
trends to focus on problem areas. Systems to model time and new 
technology factors are being developed to budget effectively to 
changing needs in a rapidly evolving technological development 
environment. 

Recommendation 2: The GAO recommends that the Secretary of Defense 
require that all new programs have manageable development cycles, 
realistic cost estimates, and have planned and programmed full funding 
for the entire development cycle. (p. 28/GAO Draft Report) 

DOD Response: Partially concur. Current DoD acquisition policy, as 
articulated in DoDI 5000.2, establishes an evolutionary, event-based 
acquisition approach as the Department's preferred strategy for rapid 
acquisition of mature technology for the user. Additionally, current 
policy requires cost estimates and, where required, independent cost 
estimates to be completed at key program milestones and full funding to 
be programmed prior to Milestone B. 

These existing policies are consistent with the GAO's recommendation. 
Additionally, the Department has recently introduced a number of new 
policies that we believe will substantively contribute to more 
predictable program outcomes: 

* Our prototyping and competition policy requires program managers to 
plan for two or more competing teams to produce prototypes of key 
system elements up to or through Milestone B. The intent is to reduce 
technical risk, validate cost estimates, evaluate manufacturing 
processes, and refine requirements. 

* We have established Configuration Steering Boards for all ACAT I 
programs to review proposed requirements changes, a significant 
contributor to increased cost and extended development cycles, and any 
technical configuration changes that have the potential to result in 
increases to cost and or lengthier schedules. Such changes will 
generally be rejected, deferring them to future development increments.
Finally, the Department is considering policy initiatives that are 
designed to reduce program instability and improve schedule and cost 
predictability. These initiatives include: (1) moving the Preliminary 
Design Review before Milestone B, to improve our understanding of cost 
and technical requirements before program initiation; and (2) requiring 
a Milestone Decision Authority-conducted post-Critical Design Review 
Assessment in order to ensure that programs are proceeding through the 
System Development and Demonstration phase consistent with sound 
engineering practice and the approved acquisition program baseline. 

Together we believe these new or pending policies will enhance existing 
policy, and substantively contribute to improved cost estimates, better 
informed decision making and shorter cycle times. 

Recommendation 2 includes a provision that all new development programs 
have "manageable development cycles." This conclusion is based on the 
commercial product development models reviewed by GAO and mentioned in 
the report, specifically those from Eli Lilly, IBM, and Motorola. While 
we generally agree that DoD product development cycles should be 
"manageable," these development cycles may bear little relation to 
those of the three commercial firms cited by GAO in the report. 

The use by GAO of three commercial firm product development models as a 
basis for determining `best practice' in DoD has several flaws. First, 
DoD as a public sector institution, in contrast to a private sector 
firm, has key differences in goals and incentives. All of the DoD 
weapon system programs cited in the report are developed through 
contracts from the public to the private sector, which contain 
significantly different incentives and provisions than the product 
development programs typically undertaken internally by firms in the 
private sector. It may be more appropriate that instead of benchmarking 
DoD against private sector firms, GAO should be benchmarking DoD 
against best practices in the public sector throughout the world. 

Further the commodity classes for comparison to DoD development 
programs for major weapon systems that GAO selected, we believe, are 
not a good comparison. The three firms cited in the report are in the 
pharmaceutical, computer products and services, and the radio and 
cellular telephony sectors. Yet the implication of the report is that 
the development cycles in these product sectors can be directly applied 
to systems like the Joint Strike Fighter program. At a minimum, GAO 
might better examine commercial product developments in comparable 
commodity sectors, such as the commercial aircraft sector, to avoid 
drawing conclusions that may be invalid based on analogies to 
dissimilar commodity sectors. 

In order to more thoroughly review the study, the DoD requested that 
the GAO share its methodology, raw data, interview notes, detailed 
analyses, working papers, etc. Although to date the GAO has not 
provided the aforementioned materials, the Department would be pleased 
to work with GAO to better understand the details of such a comparison. 

Recommendation 3: The GAO recommends that the Secretary of Defense 
require all cost estimates submitted for funding a program at Milestone 
decisions to be reported as a range of likely costs and the associated 
levels of risk and uncertainty. At Milestone A, require estimates that 
allow for a wide range of likely costs. At Milestone B, require 
estimates that, based on knowledge gained, are more precise--in line 
with best practice standards. (p. 28/GAO Draft Report) 

DOD Response: Non-concur. While we and the DoD Cost Analysis 
Improvement Group (CRAIG) agree with the spirit of the recommendation 
to consider risks at key milestones in development programs; however 
the GAO recommendation implies that the DoD should calculate a range of 
costs and associated confidence levels using Monte Carlo analysis 
techniques. The CAIG has observed limitations to this analysis 
methodology when used in practice: Most mathematical implementations of 
this methodology assume no correlation of work breakdown structure 
elements in developing distributions of potential cost outcomes. In 
most cases this simplifying assumption is incorrect, particularly for 
cost estimates developed for product-oriented work breakdown 
structures, as are commonly used in the DoD. This assumption results in 
calculated distributions of potential cost outcomes that are "too 
narrow" relative to the actual distribution of cost outcomes, providing 
misleading results. 

The CAIG does routinely calculate cost risks for major milestone 
reviews by examining the key structural elements of cost estimates, 
including schedule durations, technical risks, contract vehicles, 
incentives, and management structures. In certain situations, the CAIG 
has reported a range of life-cycle costs because of the poor quality of 
the definition of the program to be undertaken in the Cost Analysis 
Requirements Description document. For example, the CAIG reported a 
likely range of Future Combat Systems (FCS) development costs on 
several occasions, a fact that GAO did not include in its report. In 
summary, the CAIG tailors the risk analysis to the specific program and 
milestone at hand, calculating and presenting ranges of costs when it 
is appropriate to do so. The specification. of a single methodology and 
practice to be used at each and every milestone review of a Major 
Defense Acquisition Program is not appropriate. 

In the GAO report, Table 1, Development Costs Estimates and Baselines 
for 20 Major Weapon System Programs, contains information on 
development cost estimates and cost baselines. Unfortunately, the 
report does not consider causal factors bearing on the estimates: (1) 
methodological issues, (2) incomplete, errorful, or volatile 
requirements or program specification, (3) engineering change 
proposals, (4) Congressionally mandated changes in program funding 
levels or unit quantities, etc. Additionally, the report does not 
specify the metrics whereby a cost estimate is considered as 
acceptable. 

In order to assist in maintaining an awareness of overall risk with its 
acquisition programs, the Air Force is evaluating the idea of setting a 
goal of annual updates to program cost estimates for major defense 
acquisition programs, versus the DoD 5000 series requirement to update 
for each milestone review. This should bring newly-acquired knowledge 
to bear more quickly and bring problems to light in a more timely 
manner. 

[End of section] 

Related GAO Products: 

Defense Acquisitions: Assessments of Selected Weapon Programs. 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-467SP]. 
Washington, D.C.: March 31, 2008. 

Best Practices: Increased Focus on Requirements and Oversight Needed to 
Improve DOD's Acquisition Environment and Weapon System Quality. 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-294], Washington, 
D.C.: February 1, 2008. 

Cost Assessment Guide: Best Practices for Estimating and Managing 
Program Costs. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-
1134SP], Washington, D.C.: July 2007. 

Defense Acquisitions: Realistic Business Cases Needed to Execute Navy 
Shipbuilding Programs. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-943T], Washington, D.C.: July 24, 2007. 

Best Practices: An Integrated Portfolio Management Approach to Weapon 
System Investments Could Improve DOD's Acquisition Outcomes. 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388], Washington, 
D.C.: March 30, 2007. 

Space Acquisitions: DOD Needs to Take More Action to Address 
Unrealistic Initial Cost Estimates of Space Systems. [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-96], Washington, D.C.: 
November 17, 2006. 

Defense Acquisitions: Major Weapon Systems Continue to Experience Cost 
and Schedule Problems under DOD's Revised Policy. [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-06-368]. Washington, D.C.: April 
13, 2006. 

Defense Acquisitions: Actions Needed to Get Better Results on Weapons 
Systems Investments. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-
06-585T]. Washington, D.C.: April 5, 2006. 

Best Practices: Better Support of Weapon System Program Managers Needed 
to Improve Outcomes. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-
06-110]. Washington, D.C.: November 30, 2005. 

Future Years Defense Program: Actions Needed to Improve Transparency of 
DOD's Projected Resource Needs. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-514]. Washington, D.C.: May 7, 2004. 

Best Practices: Capturing Design and Manufacturing Knowledge Early 
Improves Acquisition Outcomes. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-02-701]. Washington, D.C.: July 15, 2002. 

Best Practices: Better Matching of Needs and Resources Will Lead to 
Better Weapon System Outcomes. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-01-288]. Washington, D.C.: March 8, 2001. 

Footnotes: 

[1] GAO, Best Practices: An Integrated Portfolio Management Approach to 
Weapon System Investments Could Improve DOD's Acquisition Outcomes. 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388], Washington, 
D.C.: March 30, 2007. 

[2] Global Hawk, an Air Force unmanned aircraft system, is intended to 
provide intelligence, surveillance, and reconnaissance capabilities; 
JSF is a joint Air Force, Navy, and Marine Corps program to develop and 
field stealthy fighter aircraft to replace DOD's aging fighter and 
attack aircraft; FCS is an Army program intended to provide advanced, 
networked combat and sustainment systems, unmanned ground and air 
vehicles, and unattended sensors and munitions; WIN-T is intended to 
provide the Army with a high-speed, high-capacity communications 
network; MMA is a Navy program intended to provide persistent 
antisubmarine and antisurface warfare, and intelligence, surveillance, 
and reconnaissance capabilities. 

[3] GAO, Cost Assessment Guide: Best Practices for Estimating and 
Managing Program Costs. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-1134SP] (Washington, D.C.: July 2007). 

[4] Full funding is a DOD 5000.2 requirement for formal program 
initiation of an acquisition program. In this sense, full funding means 
having an approved current (and projected) resource stream to execute 
the acquisition program; that is, program funding is included both in 
the budget and in the out-years of the Future Years Defense Program 
sufficient to cover the current and future efforts described in the 
acquisition strategy. 

[5] DOD modified the PPBS process, renamed the PPBE, in 2003. PPBE in 
its entirety is not implemented every year even though DOD must request 
funding from congress annually. Full-scale planning and programming 
activities are conducted in even-numbered years (called on-years) while 
programming adjustments are made in odd-numbered years (called off- 
years). As a result of this modification, the number of years covered 
by the FYDP now alternates between 5 years (the President's budget plus 
4 years) and 6 years (the President's budget plus 5 years). 

[6] The Milestone Decision Authority for all Major Defense Acquisition 
Programs is the Under Secretary of Defense for Acquisition, Technology 
& Logistics unless delegated to the head of a DOD component. 

[7] Although DOD policy requires full funding within the FYDP, most 
major weapon system acquisition programs have development cycles that 
extend beyond the FYDP time period. However, there is no requirement to 
constrain development cycles to fit within that same time period. 

[8] U.S.C. Title 10 ï¿½ 2366(a), which was enacted in 2006, specifies 
that a major defense acquisition program may not receive Milestone B 
approval, or Key Decision Point B approval in the case of a space 
program, until the Milestone decision authority certifies that--(1) the 
program technology has been demonstrated in a relevant environment; (2) 
the program demonstrates a high likelihood of accomplishing its 
mission; (3) the program is affordable when considering the per unit 
cost and the total acquisition cost in the context of the total 
resources available during the period covered by the future-years 
defense program submitted during the fiscal year in which the 
certification is made; (4) the department has completed an analysis of 
alternatives; (5) the program is affordable when considering the 
ability of DOD to accomplish the program's mission using alternative 
systems; (6) the Joint Requirements Oversight Council has accomplished 
its duties with respect to the program pursuant to section 181 (b) of 
this title, including an analysis of the operational requirements for 
the program; and (7) the program complies with all relevant policies, 
regulations, and directives of DOD. 

[9] Programs are not required to hold a Milestone A review, the point 
at which concept refinement ends and technology development begins. 
However, in 2007, Congress enacted legislation, which specifies that a 
major defense acquisition program beginning after March 2008 may not 
receive Milestone A approval, to begin a technology development 
program, until the Milestone Decision Authority certifies to Congress 
that (1) the system fulfills an approved initial capabilities document; 
(2) the system is being executed by an entity with a relevant core 
competency as identified by the Secretary of Defense; (3) if the system 
duplicates a capability already provided by an existing system, the 
duplication provided by such system is necessary and appropriate; and 
(4) a cost estimate for the system has been submitted. DOD is currently 
revising its policy and guidance for conducting and certifying 
Milestone A reviews. 

[10] See Related GAO Products at the end of this report. 

[11] The funding needed within the FYDP for each program was calculated 
using the funding data contained in the RDT&E Annual Funding Summary of 
the first SAR issued after system development began. The amount of 
funding established in the FYDP was calculated using each program's 
RDT&E budget justification documentation for the year in which system 
development was initiated. 

[12] DOD has a formal process for reprogramming appropriated funds, 
including if necessary, congressional notification and approval. 

[13] GAO, Space Acquisitions: DOD Needs to Take More Action to Address 
Unrealistic Initial Cost Estimates for Space Systems. [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-96], Washington, D.C.: 
November 17, 2006. 

[14] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388]. 

[15] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388]. 

[16] Assessment Panel of the Defense Acquisition Performance Assessment 
Project for the Deputy Secretary of Defense, Defense Acquisition 
Performance Assessment Report (January 2006); Defense Science Board 
Summer Study on Transformation: A Progress Assessment (February 2006); 
RAND, Historical Cost Growth of Completed Weapon System Programs 
(2006); Institute for Defense Analysis, Costs Growth in Major Weapon 
Procurement Programs, Presentation (2005). 

[17] Defense Science Board, Report of the Defense Science Board /Air 
Force Advisory Board, Joint Task Force on Acquisition of National 
Security Space Programs, May 2003 

[18] Systems engineering is a technical management tool that provides 
the knowledge necessary to translate requirements into specific, 
achievable capabilities. By using the tools of systems engineering 
during the early phases of concept refinement and technology 
development, acquisition decision makers and developers can work 
together to close gaps between requirements and available resources 
well before system development starts. 

[19] An AOA should compare the costs and benefits of alternative 
solutions for meeting a validated need. As such an AOA should assess 
the life-cycle cost, schedule, and operational effectiveness of each 
possible solution and identify a preferred alternative. 

[20] GAO, Defense Acquisitions: Assessments of Selected Major Weapon 
Programs. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-467SP]. 
Washington, D.C.: March 31, 2008; Defense Acquisitions: Major Weapon 
Systems Continue to Experience Cost and Schedule Problems under DOD's 
Revised Policy. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-
368]. Washington, D.C.: April 13, 2006. 

[21] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-368] and 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388]. 

[22] Institute for Defense Analysis, Future Combat Systems (FCS) Cost 
Review: Summary of Findings, P-4212, April 2007, p. S-2. 

[23] GAO, Tactical Aircraft: Opportunity to Reduce Risks in the Joint 
Strike Fighter Program with Different Acquisition Strategy. [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-05-271] (Washington, D.C.: March 
15, 2005). 

[24] GAO, Space Acquisitions: DOD Needs to Take More Action to Address 
Unrealistic Initial Cost Estimates for Space Systems. [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-96] (Washington, D.C.: Nov. 
17, 2006). 

[25] GAO, Defense Acquisitions: Realistic Business Cases Needed to 
Execute Navy Shipbuilding Programs. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-943T], Washington, D.C.: July 24, 2007. 

[26] For management to make good decisions about programs, our Cost 
Assessment Guide calls for a quantitative risk and uncertainty analysis 
to assess variability in an estimate. 

[27] A Defense Science Board report recommended funding space 
acquisition programs at the 80-percent confidence level (Report of the 
Defense Science Board/Air Force Advisory Board, Joint Task Force on 
Acquisition of National Security Space Programs, May 2003); and 
subsequently the Air Force issued guidance in March 2007 stating that 
an 80-percent confidence level is the objective for space programs, but 
it is too soon to assess whether this policy has resulted in more 
realistic estimates. 

[28] GAO, Future Years Defense Program: Risks in Operation and 
Maintenance and Procurement Programs. [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-01-33] (Washington, D.C.: October 
5, 2000); Best Practices: An Integrated Portfolio Management Approach 
to Weapon System Investments Could Improve DOD's Acquisition Outcomes. 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388] (Washington, 
D.C.: March 30, 2007). 

[29] GAO, Best Practices: Better Support of Weapon System Program 
Managers Needed to Improve Outcomes, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-06-110] (Washington, D.C.: November 30, 2005). 

[30] See Related GAO Products at the end of this report. 

[31] IBM's contracts are not focused solely on program costs; they also 
establish goals, objectives, and allowable deviations for other program 
measures such as schedule and revenue. 

[32] Assessment Panel of the Defense Acquisition Performance Assessment 
Project for the Deputy Secretary of Defense, Defense Acquisition 
Performance Assessment Report (Jan. 2006). 

[33] Defense Acquisitions: Assessments of Selected Major Weapon 
Programs. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-467SP]. 
Washington, D.C.: March 31, 2008. 

[34] Global Hawk, an Air Force unmanned aircraft system, is intended to 
provide intelligence, surveillance, and reconnaissance capabilities; 
JSF is a joint Air Force, Navy, and Marine Corps program to develop and 
field stealthy fighter aircraft to replace DOD's aging fighter and 
attack aircraft; FCS is an Army program intended to provide advanced, 
networked combat and sustainment systems, unmanned ground and air 
vehicles, and unattended sensors and munitions; WIN-T is intended to 
provide the Army with a high-speed, high-capacity communications 
network; MMA is a Navy program intended to provide persistent 
antisubmarine and antisurface warfare, and intelligence, surveillance, 
and reconnaissance capabilities. 

[35] Eli Lilly is one of the largest corporations in the world and 
engages in pharmaceutical research and development around the world; 
IBM is the largest supplier of hardware, software, and information 
technology services; and Motorola is a Fortune 100 global 
communications leader. We also contacted the other two companies that 
provided input to our prior review--Proctor & Gamble and Caterpillar-- 
but relevant officials were not available to meet with us for follow-up 
discussions during the timeframes of our review. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***