Defense Acquisitions: Major Weapon Systems Continue to Experience
Cost and Schedule Problems under DOD's Revised Policy (14-APR-06,
GAO-06-368).
The Department of Defense (DOD) is planning to invest $1.3
trillion between 2005 and 2009 in researching, developing, and
procuring major weapon systems. How DOD manages this investment
has been a matter of congressional concern for years. Numerous
programs have been marked by cost overruns, schedule delays, and
reduced performance. Over the past 3 decades, DOD's acquisition
environment has undergone many changes aimed at curbing cost,
schedule, and other problems. In order to determine if the policy
DOD put in place is achieving its intended goals, we assessed the
outcomes of major weapons development programs initiated under
the revised policy. Additionally, we assessed whether the
policy's knowledge-based, evolutionary principles are being
effectively implemented, and whether effective controls and
specific criteria are in place and being used to make sound
investment decisions.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-368
ACCNO: A51657
TITLE: Defense Acquisitions: Major Weapon Systems Continue to
Experience Cost and Schedule Problems under DOD's Revised Policy
DATE: 04/14/2006
SUBJECT: Cost overruns
Defense cost control
Defense procurement
Internal controls
Military policies
Performance measures
Policy evaluation
Procurement planning
Procurement policy
Schedule slippages
Weapons research and development
Weapons systems
Investment management
Army Future Combat Systems
Joint Strike Fighter program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-368
* Knowledge-Based Process Not Enforced
* Evolutionary Acquisition Is Not Being Used
* Order by Mail or Phone
United States Government Accountability Office
Report to Congressional Committees
GAO
April 2006
DEFENSE ACQUISITIONS
Major Weapon Systems Continue to Experience Cost and Schedule Problems under
DOD's Revised Policy
GAO-06-368
DEFENSE ACQUISITIONS
Major Weapon Systems Continue to Experience Cost and Schedule Problems
under DOD's Revised Policy
What GAO Found
Changes made in DOD's acquisition policy over the past 5 years have not
eliminated cost and schedule problems for major weapons development
programs. Of the 23 major programs we assessed, 10 are already expecting
development cost overruns greater than 30 percent or have delayed the
delivery of initial operational capability to the warfighter by at least 1
year. The overall impact of these costly conditions is a reduction in the
value of DOD's defense dollars and a lower return on investment. The
following table illustrates the problem.
Cost and Schedule Outcomes Sorted by Percent of Product Development Remaining
Percent cost Programs growtha
Percent of Schedule growth, development in months remaining
Aerial Common Sensor 45% 24 85%
Future Combat System 48% 48 78%
Joint Strike Fighter 30% 23 60%
Expeditionary Fighting Vehicle 61% 48 49%
C-130 Avionics Modernization Program 122% Delays anticipated Undetermined
Global Hawk (RQ-4B) 166% Delays anticipated Undetermined
Sources: DOD (data); GAO (analysis and presentation).
a
Cost growth is expressed as the percent change in program development cost
estimates in 2005 base year dollars.
Poor execution of the revised acquisition policy is a major cause of DOD's
continued problems. DOD frequently bypasses key steps of the
knowledgebased process outlined in the policy, falls short of attaining
key knowledge, and continues to pursue revolutionary-rather than
evolutionary or incremental-advances in capability. Nearly 80 percent of
the programs GAO reviewed did not fully follow the knowledge-based process
to develop a sound business case before committing to system development.
Most of the programs we reviewed started system development with immature
technologies, and half of the programs that have held design reviews did
so before achieving a high level of design maturity. These practices
increase the likelihood that problems will be discovered late in
development when they are more costly to address. Furthermore, DOD's
continued pursuit of revolutionary leaps in capability also runs counter
to the policy's guidance.
DOD has not closed all of the gaps in the policy that GAO identified
nearly 3 years ago, particularly with regard to adding controls and
criteria. Effective controls require decision makers to measure progress
against specific criteria and ensure that managers capture key knowledge
before moving to the next acquisition phase. However, DOD's policy
continues to allow managers to approach major investment decisions with
many unknowns. Without effective controls that require program officials
to satisfy specific criteria, it is difficult to hold decision makers or
program managers accountable to cost and schedule targets. In this
environment, decisionmaking transparency is crucial, but DOD is lacking in
this area as well.
United States Government Accountability Office
Contents
Abbreviations
DOD Department of Defense GAO Government Accountability Office
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
April 13, 2006
The Honorable John Ensign Chairman The Honorable Daniel K. Akaka Ranking
Minority Member Subcommittee on Readiness and Management Support Committee
on Armed Services United States Senate
The Honorable Duncan L. Hunter Chairman Committee on Armed Services House
of Representatives
DOD's planned investment in research, development, and procurement of
major weapon systems will total approximately $1.3 trillion between 2005
and 2009, with over $800 billion of that investment yet to be made. DOD is
facing a significant number of problems in managing its acquisitions.
Military operations in Afghanistan and Iraq are consuming a large share of
DOD resources and causing the department to invest more money sooner than
expected to replace or fix existing weapons. Meanwhile, DOD is intent on
transforming military operations while pursuing multiple megasystems that
are expected to be the most expensive and complex ever. These costly
conditions coupled with increases in spending for other national
priorities, such as health care and social security, make it essential
that DOD effectively leverage its investments, particularly in weapon
system acquisitions. If DOD manages its current portfolio of weapons
within traditional margins of error, the financial consequences could be
dire.
DOD's strategy for acquiring major weapon systems has traditionally been
to plan programs that would achieve a big leap forward in capability
within a single development program, a strategy that often results in
major cost and schedule problems. We have assessed weapon acquisitions as
a high-risk area for 15 years, and although U.S. weapons are among the
best in the world, the programs to acquire them have continued to produce
poor cost and schedule outcomes. However, the current defense acquisition
environment continues to be characterized by cost and schedule growth, a
lack of confidence by congressional and DOD leaders, and no appreciable
improvement in the defense acquisition system. DOD
Results in Brief
knows what to do to achieve better outcomes. It has written into policy an
approach that advocates that adequate knowledge be attained at critical
junctures before DOD managers agree to invest more money in the next phase
of weapon system development. The policy also emphasizes evolutionary
principles for acquiring weapons rather than trying to achieve a big leap
forward in capability within a single development program. We have
reported in the past that DOD's revised policy does not incorporate
adequate controls to ensure the effective implementation of a
knowledgebased, evolutionary acquisition process. However, DOD believes
that the policy includes the necessary controls to achieve effective
outcomes.
You requested that we evaluate DOD's compliance with and implementation of
its revised acquisition policy intended to produce better cost, schedule,
and performance outcomes for major acquisition programs. In order to
obtain an early assessment of the cost and schedule impact of the revised
policy, and to assess DOD's effectiveness in implementing a
knowledge-based, evolutionary acquisition approach we assessed (1) the
cost and schedule status of major weapons development programs initiated
under the revised policy, (2) whether the policy's knowledge-based,
evolutionary acquisition principles are being effectively implemented, and
(3) whether effective controls and specific criteria are in place and
being used to make sound investment decisions.
In conducting our evaluation, we reviewed pertinent acquisition statutes,
policies, and guidance; analyzed development cost and schedule data for 23
major acquisition programs approved to start system development under
DOD's revised acquisition policy between October 2000 and December 2004;
conducted case study reviews of nine of those 23 programs; and interviewed
officials from the Office of Secretary of Defense and each of the military
services. We conducted our review from May 2005 to February 2006 in
accordance with generally accepted government auditing standards.
Additional information about our methodology is contained in appendix I.
DOD's revised acquisition policy has not led to improved acquisition
program outcomes. Programs initiated under the revised policy are already
beginning to experience cost and schedule problems similar to programs
managed under prior versions of the policy. Although the programs we
reviewed have been in development for only a short period of time, nearly
half are already estimating development cost growth greater than 30
percent or are expecting to delay initial delivery to the warfighter by at
least 1 year. Program officials are facing the familiar predicament of
having to add unplanned money or time or to reduce system capabilities and
quantities after initial business cases have been approved and system
development has begun. As a result, DOD is reducing its buying power and
is not achieving the return on investment it expected when the programs
began.
Acquisition officials are not effectively implementing the revised
acquisition policy's knowledge-based process. They regularly bypass key
phases of the early acquisition process, approach key decision points with
limited knowledge about critical technologies and system design, and do
not employ evolutionary acquisition principles. Nearly 80 percent of the
programs we assessed were permitted to bypass the policy's initial major
decision review and the associated systems-engineering process that are
intended to ensure that a system's requirements match available resources
and that a sound business case is developed prior to starting system
development. By not consistently following key processes and strategies,
acquisition officials are not ensuring that a solid foundation of
knowledge about cost, schedule, and performance is established before
allowing programs to start system development, thus resulting in
unexecutable business cases. Although the policy explicitly states that
programs shall increase program knowledge by maturing technologies before
beginning system development, we found that almost three-fourths of the
programs started since the policy was revised began development with
immature critical technologies. Our analysis also indicates that decision
makers are continuing to commit programs to system demonstration and
initial manufacturing before officials have demonstrated high levels of
design knowledge, as emphasized in the policy. In addition, programs like
the Joint Strike Fighter and Future Combat System are still structured to
achieve major leaps in capability within a single development program, a
strategy that has historically proven to be problematic in terms of cost
and schedule outcomes.
Effective implementation of the revised policy is limited by the absence
of effective controls that require compliance and specific criteria for
clearly demonstrating that acceptable levels of knowledge about
technology, design, and manufacturing have been attained at critical
junctures during system development before making further investments in a
program. Without effective controls, the policy cannot prevent DOD
decision makers from starting system development even when they face
significant unknowns about technology, design, and production. Without
specific criteria-or standards against which a judgment or decision is
quantifiably based-decision makers are permitted to make decisions on the
basis of subjective judgment. We reported this condition in 2003, yet DOD
has not
Background
closed gaps in the policy. In the absence of such controls and criteria,
DOD faces the added problems of transparency and accountability because it
often does not sufficiently document the rationale for its decisions to
allow acquisition programs to advance with low levels of technology,
design, and manufacturing knowledge.
This report contains recommendations that DOD require programs to meet
specific knowledge-based criteria at each key decision points in the
acquisition process and require decision makers to provide clear and
specific rationale for their decisions. In addition, we recommend that
before programs enter system development they should be required to
complete disciplined concept and technology development phases that
include specific activities dedicated to capturing knowledge critical to
developing an executable business. DOD partially concurred with our
recommendations. DOD agrees that knowledge-based decision making is
consistent with sound business practice and stated that it would continue
to develop policy that reflects a knowledge-based approach and improves
acquisition outcomes. DOD also agrees that acquisition decisions should be
documented, decision makers should be held accountable, and that they
should provide the rationale for their decisions.
Historically, DOD's programs for acquiring major weapon systems have taken
longer, cost more, and often delivered fewer quantities and other
capabilities than planned. GAO has documented these problems for decades.
In 1970, GAO reported that considerable cost growth had been and was
continuing to occur on many current development programs. Since that
report was issued, numerous changes have been made to DOD's acquisition
process and environment to try to improve acquisition outcomes. Those
changes include numerous executive branch initiatives and legislative
actions as well as roughly 11 revisions to DOD's acquisition policy
between 1971 and 2005. Despite these efforts, defense acquisition programs
in the past 3 decades continued to routinely experience cost overruns,
schedule slips, and performance shortfalls.
Figure 1 illustrates the continued problem of development cost overruns.
The figure depicts the combined cost overruns for large development
programs (programs totaling more than $1 billion for research,
development, testing and evaluation in fiscal year 2005 dollars) in each
of the past 3 decades. The figure also identifies some of the major
studies and improvement efforts initiated during this time frame. As the
figure illustrates, efforts to improve acquisition outcomes have not been
successful in curbing acquisition cost problems. Programs initiated in the
1970s exceeded DOD's initial investment estimate by 30 percent, or $13
billion (in fiscal year 2005 dollars), and similar outcomes continued
during the subsequent decades despite numerous reform efforts and policy
revisions.
Figure 1: Development Cost Overruns by Decade (in Fiscal Year 2005 Dollars) and
Key Reform Efforts
Source: DOD (data); GAO (analysisand presentation).
Since the mid-1990s, we have studied the best practices of leading
commercial companies. Taking into account the differences between
commercial product development and weapons acquisitions, we articulated a
best practices product development model that relies on increasing
knowledge when developing new products, separating technology development
from product development, and following an evolutionary or incremental
product development approach. This knowledge-based approach requires
developers to make investment decisions on the basis of specific,
measurable levels of knowledge at critical junctures before investing more
money and before advancing to the next phase of acquisition. An
evolutionary product development process defines the individual increments
on the basis of mature technologies and a feasible design that are matched
with firm requirements. Each increment should be managed as a separate and
distinct acquisition effort with its own cost, schedule and performance
baseline. An increment that excludes one of these key elements puts an
extra burden on decision makers and provides a weak foundation for making
development cost and schedule estimates. The knowledge-based, evolutionary
approach in our model is intended to help reduce development risks and to
achieve better program outcomes on a more consistent basis.
Hoping to improve acquisition outcomes, DOD leaders initiated significant
revisions to the department's acquisition policy again in October 2000, by
adopting the knowledge-based, evolutionary system development approach. 1
We reported in November 2003, that much of the revised policy agrees with
GAO's extensive body of work and that of successful commercial firms.
DOD's revised policy emphasizes the importance of and provides a good
framework for capturing knowledge about critical technologies, product
design, and manufacturing processes. If properly implemented and enforced
this approach could help DOD's decision makers gain the confidence they
need to make significant and sound investment decisions for major weapon
systems. Furthermore, the policy's emphasis on evolutionary system
development sets up a more manageable environment for achieving knowledge.
We also noted that DOD's policy strongly suggests the separation of
technology development from system development, a best practice that helps
reduce technological risk at the start of a program and makes cost and
delivery estimates much more predictable. 2 Figure 2 depicts in general
how DOD's revised policy adopts key aspects of the best practices model.
In addition to the acquisition policy, the process used by DOD to
establish program funding, known as the Planning, Programming, Budgeting
and Execution System (PPBES), and the process used to determine system
requirements, now called the Joint Capabilities Integration and
Development System (JCIDS), also impact program outcomes. Both processes
are currently being studied to determine if any changes could be made to
improve program performance as it relates to funding and requirements.
2GAO. Defense Acquisitions: DOD's Revised Policy Emphasizes Best
Practices, but More Controls Are Needed, GAO-04-53 ( Washington, D.C.:
Nov. 10, 2003).
Figure 2: Comparison of DOD's Revised Policy and Commercial Best Practices Model
Source: DOD (data); GAO (analysisand presentation).
Although DOD took significant steps in the right direction, its policy
does not include controls that require program officials to meet the key
criteria that we believe are necessary for ensuring that acceptable levels
of knowledge are actually captured before making additional significant
investments. We previously recommended that DOD design and implement
necessary controls to ensure that appropriate knowledge is captured and
used to make decisions about moving a program forward and investing more
money at critical junctures. DOD officials acknowledged the advantages of
using knowledge-based controls, but stated that they believed the policy
already included enough controls to achieve effective program results. The
officials agreed to monitor the acquisition process to assess the
effectiveness of those controls and to determine whether additional ones
are necessary.
DOD's Revised Policy Has Not Improved Development Program Outcomes
The cost and schedule outcomes being achieved by development programs
initiated since DOD first issued its revised policy have not improved over
those achieved by programs managed under prior versions of the policy. Of
the 23 major programs we assessed, 10 have already reported estimated
development cost growth greater than 30 percent or expected delays of at
least 1 year in delivery of an initial operational capability to the
warfighter. These programs combined represent a cost increase of $23
billion (in fiscal year 2005 dollars) and an average delay in delivery of
initial capability of around 2 years. Most of the other programs were
still in the early stages as of December 2005 with over half of system
development remaining and had not yet reported an adequate amount of cost
or schedule data to effectively analyze their progress. Table 1 contains
the cost and schedule increases for the 23 programs we assessed, expressed
as a percentage of each program's development estimate.
Table 1: Cost andSchedule Outcomes for 23 Programs Initiated under the Revised
Policy (as of December 2005)
Percent growth in estimated Percent growth in estimated Program
development costa development schedule
Expeditionary Fighting Vehicle 61% 70%
Active Electronically Scanned Array radar (upgrade for F/A-18 E/F
fighter/attack aircraft) 14% 1%
Global Hawk unmanned aerial vehicle 166% Undetermined
Joint Strike Fighter 30% 23%
UH-60M helicopter upgrade 151% 25%
C-130 Avionics Modernization Program 122% Undetermined
C-5 Reliability Enhancement and Re-engining Program 0% 25%
Joint Tactical Radio System Cluster 1 31% 44%
Joint Tactical Radio System Waveform 44% Undetermined
Advanced Anti-radiation Guided Missile 7% 0%
Multi-Platform Radar Technology Insertion Program 0% Undetermined
Future Combat System 48% 53%
E-2 Advanced Hawkeye 5% 0%
Warfighter Information Network-Tactical 0% 0%
Small Diameter Bomb 0% 0%
EA-18G 7% 0%
Joint Tactical Radio System Cluster 5 0% 2%
Multi-Mission Maritime Aircraft 0% 0%
Standard Missile-6 Extended Range Active Missile Block 1 0% 0%
Aerial Common Sensor 45% 36%
B-2 Radar Modernization Program 0% 0%
Patriot/Medium Extended Air Defense System Combined Aggregate Program
(fire unit) 0% 0%
Mission Planning System 0% 0%
Sources: DOD (data); GAO (analysis and presentation).
a
Cost growth is expressed as the percent change in program development cost
estimates in fiscal year 2005 dollars.
The Army's Future Combat System is a case in point. Less than 3 years
after program initiation and with $4.6 billion invested, the Army has
already increased its development cost estimate $8.9 billion or 48 percent
and delayed delivery of initial capability by 4 years over the original
business case. Similarly, just over 1 year after initiating development of
the Aerial Common Sensor aircraft, the Army has reported that severe
weight and design problems discovered during development have stopped work
on the program. As a result, program officials are anticipating at least a
45 percent cost increase and a delay of 2 years in delivering an initial
capability to the warfighter. These two Army programs are not the only
ones experiencing problems. Table 2 contains cost and schedule data for 6
of the 10 largest development programs initiated under the revised policy,
including the Future Combat System and Aerial Common Sensor. As the table
illustrates there are several programs experiencing large cost increases
and schedule delays.
Table 2: Cost andSchedule Outcomes for 6 of the 10 Largest Development
Programs Sorted by Percent of System Development Remaining
Percent development cost Delay in delivery of initial Percent of
development Programs growth capability in months remaining
Aerial Common Sensor 45% 24 85%
Future Combat System 48% 48 78%
Joint Strike Fighter 30% 23 60%
Expeditionary Fighting Vehicle 61% 48 49%
C-130 Avionics Modernization 122% Delays anticipated due to Undetermined
due to program Program program restructure restructure
Global Hawk (RQ-4B) 166% Delays anticipated due to Undetermined due to
program program restructure restructure
Sources: DOD (data); GAO (analysis and presentation).
A good measure of acquisition performance is return on investment as
expressed in acquisition program unit cost because unit cost represents
the value DOD is getting for its acquisition dollars invested in a certain
program. The programs listed in table 2 will not achieve the return on
investment that DOD anticipated when they began development. In the case
of Joint Strike Fighter, for example, DOD initially intended to purchase
2,866 aircraft at an acquisition program unit cost of about $66 million.
The Navy has reduced the number of Joint Strike Fighter aircraft it plans
to buy; technology and design problems encountered during development have
led to the significant cost growth. As a result, the acquisition program
unit cost is now about $84 million, an increase of 27 percent. We recently
reported that the risk of even greater increases is likely because flight
testing has not yet started and the acquisition strategy involves
substantial overlap of development and production. Similar problems have
led to increases in the Future Combat System program. At program
initiation, the Army anticipated that each of 15 units would cost about
$5.5 billion to develop and deliver. Since that time, instability in the
program's technologies and requirements have led to significant cost
increases, leading to a 54 percent increase in acquisition program unit
cost, now estimated to be $8.5 billion.
Regarding all 23 development programs, DOD leaders originally planned to
invest a total of about $83 billion (fiscal year 2005 dollars) for system
development and anticipated delivering an initial operational capability
to the warfighter in 77 months on average. However, development costs have
grown and delivery schedules have been delayed significantly. DOD now
expects to invest over $106 billion in those same programs, an increase of
over $23 billion or 28 percent. The delivery of initial capability to the
warfighter is expected to take an average of 88 months or nearly 1 year
longer than originally planned. Figure 3 shows changes in these business
case elements for these programs in the short time since their initiation.
Figure3: Cost andSchedule Growth under DOD's Revised Policy
DOD Is Not Effectively Implementing the Policy's Knowledge-Based, Evolutionary
Approach
DOD is not effectively implementing the knowledge-based process and
evolutionary approach emphasized in its acquisition policy. While the
policy outlines a specific knowledge-based process of concept refinement
and technology development 3 to help ensure a sound business case is
developed before committing to a new development program, almost 80
percent of the programs we reviewed were permitted to bypass this process.
Furthermore, the policy emphasizes the need to mature all critical
technologies before starting system development and to demonstrate that
the product's design is mature before beginning system demonstration.
However, nearly three-fourths of the programs reported having immature
critical technologies when they received approval to start development,
and at least half of the programs had not achieved design maturity before
holding their design review and gaining approval to enter the system
demonstration phase of development. The policy also emphasizes the use of
an evolutionary product development approach, yet program officials
continue to structure major acquisition programs to achieve large advances
in capability within a single step development program. This strategy has
historically resulted in poor cost and schedule outcomes.
Knowledge-Based Process Not Enforced
DOD decision makers continue to approve programs for system development
that have not followed key elements of the policy's suggested
knowledge-based process. The policy requires program managers to provide
senior decision makers with knowledge about key aspects of a system at
critical investment points in the acquisition process. Our prior reviews
have identified those critical points as the start of system development
or program start (referred to as Milestone B in the DOD acquisition
policy), design readiness review separating system integration and system
demonstration, and production commitment (Milestone C in the DOD
acquisition policy). The most important point occurs at program start,
when system development begins. DOD acquisition guidance emphasizes the
importance of the acquisition phases preceding program start, noting that
the decisions made during those phases-concept
3
According to DOD Instruction 5000.2, the concept refinement phase is
intended to refine the initial concept and develop a technology
development strategy. Concept refinement ends when the decision authority
approves a preferred solution resulting from the analysis of alternatives
and approves the associated technology development strategy. After concept
refinement, a project enters technology development at Milestone A, when
the decision maker has approved the technology development strategy. The
purpose of this phase is to reduce technology risk and to determine the
appropriate set of technologies to be integrated into a full system.
Acquisition Officials Are Not Effectively Using Early Processes to Develop
Executable Business Cases
refinement and technology development-generally define the nature of an
entire acquisition program.
Acquisition officials continue to begin system development without
following early processes for developing executable business cases. A
business case should provide demonstrated evidence that (1) the
warfighter's needs are real and necessary and that they can best be met
with the chosen concept and (2) the chosen concept can be developed and
produced within existing resources-including technologies, design
knowledge, funding, and the time to deliver the product when it is needed.
Establishing a business case calls for a realistic assessment of risks and
costs; doing otherwise undermines the intent of the business case and
invites failure. This process requires the user and developer to negotiate
whatever trade-offs are needed to achieve a match between the user's
requirements and the developer's resources before system development
begins.
The revised policy and associated guidance emphasize the importance of
following a sound process of systems engineering 4 and decision making
prior to initiating a system development program. The process established
in the policy consists of two phases, concept refinement and technology
development, and a major decision review called Milestone A, which if
rigorously followed, would provide acquisition officials with an
opportunity to assess whether program officials had the knowledge needed
to develop an executable business case. However, almost 80 percent of the
programs we reviewed began system development without holding any prior
decision review. Senior officials with the Office of the Secretary of
Defense confirmed that this is a common practice among defense acquisition
programs. This practice eliminates a key opportunity for decision makers
to assess early product knowledge needed to establish a business case that
is based on realistic cost, schedule, and performance expectations.
Although program officials conduct analysis before starting a development
program, they do not consistently follow a process to capture the critical
knowledge needed to produce executable business cases, as evidenced by
4
Systems engineering is a technical management tool that provides the
knowledge necessary to translate requirements into specific, achievable
capabilities. By using the tools of systems engineering during these early
phases of concept refinement and technology development acquisition
decision makers and developers can work together to close gaps between
requirements and available resources-well before system development
starts.
Page 13 GAO-06-368 DOD Acquisition Policy
Programs Continue to Enter System Development with Immature Technologies
the poor outcomes current programs are experiencing. Officials with the
Office of the Secretary of Defense recognized this lack of rigor and
discipline in acquisition process, and in February 2004, the Under
Secretary of Defense (Acquisition, Technology and Logistics) issued a
department-wide policy memorandum directing acquisition officials to place
greater emphasis on systems engineering when planning and managing
acquisition programs. The policy requires programs to develop a systems
engineering plan that describes the programs' overall technical approach,
including processes, resources, metrics, and applicable performance
incentives. Although DOD's systems engineering initiative has the
potential to improve program performance, officials have found that the
preliminary results are mixed. Early analysis shows that implementation is
inconsistent while program officials learn to develop and implement
systems-engineering plans.
DOD decision makers continue to permit programs to enter system
development before critical technologies are mature. Our review of
technology readiness assessments and acquisition decision memorandums for
our nine case study programs found that seven of the nine programs were
approved to begin development even though program officials reported
levels of knowledge below the criteria suggested in the policy and
associated guidance, specifically in the area of technology maturity. 5
Those seven programs are not isolated cases. As illustrated in Figure 4,
13 of the programs (nearly three-fourths) that received approval to enter
system development under the revised policy did so with less than 100
percent of their critical technologies mature to the level specified by
DOD. Only 2 of those programs had more than 75 percent of their
technologies mature when they began (see appendix III for technology
maturity data for each program).
5
DOD's revised policy emphasizes the importance of reducing technology risk
and demonstrating technologies in a relevant environment (technology
readiness level 6) prior to program start. A technology readiness level of
6 means the technology should be very close to the planned form, fit, and
function of its physical configuration and that it has been tested or
proven to work in a relevant environment such as a laboratory. GAO
recommends a higher level of maturity in its best practice model based on
best commercial practices. This would require a demonstration of the
technology in the environment it is expected to be used.
Programs Continue Past Design Reviews before Design Maturity is
Demonstrated
Figure 4: Comparison of Programs with Mature versus Immature Technologies
at Start ofSystem Development
Note: This figure depicts technology maturity status for 18 of the 23
programs in our review. Technology maturity data was not available for the
other 5 programs.
Even though acquisition policy states that technologies shall be mature
before beginning system development, the practice of accepting high levels
of technology risk at program start continues to be the norm and not the
exception. An official with Office of the Secretary of Defense responsible
for reviewing and validating program assessments of technology maturity
informed us that the office generally views immature critical technologies
at the beginning of development as an acceptable risk as long as program
officials can show that they have a plan to mature the technologies by the
time the program reaches its design readiness review, which requires
additional investments to move a program from system integration into
system demonstration. Therefore, risk management plans are consistently
viewed as acceptable substitutes for demonstrated knowledge.
In addition to emphasizing the importance of capturing technology
knowledge before starting system development, DOD's policy also highlights
the importance of demonstrating design maturity before moving from the
integration phase of system development into system demonstration and
initial manufacturing. The policy establishes a design readiness review
between the two phases to determine whether a product's design is mature
and stable and whether the product is ready to move ahead. While DOD's
policy does not require programs to demonstrate any specific level of
design maturity, our past work has found that a key indicator of design
maturity is the completion of 90 percent of the system's engineering
drawings. We found that defense programs that moved forward with lower
levels of design maturity, as indicated by drawing completion, encountered
costly design changes and parts shortages that, in turn, caused labor
inefficiencies, schedule delays, and quality problems. Consequently, those
programs required significant increases in resources-time and money-over
what was estimated at the point each program entered the system
demonstration phase.
We analyzed engineering drawing completion data for 8 programs initiated
under the revised policy that have held a design review, 6 and found that
more than half of those programs had not completed 90 percent of their
design drawings before they received approval to enter the system
demonstration phase of development. We also analyzed drawing-release data
for three programs that have not yet held their design review but have
projected the number of drawings officials anticipate will be completed
when their reviews are held. Based on projections provided by program
officials, 2 of those 3 programs are expected to have less than 55 percent
of their drawings complete before they seek approval to begin system
demonstration and initial manufacturing.
Some programs did not report having a design readiness review but did
report having a critical design review. Where this was the case, we
assessed those programs' drawing data at their critical design review.
Page 16 GAO-06-368 DOD Acquisition Policy
Evolutionary Acquisition Is Not Being Used
Sources: DOD (data); GAO (analysis and presentation).
a
Program office projections.
Despite the revised policy's guidance that capabilities should be
developed and delivered in individually defined and separately managed
increments, a majority of major weapon acquisition programs we assessed
continue to be structured to achieve revolutionary increases in capability
within one development program. According to the policy, the objective of
an evolutionary approach is to balance needs and available capability with
resources and put capability into the hands of the user quickly. The
policy states that the success of the strategy depends on consistent and
continuous definition of requirements and the maturation of technologies
that lead to disciplined development and production of systems that
provide increasing capability. In this approach, requirements that cannot
be satisfied within these limits as well as available financial resources
must wait for future generations of the product and be managed as separate
system development programs with separate milestones, costs, and
schedules. In our case studies of nine acquisition programs initiated
under the revised policy, we found only one program-the Small Diameter
Bomb-that satisfied all of the criteria of an evolutionary approach. In
five case studies, we found that program officials had claimed that their
programs were evolutionary, yet our evidence shows they were not
evolutionary in practice; 7 and in three cases, program officials chose
not to use evolutionary acquisition from the outset. Table 3 summarizes
our assessment of the nine case studies.
Specific Criteria Are Needed to Ensure Disciplined and Transparent Investment
Decisions
Sources: DOD (data); GAO (analysis and presentation).
The revised acquisition policy does not contain effective controls that
require the demonstration of product knowledge measured against specific
criteria to ensure that acquisition officials make disciplined,
transparent, and knowledge-based investment decisions. The lack of
specific required criteria creates an environment in which unknowns about
technology, design, and manufacturing processes are acceptable. Decision
makers and program officials are left with no objective measures against
which to gauge a program's level of knowledge, making accountability
difficult. In the absence of criteria, transparency in acquisition
decisions is essential to ensuring accountability, but key decision
documents do not provide sufficient information about major decisions. DOD
believes that acquisition decision memorandums, used to document program
decisions, provide adequate transparency. However,
7
GAO, TACTICAL AIRCRAFT: Opportunity to Reduce Risks in the Joint Strike
Fighter Program with Different Acquisition Strategy, GAO-05-271 (
Washington, D.C.: March 15, 2005) and GAO, UNMANNED AERIAL VEHICLES:
Changes in Global Hawk's Acquisition Strategy Are Needed to Reduce Program
Risks, GAO-05-6 (Washington, D.C.: November 5, 2004).
Page 18 GAO-06-368 DOD Acquisition Policy
the decision memorandums we reviewed did not contain an explanation of the
decision maker's rationale and rarely identify remaining risks, especially
as they relate to the key knowledge standards emphasized in the policy.
Further, the timeliness, accessibility, and depth, of the data contained
in the Selected Acquisition Reports, DOD's primary means of providing
Congress with a status report of program performance, inhibits the
reports' usefulness as a management and oversight tool.
In November 2003, we reported that the revised acquisition policy lacked
many of the controls that leading commercial companies rely on to attain
an acceptable level of knowledge before making additional significant
investments. 8 Controls are considered effective if they are backed by
specific criteria and if decision makers are required to consider the
resulting data before deciding to advance a program to the next level.
Controls used by leading companies help decision makers gauge progress in
meeting cost, schedule, and performance goals and hold program managers
accountable for capturing relevant product knowledge to inform key
investment decisions. The controls we have articulated as best practices
used by successful commercial product developers are listed below in table
5.
GAO, DEFENSE ACQUISITIONS: DOD's Revised Policy Emphasizes Best Practices,
but More Controls Are Needed , GAO-04-53 ( November 10, 2003).
Page 19 GAO-06-368 DOD Acquisition Policy
Table 5: Types of Controls Considered Best Practices for Successful
Product Development
Program start (Milestone B): Start of product development
Demonstrate technologies to high readiness levels
Ensure that requirements for the product are informed by the systems
engineering process
Establish cost and schedule estimates for product on the basis of
knowledge from preliminary design using system engineering tools
Conduct decision review for program start
Design readiness review: Beginning of system demonstration
Complete 90 percent of design drawings
Complete subsystem and system design reviews
Demonstrate with prototype that design meets requirements
Obtain stakeholders' concurrence that drawings are complete and producible
Complete the failure modes and effects analysis
Identify key system characteristics
Identify critical manufacturing processes
Establish reliability targets and growth plan on the basis of demonstrated
reliability rates of components and subsystems
Conduct decision review to enter system demonstration
Production commitment (Milestone C): Initiation of low-rate production
Demonstrate manufacturing processes
Build production-representative prototypes
Test production-representative prototypes to achieve reliability goal
Test production-representative prototypes to demonstrate product in
operational environment
Collect statistical process control data
Demonstrate that critical processes are capable and in statistical control
Conduct decision review to begin production
Sources: GAO (analysis and presentation).
Some senior officials with the Office of the Secretary of Defense believe
that the effective use of controls in DOD's policy and the establishment
of more specific criteria for decision making would improve program
outcomes. They note that specific criteria need to be established and that
programs need to be held accountable to those criteria before being
permitted to proceed into the next phase. They also note that the criteria
for moving an acquisition effort from one phase of the process to the
next, primarily documented in acquisition decision memorandums as exit
criteria, are not typically specific and often do not relate to the key
knowledge-based criteria suggested in the policy.
We found this to be true for our nine case study programs. We reviewed
acquisition decision memorandums in our case studies and determined that
they were not useful in explaining the decision maker's rationale and in
almost all of the cases they did not address the key knowledge criteria
suggested in the acquisition policy. In most instances, the decision maker
simply noted that the program being assessed was ready to proceed into
system development, but did not provide an explanation of the rationale
for the decision. Senior officials with the Office of the Secretary of
Defense told us that they agree that a better explanation of the decision
maker's rationale, specifically in instances where the knowledge criteria
are not fully met, would provide transparency and ultimately allow for a
more accountable decision-making process. The following two examples
illustrate how decision documentation is lacking:
o The Future Combat System program received approval to enter system
development and demonstration in 2003, with 19 percent of its critical
technologies mature, well below the policy's standard. The acquisition
decision memorandum supporting this decision did not provide the
rationale for approving the system with such a large number of
immature critical technologies. The memo did direct an updated review
of the decision 18 months later and that the program "remain flexible
and open to accommodate trades in the system architecture and in the
individual systems' designs."
o The Joint Strike Fighter program was approved to enter system
development in 2001. The acquisition decision memorandum did not
address the fact that 75 percent of the program's critical
technologies were not mature to the policy's standard. The memorandum
did acknowledge that the program's requirements could be changed or
modified, noting that further refinements in the requirements should
be explored as a potential way to reduce program costs. However, the
memorandum did not explain why the decision maker determined that the
program should enter development without achieving the technology and
requirements knowledge emphasized in the policy.
The acquisition decision memorandums for most of the other programs we
reviewed did not specifically address critical gaps in knowledge, nor did
they effectively explain the decision makers' rationale for deeming those
programs ready to begin system development. In memos where we found a
reference to key knowledge principles, such as technology maturity, the
Conclusions
decision makers acknowledged that more effort was needed to meet the
policy's suggested criteria but considered the risk acceptable to begin
development. These memos did not explain why risks were considered
acceptable. For example, the Navy's Multi-Mission Maritime Aircraft
program had none of its critical technologies mature at program
initiation. The decision maker acknowledged the need to further mature the
critical technologies but approved the program to enter development.
Instead of holding the program to the policy's criteria for entering
development, the decision maker simply directed the Navy to work with the
Office of the Secretary of Defense to implement risk mitigation and
technology maturation plans during the integration phase of system
development.
In addition to the lack of transparency provided through acquisition
decision memoranda, we also found that the data presented to Congress in
DOD's Selected Acquisition Reports (SARs) provided only limited usefulness
as an oversight tool. Since 1969, SARs have been the primary means by
which DOD reports the status of major weapon system acquisitions to
Congress. SARs are reports that are expected to contain information on the
cost, schedule, and performance of major weapon systems in comparison with
baseline values established at program start, full-scale development, and
production decision points. Our analysis, as well as a previous GAO
review, 9 of current and historical SAR data found that the timeliness,
accessibility, and depth of the data contained in the reports limits their
usefulness as an oversight tool. Our prior review noted that a number of
opportunities exist for DOD to give Congress more complete information on
the performance of major defense acquisition programs. DOD agreed that SAR
data could be improved to make it more useful to Congress.
Failing to consistently implement the knowledge-based process and
evolutionary principles emphasized in the revised acquisition policy-
coupled with a lack of specific criteria for making key investment
decisions-are keeping DOD on its historical path of poor cost and schedule
outcomes. Most programs are incurring the same scope of cost overruns and
schedule delays as programs managed under prior DOD policies. More
consistent use of the early acquisition processes would improve the
quality and viability of program business cases by ensuring
9
GAO, DEFENSE ACQUISITIONS: Information for Congress on Performance of
Major Programs Can Be More Complete, Timely, and Accessible, GAO-05-182 (
March 28, 2005).
Page 22 GAO-06-368 DOD Acquisition Policy
they are founded on knowledge obtained from rigorous and disciplined
analysis. The initiative by Office of the Secretary of Defense to
reinstitute the use of systems engineering is a step in the right
direction. However, in order for this initiative to be effective DOD must
establish and enforce specific criteria at key decision points. Our past
work has identified and recommended criteria and controls that should be
consistently applied at major decision points. The enforcement of these
criteria is critical to ensuring that programs have the knowledge
necessary to successfully move forward through the acquisition process.
DOD officials have acknowledged the advantages of using knowledge-based
criteria and controls, but believe the policy already includes enough
controls to achieve effective program results. However, without
enforceable criteria, defense officials are challenged to determine
whether adequate knowledge has been obtained for investing taxpayer
dollars. The lack of enforceable criteria also makes it difficult to hold
defense officials accountable for their decisions.
DOD must ensure that appropriate knowledge is captured and used at
Recommendations
critical junctures to make decisions about moving a program forward and
for Executive Action investing more money. We recommend that the Secretary
of Defense require program officials to demonstrate that they have
captured appropriate knowledge at three key points-program start, design
review for transitioning from system integration to system demonstration,
and production commitment-as a condition for investing resources. At a
minimum those controls should require program officials to demonstrate
that they have achieved a level of knowledge that meets or exceeds the
following criteria at each respective decision point:
* Program start (Milestone B): Start of product development
o Demonstrate technologies to high readiness levels
o Ensure that requirements for the product are informed by the
systemsengineering process
o Establish cost and schedule estimates for product on the basis
of knowledge from preliminary design using system engineering
tools
o Conduct decision review for program start
* Design readiness review: Beginning of system demonstration
o Complete 90 percent of design drawings
o Complete subsystem and system design reviews
o Demonstrate with prototype that design meets requirements
o Obtain stakeholders' concurrence that drawings are complete
and producible
o Complete the failure modes and effects analysis
o Identify key system characteristics
o Identify critical manufacturing processes
o Establish reliability targets and growth plan on the basis of
demonstrated reliability rates of components and subsystems
o Conduct decision review to enter system demonstration
* Production commitment (Milestone C): Initiation of low-rate
production
o Demonstrate manufacturing processes
o Build production-representative prototypes
o Test production-representative prototypes to achieve
reliability goal
o Test production-representative prototypes to demonstrate
product in operational environment
o Collect statistical process control data
o Demonstrate that critical processes are capable and in
statistical control
o Conduct decision review to begin production
Furthermore, to ensure that major decisions are transparent and that
program officials and decision makers are held accountable, we recommend
that the Secretary of Defense require decision makers to include written
rationale for each major decision in acquisition decision documentation.
The rationale should address the key knowledge-based criteria appropriate
for milestone decisions, explain why a program's level of knowledge in
each area was deemed acceptable if criteria have not been met and provide
a plan for achieving the knowledge necessary to meet criteria within a
given time frame.
Agency Comments and Our Evaluation
DOD provided written comments on a draft of this report. The comments
appear in appendix II.
DOD partially concurred with our recommendation that the Secretary of
Defense should establish specific controls to insure that program
officials demonstrate that they have captured a level of knowledge that
meets or exceeds specific criteria at three key points in the acquisition
process: program start, design readiness review, and production
commitment. DOD agreed that knowledge-based decision making is consistent
with sound business practice and stated that it would continue to develop
policy that reflects a knowledge-based approach and improves acquisition
outcomes. DOD noted that it would consider our recommendations as it
reassesses the DOD acquisition business model and the knowledge required
at each decision point. We believe that DOD's plan to reassess its
business model provides a good opportunity to establish the controls and
specific criteria recommended in this report. Therefore, we are retaining
our recommendation that the Secretary of Defense should establish controls
to insure that program officials demonstrate that they have captured a
level of knowledge that meets or exceeds specific criteria at three key
points in the acquisition process.
DOD also partially concurred with our recommendation that the Secretary of
Defense require decision makers to provide written rationale in
acquisition decision documentation for each major decision. DOD agreed
that acquisition decisions should be documented, decision makers should be
held accountable, and that they should provide the rationale for their
decisions. DOD believes that the implementation of Section 801 of the
National Defense Authorization Act for FY 2006 reinforces these processes.
The act calls for the decision maker to certify that the program meets
certain requirements, such as technology maturity, prior to starting a new
development program at Milestone B. However, the act is focused on the
decision to start a development program and does not identify specific
criteria for programs to be measured against at design readiness review or
production commitment. We believe our recommendation adds transparency and
accountability to the process because it requires the decision maker to
provide the rationale for a decision to allow a program to move forward,
not only at Milestone B but at other key decision points as well.
Therefore, we are retaining our recommendation that the Secretary of
Defense require decision makers to provide written rationale for each
major decision in acquisition decision documentation.
We are sending copies of this report to the Secretary of Defense; the
Secretaries of the Air Force, Army, and Navy; and the Director of the
Office of Management and Budget. We will provide copies to others on
request. This report will also be available at no charge on GAO's Web site
at http://www.gao.gov.
If you have any questions about this report or need additional
information, please call me at (202) 512-4841 ([email protected]). Contact
points for the offices of Congressional Relations and Public Affairs are
located on the last page of this report. Key contributors to this report
were Michael Hazard, Assistant Director; Lily Chin; Ryan Consaul;
Christopher DePerro; Travis Masters; and Adam Vodraska.
Michael J. Sullivan Director, Acquisition
and Sourcing Management
Appendix I: Scope and Methodology
To assess the impact of DOD's revised acquisition policy, we analyzed cost
and schedule data for 23 major defense acquisition programs that were
approved to begin system development under the revised policy. We did not
assess space, missile defense, or ship programs. We collected our data
from Selected Acquisition Reports, presidential budget documents, ongoing
GAO work, and pertinent program officials. We utilized previous GAO
reports related to defense acquisition policies and worked with
knowledgeable GAO staff to ensure the use of current, accurate data. We
also analyzed more than 150 annual Selected Acquisition Reports covering a
36-year period from 1969 to 2005, to determine historical trends related
to outcomes of acquisition policy implementation.
We assessed whether the revised policy's knowledge-based, evolutionary
acquisition principles were being effectively implemented by conducting 9
case study reviews and analyzing design maturity data for 11 programs that
have made engineering-drawing data available to GAO. Our case study
programs were the Aerial Common Sensor, Multi-Platform Radar Technology
Insertion Program, Global Hawk Unmanned Aerial Vehicle, Small Diameter
Bomb, Future Combat System, Joint Strike Fighter, Expeditionary Fighting
Vehicle, Multi-Mission Maritime Aircraft, and the E-2 Advanced Hawkeye. We
interacted directly with numerous program officials to seek input on
current developments with their programs. We studied program documents to
assess how well programs understand and are implementing the revised
acquisition policy. We also analyzed drawing release data for those
programs that have either passed their design review or have provided GAO
with estimated drawing release data for a future design review to assess
design maturity. In several cases, we asked that program offices verify
information in these various documents.
We also reviewed Department of Defense (DOD) Directive 5000.1, DOD
Instruction 5000.2, and the Defense Acquisition Guidebook. In addition we
examined each of the military services' policy directives and guidance,
DOD memorandums to include policy intent and DOD expectations regarding
policy implementation as well as Joint Capabilities Integration and
Development System documents. We interviewed relevant officials in
Washington, D.C., from the Office of the Director, Defense Research and
Engineering, the Joint Staff, the Office of the Secretary of Defense, and
Army, Navy, and Air Force acquisition policy staff in order to better
understand the content of these documents and the intent of DOD's policy.
We conducted our review from May 2005 to February 2006 in accordance with
generally accepted government auditing standards.
Appendix II: Comments from the Department of Defense
Appendix II: Comments from the Department of Defense
Appendix II: Comments from the Department of Defense
Appendix III: Program Data for 23 Programs Initiated under DOD's Revised
Acquisition Policy (as of December 2005)
Page 31 GAO-06-368 DOD Acquisition Policy
Appendix III: Program Data for 23 Programs Initiated under DOD's Revised
Acquisition Policy (as of December 2005)
Sources: DOD (data); GAO (analysis and presentation).
Note: In this table the term "not available" means that GAO had not
received sufficient data to make an assessment of the given program's
design and/or technology maturity.
a
Milestone I was a forerunner to Milestone A, the decision review that
currently precedes the start of technology development.
b
Program office projections.
Cost growth is expressed as the percent change in program development cost
estimates in fiscal year 2005 dollars.
Related GAO Products
DOD Acquisition Outcomes: A Case for Change. GAO-06-257T. Washington,
D.C.: November 15, 2005.
Defense Acquisitions: Stronger Management Practices Are Needed to Improve
DOD's Software-Intensive Weapon Acquisitions. GAO-04-393. Washington,
D.C.: March 1, 2004.
Best Practices: Setting Requirements Differently Could Reduce Weapon
Systems' Total Ownership Costs. GAO-03-57. Washington, D.C.: February 11,
2003
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15,
2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best Practices.
GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to Better
Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better Weapon
System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31, 2000.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon System Programs
Implement Best Practices. GAO/NSIAD-99-206. Washington,D.C.: August16,
1999.
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30, 1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program
Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.
Defense Acquisition: Improved Program Outcomes Are Possible. GAO/T-
NSIAD-98-123. Washington, D.C.: March 17, 1998.
Related GAO Products
Best Practices: DOD Can Help Suppliers Contribute More to Weapon System
Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD's Environment. GAO/NSIAD-98-56. Washington, D.C.: February
24, 1998.
Best Practices: Commercial Quality Assurance Practices Offer Improvements
for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26, 1996.
(120448)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov) . Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202) 512-4800
Public Affairs
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON
RECYCLED PAPER
*** End of document. ***