Defense Acquisitions: Improvements Needed in Space Systems
Acquisition Management Policy (15-SEP-03, GAO-03-1073).
The Department of Defense is spending nearly $18 billion annually
to develop, acquire, and operate satellites and other
space-related systems. The majority of satellite programs that
GAO has reviewed over the past 2 decades experienced increased
costs and delayed schedules. DOD has recently implemented a new
acquisition management policy, which sets the stage for decision
making on individual space programs. GAO was asked to assess the
new policy.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-1073
ACCNO: A08496
TITLE: Defense Acquisitions: Improvements Needed in Space
Systems Acquisition Management Policy
DATE: 09/15/2003
SUBJECT: Military satellites
Defense procurement
Procurement policy
Policy evaluation
Data collection
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-1073
Report to the Chairman, Subcommittee on Defense, Committee on
Appropriations, House of Representatives
United States General Accounting Office
GAO
September 2003 DEFENSE ACQUISITIONS
Improvements Needed in Space Systems Acquisition Management Policy
GAO- 03- 1073
DOD*s new space acquisition policy may help provide more consistent and
robust information on technologies, requirements, and costs. For example,
the policy employs a new independent cost estimating process, independent
program reviews performed by space experts not connected with the
program, and more rigorous analyses of alternatives, requirements, and
system interdependencies. This information may help decision- makers
assess whether gaps exist between expectations and what the program can
deliver.
However, the benefits that can be derived from these tools will be limited
since the new policy does not alter DOD*s practice of committing major
investments before knowing what resources will be required to deliver
promised capability. Instead, the policy encourages development of leading
edge technology within product development, that is, at the same time the
program manager is designing the system and undertaking other product
development activities. As our work has repeatedly shown, such concurrency
increases the risk that significant problems will be discovered as the
system is integrated and built, when it is more costly and time- consuming
to fix them. Moreover, when even one technology does not mature as
expected, the entire program can be thrown off course since time and cost
for invention cannot be reliably estimated. DOD*s new acquisition policy
for its other weapon systems recognizes these risks and consequently
requires technology and product development to be done separately.
Overview of Key Decision Points
Note: According to DOD officials, while technology development is expected
to ramp down during phase B, in some instances technology development
could even continue after key decision point C or critical design review.
Thus, technology development is depicted in a lighter shade after decision
point C.
The Department of Defense is spending nearly $18 billion annually to
develop, acquire, and operate satellites and other space- related systems.
The majority of satellite programs that GAO has reviewed over the past 2
decades experienced increased costs and delayed schedules. DOD has
recently implemented a new acquisition management
policy, which sets the stage for decision making on individual space
programs. GAO was asked to assess the new policy.
GAO is recommending that DOD modify its policy to separate technology
development from product development and ensure decisions to start
programs are
based on sound criteria. DOD disagreed with our recommendations
principally because it believes that implementing them will slow down
acquisitions, increase risks, and prevent DOD from taking advantage of
cutting edge technology. Our past reviews of best practices, however, have
shown that risk and time are reduced and capability is increased when
programs begin with knowledge that technologies can
work as intended. DOD*s policy for other weapon systems incorporates this
view.
www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 1073. To view the full product,
including the scope and methodology, click on the link above. For more
information, contact Katherine Schinasi at (202) 512- 4841. Highlights of
GAO- 03- 1073, a report to the
Chairman, Subcommittee on Defense, Committee on Appropriations, House of
Representatives
September 2003
DEFENSE ACQUISITIONS
Improvements Needed in Space Systems Acquisition Management Policy
Page i GAO- 03- 1073 Space Acquisition Policy Letter 1 Results in Brief 1
Background 2 Gap between Resources and Requirements Has Undermined
Space Acquisitions 6 Space Policy May Help Increase Insight into Gaps
between Requirements and Resources 10 New Space Policy Does Not Call for a
Match between Resources and Requirements at Program Start 13 Conclusions
16 Recommendations for Executive Action 17 Agency Comments and Our
Evaluation 17 Appendix I The Department of Defense*s Current and
Planned Satellite Systems 20
Appendix II Technology Readiness Levels and Their Definitions 21
Appendix III Comments From the Department of Defense 22
Related GAO Products 29
Table
Table 1: Decision- Making Characteristics 5 Figures
Figure 1: Overview of Key Decision Points 4 Figure 2: DOD Will Be Making
Commitments before Obtaining Critical Knowledge for Space Systems 15
Contents
Page ii GAO- 03- 1073 Space Acquisition Policy Abbreviations
AEHF Advanced Extremely High Frequency CAIG Cost Analysis Improvement
Group DAB Defense Acquisition Board DOD Department of Defense DSAB Defense
Space Acquisition Board EELV Evolved Expendable Launch Vehicle GPS Global
Positioning System IPA Independent Program Assessment IPT Integrated
Product Team KDP key decision point MUOS Mobile Users Objective System
NPOESS National Polar- orbiting Operational Environmental Satellite
System NRO National Reconnaissance Office SBIRS Space- Based Infrared
System SBR Space- Based Radar
STSS Space Tracking and Surveillance System TRL Technology Readiness Level
TSAT Transformational Communications Satellite
This is a work of the U. S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
Page 1 GAO- 03- 1073 Space Acquisition Policy
September 15, 2003 The Honorable Jerry Lewis Chairman, Subcommittee on
Defense Committee on Appropriations House of Representatives
Dear Mr. Chairman: The Department of Defense (DOD) is spending more than
$18 billion annually to develop, acquire, and operate satellites and other
space- related systems. Moreover, DOD is on the threshold of investing in
several new major satellite acquisition programs. These programs are
intended to help transform how information is collected on capabilities
and intentions of potential adversaries as well as how military forces
communicate and navigate and attack targets. We reported to you in June
2003 that the majority of satellite programs we have reviewed over the
past 2 decades experienced problems during acquisition that significantly
increased costs and delayed schedules, often to the point where programs
needed to be restructured by DOD.
DOD has recently implemented a new acquisition management policy for space
systems, which sets the stage for making decisions on individual space
programs. As you requested, we assessed the new policy* specifically
whether it will enable DOD to match requirements (that is, what the system
needs to do and how well it needs to perform) to resources (time, money,
and technical knowledge) at the onset of product development. Our work
shows that achieving this match is the most critical determinant for
successful outcomes of acquisitions. DOD*s new space acquisition policy
may help provide more consistent
and robust information on technologies, requirements, and costs. For
example, the policy employs a new independent cost estimating process,
independent program reviews performed by space experts not connected with
the program, and more rigorous analyses of alternatives, requirements, and
system interdependencies. This information may help decision- makers
assess whether gaps exist between expectations and
what the program can deliver.
United States General Accounting Office Washington, DC 20548
Results in Brief
Page 2 GAO- 03- 1073 Space Acquisition Policy
However, the benefits that can be derived from these tools will be limited
since the new policy does not alter DOD*s practice of committing major
investments before knowing what resources will be required to deliver
promised capability. Instead, the policy encourages development of leading
edge technology within product development, that is, at the same time the
program manager is designing the system and undertaking other product
development activities. As our work has repeatedly shown, such concurrency
increases the risk that significant problems will be discovered as the
system is integrated and built, when it is more costly and time- consuming
to fix them. Moreover, when even one technology does not mature as
expected, the entire program can be thrown off course since time and cost
for invention cannot be reliably estimated. DOD*s new acquisition policy
for its other weapon systems recognizes these risks and consequently
requires technology and product development to be done separately.
We are making recommendations to DOD to modify its policy to separate
technology development from product development and ensure decisions to
start programs are based on sound criteria. DOD disagreed with our
recommendations principally because it believes that implementing them
will slow down acquisitions, increase risks, and prevent DOD from taking
advantage of cutting edge technology. Our past reviews of best practices,
however, have shown that risk and time are reduced and capability is
increased when programs begin with knowledge that technologies can work as
intended. DOD*s policy for other weapon systems incorporates
this view. DOD*s current space network is comprised of constellations of
satellites, ground- based systems, and associated terminals and receivers.
Among other things, these assets are used to perform intelligence,
surveillance, and reconnaissance functions; perform missile warning;
provide communication services to DOD and other government users; provide
weather and environmental data; and provide positioning and precise timing
data to U. S. forces as well as national security, civil, and commercial
users. Background
Page 3 GAO- 03- 1073 Space Acquisition Policy
DOD is now implementing a new acquisition management policy tailored to
its space systems. 1 It expects to finalize the policy this fiscal year.
The policy is similar to the one used by the National Reconnaissance
Office (NRO). The policy is different from a new acquisition management
policy DOD is implementing for most other weapons- related acquisitions in
several respects.
Key decisions, including the decision to start product development and
to start building and testing a satellite, will be made earlier in the
development process. According to DOD, this is because satellites incur
most of their costs during the early phases of development. The decision
to build and produce a satellite will be made at the same
time instead of sequentially. According to DOD, this is because satellites
are produced in very small numbers as compared to other acquisitions.
Figure 1 provides an overview of differences in key decision points. 1
Other DOD weapons- related acquisitions (e. g., aircraft, ships, and
tanks) fall under DOD*s new 5000 Series. Missile defense systems, such as
the Space Tracking and Surveillance System, fall under a process designed
and managed by the Missile Defense Agency.
Page 4 GAO- 03- 1073 Space Acquisition Policy
Figure 1: Overview of Key Decision Points Note: According to DOD
officials, while technology development is expected to ramp down during
phase B, in some instances technology development could even continue
after key decision point C or critical design review. Thus, technology
development is depicted in a lighter shade after decision point C.
The new space acquisition policy is also different than DOD*s policy for
other weapon systems in terms of decision- making support. For example,
the new policy has created an advisory board distinct from the DOD*s
Defense Acquisition Board (DAB). The Defense Space Acquisition Board
(DSAB), comprised of senior- level DOD officials and mission partners,
will advise the Under Secretary of the Air Force, as the milestone
decision authority, on whether significant investments should move forward
in the development process. Also, temporary Independent Program Assessment
teams (IPA) will be used to conduct an intensive review before key
decisions are made. Under DOD*s process for other weapon systems, standing
Integrated Product Teams (IPT) are used to help programs
Page 5 GAO- 03- 1073 Space Acquisition Policy
conduct key analyses as well as to advise the DAB. Table 1 provides more
details on these differences.
Table 1: Decision- Making Characteristics DOD Weapons Acquisitions Space
Acquisitions Milestone Decision Authority Milestone Decision Authority
Under Secretary of Defense for Acquisition, Technology and Logistics (USD
AT& L) makes decision on whether program should proceed into next phase.
Under Secretary of the Air Force makes decision on whether program should
proceed into next phase.
Advisory Board Advisory Board
Defense Acquisition Board (DAB) Composed of Vice Chairman, Joint Chiefs of
Staff (Co- chairman of DAB) Under Secretary of Defense- Comptroller Under
Secretary of Defense- Policy Under Secretary of Defense- Personnel and
Readiness Assistant Secretary of Defense for Networks and Information
Integration
Service secretaries Director of Operational Test and Evaluation Additional
advisors as invited Defense Space Acquisition Board (DSAB)
Composed of Vice Chairman, Joint Chiefs of Staff (Co- chairman of DSAB)
Under Secretary of the Air Force staff
Executive Service offices Mission partners (National Reconnaissance
Office, National Aeronautics and Space Administration, U. S. Strategic
Command, Department of Transportation) Stakeholders (Office of the
Secretary of Defense, Joint Chiefs
Staff, Office of Management and Budget) Users (e. g., combatant commands,
military services, and intelligence community)
Director of Operational Test and Evaluation Additional advisors as invited
Integrated Product Team Independent Program Assessment Team
Help programs prepare for DAB review and provide decisionmaking support.
Two teams (overarching and working level), permanently assigned to certain
weapon systems.
Comprised of different functional experts, e. g., engineering,
manufacturing, purchasing, and finance. Teams review various types of
weapon systems, so they will not necessarily include space experts.
Teams meet with programs once every few months. Because teams are
dedicated to several programs, they cannot do intensive drill downs. Time
taken to help programs prepare for review may take as long as 18 months.
Perform *drill down* reviews of programs before decisions on whether to
move programs forward are made. Temporary team Comprised of space experts
Review is done in 8 weeks (or more, if required) on- site working full-
time with program officials. Source: GAO. DOD is already applying this new
process to major satellite programs,
including the Space- Based Infrared System (High) (SBIRS- High), the
Transformational Communications Satellite (TSAT), the Advanced Extremely
High Frequency (AEHF) system, the Mobile User Objective System (MUOS), the
Global Positioning System (GPS), the National
Page 6 GAO- 03- 1073 Space Acquisition Policy
Polar- orbiting Operational Environmental Satellite System (NPOESS), and
the Space- Based Radar (SBR) system. (See app. I for a further description
of DOD*s current and planned systems.) SBR is the first system to receive
approval for the first key decision point* key decision point (KDP) A*
which begins a study phase. Other systems will come in at a later decision
point* KDP B, which starts the acquisition program, or KDP C, which starts
the process of building, testing, and launching the satellite. Some space-
related systems, such as user equipment, are produced in mass numbers.
They will be overseen under a process that is more similar to the DOD-
wide acquisition process.
The majority of satellite programs we have reviewed over the past 2
decades experienced problems during acquisition that drove up costs and
schedules and increased technical risks. Several programs were
restructured by DOD in the face of delays and cost growth. We have found
that these problems, which are common among many weapon systems, are
largely rooted in a failure to match the customer*s needs with the
developer*s resources* technical knowledge, timing, and funding* when
starting product development. In other words, commitments were made to
satellite launch dates and achieving certain capabilities without knowing
whether technologies being pursued could really work as intended. Time and
costs were consistently underestimated.
Leading commercial firms expect that their program managers will deliver
high quality products on time and within budgets. Doing otherwise could
result in losing a customer in the short term and losing the company in
the long term. Thus, these firms have adopted practices that put their
individual program managers in a good position to succeed in meeting these
expectations on individual products. Collectively, these practices ensure
that a high level of knowledge exists about critical facets of the product
at key junctures during its development and is used to deliver capability
as promised. While DOD is different from the commercial world in terms of
its need to push for cutting edge technology to maintain military
superiority, its policies for major weapon systems recognize that
maturing technology outside of product development allows needed stability
in executing budgets and allows capability to be delivered to the
warfighter sooner.
Our reviews have shown that there are three critical junctures at which
firms must have knowledge to make large investment decisions. First,
before product development is started, a match must be made Gap between
Resources and Requirements Has Undermined Space Acquisitions
Achieving a Match between Resources and Requirements Is Essential to
Success
Page 7 GAO- 03- 1073 Space Acquisition Policy
between the customer*s needs and the available resources* technical and
engineering knowledge, time, and funding. Second, a product*s design must
demonstrate its ability to meet performance requirements and be stable
about midway through development. Third, the developer must show that the
product can be manufactured within cost, schedule, and quality targets and
is demonstrated to be reliable before production begins.
The process is building block in nature as the attainment of each
successive knowledge point builds on the proceeding one. While the
knowledge itself builds continuously without clear lines of demarcation,
the attainment of knowledge points is sequential. In other words,
production maturity cannot be attained if the design is not mature, and
design maturity cannot be attained if the key technologies are not mature.
In applying the knowledge- based approach, the most leveraged decision
point of the three junctures is matching the customer*s needs with the
developer*s resources. This initial decision sets the stage for the
eventual outcome* desirable or problematic. The match is ultimately
achieved in
every development program, but in successful development programs, it
occurs before product development. In successful programs, negotiations
and trade- offs occur before product development is started to ensure that
a match exists between customer expectations and developer resources.
Technologies that are not mature continue to be developed in the
technology base (for example, a research laboratory). With achievable
requirements and commitment of sufficient investment to complete the
development, programs are better able to deliver products at cost and on
schedule. 2 Our past work has shown that space programs have not typically
achieved a match between requirements and resources before starting
product
development. Product development was often started based on a rigid set of
requirements that proved to be unachievable within a reasonable
development time frame. At times, even more requirements were added after
the program began. When problems arose, adding resources in terms
of time and money became the primary option for solving problems, since 2
Our best practice reviews are identified in the Related GAO Products at
the end of this report. In DOD, Match between Resources and
Requirements Is Seldom Achieved at Start of Product Development
Page 8 GAO- 03- 1073 Space Acquisition Policy
customer expectations about the product*s performance had already become
hardened. For example:
After starting its AEHF satellite program, DOD substantially and
frequently changed requirements. In addition, after the failure of one of
DOD*s legacy communications satellites, DOD decided to accelerate its
plans to build AEHF satellites. The contractors proposed, and DOD
accepted, a high risk schedule that turned out to be overly optimistic and
highly compressed, leaving little room for error and depending on a chain
of events taking place at certain times. Moreover, at the time DOD decided
to accelerate the program, it did not have funding needed to support the
activities and manpower needed to design and build the satellites quicker.
The effects of DOD*s inability to match requirements to resources were
significant. Cost estimates produced by the Air Force reflected an
increase from $4.4 billion in January 1999 to $5.6 billion in June 2001* a
difference of 26 percent. Although considered necessary, many changes to
requirements were
substantial, leading to cost increases of hundreds of millions of dollars
because they required major design modifications. Also, schedule delays
occurred when some events did not occur on time, and additional delays
occurred when the program faced funding gaps. Scheduling delays eventually
culminated into a 2- year delay in the launch of the first satellite. We
also reported that there are still technical and production risks that
need to be overcome in the AEHF program, such as a less- than- mature
satellite antenna system and complications associated with the production
of the system*s information security system.
The SBIRS- High 3 contract for engineering, manufacturing and
development amounted to $2.4 billion. In the fall of 2001, DOD identified
cost growth of $2 billion or more, triggering a mandatory program review
and recertification under 10 U. S. C. section 2433. Currently, SBIRS- High
is under contract for $4.4 billion. We reported that when DOD*s SBIRS-
High satellite program began in 1994, none of its critical technologies
were
mature. Moreover, according to a DOD- chartered independent review team,
the complexity, schedule, and resources required to develop SBIRS- High,
in hindsight, were misunderstood when the program began.
3 In the mid- 1990s, SBIRS was established as a *systems of systems*
approach with two components, SBIRS- High and SBIRS- Low, that were
managed by the Air Force. In 2000, SBIRS- Low was shifted back from the
Air Force to the Ballistic Missile Defense Organization, which is now the
Missile Defense Agency. In 2002, SBIRS- Low was renamed Space Tracking and
Surveillance System (STSS). While STSS is focused primarily on supporting
the missile defense mission, SBIRS- High is focused on missile warning,
missile defense, technical intelligence, and battlespace characterization
and is managed by the Air Force.
Page 9 GAO- 03- 1073 Space Acquisition Policy
This led to an immature understanding of how requirements translated into
detailed engineering solutions. Even though the program was restructured
by DOD, the independent review team noted that SBIRS- High still faced
significant risks.
DOD has initiated several programs and spent several billion dollars
over the past 2 decades to develop low- orbiting satellites that can track
ballistic missiles throughout their flight. However, it has not launched a
single satellite to perform this capability. We have reported 4 that a
primary problem affecting these programs was that DOD and the Air Force
did not relax rigid requirements to more closely match technical
capabilities that were achievable. Program baselines were based on
artificial time and/ or
money constraints. Over time, it became apparent that the lack of
knowledge of program challenges had led to overly optimistic schedules and
budgets that were funded at less than what was needed. Attempts to
stay on schedule by approving critical milestones without meeting program
criteria resulted in higher costs and more slips in technology development
efforts. For example, our 1997 and 2001 reviews of DOD*s $1.7 billion
SBIRS- Low program showed that the program would enter into the product
development phase with critical technologies that were immature and with
optimistic deployment schedules. Some of these technologies were so
critical that SBIRS- Low would not be able to perform its mission if they
were not available when needed. DOD eventually restructured the SBIRS- Low
program because of the cost and scheduling problems, and it put the
equipment it had partially built into storage. In view of the program*s
mismatch between expectations and what it could achieve, the Congress
directed DOD to restructure the program (now known as the Space Tracking
and Surveillance System or STSS) as a research and development effort.
We recently reported 5 on crosscutting factors that make it more difficult
for DOD to achieve a match between resources and requirements for space
acquisitions. In particular, space programs often involve a diverse array
of organizations with competing interests involved in overall satellite
development* from the individual military services, to testing
4 U. S. General Accounting Office, Missile Defense: Alternate Approaches
to Space Tracking and Surveillance System Need to Be Considered, GAO- 03-
597 (Washington, D. C.: May 23, 2003).
5 U. S. General Accounting Office, Military Space Operations: Common
Problems and Their Effects on Satellite and Related Acquisitions, GAO- 03-
825R (Washington, D. C.: June 2, 2003).
Page 10 GAO- 03- 1073 Space Acquisition Policy
organizations, contractors, civilian agencies, and in some cases, even
international partners and industry. This creates challenges in making
tough tradeoff decisions.
In addition, like other weapon programs, space acquisition programs have
historically attempted to satisfy all requirements in a single step,
regardless of the design challenge or the maturity of technologies to
achieve the full capability. This approach has made it more difficult to
match requirements to available resources.
DOD*s new space acquisition oversight process may help increase insight
into gaps between requirements and resources. In particular, tools being
adopted, such as technology readiness assessments, alternatives analyses,
and independent cost estimates, may help provide more consistent and
robust information on technologies, requirements, and costs. However, the
value of these tools depends largely on whether or not the knowledge is
used to make decisions. According to DOD officials, similar tools are also
being adopted by other weapon system programs.
First, DOD is requiring that all space programs conduct technology
maturity assessments before key oversight decisions to assess the maturity
level of technology. One tool used by many weapon systems is known as
Technology Readiness Levels (TRL). The tool associates different TRLs with
different levels of demonstrated performance, ranging from paper studies
to proven performance on the intended product. The
value of using a tool based on demonstrated performance is that it can
presage the likely consequences of incorporating a technology at a given
level of maturity into a product development, enabling decision- makers to
make informed choices. The tool is even more valuable if it is commonly
used. Our previous reviews have found the use of TRLs to be a best
practice. 6 (App. II describes TRL levels.)
Second, DOD is requiring space programs to more rigorously assess
alternatives, consider how their systems will operate in the context of
larger families of systems, and think through operational, technical, and
system requirements before programs are started. For example, programs 6
U. S. General Accounting Office, Best Practices: Better Management of
Technology Development Can Improve Weapon System Outcomes, GAO/ NSIAD- 99-
162 (Washington, D. C.: July 30, 1999). Space Policy May
Help Increase Insight into Gaps between Requirements and Resources
Page 11 GAO- 03- 1073 Space Acquisition Policy
will be required to develop an architecture that specifies the structure
of system components, their relationships, and the principles and
guidelines governing their design and evolution over time.
It is important for DOD to increase attention to requirements earlier in
the acquisition process and force DOD to think through whether there are
more cost- effective alternatives to pursue. A recent DOD study 7 found
that understanding of requirements often occurs too late to affordably
change the system and, more specifically, that space programs do not
always understand how systems fit in with other systems with which they
need to interact and that often a lack of mutual understanding of
requirements
exists between the government and contractors. The SBIRS independent
review team also found a need across space programs for more rigorous up
front development of requirements. In addition, in previous reviews, we
found that space programs often do not examine potentially more cost-
effective approaches. In 2001, for example, we reported 8 that DOD*s
SBIRS- Low program was not adequately analyzing alternatives to SBIRS- Low
that could satisfy critical missile defense requirements, such as Navy
ship- based radar capability. At the time, other studies supported the
possibility that other types of sensors could be used to track missiles in
the midcourse of their flight and to cue interceptors.
Third, the new policy seeks to improve the accuracy of cost estimates by
establishing an independent cost estimating process in partnership with
DOD*s Cost Analysis Improvement Group (CAIG) and by adopting methodologies
and tools used by the NRO. To ensure timely cost analyses, the CAIG will
augment its own staff with cost estimating personnel drawn
from across the entire national security space cost estimating community,
including cost estimating teams belonging to the intelligence communities,
the Air Force, NRO, the Army, and the Navy. The policy also calls on
programs to produce performance metrics that compare estimated to actual
costs. The policy allows programs to request assistance from the CAIG for
purposes other than DSAB reviews. However, there is no point in the
process that requires DOD to commit to fully fund a space program.
7 Booz Allen Hamilton, *Space Systems Development Growth Analysis,* Los
Angeles, CA, August 2, 2002. 8 U. S. General Accounting Office, Defense
Acquisitions: Space- Based Infrared System- Low At Risk of Missing Initial
Deployment Date, GAO- 01- 6 (Washington, D. C.:
February 28, 2001).
Page 12 GAO- 03- 1073 Space Acquisition Policy
Improving reliability of cost estimates is critical. Several of our
studies* such as ones on GPS, Evolved Expendable Launch Vehicle (EELV),
and AEHF* have called attention to problems with estimating system costs,
such as errors, omissions, and conflicting assumptions. For example, in
1980 we reported that the cost to acquire and maintain GPS satellites
through 2000 increased from $1.7 billion to $8.6 billion due largely to
estimates not previously included for replenishment of satellites,
launches, and user equipment. Moreover, recent DOD studies found initial
cost estimates for the AEHF program as well as SBIRS- High did not
accurately capture program content and risk and were based on optimistic
assumptions. We also reported that costs would be better estimated if DOD
required more knowledge before starting a program. Without knowing that
technologies can work as intended, for example, programs cannot reliably
estimate costs and schedules.
Another tool that could be useful in gaining insight into whether programs
are positioned for success is the IPA team. IPA teams are to be drawn from
experts who are not directly affiliated with the program. They are to
spend about 8 weeks on- site working full- time with program officials to
study the program, particularly by assessing the acquisition strategy,
contracting information, cost analyses, system engineering, and
requirements. After this study, they are to conclude their work with
recommendations to the DSAB on whether or not to allow the program to
proceed, typically using the traditional *red,* *yellow,* and *green*
assessment colors to indicate whether the program has satisfied key
criteria in areas such as requirements setting, cost estimates, and risk
reduction. The Under Secretary of the Air Force, however, makes the
decision on whether to allow the program to proceed.
IPA team studies already performed have called attention to risks faced by
the GPS III, NPOESS, and SBR programs. The NPOESS study, for example,
noted that risk mitigation plans needed to be strengthened and that
independent cost estimates needed to include the winning contractor*s
negotiated contract. The SBR study found that the program needed to better
define how the system would operate in the context of DOD*s
transformational communications architecture and work with key
intelligence systems, such as the planned Distributed Common Ground
Station. Both reviews recommended that the programs move forward (NPOESS
into the build phase and SBR into the study phase) on the condition that
these programs address areas of concern.
An IPA team studying GPS III found the program was too optimistic in
estimating resources that would be needed. For example, the study
Page 13 GAO- 03- 1073 Space Acquisition Policy
noted that the program budget was not sufficient to support the program
plan by several hundred million dollars. The team also pointed out that
the system*s architecture and acquisition strategy were not sufficiently
defined.
DOD*s new acquisition management policy for space systems does not alter
DOD*s practice of committing major investments before knowing what
resources will be required to deliver promised capability. Instead, the
policy allows programs to continue to mature technologies while they are
designing the system and undertaking other product development activities.
While space systems are different than other weapon systems in terms of
how they are developed and tested, it is still necessary to mature
technology before starting product development and match resources to
requirements in order to prevent cost increases and schedule delays.
We previously recommended that DOD should not allow technologies to enter
into a weapon system*s product development until they are assessed at a
TRL 7, meaning that a prototype has been demonstrated in an operational
environment. 9 According to DOD officials, the new space acquisition
policy does not set TRL criteria for deciding what the threshold for being
mature should be. However, DOD officials stated that technologies may well
enter into product development at a TRL 5, meaning basic components have
only been tested in a laboratory, or an even lower level of maturity. This
means that programs will design the system and conduct other program
activities at the same time they build representative models of key
technologies and test them in an
environment that simulates the conditions of space. In essence, DOD will
be concurrently building knowledge about technology and design* an
approach with a problematic history. As shown in figure 2, the knowledge
building approach for space stands in sharp contrast to that followed by
successful programs and the approach recommended by DOD*s new acquisition
policy for weapon systems.
Successful programs will not commit to undertaking product development
unless they have high confidence that they have achieved a match between
what the customer wants and what the program can deliver. Technologies
9 U. S. General Accounting Office, Best Practices: Better Management of
Technology Development Can Improve Weapon System Outcomes, GAO/ NSIAD- 99-
162 (Washington, D. C.: July 30, 1999). New Space Policy
Does Not Call for a Match between Resources and Requirements at Program
Start
Page 14 GAO- 03- 1073 Space Acquisition Policy
that are not mature continue to be developed in an environment that is
focused solely on technology development. This puts programs in a better
position to succeed because they can focus on design, system integration,
and manufacturing.
By contrast, allowing technology development to carry over into product
development increases the risk that significant problems will be
discovered late in development. Addressing such problems may require more
time, money, and effort to fix because they may require more extensive
retrofitting and redesign as well as retesting. The approach also makes it
more difficult for programs to demonstrate the same level of design
stability since technology and design activities will be done
concurrently. Further, the consequences of problems experienced during
development will be much greater for space programs since the design
review occurs at the same time as the commitment to build and deliver the
first product to a customer.
Page 15 GAO- 03- 1073 Space Acquisition Policy
Figure 2: DOD Will Be Making Commitments before Obtaining Critical
Knowledge for Space Systems
Space acquisition officials we spoke with acknowledged the added risks
that come when programs concurrently develop technologies and design the
system. However, they maintain that concurrent technology and product
development is necessary for space acquisitions for several reasons.
First, while some testing on satellites can be done on the ground in
thermovac or other environmental simulation chambers and some systems can
also be tested via aircraft, the only way to test satellites in a true
operational space environment is to build one or more demonstrator
satellites and launch them into orbit. Launching demonstrators is costly
and time consuming.
Our prior reports have recognized that space systems are uniquely
difficult to test in a true operational environment. However, DOD has
found ways to test sensors and other critical technologies on experimental
satellites and it has built and launched technology demonstrator
satellites.
Page 16 GAO- 03- 1073 Space Acquisition Policy
Second, in view of the length of time it takes to develop space systems,
DOD asserts that it will not be able to ensure that satellites, when
launched, will have the most advanced technologies, unless program
managers are continually developing technologies. DOD officials have
stated that they would reduce the added risks of their approach by not
allowing programs to start if too many technologies were deemed to be
immature or by deferring certain capabilities if it turned out that
technologies did not test well.
We agree that continuing to develop leading edge technology is important
for all system capabilities, not just space systems. However, history has
shown and we have repeatedly reported that conducting technology
development within a product environment consistently delays the delivery
of capability to the user, robs other programs of necessary funds through
unanticipated cost overruns, and consequently, can result in money wasted
and fewer units produced than originally stated as necessary. A technology
development environment is more forgiving and less costly than a delivery-
oriented acquisition program environment. Events such as test *failures,*
new discoveries, and time spent in attaining knowledge are considered
normal in this environment. Further, judgments of technology maturity have
proven to be insufficient as the basis for accurate estimates of program
risks relative to cost, schedule, and capability.
Finally, because operation and support costs make up a smaller portion
of total costs for satellites than other weapon programs, DOD asserts that
earlier insight and decisions are needed on space programs.
We agree that early insight into programs is important, as we have
reported that over 80 percent of the cost of a weapon system program is
determined by requirements set at the beginning. However, moving decisions
to an earlier point in the product development process without additional
knowledge may actually increase the risk of promising more than can be
delivered and at higher costs.
The growing importance of space systems to military and civil operations
requires DOD to develop cutting edge technologies and achieve timely
delivery of capability. DOD*s new space acquisition policy does not
position space programs to do either. By allowing major investment
commitments to continue to be made with unknowns about technology
readiness, requirements, and funding, programs will likely continue to
experience problems that require more time and money to address than
Conclusions
Page 17 GAO- 03- 1073 Space Acquisition Policy
anticipated. Over the long run, the extra investment required to address
these problems may well prevent DOD from pursuing more advanced
capabilities. By contrast, DOD is taking steps to better position other
weapon systems for success. By separating technology development and
product development, the policy will help to align customer expectations
with resources, and therefore minimize problems that could hurt the
program in its design and production phases.
In finalizing DOD*s new space acquisition management policy, we recommend
that the Secretary of the Air Force, who is DOD*s executive agent for
space, modify the policy to ensure that customer expectations can be
matched to resources before starting product development (phase B).
Specifically, we recommend that the Secretary separate technology
development from product development. To ensure that this is done, we also
recommend that the Secretary set a minimum threshold of
maturity for allowing technologies into a program. As noted in our report,
we previously recommended that DOD should not allow technologies to enter
into a weapon system*s product development until they are
assessed at a TRL 7, meaning that a prototype has been demonstrated in an
operational environment. 10 In commenting on a draft of this report, the
Assistant Secretary of Defense
for Networks and Information Integration disagreed with our finding that
the new space policy perpetuates risks for space programs since it does
not separate technology development from product development. DOD
disagreed with our recommendations as well, citing its need to keep up
with the fast- paced development of advanced technologies for space
systems and a requirement in its draft policy for technology readiness
assessments to be conducted at appropriate milestones.
In fact, it is DOD*s long- standing and continuous inability to bring the
benefits of technology to the warfighter in a timely manner that underlies
the report*s findings and recommendations. In our reviews of numerous DOD
programs, including many satellite developments, it has been clear that
committing to major investments in design, engineering, and
10 U. S. General Accounting Office, Best Practices: Better Management of
Technology Development Can Improve Weapon System Outcomes, GAO/ NSIAD- 99-
162 (Washington, D. C.: July 30, 1999). Recommendations for
Executive Action Agency Comments and Our Evaluation
Page 18 GAO- 03- 1073 Space Acquisition Policy
manufacturing capacity without knowing a technology is mature and what
resources are needed to ensure that the technology can be incorporated
into a weapon system has consistently resulted in more money, time, and
talent spent than either was promised, planned for, or necessary. The
impact of such mistakes in individual programs has also had a damaging
effect on military capability as other programs are taxed to meet
unplanned cost increases and production units are often cut because unit
costs increase and funds run out.
Although each DOD program differs in its characteristics, GAO*s work with
successful product developers in DOD and the commercial sector has found
that the process of developing leading edge technology and products that
have more capability than their predecessors does not differ. In fact,
successful product developments are marked by adherence to a disciplined
process that collects metrics and establishes and uses common and
consistent criteria for decision- making. We have found that companies
that adopt these best practices often do so out of necessity, when their
existence is threatened. While the Air Force has taken some promising
steps in drafting the policy to address DOD*s poor record of
developing satellites within cost and schedule targets and with promised
performance, it will miss an opportunity to dramatically improve outcomes
if it does not adopt similar practices. Therefore, we have not changed our
recommendation.
DOD*s detailed comments and our responses are provided in appendix III. In
conducting our review, we analyzed DOD*s new interim acquisition
management policy for space. Because of the limited time of our review, we
focused on the question of whether the policy will enable DOD to match
requirements to resources at the onset of product development, which our
work has shown to be the most critical determinant for successful outcomes
of acquisitions. We compared the new space policy to DOD*s new acquisition
policy for other weapon systems as well as our past reviews of the best
practices of commercial and military acquisitions. In addition, we
discussed this policy with Air Force space acquisition
officials. We analyzed IPA studies performed under the new policy on DOD*s
NPOESS, GPS III, and SBR programs. We also analyzed our past reviews of
space programs as well as DOD studies on the SBIRS- High program and on
space systems development growth. See Related GAO Products at the end of
this report for a list of past GAO reports we relied on. We conducted our
review from June 2003 through August 2003 in accordance with generally
accepted government auditing standards.
Page 19 GAO- 03- 1073 Space Acquisition Policy
We are sending copies of this report to the Secretaries of Defense and the
Air Force and interested congressional committees. We will also make
copies available to others upon request. In addition, the report will be
available at no charge on the GAO Web site at http:// www. gao. gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512- 4841. Key contributors to this report were
Cristina Chaplain, Jean Harker, Natalie Britton, and Bradley Terry.
Sincerely yours, Katherine V. Schinasi Director, Acquisition and Sourcing
Management
Appendix I: The Department of Defense*s Current and Planned Satellite
Systems Page 20 GAO- 03- 1073 Space Acquisition Policy
Function Current Systems Planned Systems
Missile warning and tracking Defense Support Program Space- Based
Infrared System (High)
Space Tracking and Surveillance System Intelligence, Surveillance and
Reconnaissance National Reconnaissance Office (NRO)
satellites
NRO satellites DOD*s Space- based Radar
Communications Wideband/ high capacity systems Defense Satellite
Communications System
Global Broadcasting Service Wideband Gapfiller Satellite
Advanced Wideband System Protected systems (antijam, survivable)
Milstar Advanced Extremely High Frequency
Advanced Polar System Narrowband systems Ultra High Frequency Follow-
On satellite
communications system
Mobile User Objective System Navigation, Positioning, Timing Global
Positioning System (GPS) Next Generation GPS Weather/ Environmental
Defense Meteorological Satellite Program National Polar- orbiting
Operational
Environmental Satellite System Source: GAO. Appendix I: The Department of
Defense*s Current and Planned Satellite Systems
Appendix II: Technology Readiness Levels and Their Definitions Page 21
GAO- 03- 1073 Space Acquisition Policy
Technology readiness level Description
1. Basic principles observed and reported. Lowest level of technology
readiness. Scientific research begins to be translated into applied
research and development. Examples might include paper studies of a
technology*s basic properties.
2. Technology concept and/ or application formulated. Invention begins.
Once basic principles are observed, practical applications can be
invented. The application is speculative and there is no proof or detailed
analysis to support the assumption. Examples are still limited to paper
studies. 3. Analytical and experimental critical function and/ or
characteristic proof of concept. Active research and development is
initiated. This includes analytical studies and laboratory studies to
physically validate analytical predictions of separate
elements of the technology. Examples include components that are not yet
integrated or representative.
4. Component and/ or breadboard validation in laboratory environment.
Basic technological components are integrated to establish that the pieces
will work together. This is relatively *low fidelity* compared to the
eventual system. Examples include integration of *ad hoc* hardware in a
laboratory. 5. Component and/ or breadboard validation in relevant
environment. Fidelity of breadboard technology increases significantly.
The basic technological components are integrated with reasonably
realistic supporting
elements so that the technology can be tested in a simulated environment.
Examples include *high fidelity* laboratory integration of components.
6. System/ subsystem model or prototype demonstration in a relevant
environment. Representative model or prototype system, which is well
beyond the
breadboard tested for technology readiness level (TRL) 5, is tested in a
relevant environment. Represents a major step up in a technology*s
demonstrated readiness. Examples include testing a prototype in a high
fidelity laboratory environment or in simulated operational environment.
7. System prototype demonstration in an operational environment. Prototype
near or at planned operational system. Represents a major step up from TRL
6, requiring the demonstration of an actual system prototype in an
operational environment, such as in an aircraft, vehicle or space.
Examples include testing the prototype in a test bed aircraft. 8. Actual
system completed and *flight qualified* through test and demonstration.
Technology has been proven to work in its final form and under expected
conditions. In almost all cases, this TRL represents the end of true
system development. Examples include developmental test and evaluation of
the system in its intended weapon system to determine if it meets design
specifications. 9. Actual system *flight proven* through successful
mission operations. Actual application of the technology in its final form
and under mission conditions, such as those encountered in operational
test and evaluation. In
almost all cases, this is the end of the last *bug fixing* aspects of true
system development. Examples include using the system under operational
mission conditions.
Source: GAO. Appendix II: Technology Readiness Levels and Their
Definitions
Appendix III: Comments from the Department of Defense Page 22 GAO- 03-
1073 Space Acquisition Policy
Appendix III: Comments from the Department of Defense
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
Appendix III: Comments from the Department of Defense Page 23 GAO- 03-
1073 Space Acquisition Policy
See comment 1.
Appendix III: Comments from the Department of Defense Page 24 GAO- 03-
1073 Space Acquisition Policy
See comment 4. See comment 3.
See comment 2.
Appendix III: Comments from the Department of Defense Page 25 GAO- 03-
1073 Space Acquisition Policy
See comment 7. See comment 6.
See comment 5.
Appendix III: Comments from the Department of Defense
Page 26 GAO- 03- 1073 Space Acquisition Policy
The following are GAO*s comments on the Department of Defense*s letter
dated September 5, 2003.
1. We agree that there are consistencies between the two policies in terms
of how they enhance the development of requirements. However, the policies
are very different in terms of their views on technology development.
DOD*s policy for weapon systems clearly requires technologies to be mature
(demonstrated in a relevant, preferably operational environment) before
beginning product development. The space policy does not. In fact, DOD
officials stated that, under the space policy, technologies may well enter
product development without being demonstrated in a relevant environment.
This might not occur until DOD is close to making its production decision.
In our view, this difference will be a detriment to the future success of
space programs.
2. DOD contended that our recommendation to set a minimum threshold of
maturity for allowing technologies into a program ignores differences
among programs and ignores evolutionary acquisition. We disagree with
these points. Technology maturity is fundamental to the success of all
programs and cannot be ignored as part of a satellite*s business case.
While it is possible to take a gamble on a key technology and have it work
out in the end, DOD*s experiences show that this is an unlikely result.
Moreover, this is not an approach that successful product developers
emulate. In addition, technology maturity is essential to successful
evolutionary acquisitions. The principle of evolutionary development is
reaching full capability in more doable steps. Technical maturity
essentially defines what is doable for each increment or block. 3. DOD
asserted that it is not feasible for space programs to separate
technology development from product development because it would delay
delivery of the product and make its technologies obsolete. We disagree.
Separation of technology development from product
development has been found to be essential to reducing overall development
cycle times and delivering new products within estimated resources. The
DOD policy for other weapons acquisitions is quite clear on this as well.
In successful programs, the technologies are matured, hybrid organizations
and agreements between the technologists and the product developers are
established, and preliminary designs are done, thus providing the basis
for a match between the user's needs and the developer's resources-- all
before the
GAO Comments
Appendix III: Comments from the Department of Defense
Page 27 GAO- 03- 1073 Space Acquisition Policy
commitment to product development is made. By maturing technologies before
committing significant time and money to product development and following
an evolutionary approach, the product development cycle time is reduced,
while opportunities for inserting new technologies are more frequent. 4.
DOD asserted that satellite programs cannot be demonstrated in an
operational environment (TRL 7). We disagree. NASA, the creator of TRLs,
tests some technologies to a TRL 7 if they are mission critical. Moreover,
while we recognize the difficulties in attaining this level of maturity
for space systems, the space policy does not even encourage programs to
demonstrate technologies in a relevant environment before committing to a
program. In fact, according to DOD officials, under the space policy,
technologies could enter product development with a TRL 5 or even lower.
The policy is silent on what the minimum threshold for maturity should be,
leaving that decision to the milestone decision authority.
5. DOD stated that none of our prior best practices case studies included
a commercial satellite producer, making the knowledge points irrelevant to
space systems. This assertion is wrong. In the report that first
promulgated the knowledge points (GAO/ NSIAD- 98- 56), one of the key case
studies was Hughes Space and Communications and its experience with the
HS- 702 satellite. We deliberately included Hughes because it was a low-
volume, high technology producer. Hughes insisted on having process
control for all key processes and proved them either through use on other
satellite production or through statistical process control techniques.
Hughes was also included as part of our best practice study on technology
development (GAO/ NSIAD- 99- 162).
6. DOD asserted that moving decision points to an earlier point in the
program reduces risks, rather than increases them as our report states. We
disagree. The space policy proposes to make commitments to product
development (including point estimates on cost, schedule, and performance)
before sufficient knowledge has been achieved and requires decision makers
to commit first to product development without having technology in hand
and second to production of the first two products without production
knowledge in hand. This is the traditional DOD approach, which has
consistently resulted in capability being delivered much later and much
more expensively than planned. The commitment to product development (and
the requisite
Appendix III: Comments from the Department of Defense
Page 28 GAO- 03- 1073 Space Acquisition Policy
estimates) can be done more confidently and the product development cycle
time can be much shorter only if decisions are knowledge- based.
7. While officials have told us that the intent of the policy is to
complete technology development during phase B, they acknowledged that the
policy does not identify an end point for technology development and that,
in some cases, it could continue until the point the program is ready to
begin building the first satellite.
Related GAO Products Page 29 GAO- 03- 1073 Space Acquisition Policy
Military Space Operations: Common Problems and Their Effects on Satellite
and Related Acquisitions. GAO- 03- 825R. Washington, D. C.: June 2, 2003.
Polar- Orbiting Environmental Satellites: Project Risks Could Affect
Weather Data Needed by Civilian and Military Users. GAO- 03- 987T.
Washington, D. C.: July 15, 2003.
Missile Defense: Alternate Approaches to Space Tracking and Surveillance
System Need to Be Considered. GAO- 03- 597. Washington, D. C.: May 23,
2003.
Military Space Operations: Planning, Funding, and Acquisition Challenges
Facing Efforts to Strengthen Space Control. GAO- 02- 738. Washington, D.
C.: September 23, 2002.
Polar- Orbiting Environmental Satellites: Status, Plans, and Future Data
Management Challenges. GAO- 02- 684T. Washington, D. C.: July 24, 2002.
Defense Acquisitions: Space- Based Infrared System- Low at Risk of Missing
Initial Deployment Date. GAO- 01- 6. Washington, D. C.: February 28, 2001.
Defense Acquisitions: Assessments of Major Weapon Programs.
GAO- 03- 476. Washington, D. C.: May 15, 2003.
Defense Acquisitions: Matching Resources With Requirements Is Key to the
Unmanned Combat Air Vehicle Program*s Success. GAO- 03- 598. Washington,
D. C.: June 30, 2003.
Best Practices: Better Acquisition Outcomes Are Possible If DOD Can Apply
Lessons from F/ A- 22 Program. GAO- 03- 645T. Washington, D. C.: April 11,
2003.
Best Practices: Setting Requirements Differently Could Reduce Weapon
Systems* Total Ownership Costs. GAO- 03- 57. Washington, D. C.: February
11, 2003.
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. GAO- 02- 701. Washington, D. C.: July 15,
2002. Related GAO Products Space Reports
Best Practice Reports
Related GAO Products Page 30 GAO- 03- 1073 Space Acquisition Policy
Defense Acquisitions: DOD Faces Challenges in Implementing Best Practices.
GAO- 02- 469T. Washington, D. C.: February 27, 2002. Best Practices: DOD
Teaming Practices Not Achieving Potential Results.
GAO- 01- 510. Washington, D. C.: April 10, 2001.
Best Practices: Better Matching of Needs and Resources Will Lead to Better
Weapon System Outcomes. GAO- 01- 288. Washington, D. C.: March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better Weapon
System Outcomes. GAO/ NSIAD- 00- 199. Washington, D. C.: July 31, 2000.
Defense Acquisitions: Employing Best Practices Can Shape Better
Weapon System Decisions. GAO/ T- NSIAD- 00- 137. Washington, D. C.: April
26, 2000.
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes. GAO/ NSIAD- 99- 162. Washington, D. C.: July 30,
1999.
Best Practices: Successful Application to Weapons Acquisitions Requires
Changes in DOD*s Environment. GAO/ NSIAD- 98- 56. Washington, D. C.:
February 24, 1998.
(120266)
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail
this list to you every afternoon, go to www. gao. gov and select
*Subscribe to e- mail alerts* under the *Order GAO Products* heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000
TDD: (202) 512- 2537 Fax: (202) 512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470 Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
4800
U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548 GAO*s Mission Obtaining Copies of
GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs Public Affairs
*** End of document. ***