Space Transportation: Challenges Facing NASA's Space Launch
Initiative (17-SEP-02, GAO-02-1020).
In 2001, the National Aeronautics and Space Administration (NASA)
began undertaking a new effort--the Space Launch Initiative
(SLI)--to develop a new generation of space transportation
vehicles. SLI is expected to result in development of the second
generation of reusable launch vehicles, the space shuttle being
the first generation. NASA plans to define basic requirements for
its second-generation reusable launch vehicle--that is, what the
crew size will be, what the payload capacity will be, and what
designs or architectures are worth pursuing--by November 2002.
However, considerable challenges must be addressed before NASA
can accomplish this. First, NASA has to complete a reassessment
of its overall space transportation plans. Second, NASA is
currently reassessing the future of the International Space
Station. The decisions it will make as part of this evaluation,
such as how many crew will operate the station, will have a
dramatic impact on NASA's requirements for a second-generation
vehicle. Third, NASA needs to decide whether the SLI program will
be developed jointly with the Department of Defense (DOD) and, if
so, how can it accommodate DOD's requirements for a reusable
launch vehicle. Until NASA finalizes its basic requirements for
SLI, it cannot implement management controls that are essential
to predicting what the total costs of the program will be and to
minimizing the risks with NASA's planned initial investment of
$4.8 billion. It is important for NASA to implement management
controls for SLI as soon as possible, so that it can provide its
managers and Congress with the information needed to ensure that
the program is on track and able to meet expectations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-1020
ACCNO: A05087
TITLE: Space Transportation: Challenges Facing NASA's Space
Launch Initiative
DATE: 09/17/2002
SUBJECT: Aerospace industry
Aerospace research
Cost control
Interagency relations
Internal controls
Space exploration
Transportation costs
Transportation research
NASA Space Launch Initiative
NASA X-33 Program
NASA X-34 Program
Space Transportation: Challenges Facing NASA's Space Launch
Initiative (17-SEP-02, GAO-02-1020).
In 2001, the National Aeronautics and Space Administration (NASA)
began undertaking a new effort--the Space Launch Initiative
(SLI)--to develop a new generation of space transportation
vehicles. SLI is expected to result in development of the second
generation of reusable launch vehicles, the space shuttle being
the first generation. NASA plans to define basic requirements for
its second-generation reusable launch vehicle--that is, what the
crew size will be, what the payload capacity will be, and what
designs or architectures are worth pursuing--by November 2002.
However, considerable challenges must be addressed before NASA
can accomplish this. First, NASA has to complete a reassessment
of its overall space transportation plans. Second, NASA is
currently reassessing the future of the International Space
Station. The decisions it will make as part of this evaluation,
such as how many crew will operate the station, will have a
dramatic impact on NASA's requirements for a second-generation
vehicle. Third, NASA needs to decide whether the SLI program will
be developed jointly with the Department of Defense (DOD) and, if
so, how can it accommodate DOD's requirements for a reusable
launch vehicle. Until NASA finalizes its basic requirements for
SLI, it cannot implement management controls that are essential
to predicting what the total costs of the program will be and to
minimizing the risks with NASA's planned initial investment of
$4.8 billion. It is important for NASA to implement management
controls for SLI as soon as possible, so that it can provide its
managers and Congress with the information needed to ensure that
the program is on track and able to meet expectations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-1020
ACCNO: A05087
TITLE: Space Transportation: Challenges Facing NASA's Space
Launch Initiative
DATE: 09/17/2002
SUBJECT: Aerospace industry
Aerospace research
Cost control
Interagency relations
Internal controls
Space exploration
Transportation costs
Transportation research
NASA Space Launch Initiative
NASA X-33 Program
NASA X-34 Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-1020
Report to the Subcommittee on Space and Aeronautics, Committee on Science,
House of Representatives
United States General Accounting Office
GAO
September 2002 SPACE TRANSPORTATION
Challenges Facing NASA*s Space Launch Initiative
GAO- 02- 1020
Page i GAO- 02- 1020 Space Transportation Letter 1
Results in Brief 2 Background 3 Important Decisions to Be Made Before
Requirements Can Be
Defined 7 Key Management Controls Are Not Yet Implemented 9 Conclusions 14
Recommendations 15 Agency Comments 16 Scope and Methodology 17
Appendix I Comments from the National Aeronautics and Space Administration
19
Appendix II GAO Contact and Staff Acknowledgments 22
Table
Table 1: Activities Related to the Space Launch Initiative 7
Figures
Figure 1: NASA Illustration of the 2nd Generation Reusable Launch Vehicle
4 Figure 2: Illustration of NASA*s Overall Space Transportation Plans 5
Abbreviations
DOD Department of Defense EVM Earned Value Management NASA National
Aeronautics and Space Administration RFP Request for Proposal SLI Space
Launch Initiative TRL technology readiness level Contents
Page 1 GAO- 02- 1020 Space Transportation
September 17, 2002 The Honorable Dana Rohrabacher Chairman The Honorable
Bart Gordon Ranking Member Subcommittee on Space and Aeronautics Committee
on Science House of Representatives
In 2001, the National Aeronautics and Space Administration (NASA) began
undertaking a new effort* the Space Launch Initiative (SLI)* to develop a
new generation of space transportation vehicles. SLI is expected to result
in the development of the second generation of reusable launch vehicles,
the space shuttle being the first generation. NASA plans to spend $4.8
billion on the program through fiscal year 2006. SLI is part of a broader
program* known as NASA*s Integrated Space Transportation Plan* to address
future space transportation needs. Under the plan, NASA envisions making
upgrades to extend the life of the space shuttle and undertaking longer-
term research and development of future transportation technologies and
transportation systems, including space vehicles that can reach orbit in
just one- stage.
SLI is a highly ambitious program. It will require NASA to develop and
advance new technologies, such as propulsion and airframe systems, which
in turn can potentially be used by U. S. industry to create new business
opportunities in space. The undertaking will also require a high level of
communication and coordination between a range of partners, including
private- sector contractors, academia, and the Department of Defense
(DOD). Moreover, it will require effective controls and oversight to
reduce cost, scheduling, and technical risks. NASA*s previous attempts to
develop a new generation of space vehicles were unsuccessful largely
because NASA did not successfully implement and adhere to critical project
management controls and activities.
You requested that we assess NASA*s progress with the Space Launch
Initiative, particularly with respect to defining requirements and
implementing management controls.
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 02- 1020 Space Transportation
NASA plans to define basic requirements for its second- generation
reusable launch vehicle* that is, what the crew size will be, what the
payload capacity will be, and what designs or architectures are worth
pursuing* by November 2002. But considerable challenges must be addressed
before NASA can accomplish this.
First, NASA has to complete a reassessment of its overall space
transportation plans. In doing so, it must decide whether it should
continue pursuing the development of second- generation vehicles as
planned, pursue alternative ways to develop the second generation in order
to more quickly replace the space shuttle, or postpone these efforts
altogether indefinitely until there is a major breakthrough in technology
that could vastly improve performance and reduce costs. This decision will
be difficult, given the uncertainties about the availability of
technologies needed to reduce costs and enhance performance for future
space flight.
Second, NASA is currently reassessing the future of the International
Space Station. The decisions it will make as part of this evaluation, such
as how many crew will operate the station, will have a dramatic impact on
NASA*s requirements for a second- generation vehicle. But they will be
difficult to reach, since they require NASA to come to agreement with
international partners who are concerned about planned cutbacks to the
station*s capabilities.
Third, NASA needs to decide whether the SLI program will be developed
jointly with DOD and, if so, how it can accommodate DOD*s requirements for
a reusable launch vehicle. So far, indications are that NASA and DOD will
share many of the same objectives for the vehicle, but there are
significant differences in priorities and requirements.
Until NASA finalizes its basic requirements for SLI, it cannot implement
management controls that are essential to predicting what the total costs
of the program will be and to minimizing risks with NASA*s planned initial
investment of $4.8 billion. These include cost estimates, controls
designed to provide early warnings of cost and schedule overruns, and risk
mitigation plans. Moreover, there are potential impediments to NASA*s
development and effective use of a detailed cost estimate, including the
lack of a modern integrated financial management system. Lastly, NASA does
not plan to develop several measures that are important to assessing how
the program as a whole is making progress toward achieving its key
objectives, reducing risks, and maturing technology. Results in Brief
Page 3 GAO- 02- 1020 Space Transportation
It is important for NASA to implement management controls for SLI as soon
as possible, so that it can provide its managers and the Congress with the
information needed to ensure that the program is on track and able to meet
expectations. We are making recommendations to NASA that focus on the need
to make decisions with regard to the future of NASA*s overall space
transportation plan, the future of the space station, and DOD*s
participation in the SLI program before setting requirements for SLI. We
are also making recommendations aimed at implementing management controls
for the SLI program.
In its comments to a draft of this report, NASA stated that it concurs
with the recommendations. NASA believes that much of the SLI program*s
success is directly related to the implementation of project management
controls and appropriate levels of insight. NASA*s response is included as
appendix 1.
NASA*s Space Launch Initiative is an effort to develop and build a second
generation of reusable space transportation vehicles. (See fig. 1 and
table 1.) NASA*s current transportation vehicle, the space shuttle, has
been in use for 20 years and requires a significant portion of NASA*s
resources to operate and maintain. The primary goals for SLI are to reduce
the risk of crew loss as well as substantially lower the cost of space
transportation so that more funds can be made available for scientific
research, technology development, and exploration activities. Currently,
NASA spends nearly one- third of its budget on space transportation.
Background
Page 4 GAO- 02- 1020 Space Transportation
Figure 1: Illustration of the 2nd Generation Reusable Launch Vehicle
Source: NASA.
SLI is part of a broader program* known as NASA*s Integrated Space
Transportation Plan* to address space transportation needs. Under the
plan, NASA could operate the space shuttle through 2020 and make software
and hardware upgrades to the shuttle in order to extend its use to this
point. It envisions the deployment of second- generation cargo vehicles to
begin around 2011 and crew vehicles around 2014. As with the shuttle, NASA
envisions that the second- generation vehicle will reach orbit in two
stages. NASA also anticipates building a third generation of vehicles in
2025 and even a fourth generation in 2040. (See fig. 2.) NASA anticipates
that these vehicles would reach orbit in one stage; travel beyond low-
earth orbit to far- reaching interstellar missions; and employ
revolutionary technologies, such as (1) rocket engines that breath oxygen
from the air during the climb to orbit rather than carrying heavy
oxidizers onboard, (2) propellantless electromagnetic propulsion, and (3)
solar- powered space sails.
Page 5 GAO- 02- 1020 Space Transportation
Figure 2: Illustration of NASA*s Overall Space Transportation Plans
Source: NASA.
Building the second- generation vehicle will be a considerably complex and
challenging endeavor for NASA* from both a technical and business
standpoint. For example, NASA plans to develop and advance new
technologies for the new vehicle. These include (1) new airframe
technologies that will include robust, low- cost, low- maintenance
structure, tanks, and thermal protection systems, using advanced ceramic
and metallic composite materials, and (2) new propulsion technologies,
including main propulsion systems, orbital maneuvering systems, main
engines, and propellant management. If successfully developed, NASA
believes that these new technologies could substantially reduce the costs
to maintain and operate the vehicle and increase its reliability.
NASA also plans to develop the new vehicle through partnerships with
private industry, academia, and DOD. It is partnering with the private
sector so that it can help create business opportunities in space, and it
is partnering with DOD because of military needs for a reusable launch
vehicle. Such partnerships will require a high level of coordination and
communication, since agreements need to be reached on what the basic
capabilities of the new vehicle will be, what designs or architectures 1
should be pursued, how development costs will be shared, and what
individual partner responsibilities will be.
NASA recognizes that there are substantial technical and business risks
with SLI, and it is undertaking activities aimed at reducing them. To
reduce technical risks, for example, NASA is exploring, developing, and
testing technologies to make sure that they can be used on the new
1 *Architecture* broadly includes an Earth- to- orbit reusable launch
vehicle; on- orbit transfer vehicles and upper stages; mission planning;
ground and flight operations; and support infrastructure, both on the
ground and in orbit.
Page 6 GAO- 02- 1020 Space Transportation
vehicle, rather than proceeding with the program with uncertainty about
whether technologies will be mature enough when they are needed. To reduce
business and costs risks, NASA*s goal is to ensure that there is adequate
competition in the development of the architecture and that the
architectures that NASA pursues will enable convergences with NASA*s
requirements and commercial and military missions, and will not pursue
vehicles that cannot meet program goals.
To date, NASA has already explored hundreds of concepts and technologies
that could be used for future government and commercial launch systems and
space transportation operations. The *leap- ahead* technologies studied
include crew survival systems, advanced tanks and airframe structures,
long- life rocket engines, and thermal protection systems. In March 2002,
NASA selected 15 of the most promising candidates to go forward into more
detailed development. In November 2002, NASA plans to narrow the field
down to 3 candidates and in September 2003 to at least 2 candidates. From
fiscal year 2003 through fiscal 2006, two competing system architectures
will be developed, ready for a full- scale development decision in fiscal
2006.
NASA plans to spend $4.8 billion for the SLI program through fiscal year
2006. This investment is to be used for (1) systems engineering and
activities concerning the definition of requirements; (2) efforts to
compete designs for the reusable launch vehicle as well as efforts to
reduce business and technical risks; (3) activities to develop and
demonstrate designs, technologies, and system- level integration issues
associated with such NASA- unique transportation elements as a crew
transport vehicle and cargo carriers; and (4) activities to develop
alternative access to the space station as well as to provide contingency
backup or relief for the space shuttle. NASA*s efforts to compete designs
for the vehicle and reduce technical and business risks consist of
approximately $3 billion of the planned initial investment. At the present
time, no prototype vehicle integrating all new SLI technologies is planned
before NASA*s fiscal year 2006 decision on whether to proceed to full-
scale development.
Page 7 GAO- 02- 1020 Space Transportation
Table 1: Activities Related to the Space Launch Initiative Year Month
Event Narrative
2001 Feb. SLI program approved NASA plans to budget $4. 8 billion for
the program through fiscal year 2006. 2001 May Initial contracts awarded
to
22 contractors The contracts, valued at almost $800 million, were not
intended to provide a specific vehicle design, but rather to explore
concepts
and technologies that could be used for future government and commercial
launch systems and space transportation operations. The leap- ahead
technologies explored included crew survival systems, advanced tanks and
airframe structures, long- life rocket engines, and thermal protection
systems. 2002 Mar. Interim Architecture and Technology
Review Design concepts for the second- generation vehicle narrowed down
from hundreds to 15.
2002 Nov. Systems Requirements Review This review is to focus attention
on fewer space transportation architectures and technology areas, to
select three architectures that can be pursued, and reach agreement on the
development of system requirements. 2003 Feb. Request for Proposals
(RFP) to
select SLI designs RFPs are to focus on selecting the most promising
architectures to proceed toward a detailed preliminary design.
2003 Sept. Contract awards for selection of two designs Second phase of
formulation program to further develop concepts
and risk- reduction activities. 2006 SLI full- scale development
decision Decision for selecting architecture to enter full- scale
development. 2009 Potential prototype vehicle available Reusable
unmanned vehicle with limited capabilities. 2012 Cargo vehicle available
Reusable unmanned vehicle with advanced engines and tanks. 2014 Crew
vehicle available Reusable launch vehicle with crew capabilities
Source: Discussions with and documentation from NASA.
According to a NASA official, NASA plans to define the basic requirements
for its second- generation space transportation* that is, what the crew
size will be, what the payload capacity will be, and what designs or
architectures are worth pursuing* by November 2002. These decisions will
have a significant impact on the cost, size, and design of the new
vehicle. For example, as the payload capacity increases, so does the
thrust requirement for the propulsion system, and in turn, the cost to
develop and build the system, as well as the cost to operate the system.
However, NASA is facing a considerable challenge in reaching the point to
where it can finalize SLI requirements. This is primarily because NASA
must first make some difficult broader decisions regarding the future of
space transportation and other NASA projects, including (1) whether NASA
should skip development of a second- generation vehicle in favor of
concentrating on the third generation, (2) what the future of the
International Space Station will be, and (3) what DOD*s role in developing
the new vehicle will be. Moreover, in making these decisions, NASA will
need to reach consensus with a wide range of partners who have priorities
Important Decisions
to Be Made Before Requirements Can Be Defined
Page 8 GAO- 02- 1020 Space Transportation
and concerns different from NASA*s, and it will need to make trade- offs
amid uncertainties as to the availability of advanced technologies.
First, NASA must complete its ongoing reassessment of its overall space
transportation plan. This evaluation is being done as part of NASA*s
development of a budget proposal for fiscal year 2004. The options NASA is
considering could have a far- reaching impact on the SLI program. One
option, in fact, involves postponing efforts to develop a new generation
of vehicles indefinitely until there is a major breakthrough in technology
that could vastly improve performance and reduce costs. Other options
being examined could drastically change NASA*s timetable and requirements
for SLI. For example, NASA is looking at developing a crewed vehicle more
quickly than currently planned and launching it atop an expendable rocket
until it can field a reusable launch vehicle. NASA is also considering
deferring some development efforts so that it can aggressively pursue a
goal of building a relaunchable vehicle prototype by 2009. Finalizing its
decisions on where to go with the space transportation plan will be
difficult for NASA, given the uncertainties about the availability of
technologies needed to reduce costs and achieve NASA*s performance and
safety objectives.
Second, NASA is currently reassessing the future of the International
Space Station. One decision it needs to make that could significantly
affect the SLI program is whether the station should support a crew of
seven astronauts, as originally planned, or three. Because of cost growth,
NASA plans to cut back to a crew of three, but its international partners
have not agreed yet to this decision and are concerned that the cutback
will severely undermine planned scientific research. This decision could
significantly affect the design and cost of the second- generation
vehicle. For example, with a bigger crew size, the overall vehicle will
have to be larger; a larger crew cabin and additional backup systems will
be required; and, as a result, the cost to develop the vehicle will
increase.
NASA is also planning to cut back the number of flights to the station
from seven to four per year, which could also have an impact on SLI. But
again, NASA*s international partners have concerns about this decision,
since it would limit the deliveries of resources needed to carry out
research activities. At the same time that NASA is looking at cutting back
on space station capabilities, it is planning to extend the life of its
space shuttle to 2020 with software and hardware upgrades. If NASA can
successfully extend the life of the shuttle, it may well find that a
second- generation vehicle, which is not expected to begin transporting
crews to the space station until at least 2014, would generally be
duplicative in capability and
Page 9 GAO- 02- 1020 Space Transportation
therefore unnecessary. NASA plans to make final decisions on the space
station by November/ December 2002.
Third, while DOD and NASA have explored potential common areas of interest
for a new space transportation vehicle, DOD has not yet formally defined
its requirements. In fact, it is still uncertain whether SLI will be a
joint DOD/ NASA program. A study conducted by the Air Force and NASA
earlier this year revealed that both NASA and DOD shared similar
objectives when it came to technologies needed for the new vehicle as well
as cargo lift requirements and launch architecture elements. But there
were differences with priorities and certain requirements. The Air Force
would like the vehicle to operate in stronger winds, more precipitation,
and a wider range of temperatures. It would also like the vehicle to
operate from an inland U. S. Air Force base. NASA would like the vehicle
to stay in orbit for a longer duration and have more maneuverability
because of its mission to service the International Space Station.
Additionally, NASA will have a higher weight- delivered- to- orbit
requirement for its crew vehicle.
It is apparent that some of DOD*s objectives and priorities may not match
up with NASA*s. For example, DOD envisions developing an unpiloted
vehicle, while NASA is focusing on a crewed design. DOD also envisions
developing a vehicle that could be capable of relaunching within 12 to 48
hours and making as many as 20 flights in a 2- week time frame. At this
time, NASA does not share these objectives. According to a DOD official,
DOD expects to finalize its requirements before the end of 2002. Since
DOD*s decisions will affect the size, design, and capabilities of the
vehicle, it is important that these requirements be known before NASA
finalizes its own requirements in November 2002. Otherwise, NASA will need
to reexamine its requirements after narrowing down potential system
architectures to three possibilities. While significant cost and
operational benefits may accrue with a vehicle capable of satisfying both
DOD*s and NASA*s requirements, NASA will need to guard against making
compromises that might negatively affect its goal of substantially
lowering launch costs.
NASA cannot implement key management controls for the SLI program until it
defines its basic requirements. These include cost estimates, controls
designed to provide early warnings of cost and schedule problems, as well
as risk mitigation plans. Moreover, NASA does not have plans to implement
some performance measures, including ones that would assess overall
progress toward achieving key objectives, reducing Key Management
Controls Are Not Yet Implemented
Page 10 GAO- 02- 1020 Space Transportation
risks, and maturing technology. It is essential that these controls be
implemented quickly so that NASA can predict what the total costs of the
program will be and provide assurance that its investment in SLI is being
spent wisely.
Undertaking ambitious, technically challenging efforts like SLI* which
involve multiple contractors and technologies that have to be developed
and proven* requires careful oversight and management. Importantly,
accurate and reliable cost estimates need to be developed, technical and
program risks need to be anticipated and mitigated, and performance and
readiness need to be closely monitored. Such undertakings also require a
high level of communication and coordination. Not carefully implementing
such project management tools and activities is a recipe for failure.
Without realistically estimating costs and risks, and providing budgetary
reserves needed to mitigate those risks, management may not be in a
position to effectively deal with the technical problems that cutting-
edge projects invariably face.
In fact, we found that NASA did not successfully implement and adhere to a
number of critical project management tools and activities in its previous
efforts to build new space transportation vehicles. As we testified in
June 2001, neither of NASA*s X- 33 or X- 34 programs* which attempted to
build a new transportation vehicle that would reach orbit in one stage*-
fully assessed the costs associated with developing new, unproven
technologies; provided for the financial reserves needed to deal with
technical risks and accommodate normal development delays; developed plans
to quantify and mitigate risks to NASA; or established performance targets
showing a clear path to an operational launch vehicle. 2 Underlying these
difficulties were problems with the agreements and contracts that
established the relationship between NASA and its industry partners and
the eventual erosion of commercial prospects for the development of new
reusable launch vehicles. We testified that lax management controls led to
numerous problems with both the X- 33 and X- 34 programs. Technical
problems, for example, resulted in significant schedule and cost overruns,
which NASA was ill prepared to deal with.
2 See U. S. General Accounting Office, Space Transportation: Critical
Areas NASA Needs to Address in Managing Its Reusable Launch Vehicle
Program, GAO- 02- 826T (Washington, D. C.: June 20, 2001). Importance of
Management Controls for SLI
Page 11 GAO- 02- 1020 Space Transportation
NASA has taken steps to avoid the problems it encountered with the X- 33
and X- 34 programs. In our testimony last year, for example, we pointed
out that NASA planned to increase the level of insight into SLI projects
by providing more formal reviews and varying levels of project
documentation from contractors depending on the risk involved and the
contract value. NASA also required that all proposals submitted in
response to its research announcement be accompanied by certifiable cost
and pricing data. Finally, NASA discouraged the use of cooperative
agreements, since these agreements did not prove to be effective
contracting mechanisms for research and development efforts where large
investments are required.
NASA cannot develop cost estimates for the SLI program until it defines
the basic requirements for the new transportation vehicle and narrows the
field of possible architectures. Moreover, until requirements are defined,
NASA cannot determine whether the $4.8 billion already planned for SLI
through fiscal year 2006 will achieve the desired results for the
program*s formulation. NASA guidance requires that life- cycle costs be
estimated, assessed, and controlled throughout a program*s life cycle. 3
The estimates are to be prepared to support major program reviews and the
development of budget submissions.
In our recent review of the International Space Station, we reported that
NASA is facing additional challenges to developing reliable cost estimates
and effectively using them to manage programs. 4 For example, a recent
study performed by the Rand Corporation for the Office of Science and
Technology Policy found that NASA has *very good* cost and risk modeling
capabilities. However, the study also found that NASA*s in- house
capabilities were not well integrated into the program*s planning and
management and that NASA programs had been reluctant to integrate cost
estimate and control practices that were sufficiently robust to yield
confidence in budget estimates. In addition, a task force appointed last
year to conduct an independent external review and assessment of the space
station*s cost, budget, and management found that NASA tends to take a
short- term focus on executing programs* that is, rather than
3 See NASA Policy Directive 7120. 4A, Program/ Project Management and NASA
Procedures and Guidelines 7120. 5A, Program and Project Management
Processes and Requirements. 4 See U. S. General Accounting Office, Space
Station: Actions Under Way to Manage Cost, but Significant Challenges
Remain, GAO- 02- 735 (Washington, D. C.: July 17, 2002).
Cost Estimates Cannot Be Developed Until Requirements Are Defined
Page 12 GAO- 02- 1020 Space Transportation
managing programs to come within overall cost and scheduling goals, it
manages them around annual budgets. The task force cited NASA*s culture of
managing the space station program in adherence to its annual budgets as
perhaps the single greatest factor in the space station program*s cost
growth.
We also reported that NASA*s ability to develop good cost estimates for
programs is hampered by NASA*s lack of a modern integrated financial
system to track and maintain data needed for estimating and controlling
costs. NASA has made it a top priority to develop and implement a new
system. However, the first major component of the system* the core
financial system* is not expected to be implemented until June 2003.
It will be important for NASA to overcome barriers that relate to
estimating costs for SLI and to use estimates to take a long- term
perspective in managing the SLI program. Without good cost information,
decision makers at NASA and in the Congress will not know whether the $4.8
billion investment is sufficient for the early phases of the program or
how much more it will cost to actually develop and deploy the new vehicle.
Moreover, NASA managers will lack the information they need to monitor
costs, schedule, and performance.
Other management controls that are integral to successfully managing the
SLI program cannot be implemented until NASA has a cost estimate. First,
NASA has a system* known as Earned Value Management* intended to help
provide program managers and others with early warnings of cost and
schedule problems. But this system cannot be effectively implemented
without having baseline requirements defined or detailed cost estimates,
schedules, and timelines developed for the SLI program.
Earned Value Management goes beyond the two- dimensional approach of
comparing budgeted costs with actual costs. It also attempts to compare
the value of work accomplished during a given period with the value of
work scheduled for that period. By using the value of work done as a basis
for estimating the cost and time to complete it, the earned value concept
should alert program managers to potential problems sooner than
expenditures alone can. The communities that have a vested interest in
earned value are the (1) program managers, who are charged with overall
management responsibility for acquisition programs; (2) contractors, who
are responsible for the contract*s successful execution; and (3)
overseers, such as acquisition executives, financial managers, contract
surveillance Other Management
Controls Cannot Be Implemented Without a Cost Estimate
Page 13 GAO- 02- 1020 Space Transportation
officials, and cost estimators, who are tasked with tracking and
estimating program costs.
Second, individual SLI activities have prepared risk mitigation plans.
However, a program official told us that these plans cannot be quantified
at the overall program cost and schedule level without complete cost
estimates. Moreover, without a cost estimate, risks can be measured only
on a year- by- year basis and not on a multiyear basis* for example, to
2006. Risk mitigation plans identify, assess, and document the risks
associated with the cost, resource, schedule, and technical aspects of a
project and determine the procedures that will be used to manage those
risks. In doing so, they help ensure that a system will meet performance
requirements and be delivered on schedule and within budget. NASA*s
guidance requires that these plans be developed during the formulation
phase of a project.
Agencies are required to prepare annual performance plans that establish
performance goals with measurable target levels of performance for each
program activity in the agency*s budget and to provide a basis for
comparing actual performance with performance goals. 5 Doing so enables
agencies to gauge the progress of programs like SLI and, in turn, to take
quick action when performance goals are not being met.
Individual SLI activities are using computer simulations to help gauge
whether the technologies under development can meet specific performance
targets. However, NASA has not yet established broader measures that can
show NASA decision makers and the Congress whether the program as a whole
is meeting objectives such as reducing the payload cost to approximately
$1,000 per pound and reducing the risk of crew loss to approximately 1 in
10,000 missions. Like cost and risk controls, the development of such
metrics depends on NASA*s definition of requirements and cost estimation
for SLI.
It is important that NASA develop such measures as soon as possible. We
testified last year that one problem that hampered the previous X- 33 and
X- 34 efforts was the fact that NASA had not developed performance targets
that establish a clear path leading to a reusable launch vehicle.
5 31 U. S. C. S:1115.
Measures to Assess Performance Not Implemented
Page 14 GAO- 02- 1020 Space Transportation
In addition, while NASA plans to assess the readiness of technology to
gauge the maturity of individual technologies, it does not plan to verify
and validate the maturity of technology for the SLI program as a whole. To
ensure that individual technologies are sufficiently mature by NASA*s
planned full- scale development decision for 2006, NASA intends to use
technology readiness levels (TRLs). Our prior reports have shown that
TRLs, which were pioneered by NASA, are a good way to gauge the maturity
of technologies. Readiness levels are measured along a scale of 1 to 9,
starting with paper studies of the basic concept, proceeding with
laboratory demonstrations, and ending with a technology that has proven
itself on the intended product. NASA would like to achieve a TRL of 6 by
2006 for key technologies, such as the propulsion system. At this level, a
prototype is tested in a relevant environment, such as a highfidelity
laboratory environment or in a simulated operational environment.
Currently, most of the technology areas are at levels 3 or 4. At level 3,
analytical studies and laboratory studies are performed to physically
validate analytical predictions of separate elements of the technology. At
level 4, basic technological components are integrated to establish that
the pieces will work together.
While assessing technologies separately should help NASA decide when and
where to insert new technologies into the SLI program, it is still
important for NASA to look at the readiness of the product as a whole
because how well various components being developed will work together is
unknown.
Another measure that NASA officials told us they do not plan to implement
is one that would assess the extent or percentage of total risks that have
been reduced with NASA*s initial $4.8 billion investment as well as the
amount of risk remaining. Program officials told us that it would be too
resource- intensive to develop this measure. However, in the absence of
such information, decision makers at NASA and in the Congress have little
assurance that the $4.8 billion investment in SLI can fulfill the goals
expected of the program, and they have reduced confidence that what has
already been spent has placed NASA on track to meeting its primary goals.
NASA aims to be able to define system requirements for SLI by November
2002. But meeting this goal may not be realistic. NASA must first decide
whether developing a second- generation vehicle to be deployed in 2014 to
2015 is still a worthwhile endeavor, given plans to extend the life of the
space shuttle and cut back on the space station*s capabilities, and if so,
what specific direction the program should take and how it will fit in
with Conclusions
Page 15 GAO- 02- 1020 Space Transportation
DOD efforts. Making such decisions within the short time remaining before
the SLI systems requirements review will be difficult because it will
require NASA to (1) resolve differences with its space station partners,
who have concerns about planned cutbacks to the station; (2) reach
consensus with DOD on requirements and priorities, which are now
considerably different; and (3) make trade- offs as to what capabilities
and technologies it should pursue without really knowing when advancements
can be achieved. When NASA is able to resolve these challenges and
finalize requirements for SLI, it will be critical for NASA to swiftly
implement effective management to oversee the effort. Until it does so,
NASA will not be able to assure its managers and the Congress that the
initial investment is being spent wisely and that risks are being reduced,
and it will not be able to predict what the total costs of the program
will be.
We recommend that the NASA Administrator do the following: 1. Reassess the
schedule for defining the requirements for the Space
Launch Initiative in order to ensure that the agency takes the following
actions before making final decisions on basic requirements and selecting
three architectures to pursue: (1) complete the reassessment of NASA*s
integrated space transportation plan, (2) reach consensus with its
international partners on the future of the space station, and (3) reach
consensus with the Department of Defense on its role in the SLI effort.
2. If DOD is to jointly develop the second- generation vehicle, reach
consensus with DOD on priorities and objectives for SLI and factor DOD*s
requirements into NASA*s own.
3. After NASA completes its system requirements review, ensure that a cost
estimate is promptly developed for the SLI investment and that this
estimate is detailed and reliable enough to be used to complete risk
mitigation plans and carry out earned value management activities.
4. Ensure that NASA can demonstrate how the $4.8 billion initial
investment supports the requirements that NASA decides to pursue in
November 2002.
5. After system requirements are defined, ensure that performance measures
are developed to assess the progress of the program, as a Recommendations
Page 16 GAO- 02- 1020 Space Transportation
whole, toward (1) meeting key performance objectives, including lowering
the cost of delivering payloads to low- Earth orbit to less than $1,000
per pound and reducing the risk of crew loss to approximately 1 in 10,000
missions, and (2) achieving an overall technology readiness level of 6 by
2006.
6. Ensure that measures are developed to assess the amount of risk reduced
following each year of expenditure and the amount of risk remaining.
In written comments on a draft of this report, NASA*s Associate Deputy
Administrator said the agency concurs with the recommendations, adding
that the program completed its first year as planned and within budget.
According to the Associate Deputy Administrator, the agency is reassessing
the Integrated Space Transportation Plan and working with DOD to develop
their requirements. Furthermore, NASA concurs with the recommendation
concerning the development of a cost estimate. However, the agency notes
that the RLV industry suffers uncertainties in predicting costs, primarily
because of a limited set of models and data to validate those models. NASA
also concurred with the need to validate SLI requirements and the need for
performance measures to assess the progress of the program towards meeting
objectives and achieving technology readiness levels.
Our draft report contained a recommendation that NASA develop measures to
assess the percentage of risk reduced following each year of expenditures
and the percentage of risk remaining. NASA believes that its process of
narrowing potential SLI technology alternatives to the most promising
concepts meriting further funding enables the successful candidates to be
built and become operational. In that way, the SLI program is reducing the
risks inherent in an advanced research and technology program of this
magnitude. According to NASA, it will develop a reporting mechanism that
communicates the amount of risk reduction achieved and remaining on an
annual basis. Our intent is for NASA to provide a quantifiable means of
measuring progress. Thus, we modified that recommendation accordingly.
Agency Comments
Page 17 GAO- 02- 1020 Space Transportation
To assess the coordination of requirements with DOD and other NASA
programs, we interviewed officials within the Department of Defense and
NASA. We evaluated studies conducted by DOD and NASA on efforts to address
requirements and coordination.
To address NASA*s management controls for the SLI program, we interviewed
NASA officials regarding cost estimates and the process by which cost
information is studied and communicated throughout NASA. We reviewed
NASA*s and the Office of Management and Budget*s guidance regarding earned
value management and discussed the program*s methods for coordinating
activities. We also reviewed risk mitigation plans to determine how cost
affects these plans and discussed these plans with program officials.
To assess program performance measures, we reviewed NASA*s policies and
procedures governing program management and the program*s plans for
assessing performance measures. We also interviewed SLI program officials
to understand the program*s plans to meet goals and objectives.
To accomplish our work, we interviewed officials from NASA*s headquarters,
Washington, D. C.; Marshall Space Flight Center, Alabama; Johnson Space
Center, Texas; Kennedy Space Center, Florida; and Independent Program
Assessment Office, Virginia; and DOD*s Space Command, Colorado.
We performed our review from October 2001 through August 2002 in
accordance with generally accepted government auditing standards.
Unless you publicly announce its contents earlier, we plan no further
distribution of this report until 30 days from its issue date. At that
time, we will send copies to the NASA Administrator; Director, Office of
Management and Budget; and other interested parties. We will also make
copies available to others upon request. In addition, the report will be
available at no charge on the GAO Web site at http:// www. gao. gov. Scope
and
Methodology
Page 18 GAO- 02- 1020 Space Transportation
Please contact me at (202) 512- 4841 if you or your staffs have any
questions about this report. Major contributors to this report are listed
in appendix II.
Allen Li Director Acquisition and Sourcing Management
Appendix I: Comments from the National Aeronautics and Space
Administration
Page 19 GAO- 02- 1020 Space Transportation
Appendix I: Comments from the National Aeronautics and Space
Administration
Appendix I: Comments from the National Aeronautics and Space
Administration
Page 20 GAO- 02- 1020 Space Transportation
Appendix I: Comments from the National Aeronautics and Space
Administration
Page 21 GAO- 02- 1020 Space Transportation
Appendix II: GAO Contact and Staff Acknowledgments
Page 22 GAO- 02- 1020 Space Transportation
Jerry Herley (202) 512- 7609 In addition to the person named above,
Cristina Chaplain, Ivy Hubler, Danny Owens, and Dana Solomon made key
contributions to this report. Appendix II: GAO Contact and Staff
Acknowledgments GAO Contact Acknowledgments
(120101)
The General Accounting Office, the investigative arm of Congress, exists
to support Congress in meeting its constitutional responsibilities and to
help improve the performance and accountability of the federal government
for the American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations, and
other assistance to help Congress make informed oversight, policy, and
funding decisions. GAO*s commitment to good government is reflected in its
core values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this list to you every afternoon, go to www. gao. gov
and select *Subscribe to daily E- mail alert for newly released products*
under the GAO Reports heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548
To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470
Jeff Nelligan, managing director, NelliganJ@ gao. gov (202) 512- 4800 U.
S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D. C.
20548 GAO*s Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs
Public Affairs
*** End of document. ***