NASA: Long-Term Commitment to and Investment in Space Exploration
Program Requires More Knowledge (17-JUL-06, GAO-06-817R).
The National Aeronautics and Space Administration (NASA) plans to
spend nearly $230 billion over the next two decades implementing
the Vision for Space Exploration. In January 2006, NASA publicly
released its Exploration Systems Architecture Study (ESAS), which
is an effort to identify the best architecture and strategy to
implement the President's 2004 Vision for Space Exploration
(Vision). The cost estimate for implementing the ESAS through
fiscal year 2011 exceeds $31 billion. The estimate through fiscal
year 2018 is over $122 billion, and the estimate through fiscal
year 2025 is nearly $230 billion. These estimates include the
architecture, robotic precursor missions, supporting
technologies, and funding needed to service the International
Space Station (ISS). NASA plans to implement this architecture
through a "go as you can afford to pay" approach, wherein
lower-priority efforts would be deferred, descoped, or
discontinued to allow NASA to stay within its available budget
profile. This approach assumes NASA's budget will increase
moderately to keep pace with inflation. Given the long-term
fiscal imbalances that will challenge the entire federal
government now and in the future, it would be prudent for NASA to
establish a program that reduces the risk that significant
additional funding, beyond moderate increases for inflation, will
be required to execute the program. Government leaders will have
to make difficult decisions to resolve such challenges, and the
debate over the potential cost and the federal government's role
in implementing the Vision are emblematic of the challenges the
nation will need to resolve in the years ahead. Because of the
significance of this investment, competing demands on the federal
discretionary budget, and the importance of the success of NASA's
exploration program to the future of U.S. human spaceflight,
Congress requested that GAO assess (1) the extent to which NASA
has identified the architecture and costs necessary to implement
the Vision, (2) whether NASA's exploration architecture cost
estimates fit within the agency's projected available budgets,
and (3) the risks associated with NASA's acquisition strategy for
the Crew Exploration Vehicle (CEV) project.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-817R
ACCNO: A57081
TITLE: NASA: Long-Term Commitment to and Investment in Space
Exploration Program Requires More Knowledge
DATE: 07/17/2006
SUBJECT: Aerospace contracts
Cost analysis
Cost overruns
Future budget projections
Research and development contracts
Risk assessment
Risk management
Schedule slippages
Space exploration
Target architecture
Cost estimates
Vision for Space Exploration
Crew Exploration Vehicle
Crew Launch Vehicle
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-817R
July 17, 2006
The Honorable Sherwood Boehlert
Chairman
The Honorable Bart Gordon
Ranking Minority Member
Committee on Science
House of Representatives
Subject: NASA: Long-Term Commitment to and Investment in Space Exploration
Program Requires More Knowledge
The National Aeronautics and Space Administration (NASA) plans to spend
nearly $230 billion over the next two decades implementing the Vision for
Space Exploration. In January 2006, NASA publicly released its Exploration
Systems Architecture Study (ESAS), which is an effort to identify the best
architecture and strategy to implement the President's 2004 Vision for
Space Exploration (Vision).1 The cost estimate for implementing the ESAS
through fiscal year 2011 exceeds $31 billion. The estimate through fiscal
year 2018 is over $122 billion, and the estimate through fiscal year 2025
is nearly $230 billion.2 These estimates include the architecture, robotic
precursor missions, supporting technologies, and funding needed to service
the International Space Station (ISS).3 NASA plans to implement this
architecture through a "go as you can afford to pay" approach, wherein
lower-priority efforts would be deferred, descoped, or discontinued to
allow NASA to stay within its available budget profile. This approach
assumes NASA's budget will increase moderately to keep pace with
inflation. Given the long-term fiscal imbalances that will challenge the
entire federal government now and in the future, it would be prudent for
NASA to establish a program that reduces the risk that significant
additional funding, beyond moderate increases for inflation, will be
required to execute the program.4 Government leaders will have to make
difficult decisions to resolve such challenges, and the debate over the
potential cost and the federal government's role in implementing the
Vision are emblematic of the challenges the nation will need to resolve in
the years ahead.
1 The ESAS architecture supports the development of a new Crew Exploration
Vehicle (CEV), Crew Launch Vehicle (CLV), a Cargo Launch Vehicle (CaLV),
and other supporting systems. The architecture also calls for various
Research and Technology (R&T) and Robotic Lunar Exploration Program (RLEP)
projects.
2 All cost estimates related to the Vision are reported as inflated ("real
year") dollars.
3 NASA's cost estimate through 2011-$31.2 billion-included the costs of
the R&T and RLEP projects needed to support the architecture. Its estimate
for the first lunar landing-$104 billion-did not include $18 billion in
funding for R&T and RLEP projects. To ensure consistency, the estimates
for 2018 and 2025 are presented with R&T and RLEP funding included.
4 GAO, 21st Century Challenges: Reexamining the Base of the Federal
Government, GAO-05-325SP (Washington, D.C.: Feb. 2005); 21st Century:
Addressing Long-Term Fiscal Challenges Must Include a Reexamination of
Mandatory Spending, GAO-06-456T (Washington, D.C.: Feb. 15, 2006); and
Highlights of a GAO Forum: The Long-Term Fiscal Challenge, GAO-05-282SP
(Washington, D.C.: Feb. 1, 2005).
Because of the significance of this investment, competing demands on the
federal discretionary budget, and the importance of the success of NASA's
exploration program to the future of U.S. human spaceflight, you requested
that we assess (1) the extent to which NASA has identified the
architecture and costs necessary to implement the Vision, (2) whether
NASA's exploration architecture cost estimates fit within the agency's
projected available budgets, and (3) the risks associated with NASA's
acquisition strategy for the CEV project.
We presented our preliminary findings to your staff in May 2006. Because
of your committee's interest in how NASA is implementing the Vision for
Space Exploration, we are enclosing the full briefing that supported that
May presentation with this report (see encl. II), along with a summary of
our findings and conclusions. We are recommending that the NASA
Administrator modify the current CEV acquisition strategy to ensure that
the agency does not commit itself, and in turn the federal government, to
a long-term contractual obligation prior to establishing a sound business
case at the project's preliminary design review. In written comments, NASA
nonconcurred with our recommendation and stated that it has the
appropriate level of knowledge to proceed with its current acquisition
strategy. As a result of its nonconcurrence, we are including as a matter
for congressional consideration that the Congress should consider
restricting NASA's appropriations and obligations for the CEV project to
only the amount of funding necessary to successfully complete the
project's preliminary design review.
Background
The Vision includes plans to explore the moon, Mars, and beyond.5 The
first step in implementing the Vision is to retire the space shuttle after
completing assembly of the ISS by the end of the decade. NASA currently
plans to retire the space shuttle in 2010, creating a potential gap in
U.S. human spaceflight of up to 4 years before development of the CEV and
the CLV is complete. Congress has voiced concern over the United States
not having continuous access to space, and NASA has made it a priority to
minimize the gap by accelerating the CEV project to have it in service as
close to 2010 as possible. NASA's Exploration Systems Mission
Directorate's (ESMD) Constellation program is responsible for the
development of both the CEV and the CLV. NASA awarded concept development
contracts for the CEV project to both Lockheed Martin and Northrop Grumman
in July 2005 and plans to award a contract for design, development,
production and sustainment in September 2006. That contract could extend
through 2019. For the CLV, NASA plans to award a sole-source contract for
the first stage of the CLV to ATK-Thiokol, the manufacturer of the
Shuttle's Reusable Solid Rocket Motor, in October 2006. Also, the agency
plans to award Pratt & Whitney Rocketdyne, the developer of the Space
Shuttle Main Engine (SSME) and J-2 engines, a sole-source contract for
development of the J-2X engine in November 2006. These contractors are
currently planning their respective efforts under interim contract
arrangements. NASA has started in-house preliminary design work on the CLV
upper stage structures and avionics and plans to begin awarding
competitive contracts for production of these items in May 2007.
5 The Vision includes a return to the moon that is intended ultimately to
enable future exploration of Mars and other destinations. To accomplish
this, NASA initially plans to (1) complete its work on the International
Space Station by 2010, fulfilling its commitment to 15 international
partner countries; (2) begin developing a new manned exploration vehicle
to replace the space shuttle; and (3) return to the moon no later than
2020 in preparation for future, more ambitious missions.
Despite many successes in the exploration of space, such as landing the
Pathfinder and Exploration Rovers on Mars, the loss of life, unsuccessful
missions, and unforeseen cost overruns have recently increased the level
of concern over the benefits of such exploration, particularly with regard
to human spaceflight activities. NASA has had difficulty bringing a number
of projects to completion, including several efforts to build a second
generation of reusable human spaceflight vehicle to replace the space
shuttle. NASA has attempted several expensive endeavors such as the
National Aero-Space Plane, the X-33 and X-34, and the Space Launch
Initiative, among others. While these endeavors have helped to advance
scientific and technical knowledge, none have completed their objective of
fielding a new reusable space vehicle. We estimate that these unsuccessful
development efforts have cost approximately $4.8 billion since the 1980s.
The high cost of these unsuccessful efforts and the potential costs of
implementing the Vision make it important that NASA achieve success in its
new exploration program.
Our past work has shown that developing a sound business case, based on
matching requirements to available and reasonably expected resources
before committing to a new product development effort, reduces risk and
increases the likelihood of successful outcomes.6 At the heart of a
business case is a knowledge-based approach to product development that is
a best practice among leading commercial firms and successful government
system developers. For a program to increase its chances of delivering a
successful product, high levels of knowledge should be demonstrated before
managers make significant program commitments. In essence, knowledge
supplants risk over time. This building of knowledge can be described as
three levels that should be attained over the course of the program:
(1) At program start, the customer's needs should match the
developer's available resources in terms of availability of mature
technologies, time, human capital, and funding.
(2) Midway through development, the product's design should be
stable and demonstrate that it is capable of meeting performance
requirements.
(3) By the time of the production decision, the product must be
shown to be producible within cost, schedule, and quality targets,
and have demonstrated its reliability.
6 Examples of our best practices reports include GAO, Best Practices:
Using a Knowledge-Based Approach to Improve Weapon Acquisition,
GAO-04-386SP (Washington, DC.: Jan. 2004); Space Acquisitions: Committing
Prematurely to the Transformational Satellite Program Elevates Risks for
Poor Cost, Schedule, and Performance Outcomes, GAO-04-71R (Washington,
D.C.: Dec. 4, 2003); Best Practices: Capturing Design and Manufacturing
Knowledge Early Improves Acquisition Outcomes, GAO-02-701 (Washington,
D.C.: Jul. 15, 2002); and Best Practices: Better Matching of Needs and
Resources Will Lead to Better Weapon System Outcomes, GAO-01-288
(Washington, DC.: Mar. 8, 2001).
Our work has shown that programs that have not attained the level of
knowledge needed to support a sound business case have been plagued by
cost overruns, schedule delays, decreased capability, and overall poor
performance. With regard to NASA, we have reported that in some cases the
agency's failure to define requirements adequately and develop realistic
cost estimates-two key elements of a business case-resulted in projects
costing more, taking longer, and achieving less than originally planned.
Summary
Although NASA is continuing to refine its exploration architecture cost
estimates, the agency cannot at this time provide a firm estimate of what
it will take to implement the architecture. The absence of firm cost
estimates is mainly due to the fact that the program is in the early
stages of its life cycle. According to NASA cost-estimating guidance,
early life cycle phase estimates are generally based upon parametric
models, which use data from projects with similar attributes to predict
cost because there are usually many unknowns and actual cost or
performance data are not available. NASA preliminarily identified the
resources needed to implement the architecture as outlined in the
architecture study primarily through the use of such models. NASA
conducted a cost risk analysis of its preliminary estimates through fiscal
year 2011. On the basis of this analysis and through the addition of
programmatic reserves (20 percent on all development and 10 percent on all
production costs), NASA is 65 percent confident that the actual cost of
the program will either meet or be less than its estimate of $31.2 billion
through fiscal year 2011. For the cost estimates for beyond 2011, when
most of the cost risk for implementing the architecture will be realized,
NASA has not applied a confidence level distinction. Since NASA released
its preliminary estimates, the agency has continued to make architecture
changes. For example, following the issuance of the architecture study,
NASA conducted several analysis cycles during which various aspects of the
architecture have evolved, such as the diameter of the CEV, the engine
used to support the upper stage of the CLV, and the size of the Reusable
Solid Rocket Booster on the CLV. While these changes, and others, are
appropriate for this phase of the program, when concepts are still being
developed, they leave the agency in the position of being unable to firmly
identify program requirements and needed resources, which can also be
expected at this phase of the program. According to NASA officials, once
they receive more detailed contractor inputs, the agency will be able to
produce higher-fidelity estimates of program cost. NASA plans to commit to
a firm cost estimate at the preliminary design review (PDR) in 2008, when
the program's requirements, design, and schedule will all be baselined.
NASA will be challenged to implement the architecture recommended in the
study within its projected budget. Whether using the architecture study
estimates of funds available or NASA's Fiscal Year 2007 Budget Submission
for ESMD that was based on the architecture study cost estimates, there
are years when NASA does not have sufficient funding to implement the
architecture. Some yearly shortfalls exceed $1 billion, while in other
years the funding available exceeds needed resources. NASA maintains that
the architecture could be implemented within the projected available
budgets through fiscal year 2011 when funding is considered cumulatively.
In addition, NASA preliminarily projects multibillion-dollar shortfalls
for ESMD in all fiscal years from 2014 to 2020, with an overall deficit
through 2025 of over $18 billion. In the short term, NASA is attempting to
address this problem within the Constellation program by redirecting funds
to that program from other ESMD activities to provide a significant
surplus for fiscal years 2006 and 2007 to cover projected shortfalls
beginning in fiscal year 2009. In addition, the Constellation program has
requested more funds than required for its projects in several early years
to cover shortfalls in later years. For example, the Exploration
Communication and Navigation Systems project within the Constellation
program plans to roll over $56.2 million from the fiscal year 2007 budget
to make up for budget shortfalls in fiscal years 2008, 2009, and 2010.
NASA officials stated the identified budget phasing problem could worsen
given that changes made to the exploration architecture following issuance
of the study will likely add to the near-term development costs, where the
funding is already constrained. In addition, NASA's estimates beyond 2010
are based upon a surplus of well over $1 billion in fiscal year 2011 due
to the retirement of the space shuttle fleet in 2010. However, NASA
officials said the costs for retiring the space shuttle and transitioning
to the new program are not fully understood, and thus the expected surplus
could be less than anticipated.
NASA's current acquisition strategy for the CEV places the project at risk
of significant cost overruns, schedule delays, and performance shortfalls
because it commits the government to a long-term product development
effort before establishing a sound business case. NASA plans to award a
contract for the design, development, production, and sustainment of the
CEV in September 2006-before it has developed key elements of a sound
business case, including well-defined requirements, a preliminary design,
mature technology, and firm cost estimates. The period of performance for
the contract scheduled for award in September 2006 will extend through at
least 2014, with the possibility of extending through 2019. This contract
will comprise all design, development, and test and evaluation activities,
including production of ground and flight test articles and at least four
operational CEVs. Although NASA is committing to a long-term contract, it
will not have the elements of a sound business case in place until the
project level PDR in fiscal year 2008. Awarding a contract for design,
development, production, and sustainment of the project as NASA has
planned places the CEV project at increased risk of cost growth, schedule
delays, and performance shortfalls. At PDR, NASA will likely (a) have the
increased knowledge necessary to develop a sound business case that
includes high-fidelity, engineering-based estimates of life cycle cost for
the CEV project, (b) be in a better position to commit the government to a
long-term effort, and (c) have more certainty in advising Congress on
required resources.
Implementing the Vision over the coming decades will require hundreds of
billions of dollars and a sustained commitment from multiple
Administrations and Congresses over the length of the program. The
realistic identification of the resources needed to achieve the agency's
short-term goals would provide support for such a sustained commitment
over the long term. With a range of federal commitments binding the fiscal
future of the United States, competition for resources within the federal
government will only increase over the next several decades. Consequently,
it is incumbent upon NASA to ensure that it is wisely investing its
existing resources. As NASA begins to implement the Vision with several
key acquisition decisions planned to occur this fall, it will be essential
that the agency ensure that the investment decisions it is making are
sound and are based upon high levels of knowledge. NASA should make the
prudent decision now to ensure that it has attained the appropriate level
of knowledge to support a sound business case before it commits to the
project. However, under the current acquisition strategy for CEV, key
knowledge-including well-defined requirements, a preliminary design,
mature technology, and firm cost estimates-will not be known until over a
year after the expected contract award date. Nevertheless, NASA plans to
commit the government to a long-term contract. This approach increases the
risk that the project will encounter significant cost overruns, schedule
delays, and decreased capability. Given the nation's fiscal challenges and
those that exist within NASA, the availability of significant additional
resources to address such issues, should they occur, is unlikely. With the
impending decisions pertaining to the CEV, NASA has the opportunity to
establish a firm foundation for the entire Constellation program by
ensuring that the appropriate level of knowledge is available before
proceeding with its acquisition strategy and committing the government to
a long-term design, development, and production effort.
Recommendation for Executive Action
Because of the importance of the CEV project to NASA's overall
implementation of the Vision, NASA should focus on ensuring that its
acquisition approach for the CEV project does not place the government at
risk by committing to a long-term design and development effort without
the knowledge needed to make wise investment decisions. We therefore
recommend that the NASA Administrator modify the current CEV acquisition
strategy to ensure that the agency does not commit itself, and in turn the
federal government, to a long-term contractual obligation prior to
demonstrating, through the establishment of a sound business case at the
project's preliminary design review, that the project is affordable and
executable.
Matter for Congressional Consideration
Based on its response to our report, it appears that NASA plans to proceed
with its acquisition strategy for the CEV and award a long-term contract
for the project, although it continues to lack sufficient knowledge and a
sound business case for doing so. Congress is currently being asked to
approve NASA's fiscal year 2007 funding request and will be asked to
approve fiscal year 2008 and perhaps the fiscal year 2009 funding requests
for the CEV project before NASA has demonstrated such knowledge and has
provided evidence, based on that knowledge, that the project will be
executable within existing and expected resources. In light of the fact
that NASA plans to award the contract for the CEV in September 2006,
Congress should consider restricting annual appropriations and limiting
NASA's obligations for the CEV project to only the amount of funding
necessary to support activities needed to successfully complete the
project's preliminary design review.
Agency Comments and Our Evaluation
In written comments on a draft of this report (see encl. I), NASA
nonconcurred with our recommendation that it modify the current CEV
acquisition strategy to ensure that the agency does not commit itself, and
in turn the federal government, to a long-term contractual commitment
prior to establishing a sound business case at the project's preliminary
design review. NASA stated that it has the appropriate level of knowledge
to proceed with its acquisition plan to "down select" to a single Crew
Exploration Vehicle prime contractor in September 2006. NASA added that it
is maximizing competition by soliciting from industry a development,
production, and management approach with an emphasis on life cycle cost.
In the area of technology maturity, NASA stated that it has a plan and
process in place to address the Thermal Protection and Landing subsystems
technology risks through in-house development work and collaboration with
the prime contractor. NASA also noted that during its design, development,
and test and evaluation effort, the agency will be using an end-item award
fee, which would make all award fees subject to a final evaluation to
determine how well the product met requirements, including cost and
schedule.
The CEV acquisition strategy is not knowledge-based in that it calls for
maturing technologies, designing systems, and preparing for initial
production concurrently-an approach that our work has shown carries the
increased risk of cost and schedule overruns and decreased technical
capability. Therefore, we disagree with NASA's statement that it has the
appropriate level of knowledge to proceed with its current acquisition
strategy and award a long-term contract for the project prior to obtaining
sufficient knowledge. Specifically:
o In its response, NASA suggests that there would be no benefit
in retaining two prime contractors for the CEV project through the
preliminary design review and that the best return on its
investment would be gained by down-selecting to one contractor and
awarding the contract in September 2006. Contrary to NASA's
response, addressing our recommendation would not preclude the
agency from down-selecting to one contractor. The thrust of our
recommendation is that NASA should lessen the government's
obligation to the project at such an early stage when realistic
cost estimates have yet to be established and requirements are not
fully defined, and therefore limit the scope of the contract to
activities needed to successfully complete the preliminary design
review. At that point the project should have in place a sound
business case for proceeding and hence be in a better position to
justify continued investment. Implementation of the recommendation
could be accomplished through various means, including by
retaining two contractors through the preliminary design review
and awarding a contract at that time or by down-selecting as
planned in September 2006 and limiting the scope of the contract
as described above.
o NASA's suggestion that it is maximizing competition by
soliciting from industry its development, production, and
management approach and that it will receive firm competitive
prices from industry for completion of development and
demonstration of two vehicles has little basis. First, while the
current structure will allow for competition in the short term,
the benefits of such competition will be short-lived. Without
well-defined requirements, mature technologies, an approved
preliminary design, and realistic cost estimates, NASA has
insufficient information to ensure that it is obtaining firm
competitive prices for the work conducted for the entirety of
Schedule A-especially for activities beyond the project's
preliminary design review.
Because NASA continues to refine the project's requirements, as
demonstrated by the numerous changes to the exploration
architecture as discussed in our report, it cannot provide a firm
estimate of project cost. Without such information, it will likely
be difficult for NASA to establish realistic "not-to-exceed"
prices for Schedule B activities. Under the current strategy, NASA
will not have high-fidelity, engineering-based estimates of life
cycle costs for the CEV until the preliminary design review. As
outlined in this report, projects with cost estimates based on
early, evolving designs and top-level requirements are at
increased risk of cost growth relative to estimates based on
mature designs and detailed requirements-which could be achieved
at the preliminary design review. According to NASA, it plans to
obtain this and further knowledge about program cost, schedule,
and risk elements following the contract award and in conjunction
with the contractor. In the absence of such information, it is not
clear how NASA can substantiate its statement that it has the
knowledge necessary to commit to activities beyond the project's
preliminary design review. Further, it cannot provide Congress
with assurance of the appropriateness of requested funding for the
project.
o NASA stated that its current acquisition strategy for the CEV
minimizes the government's obligation during development by
dividing the CEV contract into three separate schedules. All three
schedules, however, will be awarded in September 2006 as part of
one contract. Although NASA plans to include language in the
negotiated CEV contract to state that the minimum quantity under
Schedule B will not be applicable until that schedule's period of
performance begins in 2009-a step that would lessen the
government's obligation during production-it will continue to be
responsible for all Schedule A activities at the time of contract
award. These activities include all design, development, and test
and evaluation activities, as well as the production of two
operational vehicles. Contractually obligating the government to
even these Schedule A activities, before it has established a
sound business case to support such a commitment, is not in line
with our knowledge-based approach and is ultimately not in the
best interest of the government.
o NASA's investment in identifying and maturing the Thermal
Protection and Landing Subsystems is a step in the right direction
to ensure that these technologies are mature and available when
needed. NASA has no guarantee, however, that these critical
technologies will be mature by the time of the project's
preliminary design review-the point at which our work has shown
that technologies should be mature in order to decrease the risk
of cost and schedule growth. NASA's proposed commitment to the
project for activities beyond the preliminary design review before
retiring these technology risks increases the likelihood that the
project will experience schedule delays and cost overruns.
o NASA maintains that program risks have been marginalized and
that the agency will utilize incentives, including end-item award
fees, to ensure contractor performance. NASA suggests that the
incentives it plans to use in the form of end-item award fees will
be a powerful tool for meeting cost schedule, technical, and
quality goals. The use of these tools, however, does not
compensate for proceeding with a risky acquisition, nor do they
lessen NASA's responsibility to implement an executable program
from the start. For them to function as intended, NASA needs to
address the more fundamental issues related to its acquisition
strategy, including its lack of a sound business case for the CEV
project.
o Finally, the use of cost-reimbursable contracting, while
appropriate for early development and design efforts, places most
of the cost risk for the project on the government. Given the
nature of this effort, it is likely that the project will change
significantly as it moves forward. Therefore, any scope changes or
schedule slips could translate into additional contract cost for
NASA. Such cost impacts could be minimized if NASA limited its
contractual obligation to those activities needed to achieve a
successful preliminary design review, as we recommended. In
addition, limiting the scope of the CEV contract would allow both
NASA and Congress to assess the project's progress at the
preliminary design review and to decide if continued investment in
the project is prudent and in the best interest of the government.
It is important to note that Congress will continue to be asked to make
funding commitments in advance of CEV project events that would
demonstrate that the project has the knowledge necessary to support a
sound business case. Specifically, NASA's funding request for fiscal years
2007 and 2008 are scheduled to be approved before the CEV holds its
preliminary design review. Since the preliminary design review is
currently scheduled for March 2008, this may also be the case for fiscal
year 2009. Congress should safeguard against a situation in which
contractual and budget decisions could hinder its ability to tie further
investments in the CEV project to demonstrated progress at the preliminary
design review. As such, we have included a matter for congressional
consideration.
We also received technical comments from NASA, which have been addressed
in the report, as appropriate.
Scope and Methodology
To assess the extent to which NASA has identified the architecture and
costs necessary to implement the Vision and whether NASA's exploration
architecture cost estimates fit within the agency's projected available
budgets, we reviewed and analyzed NASA's Exploration Systems Architecture
Study, fiscal year 2007 budget request, ground rules and assumptions
provided from the Constellation program to project level management
estimators to perform the bottom up review, guidance for use in preparing
the fiscal year 2008 budget request, NASA cost-estimating guidance in the
NASA Cost Estimating Handbook, and congressional hearings and testimonies
pertaining to NASA and the Vision. We also conducted interviews with NASA
headquarters officials from the Cost Analysis Division, the Exploration
Systems Mission Directorate, and Constellation program officials,
Constellation program and CEV project officials at Johnson Space Center;
CLV project officials at Marshall Space Flight Center; and cost analysts
from the Kennedy Space Center. During these interviews, we discussed the
methodologies used in preparing the ESAS and subsequent cost estimates,
architecture changes after the ESAS and the trades being considered,
budgeting issues, and procurement strategies and activities.
To assess the risks associated with NASA's acquisition strategy for the
CEV project, we reviewed and analyzed CEV project documentation, including
draft project plans, draft requirements documents, technology development
plans, documentation included in the contract request for proposals, and
past NASA human spaceflight acquisition programs. We compared NASA's plans
for the CEV with criteria contained in GAO best practices work on systems
acquisition. We also conducted interviews with NASA headquarters officials
from the Exploration Systems Mission Directorate and Constellation Systems
officials, Constellation program and CEV project officials at Johnson
Space Center, and CLV project officials at Marshall Space Flight Center.
We conducted our work from January 2006 to May 2006 in accordance with
generally accepted government auditing standards.
As agreed with your offices, unless you announce its contents earlier, we
will not distribute this report further until 10 days from its date. At
that time, we will send copies of the report to NASA's Administrator and
interested congressional committees. We will also make copies available to
others upon request. In addition, the report will be available at no
charge on GAO's Web site at http://www.gao.gov .
Should you or your staff have any questions on matters discussed in this
report, please contact me at (202) 512-4841 or lia@gao.gov . Contact
points for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. Principal contributors to this
report were James L. Morrison, Assistant Director; Rick Cederholm; Shelby
S. Oakley; Guisseli Reyes; Sylvia Schatz; and John S. Warren, Jr.
Allen Li
Director
Acquisition and Sourcing Management
Enclosures
Enclosure I
Comments from the National Aeronautics and Space Administration
Summary
NASA: Long-Term Commitment to and Investment in Space Exploration Program
Requires More Knowledge
May 2006
Briefing for Congressional Staff
Although NASA is continuing to refine its exploration architecture cost
estimates, the agency cannot at this time provide a firm estimate of what
it will take to implement the architecture. The absence of firm cost
estimates is mainly due to the fact that the program is in its early
stages. NASA preliminarily identified the resources needed to implement
the architecture as outlined in the ESAS. However, since that time, NASA
has continued to make architecture changes. For example, following the
issuance of the ESAS, NASA undertook several analysis cycles in which
various aspects of the architecture have evolved, such as the diameter of
the CEV, the engine used to support the upper stage of the CLV, and the
size of the Reusable Solid Rocket Booster on the CLV. These changes, and
others, are appropriate for this phase of the program, when concepts are
being developed, but leave NASA in the position of being unable to firmly
identify program requirements and needed resources. NASA plans to commit
to a firm cost estimate at the preliminary design review (PDR) in 2008,
when the programs' requirements, design, and schedule will all be
baselined.
NASA will be challenged to implement the ESAS architecture with its
projected budget. Whether using the ESAS estimates of funds available or
NASA's fiscal year 2007 budget submission that was based upon the ESAS
estimates, there are years when NASA does not have sufficient funding to
implement the architecture. Some yearly shortfalls exceed $1 billion,
while in other years the funding available exceeds needed resources. NASA
maintains that the architecture could be implemented within its projected
available budgets through fiscal year 2011 when funding is considered
cumulatively. In the short term, NASA has redirected funds to the
Constellation program from other ESMD activities to provide a significant
surplus for fiscal years 2006 and 2007 to cover projected shortfalls for
the program beginning in fiscal year 2009. The identified budget phasing
problem in ESAS could worsen, given that changes to the architecture
following the ESAS will likely add to the near term development costs,
where funding is already constrained. In addition, NASA anticipates a
significant surplus in fiscal year 2011 because of the retirement of the
space shuttle fleet in 2010. However, the transition costs are not fully
understood.
NASA's acquisition strategy for the CEV places the project at risk of cost
overruns, schedule delays, and performance shortfalls because it commits
the government to a long-term product development effort before
establishing a sound business case. NASA plans to award a contract for
design, development, production, and sustainment of the CEV in September
2006-before it has developed well-defined requirements, a preliminary
design, mature technology, and firm cost estimates. This information is
not expected until the project-level PDR in fiscal year 2008. At that
point, NASA will likely (a) have the increased knowledge necessary to
develop a sound business case that includes high-fidelity,
engineering-based estimates of life cycle cost for the CEV project, (b) be
in a better position to commit the government to a long-term effort, and
(c) have more certainty in advising Congress on required resources.
In January 2006, the National Aeronautics and Space Administration (NASA)
publicly released its Exploration Systems Architecture Study (ESAS), which
aimed to identify the best architecture and strategy to implement the
President's 2004 Vision for Space Exploration (Vision). The ESAS
architecture supports development of a new Crew Exploration Vehicle (CEV),
Crew Launch Vehicle (CLV), Cargo Launch Vehicle (CaLV), and other
supporting systems, which are part of NASA's Exploration Systems Mission
Directorate's (ESMD) Constellation program. The architecture also calls
for various Research and Technology (R&T) and Robotic Lunar Exploration
Program (RLEP) projects.
The cost estimate for implementing the ESAS through fiscal year 2011
exceeds $31 billion. The estimate through fiscal year 2018 is $122 billion
and the estimate through fiscal year 2025 is nearly $230 billion. These
estimates include the architecture, robotic precursor missions, supporting
technologies, and funding needed to service the International Space
Station (ISS).
Because of the significance of this investment, competing demands on the
federal discretionary budget, and the importance of the success of NASA's
exploration program to the future of U.S. human spaceflight, we assessed
(1) the extent to which NASA has identified the architecture and costs
necessary to implement the Vision, (2) whether NASA's exploration
architecture cost estimates fit within the agency's projected budgets, and
(3) the risks associated with NASA's acquisition strategy for the CEV.
Why GAO Did This Study
Briefing
Briefing Structure
Background page 2
Findings
Firm Cost Estimates Cannot Be Developed at This Time page 3
Expected Budget Challenges Architecture Implementation page 5
Lack of Sound Business Case Puts CEV Acquisition at Risk page 7
Appendix
Scope and Methodology and Contributors page 9
Enclosure II
Page 2
Briefing for Congressional Staff
Original Exploration Systems Architecture Study Overview
The ESAS outlined the recommended architecture and strategy for
implementation of the Vision. The primary vehicles and elements of the
architecture include the CEV, the CLV, the CaLV that includes the Earth
Departure Stage (EDS), and the Lunar Surface Access Module (LSAM). The
diagram below outlines a launch mission for crew and cargo, utilizing
rendezvous locations in low-Earth and low-lunar orbits.
NASA plans to bring the President's Vision to reality over the next
several decades by
o conducting exploration activities in low-Earth orbit; for example,
flying the space shuttle to complete assembly of the ISS;
o exploring beyond low-Earth orbit; for example, establishing sustained
exploration of the moon and Mars;
o developing transportation that
supports exploration; for example, building crew exploration vehicles; and
o pursuing opportunities for international and commercial
participation.
Implementing the Vision
Background
Enclosure II
The original ESAS architecture isdescribed below. Changesmade to the
architecture since the release of ESAS are described in later sections.
CEV: The CEV is a reusable, Apollo-derived cone-shaped capsule launched
atop the CLV. The CEV consists of a Command Module (CM), a Service Module
(SM), and a Launch Abort System (LAS). The CEV is sized at 5.5 meter
diameters for lunar polar missions carrying a crew of four, and is also
reconfigurable to accommodate up to six crew members for missions to ISS.
The vehicle uses a Low Impact Docking System (LIDS) for ISS and lunar
missions. The vehicle is reusable for up to 10 missions and will land on
land with a water landing as a backup. The SM utilizes a pressure-fed
liquid oxygen (LOX)/methane propulsion system.
CLV: The CLV consists of a shuttle-derived four-segment Reusable Solid
Rocket Booster (RSRB) first stage and a newly designed upper stage with
one modified, and now expendable, SSME. It will launch 25 metric tons to
low-Earth orbit and serve as the long-term crew launch capability for the
United States.
CaLV: The CaLV will use a heritage shuttle external tank-derived
LOX/liquid hydrogen core stage propelled by five redesigned SSMEs.
Attached to this core stage are two newly developed five-segment RSRBs,
allowing over 100 metric tons to be launched to low-Earth orbit. The upper
stage, which also serves as the EDS, uses an external tank-derived
LOX/liquid hydrogen system and will employ two Saturn-derived J-2 engines.
LSAM: The LSAM is an expendable two-stage module launched atop the CaLV.
The descent stage will utilize a LOX/liquid hydrogen propulsion system
while the ascent stage will use a pressure-fed LOX-methane propulsion
system. A crew cabin will be located on the ascent stage and will have an
airlock to allow docking with the CEV. The LSAM will be able to land at
any location on the lunar surface and will house a four-member crew for up
to 7 days.
Exploration Contracts
NASA awarded concept development contracts to both Lockheed Martin and
Northrop Grumman for the CEV project in July 2005. NASA plans to
down-select to one contractor and award a contract for development,
production, and sustainment of the CEV in September 2006. That contract
could extend through 2019.
NASA plans to award a sole-source contract for the first stage of the CLV
to ATK-Thiokol, the manufacturer of the Shuttle's Reusable Solid Rocket
Motor, in October 2006. Also, the agency plans to award Pratt & Whitney
Rocketdyne, the developer of the Space Shuttle Main Engine (SSME) and J-2
engines, a sole-source contract for development of the J-2X engine in
November 2006. These contractors are currently planning their respective
efforts under interim contract arrangements. NASA has started in-house
preliminary design work on the CLV upper stage structures and avionics and
plans to begin awarding competitive contracts for production of these
items in May 2007.
Page 3
Briefing for Congressional Staff
Firm Cost Estimates Cannot Be Developed at This Time
Cost-Estimating Process
NASA's cost estimates for implementing its exploration architecture are
preliminary-a fact that NASA has acknowledged since the ESAS was publicly
released. As part of the ESAS effort, NASA laid out the cost estimates for
implementing the recommended architecture. Because the ESAS effort was an
early life cycle activity, Pre-Phase A, the majority of the individual
estimates were based upon parametric models, with little actual data.
The ESAS process evaluated the cost of various alternative exploration
architectures based upon high-level program requirements. The recommended
architecture costs totaled
o over $31 billion dollars through fiscal year 2011,
o over $122 billion through fiscal year 2018, and
o close to $230 billion through fiscal year 2025.1
NASA conducted a cost risk analysis of the estimates through fiscal year
2011. This analysis provided a 65 percent confidence level for the
estimate (i.e., NASA is 65 percent certain that the actual cost of the
program will either meet or be less than the estimate). To obtain this
level of confidence in the estimates, NASA included programmatic
reserves-20 percent on all development and 10 percent on all production
costs. NASA only conducted the risk analysis through the first flight date
of the CEV at the time of ESAS-2011-leaving the estimates through 2018 and
2025, when most of the cost risk for implementing the architecture will be
realized, with no confidence level distinction. According to NASA
officials, the cost risk analysis lacked quality because of the evolving
nature of the requirements for the architecture and the compressed time
frames with which they had to conduct the analysis. According to NASA
officials, once they receive more detailed contractor inputs, the agency
will be able to produce higher-fidelity estimates of program cost. NASA
has stated that it would not commit to a cost estimate for implementing
the exploration architecture until the Constellation program's PDR, which
will occur in late fiscal year 2008. At that time, the requirements,
design, schedule, and cost will all be baselined.
NASA refined the architecture several times since ESAS. As a result of
these changes, the costs associated with the architecture have also
changed. As part of the fiscal year 2007 budget formulation process, NASA
made two major changes to plans laid out in the ESAS. First, the
requirement for use of a LOX/methane engine on the CEV service module-a
high-risk development-was removed, and the approach for meeting the
propulsion requirement was left to the discretion of the contractor.
Second, the first flight of the CEV was delayed until no later than 2014.
1 NASA's cost estimate through 2011-$31 billion- included the costs of the
R&T and RLEP projects needed to support the architecture. Its estimate for
the first lunar landing-$104 billion-did not include $18 billion in
funding for R&T and RLEP projects. To ensure consistency, the estimates
for 2018 and 2025 are presented with R&T and RLEP funding included. The
estimates include $20 billion to service the ISS.
NASA's Cost Estimating Handbook outlines cost-estimating processes in
relation to acquisition life cycle phases.
o In Pre-Phase A, there are many unknowns. At this point, the
most effective cost-estimating approach is a parametric or
analogous methodology, i.e., data from projects with similar
attributes is used to predict the cost.
o In Phase A, conceptual designs are better defined and a better
understanding of the system requirements and technical risks
exists. But, parametric or analogous cost-estimating techniques
are still used, because detailed data may still be unavailable.
o In Phase B, system designs are defined below the subsystem
level. At this point, estimating methodologies evolve to more
detailed parametric or engineering buildup estimates supported by
technical experts. By the end of Phase B, specific data are
available to prepare a full life cycle cost estimate.
o In Phases C and D, cost estimates are refined to include actual
data. At this point, the preferred cost methodology is an
engineering buildup based on the lowest level of detail available,
including overhead, labor, and material costs.
Cost Estimating
NASA's Life Cycle for Flight Systems and Ground Support Projects through
Phase D
Enclosure II
Enclosure II
Subsequenttothe submission of NASA's fiscal year 2007 budget, the
Constellation program conducted an internal bottom-up review (BUR) of
program costs. The goal of the BUR was to identify the funding it would
take to "get the job done," which, according to the BUR guidance, means
conducting the first flight of the CEV to the ISS by 2012 and first lunar
mission by 2017. This review attempted to determine the cost impact of
several major changes that were made to the architecture. These changes
included a reduction in CEV diameter from 5.5 to 5 meters, use of a
five-segment RSRB and a Saturn-derived J-2x engine on the upper stage of
the CLV, deletion of the unpressurized cargo CEV, the addition of an ISS
docking system (Androgynous Peripheral Attachment System), and the
inclusion of a Ka Band for High Definition Television on the CEV. Some of
these architecture changes may help lessen technology development risks in
the future program due to the planned commonality between the CLV and CaLV
launch systems. While the results of this review were an attempt to
provide more fidelity to the Constellation program's cost estimates, given
the continued lack of a firm program baseline for requirements, design,
and schedule, along with a continued lack of input from contractors, it is
unlikely that the program had the level of detail available to support a
true estimate of total costs this early in the program life cycle.
ESMD is conducting a follow-on review to the Constellation program's BUR
as NASA enters its fiscal year 2008 budget formulation cycle. As part of
this latest review, NASA has continued to evaluate changes to the program
architecture and schedule, such as the use of the RS-68 engine on the CaLV
and the delay of the first lunar mission to either fiscal year 2019 or
fiscal year 2020.
The continued evolution of the exploration architecture serves to
highlight the preliminary nature of architecture itself and its associated
cost estimates. Although NASA is continuing to refine its cost estimates
for implementing the architecture to provide a more reliable estimate of
cost, history suggests that program costs could increase significantly
over estimates. In 2004, CBO reported that fulfilling the Vision could
require the addition of billions of dollars to NASA's estimates of cost or
extending the schedule for the first lunar landing by several years.
Applying NASA's average cost growth figure of 45 percent to the ESAS cost
estimates, assuming NASA business as usual, would result in an increase of
almost $14 billion over the $31 billion it estimates it will need through
2011. With a significant increase in NASA budgets unlikely, given the
current national fiscal imbalance, this level of cost growth could result
in an unsustainable long-term exploration program.
Page 4
Briefing for Congressional Staff
Firm Cost Estimates Cannot Be Developed at This Time, cont'd
Historically, NASA has shown that it lacks a clear understanding of how
much its programs will cost and how long they will take to achieve their
objectives. NASA's cost estimates have often been unreasonable when
committing to programs because of several factors, including inadequate
requirements definition; changes in program content; and inadequate
processes to establish priorities, quantify risks, and make informed
investment decisions. GAO has reported on these issues for several years
in both its high-risk series and in specific reviews of programs where
NASA failed to apply discipline to its cost estimates to ensure those
estimates were reasonable. For example, in 2002, GAO reported that since
1995, estimates for completion of the ISS had increased by $13 billion and
the scheduled completion date had slipped 4 years. Also, in 2004, GAO
conducted a review of 27 other NASA programs and reported that the initial
baseline estimates for over half of those programs were understated.
Costs for NASA programs have historically been greater, on average, than
initial estimates anticipated. A 2004 Congressional Budget Office (CBO)
examination of 72 NASA programs spanning the past 30 years found that
costs of NASA programs have increased, on average, 45 percent from initial
budget estimates.
Cost Estimate Issues
Cost Estimating
Page 5
Briefing for Congressional Staff
The NASA Administrator recently testified that the agency is facing
challenges to ensuring adequate funding for the priorities of the
President and Congress within available budgetary resources. He stated
that NASA has adopted a "go as you can afford to pay" approach to funding
its exploration missions. This approach assumes NASA's top line budget
will grow at the moderate rate identified in the President's fiscal year
2007 budget request.
Under this approach, NASA would implement its priority missions within
available resources and planned budgets through the redirection of funding
for longer-term and lower-priority R&T elements within ESMD. As a result,
several ESMD R&T programs and missions were discontinued, descoped, or
deferred. That funding, in turn, was shifted into the Constellation
Program to accelerate development of the CEV and the CLV.
NASA Funding Approach
Expected Budget Challenges Architecture Implementation
Funding Shortfalls
Enclosure II
ESAS Estimated Cost versus ESMD Projected Budget by Fiscal Year
NASAwill be challengedto implement the exploration architecture,given the
agency's expected budget profile. The ESAS effort defined the recommended
architecture and preliminary costs, which NASA contends would allow the
program to be accomplished within available budgets through fiscal year
2011. However, phasing issues still needed to be resolved. On an annual
basis, NASA cannot afford to implement the architecture, although,
cumulatively, for fiscal years 2007-2011, the agency says it has the money
available. Beginning with fiscal year 2014 and for the remainder of the
decade, where the anticipated available budgets were adjusted for
inflation, the ESAS cost projections show yearly multibillion-dollar
shortfalls with an overall deficit through 2025 of over $18 billion.
The projected ESMD available budget figures used in the ESAS were
developed well in advance of NASA's fiscal year 2007 President's budget
submission. However, using the updated budget estimates from the fiscal
year 2007 budget, the phasing issue becomes more pronounced when compared
to ESAS estimated costs. As shown in the chart below, ESAS estimates could
be accommodated within the ESMD available budget through fiscal year 2007.
From fiscal year 2008 through fiscal year 2010, however, NASA anticipates
annual budget shortfalls for implementing the architecture within ESMD to
exceed $1 billion per year. This shortfall could be partially offset, at
least within the Constellation program, by a carryover of approximately $1
billion in both fiscal years 2006 and 2007 as a result of funds redirected
from R&T activities within ESMD to that program. In addition, NASA
officials stated the Constellation program has requested more funding than
required for its projects in several years to cover shortfalls in later
years. For example, the Exploration Communication and Navigation Systems
project within the Constellation program plans to roll over $56.2 million
from the fiscal year 2007 budget to make up for budget shortfalls in
fiscal years 2008, 2009, and 2010.
Page 6
Briefing for Congressional Staff
NASA's approach, however, appears to be contrary the agency's stated "go
as you can afford to pay" approach to implement priority missions within
available resources. In addition, the surplus shown in fiscal year 2011 is
dependent upon dollars becoming available from the retirement of the space
shuttle fleet, even though NASA officials stated the costs associated with
retiring the space shuttle and transitioning to new architecture are not
fully understood and the expected surplus could be less than anticipated.
The shortfall presented by the fiscal year 2007 budget would not allow
NASA to accomplish the stated program objectives within available
resources over the next 5 years.
In addition, changes to the architecture implementation schedule have not
been consistent within the Constellation program. As previously stated,
NASA moved the scheduled initial operational capability (IOC) date of the
CEV to no later than 2014 during the fiscal year 2007 budget formulation
process. This change, along with modifications to the architecture,
allowed NASA's estimates to meet its overall budget profile, despite
continued year-to-year budget phasing issues. However, because of NASA's
focus on minimizing the gap between the retirement of the space shuttle
and the first flight of the CEV to the ISS, the program continued to
attempt to meet the earlier IOC date for the CEV through its various
analysis cycles. The earlier 2012 IOC date was retained as the planning
date during the bottom-up review process, the Phase II request for
proposal to the contractors involved CEV development, and the recent
announcement concerning its intention to purchase the J-2x engine for the
CLV from Pratt & Whitney Rocketdyne.
The 2012 date for CEV IOC, in addition to changes the Constellation
program made to the architecture during the BUR process, did not alleviate
issues with the short-term funding profile. According to Constellation
program officials, the net result of these changes will add more cost to
the early years of the program, when funding is already constrained and
phasing issues persist. Although the results of the BUR will not be
released, indications from Constellation program officials are that the
estimated costs of the program are higher than the ESAS estimated costs
and available funding per NASA's budget profile.
In the meantime, NASA continues to look for ways to resolve its budget
phasing issues, such as by making additional changes to the exploration
architecture. As the Constellation program executes its budget formulation
process for the fiscal year 2008 budget cycle, it is currently analyzing
options to the current architecture in an attempt to reduce development
and production costs. For example, NASA recently announced that it intends
to use five RS-68 engines instead of five SSMEs for the CaLV core stage,
which would also require the CaLV core stage diameter to be increased to
approximately 33 feet to accommodate the additional propellant needed by
the RS-68 engines.
ExpectedBudget Challenges Architecture Implementation, cont'd
The Vision called for retirement of the space shuttle fleet by the end of
this decade and that the CEV should be available no later than 2014,
creating a potential gap in human spaceflight of up to 4 years. The NASA
Administrator has stated that it is a priority of the agency to close this
gap and that the agency has taken steps to have the CEV in service as
close to 2010 as possible.
On the basis of lessons learned from the period between the end of the
Apollo Program and the first flight of the space shuttle, the
Administrator outlined several reasons why the CEV should not be delayed.
These reasons include the potential for
o stagnation in the aerospace industry,
o loss of critical expertise,
o withering of the industrial base,
o higher overall program costs,
o program schedule delays, and
o loss of leadership in space exploration.
Congress has also voiced its concern over the potential gap in human
spaceflight. In the National Aeronautics and Space Administration
Authorization Act of 2005, Congress stated it is the policy of the United
States to have the capability for human access to space on a continuous
basis.
Gap in Human Spaceflight
Funding Shortfall
Enclosure II
ers
NASA's acquisition strategyfor the CEV places the project at risk of cost
overruns, schedule delays, and performance shortfalls because it commits
the government to a long-term product development effort before
establishing a sound business case. In September 2006, NASA plans to award
a contract for design, development, production, and sustainment of the
CEV-before it has developed well-defined requirements, a preliminary
design, mature technology, and firm cost estimates for the project. The
CEV project might not have all the elements of a sound business case in
place until the project-level PDR in March 2008. At the completion of the
PDR, NASA will approve the selected prime contractor's preliminary design
based on detailed, validated requirements. Further, CEV project officials
indicated that the CEV project plans to retire all technology risks by the
PDR. At that point, NASA will likely have the increased knowledge
necessary to develop a sound business case that includes high-fidelity,
engineering-based estimates of life cycle cost for the CEV project. With
this business case in hand, NASA would be in a better position to commit
the government to a long-term design and development effort. NASA
officials disagree and have stated that it is appropriate for them to
proceed with the contract award because the agency is selecting "a
designer, not a design" for the CEV. In reality, by awarding a contract as
planned in September 2006, NASA is not only committing to an unknown
design but to production and long-term sustainment of the CEV as well.
The CEV contract scheduled for award in September 2006 will have three
schedules. At the time of contract award, NASA will be responsible for fee
earned and the reasonable, allowable, and allocable costs incurred by the
contractor in the performance of Schedule A and the minimum quantities
under Schedules B and C.
o Schedule A is for design, development, test and evaluation of
the CEV. Deliverables under Schedule A include all test articles
and two operational CEV vehicles-one human-rated variant and one
pressurized cargo variant.
o Schedule B is for production beyond the two operational
vehicles delivered under Schedule A. The CEV request for proposal
states that the "guaranteed minimum" quantity for Schedule B is
"two CEV," the type of which, according to NASA officials, is
undetermined.
o Schedule C is for sustainment in support of operations and in
support of Schedule B activities.
Lack of Sound Business Case Puts CEV Acquisition at Risk
GAO has frequently reported on the importance of developing a sound
business case before committing resources to a new product development
effort. The business case in its simplest form is demonstrated evidence
that (1) the need for the product is valid and that it can best be met
with the chosen concept, and (2) the chosen concept can be developed and
produced using existing and reasonably expected resources.
GAO has undertaken a best practices body of work on how leading developers
use a knowledge-based approach to develop products that reduces risks and
increases the likelihood of successful outcomes. This type of approach is
based on the premise of attaining knowledge about a program and the
resources available before making a contractual or financial commitment.
Knowledge that supports a sound business case includes well-defined
requirements, a preliminary design, mature technology, and realistic cost
estimates.
Use of this approach has enabled leading organizations to deliver
high-quality products on time and within budget. Conversely, GAO has also
reported that major systems that have not established a sound business
case have been plagued by cost overruns, schedule delays, decreased
capability, and overall poor performance. NASA's track record in
developing systems has not been good. GAO and others have reported that
NASA has had numerous problems with its programs and projects, including
underestimating program complexity and technology maturity, and inadequate
review and systems engineering processes.
Page 5Page 7
Note: Contract award for all schedules is planned for September 2006.
Schedule B and C performance periods are from 2009 to 2014 with an
additional 5-year performance option to end in 2019.
Best Practices
Briefing for Congressional StaffBriefing for Congressional Staff
CEV Project
Enclosure II
___AcroPDFMTS___AcroPDFMTS
Page 8
Briefing for Congressional Staff
Lack of Sound Business Case Puts CEV Acquisition at Risk, cont'd
NASA has tried unsuccessfully to develop a number of vehicles to replace
the shuttle over the past three decades. In the 1980s NASA initiated the
National Aero-Space Plane (NASP) to build and test a manned experimental
flight vehicle for demonstrating single-stage-to-orbit space launch and
sustained hypersonic cruise capability. NASA canceled the program as it
was experiencing cost overruns, schedule delays, and technology problems.
GAO reported that from 1986 to 1993 NASA spent $398 million for the NASP
program.
In the 1990s, NASA began the X-33 program to develop single-stage-to orbit
technology and the X-34 to demonstrate reusable two-stage-to -orbit
technologies. According to a 2006 Congressional Research Service report,
NASA terminated the X-33 and X-34 in March 2001-after spending over $1.4
billion-because the cost to complete them was too high relative to the
benefits. In 1999, GAO reported that technical problems and unrealistic
cost estimates on the X-33 project alone led to cost overruns of $75
million and over a year's delay.
In 2004, after the announcement of the Vision, NASA canceled the Space
Launch Initiative (SLI) program, which was to provide both launch
capabilities and an emergency crew return from the ISS. NASA's Inspector
General reported that NASA did not verify and validate basic requirements
for its second generation space transportation, while GAO reported that
key management controls could not be implemented until such requirements
were defined. GAO estimates that from 2001 to 2005 NASA provided the SLI
program with about $3 billion in funding.
Past Development Attempts
CEV Project
Enclosure II
An important step in developing a sound business case is defining
requirements. The acquisition strategy for the CEV lays out a series of
reviews to validate and approve CEV requirements. These reviews result in
approved system-level requirements at the October 2006 System Requirements
Review (SRR), and approved subsystem-level requirements at the April 2007
System Definition Review (SDR) and culminate with validated and approved
component-level requirements at the March 2008 PDR. Under the current CEV
strategy, NASA will select the winning contractor about 1month before the
system level requirements are approved at the SRR, over a year and a half
before detailed component-level requirements are approved at the PDR.
Another aspect of a sound business case is having mature technologies
before committing to product development. The CEV's acquisition strategy
is predicated upon using mature technologies as the basis for system
development. However, contractors will also be given discretion to include
immature technologies in areas where technology advancement is critical to
meeting requirements. NASA has independently identified technology risks
and implemented advanced technology development projects to address risks
in the areas of the thermal shielding needed for reentry and the landing
systems needed for ground landings. CEV project officials also expect that
each contractor's proposal will include additional technology development
risks. Under the current CEV strategy, NASA is awarding a contract for
product development and production of the first two variants of the CEV
before it has resolved these technology development risks.
AnswerAnswerScope and Methodology
P
age 10 Page 9
Briefing for Congressional Staff
To assess the extent to which NASA has identified the architecture and
costs necessary to implement the Vision and whether NASA's exploration
architecture fits within the agency's projected available budgets, we
reviewed and analyzed NASA's Exploration Systems Architecture Study,
fiscal year 2007 budget request, ground rules and assumptions provided
from the Constellation program to project-level management estimators to
perform the BUR, guidance for use in preparing the fiscal year 2008 budget
request, NASA cost-estimating guidance in the NASA Cost Estimating
Handbook, and congressional hearings and testimonies pertaining to NASA
and the Vision. We also conducted interviews with NASA headquarters
officials from the Cost Analysis Division, the Exploration Systems Mission
Directorate, and the Constellation Program; Constellation program and CEV
project officials at Johnson Space Center; CLV project officials at
Marshall Space Flight Center; and cost analysts from Kennedy Space Center.
During these interviews, we discussed the methodologies used in preparing
the ESAS and subsequent cost estimates, architecture changes after ESAS
and the trades being considered, budgeting issues, and procurement
strategies and activities.
To assess the risks associated with NASA's acquisition strategy for the
CEV project, we reviewed and analyzed CEV project documentation, including
draft project plans, draft requirements documents, technology development
plans, documentation included in the contract request for proposals, and
documentation for past NASA human space flight acquisition programs. We
compared NASA's plans for the CEV with criteria contained in GAO best
practices work on systems acquisition. We also conducted interviews with
NASA headquarters officials from the Exploration Systems Mission
Directorate, Constellation Program and CEV project officials at Johnson
Space Center, and CLV project officials at Marshall Space Flight Center.
If you have any questions concerning this briefing, please call Allen Li
at (202) 512-4841. Other key contributors to this briefing were James L.
Morrison, Assistant Director; Rick Cederholm; Shelby S. Oakley; Guisseli
Reyes; Sylvia Schatz; and John S. Warren, Jr.
Contributors
Appendix
Enclosure II
(120515)
*** End of document. ***