[House Hearing, 111 Congress]
[From the U.S. Government Publishing Office]


 
                    COST MANAGEMENT ISSUES IN NASA'S 
                       ACQUISITIONS AND PROGRAMS 

=======================================================================

                                HEARING

                               BEFORE THE

                 SUBCOMMITTEE ON SPACE AND AERONAUTICS

                  COMMITTEE ON SCIENCE AND TECHNOLOGY
                        HOUSE OF REPRESENTATIVES

                     ONE HUNDRED ELEVENTH CONGRESS

                             FIRST SESSION

                               __________

                             MARCH 5, 2009

                               __________

                            Serial No. 111-7

                               __________

     Printed for the use of the Committee on Science and Technology


     Available via the World Wide Web: http://www.science.house.gov

                                 ______

                        U.S. GOVERNMENT PRINTING OFFICE 

47-554 PDF                       WASHINGTON : 2009 

For sale by the Superintendent of Documents, U.S. Government Printing 
Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; 
DC area (202) 512-1800 Fax: (202) 512-2250 Mail: Stop SSOP, 
Washington, DC 20402-0001 





























                  COMMITTEE ON SCIENCE AND TECHNOLOGY

                   HON. BART GORDON, Tennessee, Chair
JERRY F. COSTELLO, Illinois          RALPH M. HALL, Texas
EDDIE BERNICE JOHNSON, Texas         F. JAMES SENSENBRENNER JR., 
LYNN C. WOOLSEY, California              Wisconsin
DAVID WU, Oregon                     LAMAR S. SMITH, Texas
BRIAN BAIRD, Washington              DANA ROHRABACHER, California
BRAD MILLER, North Carolina          ROSCOE G. BARTLETT, Maryland
DANIEL LIPINSKI, Illinois            VERNON J. EHLERS, Michigan
GABRIELLE GIFFORDS, Arizona          FRANK D. LUCAS, Oklahoma
DONNA F. EDWARDS, Maryland           JUDY BIGGERT, Illinois
MARCIA L. FUDGE, Ohio                W. TODD AKIN, Missouri
BEN R. LUJAN, New Mexico             RANDY NEUGEBAUER, Texas
PAUL D. TONKO, New York              BOB INGLIS, South Carolina
PARKER GRIFFITH, Alabama             MICHAEL T. MCCAUL, Texas
STEVEN R. ROTHMAN, New Jersey        MARIO DIAZ-BALART, Florida
JIM MATHESON, Utah                   BRIAN P. BILBRAY, California
LINCOLN DAVIS, Tennessee             ADRIAN SMITH, Nebraska
BEN CHANDLER, Kentucky               PAUL C. BROUN, Georgia
RUSS CARNAHAN, Missouri              PETE OLSON, Texas
BARON P. HILL, Indiana
HARRY E. MITCHELL, Arizona
CHARLES A. WILSON, Ohio
KATHLEEN DAHLKEMPER, Pennsylvania
ALAN GRAYSON, Florida
SUZANNE M. KOSMAS, Florida
GARY C. PETERS, Michigan
VACANCY
                                 ------                                

                 Subcommittee on Space and Aeronautics

                HON. GABRIELLE GIFFORDS, Arizona, Chair
DAVID WU, Oregon                     PETE OLSON, Texas
DONNA F. EDWARDS, Maryland           F. JAMES SENSENBRENNER JR., 
MARCIA L. FUDGE, Ohio                    Wisconsin
PARKER GRIFFITH, Alabama             DANA ROHRABACHER, California
STEVEN R. ROTHMAN, New Jersey        FRANK D. LUCAS, Oklahoma
BARON P. HILL, Indiana               MICHAEL T. MCCAUL, Texas
CHARLES A. WILSON, Ohio                  
ALAN GRAYSON, Florida                    
SUZANNE M. KOSMAS, Florida               
BART GORDON, Tennessee               RALPH M. HALL, Texas
              RICHARD OBERMANN Subcommittee Staff Director
            PAM WHITNEY Democratic Professional Staff Member
             ALLEN LI Democratic Professional Staff Member
            KEN MONROE Republican Professional Staff Member
            ED FEDDEMAN Republican Professional Staff Member
                    DEVIN BRYANT Research Assistant





























                            C O N T E N T S

                             March 5, 2009

                                                                   Page
Witness List.....................................................     2

Hearing Charter..................................................     3

                           Opening Statements

Statement by Representative Gabrielle Giffords, Chairwoman, 
  Subcommittee on Space and Aeronautics, Committee on Science and 
  Technology, U.S. House of Representatives......................    17
    Written Statement............................................    18

Statement by Representative Pete Olson, Minority Ranking Member, 
  Subcommittee on Space and Aeronautics, Committee on Science and 
  Technology, U.S. House of Representatives......................    19
    Written Statement............................................    20

                               Witnesses:

Mr. Christopher J. Scolese, Acting Administrator, National 
  Aeronautics and Space Administration (NASA)
    Oral Statement...............................................    21
    Written Statement............................................    23
    Biography....................................................    27

Ms. Cristina T. Chaplain, Director, Acquisition and Sourcing 
  Management, Government Accountability Office (GAO)
    Oral Statement...............................................    28
    Written Statement............................................    29
    Biography....................................................    33

Mr. Gary P. Pulliam, Vice President, Civil and Commercial 
  Operations, The Aerospace Corporation
    Oral Statement...............................................    33
    Written Statement............................................    35
    Biography....................................................    40

Discussion
  Impediments to Performance in NASA Projects....................    40
  NASA Implements Incentive and Punitive Measures to Increase 
    Accountability...............................................    41
  NASA's Problems Could Be Fixed With a Small Fraction of 
    Stimulus.....................................................    43
  Cost Management of Human Flight Missions Compared to Robotic 
    Missions.....................................................    43
  Should NASA Freeze Cost Estimate Design?.......................    44
  Long-duration Human Space Flight and Its Effect on Cost........    45
  Shrinking Industrial Base Adds to Cost.........................    45
  Growing American Industrial Base Means Changes to ITAR.........    46
  External Factors in NASA's Cost Growth.........................    46
  NASA Success Has Led to Heightened Public Expectations.........    47
  GAO Recommendations for NASA's Future Human Space Flight.......    48
  Cost Overruns in Climate Change Missions.......................    49
  Contractor Performance Contributes to Cost Overruns............    50
  NASA's Civil Servant Capacity to Provide Technological 
    Expertise....................................................    51
  NASA Life Cycle Performance....................................    51
  Firm-fixed Contracts Incentive to Avoid Cost Overruns..........    52
  More Cost Overruns Flexibility in Human Space Flight...........    54
  A Fixed Date to End the Space Shuttle Increase Cost Stability..    55
  How New Contracting Procedures Will Effect NASA................    56
  NASA Addresses Launch Conflicts and Increased Cost.............    56
  Comparison Between NASA and DOD on Cost Estimating, Budgeting, 
    and Programmatic Control.....................................    58
  MSL Dramatic Cost Estimate Change..............................    59
  Bad Judgment and Irrational Optimism Affects Cost Estimates....    59

              Appendix: Answers to Post-Hearing Questions

Mr. Christopher J. Scolese, Acting Administrator, National 
  Aeronautics and Space Administration (NASA)....................    64

Ms. Cristina T. Chaplain, Director, Acquisition and Sourcing 
  Management, Government Accountability Office (GAO).............    69

Mr. Gary P. Pulliam, Vice President, Civil and Commercial 
  Operations, The Aerospace Corporation..........................    72


       COST MANAGEMENT ISSUES IN NASA'S ACQUISITIONS AND PROGRAMS

                              ----------                              


                        THURSDAY, MARCH 5, 2009

                  House of Representatives,
             Subcommittee on Space and Aeronautics,
                       Committee on Science and Technology,
                                                    Washington, DC.

    The Subcommittee met, pursuant to call, at 10:00 a.m., in 
Room 2318 of the Rayburn House Office Building, Hon. Gabrielle 
Giffords [Chairwoman of the Subcommittee] presiding.

[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

                            hearing charter

                 SUBCOMMITTEE ON SPACE AND AERONAUTICS

                  COMMITTEE ON SCIENCE AND TECHNOLOGY

                     U.S. HOUSE OF REPRESENTATIVES

                    Cost Management Issues in NASA's

                       Acquisitions and Programs

                        thursday, march 5, 2009
                         10:00 a.m.-12:00 p.m.
                   2318 rayburn house office building

I. Purpose

    The House Committee on Science and Technology's Subcommittee on 
Space and Aeronautics will convene a hearing to review the status of 
the National Aeronautics and Space Administration's (NASA) efforts to 
improve the cost management of its acquisitions and programs. The 
hearing will focus on (1) the results of the Government Accountability 
Office's (GAO) just-completed assessments of selected large-scale NASA 
projects and its designation of NASA acquisition management as a 
``high-risk'' area, (2) the causes of cost growth and schedule delays 
in NASA acquisitions and (3) the Agency's progress in addressing them.

II. Witnesses

Christopher Scolese
Acting Administrator
National Aeronautics and Space Administration

Cristina T. Chaplain
Director
Acquisition and Sourcing Management
U.S. Government Accountability Office

Gary P. Pulliam
Vice President
Civil and Commercial Operations
The Aerospace Corporation

III. Overview

    When programs cost more to build and take longer than planned, NASA 
is able to accomplish less with the resources it is allocated. 
Confronted with specific instances of cost growth and schedule delay, 
the Agency is forced to either seek additional funds or make difficult 
trade-offs among its portfolio of projects such as shortening missions 
or removing instruments. An important factor in mitigating cost growth 
and schedule delay is accurate information with which to make decisions 
and cost projections. For more than a decade, GAO has identified NASA's 
contract management as a high-risk area--in part because of NASA's 
inability to collect, maintain, and report the full cost of its 
programs and projects. GAO has said that in the absence of such 
information, NASA would be challenged to manage its programs and 
control program costs. GAO has also underlined the importance of sound 
cost management in other reports. For example, in reporting on NASA's 
initial efforts to implement President Bush's 2004 Vision for Space 
Exploration, GAO said that in the past, NASA has had difficulty meeting 
cost, schedule, and performance objectives for some of its projects 
because it failed to adequately define project requirements and 
quantify resources.
    It is important to note that while essential to ensuring timely, 
effective and efficient acquisition of goods and services, integrating 
sound cost management in program management is not enough. For example, 
there must be transparency and clarity in the decision about whether 
design and development is appropriately performed internally or 
acquired from external sources. In addition, an emerging issue 
requiring NASA's focused attention is the impact of the growing number 
of bid protests lodged by vendors not selected in response to several 
of its procurements, such as those for new space suits (since resolved) 
and Commercial Resupply Services for the International Space Station. 
Awaiting the outcome of such bid protests can have adverse impacts on 
the Agency's planned schedules and program budgets. Potential means of 
minimizing the impact of protests range from ensuring proposal 
evaluations are defensible to guarding against any appearances of 
conflict of interest in evaluation panelists and advisory review group 
members chosen. Finally, successful acquisition outcomes require a 
skilled and motivated acquisition work force. The NASA Project 
Management Study completed in 1981 concluded that

         ``Good people are the key to good project management. Sound 
        project planning, management practices, and source evaluation 
        approaches are all important. However, they cannot substitute 
        for having high quality, and highly-motivated people 
        responsible for project management; both inside and outside of 
        government.''

    As GAO has indicated, NASA's need to effectively manage its 
programs will gain even more importance as the Agency seeks to manage 
its wide-ranging portfolio in an increasingly constrained fiscal 
environment. While today's hearing will focus specifically on cost 
management, the Committee on Science and Technology will continue to 
monitor and review a range of issues that impact NASA's ability to 
acquire needed goods and services in a timely, cost effective and 
efficient manner.

IV. Potential Hearing Issues

    The following are some of the potential issues that might be raised 
at the hearing:

          What are the main causes of cost growth and schedule 
        delays in NASA programs and projects? Is there a consensus on 
        what causes cost growth and schedule delay? Are there any 
        similarities with those experienced by the Department of 
        Defense and other federal agencies in their acquisition of 
        space systems?

          What has NASA done to mitigate cost growth and 
        schedule delay? Can other federal agencies benefit from NASA's 
        corrective actions?

          In light of continued instances of cost growth and 
        schedule delay in key programs such as Mars Science Lander and 
        Glory, how effective have NASA latest efforts been?

          What more needs to be done to mitigate cost growth 
        and schedule delay in NASA programs?

          Why is NASA acquisition management still 
        characterized by GAO as a high-risk area after 18 years?

          What has NASA done in response to GAO's 
        characterization that NASA's acquisition management is a high-
        risk area?

          What must NASA do to warrant removal from GAO's high-
        risk list?

V. Background

    To effectively use public funds in carrying out its activities, the 
Federal Government is expected to employ sound management practices and 
processes, including the measurement of program performance. The 
Congress, Executive Branch officials, and the public want to know 
whether federal programs are achieving stated goals and what their 
costs are.

The Importance of Developing Reliable Cost Estimates
    As stated in GAO's ``Cost Estimating and Assessment Guide: Best 
Practices for Developing and Managing Capital Program Costs'' [GAO-09-
3SP], cost estimates are necessary for government programs for many 
reasons: supporting decisions about whether to fund one program over 
another, developing annual budget requests, and evaluating resource 
requirements at key decision points. Moreover, as stated in GAO's 
guide, having a realistic estimate of projected costs makes for 
effective resource allocation, and increases the probability of a 
program's success.
    GAO's guide defines a cost estimate as the summation of individual 
cost elements, using established methods and valid data to estimate the 
future costs of a program, based on what is known today. The management 
of a cost estimate involves continually updating the estimate with 
actual data as they become available, revising the estimate to reflect 
program changes, and analyzing differences between estimated and actual 
costs.
    The guide further states that the ability to generate reliable cost 
estimates is a critical function. Without this ability, agencies are at 
risk of experiencing cost overruns, missed deadlines, and performance 
shortfalls--all recurring problems that GAO's program assessments have 
revealed. Furthermore, cost increases often mean that the government 
cannot fund as many activities as planned or deliver them when 
promised.

Cost Growth and Schedule Delays in NASA Programs
    The need to mitigate cost growth and schedule delay in NASA 
programs is not a new concern. As early as 1981, it was identified by 
the NASA Project Management Study. In an article featured in NASA's ASK 
project management publication, Dr. C. Howard Robins, former Deputy 
Associate Administrator for Space, said that the study, colloquially 
referred to as the ``Hearth Study,'' would ``come to be viewed within 
NASA as a landmark.'' Both the House Committee on Science and 
Technology and House Committee on Appropriations requested the study 
due to congressional concerns about cost and schedule performance 
problems. The House Science and Technology Committee letter dated 
September 19, 1980 to NASA stated:

         ``A number of large projects, for example Galileo, LANDSAT-D, 
        space telescope, and ISPM, are experiencing cost and schedule 
        problems.''

         ``The Committee encourages NASA to take the necessary steps to 
        minimize the cost and schedule impact of the problems 
        associated with these ongoing programs. Further, the Committee 
        has authorized new starts in the space and applications area 
        and is anxious that sound project management principles be 
        applied from the beginning of these new programs.''

    The 1981 study was the first multi-project study of program/project 
management by NASA and also the first study of the topic by an agency-
wide NASA team. Langley Research Center Director Donald P. Hearth led a 
team that studied thirteen robotic projects, including projects such as 
Viking and Voyager, undertaken over a twenty-two-year period.
    The study found significant problems, including inadequate project 
definition and over-optimism during advocacy. Several recommendations 
were made, such as requiring a formal definition review prior to the 
NASA decision to include the project in its budget request. A more 
detailed list of conclusions and summary recommendations made in the 
Hearth Study are included in Appendix A. More importantly, the study 
said that its conclusions and recommendations should be viewed with the 
following comments in mind:

         ``During recent years, several projects have experienced major 
        cost increases without apparent forewarning. This has damaged 
        NASA's credibility and reputation for successful project 
        management. Actions by NASA management are, therefore, 
        necessary; particularly, in light of NASA's external 
        environment and the pressures on government budgets.

         The Study Team verified, from its examination of a group of 
        representative projects that the cost performance of a project 
        is closely related to the application of sound project 
        management principles and/or the use of available management 
        tools. Therefore, the Study Team's Conclusions and 
        Recommendations are not intended to suggest the superposition 
        of either an additional hierarchy of management, or the 
        addition of new management tools within the current NASA 
        system. Rather they stress the need for continuing application 
        of the basic principles of sound project management by NASA, 
        refinement of existing management tools, and the continuing 
        verification, by NASA's top management, that the principles are 
        being followed and available tools are being used.''

    In ensuing years, NASA cost growth and schedule issues were 
conducted in its robotic and human space flight mission areas, 
including the International Space Station. More recently, the issues of 
cost growth and schedule delays in NASA programs have been addressed in 
legislation and analyzed in studies by GAO, the National Research 
Council (NRC), and NASA itself.

            NASA Authorization Act of 2005
    Cost growth and schedule delay were addressed in the NASA 
Authorization Act of 2005 (P.L. 109-155). Provisions were enacted to 
help NASA and Congress spot potential cost growth and schedule problems 
early in the development phase of a major program. Rather than 
discouraging risk taking, these provisions were intended to encourage 
NASA managers to identify risks as early as possible, when they are 
more readily managed and solutions are more easily implemented.

          Under the 2005 Act, a Baseline Report is required 
        whenever a major program completes required reviews and is 
        approved to proceed to implementation. NASA's policy defines a 
        project life cycle in two phases--the formulation and 
        implementation phases. During the formulation phase, projects 
        develop and define requirements and lead up to a preliminary 
        design review. Projects also complete development of mission-
        critical or enabling technology with associated demonstrations. 
        The implementation phase begins after project confirmation.

           After completing the Baseline Report, the Act requires NASA 
        to report periodically on a major program through an Annual 
        Report, which is provided as part of the annual agency budget 
        submittal to the Congress, until the program enters operation. 
        The provision defines a major program as an activity with a 
        life cycle cost estimate greater than $100 million. Having 
        established the baseline, the 2005 legislation sets thresholds 
        that, if exceeded, require agency action. Notification to 
        Congress and an internal evaluation are required in the event 
        that any major program exceeds its originally estimated 
        development cost by more than 15 percent or exceeds its 
        originally planned schedule by more than six months. The Act 
        also requires Congress to evaluate whether to continue the 
        major program in the event that it exceeds its originally 
        estimated development cost by more than 30 percent or $1 
        billion.

          The NASA Authorization Act of 2000 was amended to 
        better reflect current mission cost categories by increasing 
        the cost threshold that could trigger an independent cost 
        analysis from $150,000,000 to $250,000,000 and by requiring the 
        Administrator, rather than the chief financial officer, to 
        conduct the independent cost analysis.

            NASA Authorization Act of 2008
    Concerns regarding the increasing number of Earth science missions 
that were exceeding the 15 percent threshold established in the NASA 
Authorization Act of 2005 prompted a requirement in the NASA 
Authorization Act of 2008 for an independent review of the situation. 
Specifically, the Act directs the NASA Administrator to arrange for an 
independent external assessment to identify the primary causes of cost 
growth in large, medium, and small space and Earth science spacecraft 
mission classes. The external assessment is to also identify 
recommendations and provide a report within 15 months of the enactment 
of the Act. The National Research Council has been tasked by NASA to 
perform this review.
    In addition, the Glory program was reauthorized in the NASA 
Authorization Act of 2008, responding to the requirement in the 2005 
NASA Authorization Act that Congress evaluate whether to continue a 
major program in the event that it exceeds its originally estimated 
development cost by more than 30 percent.

            GAO Reports
    GAO has issued a number of reports dealing with cost and schedule 
problems in NASA's programs and with NASA's acquisition process:

          In its report of May 2004 on what it described as 
        NASA's lack of disciplined cost-estimating processes [GAO-04-
        642], GAO stated that the considerable flux it found in NASA's 
        program cost estimates--both increases and decreases--was an 
        indication that NASA lacked a clear understanding of how much 
        its programs will cost and how long they will take to achieve 
        their objectives. GAO found that the development cost estimates 
        for more than half of the 27 programs it reviewed had 
        increased, and that for some programs, this increase was 
        significant--as much as 94 percent. GAO also reported that 
        NASA's basic cost-estimating processes--an important tool for 
        managing programs--lacked the discipline needed to ensure that 
        program estimates were reasonable. GAO recommended that NASA 
        take a number of actions to better ensure that the Agency's 
        initiatives result in sound cost-estimating practices and are 
        integrated into the project approval process. NASA concurred 
        with GAO's recommendations.

          In March 2005, GAO reported [GAO-06-634] that the 
        James Webb Space Telescope (JWST) program increased its life 
        cycle cost estimate from $3.5 billion to $4.5 billion and 
        extended its schedule by almost two years. More than a third of 
        the cost increase was caused by requirement additions and other 
        changes. An increase in the program's contingency funding 
        [``reserves''] accounted for the remainder--about 12 percent--
        of the growth. About half of the cost growth was due to 
        schedule slippage. A delay by the Administration in approving 
        the use of a European Space Agency-supplied Ariane 5 launch 
        vehicle resulted in a one-year delay; an additional 10-month 
        slip was caused by NASA's budget profile limitations in fiscal 
        years 2006 and 2007.

           GAO reported that although the JWST program revised its 
        acquisition strategy to conform to NASA's acquisition policies, 
        the program still faced considerable challenges because it has 
        not fully implemented a ``knowledge-based'' approach to its 
        acquisition. For example, GAO noted that when program officials 
        initiated work and before the JWST program revised its 
        acquisition strategy, these officials had intended to have NASA 
        commit to program start with immature technologies and without 
        a preliminary design. Despite the program's change in 
        acquisition strategy to address GAO's concerns, GAO concluded 
        that the revised plan still might not permit the maturity of 
        key technologies to be adequately tested prior to program 
        start. Consequently, GAO recommended that the NASA 
        Administrator direct the JWST program to (1) fully apply a 
        knowledge-based acquisition approach to ensure that adequate 
        knowledge is attained at key decision points and (2) continue 
        to adhere to NASA acquisition policy and go forward only after 
        demonstrating that it is meeting incremental knowledge markers 
        and has sufficient funds to execute the program. NASA concurred 
        with GAO's recommendations.

          Following a review requested by this Committee's 
        then-Ranking Member, Rep. Bart Gordon, GAO reported in December 
        2005 [GAO-06-218] that while NASA's revised policy for 
        developing flight systems and ground support projects 
        incorporated some of the best practices used by successful 
        developers, it lacked certain key criteria and major decision 
        reviews that support a knowledge-based acquisition framework. 
        For example, NASA's policy requires projects to conduct a major 
        decision review before moving from formulation to 
        implementation and that prior to moving from formulation to 
        implementation, projects must validate requirements and develop 
        realistic cost and schedule estimates. However, as GAO found, 
        NASA's policies did not require projects to demonstrate 
        technologies at high levels of maturity before program start. 
        By not establishing a minimum threshold for technology 
        maturity, GAO said that NASA increased the risk that design 
        changes would be required later in development, when such 
        changes are typically more costly to make. GAO made several 
        recommendations to help ensure NASA uses a knowledge-based 
        acquisition approach in making informed investment decisions. 
        NASA concurred with GAO's recommendations.

           In releasing GAO's report, Rep. Gordon said:

                   ``As NASA embarks on an initiative to return 
                American astronauts to the Moon--an endeavor estimated 
                to cost more than $100 billion over the next 13 years--
                we need to have confidence that the Agency will be good 
                stewards of taxpayer dollars.'' He added ``In its 
                report out today, the GAO offers some common sense 
                recommendations aimed at reducing the chances that 
                NASA's projects will suffer cost growth and schedule 
                delays. I hope NASA will take the GAO's guidance 
                seriously.''

          This week, GAO released its report [GAO-09-306] 
        assessing the status of 18 large-scale projects at NASA. GAO's 
        independent assessment was conducted in response to the 
        explanatory statement of the House Committee on Appropriations 
        accompanying the Consolidated Appropriations Act of 2008; the 
        Committee on Science and Technology was a co-requester of the 
        assessment. Ms. Cristina Chaplain, a witness at this hearing, 
        directed GAO's work and will highlight the report's findings to 
        the Subcommittee. GAO compared projects against best practice 
        criteria for system development including attainment of 
        knowledge on technologies and design. The office found that 10 
        out of 13 projects that had entered the implementation phase of 
        the project life cycle experienced significant cost and/or 
        schedule growth. For those projects, GAO found that development 
        costs increased by an average of 13 percent from baseline cost 
        estimates that were established just two or three years ago; 
        average launch delay was 11 months.

           As an illustration, the development cost of the Mars Science 
        Laboratory (MSL) increased in the past year by over $200 
        million--more than a 26 percent increase and now stands at over 
        $1.2 billion. GAO anticipates that the MSL's development cost 
        will be even greater due to the launch being delayed from 
        October 2009 to 2011, a 25-month delay. Initially scheduled for 
        September 2009, the next window of opportunity for a Mars 
        launch occurs in the October/November 2011 timeframe. NASA 
        notified the Committee of that delay in December 2008, with the 
        Agency stating that a 2009 launch would be too risky because of 
        technical uncertainties. Regarding the challenges faced by MSL, 
        GAO reported that the program relied on several heritage 
        technologies that had to be re-designed, re-engineered, or 
        replaced. For example, the heat shield made of a light-weight 
        material had flown on previous missions and was considered 
        nearly ready. But a setback in testing forced NASA to select a 
        new and less mature technology. Also, the initial decision to 
        use dry lubricated lightweight titanium gears for rover 
        actuators had to be revisited when NASA found, during 
        fabrication, that the gears would not meet its durability 
        needs. As a result, the project has had to revert to heavier 
        stainless steel gears with a wet lubricant used by prior 
        projects. To keep the lubricant from freezing in Martian 
        temperatures, the project also had to add heaters. GAO said 
        this increased the mass of the MSL's rover.

           The underestimation of complexity resulting from the planned 
        use of new or heritage technology is not unique to the MSL 
        mission. GAO said that many of the projects reviewed indicated 
        that they had experienced challenges in developing new 
        technologies or retrofitting older technologies as well as in 
        managing their contractors. From a general standpoint, NASA 
        projects faced difficulty understanding the risks and 
        challenges they were up against when they started their 
        efforts. Challenges GAO identified included technology 
        maturity, design stability, complexity of heritage technology, 
        contractor performance, and performance by a development 
        partner such as an international space agency. GAO did not make 
        recommendations in this report as it acknowledged that NASA was 
        undertaking an array of initiatives aimed at improving program 
        management, cost estimating, and contractor oversight. However, 
        GAO said that NASA would benefit from a more disciplined 
        approach to its acquisitions and called for continued attention 
        to NASA's efforts to enable the Agency to maximize the 
        effectiveness of its acquisition investments.

            NRC's Review of NASA's Beyond Einstein Program
    NRC released a report in September 2007 entitled ``NASA's Beyond 
Einstein Program: An Architecture for Implementation.'' Prompted by 
Congress and the Office of Science and Technology Policy, NASA and the 
Department of Energy asked the committee to assess the five proposed 
mission concepts for achieving the goals of the Beyond Einstein space-
based physics research initiative, and recommend one for first 
development and launch.
    As part of its charge, the committee was tasked with determining 
the realism of preliminary technology and management plans, and cost 
estimates of the candidate Beyond Einstein mission set. Five mission 
areas--Joint Dark Energy Mission, Black Hole Finder Probe, Inflation 
Probe, and Einstein Great Observatories--comprised 11 mission 
candidates. Criteria used by the committee included plans for the 
maturing of critical mission technology, technical performance margins, 
schedule margins, risk-mitigation plans, and the proposal's estimated 
costs versus independent probable cost estimates prepared by the 
committee.
    The committee worked with an experienced outside contractor to 
develop independent cost estimates and a probable cost range for each 
candidate mission. The probable cost ranges were also compared with 
those of previous missions of similar scope and complexity. In all 
cases, the committee found higher costs and longer schedules than those 
estimated by the mission teams. The committee observed that this is 
typical of the differences between the estimates developed by mission 
teams and by independent cost estimators at this early stage of a 
program. Given the long history of missions comparable to the Beyond 
Einstein mission candidates, the committee said that it believed that 
the most realistic cost range for each of these missions is 
significantly more than the current estimates provided by the research 
teams.
    In discussing its assessment of mission readiness, the committee 
stressed the importance of technology readiness as a key consideration 
in the decision to proceed to mission development. The committee said 
that ideally, mission development should not commence until all new 
technologies necessary for mission success have reached a certain level 
of technology readiness. Experience has shown, the committee added, 
``that NASA and other missions pay the price when a mission enters 
development prematurely.''

            NASA Self-Examinations
    NASA research on incidents of cost and schedule growth and their 
causes is conducted by a number of organizations and individuals. 
Typically, according to NASA, this research is intended to assist the 
organization in evaluating performance trends; evaluate the 
effectiveness of their own organizational processes, tools and methods; 
and develop proposals for changes to their organization, processes, 
methods, and tools. NASA's Office of Program Analysis and Evaluation 
(PA&E) performs studies on an on-going basis and the topic of cost 
growth is frequently discussed at NASA Project Management Workshops and 
Cost Community Events such as a Cost Symposium. Because there is no 
clearinghouse for all cost estimating research being undertaken within 
NASA, the full extent of the Agency's research in cost growth and 
schedule delay cannot be fully characterized. Nonetheless, the 
following four examples are illustrative of self examinations the 
Agency has undertaken in recent years.

          In February 2004, NASA completed an analysis 
        comparing initial and final budget estimates of development 
        costs for 45 recent projects and computed percent budget growth 
        as a surrogate for cost growth. The analysis found that an 
        average cost growth of 36 percent and a median growth of 26 
        percent; 35 of 45 projects exceeded the initial budget 
        estimate. The relative change from the total of the 45 initial 
        budgets to the total of the 45 final budgets indicated a total 
        growth of 28 percent. In comparing historical budget growth 
        trends in the Department of Defense (DOD) and NASA as well as 
        describing the cost-estimating process changes made by DOD, 
        NASA analysts suggested that NASA cost-estimating processes 
        were in need of reform. But the analysts also listed a number 
        of changes already in progress that would have beneficial 
        impact, such as the then near-release of an update of NPR 
        7120.5 codifying the requirement for an Independent Program 
        Assessment Office project review prior to the two key project 
        milestones and requirements for a Cost Analysis Requirements 
        Description (CARD) and a full continuum of sound cost- and 
        program management practices; updating of NASA's Cost 
        Estimating Handbook; and development of training tools for 
        program managers. But the analysts also indicated that still 
        needed was a method for capturing project cost, technical, and 
        schedule data recorded in a standardized format and collected 
        at a reasonable frequency.

          At a NASA Cost Symposium in July 2007, analysts from 
        the Aerospace Corporation and NASA conducted a presentation 
        entitled ``Using Historical NASA Cost and Schedule Growth to 
        set Future Program and Project Reserve Guidelines.'' Analysts 
        discussed their investigation of the cost and schedule growth 
        history for 40 science missions--the ``mission set.'' By 
        looking at historical schedule and cost growth, analysts sought 
        to determine whether the past could be used to establish 
        guidelines for the levels of reserves needed for future 
        missions.

           Reserves are unallocated funds that are provided to counter 
        risks to costs and schedule that are unanticipated; they reduce 
        the probability that actual costs will overrun estimates. In 
        essence, they act as contingency funds to address circumstances 
        or outcomes that were not conceived of by an observer at a 
        given point in time--what is commonly known in project planning 
        as ``unknown unknowns.'' In contrast, ``known unknowns'' refers 
        to circumstances or outcomes that are known to be possible, but 
        for which it is unknown whether or not those outcomes or 
        circumstances will be realized.

           Examination of the historical data set by the analysts from 
        Aerospace and the Science Mission Directorate (SMD) showed that 
        the majority of projects had experienced cost and schedule 
        growth and that this cost and schedule growth was substantial. 
        The average cost and schedule growth for the mission set was 27 
        percent and 22 percent, respectively. Analysts said the data 
        highlighted that the primary internal reason for cost and 
        schedule growth was instrument development issues, and the 
        fundamental external reason for the growth was launch vehicle 
        delay.

           Analysis of project reserves was challenging to the 
        analysts. This is because reserve levels are not explicitly 
        identified in NASA budget documents. Using NASA backup budget 
        documents and other sources, analysts were able to identify 
        reserve values for eighteen of the forty missions were 
        obtained. The cost reserve levels held by each mission varied 
        from 10 to 30 percent while the average reserve was on the 
        order of 18 percent. Additionally, although specific schedule 
        reserve could not be identified from the budget, a general 
        industry rule of thumb that was prevalent when these missions 
        were developed was that a mission should carry one-month of 
        schedule reserve for each year of development. This equates to 
        an 8.3 percent schedule reserve for the project.

           Suggestions provided by analysts from Aerospace and SMD 
        included doing the following:

                  Requiring better technical and programmatic 
                definitions at the beginning of a project

                  Independently assessing design and cost/schedule 
                assumptions

                  Performing earlier instrument development to reduce 
                risk

                  Holding instrument CDR prior to spacecraft and 
                mission PDR

                  Considering increased cost and schedule reserves for 
                projects, some to be held outside the project.

           ``Best Practices'' for the control of cost and schedule in a 
        project were also identified, including:

                  Proper mission scoping

                  Robust initial cost and schedule estimate

                  Monthly estimates to complete

                  Importance of managing to schedule

                  Effective Use of Earned Value Management (EVM). Both 
                the IMAGE and Stardust missions used EVM. EVM is a 
                technique that compares the value of work accomplished 
                during a given period with the work scheduled for that 
                period. By using the value of completed work as a basis 
                for estimating cost and time needed to complete the 
                program, earned value can alert program managers to 
                potential problems early in the program. As was stated 
                for the IMAGE mission: ``The Earned Value system worked 
                well as an early indicator of cost problems ahead.''

           Analysts also stated that the real problem is that there is 
        no incentive for any project manager to underrun cost 
        estimates. They said that in today's culture, an underrun is 
        considered evidence that the project manager did not do enough 
        testing or analysis or should have added another instrument or 
        made the resolution better. A secondary problem identified by 
        the analysts was that project managers do not have the 
        authority to control costs, such as not being able to remove 
        excess personnel without Center Director approval. Until more 
        control is given to the project manager and incentives are put 
        into place to return funding, analysts concluded that cost 
        growth will still occur.

          In March 2008, NASA's SMD, assisted by the Science 
        Applications International Corporation (SAIC) presented a 
        summary overview entitled ``SMD Cost/Schedule Performance 
        Study'' before the NASA Advisory Council's Planetary Science 
        Subcommittee. The objective of the study was to evaluate the 
        cost/schedule performance record of selected SMD flight 
        projects to determine key drivers of cost/schedule performance, 
        and implementation approaches that enhance performance of SMD 
        missions. Project Managers and other key staff members were 
        interviewed to collect narrative descriptions to compare with 
        and explain the detailed historical data. Among its findings, 
        the study showed that:

                  Cost history data for 21 of the 24 projects studied 
                indicated cost growth. Total cost growth from the start 
                of the design phase to Estimate-to-Complete (ETC) at 
                Launch for all projects studied represented a combined 
                impact of $2.0 billion to SMD's mission portfolio.

                  Schedule history data indicated schedule slips for 
                19 of the 24 projects studied. The delays ranged from 
                five to 42 months.

                  Interview comments by eight projects cited early 
                planning deficiencies as a significant source of 
                development problems (underestimates, inexperience, 
                inadequate early technology investment, and/or design 
                heritage that was not realized).

                  The four projects that reported using EVM as a 
                management tool showed lower average growth in 
                development costs compared to projects that did not use 
                EVM.

           Regarding the key drivers that affected cost/schedule 
        performance for SMD projects, internal factors identified were 
        over-optimism early in the project's formulation phase, as 
        instrument development complexity. Launch service issues and 
        unstable or inadequate initial funding profiles were cited as 
        the most common external factors affecting cost and schedule. 
        Among the study's recommendations to mitigate cost growth and 
        schedule slips was one that SMD require more rigor in the 
        process used to generate early cost and schedule estimates and 
        establish a minimum set of requirements for a credible basis of 
        estimate for mission concept costing. It was also recommended 
        that projects be encouraged to include more conservatism in 
        base estimates early in the process and be required to 
        carefully evaluate all key project assumptions including design 
        heritage credits.

          At a presentation before the Goddard Space Flight 
        Center Symposium in June 2008, a member of the Aerospace 
        Corporation discussed perspectives on mission cost and schedule 
        performance trends, building on his team's review of 40 NASA 
        robotic science missions. The team's findings included the 
        following:

                  While estimates become more accurate as a project 
                matures, the greatest growth manifests itself late in 
                the project during integration and test.

                  Data highlighted that the primary reason for cost 
                and schedule growth is internal project technical and 
                development issues often associated with instruments.

                  Initial project estimates may be unreliable due to 
                design and technology immaturity and inherent optimism.

                  Better technical and programmatic appraisal early in 
                the life cycle is needed along with independent 
                assessment of design and programmatic assumptions.

           In addition, the team analyzed the relationship between 
        cost, schedule and complexity. A complexity index was 
        established for the projects reviewed based on performance, 
        mass, power and technology choices. The team plotted missions' 
        cost versus complexity index and found a near linear rising 
        ``band'' where successful missions cluster. On the other hand, 
        those missions failed that were below that clustered range. 
        This led the team to characterize this area as the ``no-fly 
        zone.''

GAO's Characterization of NASA Acquisition Management as High-Risk
    Since 1990, GAO has periodically reported on government operations 
that it identifies as ``high-risk.'' This effort has brought focus to 
problems impeding effective government and costing the government 
billions of dollars each year. GAO's high-risk status reports are 
provided at the start of each new Congress. Historically, high-risk 
areas have been so designated because of traditional vulnerabilities 
related to their greater susceptibility to fraud, waste, abuse, and 
mismanagement. As GAO's high-risk program has evolved, it has 
increasingly used the high-risk designation to draw attention to areas 
associated with broad-based transformations needed to achieve greater 
economy, efficiency, effectiveness, accountability, and sustainability 
of selected key government programs and operations. In 1990, GAO 
designated NASA's contract management as high-risk in view of 
persistent cost growth and schedule slippage in the majority of its 
major projects. Since that time, GAO's high-risk work has focused on 
identifying a number of causal factors, including antiquated financial 
management systems, poor cost estimating, and undefinitized contracts.
    In the January 2009 update of its high-risk list [GAO-09-271], GAO 
reported that since the 2007 high-risk update, NASA had taken 
significant steps to improve its acquisition management with the 
implementation of new policies and procedures and the development of a 
corrective action plan to address weaknesses in areas identified as 
high-risk by GAO. For example, NASA revised its acquisition and 
engineering polices to incorporate elements of a knowledge-based 
approach that should allow the Agency to make informed decisions. 
According to GAO, NASA is also instituting a new approach whereby 
senior leadership is reviewing acquisition strategies earlier in the 
process and developed broad procurement tenets to guide the Agency's 
procurement practices. Among procurement policy reforms, GAO noted that 
an earned value management procurement policy has been established and 
a requirement that all award fee contracts undergo a cost-benefit 
analysis has been codified to improve the likelihood that NASA is using 
its resources most effectively. GAO noted NASA's broad plan for 
reducing acquisition risk and observed that successful implementation 
of both the plan and revised policies should stem cost growth and 
schedule slippage.
    However, GAO said that because cost growth and schedule delays 
persist, this area--now titled ``acquisition management'' because of 
the scope of issues that need to be resolved--remains high-risk. GAO 
added that to maximize NASA's investment dollars, implementation needs 
to be complemented by vigorous executive leadership to foster the 
expansion of a business-oriented culture and a sustained commitment to 
identify and take action on projects that are not achieving cost, 
schedule or performance goals upon which they were based when they were 
initiated. Ms. Cristina Chaplain, who directed GAO's effort looking at 
NASA, is a witness at today's hearing and will highlight her team's 
findings.

Similarities Between NASA and DOD in Their Acquisition of Space Systems
    GAO has reported that the costs for DOD space acquisitions over the 
past several decades have consistently been underestimated--sometimes 
by billions of dollars. For example, Space Based Infrared System High 
program costs were originally estimated at $4 billion, but the program 
is now estimated to cost over $10 billion. Estimated costs for the 
National Polar-orbiting Operational Satellite System program--conducted 
jointly by DOD, the National Oceanic and Atmospheric Administration and 
NASA--have grown from almost $6 billion at program start to over $11 
billion.
    GAO found in November 2006 [GAO-07-96] that, for the most part, 
cost growth in DOD space acquisitions has been caused by the tendency 
to start programs before knowing whether requirements can be achieved 
within available resources--largely because of pressures to secure 
funding. GAO reported that unrealistic program office cost estimates 
exacerbated space acquisition problems and that with budgets originally 
set at unrealistic amounts, DOD has had to resort to continually 
shifting funds to and from programs, and such shifts have had costly, 
reverberating effects.
    GAO's analyses of six ongoing space programs shows some parallels 
with challenges faced by NASA. GAO found that original cost estimates 
were particularly unrealistic regarding the potential for savings from 
increased contractor program management responsibilities, the constancy 
and availability of the industrial base, savings that could be accrued 
from heritage systems, the amount of weight growth that would occur 
during a program, the availability of mature technology, the stability 
of funding, the stability of requirements, and the achievability of 
planned schedules. Ms. Cristina Chaplain, who directed GAO's effort 
looking at DOD's space acquisitions, is a witness at today's hearing 
and will highlight her team's findings. In addition, Mr. Gary P. 
Pulliam, from the Aerospace Corporation, has been asked to comment on 
whether there are any similarities in cost growth and schedule delays 
experienced by NASA and the Department of Defense/other federal 
agencies in their acquisition of space systems, and whether there are 
any ``lessons learned'' that would be applicable to these 
organizations.

Latest Actions by NASA to Address Cost Growth and Schedule Delay in Its 
        Programs
    In addition to agreeing to the recommendations made by GAO, NASA 
has implemented corrective actions on its own to address the issue of 
cost and schedule performance. For example, it has:

          Issued the 2008 NASA Cost Estimating Handbook (CEH), 
        a reorganized and updated version of the 2004 handbook. 
        According to NASA, the handbook provides useful information on 
        cost estimating for the entire NASA Cost Estimating Community. 
        It is meant to be both informative for the new cost estimator 
        and a good reference document for the experienced cost 
        estimator. Explanatory material accompanying the handbook 
        indicates that based on the extensive feedback from the NASA 
        Cost Estimating Community, the 2008 edition of the handbook has 
        been streamlined to make references easy to find, simplified to 
        make new initiatives easy to understand, and clarified to 
        communicate key policy messages efficiently. The material also 
        says that the handbook's information provides NASA-relevant 
        perspectives and NASA-centric data useful in the NASA 
        environment and facilitates the development of reliable, 
        comprehensive, defensible, and well documented cost estimates.

          Instituted a policy of budgeting to the 70 percent 
        confidence level. The policy, which is applicable to space 
        flight and information technology programs and projects, is 
        institutionalized in a new NASA Policy Directive (NPD 1000.5), 
        effective January 15, 2009. Programs are to be budgeted at a 
        confidence level of 70 percent or the level approved by an 
        authority of the Agency-level management council. As an 
        example, a 70 percent confidence level is the point on the 
        joint cost and schedule probability distribution where there is 
        a 70 percent probability that the project will be completed at, 
        or lower than, the estimated cost and at or before the 
        projected completion date. In the case of the Constellation 
        program, the confidence level was set at 65 percent by then-
        Administrator Michael Griffin due to programmatic conclusions 
        regarding the amount of technology heritage that would inform 
        the Constellation designs.

          Emphasized educating NASA staff on the need for 
        probabilistic cost and schedule estimating, how to do it, and 
        providing enabling tools.

          Implemented independent assessments of projects 
        through Standing Review Boards.

          Conducted in depth interviews with past NASA Program 
        Managers to better understand root causes of cost growth and 
        schedule delay. In particular, NASA recognized the need to 
        fully understand which factors contributed the most.

          Collected ideas to improve cost and schedule 
        estimates, such as spending more on R&D to mature technology 
        readiness levels, developing instruments first, demanding 
        better data to support claims at decision gates, and keeping 
        requirements stable.

          Established, under the Office of the Chief Engineer, 
        the Academy of Program/Project & Engineering Leadership (APPEL) 
        which provides leadership, advice, direction, and support for 
        the development and education of the NASA program/project 
        management and engineering community. Among its numerous 
        functions, the Academy facilitates the dissemination of 
        ``lessons learned'' and ``best practices'' through knowledge 
        sharing activities, including conferences, forums, 
        publications, case studies, and communities of practice.

Risk Management and the Challenge of Containing Project Costs
    Meeting technical and safety goals while also meeting programmatic 
constraints related to cost and schedule is a tremendous challenge. To 
that end, identifying and managing risks can be of significant help, as 
they are closely related to cost management efforts--initially in the 
planning of the project when costs are estimated and later during 
development when cost fluctuations invariably occur. Since mission 
success is the primary goal of any NASA activity, the Agency has 
recognized that effective risk management is critical to achieving that 
mission success. The implementation of a thorough, disciplined risk 
management approach is now required of all NASA programs and projects.
    Because of the pressure to contain costs, difficult decisions often 
need to be made when unplanned increases occur. To manage cost 
increases, particularly when increased funding is not provided, NASA 
projects have in the past altered (1) the scope of the project, 
including the elimination of scientific instruments, (2) management 
oversight by reducing the number of personnel assigned to that 
function, and (3) the testing sequence or reduced the testing 
requirements.
    When performed without sufficient recognition of risks, making such 
alterations can lead to catastrophic results as was demonstrated by the 
``lessons learned'' activity following the failure of the Mars Climate 
Orbiter probe. That spacecraft, developed under the Faster, Better, 
Cheaper (FBC) mantra advanced by NASA in the 1990s, was lost as it was 
landing on Mars in September 1999. In its report on Project Management 
in NASA dated March 13, 2000, the Mars Climate Orbiter Mishap 
Investigation Board stated that:

         ``Greater attention needs to be paid to risk identification 
        and management. Risk management should be employed throughout 
        the life cycle of the project, much the way cost, schedule and 
        content are managed. Risk, therefore, becomes the ``fourth 
        dimension'' of project management--treated equally as important 
        as cost and schedule.''

    The Board also said that it saw strong evidence that the systems 
engineering team and the systems processes were inadequate on the 
project, adding that:

         ``Inadequate independent verification and validation of Mars 
        Climate Orbiter ground software (end-to-end testing to validate 
        the small forces ground software performance and its 
        applicability to the software interface specification did not 
        appear to be accomplished).''

    With regards to reduced oversight, the Board noted:

         ``To exacerbate this situation, the mission was understaffed, 
        with virtually no Jet Propulsion Laboratory oversight of 
        Lockheed Martin Astronautics' subsystem developments. Thus, as 
        the mission workforce was reduced and focus shifted from 
        spacecraft development to operations, several mission critical 
        functions--such as navigation and software validation--received 
        insufficient management oversight.''

    The lesson learned, the Board said was that:

         ``In the era of ``Faster, Better, Cheaper,'' projects and line 
        organizations need to be extremely vigilant to ensure that a 
        Mission Success First attitude propagates through all levels of 
        the organization. A proper balance of contractor and project 
        oversight by technical divisions at NASA field centers is 
        required to ensure mission success and to develop a sense of 
        ownership of the project by the institution.''

Appendix A

                     NASA PROJECT MANAGEMENT STUDY

                             (January 1981)

                    (Known as the ``Hearth Study'')

OBJECTIVES

  To assess project management in NASA

  To identify generic reasons which aggravate cost and schedule 
growth

  To recommend appropriate actions by NASA

SUMMARY CONCLUSIONS

  Significant contributors to cost growth of several NASA 
projects

          Technical complexity of projects

          Inadequate definition prior to commitment

          Effect of low contractor bids

          Poor tracking of contractor accomplishments

  Significant contributors to good cost performance of several 
NASA projects

          The function of the NASA project manager

          Adequate definition prior to commitment

          Proper planning and management of reserves

          Early understanding between NASA and implementing contractor

          Good implementation by NASA and contractor(s)

  Not significant factors in cost growth of several NASA 
projects

          Inability to make cost estimates when project well defined

          Non-utilization of classified technologies

          Excessive influence of ``users''

  Difficulty to quantify effect of high inflation has 
contributed to cost growth in recent years

  Use and definition of reserves not consistent within NASA

  Ground segments have experienced cost growth and are not well 
defined prior to implementation

  Management of some projects assigned to multiple NASA Centers 
without timely interface definition

  Concurrent developments increase risk substantially

  Other concerns

          Industry's workload, interest in NASA work, etc.

          Composition of NASA workforce.

SUMMARY OF RECOMMENDATIONS

  Continue to pursue technically-advanced projects. Expect cost 
growth in some future projects.

  Require pre-project analysis and definition phases. 
Sufficient definition funding in NASA budget. Formal definition 
reviews. Require approved project initiation agreement and project 
plan.

  Select contractors primarily on technical considerations, 
management plans, past performance, etc.

  Issue NASA Policy to have adequate visibility of contractor 
activity. Center Directors responsible for policy implementation. 
Requires strong NASA in-house capability and adequate center resources.

  Fund implementing contractor at low level to develop thorough 
understanding. NASA project manager reconfirms or changes initial 
commitment.

  Provide completion costs of major projects in terms of 
budget-years dollars.

  Issue NASA Policy to have adequate reserves in all major 
projects. Based on definition maturity, risk, technical complexity, and 
concurrent developments. Managed by Headquarters program manager and 
project manager.

  General practice, minimize NASA management interfaces. When 
teaming of NASA Centers is appropriate, define interfaces prior to 
project implementation.

  Revise and re-issue NASA management Instructions defining 
project management policies.
    Chairwoman Giffords. Good morning, everyone. I am very 
pleased to welcome all of you to the first hearing on the 
Subcommittee for Space and Aeronautics of the 111th Congress. I 
am very much looking forward to working with the Ranking Member 
Pete Olson, who represents a district that has JSC in it and my 
colleagues on both sides of the aisle, I am very pleased that 
we have Members that make up many of the NASA facilities and a 
real passion on this committee or this subcommittee for space 
and aeronautics.
    We have a lot to do this year, but I am confident that we 
will have a productive and a cordial year ahead of us. I think 
that all Members of the Subcommittee would agree that NASA is 
one of America's greatest achievements, and as we look forward 
to the next two years on the Subcommittee, Congress and the 
American people will be looking to NASA to help solve some of 
our greatest challenges: development of Orion, the next 
exploration vehicle, as the Shuttle is scheduled to retire in 
2010; the ability to forecast and understand changes in climate 
as our planet continues to warm; other countries competing in 
space exploration for both military, also technological and 
scientific purposes; and the continuing decline that we have in 
numbers of STEM field graduates. The United States is not 
graduating enough students inspired to pursue career areas in 
these important areas of science, technology, engineering, and 
mathematics.
    What I like about this committee, well, I guess there is a 
lot I like about this committee, but one is this quote that 
stands above us. ``Where there is no vision the people will 
perish.'' So I think about NASA, and I want the American people 
to know as we continue to look for all of you that vision, that 
you have inspired us, that you have captured our imaginations, 
and you have transformed our nation.
    Our jobs as Members of Congress is to give you the tools 
and the resources you need to do your jobs, but we also have to 
be responsible as good stewards of the American taxpayers for 
the dollars that are allocated to your agency. And this is a 
responsibility that the Subcommittee Members take very 
seriously.
    With that the legislative session, the hearing today is 
going to be on cost management issues in NASA's acquisition and 
programs. I want everyone to know that this subcommittee 
hearing, just like all the Committee hearings, will be cast on 
the web, but I believe this is a good start for us, because it 
is appropriate for us to take some time to look at our 
oversight activities. It is clear that good cost and schedule 
management will be critical to the success of NASA's planned 
robotic and human space flight activities. It is good common 
sense given that NASA will always be working with a constrained 
budget and competing priorities.
    As has been summarized in numerous reports and studies, 
NASA has suffered cost growth and schedule delays of a number 
of projects and programs in recent years. The stated causes of 
the cost growth and delays have varied, and the prescriptions 
to fix the problems offered by individuals inside and outside 
the Agency are also varied, and indeed, sometimes have been in 
conflict.
    I want to use this hearing to start to sort out the facts 
and to explore what can and should be done to reduce the 
instances of cost and scheduled growth that we see at NASA. As 
was noted in the charter for today's hearing, cost and 
scheduled growth has been a concern at NASA since the early 
1980s, if not earlier. Moreover, from my vantage point on the 
House Armed Services Committee it is clear that our military 
space programs suffer from similar problems.
    However, it is also clear that NASA, DOD, and the other 
agencies of the Federal Government involved in space activities 
have many dedicated and competent scientists and engineers 
working long hours to try to deliver complex and successful 
projects. That tells me that dealing with these costs and 
schedule issues is hard, and that there is no simple fix or 
situation. If that was the case, certainly we would have 
resolved this a long time ago.
    So we need to figure out why preventing cost and schedule 
growth in our space project is so hard, but more importantly 
what we can do about it to be on a better path for the future. 
I suspect that getting on that better path will involve the 
need for improved practices within NASA in the Agency's 
oversight of its contractors and in its collaborations with our 
international partners, as well as better efforts by us, those 
of us in Congress, and the White House to reduce the vagaries 
of the budgeting process that introduce additional instability 
in NASA programming planning.
    It may not be possible to achieve perfection, but we 
certainly need to do our best to ensure that NASA is making the 
best use of its funds it has been given. We owe that to the 
American taxpayers as well as to those who are working so hard 
to advance this nation's agenda in space and aeronautics 
research.
    Today's hearing should be viewed as simply a first step in 
this subcommittee's oversight of NASA's acquisition and program 
management. I am very pleased that we have an incredibly 
knowledgeable panel of witnesses here before us today. I want 
to welcome each of you, and I look forward to hearing your 
testimony.
    With that I now yield to Mr. Olson for any opening 
statement that he would care to make.
    [The prepared statement of Chairwoman Giffords follows:]
          Prepared Statement of Chairwoman Gabrielle Giffords
    Good morning. I'm pleased to welcome everyone to this the first 
hearing of the Subcommittee on Space and Aeronautics in the 111th 
Congress.
    I'm very much looking forward to working with Ranking Member Olson 
and my colleagues on both sides of the aisle this year to tackle the 
important space and aeronautics issues facing the Nation.
    We have a lot to do, but I am confident that we will have a 
productive--and collegial--year ahead of us.
    I think all Members of the Subcommittee will agree with me that 
NASA is one of America's greatest achievements.
    As we look to the next two years, Congress and the American people 
will be looking to NASA to help solve some of our greatest challenges: 
the development of our next exploration vehicle as the Shuttle is 
scheduled to retire in 2010; the ability to forecast and understand 
changes in climate as our planet continues to warm; other countries 
competing in space exploration for both military, technological and 
scientific purposes; the continuing decline in numbers of STEM field 
graduates as are not graduating enough students inspired to pursue a 
career in these important areas.
    This quote that appears above us --where there is no vision the 
people perish.
    So as we look to the next two years, I want the American people to 
continue to look to NASA for that vision, and our job as Members of 
Congress is to ensure that NASA has the resources and tools you need to 
carry out the vision including the many tasks that we have given you.
    With that, however, goes the responsibility of ensuring that NASA 
is being a good steward of the resources provided to you by our 
constituents.
    I take that responsibility seriously, as I know my colleagues do, 
and that is why we are kicking off this legislative session with 
today's hearing--a hearing that is also being webcast, as are all of 
the Science and Technology Committee hearings.
    The topic of today's hearing--Cost Management Issues in NASA's 
Acquisitions and Programs--is an appropriate one with which to begin 
the Subcommittee's oversight activities.
    It is clear that good cost and schedule management will be critical 
to the success of NASA's planned robotic and human space flight 
activities.
    That is only good common sense, given that NASA will always be 
working with constrained budgets and competing priorities.
    As has been summarized in numerous reports and studies, NASA has 
suffered cost growth and schedule delays in a number of projects and 
programs in recent years.
    The stated causes of the cost growth and delays have varied, and 
the prescriptions to fix the problems offered by individuals inside and 
outside of the Agency have also varied--and indeed sometimes have been 
in conflict.
    I want to use this hearing to start to sort out the facts and to 
explore what can and should be done to reduce the instances of cost and 
schedule growth that we see at NASA.
    As was noted in the charter for today's hearing, cost and schedule 
growth has been a concern at NASA since the early 1980s, if not 
earlier.
    Moreover, from my vantage point on the House Armed Services 
Committee, it is clear that our military space programs suffer from 
similar problems.
    However, it is also clear that NASA, DOD, and the other agencies of 
the Federal Government involved in space activities have many dedicated 
and competent scientists and engineers working long hours to try to 
deliver successful projects.
    That tells me that dealing with these cost and schedule issues is 
hard, and that there's no simple fix or the situation would have been 
resolved long ago.
    We need to find out why preventing cost and schedule growth in our 
space projects is so hard, and more importantly, what we can do to put 
us on a better path for the future.
    I suspect that getting on that better path will involve the need 
for improved practices within NASA, in the Agency's oversight of its 
contractors, and in its collaborations with its international 
partners--as well as better efforts by Congress and the White House to 
reduce the vagaries of the budgeting process that introduce additional 
instability in NASA's program planning.
    It may not be possible to achieve perfection, but we certainly need 
to do our best to ensure that NASA is making the best use of the funds 
that it is given.
    We owe that to the American taxpayers, as well as to those who are 
working so hard to advance the Nation's agenda in space and aeronautics 
research.
    Today's hearing should be viewed as simply a first step in this 
subcommittee's oversight of NASA's acquisition and program management.
    I am very pleased that we have a very knowledgeable panel of 
witnesses here today to help us in that work.
    I want to welcome each of you, and I look forward to your 
testimony.
    With that, I will now yield to Mr. Olson for any opening statement 
that he would care to make.

    Mr. Olson. Madam Chairwoman, thank you for convening this 
hearing today. The subject matter we are going to discuss is 
always of great concern but particularly now in the current 
budget environment. I, too, want to welcome and thank our 
witnesses for taking the time to brief us on the GAO report, 
assessments of selected large-scale projects at NASA.
    Since this is our first Subcommittee hearing of the 111th 
Congress, I would like to offer my gratitude to Chair Gordon 
and Ranking Member Hall for this opportunity to serve. I would 
especially like to say that I am extremely pleased to be 
working with our Chairwoman Gabrielle Giffords. In every 
conversation we have had she has been more than gracious in the 
welcoming of my thoughts and perspectives. I want to thank you 
for that. And as further encouragement it turns out our 
thoughts and our perspectives are often the same. Our goals are 
similar, and our respect for the men and women of our nation's 
space and aeronautics agencies and industries are beyond 
measure.
    Our nation faces enormous challenges. In their own way the 
sectors we will be dealing with can help by offering ways to 
meet those challenges, whether by creating jobs, performing 
cutting-edge scientific research, or serving as the inspiration 
to a new generation of explorers. We must work to serve--our 
work must serve as a means to help those who are doing the work 
to fulfill their missions in the most effective way possible.
    And along those lines today's hearing on cost effectiveness 
in particular is a very critical one. Right now NASA has as 
much room for error in their budgeting as they do for one of 
their manned space missions. That much. The Agency needs to 
have a well-balanced, well-managed, and cost-efficient system 
of budgeting and scheduling for their future missions.
    I am glad to read that NASA, according to the GAO, has 
``developed a comprehensive plan to address systematic 
acquisition management weaknesses.'' I look forward to learning 
how this was done, what challenges remain, and how this will 
apply to future missions. But the GAO also writes that NASA 
``would benefit from a more-disciplined approach to its 
acquisitions.''
    There are many Members of this--in this Congress, myself 
and the Chairwoman chief among them, who stand ready and 
willing to stand up for increased NASA funding. To be effective 
in doing so we need to be able to show our colleagues and most 
importantly the American people that their tax dollars are 
being invested wisely.
    Thank you again for being here, and I look forward to 
hearing your statements into our discussion. Thank you.
    [The prepared statement of Mr. Olson follows:]
            Prepared Statement of Representative Pete Olson
    Madame Chairwoman, thank you for convening this hearing today. The 
subject matter we are going to discuss is always of great concern, but 
particularly now in this current budget environment. I too want to 
welcome and thank our witnesses for taking the time to brief us on the 
GAO report assessing selected large-scale projects at NASA.
    Since this is our first subcommittee hearing of the 111th Congress, 
I'd like to offer my gratitude to Chair Gordon and Ranking Member Hall 
for this opportunity to serve. I would especially like to say that I'm 
extremely pleased to be working with Chairwoman Gabrielle Giffords. In 
every conversation we have had, she has been more than gracious and 
welcoming of my thoughts and perspectives. I thank you for that. As a 
further encouragement, it turns out our thoughts and perspectives are 
often the same. Our goals are similar, and our respect for the men and 
women of our nation's space and aeronautics agencies and industries are 
beyond measure.
    Our nation faces enormous challenges. In their own way, the sectors 
we will be dealing with can help by offering ways to meet those 
challenges. Whether by creating jobs, performing cutting edge 
scientific research, or serving as the inspiration to a new generation 
of explorers, our work must serve as a means to help those who are 
doing the work to fulfill their missions in the most effective way 
possible.
    And along those lines, today's hearing on cost effectiveness in 
particular is a very critical one. Right now, NASA has as much room for 
error in their budgeting as they do for one of their space missions. 
The Agency needs to have a well balanced, well managed, and cost 
efficient system of budgeting and scheduling for their future missions.
    I am glad to read that NASA, according to GAO, has ``developed a 
comprehensive plan to address systemic acquisition management 
weaknesses'' and I looking forward to learning how this was done, what 
challenges remain, and how this will apply to future missions. But GAO 
also writes that NASA ``would benefit from a more disciplined approach 
to its acquisitions.''
    There are many members in this Congress, myself and the Chairwoman 
chief among them, who stand ready and willing to stand up for increased 
NASA funding. To be effective in doing so, we need to be able to show 
our colleagues, and most importantly the American people, that their 
tax dollars are being invested wisely.
    Thank you again for being here and I look forward to hearing your 
statements and to our discussion.

    Chairwoman Giffords. Thank you, Mr. Olson. If there are 
Members who wish to submit additional opening statements, your 
statements will be added to the record at this point.
    At this time I would like to introduce our witnesses. First 
up we have Mr. Christopher Scolese, who is currently the Acting 
Administrator at NASA. I would note that we will be having 
hearings on NASA's fiscal year 2010, budget request in the near 
future, and Members will have plenty of opportunities to ask 
questions of the Agency on the budget request at those 
hearings. So I would like to ask Members to confine their 
questions to Acting Administrator Scolese today to the topic of 
this hearing.
    We also have with us Ms. Cristina Chaplain, who is the 
Director of Acquisition and Sourcing Management at the 
Government Accountability Office, and we have Mr. Gary Pulliam, 
who is Vice President for Civil and Commercial Operations at 
the Aerospace Corporation. Welcome.
    As our witnesses should know, you will each have five 
minutes for your spoken testimony. Your written testimony will 
be included in the record for the hearing. When you have all 
completed your spoken testimony, we will begin with questions. 
Each Member, myself included, will have five questions for the 
panelists, and I would like to start with Mr. Scolese.

  STATEMENT OF MR. CHRISTOPHER SCOLESE, ACTING ADMINISTRATOR, 
      NATIONAL AERONAUTICS AND SPACE ADMINISTRATION (NASA)

    Mr. Scolese. Thank you, Ms. Chairwoman, and Members of the 
Subcommittee. Thank you for the opportunity to appear today to 
discuss NASA's efforts to improve the cost and schedule of the 
Agency's projects.
    NASA missions have allowed us to rove the surface of Mars, 
other planets, to send people to live and work in space, to 
improve our understanding of the universe, and to better 
understand our Earth. So we recognize the importance of 
delivering missions on cost and on schedule and developing 
clear and stable baselines for planning.
    We continually strive to improve our tools so that we can 
develop better estimates and implement corrective actions, and 
we are pleased that the Government Accountability Office 
recognizes these efforts. Today I will outline the progress 
NASA has made to date.
    NASA is fortunate to be entrusted with conducting 
revolutionary missions in Earth and space science, aeronautics, 
and human space flight. These missions are often one of a kind 
and require new capabilities and new technologies to meet 
mission goals. These new developments, while exciting and 
offering benefits to society, seldom have clear analogs to past 
missions, thus presenting challenges for the development of 
cost and schedule estimates. As a result, there is no perfect 
formula for estimating cost, so we must rely on a combination 
of experience, cost models based on past missions, and actual 
costs when available.
    We recognize two types of growth to understand our 
projects. First is internal growth, resulting from actions by 
the project. Examples are optimism in estimates of cost, 
optimisms in estimates of schedule or technology readiness, 
under-estimation of mission complexity, or the over-estimation 
of the utility of heritage components, components that have 
flown before. Growth may also be caused by poor management. We 
don't see this very often, but we do see it.
    The other type of growth result from external factors that 
are beyond the control of the project, and I mean, the project. 
Sometimes they are beyond the control of NASA, sometimes they 
are beyond the control of the government, but strictly speaking 
we are talking about the project. These include partner 
performance, industrial base issues, launch manifest issues, or 
changes in planned budget or budget profile.
    However, not all external factors affecting a project are 
due to poor performance. A project can be impacted by the 
success of other missions, missions that last well beyond 
expectation in terms of discovery and lifetime.
    We are planning to conduct further study to improve our 
understanding of these factors and to allow us to better 
estimate project life cycle costs. The existing analysis 
indicates that early in the project development estimates can 
be driven by the optimism I mentioned earlier. Typically, NASA 
uses these early estimates for planning purposes to identify 
the rough cost and schedule range for the proposed mission. It 
is important to note that we do not consider these early 
estimates as cost commitments.
    As a project advances, we develop a better understanding of 
the challenges, risks, technologies, and therefore, costs and 
schedule. We believe that we have the best estimate at a time 
when decisive action can still be taken at the completion of a 
properly-funded preliminary design. Thus, it is at this point 
that NASA makes our cost and schedule commitment for the 
project to the Congress.
    I am pleased to say that we have put improvements in place 
in our processes and tools to improve our cost and schedule 
performance. This was recognized by the GAO in the high-risk 
series. Improvements include standardization of project life 
cycle milestones and reports to ensure that all projects are 
measured consistently and are reporting to our stakeholders is 
likewise more consistent.
    Use of historical cost analysis, where possible, to 
identify areas that need to be addressed with corrective 
actions. Implementation of joint cost and schedule 
probabilistic estimation tools to better provide estimates for 
both early phase planning and later to support our commitment 
when we make it.
    Implementation of a rigorous monthly review process to 
provide ongoing senior management review of program project and 
institutional performance so we can correct problems before 
they become serious. Use of independent review boards to 
evaluate project performance at key decision points. Formal 
commitment of cost and schedule following the preliminary 
design review, and better use of tools such as earned value 
management.
    Also, annual assessments of proposed new missions versus 
the approved activities that are already in place to determine 
industrial base, partner, and NASA ability to accomplish those 
objectives.
    We recognize that cost and schedule estimation of a one-of-
a-kind mission is difficult. Therefore, multiple techniques are 
required to improve estimates and performance. Let me assure 
you and the American public that we are committed to this 
improvement and to working with the GAO and Congress to provide 
consistent metrics on our performance. There are many 
improvements already in place. Others are underway. From these 
we have developed and will continue to develop improved NASA 
processes yielding results now and in the years to come.
    Thank you for the time to speak today, and I am ready for 
questions.
    [The prepared statement of Mr. Scolese follows:]
              Prepared Statement of Christopher J. Scolese
    Ms. Chairwoman and Members of the Subcommittee, thank you for the 
opportunity to appear today to discuss NASA's progress in managing the 
cost and schedule of the Agency's projects. NASA missions have allowed 
us to rove the surface of other planets, to send people to live and 
work in space, to improve our understanding of the Universe, and to 
better understand our Earth. NASA recognizes the importance of 
delivering missions on cost and on schedule, and developing clear and 
stable baselines for planning. We strive to continually improve our 
tools to identify issues so we can implement corrective action. Today, 
my testimony will outline NASA's progress to date and the actions the 
Agency is taking to continue to improve its performance. We are pleased 
that the Government Accountability Office (GAO) recognizes our efforts 
to mitigate acquisition management risk and lay a foundation to improve 
project cost and schedule performance.

Federal Research and Development Environment

    As one of the Federal Government's research and development (R&D) 
organizations, NASA functions in an environment where we must accept 
and manage considerable risk and uncertainty. NASA develops scientific 
instruments, spacecraft, and new launch systems that redefine state-of-
the-art. The Agency strives to standardize and reuse systems and 
capabilities where feasible. However, where we endeavor to achieve the 
next goal, develop the next technology, and make the next discovery, we 
venture beyond the realm of past experience and into an environment of 
uncertainty and higher risk. This is just one of the facts of life in 
an aggressive and exciting R&D environment.
    Let me take a moment to share some examples with you, partially 
because they are illuminating, and partially because they show why 
people really love working at NASA.
    The International Space Station (ISS), permanently crewed since 
November 2000, is being built by over a dozen nations. The ISS already 
has the American Destiny and European Columbus science laboratories on 
board and, with the flight of STS-127 later this year, the Japanese 
Kibo laboratory will be complete. Upon its completion next year, the 
ISS will have a mass of over 900,000 pounds and be a world-class 
research center for conducting experiments in life and materials 
sciences; it will also serve as a training ground for long-duration 
human space missions. The ISS has repeatedly demonstrated the ability 
of nations to work together on complex projects: with Station 
components being designed and built in different countries, many were 
actually assembled for the first time in orbit. Now, international 
crews are operating, repairing, and utilizing the ISS for the benefit 
of the world. This kind of cooperation is essential if we are to 
continue to expand our reach beyond our planet. Research results have 
already improved medical science here on Earth: as you probably know, 
experiments conducted aboard the Space Shuttle and the ISS have been 
useful in demonstrating techniques for the development of salmonella 
vaccines. The ISS Program represents unprecedented international 
cooperation on a peacetime task of immense technical complexity.
    In the past five years, NASA has landed three vehicles on the 
surface of Mars--each without human intervention. The planning and on-
board capabilities to avoid obstacles make these landings some of the 
most difficult accomplishments imaginable. Think of shooting a 
basketball from Washington, DC, and making a perfect shot through a 
basketball hoop located at in Los Angeles without hitting the rim, 
while the rim is moving. The discoveries made by these rovers and their 
companion orbiters have changed our view of Mars. We now know that, at 
one time, Mars was indeed a wet planet, and our vehicles have found ice 
on its surface. More mysteries remain to be unlocked. The Mars Science 
Laboratory (MSL) is the next in the series of missions to Mars. MSL is 
significantly more complex than its predecessors, as it builds upon the 
lessons and discoveries they made to address the next level of 
scientific questions. As a result, the MSL vehicle is much larger--
about the size of a Mini-Cooper--than the Mars Rovers Spirit and 
Opportunity--roughly the size of a coffee table--so it requires a new 
type of landing system.
    The Nation and the world benefit from NASA's breakthrough research 
in Earth science and technology on a daily basis. This legacy began in 
April 1960 when NASA launched the world's first environmental 
satellite. The focus then was to improve weather forecasts. Our focus 
now is much more challenging. NASA conducts a comprehensive research 
program to advance fundamental knowledge on the most important 
scientific questions on the global and regional integrated Earth 
system. NASA presently operates 15 on-orbit Earth science missions, 
making measurements ranging from precision sea level through 
atmospheric chemistry and composition, and winds through ocean color 
and land vegetation, as well as ice cover and surface temperature. 
NASA's robust research and analysis develops outstanding scientific 
advances that improve climate projections and provide societal 
applications. NASA has six missions in formulation and development, and 
is pleased to have a first-ever National Research Council Decadal 
Survey for Earth science and applications that establishes NASA's 
priorities for satellite missions to study changes in the Earth's 
climate and environment. Achieving simultaneity of NASA's outstanding 
measurements is a major challenge for progress in understanding the 
changing climate, its interaction with life, and how human activities 
affect the environment.
    As you can imagine, the NASA and Earth science communities are 
saddened at the loss of a key Earth science asset when the NASA 
Orbiting Carbon Observatory satellite failed to reach orbit last week 
following launch. NASA immediately convened a Mishap Investigation 
Board to determine the cause of the launch failure. In addition, we are 
assessing options for its replacement. Although rare, these kinds of 
events demonstrate the need for flexibility in NASA's ongoing 
portfolio.
    The scientific and technical results across NASA's portfolio are 
substantial, and often extraordinary. However, as we push the 
performance envelope on several fronts, NASA's specific cost and 
schedule performance has, indeed, been less than desired in the past. 
It is NASA's responsibility to maximize the value of the American 
taxpayer's dollars. We already have some tools in place, but we also 
have plans to incorporate additional tools and make better use of 
existing tools and processes to improve our delivery of missions on 
cost and on schedule.

Potential Causes of Cost Growth and Schedule Delay

    NASA puts great effort into managing the environment of uncertainty 
that naturally surrounds a project. Some uncertainties are within the 
realm of the project's control. Proposers can be overly optimistic in 
their efforts to provide the most attractive package in a competition. 
The cost savings assumed based on the use of ``heritage technology'' 
for spacecraft or instruments can be over estimated. New technology 
development can ultimately be much more challenging than anticipated. 
Sometimes inadequate time is planned for early engineering efforts and 
refinement of requirements. These are all areas within project 
accountability and the majority of this statement outlines the steps 
NASA has taken to address these issues.
    I would like to digress for a moment to add a bit of ``ground 
truth'' on cost or schedule variances. NASA focuses a great deal of 
effort on measuring variations from plans and responding to trend 
patterns reported in monthly Baseline Performance Reviews, and in 
program and project reviews. NASA's renewed emphasis on the use of 
various tools such as Earned Value Management also help provide 
indications of problems early enough to take corrective action.
    Reports of apparent cost growth can be misleading. If one measures 
project cost or schedule from the very earliest conceptual phase, as 
compared to measuring cost after the preliminary design is complete, 
the project typically appears to have incurred significant growth. NASA 
commits to project cost and schedule estimates at the completion of the 
preliminary design phase when technology readiness is better 
understood, preliminary designs are complete, and partner arrangements 
and industrial base considerations are better understood. This 
information provides a much better basis for estimating cost and 
schedule. While useful and necessary for the initial planning phase of 
a mission, early estimates are, at best, educated guesses made with 
preliminary conceptual information. As an example, although there 
remains plenty of room for improvement in the case of MSL, one of these 
early conceptual estimates quoted in the press for MSL was not even an 
estimate produced by NASA.
    Other events can occur that are not within the control of the 
project, but are typically under the control, and within the 
accountability, of the overall program or the Agency. Owing to other 
stresses in the host program, funding flexibility to address problems 
may be inadequate, there may be inadequate validation of cost and 
schedule assumptions, or performance on one project may negatively 
affect others. This last point needs clarification. Not all projects 
that adversely impact other projects are poor performers. Sometimes 
they are stellar performers. For example, because on-orbit lifetime of 
a mission is difficult to predict from afar, projects already in 
operation that extend well beyond the original planned operational life 
may require more funding, resulting in the need to obtain resources 
from other sources, often projects in development. As an example, the 
Spirit and Opportunity Rovers on Mars were planned for approximately 
three months of operation, but are now past five years of operations 
and are still returning valuable data. NASA also tries to estimate 
these costs and control impacts by having a group of independent 
experts periodically review these extraordinary missions to assess 
their value and the likelihood that they will operate until the end of 
the projected budget horizon. However, who could have guessed that the 
Terra Earth Science mission--approaching its 10th anniversary--would 
operate over twice its design life, or that the Voyagers--at over 30 
years in space--would still be operational outside of our solar system?
    Of course, some events occur that are not under the control of the 
project or the Agency, although we take measures to mitigate the 
attendant risk. In the case of the Solar Dynamics Observatory, national 
launch manifest priorities--not project performance--resulted in delays 
of about a year, with the attendant cost growth. In the case of the 
Glory project--a first-of-a-kind Earth science mission--the mission 
experienced unexpected problems due to a loss of contractor expertise, 
which is illustrative of challenges in the aerospace industrial base. 
Simply put, the number of capable suppliers has substantially 
contracted and the demand is such that the skills of the remaining 
suppliers are difficult to maintain. Contributions from our 
international partners can be late. Launch vehicle delays or price 
increases have also had significant impacts. External changes in budget 
profiles, including the unavoidable impacts of Continuing Resolutions, 
can also occur. Out of the ten NASA projects in the GAO QuickLook 
Report that exceeded the Congressionally-mandated cost and schedule 
thresholds, approximately half did so as a result of external factors; 
some with limited solution options open to NASA.
    In an effort to better understand the extent to which our 
performance has been impacted by events that are beyond the control of 
the project and program, we have initiated a study of NASA and 
Department of Defense projects with the objective of being able to 
quantitatively separate internal and external growth. This will enable 
the Agency to better compare the results of a project's detailed cost 
estimate with the results of analytical cost estimates based upon 
historical performance. NASA currently anticipates completing this 
study by the end of calendar year 2009. We will keep the Congress 
informed of our progress in evaluating these factors.

Historical Cost and Schedule Studies

    Over time, various NASA organizations have studied cost and 
schedule growth after the fact. Most of the studies were focused on a 
specific question, or measured cost or schedule from different points 
in a project's life cycle. Additionally, the individual research tasks 
utilized different data, methods, and approaches, and thus are not 
directly comparable.
    To provide a proactive means to control costs, NASA has implemented 
monthly reviews--using common data set requirements and consistent data 
and analyses that are centrally coordinated--to produce results that 
are comparable from project to project and from year to year. It is 
this data that is now reported both internally to NASA and to the 
Administration and externally to the Congress. The January 2009 update 
to the GAO High-Risk Series notes a number of these changes that have 
improved NASA's standard reporting.
    Additionally, NASA is using the research on historical cost and 
schedule performance to identify areas that need to be addressed with 
corrections to tools or processes. A number of changes have been 
initiated that address common issues such as optimism in cost estimates 
and schedules, inadequate identification of risks, and unrealistic 
assumptions on technology maturity, along with external issues such as 
instability in funding, launch vehicle issues, and the performance of 
partners.

Steps Already Taken

    The Agency has undertaken a number of actions to address cost and 
schedule growth through modifications to NASA's project lifecycle. 
These actions are also noted in the NASA High-Risk Corrective Action 
Plan, which the Agency developed in recognition of the complexity and 
cross-functional nature of the issues identified in the GAO High-Risk 
Series. While NASA continues to address the issues outlined in the GAO 
High-Risk series, we were pleased that the January 2009 update to the 
series highlighted the efforts we have made to improve NASA acquisition 
management.
    Some actions that NASA has taken relate to the definition of a 
project life cycle that is now used by all space flight projects. 
Examples include:

          The project life cycle has six phases that each space 
        flight project now must address. This is a change from the 
        past, where different types of projects followed different 
        paths, so that comparisons were more difficult to make, and 
        most importantly, progress across NASA was difficult to assess.

          To ensure that we have an unbiased assessment of 
        project performance and plans, NASA has implemented the use of 
        Standing Review Boards to evaluate the project at each key 
        decision point in the project's life cycle. The Standing Review 
        Boards are composed of discipline experts who are independent 
        of the project being reviewed. The Boards provide the Agency 
        with independent advice on project design implementation, 
        manufacturing plans, cost and schedule planning, risks, and 
        margins. This change helps address past performance issues 
        related to optimism, inadequate evaluation of technology 
        maturity, heritage assumptions, etc.

          NASA commits to the project content, cost, and 
        schedule baseline only after successful completion of the Key 
        Decision Point C (KDP-C). At that point in the life cycle, 
        following the completion of the Preliminary Design Review, 
        project management has a more thorough understanding of the 
        technological maturity, complexity, and risk associated with 
        the project. As a number of risks have been retired by that 
        point, and the implications of the project requirements are 
        better understood, the baseline established at KDP-C provides a 
        more meaningful basis for measuring cost and schedule 
        performance. Several NASA research efforts confirm that the 
        Agency's cost and schedule performance is better when measured 
        from the KDP-C gate than when measured from the earlier 
        milestones.

Recent Actions

    In January 2009, NASA adopted a new acquisition strategy policy, 
which improves its ability to manage performance risk (including the 
adoption of probabilistic cost and schedule estimating methods). Among 
its features, the new policy requires space flight and information 
technology projects and programs to develop joint cost and schedule 
probabilistic estimates. Probabilistic estimating provides NASA with an 
approach that fully integrates technical, cost, and schedule plans and 
risks to develop both an understanding of the sensitivity of parameters 
to each other and the most likely estimate. Using this approach allows 
NASA to understand and document how the mitigation of technical risks 
would enable an increase in the project confidence level. Conversely, 
the introduction of a budget reduction would have the effect of 
increasing technical and schedule risks and thus lower the confidence 
level for the project. The use of probabilistic estimates also 
generates baseline values that include funding to address impacts 
associated with contingencies and uncertainties, such as industrial 
base, partner performance and technology optimism.
    The introduction of probabilistic joint cost and schedule 
estimating puts NASA on the leading edge of applying these techniques 
in both the federal and space sectors. Because this estimating approach 
requires the employment of new tools and techniques, full 
implementation will take some time to deploy; we are currently 
estimating at least two years to develop the tools, training, and 
understanding across the Agency. Given the deployment and the typical 
project development cycle of three to five years, it is unlikely that 
NASA will be able to evaluate the impact of these changes for a few 
more years. The recent GAO QuickLook Report underlines the fact that it 
takes time to realize the results from policy and process changes. 
Further, as we implement this joint confidence level policy, we are 
looking back at existing projects in development to ascertain risks and 
make adjustments where prudent to improve our cost and schedule 
posture.
    As noted earlier in this testimony, there have been issues with the 
consistency of historical data used for various cost research studies. 
In another recent action, NASA has taken steps to improve and bring 
consistency to the cost and schedule data collection that is now 
included in the Cost Analysis Data Requirement documents. This effort 
is also part of the NASA High Risk Corrective Action Plan. These 
documents serve to collect data in a standard format to allow us to 
assess performance on current projects and to provide a reference for 
future activities. At this time, NASA has completed detailed 
documentation on 38 historical projects and has captured data from 90 
KDPs on current projects.
    NASA is committed to using our tools and processes to identify 
issues and take corrective actions to address those issues. The steps 
that we have taken to standardize our project life cycle, to utilize 
Standing Review Boards to provide focused assessments at Key Decision 
Points, the renewed emphasis on tools such as Earned Value Management, 
the institution of strengthened acquisition planning and monthly 
reviews, and the use of joint cost and schedule confidence levels in 
our decision-making, have all moved NASA along a path towards improving 
our delivery of projects on time and within budget.

Conclusion

    In closing, cost and schedule estimation and performance are 
extremely important, and the Agency has taken a number of steps in 
recent years that have been acknowledged in the January 2009 update to 
the GAO High-Risk Series. We understand and support transparency and 
accountability in NASA project cost and schedule assessment.
    NASA is dedicated to the continuous improvement of its acquisition 
management processes and performance. There are many improvement 
efforts already in place, and others are underway. From these, we have 
developed--and will continue to develop--significantly improved NASA 
processes yielding results now and in the years to come.
    I would be happy to respond to any questions you or the other 
Members of the Subcommittee may have.

                  Biography for Christopher J. Scolese
    Since January 20, 2009, Mr. Christopher J. Scolese has been serving 
as the Acting Administrator of the National Aeronautics and Space 
Administration (NASA). As the Acting Administrator, Mr. Scolese is 
responsible for leading the development, design, and implementation of 
the Nation's civil space program. As such, Mr. Scolese provides overall 
leadership for NASA's multiple field installations, works closely with 
the Executive and Legislative branches to ensure that NASA is 
supporting appropriate national policy, and leads an international 
collaboration in carrying out high-profile space missions including the 
Space Shuttle, the International Space Station, the Hubble Space 
Telescope, and a multitude of other scientific and technological 
efforts.
    In addition, Mr. Scolese is still serving in the position of 
Associate Administrator, NASA's highest-ranking civil servant. As 
Associate Administrator, Mr. Scolese is responsible for the oversight 
and integration of NASA's programmatic and technical efforts to ensure 
the successful accomplishment of the Agency's overall mission.
    Previously, Mr. Scolese served as NASA's Chief Engineer. As Chief 
Engineer, Mr. Scolese was responsible for ensuring that development 
efforts and mission operations within the Agency were planned and 
conducted on a sound engineering basis, as well as for the long-term 
health of the NASA engineering workforce.
    Formerly, Mr. Scolese was the Deputy Director of the Goddard Space 
Flight Center where he assisted the Director in overseeing all 
activities. He also served as the Deputy Associate Administrator in the 
Office of Space Science at NASA Headquarters. In this position, he was 
responsible for the management, direction and oversight of NASA's Space 
Science Flight Program, mission studies, technology development and 
overall contract management of the Jet Propulsion Laboratory.
    Mr. Scolese also served as the Earth Orbiting Satellite (EOS) 
Program Manager and the Deputy Director of Flight Programs and Projects 
for Earth Science at Goddard. In these positions, he was responsible 
for the operation and development of all Earth Science missions 
assigned to Goddard. While there, he also served as the EOS Terra 
Project Manager. In addition, Mr. Scolese was the EOS Systems Manager 
responsible for the EOS system architecture and the integration of all 
facets of the project. During his tenure at Goddard, he chaired the EOS 
Blue Team that re-scoped the EOS Program; he supported the EOS 
investigators in the development of the EOS payloads in the 
restructured EOS; and he has been responsible for the adoption of 
common data system architecture on EOS and some other Earth orbiting 
spacecraft.
    Prior to his 1987 appointment at Goddard, Mr. Scolese's experience 
included work in industry and government. While a senior analyst at the 
General Research Corporation of McLean, Va., he participated in several 
SDIO programs. He was selected by Admiral Hyman Rickover to serve at 
Naval Reactors where he was associated with the development of 
instrumentation, instrument systems and multi-processor systems for the 
U.S. Navy and the DOE while working for NAVSEA.
    Mr. Scolese is the recipient of several honors including the 
Presidential Rank Award of Meritorious Executive, Goddard Outstanding 
Leadership, two NASA Outstanding Leadership Medals and the American 
Institute of Aeronautics and Astronautics (AIAA) National Capital 
Section Young Engineer/Scientist of the Year award. He was recognized 
as one of the outstanding young men in America in 1986, was a member of 
college honor societies including Eta Kappa Nu and Tau Beta Pi, and was 
recipient of the 1973 Calspan Aeronautics award. He is a Fellow of the 
AIAA and a member of the Institute of Electrical and Electronics 
Engineers. He also served as a member of the AIAA Astrodynamics 
Technical Committee and chaired the National Capitol Section Guidance 
Navigation and Control Technical Committee.

    Chairwoman Giffords. Thank you, Mr. Scolese.
    Ms. Chaplain, please.

 STATEMENT OF MS. CRISTINA T. CHAPLAIN, DIRECTOR, ACQUISITION 
AND SOURCING MANAGEMENT, GOVERNMENT ACCOUNTABILITY OFFICE (GAO)

    Ms. Chaplain. Madam Chairwoman and Members of the 
Subcommittee, thank you for inviting me to discuss our work on 
NASA's major acquisitions. In addition to my formal written 
statement, I would like to point out that GAO has just issued 
its first annual comprehensive assessment of major NASA 
programs, which is available on the GAO.gov website.
    To put this work into context I would like to start with 
our designation of NASA's contract management as a high-risk 
area in 1990. We put NASA on GAO's high-risk list because our 
work was continually finding there was little emphasis on end 
results, performance, and cost control. At the time NASA found 
itself procuring expensive hardware that did not always work 
properly.
    Numerous reviews following that report identified some more 
issues. Fourteen years later in a review of NASA's cost 
estimating practices, we concluded that acquisition problems 
still existed and found significant cost growth across a 
portfolio of 27 programs. In that review we found NASA lacked 
basic cost-estimating processes needed to establish priorities, 
quantify risks, and make informed investment decisions.
    We also found that programs were being moved forward into 
the development phases without a clear understanding of whether 
their requirements could be achieved within available funding, 
technologies, expertise, and other resources.
    Five years later in this most recent review we are finding 
that improvements have been made, but problems still exist. To 
NASA's credit much has been done to address the causal factors 
identified in our previous work, particularly in the areas of 
cost estimating, cost reporting, program management, and 
oversight.
    In addition, Congress has also done its part by instilling 
requirements that make cost growth more transparent and limit 
the circumstances in which programs can be re-baselined and by 
asking GAO to conduct an independent, annual assessment of 
major projects, many of which do not receive much external 
oversight.
    Because baselines were set just two or three years ago, the 
true magnitude of cost growth in the programs in our assessment 
is understated. Nevertheless, it is clear that programs still 
anticipate growth and cost and schedule. In just two or three 
years 10 of 13 projects we assessed in implementation 
experienced an average of 13 percent cost growth, and they had 
an average launch delay of 11 months.
    Many of the projects we reviewed indicated they have 
experienced challenges in developing new technologies or 
retrofitting older technologies, as well as in managing their 
contractors and more generally understanding the risks and 
challenges they were up against when they set their baselines.
    In conducting this work, we assessed the level of knowledge 
programs have about technologies, design, and their contractors 
at certain points in the development process, while at the same 
time tracking other causal factors that we have identified in 
previous reviews such as funding instability, requirements 
growth, program and contract management weaknesses.
    This methodology is designed to provide more insight into 
why programs encounter problems, as well as what actions are 
needed to address problems. While it is beneficial to have a 
wealth of policies and procedures aimed at ensuring successful 
execution, it is still necessary for us and the Congress to see 
what is happening on the ground and what types of unknowns 
exist in programs despite what policies encourage.
    This assessment is not meant to create a debate over the 
uniqueness of NASA's projects or what specific criteria should 
be applied to assess what types of unknowns and problems exist 
in programs and implementation, but rather to serve as a useful 
oversight and management tool. We remain open to working with 
NASA to determine the best way forward for measurement. Though 
the criteria we have used has been time-tested, it is based on 
sound program management practices, and it has been well-
received in communities involving all types of complex and 
technical development efforts, including the defense space 
community.
    In conclusion, what is fundamentally important today is 
that NASA is being asked to undertake new missions in space 
science and aeronautics. As such, it is vital that NASA and 
Congress clearly understand the costs and uncertainties of 
programs proposed for authorization and during their execution. 
It is equally important that problems be transparent, well 
understood, and tracked. We believe our report sets the course 
for doing so.
    Thank you, and I am happy to answer any questions.
    [The prepared statement of Ms. Chaplain follows:]
               Prepared Statement of Cristina T. Chaplain

Madam Chairwoman and Members of the Subcommittee:

    Thank you for inviting me to discuss the National Aeronautics and 
Space Administration's (NASA) oversight and management of its major 
projects. As you know, in 1990, GAO designated NASA's contract 
management as high-risk in view of persistent cost growth and schedule 
slippage in the majority of its major projects. Since that time, GAO's 
high-risk work has focused on identifying a number of causal factors, 
including antiquated financial management systems, poor cost 
estimating, and undefinitized contracts. Because cost growth and 
schedule delays persist, this area--now titled acquisition management 
because of the scope of issues that need to be resolved--remains high-
risk.
    To its credit, NASA has recently made a concerted effort to improve 
its acquisition management. In 2007, NASA developed a comprehensive 
plan to address systemic weaknesses related to how it manages its 
acquisitions. The plan specifically seeks to strengthen program/project 
management, increase accuracy in cost estimating, facilitate monitoring 
of contractor cost performance, improve agency-wide business processes, 
and improve financial management.
    While we applaud these efforts our recent work has shown that NASA 
needs to pay more attention to effective project management. It needs 
to adopt best practices are adopted that focus on closing gaps in 
knowledge about requirements, technologies, funding, time and other 
resources before it makes commitments to large-scale programs. For 
instance, the Mars Science Laboratory, which was already over budget, 
recently announced a two-year launch delay. Current estimates suggest 
that the price of this delay may be $400 million--which drives the 
current project life cycle cost estimate to $2.3 billion; up from its 
initial confirmation estimate of $1.6 billion. Also, in just one year, 
the development costs of NASA's Glory mission increased by 54 percent, 
or almost $100 million, because of problems NASA's contractor is having 
developing a key sensor. Total project costs for another project, 
Kepler, have increased almost another $100 million within two fiscal 
years because of similar issues. Taken together, these and other 
unanticipated cost increases hamper NASA's ability to fund new 
projects, continue existing ones, and pave the way to a post-Shuttle 
space exploration environment.
    Given the constrained fiscal environment and pressure on 
discretionary spending it is critical that NASA get the most out of its 
investment dollars for its space systems. The Agency is increasingly 
being asked to expand its portfolio to support important scientific 
missions including the study of climate change. Therefore, it is 
exceedingly important that these resources be managed as effectively 
and efficiently as possible for success. The recent launch failure of 
the Orbiting Carbon Observatory is an all-too-grim reminder of how much 
time, hard work, and resources can be for naught when a space project 
cannot execute its mission.
    In response to congressional direction, we have prepared a 
comprehensive report on the management and oversight of NASA's major 
projects. It contains summaries of 18 projects with a combined life 
cycle cost exceeding $50 billion. It also contains an assessment of 
issues affecting projects across-the-board. A copy of this report is 
now available on GAO's website (www.gao.gov).\1\ In conducting this 
work, we compared projects against best practice criteria for system 
development including attainment of knowledge on technologies and 
design, as well as various aspects of program management. We expect to 
continue this assessment on an annual basis and to continually refine 
our examination so that our work can inform your oversight and NASA's 
own efforts to improve in the high-risk area of acquisition management.
---------------------------------------------------------------------------
    \1\ GAO, NASA: Assessments of Selected Large-Scale Projects. GAO-
09-306SP (Washington, D.C.: Mar. 2, 2009).
---------------------------------------------------------------------------
    In responding to our report, NASA asserted that the unique nature 
of its work and external factors beyond its control make it difficult 
to apply the same criteria that we apply to other major government 
acquisitions, particularly those with large production runs. We 
disagree. The criteria we used to assess NASA's projects represent 
commonly accepted, fundamental tenets of disciplined project 
management, regardless of complexity or quantity. In fact, the concept 
of the knowledge-based approach we use has been adopted in NASA's own 
acquisition policy. Key criteria that we use have been developed by 
NASA and/or incorporated into its engineering policy. Moreover, facing 
long-standing cost and schedule growth and performance shortfalls, the 
Department of Defense (DOD) acknowledges the need for a knowledge based 
approach in the Air Force's ``back to basics'' policy for space 
systems. Lastly, we remain open to discussions with NASA as to whether 
additional criteria can and should be applied to its systems to ensure 
that decisions to move forward in development are well-informed and 
ultimately, that taxpayer dollars are well spent.
    Today I will be highlighting the results of this work, the actions 
NASA is taking to address the concerns raised in our high-risk report 
and better position its projects to meet their goals, and what we 
believe is necessary to make these actions successful. Because we also 
have responsibility for examining military space systems, we will also 
highlight common challenges with space acquisitions within NASA and the 
Department of Defense (DOD). This testimony is based on previously 
issued GAO work, which was conducted in accordance with generally 
accepted government auditing standards.

Acquisition Management Problems Persist

    We assessed 18 projects in NASA's current portfolio. Four were in 
the ``formulation'' phase, a time when system concepts and technologies 
are still being explored, and 14 were in the ``implementation'' 
phase,\2\ where system design is completed, scientific instruments are 
integrated, and a spacecraft is fabricated. When implementation begins, 
it is expected that project officials know enough about a project's 
requirements and what resources are necessary to meet those 
requirements that they can reliably predict the cost and schedule 
necessary to achieve its goals. Reaching this point requires 
investment. In some cases, projects that we reviewed spent two to five 
years and up to $100 million or more before being able to formally set 
cost and schedule estimates.
---------------------------------------------------------------------------
    \2\ We only received data for 13 of the 14 projects in 
implementation. NASA did not provide cost or schedule data for the 
James Webb Space Telescope, which is in implementation.
---------------------------------------------------------------------------
    Ten of the projects in our assessment for which we received data 
and that had entered the implementation phase experienced significant 
cost and/or schedule growth from their project baselines.\3\ Based on 
our analysis, development costs for projects in our review increased by 
an average of almost 13 percent from their baseline cost estimates--all 
in just two or three years--including one that went up more than 50 
percent. It should be noted that a number of these projects had 
experienced considerably more cost growth before a baseline was 
established in response to statutory reporting requirement. Our 
analysis also shows that projects in our review had an average delay of 
11 months to their launch dates.
---------------------------------------------------------------------------
    \3\ For purposes of our analysis, significant cost and schedule 
growth occurs when a project's cost and/or its schedule growth exceeds 
the thresholds established for Congressional reporting per the National 
Aeronautics and Space Administration Authorization Act of 2005, Pub. L. 
No. 109-161,  103; 42 U.S.C.  16613 (b), (f) (4).
---------------------------------------------------------------------------
    We found challenges in five areas that occurred throughout the 
various projects we reviewed that can contribute to project cost and 
schedule growth. These are not necessarily unique to NASA projects and 
many have been identified in many other weapon and space systems that 
we have reviewed and have been prevalent in the Agency for decades.

          Technology maturity. Four of the 13 projects in our 
        assessment for which we received data and that had entered the 
        implementation phase did so without first maturing all critical 
        technologies, that is they did not know that technologies 
        central to the project's success could work as intended before 
        beginning the process of fabricating the spacecraft. This means 
        that knowledge needed to make these technologies work remained 
        unknown well into development. Consequences accrue to projects 
        that are still working to mature technologies well into system 
        development, when they should be focusing on maturing system 
        design and preparing for production. Simply put, projects that 
        start with mature technologies experience less cost growth than 
        those that start with immature technologies.

          Design stability. The majority of the projects in our 
        assessment that held a critical design review did so without 
        first achieving a stable design. If design stability is not 
        achieved, but a product development continues, costly re-
        designs to address changes to project requirements and 
        unforeseen challenges can occur. All of the projects in our 
        assessment that had reached their critical design review and 
        that provided data on engineering drawings experienced some 
        growth in the total number of design drawings after their 
        critical design review. Growth ranged from eight percent to, in 
        the case of two projects, well over 100 percent. Some of this 
        increase can be attributed to change in system design after 
        critical design review.

          Complexity of heritage technology. More than half the 
        projects in the implementation phase--eight of them--
        encountered challenges in integrating or modifying heritage 
        technologies. Additionally, two projects in formulation--Ares I 
        and Orion--also encountered this problem. We found that the 
        projects that relied on heritage technologies underestimated 
        the effort required to modify them to the necessary form, fit, 
        or function.

          Contractor performance. Six of the seven projects 
        that cited contractor performance as a challenge also 
        experienced significant cost and/or schedule growth. Through 
        our discussions with the project offices, we were informed that 
        contractors encountered technical and design problems with 
        hardware that disrupted development progress.

          Development partner performance. Five of the thirteen 
        projects we reviewed encountered challenges with a development 
        partner. In these cases, the development partner could not meet 
        its commitments to the project within planned timeframes. This 
        may have been a result of problems within the specific 
        development partner organization or as a result of problems 
        faced by a contractor to that development partner.

Common Acquisition Management Challenges Persist between NASA and DOD

    The challenges we identified in the NASA assessment are similar to 
ones we have identified in other weapon systems, including Defense 
space systems. For example, we testified last year that DOD space 
system cost growth was attributable to programs starting before they 
have assurance that capabilities being pursued can be achieved within 
available resources and time constraints. For example, DOD's National 
Polar Orbiting Environmental Satellite System (NPOESS) has doubled in 
cost from $6 billion to $12 billion due to challenges with maturing key 
technologies. We have also tied acquisition problems in space systems 
to inadequate contracting strategies and contract and program 
management weaknesses. Further, we issued a report in 2006 that found 
DOD space system cost estimates were consistently optimistic. For 
example, DOD's Space-Based Infrared High System was originally expected 
to cost about $4 billion and is now expected to cost nearly $12 
billion.
    We have found these problems are largely rooted in the failure to 
match the customer's needs with the developer's resources--technical 
knowledge, timing, and funding--when starting product development. In 
other words, commitments were made to achieving certain capabilities 
without knowing whether technologies and/or designs being pursued could 
really work as intended. Time and costs were consistently 
underestimated. As we have discussed in previous work on space systems 
at both DOD and NASA, a knowledge-based approach to acquisitions, 
regardless of the uniqueness or complexity of the system is beneficial 
because it allows program managers the opportunity to gain enough 
knowledge to identify potential challenges earlier in development and 
make more realistic assumptions about what they can achieve.

NASA Is Making a Concerted Effort to Reduce High Risk in Acquisition 
                    Management But More Needs to Be Done

    NASA has also taken significant steps to improve in the high-risk 
area of acquisition management. For example, NASA revised its 
acquisition and engineering polices to incorporate elements of a 
knowledge-based approach that should allow the Agency to make informed 
decisions. The Agency is also instituting a new approach whereby senior 
leadership is reviewing acquisition strategies earlier in the process 
and has developed broad procurement tenets to guide the Agency's 
procurement practices. Further, NASA is working to improve management 
oversight of project cost, schedule, and technical performance with the 
establishment of a baseline performance review with senior management. 
In order to improve it's contracting and procurement process, NASA has 
instituted an agency wide standard contract-writing application 
intended to ensure all contracts include the most up-to-date NASA 
contract clauses and to improve the efficiency of the contracting 
process. NASA is also requiring project managers to quantify the 
program risks they identify and collect more consistent data on project 
cost and technologies. It is taking other actions to enhance cost 
estimating methodologies and to ensure that independent estimates are 
used.
    These changes brought the policy more in line with best practices 
for product development. However, the Agency still lacks defined 
requirements across centers and mission directorates for consistent 
metrics that demonstrate knowledge attainment through the development 
cycle. In order for a disciplined approach to take hold, we would 
expect project officials across the Agency to be held accountable for 
following the same required policies.
    More steps also need to be taken to manage risk factors that NASA 
believes are outside of its control. NASA asserts that contractor 
deficiencies, launch manifest issues, partner performance, and funding 
instability are to blame for the significant cost and schedule growth 
on many of its projects that we reviewed. Such unforeseen events, 
however, should be addressed in project-level, budgeting and resource 
planning through the development of adequate levels of contingency 
funds. NASA cannot be expected to predict unforeseen challenges, but 
being disciplined while managing resources, conducting active oversight 
of contractors, and working closely with partners can put projects in a 
better position to mitigate these risks should they occur. 
Realistically planning for and retiring technical or engineering risks 
early in product development allows the project to target reserves to 
issues NASA believes are outside of its control.
    In conclusion, managing resources effectively and efficiently as 
possible is important more than ever for NASA. The Agency is 
undertaking a new multi-billion dollar program to develop the next 
generation of spacecraft for human space flight and at a time when it 
is faced with increasing demands to support important scientific 
missions, including the study of climate change, and to increase 
aeronautics research and development. By allowing major investment 
commitments to continue to be made with unknowns about technology and 
design readiness, contractor capabilities, requirements, and/or 
funding, NASA will merely be exacerbating the inherent risks it already 
faces in developing and delivering new space systems. Programs will 
likely continue to experience problems that require more time and money 
to address than anticipated. Over the long run, the extra investment 
required to address these problems may well prevent NASA from pursuing 
more critical science and space exploration missions. By contrast, by 
continuing to implement its acquisition management reforms and ensuring 
programs do not move forward with such unknowns, NASA can better align 
customer expectations with resources, minimize problems that could hurt 
programs, and maximize it ability to meet increased demands.
    Madam Chairwoman, this concludes my statement. I will be happy to 
answer any questions that you have.

                   Biography for Cristina T. Chaplain
    Ms. Chaplain currently serves as a Director, Acquisition and 
Sourcing Management, at the U.S. Government Accountability Office. She 
has responsibility for GAO assessments of military and civilian space 
acquisitions. Ms. Chaplain has also led a variety of DOD-wide 
contracting-related and best practice evaluations for the GAO. Before 
her current position, Ms. Chaplain worked with GAO's financial 
management and information technology teams. Ms. Chaplain has been with 
GAO for 18 years. She received a Bachelor's degree, magna cum laude, in 
International Relations from Boston University and a Master's Degree in 
Journalism from Columbia University.

    Chairwoman Giffords. Thank you, Ms. Chaplain.
    Mr. Pulliam.

  STATEMENT OF MR. GARY P. PULLIAM, VICE PRESIDENT, CIVIL AND 
        COMMERCIAL OPERATIONS, THE AEROSPACE CORPORATION

    Mr. Pulliam. Good morning. Madam Chairwoman and Members of 
the Subcommittee, I am pleased to represent the Aerospace 
Corporation and to appear before you today as you deliberate 
cost management issues in NASA's acquisitions and programs.
    As a private, non-profit corporation, the Aerospace 
Corporation has provided engineering and scientific services to 
government space organizations for almost 50 years. As its 
primary activity Aerospace operates a federally-funded research 
and development center, but we also undertake projects for NASA 
and other civil agencies.
    The Subcommittee asked us to focus on the main causes of 
cost and schedule delays at NASA, how effective NASA has been 
at mitigating these problems, and similarities we see in other 
agencies. While there are many factors affecting cost and 
schedule growth, I will briefly discuss four main causes as we 
see them.
    First, sometimes NASA is too optimistic in its initial 
designs. This is understandable because NASA's job is to push 
science to new frontiers, yet there is palpable pressure for 
mission execution at the lowest-possible cost. The net result 
can be a less-than-complete appreciation for the complexity of 
the technical baseline. This can lead to artificially-low 
initial cost estimates and optimistic schedules. The cost-
estimating process itself can introduce optimism depending on 
what data and what missions are included. These factors taken 
together can increase the likelihood that the initial cost 
estimate will not be an accurate indicator of the final program 
cost.
    Second, there can be scope changes as the design evolves. 
Designs and technologies mature as projects proceed through 
development. This, too, is understandable and perhaps even 
desirable from a science perspective. The project manager wants 
to deliver maximum value to his customer.
    However, the effects are clear. An instrument that is more 
complex than originally thought will likely require more mass 
or more power. We should not be surprised that these changes 
from the original design produce changes in cost and schedules 
as well.
    Third, the inherent difficulty of developing world-class 
technologies contributes to cost and schedule growth. 
Technology immaturity is most often apparent in the advancement 
of science through new instrument development. Delays in 
instrument development can lead to schedule delays for the 
entire project. NASA is working toward reducing risks 
associated with immature technologies but has reduced 
technology development in some other areas.
    Fourth, we must acknowledge that external influences can 
have a major effect on cost and schedule performance. From the 
program or project manager's perspective whether that change 
comes from the Congress or from inside NASA the effects are the 
same. Project changes can set off chain reactions all across an 
agency. Cost growths in one program may result in reducing 
funding from other programs that were performing well, making 
them less executable. Missions can be delayed or canceled 
because existing programs are consuming available budget. Each 
of these actions has a negative impact on established costs and 
schedule.
    In looking at the effectiveness NASA has in mitigating cost 
and schedule growth we believe NASA deserves some credit for 
their efforts. In the past few years NASA has initiated several 
measures specifically designed to solve this problem. Some are 
strategic such as budgeting at a higher confidence level, and 
some are tactical, such as collecting the necessary data for 
sound cost estimating. NASA should be commending, commended for 
investing in new and innovative technologies and techniques.
    One example is schedule estimating, which is a relatively 
new capability in our industry. Another is affordability 
analysis, which allows examination of the portfolio 
interactions, long-range planning, and evaluation of costs, 
risks, and program reserves. These are ground-breaking efforts, 
and they will produce positive results.
    Even with these management actions, however, there is still 
uncertainty in the cost management process. NASA is attempting 
to manage this uncertainty through establishing cost estimates 
at a higher level of confidence than in previous years. While 
commendable, this higher level of confidence is valid only if 
the baseline remains stable, if technology maturation is 
controlled, and if external influences are understood and 
managed. Effective cost and schedule management requires a 
project manager to accept, be accountable for, and execute to a 
valid baseline. NASA is studying how to reconcile project 
estimates with independent estimates. Greater transparency into 
both processes and reconciliation of these basis of estimate 
will yield positive results.
    Our overall assessment of NASA's mitigation efforts is that 
the Agency is implementing many measures which should provide 
increased capability for cost and schedule management, and as a 
result better cost and schedule performance.
    And finally, the Committee asked us to look at similarities 
between NASA and other federal agencies. We found far more 
similarities than differences. As outlined above, optimism, 
baseline growth, technology changes, and external influences 
are not unique to NASA. Dozens of commissions and panels have 
studied cost and schedule for decades, and the themes are 
generally consistent. While each federal agency can point to 
their unique problems and circumstances, the fundamental 
challenges of good cost and schedule estimating and performance 
are remarkably similar across federal agencies.
    Madam Chairwoman, I am pleased to present our findings and 
assessments to the Subcommittee, and I look forward to your 
questions.
    [The prepared statement of Mr. Pulliam follows:]
                 Prepared Statement of Gary P. Pulliam

Madam Chairwoman and Members of the Subcommittee:

    I am pleased to present The Aerospace Corporation's findings, 
assessments, and recommendations on cost and schedule management issues 
in NASA's programs.

The Aerospace Corporation

    The Aerospace Corporation is a private, nonprofit corporation, 
headquartered in El Segundo, California. Aerospace was created in 1960 
at the recommendation of Congress and the Secretary of the Air Force to 
provide research, development, and advisory services to the United 
States government in the planning and acquisition of space, launch, and 
ground systems and their related technologies. We provide a stable, 
objective, expert source of engineering analysis and advice to the 
government, free from organizational conflict of interest. We are 
focused on the government's best interests, with no profit motive or 
predilection for any particular design or technical solution.
    Aerospace does not compete with industry for government contracts, 
and we do not manufacture products. The government relies on Aerospace 
for objective development of pre-competitive system specifications and 
impartial evaluation of competing concepts and engineering hardware 
developments to ensure that government procurements can meet the user's 
needs in a cost-and-performance-effective manner.
    Aerospace employs about 4,000 people of whom 2,700 are scientists 
and engineers with expertise in all aspects of space systems 
engineering and technology. As its primary activity, Aerospace operates 
a Federally Funded Research and Development Center sponsored by the 
Under Secretary of the Air Force, and managed by the Space and Missile 
Systems Center in El Segundo, California. Our principal tasks are 
systems planning, systems engineering, integration, flight readiness 
verification, operations support, and anomaly resolution for National 
Security Space (NSS) systems. Through our comprehensive knowledge of 
space systems and our sponsor's needs, our breadth of staff expertise, 
and our long-term, stable relationship with the government, we are able 
to integrate technical lessons learned across all NSS space programs 
and develop system-of-systems architectures that integrate the 
functions of many separate space and ground systems.
    The Aerospace Corporation also undertakes projects for civil 
agencies, including the National Aeronautics and Space Administration 
(NASA). Such projects contribute to the common good of the Nation while 
broadening the knowledge base of the corporation. Aerospace's support 
to NASA includes work on solutions to the foam and ice debris damage 
that resulted in the loss of the Space Shuttle Columbia, analysis of 
alternatives to robotically servicing the Hubble Space Telescope, and 
contributions to the Mars Exploration Rovers program. Our support to 
NASA includes its headquarters and virtually all directorates as well 
as almost every NASA Center. NASA and the NSS clients emphasize 
different areas when they task Aerospace. NASA requests far less 
support but proportionately more programmatic and budgeting support 
while the NSS clients place primary emphasis on technical support. 
While Aerospace certainly does not have full and complete insight into 
all NASA programs and projects, nor do we support all NASA programs, we 
have a unique relationship with NASA and have unique insights which we 
are privileged to share with the Committee.
    The Subcommittee asked us to focus our testimony on: 1) Identifying 
the main causes of cost growth and schedule delays in NASA programs and 
projects found during the course of The Aerospace Corporation's body of 
work at NASA; 2) Assessing the effectiveness of NASA's efforts in 
mitigating them; and 3) Identifying, in the context of The Aerospace 
Corporation's work at other federal agencies, any similarities in cost 
growth and schedule delays experienced at NASA.

Identifying the main causes of cost growth and schedule delays in NASA 
                    programs and projects found during the course of 
                    The Aerospace Corporation's body of work at NASA

    Aerospace has enjoyed a relationship with NASA for many years. We 
have studied NASA project cost and schedule for the Administrator, 
Associate Administrator, the Office of Program Analysis and Evaluation, 
Headquarters Mission Directorates, and many NASA Centers. Our work with 
NASA reveals that cost growth and schedule delays result from a variety 
of complex reasons.
    In recent years, NASA has commissioned several studies to determine 
the primary contributing factors to cost and schedule growth. These 
studies, as well as others in the field, identified several common 
themes: significant optimism in initial designs, changes in scope 
associated with the evolution of the design over time, the inherent 
technical difficulty of developing world class technologies, and the 
effect of external influences on the project such as funding 
instability. Although the conclusions stated above are primarily drawn 
from the analysis of a subset of NASA's science missions, we believe 
that the observations are applicable to a broader array of NASA 
projects.

Optimism in Initial Design

    NASA, as part of its charter, conducts unprecedented exploration 
and science. These missions continually push the envelope of the 
capabilities required by its human space flight and scientific 
instruments and spacecraft. The novelty and value of these science 
instruments are indisputable, as evidenced by the recent discovery by 
the Fermi Gamma Ray Space Telescope of the highest-energy gamma-ray 
burst ever recorded. At the same time, there is significant competitive 
pressure, both within NASA and among its contractors, to initiate a 
mission at the lowest possible cost. As noted by former NASA 
Administrator James Webb, it is not unusual for teams to ``put their 
best foot forward'' when proposing a new mission. In a recent study of 
the cost and schedule growth of 40 NASA science missions, only five of 
the 40 missions investigated resulted in no cost and schedule growth 
while over a quarter experienced cost growth greater than 40 percent 
above and beyond the project's internal cost reserves. In some cases, 
the content or complexity of the technical baseline is under-
appreciated. In other cases, the initial estimate of technical 
resources such as mass or power is inadequate or reliance on heritage 
systems is overstated. The initial inadequate technical baseline and/or 
poorly defined requirements lead to an artificially low initial cost 
estimate resulting in significant cost growth beyond the project's 
internal cost reserves. Furthermore, optimism may be introduced into 
the cost estimating process from empirical cost models that do not 
incorporate canceled missions, missions currently in development that 
are experiencing difficulties, or missions whose actual costs have been 
omitted or modified based on ``unusual'' circumstances. Another key 
driver of a project's final cost is schedule risk, which is often not 
adequately captured, making the initial schedule incompatible with the 
budget, resulting in an overall plan that is not executable. In 
summary, the optimism in the initial design starts the cycle, which is 
exacerbated by limitations in the cost estimating process.

Scope Changes as the Design Evolves

    The natural progression of a mission from its early conceptual 
design through its detailed design and implementation typically 
requires that resources (weight, power, performance, etc.) be added to 
meet stressing requirements. This growth in required spacecraft 
resources results in an associated cost growth. The understatement of 
the required resources is built into how the cost of the initial 
technical baseline is estimated. For example, while a recent historical 
study of robotic science missions observed that mass grew over 40 
percent on average from initial design inception to flight design, 
large mass growth factors are typically not applied in determining a 
cost estimate. Often, the complexity of the development effort, 
underestimated at the outset, is more fully understood as the 
development progresses. While the accuracy of project estimates 
improves over time, cost growth, over and above reserves, still occurs 
deep into the project life cycle. In short, the concept that is 
proposed is often not what is built. The initial cost estimate is 
likewise not representative of the final, as-built configuration due to 
required changes as the understanding of the design evolves. In 
essence, cost estimators are trying to estimate a moving target as 
projects progress toward their final design form.

Inherent Difficulty of Developing World Class Technologies

    NASA is continually pushing the technological envelope to reach its 
science objectives. The difficulty of landing a piece of hardware the 
size of a small car on the surface of another planet is only one 
example of the challenges that NASA faces on a regular basis. Each NASA 
development is unique, technically challenging and inherently 
difficult. To confront these challenges, technology is essential. The 
lack of mature critical technologies at project start contributes to 
the cost and schedule growth. A generally accepted risk avoidance 
practice is to fund focused technology development prior to system 
development. NASA, however, has reduced technology development funding 
in many areas due to budget constraints. Technology immaturity in 
science missions is often most apparent in instrument development, as 
opposed to spacecraft. Instrument development difficulties often lead 
to schedule delays in which a ``marching army'' cost is incurred 
awaiting instrument delivery. Additional investment to mature 
instruments, prior to the start of full project development, could 
potentially lead to reduced cost and schedule growth for science 
missions.

External Influences

    External influences can have a major effect on cost and schedule 
performance. From the program or project manager's perspective, whether 
change comes from Congress or from inside NASA, the effects are the 
same. Examples of external influences outside a project's control 
include budget modifications, funding instability, changes in 
requirements or priorities, and launch vehicle delays. The project 
manager depends on access to unallocated budget, or reserves, to 
address problems. When Headquarters or Congress reassigns budget or 
change priorities, it is often at the cost of increased execution risks 
that fall outside of a given project's ability to accommodate within 
reserves. Within a portfolio, cost growth in one project may result in 
reducing funding to other projects making them all less executable. The 
resulting domino effect impacts all missions that follow as missions 
that have not started are postponed or missions early in their 
development are stretched to fit annual budget constraints.

Assessing the effectiveness of NASA's efforts in mitigating them (cost 
                    and schedule delays)

    NASA initiated several measures to mitigate cost and schedule 
growth since the middle of this decade. Some of these measures are 
strategic in nature, such as budgeting at the 70 percent confidence 
level, and some are more tactical, such as the collection of historical 
data to provide a sound basis for new cost and schedule methodology 
development. Schedule estimation, which is a relatively new capability 
within the industry, is an area in which NASA is investing to improve 
the state of the practice commensurate with the more mature cost 
analysis methodologies. Affordability analysis, which allows 
examination of portfolio interactions, longer-range planning/analysis, 
and evaluation of cost risk and reserve policies, is another capability 
in which NASA has invested. Several introspective studies were 
commissioned to more fully understand the reasons for cost and schedule 
growth and provide recommendations on how to limit growth. The majority 
of these studies received peer review and have been published in the 
public domain. The progress in each of these areas is commendable.
    The collection of cost, schedule, and technical data is vital to 
developing representative cost and schedule models that are based in 
historical fact. NASA has embarked on an initiative to collect data, 
the Cost Analysis Data Requirement (CADRe) initiative. Aerospace is a 
contributor to this effort. Prior to the CADRe initiative, NASA's 
historical cost and schedule data collection from the early 1990s had 
been scarce and was based primarily upon the ability of individual 
organizations or programs to gather their own cost data. The CADRe 
initiative has institutionalized collection of data at specific 
milestones for a large set of missions across a large number of 
organizations. This data is invaluable in understanding and analyzing 
the cost and schedule growth of NASA projects and identifying 
contributing factors and causal relationships.
    In spite of these efforts, significant uncertainty remains in the 
cost estimating process. To offset this uncertainty, NASA has moved to 
estimating cost in a probabilistic fashion where a range of cost is 
estimated with associated confidence levels. NASA has also instituted a 
new requirement for budgeting projects at a higher level of confidence 
than previously experienced with a goal of giving projects a 70 percent 
chance of successfully meeting their budget. The validity of this 
approach, however, depends on the stability and soundness of the 
baseline. Every project has a budget estimate set by many inputs. 
Significant changes in these underlying assumptions and technical 
baseline will reduce the program's budget confidence. Furthermore, 
substantial differences of opinion remain within the cost-estimating 
community on how to develop and interpret probabilistic estimates.
    For effective NASA cost and schedule performance execution, the 
project must manage to a valid baseline estimate. One area of concern 
for the NASA project managers is the relevance and utility of 
independent cost estimates they do not own. Different methodologies are 
used by the project and independent estimate such that there is not a 
common understanding of the basis of estimate for each. Projects 
typically use bottoms up estimates that do not necessarily incorporate 
all of the risks. The disconnect between independent cost estimates and 
project estimates is exacerbated by the fact that unanticipated risks 
often manifest themselves late in development cycle during integration 
and test, when it is often too late to make adjustments. Greater 
transparency into the basis of estimate for each approach is important 
and needs to be communicated in ways that both the cost estimating 
community and project managers understand and recognize. One effort 
underway to strengthen the connection between an independent cost 
estimate and the project estimate is to include the effects of risk and 
risk mitigation to promote the project's ownership of the estimate. 
Incorporating the project's assessment of risks into the cost 
estimating process earlier and more often can put greater validity into 
the project's baseline cost estimate and provide a more robust reserve 
posture and promote the project's ownership of the estimate. NASA is 
using aspects of this philosophy on some of its projects. Continued 
expansion of its use should reduce unexpected cost and schedule growth 
in the future.
    The results of these measures have not yet had time to reach 
fruition as missions developed under the new initiatives have not yet 
been fully deployed. New methodologies such as schedule analysis tools 
and strategic mission portfolio models take time to influence project 
and program design in order to develop more robust project and program 
plans. Although it is too early to make an assessment, the studies that 
NASA has conducted and the initiatives that NASA has begun should move 
the Agency toward a more positive outcome and improve the ability to 
predict and control cost and schedule in its future.

Identifying, in the context of The Aerospace Corporation's work at 
                    other federal agencies, any similarities in cost 
                    growth and schedule delays experienced by NASA

    NASA is not alone in facing challenges in cost and schedule growth. 
The causes outlined above including optimism, growth, technology, and 
external influences, are not unique to NASA. The military procurement 
system has been analyzed for decades. Dozens of major commissions, 
panels, and academic studies have echoed these same issues, and we 
generally concur with the findings. The Aerospace Corporation's current 
and previous Presidents supported the May 2003 Defense Science Board/
Air Force Scientific Advisory Board Joint Task Force on Acquisition of 
National Security Space Programs. That group reported:

         The space acquisition system is strongly biased to produce 
        unrealistically low cost estimates throughout the acquisition 
        process. These estimates lead to unrealistic budgets and 
        unexecutable programs.

    The Task Force went on to note the need for new technology and the 
impact of technology risk on cost and schedule risk.
    In its most recent critique of defense acquisition, the Government 
Accountability Office noted:

         Invariably, the Department of Defense and the Congress end up 
        continually shifting funds to and from programs--undermining 
        well-performing programs to pay for poorly performing ones. At 
        the program level, weapon system programs are initiated without 
        sufficient knowledge about requirements, technology, and design 
        maturity. Instead, managers rely on assumptions that are 
        consistently too optimistic, exposing programs to significant 
        and unnecessary risks and ultimately cost growth and schedule 
        delays.'' (Defense Management: Actions Needed to Overcome Long-
        standing Challenges with Weapon Systems Acquisition and Service 
        Contract Management. GAO-09-362T, February 11, 2009)

    While each federal agency can point to unique problems and 
circumstances which impact project development, the fundamental 
challenges of good cost and schedule estimating and performance are 
remarkably similar across federal agencies.

Conclusion

    NASA's challenging mission includes a varied portfolio and 
substantial technological challenges. Many factors contribute to cost 
and schedule growth, but optimism in initial designs, changes in scope 
over time, the inherent technical difficulty of maturing technologies, 
and external influences are common themes we found. Many of these 
conditions and constraints exist for other federal agencies. NASA has 
initiated several measures to mitigate cost and schedule growth and 
these efforts should provide positive results over the next few years.
    The Aerospace Corporation is pleased that the Subcommittee 
requested we offer our views and stand ready for your questions.

Further Reading

 1)  Bearden, D., Boudrough R., and Wertz J., Chapter on ``Cost 
Modeling,'' Reducing the Cost of Space Systems, Microcosm Press, 1998.

 2)  Apgar, H., Bearden D. and Wong R., Chapter on ``Cost Modeling,'' 
Space Mission Analysis and Design (SMAD) 3rd edition, Microcosm Press, 
1999.

 3)  Bearden, David A., ``A Complexity-based Risk Assessment of Low-
Cost Planetary Missions: When is a Mission Too Fast and Too Cheap?'' 
Fourth IAA International Conference on Low-Cost Planetary Missions, 
JHU/APL, Laurel, MD, 2-5 May, 2000.

 4)  Bearden, David A., ``Small Satellite Costs,'' Crosslink Magazine, 
The Aerospace Corporation, Winter 2000-2001.

 5)  Bitten, R., Lao, N., Muhle, J., ``Joint Government/Industry Space 
Programs: Lessons Learned And Recommendations,'' SPACE 2001 Conference, 
28-30 August 2001.

 6)  Bitten R.E., Bearden D.A., Lao N.Y. and Park, T.H., ``The Effect 
of Schedule Constraints on the Success of Planetary Missions,'' Fifth 
IAA International Conference on Low-Cost Planetary Missions, 24 
September 2003.

 7)  Bitten R.E., Emmons D., Min I.A., and Radcliffe T.O., ``An 
Integrated Architecture Design/Cost/Scheduling Approach for Future 
Space Exploration Program Affordability,'' 1st Space Exploration 
Conference, 30 January-1 February 2005.

 8)  Bearden, D.A., ``Perspectives on NASA Robotic Mission Success with 
a Cost and Schedule-constrained Environment,'' Aerospace Risk 
Symposium, Manhattan Beach, CA, August 2005.

 9)  Bitten R.E., Bearden D.A., Emmons D.L., ``A Quantitative 
Assessment of Complexity, Cost, and Schedule: Achieving A Balanced 
Approach For Program Success,'' 6th IAA International Low Cost 
Planetary Conference, Japan, 11-13 October 2005.

10)  Bitten R.E., ``Determining When A Mission Is ``Outside The Box'': 
Guidelines For A Cost-Constrained Environment,'' 6th IAA International 
Low Cost Planetary Conference, 11-13 October 2005.

11)  Bitten R., Emmons D., Freaner C., ``Using Historical NASA Cost and 
Schedule Growth to Set Future Program and Project Reserve Guidelines,'' 
IEEE Aerospace Conference, Big Sky, Montana, 3-10 March 2007.

12)  Emmons D., ``A Quantitative Approach to Independent Schedule 
Estimates of Planetary & Earth-orbiting Missions,'' 2008 ISPA-SCEA 
Joint International Conference, Netherlands, 12-14 May 2008.

13)  Freaner C., Bitten R., Bearden D., and Emmons D., ``An Assessment 
of the Inherent Optimism in Early Conceptual Designs and its Effect on 
Cost and Schedule Growth,'' 2008 SSCAG/SCAF/EACE Joint International 
Conference, Noordwijk, The Netherlands, 15-16 May 2008.

14)  Bearden D. ``Perspectives on NASA Mission Cost and Schedule 
Performance Trends,'' Presentation at GSFC Symposium, 3 June 2008.

                     Biography for Gary P. Pulliam
    Gary P. Pulliam is Vice President of Civil and Commercial 
Operations. He was appointed to this position in December 2004. Pulliam 
directs all civil and commercial business at Aerospace and is 
responsible for contracts valued at $90 million annually. Key customers 
include the National Aeronautics and Space Administration, the National 
Oceanic and Atmospheric Administration, and a wide number of other 
civil and commercial organizations in the United States and overseas.
    In addition to his responsibilities in Civil and Commercial 
Operations, Pulliam is Corporate Director of Government Relations.
    Pulliam joined The Aerospace Corporation in 1994 as Director of 
Government Operations after serving for five years as Chief of Staff 
for U.S. Representative Earl Hutto of Florida's first congressional 
district. He concurrently was a professional staff member for the House 
Armed Services Committee, supporting Chairman Hutto, and was the 
Congressman's campaign manager.
    Pulliam was appointed General Manager in charge of non-Defense 
Department business at Aerospace in 1997. He has continued to handle 
government relations responsibilities while managing increasingly 
important civil and commercial programs.
    During a 20-year career in the Air Force, Pulliam served as a pilot 
and instructor and held assignments at the Aeronautical Systems Center 
in Dayton, Ohio. He also held several positions at the Pentagon, 
including an assignment as legislative liaison in the Office of the 
Secretary of the Air Force.
    He holds a Bachelor's degree in English from Clemson University and 
earned a Master's in operations management at the University of 
Arkansas. He also is a graduate of Harvard University's Kennedy School 
of Senior Managers in Government Program.
    The Aerospace Corporation, based in El Segundo, California, is an 
independent, nonprofit company that provides objective technical 
analyses and assessments for national security space programs and 
selected civil and commercial space programs in the national interest.

                               Discussion

              Impediments to Performance in NASA Projects

    Chairwoman Giffords. Thank you, Mr. Pulliam. At this point 
we are going to begin our first round of questions. The Chair 
recognizes herself for five minutes.
    Beginning with Mr. Scolese, we have heard testimony from 
Ms. Chaplain and Mr. Pulliam about the factors that have 
historically led to cost scheduled growth. They have also been 
able to identify some of the standard approaches that should be 
considered in attempting to mitigate such growth. But none of 
these approaches or none of these factors are certainly new, 
dating back to 1981, to the Hearth Report, certainly earlier 
than that there have been similar causes and mitigation 
strategies for cost and schedule growth in NASA and certainly 
for projects.
    Yet we are sitting here today because these projects 
continue to persist. I see from your biography that you have 
been on the front lines of the battle to control costs as a 
NASA Project and Program Manager in a variety of important 
positions. So in addition to knowing theoretically what should 
be done, you have actually experienced first-hand the real life 
challenges of attempting to manage cost and schedule.
    What from your perspective specifically are the most 
important real world impediments to consistently achieving on-
budget, on-schedule performance in NASA's projects and 
programs? And following that are we just going to have to 
accept some level of cost growth if we want to conduct space 
research and exploration? I realize that this is very 
challenging what we are asking you to do, but--or is it 
possible that measures that you and the other witnesses have 
outlined can make a significant difference?
    Mr. Scolese. Well, to answer the question first, you are 
correct. Many of these techniques have been talked about in the 
past, and Gary and Cristina discussed techniques that we are 
using. One of the things we are doing different today is we are 
using all of them in a consistent framework, so that we are not 
just doing cost estimation, we are not just improving our 
models, we are not just using EVM. We are taking all of those 
tools, and we are using all of them.
    In addition, we are having other people look at it, people 
that are independent of the performing organization, whether it 
is a project or a program or a center to go off and look at it 
because when you are sitting there, and you are looking at it, 
and you are doing it day to day, you think you can solve the 
problem. Otherwise you probably wouldn't be in the job. But 
somebody else looking at it is going to be a little bit more 
skeptical and look at what you are doing a little bit more 
clearly and bring that back for others to go off and evaluate 
and see if we can't make adjustments to help do it, to help 
correct problems, whether they are due to the project or they 
are outside of the project's control. That can be done if we 
know what they are early enough.
    You also asked, having lived in the trenches, what are the 
things that are most important to a project in order to be 
successful. Probably the most important thing is stability. One 
of the things that is extremely difficult in the project 
environment is uncertainty in your budget or uncertainty in 
requirements that you don't control. Once they are stable and 
they usually become stable later in the program development, 
things go fairly well. I mean, there can be surprises. There is 
no doubt about it. We are doing things that oftentimes haven't 
been done before, so we do get surprised. But stability was one 
of the things that we really looked for, and we really looked 
for, and we really felt that we needed in order to be 
successful.
    And last you asked do we have to live with consistent cost 
overruns. I think the answer is no, and I think you have heard 
from the three of us that we can do better. If implement these 
tools, we can better predict what our costs will be. Will they 
be totally gone? No. I don't think so, because what we do is 
very different. We have high-risks, and therefore, there is 
probability of failure and there is probability of cost growth. 
But overall we can do better, and I think when we put all these 
tools together you will see our performance improve.

      NASA Implements Incentive and Punitive Measures to Increase 
                             Accountability

    Chairwoman Giffords. Mr. Scolese, when instances of cost 
growth arise, many in the public want to know who exactly is 
accountable. It seems that assigning accountability can be 
difficult when there are a number of contributing factors that 
have led to the cost growth, but the desire for accountability 
at heart is based on the belief that it can contribute to 
preventing future cost and schedule growth at the Agency.
    As you know, in trying to deal with such growth and if some 
have called for punitive measures and some have called for 
providing incentives for good performance, what mechanisms does 
NASA currently have for penalizing poor cost and schedule 
performance, and conversely, what kinds of incentives does NASA 
provide for good performance?
    Mr. Scolese. Well, first and most important is, as you 
said, we need to understand what caused the growth. If it was 
something that was clearly caused by the project, the 
performance of the project, then, indeed, we do take actions, 
and over the last several years, over the life cycle of 
projects coming to completion now, we have replaced probably 
about 10 percent of our project managers. So there is strong 
accountability there for performance, if it is due to the 
project.
    Oftentimes it is not due to the project. It is due to that 
lack of stability, it is due to unforeseen technical problems. 
It is due to taking over a project that was originally in 
trouble. So first we look at what the causes are and then we 
take the appropriate actions as needed.
    In addition, we clearly hold accountable all of the people 
that are in the project. If a contractor is not performing, we 
have award fee scores. How much additional fee or how much fee 
we provide to them for their performance. Theoretically, if 
they were perfect, they would get 100 percent, and if they were 
terrible, they would get zero percent. So we evaluate based on 
that.
    I do want to say, though, that when we do that, we 
recognize that many of the people, most of the people that are 
performing on these projects are very good people. They have 
distinguished themselves in other fields, in other areas within 
NASA. So while they may not be successful here, we rely on our 
people to do our missions. Without them we can't do them. We 
rely on our contractor personnel and our partner personnel to 
do our missions.
    So when we do have to take action, we recognize these 
people have distinguished themselves somewhere else, and we 
move them onto other activities. So there is no public, you 
know, discussion of this. They move onto other things where 
they were at one time very good.
    But overall I think we have the tools to reward people for 
doing good performance. We have the tools to penalize people 
and organizations for doing bad performance.
    Chairwoman Giffords. Thank you, Mr. Scolese.
    The Chair will now recognize Mr. Olson for five minutes.
    Mr. Olson. And thank you, Madam Chairwoman. With your 
permission I would like to extend my time to the Ranking Member 
of the Full Committee and a fellow Texan, Mr. Hall.
    Mr. Hall. Thank you. I am not sure I thank you, but I 
didn't get to hear all of the--but, you know, we have a 
problem. We have a new Administrator coming aboard, and I don't 
think there is any secrets that Chair Gordon and I joined as 
Ranking Member requested the President to maintain Mike 
Griffin, and I know everything he did was not perfect, but he 
was always upright with us and made projections that we doubted 
but understood. And we are here with a four-year gap in there 
that concerns everybody, and we don't want to depend on nations 
that we don't really trust, but we are under some type of 
contract with them now.

        NASA's Problems Could Be Fixed With a Small Fraction of 
                                Stimulus

    I understand the acting Administrator can tell us about 
that, but I guess what I am trying to say is that, ``be not the 
first by whom the new is tried nor yet the last to lay the old 
aside.'' I don't know about the lone experience of schedule and 
cost growth and very complex robotic missions, and I think 
maybe we ought to just impose higher costs and schedules there 
in the projects' outset at the very beginning rather than 
trying to squeeze in too small reserves when the project 
encounters difficulties.
    But no matter what, where we are now and we need to somehow 
squeeze together those four years between the time that we can 
quit using Discovery and Endeavor and Atlantis, and get the 
project Constellation underway. Somehow--and with the minute 
budget compared to the overall budget that we extract for this 
project, it just seems to make sense to me, and with the 
offsets of--and the $850 billion expenditure and throwaway that 
the former President launched and the stimulus program of $700 
or $800 billion. It makes our project look like a very small 
cost and easy to add to.
    So I hope that we can all get together and push and carry 
out the program that had been set forward. The--and the hopes 
are--and it makes our expenditures look minute compared to the 
give-aways and the waste-aways and the throw-aways that they 
are, because I believe Jay Leno was right when he said that he 
thought the automobile-makers ought to keep on making 
automobiles and those guys on Wall Street ought to start making 
license tags.
    But we have such a small comparative budget and request for 
such an important program project. I just hope we will be more 
realistic in our approaches. And I didn't get to hear all the 
testimony, but that is the only thing I can offer.
    I yield back and thank you, Madam Chair.
    Chairwoman Giffords. Thank you, Mr. Hall. I think everyone 
on this committee agrees that we always look forward to your 
comments and you're an incredible asset to this subcommittee.
    The Chair will now recognize Ms. Fudge, who has Glenn in 
her district. Ms. Fudge, please.

     Cost Management of Human Flight Missions Compared to Robotic 
                                Missions

    Ms. Fudge. Thank you, Madam Chair.
    My first question is for Mr. Scolese. Are--do you believe 
that human space flight projects are likely to encounter cost 
and schedule growth? Are they more likely than if you use 
robotic science projects?
    Mr. Scolese. I don't think they are inherently different in 
terms of their purpose, but the difference is with human space 
flight programs we tend to do developments on such long time 
scales that they are separated by such a large amount of time. 
We started designing the Shuttle in the 1970s and began flying 
it in the 1980s. We did the Station in the 1990s, and we are 
doing Constellation now in the 2000s. And that presents some 
challenges because with robotic missions we tend to do several 
per year. With our aeronautics activities we tend to do several 
per year.
    So we don't have as strong a historical database to allow 
us to make the predictions that we can do with the robotic 
missions. So from that standpoint I think we have greater 
uncertainties in our initial estimates for the human missions 
than we do for the robotic missions, but that is because we 
haven't done so many of them. You can basically count on one 
hand all the human missions or the human space flight 
activities that we have done in the four decades, five decades 
that the space program has been around.
    So that presents a challenge for us. But to say that they 
are inherently different or that one can do different things 
than the other, that is certainly true, but that doesn't add to 
it. It is the fact that we don't do as many human missions as 
we do robotic missions.
    Ms. Fudge. Thank you. Mr. Pulliam----
    Mr. Pulliam. Yes, ma'am.

                Should NASA Freeze Cost Estimate Design?

    Ms. Fudge.--you stated in your testimony that the initial 
cost estimate is not representative of the final as built 
configuration due to required changes. Should NASA either 
freeze requirements early on or not proceed to implementation 
until it establishes greater knowledge of requirements and the 
resources needed?
    Mr. Pulliam. Thank you. It is among the most difficult 
problems to try to solve at NASA or other agencies spending 
taxpayer dollars. As we noted, when you start a program, you 
don't know everything that you need to know to execute that 
program. Sometimes the technologies just are in development, 
and that goes the way it goes.
    There is this optimism at the beginning. That is in part 
fostered by an increasingly competitive environment for NASA 
programs, so contractors and bidders are likely to bid more and 
more against the mission that is available. That contributes to 
optimism.
    So while one could just freeze design and say we are not 
going to allow any of this, our view is you would wind up not 
being able to accomplish the mission that is so important to 
NASA, and that is to put the world's best technology up for the 
benefit of mankind.
    So it is a difficult dilemma, that the answer in our view 
is somewhere along the way. You have to manage this technology 
maturity in a planned organizational way, understand the path 
that technology maturity is on, and understand the changes that 
accompany that to both your cost and your schedule. And then as 
NASA has stated, confirm that baseline at some point, at which 
time you have an executable program.
    So we wouldn't suggest that technology just continues to 
roam indefinitely, nor would we suggest locking it down so 
early solely for the benefit of cost and schedule because you 
would lose out on what you are trying to accomplish to some 
extent.

        Long-duration Human Space Flight and Its Effect on Cost

    Ms. Fudge. Thank you. And this is for any member of the 
panel. Certainly assuring the health and safety of our crews is 
critical to the success of NASA's exploration mission, and I 
guess because of NASA Glenn, we have had some discussions about 
the lack of timely demonstrations of hardware and techniques to 
mitigate the adverse effects of long duration space flights.
    Does that cost become one of the costs that you can't 
contain, or is that one of the issues that arises that creates 
some growth in the cost?
    Mr. Scolese. It certainly could, and that is one of the 
things that, of course, we have to address if we are going to 
do long-duration space flight. The Space Station is the tool to 
help us do that, and the better we can utilize it, the better 
we can retire those risks and develop better cost estimations, 
schedule estimates, and better estimates overall about when we 
can start sending people on those long-duration missions beyond 
low Earth orbit.
    So, yes, it does add an uncertainty to what we are doing. 
We are looking at that for the, you know, utilization of the 
Space Station. I am certain some of the efforts that Glenn is 
pursuing in those areas would be included in that.
    Ms. Fudge. Thank you, Madam Chair. I yield back.
    Chairwoman Giffords. Great. Thank you, Ms. Fudge.
    Mr. Olson, please.

                 Shrinking Industrial Base Adds to Cost

    Mr. Olson. Thank you, Madam Chairwoman. And, Mr. Scolese, I 
have a question for you. In your opening statement you 
mentioned the issues with the American industrial base, and I 
was just wondering if you could elaborate on the impact that a 
dwindling industrial base has on NASA and its maintaining a 
cost-effective schedule.
    Mr. Scolese. Yes, sir. As the industrial base has 
collapsed, of course, one of the initial or I shouldn't say 
collapsed. That is too strong of a word. As the industrial base 
has shrunk and consolidated that provides us less opportunity 
for competition. So oftentimes we have to go with the supplier 
that exists, and we don't have a choice.
    The bigger problem probably is the loss of expertise that 
was mentioned earlier in the opening statements. We don't have 
as many scientists and engineers and technicians to go off and 
build our instruments, build our spacecraft, build the 
components that go into those instruments or spacecraft, and 
that has created some degree of a problem for us.
    In addition, it has some unintended consequences because we 
have to go overseas for many of our components, and that, of 
course, hurts American industry.
    A third factor that plays into this is with lack of some 
parts we have had a rise in the number of counterfeit parts, 
things that we can't deal with. This is a worldwide problem. It 
is not a U.S. problem. This is a worldwide problem, and in 
dealing with that you find out late typically when you get 
counterfeit parts. We do inspections, we do all the things that 
you are supposed to do. That adds cost clearly, but when you 
find out about them, if you don't find out about them at 
receipt, you find out about it when you are in test or you find 
out about it when you are sitting on top of the rocket or 
worse, you find out about it when you are in space. And all of 
those have cost implications.
    One of the things that could help that, of course, is 
having, you know, more missions that are available along the 
lines of what we are talking about here. If we can reduce our 
costs, improve our cost estimations, we can provide, you know, 
opportunities to do that. But I think what I said pretty much 
summarizes the concerns that we see with the consolidation of 
the industrial base.

         Growing American Industrial Base Means Changes to ITAR

    Mr. Olson. Thank you very much. In addition to having more 
missions, any suggestions what we can do as a nation to help 
grow the American industrial base to ensure we are competitive 
and we have the capacity here in America to perform the 
missions that NASA performs?
    Mr. Scolese. Yes, sir. One area clearly is, I believe there 
was a hearing last week about the ITAR activities, and we are 
all very concerned about our national defense and recognize 
what ITAR's purpose is. But its implementation has been, has 
had an effect, a negative effect on our ability of our industry 
to compete overseas.
    It has also made it very difficult for us as the Nation's 
Space Program to also work with our partners overseas because 
we can't necessarily share information with them in ways that 
are as effective as they could be. It has also delayed our 
ability to make agreements, which has caused cost growth that 
we have talked about.
    When it takes a year or two to come to an agreement about a 
mission that you have agreed to do, you have clearly built risk 
into it as you can't talk to the international organizations 
that are providing resources. But for our industry they lack 
the ability to go off and compete because a contract on a 
communications satellite usually has about what, 30 days or 60 
days, to put in a bid. It takes that long to get through the 
process.
    So reducing some of those requirements or streamlining the 
process, and there is other people that know more about it than 
I do, could certainly help our industrial base be able to 
compete on the open market.

                 External Factors in NASA's Cost Growth

    Mr. Olson. Thank you for that question. That is a nice 
segue to my final question here, but in your testimony you 
indicated that ten of the projects that--of the ten projects 
that exceeded the costs five of those were due to external 
factors out of NASA's control. The ITAR thing is probably a 
great example.
    What are these external factors? Could you elaborate, and 
what can we do to help mitigate?
    Mr. Scolese. Well, not all can be mitigated easily. Three 
of them, I think four of them, in fact, were due to issues with 
either the launch manifest or with partner performance, and we 
have a backed up launch manifest right now. Two of the missions 
were delivered on time and unfortunately, the launch manifest 
is backed up to the point where one, the solar dynamics 
observatory will be delayed almost a year. That is something 
that we couldn't foresee. We are, of course, working with our 
colleagues in the Defense Department and the commercial 
industry to try and improve the launch manifest to improve our 
launch posture, but that is one area where we could, you know, 
clearly see some help.
    The other missions were affected by partner performance, 
international or U.S. partner performance, where they ran into 
difficulties that were unforeseen to us, and I am not sure 
there is much more that could be done that we haven't talked 
about. They have the same issues as was mentioned earlier that 
we do in terms of optimism and over-estimation, as well as 
stability of their funding requirements.
    And in at least one case it was due to, you know, 
industrial concerns, where, you know, if you went to a 
government lab or you went to a university or you went to 
industry, you might have had the same problems because of the 
lack of expertise. But that was an issue that we experienced.
    So it was those three things; the launch manifest, which is 
something we can do by, you know, encouraging a robust launch 
capability in this country, by partner issues, which were for a 
variety of different reasons that could have been unforeseen.
    Mr. Olson. Thank you very much, and I yield my time.
    Chairwoman Giffords. Great. Thank you, Mr. Olson.
    And just for the record, I think it is important to note 
that Mr. Olson and all of these Members down on this side 
either represent a NASA center or are adjacent to a NASA center 
and have NASA employees in it.
    And with that, Mr. Griffith, who represents Marshall.

         NASA Success Has Led to Heightened Public Expectations

    Mr. Griffith. Thank you, Madam Chair, and thank you, panel, 
for being here, and I would like to make a comment. Many of the 
questions that I had have been asked.
    One of the frustrations that I think that we are hearing is 
that cost overruns are inherent in any scientific endeavor. 
They are not budgeted, cannot be budgeted. Burn rates for 
discovering the unknown are going to be there. We understand 
that. I think it is unfortunate in a way that NASA has been so 
successful, has held themselves to such a standard of 
excellence that the general public expects that NASA is like 
starting their car in the morning. And that is really not that 
much science to it anymore, and there is not that much danger 
anymore, and there is not that much that really goes into it. 
We are just building rockets and exploding them into space.
    Nothing could be further from the truth. We are still on 
the frontier. China walked in space two months ago. Japan is on 
the way up, India is on the way up, Russia, of course, has been 
up. For the last 50 years NASA has established excellence in 
the pursuit of science and now you are going to become part of 
the high ground and the national defense. I expect there to be 
cost overruns, not only for safety issues but because we are 
going to be pushing the envelope because we will remain number 
one in space.
    So I think my question to you would be can we do a better 
job letting the public know what a huge scientific endeavor 
this is and how much we really don't know when we send humans 
into space? And can we maybe reeducate the public that we are 
on the great frontier again, trying to achieve the high ground, 
competing not only with Russia now but China, India, Japan, and 
certainly others?
    So I appreciate our attention to the detail of costs, but 
when we were discovering the vaccine for polio, we did not ask 
that question. We needed Dr. Salk and Dr. Sabin to get that 
polio vaccine. I think we are going to have the same attitude 
towards NASA. We want you to do what you need to do, and we 
want to help you do that, so my question is can we educate the 
public so they are not frustrated about the scientific method? 
And the other thing, what can we do to help you?
    And thank you very much, and I will yield my time.
    Mr. Scolese. Well, thank you for that. I think if we could 
copy that speech we would go a long way towards addressing that 
question, and you bring up, you know--actually I only want to 
correct one thing that you said. We don't like to explode our 
things into space.
    But you bring up, you know, a very important point. We 
can't lose sight of the need for mission success or safety. We 
have to balance the success of the mission, the risk of the 
mission with the cost and the schedule. We can't focus on any 
one of those three or four items in order to be successful.
    And, yes, sir, we can do a better job of explaining that, 
and we will.
    Chairwoman Giffords. Thank you, Mr. Griffith.
    Mr. McCaul, please.
    Mr. McCaul. Thank you, Madam Chair, and let me congratulate 
you and the Ranking Member on your new positions and----
    Chairwoman Giffords. Thank you.

        GAO Recommendations for NASA's Future Human Space Flight

    Mr. McCaul.--we are very excited to have you.
    And you, Mr. Scolese, in your new position. I have two 
areas of questioning. The first I think one of the greatest--
and NASA has many missions, but I think one of--the one with 
the greatest vision is the one of going to the moon, Mars and 
beyond. There is a proverbs quote up here, ``Where there is no 
vision, the people perish.'' This is the greatest vision for 
mankind. It was Kennedy's vision.
    This has been estimated to cost about $100 billion over the 
next 13 years, and the Chair of this committee, Bart Gordon, 
took a look at GAO's recommendations and said that they were 
common sense and hopes that you at NASA will take a look at 
these recommendations and implement them. And hopes that you 
will take these seriously.
    I want to just get your initial--Ms. Chaplain, if you could 
just briefly summarize the recommendations as they pertain to 
that specific mission, and Mr. Scolese, how you intend to work 
with GAO on that.
    Ms. Chaplain. Our recommendations have been pretty 
consistent across a lot of projects. Basically we would like to 
see NASA pushing technology is one thing, and they have, each 
project has two to five years to do that and to learn about 
what they are doing and to set requirements. But when they get 
to the point where they are ready to commit to Congress and get 
authorized to move ahead as a formal program, that is where we 
would like to see requirements get settled and technology to be 
understood.
    If that happens, you have a very solid foundation for 
moving forward. So I don't want there to be a misunderstanding 
that GAO is saying there is a bad thing to have cost growth and 
lots of things going on when you are trying to learn to do 
something that hadn't done, been done before. We really 
encourage that to happen, but there is a time and place for 
that to happen. Once a program is ready to move forward, you do 
need stability to make it executable.
    And I think, you know, across the board NASA has 
consistently agreed with these recommendations. It has built it 
into its policy. It is reflected in a lot of the actions it is 
undertaking right now. So the thing to do is just to sustain 
the attention to these improvement efforts and to adhere to 
these good practices that are reflected in their policies going 
forward, especially with these big ticket programs that are 
ready to enter these implementation phases, including areas of 
Orion, James Webb Telescope.
    Mr. McCaul. Okay. Mr. Scolese.
    Mr. Scolese. I think Cristina said it right. We are--we 
have taken those seriously. We are working on all of those. I 
would like to add one of the things that we are doing to make 
sure that we are, in fact, implementing that guidance or 
something that resembles that guidance, because we do conduct 
our independent reviews at each milestone to make sure that, in 
fact, we are doing what we said we were going to do, and we 
have independent people looking at it. And then we monitor that 
monthly to go off and make sure that all the pieces are still 
coming together.
    One of the challenges that we have is the stability, and it 
is getting to the point where we can make that commitment. As 
you can imagine as you are maturing your requirements and 
maturing your technologies, there is a lot of people that want 
you to go faster and commit sooner. We can't take forever. We 
recognize that, but we need to be able to take the time to get 
that done, and the support that we can get to allow us to do 
that would be greatly appreciated.
    And it typically takes, you know, sometimes two, three, and 
in some cases as in the JWST, James Webb Space Telescope, took 
us almost a decade to get to a point where we felt we had 
technologies mature enough to proceed, which we really just 
decided on this year.
    Mr. McCaul. Now, I just want to let you know, we in the 
Congress support you in that effort and stay in close 
communication with us.

                Cost Overruns in Climate Change Missions

    Lastly on--in the stimulus bill there was money 
appropriated for the purpose of climate change. Can you, Ms. 
Chaplain, speak to prior programs related to this, that 
particular mission and some of the cost overruns? Specifically 
I know the Glory Program had some cost overruns.
    Ms. Chaplain. Yeah. Several of the climate and 
environmental programs have had cost overruns. A lot of them 
have been attributable to contractor performance and not really 
recognizing that the contractor may not have had the expertise 
to develop some of the key sensors for satellites. What we 
would like to see is more attention paid up-front into what 
contractors can do and cannot do and to make sure you have a 
plan for dealing with any risks that you see in the contractor.
    We also like to see with these programs more stability with 
requirements once they start, and we also would like to see not 
too many expectations placed on any one program. The NPOESS 
Program, for example, has more than ten sensors on it, makes it 
very difficult to design and execute that program with so many 
sensors and so many problems associated with each one of them 
and the integration process and so forth.
    So there is a lot of risks associated with these kinds of 
satellites but a lot that can be done to mitigate those risks.
    Mr. McCaul. Thank you, Madam Chair.
    Chairwoman Giffords. Thank you, Mr. McCaul.
    Another terrific asset we have on this subcommittee is Ms. 
Edwards, who used to work at Goddard and now represents the 
interests of Goddard, so, please, five minutes.
    Ms. Edwards. Thank you, Madam Chairwoman, and thank you to 
the panel.

          Contractor Performance Contributes to Cost Overruns

    I just want to focus this morning on contractor performance 
and the ways in which contractor performance contributes to 
overrunning cost estimates, and so Mr. Scolese, I think you 
came to Goddard just as I was departing Goddard, and Ms. 
Chaplain, both of you in your testimony talked about that. And 
so I am wondering what there is about contractor performance 
and whether there are positive incentives and not just negative 
incentives to increase performance to contract and to avoid 
overruns, and then secondly, I wonder if you could speak to the 
internal capacity at NASA that actually might help us better 
oversee some of the highly-technical and scientific projects in 
which NASA is engaged.
    Mr. Scolese. Yes. In regard to the contracts and what our 
mechanisms are, we do have positive feedback mechanisms. I 
mentioned earlier that many of our contracts have award fee 
provisions, and that allows us to reward performance 
periodically all the way up until in many cases the end of the 
planned mission life. So even if--after it is launched, we can 
still go off and say, if you performed well, you can get, you 
know, a fee on that performance.
    So we have the positive incentives to go off and do that, 
and also it can serve as a negative incentive by getting lower 
scores. So we do try and find ways to motivate and reward, you 
know, performance. We do the same with our civil servants as 
you know, although it is not quite the same as that, with 
awards and metals and monetary awards when that is appropriate.
    So I think we have, you know, many of the incentives in 
place, and of course, one of the biggest incentives for both 
the contractor and the team that is doing it is to see the 
successful performance of the mission. You know, oftentimes, 
you know, it is answering a scientific question that hasn't 
been asked before, and the results can yield as they did with 
COBE with a Nobel Prize. So there is some great rewards that 
come from the scientific discovers that can be made.

        NASA's Civil Servant Capacity to Provide Technological 
                               Expertise

    Ms. Edwards. But does--what I wonder, though, is internally 
within the civil servant component of NASA, of the Agency, do 
you have the capacity to provide the kind of technological 
expertise that is needed over these projects to ask the right 
questions of contractors over the lifespan of a project to 
ensure that we are, you know, tamping down on costs where 
necessary and estimating what the overall cost of the project 
will be?
    Mr. Scolese. Yes. I do believe that we have that. We have, 
you know, a very good workforce. What we haven't always given 
them is the tools and the support that they needed. We have, as 
you well know, some of the brightest people around that are 
working these activities.
    What we haven't always done is the things that I tried to 
outline here is give them a forum to speak up when they have 
issues, to give them the opportunity to speak with senior 
management who could do things, whether it is adding additional 
resources, additional people if the project needs additional 
people, additional technical expertise if we need additional 
technical expertise. We can reach into the Agency at other 
centers. That is often been a problem where centers don't know 
what each other's capabilities are. We have broken down those 
barriers with organizations like the NASA engineering and 
safety center so that we can go off and share resources amongst 
the centers.
    We also have the ability to go outside of the Agency to 
organizations like Aerospace Corporation and others. So we made 
that more available to people.
    In addition, by having, as I mentioned, and all the centers 
now do this, having monthly reviews or periodic reviews where 
you can bring up issues to senior management, we can take 
actions. They can be as simple as picking up the phone and 
calling a contractor and saying, you are not doing what we 
think you should be doing, and here is why and having the 
details. Or it could be, you know, more invasive than that.

                      NASA Life Cycle Performance

    Ms. Edwards. Let me just interrupt you because I have 
probably just enough time for this last thing, and that is 
whether you are able to--are there interim assessments, and can 
you look at international agencies in addition to the 
Department of Defense to learn about where you can better 
assess over the life of the project whether you are meeting 
cost estimates. And so looking at like agencies or over the 
lifetime of a project, and you can think about this later. I am 
looking, for example, at Landsat, and Landsat over a period of 
time, are there things that we have learned about those 
assessments that actually might help us for other projects.
    Mr. Scolese. Yes. In fact, we are doing that. We have been 
doing that, and we are going to embark on a broader study to go 
off and look at performance of other agencies and other 
activities. I am not sure that we could get the information on 
the international ones, but we will certainly go off and look 
and see if that is doable.
    Ms. Edwards. Thank you.
    Chairwoman Giffords. Thank you, Ms. Edwards.
    Mr. Grayson.

         Firm-fixed Contracts Incentive to Avoid Cost Overruns

    Mr. Grayson. Thank you, Madam Chair. Mr. Scolese, what 
percentage of NASA's contract dollars are spent on firm-fixed 
priced contracts?
    Mr. Scolese. I will have to get you the exact details on 
that. I don't have that off the top of my head, but I can tell 
you that where we can do firm-fixed price we do, and I can give 
you a couple of examples here and then for the record I will 
provide you with more details.
    On the tracking and data relay satellite system, 
communication satellites, those were fixed-price contracts. For 
portions of the GOES satellite that we do for NOAA, those were 
fixed price. For many of our small Explorer missions the 
spacecraft bus is often a fixed-price bus.
    What percentage of our contracts beyond--in precise terms I 
can't give you off the top of my head and will get that to you.
    Mr. Grayson. Well, for firm-fixed price contracts, the 
contractor pays 100 percent of the cost of overruns. Correct?
    Mr. Scolese. If it is caused by them. That is correct. Yes.
    Mr. Grayson. And for cost reimbursement contracts, the 
contractor is not legally obliged to pay any part of the cost 
overrun. Is that correct?
    Mr. Scolese. That is correct.
    Mr. Grayson. So is it fair to say that if NASA did more 
firm-fixed price contracts we would be giving the contractor an 
incentive, a real incentive to avoid cost overruns?
    Mr. Scolese. We look very carefully at what type of a 
contract we use to try and balance the risk that is going to be 
in there. Typically we do firm-fixed price contracts when we 
understand the requirements so that a contractor will, in fact, 
bid on it and know that they have a chance of being successful.
    We use cost reimbursement type contracts when there is some 
uncertainty in the, in either the requirements that we have or 
in the ability to perform. So we do a very careful risk benefit 
relationship. Where possible and as often as possible we try 
and use fixed-price contracts. But that isn't the only remedy 
that we can use.
    Mr. Grayson. Mr. Scolese, it seems that have here two 
extremes. We have one condition where the contractor pays all 
the cost of overruns. The other condition where the contractor 
is legally obliged to pay none of the cost of overruns. 
Wouldn't it be useful to have something in-between?
    Mr. Scolese. Well, in fact, we actually do try and work 
that way, and I just want to, you know, on the firm-fixed 
price, if we change something, we pay. If they can't meet the 
original specification, they pay. And, yes, I mean, we actually 
try and work those cost reimbursable contracts where we have 
negotiations, they will not be reimbursed for all the costs if 
we feel it was their fault to cause it. We can't guarantee that 
at the beginning of the contract or we can't guarantee that at 
the beginning of the negotiations. But we do work hard to try 
and make that stick and to assign responsibility where 
responsibility lies.
    Mr. Grayson. What percentage of NASA's contracts, if any, 
are awarded under invitations for bids, sealed bids, rather 
than through competitive proposals?
    Mr. Scolese. I don't have the answer to that. Most of our 
stuff is done competitively, but there may be some 
institutional activities that are done by sealed bids. I can't 
answer that off the top of my head.
    Mr. Grayson. Now, when there are competitive proposals 
which I am sure you will agree is the predominant form of 
NASA's contracting, when there are competitive proposal's, 
there is a cost technical tradeoff in those proposals. Correct?
    Mr. Scolese. Yes.
    Mr. Grayson. All right. Now, you can correct me if I am 
wrong, but I believe that there is nothing in the statutes that 
indicates to NASA what that tradeoff should be. Is that 
correct?
    Mr. Scolese. What the tradeoff between the cost and the 
technical should be?
    Mr. Grayson. Right. How much of an emphasis should be put 
on cost versus technical?
    Mr. Scolese. No. We--there isn't a statute for that. We 
determine that before the contract is released for bid.
    Mr. Grayson. Right. But the Agency determines that in its 
sole discretion without any guidance from us. Correct?
    Mr. Scolese. Yes. That is correct.
    Mr. Grayson. Now, if we were trying to save money, it seems 
that one way we could try to do that would be to emphasize cost 
more in this cost technical tradeoff. Is that a fair statement?
    Mr. Scolese. Well, we do emphasize cost. It is a fair 
statement, but we have to manage the cost and the schedule and 
the risk and the technical performance. All of those factors 
have to be considered. Yes.
    Mr. Grayson. Now, returning again to the tradeoff between 
having a firm-fixed price contract where the contractor bears 
the risk and the cost reimbursement contract where the 
contractor legally bears none of the risk, apart from the 
question of whether we should have something between those two, 
would you agree with me that now it is the Agency that makes 
that decision and not Congress?
    Mr. Scolese. Yes. I would agree with that.
    Mr. Grayson. All right. Now, again, if we were trying to 
avoid cost overruns, do you think it might be a good idea for 
us to give you some direction about when to use a cost 
reimbursement contract and when to have the contractor bear the 
risk of the cost overrun?
    Mr. Scolese. No, sir. I think, you know, those are very 
difficult decisions that have to be made as we are developing 
our requirements, and you know, when we have very stable 
requirements, firm-fixed price contracts are absolutely the 
right thing to do. And that determination, as was mentioned 
earlier, comes after we have done some definition of what it is 
that we want to achieve. And as I tried to mention earlier, we 
do, in fact, do that. Many of our spacecraft for the small 
Explorers as an example are, in fact, firm-fixed price because 
we understand our requirements very well. And we have good 
performance there.
    So I think it is going to be, it would be very difficult to 
look at each and every one of our missions to try and determine 
which should be firm-fixed price, which should be cost plus, 
and which portion of the mission should be firm-fixed price and 
which portion should be cost plus. Because in any given mission 
you will have different types of contracts for different 
components that are being delivered.
    Mr. Grayson. Thank you. I will look forward to receiving 
the information that you promised.
    Thank you, Madam Chairwoman.
    Chairwoman Giffords. Thank you, Mr. Grayson.
    Now we have Ms. Kosmas with us, also a Central Floridian, 
representing the Kennedy Space Center.

          More Cost Overruns Flexibility in Human Space Flight

    Ms. Kosmas. Thank you. I am honored to be here. Thank you, 
Madam Chair, and thank you to the panel for being here.
    I want to start by saying I am very excited to be 
representing the Kennedy Space Center and thrilled to have been 
a neighbor and watched the launches for 35 years in my time in 
Central Florida.
    I want also to echo the comments made by Parker Griffith, 
my colleague here, about how it is very important to me that we 
maintain the U.S. supremacy in man space exploration 
specifically and also the other areas in which we use space 
exploration, whether it is for national defense, whether it is 
for environment, whether it is for weather prediction, or 
whatever the use might be, but man space flight, of course, is 
of a high interest to me.
    So with reference to the comments made by Congressman 
Grayson and the cost analysis, would you say that you put a 
greater emphasis or that you allow a greater latitude or 
flexibility in terms of cost overruns when there is a 
significant safety factor involved, particularly with regard to 
human space flight?
    Mr. Scolese. Well, certainly that goes exactly to the risk 
discussion that we were just talking about. Clearly, if we have 
a mission, we err on the side of safety. There is no question 
about that. Clearly we can't allow it to go so far beyond what 
our ability to pay is, and we would have to take a step back 
and see if we are doing it the right way and ask that question. 
But clearly we are not going to scrimp when it comes to safety.
    Ms. Kosmas. Well, good. I mean, I assumed that to be the 
case, but I thought it needed to be pointed out perhaps----
    Mr. Scolese. Thank you.

     A Fixed Date to End the Space Shuttle Increase Cost Stability

    Ms. Kosmas.--that that is an area in which cost overruns 
maybe are--we have learned from experience that certain things 
require greater care in terms of the safety risk factor.
    Which brings me to the second part of my question which has 
to do with, in my district, of course, we are concerned about 
the end of the Shuttle launch and the beginning of the 
Constellation Program and the difficulties that that poses for 
us in maintaining a professional workforce in the area. And 
wanted to ask you with regard to the safety aspect of putting a 
hard and fast date on the end of the Shuttle Program, do you 
think that that is a wise thing to do?
    Mr. Scolese. I won't comment on the wise part, but I will 
say that clearly that having a fixed date adds to the, one of 
the things that we mentioned earlier, which is stability. We 
need that same workforce, portions of that same workforce, to 
go off and build the next system, design the next system, test 
the next system. So we need to move some of those talented 
people over to that activity. They are doing a wonderful job in 
flying the Shuttle, but we also need them in other areas as 
well. So we need to start transitioning some of those people 
over.
    We also need some of those facilities. As you know, you 
were down there just recently, and you saw the three lightening 
towers going up on 39B. We need to have those facilities 
available to begin launching the new systems.
    So from a stability standpoint, from a risk standpoint, 
having a fixed date to allow us to start transitioning those 
people and those resources is important to us, and I think 
2010, is a reasonable date for us to go off and do that by. We 
have, as you know, we have the hardware to get us there. We 
would have to build additional external tanks to go beyond 
that, but so I think 2010, is a reasonable date, and it 
provides us a reasonable risk posture to proceed.
    Ms. Kosmas. You think it does provide a reasonable risk 
posture----
    Mr. Scolese. Yes.
    Ms. Kosmas.--for you to proceed? Okay.
    Mr. Scolese. Yes.
    Ms. Kosmas. Given the flexibility shall we say and time for 
rocket launches to take off and that provides a flexibility in 
time, how much flexibility do you think would be reasonable 
past 2010?
    Mr. Scolese. You mean to fly the Shuttle or to----
    Ms. Kosmas. Yes. To fly the Shuttle.
    Mr. Scolese. We will have to get back to you on that. We 
have a report actually that is coming in on----
    Ms. Kosmas. Okay.
    Mr. Scolese.--the answer specifically that question.
    Ms. Kosmas. Okay. Thanks very much.

            How New Contracting Procedures Will Effect NASA

    Chairwoman Giffords. Thank you, Ms. Kosmas. That completes 
our first round of the questions. We know that votes will be 
called very shortly, but we are going to attempt to do a second 
round.
    So with that, Mr. Scolese, President Obama yesterday singed 
a Presidential memorandum to overhaul the way the government 
does its contracting. He indicated that your agency would be 
working with other agencies as well as OMB to come up with new 
procedures and practices.
    Can you tell us today anything about the impact this 
overhaul will have on NASA contracting?
    Mr. Scolese. Actually, we took a quick look at it, and we 
think that we are largely in line with the provisions in there, 
the utilization of fixed-price contracts as we discussed 
earlier. We do that where that is appropriate on space hardware 
as well as, you know, institutional items. So we are certainly 
going to look at it and see where we can improve.
    The other provision to use competition we regularly use 
competition and require competitive activities. We require a 
justification when we do not use competitive bids, so we do 
that where there is one supplier, for instance, as is the case 
for large solids, the first stage of the Ares vehicle. There is 
only one supplier of large solids, so we had to do fixed price 
or, I mean, sole source there.
    But predominantly we use competition, and then as far as 
the A-176, we have had some experience with that but not as 
much, and that requires a little bit more of our looking at it. 
But it is along the lines of the way we have proceeded to 
maintain inherently governmental responsibilities within the 
government.

           NASA Addresses Launch Conflicts and Increased Cost

    Chairwoman Giffords. Thank you, and our subcommittee will 
likely have you come back and report on how things are going.
    I also understand that NASA is very concerned as is 
indicated by your testimony earlier about the crowded manifest 
for launch vehicles in 2011, and the fact that the Mars Science 
Labs launch window that year will be very tight. Evidently NASA 
needs to contend not only with the Juno Mission to Jupiter but 
also with DOD launches as well.
    However, this is not the first time that this has happened. 
As you know, launch delays increase project costs. We have 
heard a bit about that, but what is NASA doing to mitigate 
launch conflicts?
    And also, Mr. Pulliam, based on the Aerospace Corporation's 
work at DOD, what can NASA do better to mitigate launch 
conflicts, and is this problem going to get better, or is it 
going to get worse?
    Mr. Scolese. Well, what we are doing to resolve the problem 
is, of course, we are working with United Launch Alliance, we 
are working with our colleagues in the Department of Defense 
and Department of Commerce to try and make sure that we have a 
manifest that is doable, and we--and that we can have 
confidence in. We are behind right now. There is no question 
about it. The launch manifest as you mentioned is backed up, 
causing some missions to slip.
    What the exact way to fix that problem is going to take 
some time. We are also looking within NASA, I am certain you 
have heard about the COTS activities and about our launch 
vehicle on-ramps for the NASA launch services, we are also 
looking at other suppliers as they come online to fly our 
vehicles. And where partnerships make sense we use 
international launches on a not-for-fee basis. That is their 
contribution to us.
    So we are looking at all three of those things; working 
with our colleagues in Defense Department and ULA to improve 
the manifest, improve the capability to launch more frequently, 
we are encouraging the development of other launch vehicles, 
particularly the medium-class launch vehicle that will end when 
the Delta II stops producing, which carries most of our science 
payloads, and we also, where it makes sense and where it is 
beneficial to the United States, are looking at international 
launches as well.
    Mr. Pulliam. I would agree with the statements Chris has 
made. You know, we see the clogging of the manifest just from 
inside the DOD perspective even before you integrate the NASA 
launches into that. So it is clear, and that clogging of the 
manifest comes and goes as missions encounter these 
difficulties we have talked about and perhaps move into a new 
area of the launch manifest where maybe it wasn't so bad 
before. Now it is worse than you thought.
    So it is an ever-changing game. So even though we see a 
clogged manifest in the future, that will change some as we 
approach it as well.
    I think the answers to it are as they are with most of 
these other problems. It requires a very close interaction 
between NASA and the DOD on which instruments need to go when. 
If it is a planetary mission, then surely we need to hold onto 
those windows, lest we lose two years. If the DOD has a mission 
that is more important to national security, then perhaps that 
takes the precedence, and I think we do that now. So that 
coordination and cooperation needs to continue.
    And then finally I would say we need to look at what is on 
the critical path as we try to use our facilities in the most 
efficient way. Is it just the physical turn of the pad? Does it 
have something to do with range capabilities? You know, what is 
it that could be viewed from a multi-user perspective to see if 
we want to operate at the maximum efficiency? Are we doing that 
now? Are there upgrades that are required? Is there 
infrastructure that we don't know have, and to look far enough 
in advance to know what that is so that we don't wind up just 
accepting whatever capacity there happens to be, perhaps based 
on a time when the demand wasn't quite so high.
    Chairwoman Giffords. Thank you.
    Mr. Olson.
    Mr. Olson. Thank you, Madam Chairwoman, and Mr. Scolese, 
give you a little break.
    Mr. Pulliam----
    Mr. Pulliam. My turn.
    Mr. Scolese. Thank you.
    Mr. Olson. Spread the love a little bit so to speak.
    Mr. Pulliam. Okay.
    Mr. Olson. Just wanted to talk about NASA and in comparison 
to the Department of Defense, and in your testimony you 
compared NASA with the National Security Space Community. You 
said NASA was far less, received requests far less overall 
support from the Aerospace Corporation but proportionately more 
programmatic in budgeting support.
    Mr. Pulliam. Yes.

    Comparison Between NASA and DOD on Cost Estimating, Budgeting, 
                        and Programmatic Control

    Mr. Olson. And so between NASA and the National Security 
Space Community, who in your view does a better job with cost 
estimating, budgeting, maintaining programmatic control over 
their programs? What are the differences, and why do you think 
those differences exist?
    Mr. Pulliam. Thank you. It is true that the predominant 
customer for the Aerospace Corporation is the Department of 
Defense Space Program. That is why we were created back in 
1960. NASA is the company's third largest customer right behind 
DOD and NRO. So we are privileged to work with them, and we did 
want to make the point that--and we think it accrues to NASA's 
credit while the Air Force typically uses Aerospace more for 
the in-line, hardcore technical and scientific kinds of things 
that we are expected to do, NASA has that capacity more capably 
in its organic workforce, so they turn to Aerospace as we would 
expect federal agencies to do to see what we have that serves 
them in the accomplishment of their national mission.
    And that has turned out to be this body of work and cost 
estimating, looking at the parameters that go into that, and we 
think NASA in partnership with Aerospace is doing some things 
that other agencies frankly aren't doing. And it is going to 
pay off, and we are going to see the dividends of that.
    Which organization is doing best? We can cite examples 
across both agencies where there have been spectacular failures 
of managing costs and schedule. We did find that the causes and 
fundamental elements seem to be about the same. The Department 
of Defense uses other very capable contractors to help them 
with their cost estimating, so we are not as deeply involved 
with them as we are with NASA in trying to figure out the 
doctrine and the managerial techniques.
    But I think it sounds a bit like a cop-out, but I think the 
agencies have essentially the same problems, and they are 
running at similar kind of overhead rates. NASA can say, yeah, 
but we had this mission that really pushes the science in a way 
that no other mission has. DOD can respond by saying, yeah, but 
we have this multi-military user community that requires things 
to change in technology and schedule and growth.
    So it just needs to be managed uniquely from each agency's 
perspective, but in full appreciation of best practices that 
might be out there, and if I could just take one more moment, 
there is a lot of really fine work being done that was begun at 
the GAO and has migrated into NASA and to the Aerospace 
Corporation in looking at major acquisition milestones called 
key decision points.
    And looking in advance of those at the kind of knowledge 
one needs to have in order to be ready for those milestones, 
and those milestones play directly into cost and schedule. I 
know NASA and the Aerospace Corporation and GAO are all very 
interested in continuing to define what it takes to be 
successful at every part of the program, and that is going to 
pay off as well.

                   MSL Dramatic Cost Estimate Change

    Mr. Olson. Thank you very much for that answer.
    Mr. Scolese, I couldn't let you get away clean free. I just 
want to talk very briefly about the Mars Science Lab, and it is 
my understanding that the original cost estimate for the Mars 
Science Lab was in the $600 million range. And that NASA's 
initial confirmation estimate was closer to $1.6 billion. That 
first estimate did not come from NASA as I understand it. And 
you have, NASA has recently announced a two-year delay in the 
MSL with a cost increase of $400 million.
    I just wanted you to elaborate on how those numbers were 
obtained and what is being done to eliminate unrealistic 
estimates.
    Mr. Scolese. Yes. That first number that you mentioned, the 
roughly $600 million, came out of what is called a decadal 
survey that we do with the National Academies where they look 
at our objectives in the future, and they try and prioritize 
them and say these are the science goals that you really should 
think about doing. And we work with them to try and accomplish 
those goals.
    The Mars Science Lab was one of those, and at that time it 
was estimated at $600 million. When we went off and did our 
studies, as was mentioned here, we did the work to mature the 
technologies to get a better understanding of it, we estimated 
the cost to be $1.6 billion. Clearly, we still underestimated 
the complexity of it, and we should and could and can and will 
do better in the future.
    We are also working with the National Academy to help them 
improve their earlier cost estimates that you see there, 
because it often gets out, you know, one way or the other that, 
you know, somebody thought this was going to cost whatever was 
said at the earliest phase. So we are working with them to help 
them obtain the tools and to get resources, and the Academy is 
doing it as well. So we are doing it together, and we are doing 
it willingly so that we can go off and have a better 
understanding of what those missions would cost so they can 
better establish priorities.
    Mr. Olson. Thanks for that answer, and Madam Chairwoman, I 
yield back my time.
    Chairwoman Giffords. Thank you. We have with us Mr. 
Rohrabacher, and of course, votes have been called, but we are 
going to have some minutes to hear.

      Bad Judgment and Irrational Optimism Affects Cost Estimates

    Mr. Rohrabacher. Thank you very much. I apologize for being 
late to the hearing. As usual we had two important hearings at 
exactly the same time.
    I would like to ask a general question just--when we take a 
look at this Mars Science Lab and we take a look at, I guess it 
is NPOESS, I guess.
    Mr. Scolese. NPOESS.
    Mr. Rohrabacher. NPOESS. I have never been able to 
pronounce that right, and I have been looking at this for 
years, but with that said, you know which project I am talking 
about.
    Mr. Scolese. Yes.
    Mr. Rohrabacher. When we look at that and we look at this 
whole Mars issue today, are we talking about basically--and 
these other problems, are we basically talking about 
intentional wrongdoing in terms of lying about a bid. You know, 
when you are offering, intentionally offering a bid that you 
cannot handle, that is a lie, and that is an intentional 
wrongdoing. Then you have another option. Bad judgment, you 
know, unintentionally doing something that ends up causing harm 
like miscalculating what you really can accomplish for a 
certain degree. Or are we talking about systematic flaws?
    Because we have had consistently over the years examples 
after-example after example after example of overruns and of 
people not really being able to fulfill the obligation they 
have taken on when a contract has been issued. So what are you 
talking about? Are we talking about a systematic flaw? Should 
we be doing something else? Are we talking about just bad 
judgments that were made but with no ill, nothing, you know, 
wrong. People do make bad judgments at times. Or are we talking 
about intentional wrongdoing?
    Mr. Scolese. Well, let me start off by saying NPOESS isn't 
a NASA project, so I can't comment, you know, in any degree of 
detail on that, but your question is broader than that. And I 
don't believe that intentional wrongdoing is there ever. I 
suspect it probably happens occasionally, but that is not what 
we see. It is more in your second one, which is, you know, bad 
judgment, although I would like to call it optimism.
    Mr. Rohrabacher. All right.
    Mr. Scolese. The people that are proposing these missions 
that want to do them are extremely enthusiastic about it, and 
they are optimistic about the technologies that are out there, 
they are optimistic about their ability to do the job.
    Mr. Rohrabacher. Irrational optimism can be very costly.
    Mr. Scolese. Yes, it can. Yes, sir. And that is why get to 
this, to the third part, which is really the answer to your 
question, which is it is systematic. And that is what we were 
trying to talk about here, what we are doing, and that is what 
the GAO has pointed out that we need to do.
    It is not one thing that is going to improve our 
performance. It is many things that are going to improve our 
performance. We have to train our people, and we do, and we are 
improving our training process so that they will be able to 
recognize their optimism as well as others optimism. We have to 
monitor the performance of what is going on so that we can see 
if optimism is driving the estimate or if, in fact, poor 
performance is causing us to have some overruns. We have to put 
those tools in place so that we can catch them early enough so 
that they don't become, you know, surprises for sure but they 
don't become large cost impacts because we can then apply 
additional resources without maybe having to increase the 
budget or change the schedule.
    Mr. Rohrabacher. So training, monitoring oversight, but let 
me add one other word here that I over the years that I just 
never hear and that is accountability, and if someone has a bad 
judgment or if someone is in, especially if someone is 
intentionally doing something wrong, I have not found the 
ability of people to be fired or to be charges being brought 
against people. And I just, especially if they are within the 
system and making a bad judgment as compared to a contractor 
from the outside. Shouldn't, isn't there a problem with 
accountability here?
    How many people do you know have been fired from their job 
at NASA?
    Mr. Scolese. If you mean by fired sent away from NASA----
    Mr. Rohrabacher. Yeah.
    Mr. Scolese.--you mean for wrongdoing, you know, they 
violated the law, I wouldn't have the names but I----
    Mr. Rohrabacher. Well, I wouldn't say violated the law. If 
someone just has very bad judgment, I remember with Hubble 
Telescope there was a bad decision made somewhere there where 
they sent it up and the mirrors weren't exactly right, and it 
was, it cost the taxpayers a lot of money.
    We should--I know my time is up, so I will just say that we 
need to be holding people more accountable for bad judgments, 
irrational optimism. I know you are going to go to work on the 
systematic problems as well.
    Mr. Scolese. And we do the other, we do take into account 
those types of things. We do hold people accountable, and we 
have, in fact, replaced project managers, project personnel, 
other personnel if they are not performing. But we also have to 
recognize what caused them not to perform, and if it was 
certainly their fault, we do remove them. To say that we fire 
them, I can't necessarily say that, because oftentimes these 
people have performed spectacularly in other areas, so we will 
remove them from their current job and move them to another.
    Mr. Rohrabacher. Okay. Thank you very much.
    Chairwoman Giffords. Thank you, Mr. Rohrabacher. Thank you, 
Mr. Scolese. Just in closing, I know that a lot of the work 
that NASA has done in the budgetary areas were not reflected in 
this GAO report. So when do you expect the Congress to be able 
to see the result of the work that you have done?
    Mr. Scolese. Well, as the GAO report said, you know, these 
tools have been put in place in the last couple of years. Our 
missions typically last or our mission cycle is typically three 
to five years, so you should be seeing some of the results 
soon. I think you will see our estimates, you know, better, but 
probably you will see the real benefit of what we are doing in 
the next couple of years, the next one to two years as we start 
delivering on those missions that we started this process on 
about two years ago.
    Chairwoman Giffords. Thank you. Before we bring the hearing 
to a close, I want to thank the witnesses for testifying before 
the Subcommittee today. I want to thank my Ranking Member as 
well for our first successful hearing and also as you saw there 
is a real passion for space in this subcommittee, and a lot of 
Members, new Members to the Congress represent NASA's 
interests, the American people's interests. So I look forward 
to a fruitful, productive next couple of years.
    The record will remain open for two weeks for additional 
statements from Members and for answers to any of the follow-up 
questions the Subcommittee may ask of our witnesses. The 
witnesses are excused, and the hearing is now adjourned. Thank 
you.
    [Whereupon, at 11:37 a.m., the Subcommittee was adjourned.]
                               Appendix:

                              ----------                              


                   Answers to Post-Hearing Questions




                   Answers to Post-Hearing Questions
Responses by Christopher J. Scolese, Acting Administrator, National 
        Aeronautics and Space Administration (NASA)

Questions submitted by Chairwoman Gabrielle Giffords

Q1.  NASA has noted on several occasions that problems with contractor 
performance have led to cost overruns and schedule delays. What 
corrective actions is NASA undertaking to ensure better contractor 
performance and improve its contractor oversight for major acquisition 
projects?

A1. NASA has instituted a multi-pronged approach to strengthen 
acquisition planning and execution, increase management oversight, and 
assess and address root causes for problems with contractor 
performance. NASA has re-visited forums and metrics used to monitor, 
track and report contract performance. NASA also is taking a closer 
look at how contractor performance contributes to program and project 
performance success.
    As outlined in NASA Policy Directive 1000.5, ``Policy for NASA 
Acquisition'' (issued January 2009), and NASA Procedural Requirements 
7120.5D, ``NASA Space Flight Program and Project Management 
Requirements,'' the Agency has instituted a strategic acquisition 
planning and authorization process designed to strengthen program and 
project formulation. This process begins with an Acquisition Strategy 
Planning (ASP) Meeting, which approves a new or substantially changed 
program or project triggered by Agency requirements or legislative 
direction. The ASP is followed by an Acquisition Strategy Meeting (ASM) 
to ensure program planning is in place and validates make/buy rationale 
and any partnership decisions. The process culminates in a Procurement 
Strategy Meeting (PSM) to approve the procurement strategy for 
individual procurements. PSMs are conducted in accordance with Federal 
Acquisition Regulations for each contract within a Project and focus on 
the procurement process. These events are part of the normal program 
and project formulation and implementation activities and lay a strong 
foundation from which contracts will be managed.
    Further, the Agency has instituted a forum to monitor the Agency's 
largest and most complex contracts at a Senior Management Review of 
program and project performance. On a monthly basis, all large 
contracts from NASA's major projects are tracked for current value 
compared with original value, award fee status, current and upcoming 
significant procurement actions (including undefinitized contract 
actions, restructures, etc.). Planned procurements that are within a 
six-month horizon are also highlighted and discussed. Each NASA Mission 
Directorate program and project is reviewed quarterly and their 
associated contracts are addressed in more detail. This forum and the 
data reviewed allow the Agency to focus on contract management and 
proactively address issues, which should lead to project cost and 
schedule growth.

Q2.  You state in your testimony that NASA has initiated a number of 
changes that address common issues such as optimism in cost estimates 
and schedules and unrealistic assumptions on technology maturity, just 
to name a few. How do you plan to ensure that current and future major 
acquisition projects do not exceed the Congressionally-mandated cost 
and schedule thresholds?

A2. NASA will apply improved assessment and reporting processes to the 
current missions, but also recognizes that a number of these baseline 
commitments were made prior to the introduction of these changes, and 
we anticipate that it will be some time prior to realizing the full 
results of the changes. If cost and schedule growth do occur in the 
interim, NASA believes that our improved reporting processes will 
better enable timely notification on the issues. NASA will continue to 
take the steps necessary to rebalance the Agency portfolio to 
accommodate those problems. It is anticipated that these initiatives 
will continue to improve cost and schedule performance in the future.

Q3.  You highlight in your testimony the use of Standing Review Boards 
to provide an unbiased assessment of project performance at key 
decision points. However, the independence of some of the discipline 
experts on past boards has been questioned by the Agency's own 
Inspector General. What is NASA doing to ensure that organizational 
conflicts of interest no longer occur with Standing Review Board 
members?

A3. NASA is updating the Agency policy ``NASA Space Flight Management 
Requirements'' (NPR 7120.5D) with a NASA Interim Directive (NID) that 
will establish the Standing Review Board handbook as the source of 
guidance for establishing Standing Review Boards and assessing members 
for potential organizational or personal conflict of interest. The NASA 
Office of the General Counsel has reviewed the policy and procedures in 
the NPR 7120.5D, and the Standing Review Board handbook, which guides 
the establishment of Standing Review Boards to ensure that they will be 
established in a manner consistent with federal law. The handbook 
includes procedures that ensure identification of Standing Review Board 
members who have conflicts of interest.
    The handbook includes NASA's procedures to determine whether 
nominees are employed by companies performing work for the program or 
project under review. Ethics officials will verify Standing Review 
Board participant independence. For those individuals determined to 
have conflicts of interest, and whose value to the Board's work is 
viewed by the convening authorities as critical, a waiver will be 
submitted to ethics officials to request consideration to allow that 
individual to participate on that Standing Review Board. As described 
in the handbook, eligibility of Standing Review Board members, with 
regard to their independence, will be reviewed on an annual basis.
    The NASA Inspector General has accepted these updates as responsive 
to their concerns.

Q4.  Why is NASA's cost cap for the Joint Dark Energy Mission (JDEM) 
lower than the National Research Council panel that reviewed the 
mission thinks is realistic? Please provide a specific rationale for 
the lower cost cap.

A4. NASA agrees with the National Research Council (NRC) that JDEM is a 
>$1 billion-class mission. However, NASA does not expect to pay the 
entire cost of the mission. NASA has been working in partnership with 
the Department of Energy and is now engaged in discussions with the 
European Space Agency (ESA) with the goal of achieving an exciting and 
productive dark energy mission with our combined resources. NASA and 
its partners have much work to do before establishing a firm cost 
commitment at a formal mission confirmation review.

Q5.  Pursuant to the NASA Authorization Act of 2005 (P.L. 109-155), on 
May 1, NASA notified the Committee that the MSL project would exceed 
the Baseline development cost by 15 percent or more. The Act further 
requires NASA to notify the Committee after NASA makes a determination 
that the development cost has exceeded the Baseline by 30 percent. Has 
MSL reached the 30 percent cost growth threshold? If so, when was this 
determination made? If not, do you anticipate that the project will 
reach the 30 percent cost growth threshold and if so, when?

A5. In its letter of December 4, 2008, NASA informed the Committee of 
its decision to defer launch of MSL to the 2011 launch window. NASA 
noted MSL would require additional funding of approximately $400 
million to support this schedule slip. While the letter does not 
describe this prospective budget requirement in percentage terms, this 
communication represents a determination that MSL will exceed its 
baseline budget requirement by more than 30 percent by virtue of the 
additional estimated funding required to accommodate the schedule slip 
to 2011. NASA is preparing a formal detailed report pursuant to the 
requirements of the NASA Authorization Act of 2005 and plans to submit 
this report immediately following release of the FY 2010 budget 
request. It should be noted that NASA will continue to review the MSL 
program consistent with it program management processes and further 
refinements can be expected as the review process progresses.

Q6.  The Aerospace Industries Association recently advocated that DOD 
budget for an 80 percent probability of success. From NASA's 
perspective, what are the pros and cons of using a higher confidence 
level than the 70 percent probability currently used by NASA?

A6. Prior to making the decision to adopt a 70 percent confidence 
level, NASA evaluated the use of higher confidence levels and concluded 
that the use of the 70 percent level allowed the optimal balancing of 
risk across the Agency budget portfolio.

Q7.  Should there be a reasonable level of reserves included in the 
estimated cost of a program, and if so, what would you define as 
``reasonable''? What are the pros and cons of a higher level of 
reserves?

A7. The Agency has changed its approach to project estimating and 
reserves. The use of probabilistic estimation represents a major shift 
in the methods utilized to size and manage project and program 
resources. This new paradigm requires that the Agency provide resources 
and schedule sufficient to assure a specified probability of success 
for the project and program. The probabilistic estimates are to be 
sized to provide for all requirements, both anticipated and 
unanticipated.

Q8.  An analysis by the Aerospace Corporation concluded that cost 
growth problems would continue until project managers were given 
greater control. What are the pros and cons of giving project managers 
greater control?

A8. The study suggested that the project manager needed to have full 
authority to control costs (e.g., decline to allow the Center Director 
to assign additional human resources to the project). The roles and 
responsibilities of the Center Director and project manager differ. The 
project manager's role is to assure success of his project within 
budget and schedule. This role might lead the project manager to make 
decisions that were good for the project at the expense of maintaining 
Agency capability. The Center Director, on the other hand, is 
responsible for maintaining the skills necessary to support all current 
and projected projects that are to be executed by the Center.
    This split and balancing of roles and responsibilities is 
purposeful and important. These sometimes conflicting roles were 
specifically created to encourage a balancing of short-term (project) 
needs with longer-term institutional needs. Finally, as a check and 
balance, decisions on allocating human capital are approved by NASA 
leadership at the Mission Directorate, Associate Administrator, and 
Administrator levels.

Q9.  In his questioning during the hearing, Rep. Grayson raised the 
issue of the contrasting approaches of having vendors be responsible 
for overruns in fixed price contracts and not being obliged to pay for 
cost growth in cost reimbursement contracts. In your response to Mr. 
Grayson, you acknowledged that NASA uses fixed price contracts when the 
Agency understands the requirements ``so that a contractor will, in 
fact, bid on it and know that they have a chance of being successful.'' 
In contrast, you said that NASA uses cost reimbursement type contracts 
when there is some uncertainty in the requirements. Since NASA's 
program management process fully recognizes that a project usually 
starts with uncertainty but then matures through success design 
reviews, could NASA use a cost reimbursement contract for the work up 
conducted through the Preliminary Design Review (PDR) or Critical 
Design Review (CDR) and then subsequently utilize a fixed price 
contract? Has such a hybrid approach ever been taken, and if so, what 
were the results?

A9. NASA's goal is to ensure that the cost risk for each requirement is 
properly allocated between NASA and industry. Cost-reimbursement award-
fee contracts are generally most appropriate for use on NASA's high-
risk and complex science missions and Design, Development, Test and 
Evaluation (DDT&E) contracts. During the development phase of a 
project, NASA should take on the cost risk because of the difficulty of 
developing firm estimates for the cost of the work to be performed. Use 
of fixed price type contracts under these circumstances would 
invariably result in contractors proposing significantly higher prices 
to compensate for the high-risk. In order to mitigate the Government's 
risk under these cost type contracts, NASA utilizes incentive 
arrangements (i.e., award fee incentives, performance fee incentives, 
etc.) in conjunction with our cost-reimbursement contracts. The 
incentive tee arrangements contain clear and unambiguous evaluation 
criteria that are linked to cost, schedule, and technical performance 
requirements of the contract that provide contractors with distinct 
incentive to control costs and develop a high quality product. These 
practices are in line with the White House memorandum on Government 
Contracting, dated March 4, 2009, and demonstrate NASA's commitment to 
contract oversight and risk mitigation.
    NASA program and project offices work together to develop 
requirements and workload projections with the goal of moving toward 
fixed price contracts as soon as possible. As products and services 
mature, NASA considers a movement toward fixed price contracts for 
production and operations. For fixed price contracts, payment amounts 
can be based on performance as measured by standards and metrics. NASA 
will have examples of such an approach once a system is fully developed 
and going into maximum production. At this time, the hybrid approach 
involving costs-reimbursement and fixed price contracts has not yet 
been tested.

Question submitted by Representative Ralph M. Hall

Q1.  Given the long experience of schedule and cost growth in complex 
missions, why not simply impose higher cost and schedule reserves at 
the project's outset, instead of trying to operate with a too-small 
reserve when the project encounters difficulties?

A1. NASA has undertaken a number of initiatives intended to improve the 
quality of initial cost and schedules. These initiatives are documented 
in the NASA High-Risk Corrective Action Plan and have been reviewed in 
the recent GAO QuickLook Report. In particular, the NASA transition to 
the use of joint confidence level calculation at a 70 percent 
confidence level will essentially increase the ``set point'' for 
project cost and schedule estimates. Probabilistic estimating provides 
NASA with an approach that fully integrates technical, cost, and 
schedule plans and risks to develop both an understanding of the 
sensitivity of parameters to each other and the most likely estimate. 
Using this approach allows NASA to understand and document how the 
mitigation of technical risks would enable an increase in the project 
confidence level. Conversely, the introduction of a budget reduction 
would have the effect of increasing technical and schedule risks and 
thus lower the confidence level for the project. The use of 
probabilistic estimates also generates baseline values that include 
funding to address impacts associated with contingencies and 
uncertainties, such as industrial base, partner performance and 
technology optimism. By controlling the confidence levels at the 
program (rather than just project) level, NASA intends to be more 
vigilant in evaluating the state of the program's portfolio prior to 
undertaking additional projects. NASA expects that these steps will 
enable NASA to better control the tendency to undertake too much with 
too little.

Questions submitted by Representative Pete Olson

Q1.  When NASA experiences large cost increases without advanced 
warning it damages the Agency's credibility and reputation with 
stakeholders including Congress. Would you detail for us the actions 
the Agency has taken to improve its ability to forecast increases 
earlier so problems can be more readily managed and solutions more 
easily implemented?

A1. NASA has undertaken a number of initiatives intended to forecast 
and mitigate problems as early as possible. These initiatives include 
increased ongoing performance monitoring and more extensive review at 
key life cycle decision points, supported by expert, independent 
assessments.
    NASA has an ongoing performance review process to ensure that it 
can forecast increases, as early as possible, and work a solution. NASA 
uses several methods of tracking cost, schedule and science/technical 
requirements, and at multiple management levels.
    NASA projects employ various tools, including Earned Value 
Management, to monitor progress toward the baseline cost, schedule and 
technical deliverable. Center and Mission Directorate management review 
progress on a monthly or quarterly basis. Additionally, the Baseline 
Performance Review (BPR) process was implemented in 2007. The BPR is a 
monthly review, which assesses all Agency programs, with Agency Senior 
Management. On a quarterly basis, each Mission Directorate is 
spotlighted, providing a more in-depth assessment of its portfolio. 
This process provides not only individual program assessment but also a 
method to identify and address systemic programmatic and institutional 
issues that may affect multiple programs.
    In addition, the Agency implemented Key Decision Points (KDPs) with 
the release of NASA Procedural Requirements, NPR 7120.51D, NASA Space 
Flight Program and Project Management Requirements (effective March 
2007). KDPs serve as gates through which programs and projects must 
pass before proceeding to the next phase. Each KDP is preceded by one 
or more project or program-level reviews. Standing Review Boards are 
established to provide an expert, independent assessment throughout the 
Programs and Project's life cycle. These assessments are provided in 
support of the KDPs, with their views and recommendations.

Questions submitted by Representative Charles A. Wilson

Q1.  Experience has shown that there are major risks to program cost 
and schedule when insufficient large scale testing is performed during 
hardware development. What are NASA's plans for Orion and Ares I for 
large scale acoustic, vibration, thermal vacuum, and electromagnetic 
interference testing?

A1. Both Orion and Ares I will conduct testing starting at the 
component level, through system testing and then to large-scale 
testing. All component and systems will undergo acoustics, vibration, 
thermal, vacuum and electromagnetic interference and compatibility 
(EMI/EMC) testing for flight qualification. There will be vehicle 
specific (Orion-only, Ares-only) testing as well as integrated testing, 
including integrated vehicle (Orion and Ares) and ground systems as 
well as flight vehicle testing.
    For large-scale testing to support qualification, Orion will 
undergo a complete suite of integrated qualification tests for (EMI/
EMC) at the Integrated Environmental Test Facility at Glenn Research 
Center's Plum Brook Station in Ohio. The facility is undergoing 
modifications to support this testing.
    For Ares I, the size of the launch vehicle dictates a different 
hardware qualification flow. The components and systems will be tested 
to the established environmental (acoustics, vibration, thermal, and 
vacuum) as well as EMI/EMC requirements. The Ares I project is planning 
to use several full-scale Structural Test Articles to qualify the stage 
and tank structure. An entire Integrated System Test Article, including 
flight avionics, will be built. This test article will undergo a 
complete test firing. For the Ares I First Stage, three qualification 
motor firings are planned.
    An integrated Orion/Ares stack will undergo a mated Integrated 
Vehicle Ground Vibration Test at Marshall Space Flight Center in 
Alabama. Plans are also being developed to conduct a complete Flight 
Element Integration Test on the integrated Orion/Ares I stack at 
Kennedy Space Center in Florida.
    And finally, the Constellation test program will culminate with the 
Orion 1 flight test. Orion 1 is an un-crewed, fully functional orbital 
vehicle which will be launched on the Ares I. This flight has been 
designed to test and evaluate the integrated systems, exposed to the 
natural and induced environments experienced through the entire 
mission.
                   Answers to Post-Hearing Questions
Responses by Cristina T. Chaplain, Director, Acquisition and Sourcing 
        Management, Government Accountability Office (GAO)

Questions submitted by Chairwoman Gabrielle Giffords

Q1.  In your latest review, you say that NASA has attributed many of 
the issues on its projects to contractor performance. The 
identification of contractor performance issues begs the question of 
whether NASA undertakes adequate contractor oversight.

        a.  Do you have observations on the level of oversight of 
        contractors that NASA provides?

        b.  In your opinion, is NASA doing enough up-front thinking and 
        communication about the project and its requirement in order to 
        clearly lay out for the contractor what work needs to be 
        accomplished?

A1. Our assessment of selected large-scale NASA project focused on the 
status of major projects and the level of knowledge being attained at 
critical junctures. While we discussed management and oversight issues 
with both project managers and contractors, we did not specifically 
seek to measure the level of oversight NASA provided to its 
contractors. However, we observed instances where contractor 
capabilities were overestimated and could have been better understood 
up-front. For example, the Dawn contractor had no experience in deep 
space missions and more specifically, in ion propulsion systems for 
such missions, and officials from the company acknowledged they had 
difficulty developing the ion propulsion system for the spacecraft. In 
addition, it appeared to us that the contractor NASA used for the Glory 
and NPP projects overestimated its capacity to provide the work when 
promised, which was exacerbated when it closed one development facility 
and had difficulty hiring experienced staff at its new facility. In 
both these cases, the contractors cited communication gaps with NASA 
when their projects were experiencing problems. Given these and other 
instances, we believe NASA could do more up-front assessments about 
contractor capabilities and better plan for potential problems when 
contractors have not had prior experience with particular technologies 
or types of missions.

Q2.  What steps can Congress take to improve its oversight of NASA's 
acquisition of major projects in an effort to help address some of the 
issues identified by the GAO?

A2. The Congress has laid a good foundation for acquisition oversight 
by requiring NASA to establish baselines for major programs and report 
back when certain thresholds are breached. It can continue to build on 
this foundation by continually examining factors behind cost and 
schedule growth. Moreover, Congress can also support change though its 
own decisions about whether to authorize and appropriate funds for a 
program or project that is not performing well. Specifically, decisions 
to move projects into implementation should be based on whether a sound 
business case has been established, i.e., there is sufficient knowledge 
that requirements can be achieved within available time, funding, 
technology, workforce, and industrial base constraints.

Q3.  What corrective actions already undertaken by NASA show the most 
promise for dealing with cost management issues?

A3. Since NASA only recently issued its Corrective Action Plan, it is 
too early to determine whether any of these actions shows more promise 
than others. If these corrective actions are implemented in the spirit 
intended, they all have merit. Moreover, the success of any one 
improvement effort, such as cost estimating reform, depends on the 
success of others, such as those focused on producing more attention 
and oversight from senior leadership. GAO remains very interested to 
see NASA's commitment to following through on this plan and we await 
the results.

Q4.  Your prepared statement indicates similarities between what NASA 
faces with cost and schedule issues and what has been experienced by 
DOD and other federal agencies in their acquisition of space systems. 
Are some of the corrective actions undertaken by NASA of possible 
benefit to other agencies who conduct space acquisitions? Conversely, 
are there any ``lessons learned'' from those other agencies?

A4. It is difficult to determine which actions the Department of 
Defense and others can benefit from because NASA is still implementing 
its plan and not enough time has passed to determine impact. In 
addition, the DOD and National Reconnaissance Office space communities 
are implementing some similar reforms, notably in the area of cost 
estimating. In fact, cost estimators from all three communities have 
been sharing lessons learned in this regard. All three communities also 
seem to recognize the need to build knowledge about technologies and 
other resources before making long-term commitments to programs. 
Whether they can sustain the commitments they are making in this 
regard, however, remains to be seen.

Q5.  Your statement points out the MSL, ``which was already over budget 
recently announced a two-year launch delay,'' which you indicate will 
increase the cost of the mission. While that is accurate, would GAO 
recommend that NASA should have done something other than take the two-
year launch delay? What would GAO do differently at this point in the 
project?

A5. Clearly, at the time NASA announced the delay, it had no 
alternative but to delay the launch two years, and to fund the delay by 
diverting funding from other programs, or to cancel the program 
altogether. Our recommendations focus instead on preventing NASA from 
being faced with a similar situation in the future. Had the Mars 
Science Laboratory program followed a best practice approach, it would 
have had more knowledge about the technologies--both critical and 
heritage--it was pursuing at the time it estimated cost and schedule. 
There would have also been more robust planning for potential risks and 
more oversight from the Agency if cost, schedule, and performance 
issues did surface.

Q6a.  How successful have other agencies been at using confidence 
levels?

A6a. It is still too early to tell. NASA has just started to implement 
this policy and so has the FAA, DOD and other agencies.

Q6b.  Has this technique mitigated cost growth?

A6b. We do not know yet. Other agencies, including DOD, have only 
recently begun applying higher confidence levels to estimates of space 
and other complex programs. It is too early to determine their success 
in mitigating cost growth. It should be stressed, however, that higher 
confidence levels will not work as a practice if estimates are being 
made when there are significant unknowns about technologies, 
contractors' capabilities, funding availability, and requirements, 
among other factors. Moreover, higher confidence levels will not work 
unless they consider project risk assessments as well as cost and 
schedule risk analyses that capture the effects of funding cuts and 
lack of contractor capabilities.

Q6c.  Do you agree with NASA's choice of 70 percent confidence level or 
do you advocate a higher or lower level?

A6c. We have not made a recommendation or analyzed how high a 
confidence level should be for space programs but are encouraged that 
there is a desire to increase the confidence in estimating and that 
there are efforts underway to do so. Experts agree that technically 
risky programs that push the edge of technology should be funded at 
higher confidence levels (above 50 percent) since there are so many 
unknowns. Some experts have recommended that funding be at the 80 
percent level for new endeavors where no historical data exists to 
cover the ``unknowns'' (e.g., going to Mars).

Q7.  Why is NASA acquisition management still characterized by GAO as a 
high-risk area after 18 years?

A7. Historically, high-risk areas have been so designated because of 
traditional vulnerabilities related to their greater susceptibility to 
fraud, waste, abuse, and mismanagement. Since we designated NASA 
contract management as high-risk in 1990, our high-risk reports have 
focused on a variety of aspects related to controlling costs and risks 
in large-scale efforts--including, for example, definitizing contracts, 
implementing a new financial management system, and improving cost 
estimating. The underlying constant has been that programs are 
consistently over cost and behind schedule though the extent of cost 
growth had been hard to track until recently because of frequent re-
baselining. Because cost growth and schedule delays persist due to much 
more than risks inherent in spacecraft development, this area--now 
titled acquisition management because of the scope of issues that need 
to be resolved--remains high-risk.

Q8.  What must NASA do to warrant removal from GAO's high-risk list?

A8. NASA has taken significant steps in this direction but still faces 
substantial work on difficult initiatives where risks remain high. NASA 
has laid out a broad plan for reducing acquisition risk and taken steps 
to reflect best practices in policies. The plan specifically seeks to 
strengthen program and project management, increase accuracy in cost 
estimating, facilitate monitoring of contractor cost performance, 
improve agency-wide business processes, and improve financial 
management. Much work remains to achieve this plan. Some of the 
potentially most important initiatives in reducing cost growth and 
schedule slippage, and some of the most difficult parts of some 
initiatives have yet to be addressed. This will be difficult given the 
pressures the Agency is facing as it moves forward with the Ares I, 
Orion, and other expensive, highly anticipated programs such as the 
James Webb telescope. For instance, the desire to close the gap in 
human space flight may pressure NASA to move ahead with its Ares and 
Orion programs while there are still unknowns about technologies, 
design, and producibility, and to defer testing that is important to 
reducing cost and schedule risk. Coming off the high-risk list would 
also require NASA to find ways to better anticipate and mitigate what 
it currently believes is outside a program's control. For example, 
potential delays due to a crowded launch manifest could be better 
analyzed and factored into schedule estimates. Contractor capabilities 
could also be better understood up-front. NASA could work with the DOD 
on both issues, in fact, to develop more strategic approaches to 
problems in the launch manifest and the space industrial base.

Q9.  During the hearing, Rep. Grayson raised the issue of contrasting 
approaches of having vendors be responsible for overruns in fixed-price 
contracts and not being obliged to pay for cost growth in cost-
reimbursement contracts. Since NASA's program management process fully 
recognizes that a project usually starts with uncertainty but then 
matures through successful design reviews, could NASA use a cost-
reimbursement contract for the work conducted through the Preliminary 
Design Review (PDR) or Critical Design Review (CDR) and then 
subsequently utilize a fixed-price contract? Has such a hybrid approach 
ever been taken, and if so, what were the results?

A9. We do not know of instances where the hybrid approach described has 
been used for NASA or other space systems. For the hybrid approach to 
work, the contracts would need to be applied at a point where there is 
a high degree of certainty about technology, requirements, funding, 
etc. In DOD space programs, this is often not considered to occur until 
two satellites have been built. While we believe NASA can reduce 
critical unknowns about programs before programs enter implementation, 
there are other uncertainties that could unfairly impact a contractor 
under a fixed-price contract, such as a late delivery of a satellite 
component by one of NASA's international partners or an unforeseen 
problem with a launch vehicle. Moreover, in applying the hybrid 
approach, the government would need to be willing to stop programs that 
did incur cost increases, as a contractor would be unlikely to be 
willing to sustain long-term losses. This has been difficult to do in 
the past. For example, when it became apparent that contractors were 
sustaining losses for DOD's advanced extremely high frequency satellite 
program as well as the evolved expendable launch vehicle, the 
government did not stop the programs but rather converted to cost-plus 
arrangements. Conversely, when NASA stopped fixed-price arrangements 
under the X-33 and X-34 programs, it lost several years in its efforts 
to develop a successor to the Space Shuttle. Another caution is that a 
number of space programs that have tried to use fixed-price approaches 
in the past assumed that there would be a commercial market that would 
create a future demand for the outcome of their work with the 
government, which in turn, created an incentive for a company to work 
under a fixed price arrangement. However, this assumption later proved 
to be erroneous.

Question submitted by Representative Ralph M. Hall

Q1.  Given the long experience of schedule and cost growth in complex 
missions, why not simply impose higher cost and schedule reserves at 
the project's outset, instead of trying to operate with a too-small 
reserve when the project encounters difficulties?

A1. We would encourage NASA to do so given the history of cost and 
schedule problems, though NASA would need to concurrently ensure that 
programs are not incentivized to use up all of their reserves. 
Ultimately, establishing higher reserves may mean starting fewer 
programs--an approach the Administration, Congress and the many 
communities involved with NASA's major programs would need to support.
                   Answers to Post-Hearing Questions
Responses by Gary P. Pulliam, Vice President, Civil and Commercial 
        Operations, The Aerospace Corporation

Questions submitted by Chairwoman Gabrielle Giffords

Q1.  In your statement, you speak of teams wanting to put their best 
foot forward when being faced with the competitive pressure of 
initiating a mission at the lowest cost. You go on to say that, in some 
cases, underestimated content or complexity is often the result. How 
can we incentivize NASA teams to be realistic in identifying both their 
requirements and resource needs?

A1. Realism in identifying both the requirements and resource needs of 
a mission requires the proper balance between the complexity of science 
proposed for a mission and the risk of growth in cost and schedule for 
this mission.
    For NASA missions proposed through the Announcement of Opportunity 
(AO) process, the complexity of science that the proposer will pursue 
is decided by the proposing team, led by the Science Principal 
Investigator. Because of the open-ended nature of the science request 
for AOs, the proposer is incentivized to provide the maximum science 
capability within a fixed cost and schedule constraint. In order to 
capture the opportunity, the proposer typically pushes the envelope of 
performance within this cost and schedule cap, without proper regard to 
the likelihood of the mission exceeding the cost and schedule ``box.'' 
Too often the result is cost and schedule growth as the design matures 
and the concept is determined to be more difficult to develop than 
originally envisioned.
    For competed NASA missions, a selection that balances risk with 
science value has the potential to significantly reduce cost and 
schedule growth. A mission that has robust technical margins and is 
clearly within the cost and schedule envelope should experience limited 
cost and schedule growth. A selection such as this would be considered 
a low risk selection. If NASA solicitations promoted low risk missions 
and selected missions that were considered low risk, then the 
likelihood cost and schedule growth would be reduced. Further, it is 
expected that these NASA actions would incentivize the proposing teams 
to be more realistic about their requirements and resources needed.
    It must be understood, however, that if NASA selects only low risk 
missions, the potential science value of these may be reduced as well. 
It is hard to argue that a mission like Kepler, which is attempting to 
discover Earth-like planets, is not exciting science. At the same time, 
Kepler experienced significant cost and schedule growth. If not for 
that allowance of cost and schedule growth, the Kepler mission could 
not have been developed.
    One way to potentially balance cost and schedule risk versus 
science value is to employ a strategy that matures a technology through 
a focused technology development program prior to becoming a candidate 
NASA project. A generally accepted risk avoidance practice is to fund 
focused technology development prior to system development. However, 
due to budget constraints, NASA has recently reduced technology 
development funding in many areas.

Q2.  Your statement indicates similarities between what NASA faces with 
cost and schedule issues and what has been experienced by DOD and other 
federal agencies in their acquisition of space systems. Are some of the 
corrective actions undertaken by NASA of possible benefit to other 
agencies who conduct space acquisitions? Conversely, are there any 
``lessons learned'' from those other agencies?

A2. The DOD-initiated Cost Analysis Requirements Descriptions (CARD) 
inspired the NASA Cost Analysis Data Requirement (CADRe) effort. The 
CADRe effort that NASA has initiated is an excellent method to capture 
the cost, schedule, and technical data in a uniform way throughout the 
life cycle of a mission. Placing this data in a central repository, 
which is accessible by the community, is a significant step forward in 
terms of documenting and sharing data. CADRe allows NASA to perform 
detailed studies within a program as well as across NASA programs. 
CADRE also provides the comprehensive information required for the 
prediction of cost, schedule, and performance. The DOD could use a 
similar approach to share program development data among its 
acquisition community.
    The concept of budgeting to a confidence level, which NASA has 
recently implemented, was first adopted by the DOD. DOD also pioneered 
the use of Earned Value Management (EVM) on all of its acquisitions, 
and NASA followed with the requirement of EVM for developments over a 
certain dollar threshold. Although implementation of budgeting to a 
confidence level and the use EVM does not assure success, it should 
reduce the likelihood and magnitude of cost and schedule growth. In 
2007, NASA again followed DOD's lead in introducing Key Decision Point 
(KDP) processes and criteria into the NASA Project development life 
cycle. The purpose of KDP processes is to use explicit criteria to 
decide whether a particular program or project is ready to move on to 
the next phase in its life cycle.

Q3.  You state in your prepared statement that: ``the project must 
manage to a valid baseline estimate.'' You further note that ``One area 
of concern for the NASA project managers is the relevance and utility 
of independent cost estimates they do not own. Different methodologies 
are used by the project and independent estimates such that there is 
not a common understanding of the basis for estimate for each.'' You 
note that ``greater transparency into the basis of estimate for each 
approach is important.'' Could you please elaborate on your concerns 
about independent cost estimate at NASA and what specific types of 
actions NASA might take to ensure transparency in its estimates?

A3. NASA independent cost estimates are typically conducted with cost 
models using input parameters that are not fully understood by Project 
Managers. Consequently, Project Managers often have limited insight 
into how an independent cost estimate is developed, and thus its 
validity, relevance, and utility. Furthermore, it is often difficult 
for NASA cost analysts to communicate the intricacies of their estimate 
to the project team. The project team uses a different cost 
methodology, estimating the effort of tasks from the bottom-up, based 
on the knowledge of the individuals working on the project. These 
differences in assumptions and methodologies make reconciliation of 
cost estimates difficult.
    NASA cost analysts must do a better job of demonstrating how their 
estimates compare with historical cost data in order to provide Project 
Managers confidence that their models are predictive. The collection of 
CADRe data provides the basis for such a comparison. NASA cost analysts 
must also fully explain the methodology, as well as the data inputs, to 
Project Managers. In particular, when subjective data inputs are 
introduced into the cost models, and used in a certain manner, NASA 
cost analysts must provide Project Managers with a full understanding 
of the basis for using these inputs. This increased transparency would 
likely result in Project Managers having an increased understanding and 
confidence in the independent cost estimates.

Q4.  In describing what else needs to be done, your testimony 
highlights the need to strengthen the connection between an independent 
cost estimate and the project estimate to include the effects of risk 
and risk mitigation. Why is the project's ownership of the cost 
estimate so important?

A4. The project must ``own'' the independent cost estimate if it is 
expected to manage to it. The project must believe that the baseline 
cost estimate is valid and achievable so that they can steadfastly work 
toward executing the plan. Too often, the independent cost estimate is 
unrelated to project risk management results, and there is a disconnect 
between the risk identification process and the cost estimate. It is 
critical to link these two processes.
    Projects use risk identification to measure their robustness. 
Furthermore, an economic assessment can determine if a project should 
implement a risk mitigation effort. The identification of risk and risk 
mitigation activities helps the project to fully understand how the 
baseline plan would be affected by certain risks. Space system 
development projects have processes for identifying and managing risks, 
such as the 5 x 5 matrix process that classifies likelihood and 
consequences for each risk. A probabilistic cost estimate approach that 
incorporates all discrete risks may be a way to ensure that the Project 
Manager and the independent cost estimate team have a common language 
and understanding of risk.

Q5.  Your testimony refers to NASA's recent requirement to develop 
budgets with a 70 percent confidence level. Your statement also notes 
that ``The validity of this approach, however, depends on the stability 
and soundness of the baseline'' budget. What is needed to ensure that 
the requirement for a 70 percent confidence level in developing NASA 
budget estimates will be effective?

A5. A stable baseline budget is critical to the development of a 
project plan that can be effectively managed and executed. A sound, 
stable baseline plan requires the development early in a project of the 
70 percent confidence level that assesses all potential outcomes and 
incorporates all potential risks. The development of the 70 percent 
confidence level entails identifying the potential risks and 
quantifying their potential impact in terms of cost or schedule growth. 
The development of this comprehensive risk plan and a robust confidence 
level assessment allows for a stable and comprehensive baseline plan.
    Note that Earned Value Management (EVM) is much more effective if 
the measurement of progress is based on performance against a stable 
plan. Multiple replanning or re-baselining activities make EVM 
significantly less effective. Replanning and/or re-baselining are 
sometimes necessary due to subsequent project changes. However, a 
stable, robust baseline plan from the outset should reduce the need for 
replanning and lead to a more efficiently managed project.

Q6.  How successful have other agencies been at using confidence 
levels? Has this technique mitigated cost growth? Do you agree with 
NASA's choice of a 70 percent confidence level or do you advocate a 
higher or lower level?

A6. Many Blue Ribbon panels advocate creating a program reserve by 
budgeting to a higher confidence level as good practice. However, it is 
too early to tell how effective budgeting to a higher confidence level 
is in controlling cost and schedule growth. The DOD started using cost 
confidence levels to improve program budgeting about eight years ago. 
It has not been uniformly applied across DOD space programs; and given 
the long development timelines for major space programs, there are few 
completed programs that were initiated using this budget approach. In 
addition, the resulting increased funding from budgeting to higher 
confidence levels have not been consistently applied within the 
programs. Since several factors contribute to a project's cost and 
schedule growth, it may be difficult to determine if solely setting a 
cost confidence level has reduced cost growth in the DOD until more 
data becomes available.

Q7.  In the past year, we have seen a growing number of protests 
associated with NASA contract awards. To what do you attribute this 
growing number of protests? Are there some steps NASA could take to 
minimize vendors' need to file such protests? Is this phenomenon unique 
to NASA or are you observing a similar trend at DOD?

A7. Since Aerospace has not participated in a NASA source selection 
that has lead to protests, we cannot speak to specific experience on 
the subject. It could be postulated, however, that as mergers occur and 
fewer and fewer contractors are competing for fewer and fewer missions, 
it is inevitable that more protests would be filed. Many elements of 
NASA proposal evaluations are subjective in nature and, therefore, are 
open to debate and potential protest. For example, it is difficult to 
determine objectively the relative science value of a mission visiting 
an asteroid vs. one that visits a planet. Even with more objective 
evaluation criteria, such as those for DOD procurements, there still 
may be an incentive for a company to file a protest if it believes that 
its proposal has been misinterpreted.

Q8.  During the hearing, Rep. Grayson raised the issue of contrasting 
approaches of having vendors be responsible for overruns in fixed price 
contracts and not being obliged to pay for cost growth in cost 
reimbursement contracts. Since NASA's program management process fully 
recognizes that a project usually starts with uncertainty but then 
matures through success design reviews, could NASA use a cost 
reimbursement contract for the work up conducted through the 
Preliminary Design Review (PDR) or Critical Design Review (CDR) and 
then subsequently utilize a fixed price contract? Has such a hybrid 
approach ever been taken, and if so, what were the results?

A8. Given the nature of NASA's primary objective, which is to develop 
missions that provide first-of-a-kind, world-class science, it is our 
opinion that using a fixed price contract structure would not be 
appropriate for the majority of NASA's contracts. The commercial 
communications industry, which has a continuing product line and very 
well defined and stable requirements from the customer, uses this 
approach with great success. As stated in the written testimony, 
however, since NASA does not typically have a mature design and stable 
requirements until late in the design process, a fixed price contract 
structure must be applied with great caution and only in certain 
circumstances. For projects where uncertainty is high and the potential 
risks are significant, a fixed price contract would not be beneficial 
either for NASA or its contractors.

Q9.  Should there be a reasonable level of reserves included in the 
estimated cost of a program and, if so, what would you define as 
``reasonable''? What are the pros and cons of a higher level of 
reserves?

A9. Aerospace was involved in a previous study which suggested nominal 
reserve levels based on historical data for a specific set of robotic 
science missions. These nominal reserve levels were not intended to 
provide a hard and fast rule for specifying a percentage of reserves 
that all projects should carry. Aerospace recommends that the level of 
reserves should be commensurate with a given project's risks and 
criticality. Reserves could be defined by the confidence level 
requirement of the budget (i.e., the higher the confidence level, the 
greater the magnitude of the reserves). Budgeting using confidence 
levels allows the magnitude of a project's reserves to vary 
proportionately with each project's 70 percent confidence level, thus 
reflecting each project's unique risks.
    It is critical that NASA hold funds in reserve to manage its 
portfolio of missions. A project will use its available funding, 
including reserve, in order to mitigate risks and ensure mission 
success. This it is imperative to allocate some portion of the reserve 
funding at a higher level, such as to a Program, which is a collection 
of Projects, or to NASA Headquarters (HQ). This tiered reserve 
allocation allows funding at the Program or HQ level to be quickly 
allocated to other Projects that exceed their initial allocation. It 
also provides for a more flexible portfolio management approach, 
allowing less reserve to be held for any single project.

Question submitted by Representative Ralph M. Hall

Q1.  Given the long experience of schedule and cost growth in complex 
missions, why not simply impose higher cost and schedule reserves at 
the project's outset, instead of trying to operate with a too-small 
reserve when the project encounters difficulties?

A1. Since mission success is the objective of Project Managers, 
projects tend to use all of the cost and schedule reserves that they 
control in order to reduce mission risk. Hence, any additional project 
cost and schedule reserves provided would typically be used to further 
reduce mission risk through performing additional testing, developing 
additional testbeds, purchasing additional spare hardware, etc. Having 
higher project cost and schedule reserves may decrease perceived cost 
and schedule growth; however, in the end, the actual cost and schedule 
of missions may be greater. Given NASA's fixed annual budget, the 
approach of imposing higher cost and schedule reserves for could lead 
to a reduction in the number of missions that NASA is able to develop.
    To be able to effectively manage its portfolio of missions, 
however, it is critical that NASA hold funds in reserve. Assuming that 
a project will use its available funding including reserve, it is 
imperative that NASA allocate some portion of the reserve funding at a 
higher level, such as to a Program, which is a collection of Projects, 
or to NASA Headquarters (HQ). This tiered reserve allocation allows 
funding at the Program or HQ level to be allocated to those Projects 
that exceed their initial allocation. This approach also provides for a 
more flexible portfolio management approach, allowing less reserve to 
be held for any single Project. A tiered reserve, which distributes 
reserves at different organizational levels, represents a balanced 
approach. While it provides some reserves for the Project Managers to 
rapidly address development problems before their costs escalate, it 
does not release all reserves to the Projects where the funds would 
most likely be used up-front to plan additional mission risk reduction 
activities.

Questions submitted by Representative Pete Olson

Q1.  Are there valid reasons why it might be necessary for a project to 
enter the implementation phase without having fully matured critical 
technologies? And if so, are there ways to compensate for the increased 
risk?

A1. For the majority of NASA missions, it is prudent for NASA to fully 
develop its critical technologies prior to the start of the 
implementation phase. However, NASA may be required to start mission 
implementation without having fully matured critical technologies if 
the timeliness of a mission is such that it must be fielded on a highly 
expedited schedule. Although such a timeliness requirement is more 
prevalent in national security space systems, NASA could have a 
compelling need (e.g., a rescue or replacement of a critical NASA 
satellite or capability) that requires such timeliness. When a project 
must enter the implementation phase without having fully matured 
critical technologies, the project should compensate for the increased 
risk by developing alternative technologies in parallel. Then if a 
critical technology is not ready, an alternative technology could be 
implemented. Note that developing alternative technologies in parallel 
to compensate for increased risk may result in higher mission cost and 
decreased mission capability.

                                   
