    Polar-Orbiting Operational Environmental Satellites:
    Restructuring Is Under Way, but Technical Challenges and Risks
    Remain (27-APR-07, GAO-07-498).

    The National Polar-orbiting Operational Environmental Satellite
    System (NPOESS) is a tri-agency acquisition--managed by the
    Departments of Commerce and Defense and the National Aeronautics
    and Space Administration--which experienced escalating costs,
    schedule delays, and technical difficulties. These factors led to
    a June 2006 decision to restructure the program thereby
    decreasing the program's complexity, increasing its estimated
    cost to $12.5 billion, and delaying the first two satellites by 3
    to 5 years. GAO was asked to (1) assess progress in restructuring
    the acquisition, (2) evaluate progress in establishing an
    effective management structure, (3) assess the reliability of the
    cost and schedule estimate, and (4) identify the status and key
    risks facing the program's major segments. To do so, GAO analyzed
    program and contractor data, attended program reviews, and
    interviewed program officials
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-07-498
        ACCNO:   A68845
      TITLE:     Polar-Orbiting Operational Environmental Satellites:
    Restructuring Is Under Way, but Technical Challenges and Risks
    Remain
         DATE:   04/27/2007
      SUBJECT:   Contract modifications
    	     Cost analysis
    	     Cost control
    	     Cost overruns
    	     Earth resources satellites
    	     Federal procurement
    	     Interagency relations
    	     Procurement planning
    	     Program evaluation
    	     Program management
    	     Risk assessment
    	     Schedule slippages
    	     Strategic planning
    	     Work measurement
    	     Cost estimates
    	     Cost growth
    	     Policies and procedures
    	     National Polar-Orbiting Operational
    	     Environmental Satellite System

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-07-498

         * [1]Results in Brief
         * [2]Background

              * [3]Polar Satellite Data and Products
              * [4]NPOESS Overview
              * [5]NPOESS Acquisition Strategy
              * [6]NPOESS Experienced Cost Increases, Schedule Delays, and Tech
              * [7]Nunn-McCurdy Process Led to a Decision to Restructure the NP
              * [8]Earned Value Management Techniques Provide Insight on Progra

         * [9]NPOESS Acquisition Restructuring Is Well Under Way, but Key
         * [10]Progress Has Been Made in Establishing an Effective NPOESS M

              * [11]NPOESS Program Has Made Progress in Establishing an Effectiv
              * [12]NPOESS Program Has Filled Key Vacancies but Lacks A Programw

         * [13]Methodology Supporting the June 2006 Cost and Schedule Estim

              * [14]Certified Program Estimates Were Developed Using an Acceptab
              * [15]Recent Events Could Lead to Increased Program Costs or Delay

         * [16]Major Program Segments Are Under Development, but Significan

              * [17]Space Segment--Progress Made, but Key Sensors Continue to Fac

                   * [18]Earned Value Data Show Problems on the Space Segment
                   * [19]Program Office Is Monitoring Sensor Risks and Evaluating
                     Opt

              * [20]Ground Segment--Progress Has Been Made, but Work Remains
              * [21]Ground Segment Cost and Schedule Are on Track; Work and Risk

                   * [22]Program Office Has Plans to Address IDPS Risks

              * [23]Launch Segment--NPP Launch Preparation Has Begun, while NPOES

         * [24]Conclusions
         * [25]Recommendations for Executive Action
         * [26]Agency Comments and Our Evaluation
         * [27]GAO Contact
         * [28]Staff Acknowledgments
         * [29]GAO's Mission
         * [30]Obtaining Copies of GAO Reports and Testimony

              * [31]Order by Mail or Phone

         * [32]To Report Fraud, Waste, and Abuse in Federal Programs
         * [33]Congressional Relations
         * [34]Public Affairs

    Report to Congressional Requesters

    United States Government Accountability Office

    GAO

    April 2007

    POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITES

    Restructuring Is Under Way, but Technical Challenges and Risks Remain

    GAO-07-498

    Contents

    Letter 1

    Results in Brief 2
    Background 5
    NPOESS Acquisition Restructuring Is Well Under Way, but Key Steps Remain
    To Be Completed 21
    Progress Has Been Made in Establishing an Effective NPOESS Management
    Structure, but Executive Turnover Will Increase Risks, and Staffing
    Problems Remain 26
    Methodology Supporting the June 2006 Cost and Schedule Estimate Was
    Reliable, but Recent Events Could Increase Program Costs 31
    Major Program Segments Are Under Development, but Significant Risks Remain
    34
    Conclusions 42
    Recommendations for Executive Action 43
    Agency Comments and Our Evaluation 43
    Appendix I Objectives, Scope, and Methodology 46
    Appendix II Overview of Satellite Data Processing Algorithms and the
    Calibration and Validation Process 48
    Appendix III Comments from the Department of Commerce 51
    Appendix IV Comments from the Department of Defense 54
    Appendix V Comments from the National Aeronautics and Space Administration
    56
    Appendix VI GAO Contact and Staff Acknowledgments 57

    Tables

    Table 1: Expected NPOESS Instruments as of August 31, 2004 (critical
    sensors are in bold) 13
    Table 2: Summary of Changes to the NPOESS Program 17
    Table 3: Key Program Milestones 18
    Table 4: Changes to NPOESS Instruments (critical sensors are in bold) 19
    Table 5: Planned Configuration of Sensors on NPP and NPOESS Satellites 20
    Table 6: Elements of a Sound Cost Estimating Methodology 32
    Table 7: Status of Selected Components of the Space Segment, as of April
    2007 35
    Table 8: Status of Ground Segment Components 39

    Figures

    Figure 1: Configuration of Operational Polar Satellites 6
    Figure 2: A Generic Data Relay Pattern for Polar Meteorological Satellite
    Systems 7
    Figure 3: Satellite Data Processing Steps 8
    Figure 4: Analysis of Ozone Concentration from POES Satellite Data 9
    Figure 5: POES Image of Hurricane Katrina in 2005 10
    Figure 6: Organizations Coordinated by the NPOESS Integrated Program
    Office 12
    Figure 7: Progress on Planned Milestones in Fiscal Year 2006, as of
    October 1, 2006 23
    Figure 8: Progress on Planned Milestones in Fiscal Year 2007, as of
    January 20, 2007 24
    Figure 9: Overview of New Management Structure 27
    Figure 10: Cumulative Cost and Schedule Variance for the NPOESS Space
    Segment over a 13-Month Period 37
    Figure 11: Cumulative Cost and Schedule Variance for the NPOESS IDPS
    Development over a 13-Month Period 40
    Figure 12: Cumulative Cost and Schedule Variance for the NPOESS Command,
    Control, and Communications Segment over a 13-Month Period 41
    Figure 13: Satellite Data Processing Algorithms 49
    Figure 14: Describes the High-Level Calibration and Validation Process 50

    Abbreviations

    ATMS advanced technology microwave sounder
    CMIS conical-scanned microwave imager/sounder
    CrIS cross-track infrared sounder
    DMSP Defense Meteorological Satellite Program
    DOD Department of Defense
    EDR environmental data record
    IDPS interface data processing system
    NASA National Aeronautics and Space Administration
    NESDIS National Environmental Satellite Data and Information Service
    NOAA National Oceanic and Atmospheric Administration
    NPOESS National Polar-orbiting Operational Environmental Satellite System
    NPP NPOESS Preparatory Project
    POES Polar-orbiting Operational Environmental Satellites
    OMPS ozone mapper/profiler suite
    VIIRS visible/infrared imager radiometer suite

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States Government Accountability Office
    Washington, DC 20548

    April 27, 2007

    The Honorable Nick Lampson
    Chairman
    The Honorable Bob Inglis
    Ranking Republican Member
    Subcommittee on Energy and Environment
    Committee on Science and Technology
    House of Representatives

    The Honorable David Wu
    House of Representatives

    The Honorable Vernon J. Ehlers
    House of Representatives

    The planned National Polar-orbiting Operational Environmental Satellite
    System (NPOESS) program is expected to be a state-of-the-art,
    environment-monitoring satellite system that will replace two existing
    polar-orbiting environmental satellite systems. Polar-orbiting satellites
    provide data and imagery that are used by weather forecasters,
    climatologists, and the military to map and monitor changes in weather,
    climate, the oceans, and the environment. The NPOESS program is considered
    critical to the United States' ability to maintain the continuity of data
    required for weather forecasting (including severe weather events such as
    hurricanes) and global climate monitoring through the year 2026. The
    planned National Polar-orbiting Operational Environmental Satellite System
    (NPOESS) program is expected to be a state-of-the-art,
    environment-monitoring satellite system that will replace two existing
    polar-orbiting environmental satellite systems. Polar-orbiting satellites
    provide data and imagery that are used by weather forecasters,
    climatologists, and the military to map and monitor changes in weather,
    climate, the oceans, and the environment. The NPOESS program is considered
    critical to the United States' ability to maintain the continuity of data
    required for weather forecasting (including severe weather events such as
    hurricanes) and global climate monitoring through the year 2026.

    Three agencies share responsibility for the NPOESS acquisition: the
    Department of Commerce's National Oceanic and Atmospheric Administration
    (NOAA), the Department of Defense (DOD)/United States Air Force, and the
    National Aeronautics and Space Administration (NASA). To manage the NPOESS
    program, these agencies established a tri-agency integrated program
    office. In recent years, this program has experienced escalating costs,
    schedule delays, and technical difficulties, leading to a June 2006
    decision to restructure the program. This decision decreased the
    complexity of the program by reducing the number of satellites and
    sensors, increased the estimated cost of the program to $12.5 billion, and
    delayed the launches of the first two satellites by 3 to 5 years. Three
    agencies share responsibility for the NPOESS acquisition: the Department
    of Commerce's National Oceanic and Atmospheric Administration (NOAA), the
    Department of Defense (DOD)/United States Air Force, and the National
    Aeronautics and Space Administration (NASA). To manage the NPOESS program,
    these agencies established a tri-agency integrated program office. In
    recent years, this program has experienced escalating costs, schedule
    delays, and technical difficulties, leading to a June 2006 decision to
    restructure the program. This decision decreased the complexity of the
    program by reducing the number of satellites and sensors, increased the
    estimated cost of the program to $12.5 billion, and delayed the launches
    of the first two satellites by 3 to 5 years.

    This report responds to your request that we (1) evaluate the NPOESS
    program office's progress in restructuring the acquisition, (2) evaluate
    the This report responds to your request that we (1) evaluate the NPOESS
    program office's progress in restructuring the acquisition, (2) evaluate
    the program office's progress in establishing an effective management
    structure, (3) assess the reliability of the life cycle cost estimate and
    proposed schedule, and (4) identify the status and key risks facing the
    program's major segments and evaluate the adequacy of the program's
    efforts to mitigate these risks.

    To evaluate the program office's progress in restructuring the
    acquisition, we assessed program documentation, attended management status
    briefings, and interviewed program officials. To determine progress in
    establishing a new management structure, we assessed the status of efforts
    to implement past recommendations regarding the management structure and
    staffing, attended senior-level management review meetings, reviewed
    program documents, and interviewed program officials. To assess the cost
    estimate, we evaluated the methodology and assumptions used to develop the
    estimate and interviewed program officials to identify any assumptions
    that may have changed. To determine the status, risk, and risk mitigation
    efforts for the program, we analyzed monthly program management documents
    and interviewed NOAA, NASA, and DOD officials to determine concerns with
    these mitigation efforts. In addition, this report builds on other work we
    have done on environmental satellite programs over the last several
    years.^1

    We conducted our work at the NPOESS Integrated Program Office headquarters
    and at DOD, NOAA, and NASA facilities in the Washington, D.C.,
    metropolitan area. We performed our work from July 2006 to April 2007 in
    accordance with generally accepted government auditing standards. Appendix
    I contains additional details on our objectives, scope, and methodology.

    Results in Brief

    The NPOESS program office has made progress in restructuring the
    acquisition by establishing and implementing interim program plans guiding
    the contractors' work activities in 2006 and 2007; however, important
    tasks leading up to finalizing contract changes remain to be completed.
    While the program office developed key acquisition documents, including a
    memorandum of agreement on the roles and responsibilities of the three
    agencies, a revised acquisition strategy, and a system engineering plan,
    the responsible executives in the three agencies have not yet approved
    these documents--even though they were due by September 1, 2006.
    Finalizing these documents is essential to ensure interagency agreement
    and will allow the program office to move forward in completing other
    activities related to restructuring the program. These activities include
    conducting an integrated baseline review with the contractor to reach
    agreement on the schedule and work activities and finalizing changes to
    the NPOESS development and production contract--thereby allowing the
    program office to lock down a new acquisition baseline cost and schedule.
    Until the key acquisition documents are approved by the appropriate
    executives in each agency, the program faces increased risk that
    restructuring activities will not be completed in time to allow it to move
    forward in fiscal year 2008 with a new program baseline in place. This
    places the NPOESS program at risk of continued delays and future cost
    increases.

    ^1GAO, Polar-orbiting Operational Environmental Satellites: Cost Increases
    Trigger Review and Place Program's Direction on Hold, [35]GAO-06-573T
    (Washington, D.C.: Mar. 30, 2006); GAO, Polar-orbiting Operational
    Environmental Satellites: Technical Problems, Cost Increases, and Schedule
    Delays Trigger Need for Difficult Trade-off Decisions, [36]GAO-06-249T
    (Washington, D.C.: Nov. 16, 2005); GAO, Polar-orbiting Environmental
    Satellites: Information on Program Cost and Schedule Changes,
    [37]GAO-04-1054 (Washington, D.C.: Sept. 30, 2004); GAO, Polar-orbiting
    Environmental Satellites: Project Risks Could Affect Weather Data Needed
    by Civilian and Military Users, [38]GAO-03-987T (Washington, D.C.: July
    15, 2003); and GAO, Polar-orbiting Environmental Satellites: Status,
    Plans, and Future Data Management Challenges, [39]GAO-02-684T (Washington,
    D.C.: July 24, 2002).

    The program office has also made progress in establishing an effective
    management structure by adopting a new organizational framework with
    increased oversight from program executives and by instituting more
    frequent and rigorous program management reviews; however, planned changes
    in executive management will likely increase program risk, and the program
    lacks a process and plan for identifying and filling staffing shortages.
    As a result, the program experienced delays in beginning key activities
    such as cost estimating and contract revisions. Until this process is in
    place and working, the NPOESS program faces increased risk of further
    delays.

    The methodology supporting a June 2006 cost and schedule estimate was
    reliable, but recent events could lead to increased program costs and
    delay schedules. DOD's independent cost estimating group used an
    acceptable methodology in developing a June 2006 cost estimate of $11.5
    billion for the acquisition portion of the restructured program with the
    expectation of initial satellite launch in January 2013. Consistent with
    DOD direction, this estimate did not include roughly $1 billion in
    operations and support costs--bringing the total life cycle cost estimate
    to $12.5 billion. However, the program continues to experience technical
    problems on key sensors, and program costs will likely be adjusted during
    upcoming negotiations on contract changes. The NPOESS program office is
    developing its own cost estimate to further refine the one developed in
    June 2006 to help it negotiate contract changes. A new baseline cost and
    schedule will be established once the contract is finalized--an event that
    the Program Director expects to occur by July 2007.

    Development and testing of major program segments--including key sensors
    and the ground systems--are under way, but significant risks remain. For
    example, work continues on key sensors, but two sensors--the
    Visible/Infrared Imager Radiometer Suite and the Cross-track Infrared
    Sounder--continue to experience significant difficulties. Specifically,
    the former encountered three significant problems with image quality and
    reliability during environmental testing of the engineering unit, and the
    latter suffered a major structural failure during vibration testing.
    Additionally, while significant progress has been made in reducing delays
    in the NPOESS data processing system, much work remains in refining the
    algorithms needed to translate sensor observations into usable weather
    products. Given the tight time frames for completing key sensors,
    integrating them with the demonstration spacecraft (called the NPOESS
    Preparatory Project or NPP) and getting the ground-based data processing
    systems developed, tested, and deployed, it will be important for the
    Integrated Program Office, the Program Executive Office, and the Executive
    Committee to continue to provide close oversight of milestones and risks.

    We are making recommendations to the Secretaries of Commerce and Defense
    and to the Administrator of NASA to ensure that the appropriate executives
    finalize key acquisition documents by the end of April 2007 in order to
    allow the restructuring of the program to proceed. We are also making
    recommendations to the Secretary of Defense to direct the Air Force to
    delay reassigning the recently appointed Program Executive Officer until
    key program risks are resolved. We are also making recommendations to the
    Secretary of Commerce to ensure that NPOESS program authorities develop
    and implement a written process for identifying and addressing human
    capital needs and that they establish a plan to immediately fill needed
    positions.

    The Department of Commerce, DOD, and NASA provided written comments on our
    draft report (see apps. III, IV, and V). All three agencies agreed that it
    was important to finalize key acquisition documents in a timely manner,
    and DOD proposed extending the due dates for the documents to July 2,
    2007. In addition, the Department of Commerce concurred with our
    recommendation to identify and address human capital needs and immediately
    fill open positions in the NPOESS program office. Commerce noted that NOAA
    was taking actions in both areas. However, DOD did not concur with our
    recommendation to delay reassigning the Program Executive Officer, noting
    that the Program Director responsible for the acquisition program would
    remain in place for 4 years. While it is important that the System Program
    Director remain in place to ensure continuity in executing the
    acquisition, this position does not ensure continuity in the important
    oversight and coordination functions provided by the current Program
    Executive Officer. We remain concerned that reassigning the Program
    Executive at a time when NPOESS is still facing critical cost, schedule,
    and technical challenges will place the program at further risk.

    All three agencies also provided technical comments, which we have
    incorporated in this report as appropriate.

    Background

    Since the 1960s, the United States has operated two separate operational
    polar-orbiting meteorological satellite systems: the Polar-orbiting
    Operational Environmental Satellite (POES) series--managed by NOAA, and
    the Defense Meteorological Satellite Program (DMSP)--managed by the Air
    Force. These satellites obtain environmental data that are processed to
    provide graphical weather images and specialized weather products. These
    satellite data are also the predominant input to numerical weather
    prediction models, which are a primary tool for forecasting weather 3 or
    more days in advance--including forecasting the path and intensity of
    hurricanes. The weather products and models are used to predict the
    potential impact of severe weather so that communities and emergency
    managers can help prevent and mitigate their effects. Polar satellites
    also provide data used to monitor environmental phenomena, such as ozone
    depletion and drought conditions, as well as data sets that are used by
    researchers for a variety of studies such as climate monitoring.

    Unlike geostationary satellites, which maintain a fixed position relative
    to the earth, polar-orbiting satellites constantly circle the earth in an
    almost north-south orbit, providing global coverage of conditions that
    affect the weather and climate. Each satellite makes about 14 orbits a
    day. As the earth rotates beneath it, each satellite views the entire
    earth's surface twice a day. Currently, there are two operational POES
    satellites and two operational DMSP satellites that are positioned so that
    they can observe the earth in early morning, midmorning, and early
    afternoon polar orbits. Together, they ensure that, for any region of the
    earth, the data provided to users are generally no more than 6 hours old.
    Figure 1 illustrates the current operational polar satellite
    configuration. Besides the four operational satellites, six older
    satellites are in orbit that still collect some data and are available to
    provide limited backup to the operational satellites should they degrade
    or fail. In the future, both NOAA and the Air Force plan to continue to
    launch additional POES and DMSP satellites every few years, with final
    launches scheduled for 2009 and 2012, respectively.^2

    Figure 1: Configuration of Operational Polar Satellites

    Each of the polar satellites carries a suite of sensors designed to detect
    environmental data that are either reflected or emitted from the earth,
    the atmosphere, and space. The satellites broadcast a subset of these data
    in real time to properly equipped field terminals that are within a direct
    line of sight; these field terminals are located at universities, on
    battlefields, or on ships. Additionally, the polar satellites store the
    observed environmental data and then transmit them to NOAA and Air Force
    ground stations when the satellites pass overhead. The ground stations
    then relay the data via communications satellites to the appropriate
    meteorological centers for processing.

    ^2Three DMSP satellites and one POES satellite remain to be launched.

    Under a shared processing agreement among four satellite data processing
    centers--NOAA's National Environmental Satellite Data and Information
    Service (NESDIS), the Air Force Weather Agency, the Navy's Fleet Numerical
    Meteorology and Oceanography Center, and the Naval Oceanographic
    Office--different centers are responsible for producing and distributing,
    via a shared network, different environmental data sets, specialized
    weather and oceanographic products, and weather prediction model outputs.

    Each of the four processing centers is also responsible for distributing
    the data to its respective users. For the DOD centers, the users include
    regional meteorology and oceanography centers, as well as meteorology and
    oceanography staff on military bases, the Naval Fleet, and mobile field
    sites. NESDIS forwards the data to NOAA's National Weather Service for
    distribution and use by government and commercial forecasters. The
    processing centers also use the Internet to distribute data to the general
    public. NESDIS is responsible for the long-term archiving of data and
    derived products from POES and DMSP. Figure 2 depicts a generic data relay
    pattern from the polar-orbiting satellites to the data processing centers
    and field terminals.

    Figure 2: A Generic Data Relay Pattern for Polar Meteorological Satellite
    Systems

    Polar Satellite Data and Products

    Polar satellites gather a broad range of data that are transformed into a
    variety of products. Satellite sensors observe different bands of
    radiation wavelengths, called channels, which are used for remotely
    determining information about the earth's atmosphere, land surface,
    oceans, and the space environment. When first received, satellite data are
    considered raw data. To make them usable, the processing centers format
    the data so that they are time-sequenced and include earth location and
    calibration information. After formatting, these data are called raw data
    records. The centers further process these raw data records into
    channel-specific data sets, called sensor data records and temperature
    data records. These data records are then used to derive weather and
    climate products called environmental data records (EDR). EDRs include a
    wide range of atmospheric products detailing cloud coverage, temperature,
    humidity, and ozone distribution; land surface products showing snow
    cover, vegetation, and land use; ocean products depicting sea surface
    temperatures, sea ice, and wave height; and characterizations of the space
    environment. Combinations of these data records (raw, sensor, temperature,
    and environmental data records) are also used to derive more sophisticated
    products, including outputs from numerical weather models and assessments
    of climate trends. Figure 3 is a simplified depiction of the various
    stages of satellite data processing, and figures 4 and 5 depict examples
    of EDR weather products.

    Figure 3: Satellite Data Processing Steps

    Figure 4: Analysis of Ozone Concentration from POES Satellite Data

    Figure 5: POES Image of Hurricane Katrina in 2005

    NPOESS Overview

    With the expectation that combining the POES and DMSP programs would
    reduce duplication and result in sizable cost savings, a May 1994
    Presidential Decision Directive required NOAA and DOD to converge the two
    satellite programs into a single satellite program capable of satisfying
    both civilian and military requirements.^3 The converged program, NPOESS,
    is considered critical to the United States' ability to maintain the
    continuity of data required for weather forecasting and global climate
    monitoring through the year 2026. To manage this program, DOD, NOAA, and
    NASA formed the tri-agency Integrated Program Office, located within NOAA.

    Within the program office, each agency has the lead on certain activities:
    NOAA has overall program management responsibility for the converged
    system and for satellite operations; DOD has the lead on the acquisition;
    and NASA has primary responsibility for facilitating the development and
    incorporation of new technologies into the converged system. NOAA and DOD
    share the costs of funding NPOESS, while NASA funds specific technology
    projects and studies. Figure 6 depicts the organizations that make up the
    NPOESS program office and lists their responsibilities.

    ^3Presidential Decision Directive NSTC-2, May 5, 1994.

    Figure 6: Organizations Coordinated by the NPOESS Integrated Program
    Office

    The NPOESS program office is overseen by an Executive Committee, which is
    made up of the Administrators of NOAA and NASA and the Undersecretary of
    the Air Force.

    NPOESS Acquisition Strategy

    NPOESS is a major system acquisition that was originally estimated to cost
    about $6.5 billion over the 24-year life of the program from its inception
    in 1995 through 2018. The program is to provide satellite development,
    satellite launch and operation, and ground-based satellite data
    processing. These deliverables are grouped into four main categories: (1)
    the space segment, which includes the satellites and sensors; (2) the
    integrated data processing segment, which is the system for transforming
    raw data into EDRs and is to be located at the four processing centers;
    (3) the command, control, and communications segment, which includes the
    equipment and services needed to support satellite operations; and (4) the
    launch segment, which includes the launch vehicle services.

    When the NPOESS engineering, manufacturing, and development contract was
    awarded in August 2002, the cost estimate was adjusted to $7 billion.
    Acquisition plans called for the procurement and launch of six satellites
    over the life of the program, as well as the integration of 13
    instruments--consisting of 10 environmental sensors and three subsystems.
    Together, the sensors were to receive and transmit data on atmospheric,
    cloud cover, environmental, climatic, oceanographic, and solar-geophysical
    observations. The subsystems were to support nonenvironmental search and
    rescue efforts, sensor survivability, and environmental data collection
    activities. The program office considered 4 of the sensors to be critical
    because they provide data for key weather products; these sensors are in
    bold in table 1, which describes each of the expected NPOESS instruments.

    Table 1: Expected NPOESS Instruments as of August 31, 2004 (critical
    sensors are in bold)

    Source: GAO, based on NPOESS Integrated Program Office data.

    In addition, NPP was planned as a demonstration satellite to be launched
    several years before the first NPOESS satellite in order to reduce the
    risk associated with launching new sensor technologies and to ensure
    continuity of climate data with NASA's Earth Observing System satellites.
    NPP is to host three of the four critical NPOESS sensors (VIIRS, CrIS, and
    ATMS), as well as one other noncritical sensor (OMPS). NPP is to provide
    the program office and the processing centers an early opportunity to work
    with the sensors, ground control, and data processing systems.

    When the NPOESS development contract was awarded, the schedule for
    launching the satellites was driven by a requirement that the satellites
    be available to back up the final POES and DMSP satellites should anything
    go wrong during the planned launches of these satellites. In general,
    satellite experts anticipate that roughly 1 out of every 10 satellites
    will fail either during launch or during early operations after launch.

    Early program milestones included (1) launching NPP by May 2006, (2)
    having the first NPOESS satellite available to back up the final POES
    satellite launch in March 2008, and (3) having the second NPOESS satellite
    available to back up the final DMSP satellite launch in October 2009. If
    the NPOESS satellites were not needed to back up the final predecessor
    satellites, their anticipated launch dates would have been April 2009 and
    June 2011, respectively.

    NPOESS Experienced Cost Increases, Schedule Delays, and Technical Problems over
    Several Years

    Over the last few years, NPOESS has experienced continued cost increases
    and schedule delays, requiring difficult decisions to be made about the
    program's direction and capabilities. In 2003, we reported that changes in
    the NPOESS funding stream caused a delay in the program's schedule.^4
    Specifically, in late 2002, DOD shifted the expected launch date for its
    final DMSP satellite from 2009 to 2010. As a result, the department
    reduced funding for NPOESS by about $65 million between fiscal years 2004
    and 2007. According to program officials, because NOAA was required to
    provide the same level of funding that DOD provides, this change triggered
    a corresponding reduction in funding by NOAA for those years. As a result
    of the reduced funding, program officials were forced to make difficult
    decisions about what to focus on first. The program office decided to keep
    NPP as close to its original schedule as possible because of its
    importance to the eventual NPOESS development and to shift some of the
    program's deliverables to later years. This shift affected the NPOESS
    deployment schedule. To plan for this shift, the program office developed
    a new program cost and schedule baseline.

    ^4 [40]GAO-03-987T .

    After this new baseline was completed in 2004, we reported that the
    program office increased the NPOESS cost estimate from about $7 billion to
    $8.1 billion; delayed key milestones, including the planned launch of the
    first NPOESS satellite--which was delayed by 7 months; and extended the
    life of the program from 2018 to 2020.^5 The cost increases reflected
    changes to the NPOESS contract, as well as increased program management
    funds. According to the program office, contract changes included
    extension of the development schedule, increased sensor costs, and
    additional funds needed for mitigating risks. Increased program management
    funds were added for noncontract costs and management reserves.

    At that time, we also noted that other factors could further affect the
    revised cost and schedule estimates. Specifically, the contractor was not
    meeting expected cost and schedule targets on the new baseline because of
    technical issues in the development of key sensors, including the critical
    VIIRS sensor. Based on its performance through May 2004, we estimated that
    the contractor would most likely overrun its contract at completion in
    September 2011 by $500 million--thereby increasing the projected life
    cycle cost to $8.6 billion. In addition, we reported that risks associated
    with the development of the critical sensors, integrated data processing
    system, and algorithms, among other things, could contribute to further
    cost increases and schedule slips--and we noted that continued oversight
    was critical. The program office's baseline cost estimate was subsequently
    adjusted to $8.4 billion.

    In mid-November 2005, we reported that NPOESS continued to experience
    problems in the development of a key sensor, resulting in schedule delays
    and anticipated cost increases.^6 At that time, we projected that the
    program's cost estimate had grown to about $10 billion based on contractor
    cost and schedule data. We reported that the program's issues were due, in
    part, to problems at multiple levels of management--including
    subcontractor, contractor, program office, and executive leadership.
    Recognizing that the budget for the program was no longer executable, the
    NPOESS Executive Committee planned to make a decision in December 2005 on
    the future direction of the program--what would be delivered, at what
    cost, and by when. This involved deciding among options involving
    increased costs, delayed schedules, and reduced functionality. We noted
    that continued oversight, strong leadership, and timely decision making
    were more critical than ever and we urged the committee to make a decision
    quickly so that the program could proceed.

    ^5 [41]GAO-04-1054 .

    ^6 [42]GAO-06-249T .

    However, we subsequently reported that, in late November 2005, NPOESS cost
    growth exceeded a legislatively mandated threshold that requires DOD to
    certify the program to Congress.^7 This placed any decision about the
    future direction of the program on hold until the certification took place
    in June 2006. In the meantime, the program office implemented an interim
    program plan for fiscal year 2006 to continue work on key sensors and
    other program elements using fiscal year 2006 funding.

    Nunn-McCurdy Process Led to a Decision to Restructure the NPOESS Program

    The Nunn-McCurdy law^8 requires DOD to take specific actions when a major
    defense acquisition program exceeds certain cost thresholds. In November
    2005, key provisions of the act required the Secretary of Defense to
    notify Congress when a major defense acquisition was expected to overrun
    its project baseline by 15 percent or more and to certify the program to
    Congress when it was expected to overrun its baseline by 25 percent or
    more.^9 At that time, NPOESS exceeded the 25 percent threshold, and DOD
    was required to certify the program. Certifying a program entailed
    providing a determination that (1) the program is essential to national
    security, (2) there are no alternatives to the program that will provide
    equal or greater military capability at less cost, (3) the new estimates
    of the program's cost are reasonable, and (4) the management structure for
    the program is adequate to manage and control costs. DOD established
    tri-agency teams--made up of DOD, NOAA, and NASA experts--to work on each
    of the four elements of the certification process.

    In June 2006, DOD (with the agreement of both of its partner agencies)
    certified a restructured NPOESS program, estimated to cost $12.5 billion
    through 2026. This decision approved a cost increase of $4 billion over
    the prior approved baseline cost and delayed the launch of NPP and the
    first two satellites by roughly 3 to 5 years. The new program also
    entailed establishing a stronger program management structure, reducing
    the number of satellites to be produced and launched from 6 to 4, and
    reducing the number of instruments on the satellites from 13 to
    9--consisting of 7 environmental sensors and 2 subsystems. It also
    entailed using NPOESS satellites in the early morning and afternoon orbits
    and relying on European satellites for midmorning orbit data.^10 Table 2
    summarizes the major program changes made under the Nunn-McCurdy
    certification decision.

    ^7 [43]GAO-06-573T .

    ^810 U.S.C. S 2433 is commonly referred to as Nunn-McCurdy.

    ^910 U.S.C. S 2433 (e)(2) has recently been amended by Pub. L. No.
    109-163, S 802 (Jan. 6, 2006) and Pub. L. No. 109-364, S 213 (a) (Oct. 17,
    2006).

    Table 2: Summary of Changes to the NPOESS Program

    Source: GAO analysis of NPOESS Integrated Program Office data.

    The Nunn-McCurdy certification decision established new milestones for the
    delivery of key program elements, including launching NPP by January
    2010,^11 launching the first NPOESS satellite (called C1) by January 2013,
    and launching the second NPOESS satellite (called C2) by January 2016.
    These revised milestones deviated from prior plans to have the first
    NPOESS satellite available to back up the final POES satellite should
    anything go wrong during that launch.

    ^10The European Organization for the Exploitation of Meteorological
    Satellite's MetOp program is a series of three polar-orbiting satellites
    dedicated to operational meteorology. MetOp satellites are planned to be
    launched sequentially over 14 years.

    ^11According to program officials, although the Nunn-McCurdy certification
    decision specifies NPP is to launch by January 2010, NASA plans to launch
    it by September 2009 to reduce the possibility of a climate data
    continuity gap.

    Delaying the launch of the first NPOESS satellite means that if the final
    POES satellite fails on launch, satellite data users would need to rely on
    the existing constellation of environmental satellites until NPP data
    becomes available--almost 2 years later. Although NPP was not intended to
    be an operational asset, NASA agreed to move NPP to a different orbit so
    that its data would be available in the event of a premature failure of
    the final POES satellite. However, NPP will not provide all of the
    operational capability planned for the NPOESS spacecraft. If the health of
    the existing constellation of satellites diminishes--or if NPP data is not
    available, timely, and reliable--then there could be a gap in
    environmental satellite data. Table 3 summarizes changes in key program
    milestones over time.

    Table 3: Key Program Milestones

    Source: GAO analysis, based on NPOESS Integrated Program Office data.

    ^aPOES and DMSP are not part of the NPOESS program. Their launch dates are
    provided to indicate the increased risk of satellite data gaps between
    when these systems launch and when the NPOESS satellites launch.

    ^bAlthough the certification decision specified NPP is to launch by
    January 2010, NASA plans to launch it by September 2009 to reduce the
    possibility of a gap in climate data continuity.

    In order to reduce program complexity, the Nunn-McCurdy certification
    decision decreased the number of NPOESS sensors from 13 to 9 and reduced
    the functionality of 4 sensors. Specifically, of the 13 original sensors,
    5 sensors remain unchanged, 3 were replaced with less capable sensors, 1
    was modified to provide less functionality, and 4 were cancelled. Table 4
    shows the changes to NPOESS sensors, including the 4 identified in bold as
    critical sensors.

    Table 4: Changes to NPOESS Instruments (critical sensors are in bold)

    Source: GAO analysis of NPOESS Integrated Program Office data.

    ^aWhile direct program funding for these instruments was eliminated, the
    instruments could be reintegrated on NPOESS satellites should other
    parties choose to fund them. The Nunn-McCurdy decision requires the
    program office to allow sufficient space on the spacecraft for these
    instruments and to provide the funding needed to integrate them.

    The changes in NPOESS sensors affected the number and quality of the
    resulting weather and environmental products, called EDRs. In selecting
    sensors for the restructured program, the Nunn-McCurdy process placed the
    highest priority on continuing current operational weather capabilities
    and a lower priority on obtaining selected environmental and climate
    measuring capabilities. As a result, the revised NPOESS system has
    significantly less capability for providing global climate measures than
    was originally planned. Specifically, the number of EDRs was decreased
    from 55 to 39, of which 6 are of a reduced quality. The 39 EDRs that
    remain include cloud base height, land surface temperature, precipitation
    type and rate, and sea surface winds. The 16 EDRs that were removed
    include cloud particle size and distribution, sea surface height, net
    solar radiation at the top of the atmosphere, and products to depict the
    electric fields in the space environment. The 6 EDRs that are of a reduced
    quality include ozone profile, soil moisture, and multiple products
    depicting energy in the space environment.

    Given the changes in planned sensors, program officials established a
    planned configuration for NPP and the four satellites of the NPOESS
    program, called C1, C2, C3, and C4 (see table 5). Program officials
    acknowledged that this configuration could change if other parties decided
    to develop the sensors that were cancelled. However, they noted that the
    planned configuration of the first satellite cannot change without
    increasing the risk that the launch will be delayed.

    Table 5: Planned Configuration of Sensors on NPP and NPOESS Satellites

    Source: GAO analysis of NPOESS Integrated Program Office data.

    Earned Value Management Techniques Provide Insight on Program Cost and Schedule

    To be effective, project managers need current information on a
    contractor's progress in meeting contract deliverables. One method that
    can help project managers track this progress is earned value management.
    This method, used by DOD for several decades, compares the value of work
    accomplished during a given period with that of the work expected in that
    period.

    Differences from expectations are measured in both cost and schedule
    variances. Cost variances compare the earned value of the completed work
    with the actual cost of the work performed. For example, if a contractor
    completed $5 million worth of work and the work actually cost $6.7
    million, there would be a -$1.7 million cost variance. Schedule variances
    are also measured in dollars, but they compare the earned value of the
    work completed with the value of work that was expected to be completed.
    For example, if a contractor completed $5 million worth of work at the end
    of the month but was budgeted to complete $10 million worth of work, there
    would be a -$5 million schedule variance. Positive variances indicate that
    activities are costing less or are completed ahead of schedule. Negative
    variances indicate activities are costing more or are falling behind
    schedule. These cost and schedule variances can then be used in estimating
    the cost and time needed to complete the program.

    NPOESS Acquisition Restructuring Is Well Under Way, but Key Steps Remain To Be
    Completed

    Since the June 2006 decision to revise the scope, cost, and schedule of
    the NPOESS program, the program office has made progress in restructuring
    the satellite acquisition; however, important tasks leading up to revising
    and finalizing contract changes remain to be completed. Restructuring a
    major acquisition program like NPOESS is a process that involves
    identifying time critical and high priority work and keeping this work
    moving forward, while reassessing development priorities,
    interdependencies, deliverables, risks, and costs. It also involves
    revising important acquisition documents including the memorandum of
    agreement on the roles and responsibilities of the three agencies, the
    acquisition strategy, the system engineering plan, the test and evaluation
    master plan, the integrated master schedule defining what needs to happen
    by when, and the acquisition program baseline. The Nunn-McCurdy
    certification decision required the Secretaries of Defense and Commerce
    and the Administrator of NASA to sign a revised memorandum of agreement by
    August 6, 2006. It also required that the program office, Program
    Executive Officer, and the Executive Committee revise and approve key
    acquisition documents including the acquisition strategy and system
    engineering plan by September 1, 2006, in order to proceed with the
    restructuring. Once these are completed, the program office can proceed to
    negotiate with its prime contractor on a new program baseline defining
    what will be delivered, by when, and at what cost.

    The NPOESS program office has made progress in restructuring the
    acquisition. Specifically, the program office has established interim
    program plans guiding the contractor's work activities in 2006 and 2007
    and has made progress in implementing these plans. Specifically, the
    program office reported that it had completed 156 of 166 key milestones^12
    during fiscal year 2006--including completing ambient and thermal vacuum
    testing of the VIIRS engineering unit. Of the 10 remaining milestones
    resulting from unanticipated problems in the development of VIIRS and
    CrIS, 5 have since been completed, and 5 are still pending. The program
    office plans to complete 222 milestones in fiscal year 2007--including
    completing performance tests on the OMPS (nadir) sensor--and notes that
    they are slightly ahead of plans in that they have completed 62 milestones
    through January 20, 2007, which is 2 more than had been planned. Figures 7
    and 8 depict the program office's progress against key milestones in
    fiscal year 2006 and to date in fiscal year 2007.

    ^12The NPOESS program office selected key milestones from a much larger
    set of ongoing and planned milestones in order to track progress.

    Figure 7: Progress on Planned Milestones in Fiscal Year 2006, as of
    October 1, 2006

    Figure 8: Progress on Planned Milestones in Fiscal Year 2007, as of
    January 20, 2007

    The program office has also made progress in revising key acquisition
    documents. It revised the system engineering plan, the test and evaluation
    master plan, and the acquisition strategy plan, and obtained approval of
    these documents by the Program Executive Officer. The program office and
    contractor also developed an integrated master schedule for the remainder
    of the program--beyond fiscal year 2007. This integrated master schedule
    details the steps leading up to launching NPP by September 2009, launching
    the first NPOESS satellite in January 2013, and launching the second
    NPOESS satellite in January 2016. Near-term steps include completing and
    testing the VIIRS, CrIS, and OMPS sensors; integrating these sensors with
    the NPP spacecraft and completing integration testing; completing the data
    processing system and integrating it with the command, control, and
    communications segment; and performing advanced acceptance testing of the
    overall system of systems for NPP.

    However, key steps remain for the acquisition restructuring to be
    completed. These steps include obtaining the approval of the Secretaries
    of Commerce and Defense and the Administrator of NASA on the memorandum of
    agreement among the three agencies, and obtaining the approval of the
    NPOESS Executive Committee on key acquisition documents, including the
    system engineering plan, the test and evaluation master plan, and the
    acquisition strategy. These approvals are currently over 6 months past
    due. Agency officials noted that the September 1, 2006, due date for the
    key acquisition documents was not realistic given the complexity of
    coordinating documents among three different agencies, but did not provide
    a new estimate for when these documents would be approved.

    Finalizing these documents is critical to ensuring interagency agreements
    and will allow the program office to move forward in completing other
    activities related to restructuring the program. These activities include
    conducting an integrated baseline review with the contractor to reach
    agreement on the schedule and work activities, and finalizing changes to
    the NPOESS development and production contract--thereby allowing the
    program office to lock down a new acquisition baseline cost and schedule.
    The program office expects to conduct an integrated baseline review by May
    2007 and to finalize the contract changes by July 2007. Until key
    acquisition documents are finalized and approved, the program faces
    increased risk that it will not be able to complete important
    restructuring activities in time to move forward in fiscal year 2008 with
    a new program baseline in place. This places the NPOESS program at risk of
    continued delays and future cost increases.

    Progress Has Been Made in Establishing an Effective NPOESS Management Structure,
    but Executive Turnover Will Increase Risks, and Staffing Problems Remain

    The NPOESS program has made progress in establishing an effective
    management structure, but--almost a year after this structure was endorsed
    during the Nunn-McCurdy certification process--the Integrated Program
    Office still faces staffing problems. Over the past few years, we and
    others have raised concerns about management problems at all levels of the
    NPOESS program, including subcontractor and contractor management, program
    office management, and executive-level management.^13 Two independent
    review teams also noted a shortage of skilled program staff, including
    budget analysts and system engineers. Since that time, the NPOESS program
    has made progress in establishing an effective management
    structure--including establishing a new organizational framework with
    increased oversight by program executives, instituting more frequent
    subcontractor, contractor, and program reviews, and effectively managing
    risks and performance. However, DOD's plans for reassigning the Program
    Executive Officer in Summer 2007 increase the program's risks.
    Additionally, the program lacks a staffing process that clearly identifies
    staffing needs, gaps, and plans for filling those gaps. As a result, the
    program office has experienced delays in getting core management
    activities under way and lacks the staff it needs to execute day-to-day
    management activities.

    NPOESS Program Has Made Progress in Establishing an Effective Management
    Structure and Increasing Oversight Activities, but Executive Turnover Will
    Increase Program Risks

    The NPOESS program has made progress in establishing an effective
    management structure and increasing the frequency and intensity of its
    oversight activities. Over the past few years, we and others have raised
    concerns about management problems at all levels of management on the
    NPOESS program, including subcontractor and contractor management, program
    office management, and executive-level management. In response to
    recommendations made by two different independent review teams, the
    program office began exploring options in late 2005 and early 2006 for
    revising its management structure.

    In November 2005, the Executive Committee established and filled a Program
    Executive Officer position, senior to the NPOESS Program Director, to
    streamline decision making and to provide oversight to the program. This
    Program Executive Officer reports directly to the Executive Committee.
    Subsequently, the Program Executive Officer and the Program Director
    proposed a revised organizational framework that realigned division
    managers within the Integrated Program Office responsible for overseeing
    key elements of the acquisition and increased staffing in key areas. In
    June 2006, the Nunn-McCurdy certification decision approved this new
    management structure and the Integrated Program Office implemented it.
    Figure 9 provides an overview of the relationships among the Integrated
    Program Office, the Program Executive Office, and the Executive Committee,
    as well as key divisions within the program office.

    ^13 [44]GAO-06-249T ; U.S. Department of Commerce, Office of the Inspector
    General, Poor Management Oversight and Ineffective Incentives Leave NPOESS
    Program Well Over Budget and Behind Schedule, OIG-17794-6-0001/2006
    (Washington, D.C.: May 2006). In addition, two independent teams reviewed
    the NPOESS program in 2005: A NASA-led Independent Review Team
    investigated problems with the VIIRS sensor and the impact on NPP, and a
    DOD-led Independent Program Assessment Team assessed the broader NPOESS
    program. The teams briefed the NPOESS Executive Committee on their
    findings in August 2005 and November 2005, respectively.

    Figure 9: Overview of New Management Structure

    Operating under this new management structure, the program office
    implemented more rigorous and frequent subcontractor, contractor, and
    program reviews, improved visibility into risk management and mitigation
    activities, and institutionalized the use of earned value management
    techniques to monitor contractor performance. Specifically, program
    officials and the prime contractor now review the subcontractors' cost and
    schedule performance on a weekly basis. The information from these
    meetings feeds into monthly government meetings with the prime contractor
    to review progress against milestones, issues, and risks. Further, the
    Program Director conducts monthly reviews with each technical division
    lead to review the divisions' achievements, risks, and plans. Program
    officials note that these frequent reviews allow information on risks to
    be quickly escalated from subcontractors to contractors, to the program
    component level, and to the Program Director--and they allow program
    officials to better manage efforts to reduce risks. The program office
    also reported that all division leads were trained in earned value
    management techniques and were effectively using these techniques both to
    monitor subcontractor and contractor performance on a weekly basis and to
    identify potential problems as soon as possible.

    In addition to these program office activities, the Program Executive
    Officer implemented monthly program reviews and increased the frequency of
    contacts with the Executive Committee. Specifically, the Program Executive
    Officer holds monthly program management reviews where the Program
    Director and program division leads (for example, those in charge of
    systems engineering or ground systems) provide briefings on the program's
    earned value, progress, risks, and concerns. We observed that these
    briefings allow the Program Executive Officer to have direct insight into
    the challenges and workings of the Integrated Program Office and allow
    risks to be appropriately escalated and addressed. These meetings also
    provide an open forum for managers to raise concerns and ask questions
    about operational challenges. For example, when NASA officials expressed
    concerns that vibration levels used during testing were higher than
    necessary and were causing damage to key sensor components, the Program
    Director and Program Executive Officer immediately established a forum to
    discuss and mitigate this issue. The Program Executive Officer briefs the
    Executive Committee in monthly letters, apprising committee members of the
    program's status, progress, risks, and earned value and the Executive
    Committee now meets on a quarterly basis--whereas in the recent past, we
    reported that the Executive Committee had met only five times in 2
    years.^14

    14 [45]GAO-06-249T .

    While the NPOESS program has made progress in establishing an effective
    management structure, this progress is currently at risk. We recently
    reported that DOD space acquisitions are at increased risk due in part to
    frequent turnover in leadership positions, and we suggested that
    addressing this will require DOD to consider matching officials' tenure
    with the development or delivery of a product.^15 In March 2007, NPOESS
    program officials stated that DOD is planning to reassign the recently
    appointed Program Executive Officer in Summer 2007 as part of this
    executive's natural career progression. As of March 2007, the Program
    Executive Officer has held this position for 16 months. Given that the
    program is currently still being restructured, and that there are
    significant challenges in being able to meet critical deadlines to ensure
    satellite data continuity, such a move adds unnecessary risk to an already
    risky program.

    NPOESS Program Has Filled Key Vacancies but Lacks A Programwide Staffing Process

    The NPOESS program office has filled key vacancies in recent months but
    lacks a staffing process that identifies programwide staffing requirements
    and plans for filling those needed positions. Sound human capital
    management calls for establishing a process or plan for determining
    staffing requirements, identifying any gaps in staffing, and planning to
    fill critical staffing gaps. Program office staffing is especially
    important for NPOESS, given the acknowledgment by multiple independent
    review teams that staffing shortfalls contributed to past problems.
    Specifically, these review teams noted shortages in the number of system
    engineers needed to provide adequate oversight of subcontractor and
    contractor engineering activities and in the number of budget and cost
    analysts needed to assess contractor cost and earned value reports. To
    rectify this situation, the June 2006 certification decision directed the
    Program Director to take immediate actions to fill vacant positions at the
    program office with the approval of the Program Executive Officer.

    Since the June 2006 decision to revise NPOESS management structure, the
    program office has filled multiple critical positions, including a budget
    officer, a chief system engineer, an algorithm division chief, and a
    contracts director. In addition, on an ad hoc basis, individual division
    managers have assessed their needs and initiated plans to hire individuals
    for key positions. However, almost a year after the certification, the
    program office still lacks a programwide process for identifying and
    filling all needed positions. As a result, division managers often wait
    months for critical positions to be filled. For example, in February 2006,
    the NPOESS program estimated that it needed to hire up to 10 new budget
    analysts. As of September 2006, none of these positions had been filled.
    Today, program officials estimate that they only needed to fill 7 budget
    analyst positions, of which 2 positions have been filled, and 5 remain
    vacant. Additionally, even though the certification decision directed
    immediate action to fill critical vacancies, the program still has
    vacancies in 5 systems engineering positions and 10 technical manager
    positions. The majority of the vacancies--4 of the 5 budget positions, 4
    of the 5 systems engineering positions, and 8 of the 10 technical manager
    positions--are to be provided by NOAA. NOAA officials noted that each of
    these positions is in some stage of being filled--that is, recruitment
    packages are being developed or reviewed, vacancies are being advertised,
    or candidates are being interviewed, selected, and approved.

    ^15GAO, Space Acquisitions: Improvements Needed in Space Acquisitions and
    Keys to Achieving Them, [46]GAO-06-626T (Washington, D.C.: Apr. 6, 2006).

    The program office attributes its staffing delays to not having the right
    personnel in place to facilitate this process--and did not even begin to
    develop a staffing process--until November 2006. Program officials noted
    that the tri-agency nature of the program adds unusual layers of
    complexity to the hiring and administrative functions because each agency
    has its own hiring and performance management rules. In November 2006, the
    program office brought in an administrative officer who took the lead in
    pulling together the division managers' individual assessments of needed
    staff--currently estimated to be 25 vacant positions--and has been working
    with the division managers to refine this list. This new administrative
    officer plans to train division managers in how to assess their needs and
    to hire needed staff and to develop a process by which evolving needs are
    identified and positions are filled. However, there is as yet no date set
    for establishing this basic programwide staffing process.

    As a result of the lack of a programwide staffing process, there has been
    an extended delay in determining what staff are needed and in bringing
    those staff on board--which has resulted in delays in performing core
    management activities such as establishing the program office's cost
    estimate and bringing in needed contracting expertise. Additionally, until
    a programwide staffing process is in place, the program office risks not
    having the staff it needs to execute day-to-day management activities.

    Methodology Supporting the June 2006 Cost and Schedule Estimate Was Reliable,
    but Recent Events Could Increase Program Costs

    In June 2006, DOD certified a restructured NPOESS program that was
    estimated to cost $11.5 billion for the acquisition portion of the
    program^16 and scheduled to launch the first satellite in 2013. The Office
    of the Secretary of Defense's Cost Analysis Improvement Group (cost
    analysis group)--the independent cost estimators charged with developing
    the estimate for the acquisition portion of the program--used an
    acceptable methodology to develop this estimate. When combined with an
    estimated $1 billion for operations and support after launch, this brings
    the program life cycle cost to $12.5 billion. Recent events, however,
    could further increase program costs or delay schedules. Specifically, the
    program continues to experience technical problems on key sensors, and
    costs and schedules will be adjusted during negotiations on contract
    changes. The NPOESS program office is developing its own cost estimate to
    refine the one developed in June 2006 that it will use to negotiate
    contract changes. A new baseline cost will be established once the
    contract is finalized.

    Certified Program Estimates Were Developed Using an Acceptable Methodology

    The cost and schedule estimate for the restructured NPOESS program was
    developed by DOD's cost analysis group using an acceptable methodology.
    Cost-estimating organizations throughout the federal government and
    industry use certain key practices--related to planning, conducting, and
    reporting the estimate--to ensure a sound estimate. Table 6 lists the
    elements of a sound cost estimating methodology. In addition, to ensure
    the validity of the data assumptions that go into the estimate, leading
    organizations use actual historical costs and seek an independent
    validation of critical cost drivers.

    ^16The acquisition portion of the program includes satellite development,
    production, and launch. It does not include operations and support costs
    after launch.

    Table 6: Elements of a Sound Cost Estimating Methodology

    Source: GAO analysis of leading practices.

    DOD's cost analysis group used an acceptable methodology in developing the
    NPOESS cost estimate in that they planned, conducted, and reported the
    estimate consistent with leading practices. The cost analysis group's cost
    estimating approach was largely driven by the program's principal "ground
    rule" to maintain the continuity of weather data without a gap.
    Specifically, the cost analysis group assessed two risks: (1) the
    uncertainty of the health of the current polar-satellite constellation and
    (2) the uncertainty of when the new satellite system could be delivered
    (including the time needed to evaluate new satellites once in orbit). The
    resulting analysis showed that the restructured NPOESS system could be
    delivered and the first satellite launched by 2013 with a high level of
    confidence in maintaining satellite data continuity.^17

    To determine specific costs, the group used the existing work breakdown
    structure employed by the program office as the basis for performing its
    work. This work breakdown structure consists of seven major elements,
    including ground systems; spacecraft; sensors; assembly, integration and
    test; system engineering/contractor program management; government program
    management; and launch.

    ^17The cost analysis group determined that there was a 90 percent
    confidence level that there would be no weather coverage gap.

    The cost analysis group also took steps to ensure the validity of the data
    that went into the estimate. For each element, the cost analysis group
    visited all major contractor sites to collect program data including

               o schedule (including the original, rebaselined, and current
               schedules, and risks affecting the current schedule);
               o current staffing profile by month;
               o the history of staffing used;
               o the qualifications of people charging the program;
               o the program's technical approaches;
               o system diagrams;
               o bills of materials;
               o funding profile; and
               o the contractor's program legacy (a justification that the
               contractor has worked on similar projects in the past and that the
               contractor should be able to adapt that knowledge to the current
               work).

               The cost analysis group also compared this data with contractor
               labor rates from the Defense Contract Management Agency and
               obtained NASA's validation of the costs associated with the most
               significant cost driver, the VIIRS sensor.

               Since schedule was the primary uncertainty factor in the cost
               analysis, it also was the driver of overall costs. Specifically,
               the cost analysis group took its risk-adjusted schedule durations
               for the major cost elements and adjusted the contractor-submitted
               manning profiles accordingly. They then used NPOESS historical
               data on labor rates and materials to calculate the cost of these
               elements.

               Consistent with DOD practice, the cost analysis group established
               its cost estimate at a 50 percent confidence level.^18 However,
               cost analysts could not provide an upper limit for potential cost
               growth, explaining that the program contains "failsafe" measures
               to use alternative technologies (such as using legacy systems) if
               schedules are delayed and costs increase. As a result, cost
               analysts reported that they have a high level of confidence that
               acquisition costs will not exceed $11.5 billion--but a lower level
               of confidence that the configuration of sensors will remain
               unchanged.

    ^18A 50 percent level of confidence indicates that NPOESS has a 50 percent
    chance that the restructured program (as defined in the Nunn-McCurdy
    certification decision) will be delivered as planned at the acquisition
    cost of $11.5 billion.

               Recent Events Could Lead to Increased Program Costs or Delayed
    			  Schedules

               While the June 2006 cost estimate for the acquisition portion of
               the program was reasonable at the time it was made, several recent
               events could cause program life cycle costs to grow or schedules
               to be delayed. Specifically, the program continues to experience
               technical problems on key sensors. The CrIS sensor being developed
               for the NPP satellite suffered a major structural failure in
               October 2006. A failure review board is currently working to
               resolve the root causes of the failure. While program officials
               note that they should be able to cover costs related to
               investigating the problem, the full cost and schedule to fix the
               sensor is not yet known. Also, VIIRS development, which has been
               the program's primary cost driver, is not yet complete and
               continues to be a high-risk development. This too, could lead to
               higher final program costs or delayed schedules.

               Program costs are also likely to be adjusted during upcoming
               negotiations on contract changes. The NPOESS program office is
               developing its own cost estimate to refine the one developed in
               June 2006. Program officials plan to use this revised cost
               estimate to negotiate contract changes. A new baseline cost will
               be established once the contract is finalized--an event that the
               Program Director expects to occur by July 2007.

    			  Major Program Segments Are Under Development, but Significant Risks
    			  Remain

               Major segments of the NPOESS program--the space segment, the
               ground systems segment, and the launch segment--are under
               development; however, significant problems have occurred and risks
               remain. The program office is aware of these risks and is working
               to mitigate them, but continued problems could affect the
               program's overall cost and schedule. Given the tight time frames
               for completing key sensors, integrating them on the NPP
               spacecraft, and getting the ground-based data processing systems
               developed, tested, and deployed, it will be important for the
               NPOESS Integrated Program Office, the Program Executive Office,
               and the Executive Committee to continue to provide close oversight
               of milestones and risks.

    			  Space Segment--Progress Made, but Key Sensors Continue to Face
    			  Major Risks

               The space segment includes the sensors and the spacecraft. Four
               sensors are of critical importance--VIIRS, CrIS, OMPS, and
               ATMS--because they are to be launched on the NPP satellite.
               Initiating work on another sensor, the Microwave imager/sounder,
               is also important because this new sensor--replacing the cancelled
               CMIS sensor--will need to be developed in time for the second
               NPOESS satellite launch. Over the past year, the program made
               progress on each of the sensors and the spacecraft. However, two
               sensors, VIIRS and CrIS, have experienced major problems. The
               status of each of the components of the space segment is described
               in table 7.

    Table 7: Status of Selected Components of the Space Segment, as of April
    2007

    Source: GAO analysis of NPOESS Integrated Program Office data.

    ^aThe three problems are (1) band-to-band co-registration, an issue in
    which band registration shifts with different temperatures; (2)
    cross-talk, which involves information from sensor cells leaking into
    other cells; and (3) line-spread function issues, in which the
    instrument's focus changes with changes in temperature.

      Earned Value Data Show Problems on the Space Segment

    Earned value management tools are used to compare the value of work
    accomplished with the work expected during a given time period, and any
    differences are measured in cost and schedule variances. The NPOESS space
    segment experienced negative cost and schedule variances between January
    2006 and January 2007 (see fig. 10).

    Figure 10: Cumulative Cost and Schedule Variance for the NPOESS Space
    Segment over a 13-Month Period

    From January 2006 to January 2007, the contractor exceeded cost targets
    for the space segment by $17 million--which is 4 percent of the space
    segment budget for that time period. Similarly, the contractor was unable
    to complete $14.6 million worth of work in the space segment. The main
    factors in the cost and schedule variances were due to underestimation of
    the scope of work, pulling resources from lower priority tasks to higher
    priority items, and unforeseen design issues on key sensors. For example,
    VIIRS continued to experience negative cost variance trends due to
    unplanned efforts, which included refurbishing and recertifying the VIIRS
    calibration chamber, completing the testing of the engineering design
    unit, and resolving a problem with the testing equipment needed to adjust
    VIIRS' temperature during a key test.

    Unplanned efforts for CrIS that attributed to the negative cost and
    schedule variances included additional time required for testing and
    material management. The schedule variances for VIIRS and CrIS were mainly
    due to resources being pulled from other areas to support higher priority
    tasks, extended testing and testing delays, management changes, and
    improper material handling. Further, there is a high likelihood that CrIS
    will continue to experience cost and schedule variances against the fiscal
    year 2007 interim program plan until the issues that caused its structural
    failure are addressed.

      Program Office Is Monitoring Sensor Risks and Evaluating Options

    Program officials regularly track risks associated with various NPOESS
    components and work to mitigate them. Having identified both VIIRS and
    CrIS as high risk, OMPS as a moderate risk, and the other components as
    low risk, the program office is working closely with the contractors and
    subcontractors to resolve sensor problems.

    Program officials have identified work-arounds that will allow them to
    move forward in testing the VIIRS engineering unit and have approved the
    flight unit to proceed to a technical readiness review milestone in May
    2007. Regarding CrIS, as of March 2007, a failure review board identified
    root causes of its structural failure, identified plans for resolving
    them, and initiated inspections of sensor modules and subsystems for
    damage. An agency official reported that there is sufficient funding in
    the fiscal year 2007 program office's and contractor's management reserve
    funds to allow for troubleshooting both VIIRS and CrIS problems. However,
    until the CrIS failure review board fully determines the amount of rework
    that is necessary to fix the problems, it is unknown if additional funds
    will be needed or if the time frame for CrIS' delivery will be delayed.
    According to agency officials, CrIS is not on the program schedule's
    critical path, and there is sufficient schedule margin to absorb the time
    it will take to conduct a thorough failure review process.

    Managing the risks associated with the development of VIIRS and CrIS are
    of particular importance because these are to be demonstrated on the NPP
    satellite currently scheduled for launch in September 2009. Additionally,
    any delay in the NPP launch date could affect the overall NPOESS program
    because the success of the program depends on the lessons learned in data
    processing and system integration from the NPP satellite.

    Ground Segment--Progress Has Been Made, but Work Remains

    Development of the ground segment--which includes the interface data
    processing system, the ground stations that are to receive satellite data,
    and the ground-based command, control, and communications system--is under
    way and on track. However, important work pertaining to developing the
    algorithms that translate satellite data into weather products within the
    integrated data processing segment remains to be completed. Table 8
    describes each of the components of the ground segment and identifies the
    status of each. Additionally, appendix II provides an overview of
    satellite data processing algorithms.

    Table 8: Status of Ground Segment Components

    Source: GAO analysis of NPOESS Integrated Program Office data.

    Ground Segment Cost and Schedule Are on Track; Work and Risks Remain

    Using contractor-provided data, our analysis indicates cost and schedule
    performance on key elements of the NPOESS ground segment were generally on
    track or positive against the fiscal year 2006 and 2007 interim program
    plans. For the IDPS component, the contractor completed slightly less work
    than planned and finished slightly under budget. This caused cost and
    schedule variances of less than 1 percent off of expectations. (see fig.
    11). For the command, control, and communications component, the
    contractor was able to outperform its planned targets by finishing under
    budget by $3 million (6.2 percent of the budget for this time period) and
    by completing $31,000 (less than 1 percent) worth of work beyond what was
    planned (see fig. 12).

    Figure 11: Cumulative Cost and Schedule Variance for the NPOESS IDPS
    Development over a 13-Month Period

    Figure 12: Cumulative Cost and Schedule Variance for the NPOESS Command,
    Control, and Communications Segment over a 13-Month Period

      Program Office Has Plans to Address IDPS Risks

    The NPOESS program office plans to continue to address risks facing IDPS
    development. Specifically, the IDPS team is working to reduce data
    processing delays by seeking to limit the number of data calls, improve
    the efficiency of the data management system, increase the efficiency of
    the algorithms, and increase the number of processors. The program office
    also developed a resource center consisting of a logical technical
    library, a data archive, and a set of analytical tools to coordinate,
    communicate, and facilitate the work of algorithm subject matter experts
    on algorithm development and calibration/validation preparations. Managing
    the risks associated with the development of the IDPS system is of
    particular importance because this system will be needed to process NPP
    data.

    Launch Segment--NPP Launch Preparation Has Begun, while NPOESS Launch Planning
    Remains a Future Event

    Different agencies are responsible for launching NPP and NPOESS. NASA is
    responsible for the NPP launch and began procuring the launch vehicle for
    NPP in August 2006. Program officials expect to have it delivered by July
    2009, less than 2 months before the scheduled NPP launch in September
    2009.

    The NPOESS Integrated Program Office is responsible for launching the
    NPOESS satellites. According to program officials, the Air Force is to
    procure launch services for the program through DOD's Evolved Expendable
    Launch Vehicle contract. These services are to be procured by January
    2011, 2 years before the first scheduled launch.

    Conclusions

    NPOESS restructuring is well under way, and the program has made progress
    in establishing an effective management structure. However, key steps
    remain in restructuring the acquisition, including completing important
    acquisition documents such as the system engineering plan, the acquisition
    program baseline, and the memorandum of agreement documenting the three
    agencies' roles and responsibilities. Until these key documents are
    finalized, the program is unable to finalize plans for restructuring the
    program. Additionally, the program office continues to have difficulty
    filling key positions and lacks a programwide staffing process. Until the
    program establishes an effective and repeatable staffing process, it will
    have difficulties in identifying and filling its staffing needs in a
    timely manner. Having insufficient staff in key positions impedes the
    program office's ability to conduct important management and oversight
    activities, including revising cost and schedule estimates, monitoring
    progress, and managing technical risks. The program faces even further
    challenges if DOD proceeds with plans to reassign the Program Executive
    Officer this summer. Such a move would add unnecessary risk to an already
    risky program.

    In addition, the likelihood exists that there will be further cost
    increases and schedule delays because of technical problems on key sensors
    and pending contract negotiations. Major program segments--including the
    space and ground segments--are making progress in their development and
    testing. However, two critical sensors have experienced problems and are
    considered high risk, and risks remain in developing and implementing the
    ground-based data processing system. Given the tight time frames for
    completing key sensors, integrating them, and getting the ground-based
    data processing systems developed, tested, and deployed, continued close
    oversight of milestones and risks is essential to minimize potential cost
    increases and schedule delays.

    Recommendations for Executive Action

    Because of the importance of effectively managing the NPOESS program to
    ensure that there are no gaps in the continuity of critical weather and
    environmental observations, we are making recommendations to the
    Secretaries of Defense and Commerce and to the Administrator of NASA to
    ensure that the responsible executives within their respective
    organizations approve key acquisition documents, including the memorandum
    of agreement among the three agencies, the system engineering plan, the
    test and evaluation master plan, and the acquisition strategy, as quickly
    as possible but no later than April 30, 2007.

    We are also recommending that the Secretary of Defense direct the Air
    Force to delay reassigning the recently appointed Program Executive
    Officer until all sensors have been delivered to the NPOESS Preparatory
    Program; these deliveries are currently scheduled to occur by July 2008.

    We are also making two additional recommendations to the Secretary of
    Commerce. We recommend that the Secretary direct the Undersecretary of
    Commerce for Oceans and Atmosphere to ensure that NPOESS program
    authorities develop and implement a written process for identifying and
    addressing human capital needs and for streamlining how the program
    handles the three different agencies' administrative procedures, and
    establish a plan for immediately filling needed positions.

    Agency Comments and Our Evaluation

    We received written comments on a draft of this report from the Deputy
    Secretary of the Department of Commerce (see app. III), the Deputy
    Assistant Secretary for Networks and Information Integration of the
    Department of Defense (see app. IV), and the Deputy Administrator of the
    National Aeronautics and Space Administration (see app. V). All three
    agencies agreed that it was important to finalize key acquisition
    documents in a timely manner, and DOD proposed extending the due dates for
    the documents to July 2, 2007. Because the NPOESS program office intends
    to complete contract negotiations by July 4, 2007, we remain concerned
    that any further delays in approving the documents could delay contract
    negotiations and thus increase the risk to the program.

    In addition, the Department of Commerce agreed with our recommendation to
    develop and implement a written process for identifying and addressing
    human capital needs and to streamline how the program handles the three
    different agencies' administrative procedures. The department also agreed
    with our recommendation to plan to immediately fill open positions at the
    NPOESS program office. Commerce noted that NOAA identified the skill sets
    needed for the program and has implemented an accelerated hiring model and
    schedule to fill all NOAA positions in the NPOESS program. The department
    also stated that the Program Director will begin presenting the detailed
    staffing information at monthly program management reviews, including
    identifying any barriers and recommended corrective actions. Commerce also
    noted that NOAA has made NPOESS hiring a high priority and has documented
    a strategy--including milestones--to ensure that all 20 needed positions
    are filled by June 2007.

    DOD did not concur with our recommendation to delay reassigning the
    Program Executive Officer, noting that the NPOESS System Program Director
    responsible for executing the acquisition program would remain in place
    for 4 years. The Department of Commerce also noted that the Program
    Executive Officer position is planned to rotate between the Air Force and
    NOAA. Commerce also stated that a selection would be made prior to the
    departure of the current Program Executive Officer to provide an overlap
    period to allow for knowledge transfer and ensure continuity.

    However, over the last few years, we and others (including an independent
    review team and the Commerce Inspector General) have reported that
    ineffective executive-level oversight helped foster the NPOESS program's
    cost and schedule overruns. We remain concerned that reassigning the
    Program Executive at a time when NPOESS is still facing critical cost,
    schedule, and technical challenges will place the program at further risk.

    While it is important that the System Program Director remain in place to
    ensure continuity in executing the acquisition, this position does not
    ensure continuity in the functions of the Program Executive Officer. The
    current Program Executive Officer is experienced in providing oversight of
    the progress, issues, and challenges facing NPOESS and coordinating with
    Executive Committee members, as well as DOD authorities responsible for
    executing Nunn-McCurdy requirements. Additionally, while the Program
    Executive Officer position is planned to rotate between agencies, the
    memorandum of agreement documenting this arrangement is still in draft and
    should be flexible enough to allow the current Program Executive Officer
    to remain until critical risks have been addressed.

    Further, while Commerce plans to allow a period of overlap between the
    selection of a new Program Executive Officer and the departure of the
    current one, time is running out. The current Program Executive Officer is
    expected to depart in early July 2007 and, as of mid-April 2007, a
    successor has not yet been named. NPOESS is an extremely complex
    acquisition, involving three agencies, multiple contractors, and advanced
    technologies. There is not sufficient time to transfer knowledge and
    develop the sound professional working relationships that the new Program
    Executive Officer will need to succeed in that role. Thus, we remain
    convinced that given NPOESS's current challenges, reassigning the current
    Program Executive Officer at this time would not be appropriate.

    All three agencies also provided technical comments, which we have
    incorporated in this report as appropriate.

    As agreed with your offices, unless you publicly announce the contents of
    this report earlier, we plan no further distribution until 30 days from
    the report date. At that time, we will send copies of this report to
    interested congressional committees, the Secretary of Commerce, the
    Secretary of Defense, the Administrator of NASA, the Director of the
    Office of Management and Budget, and other interested parties. In
    addition, this report will be available at no charge on our Web site at
    http://www.gao.gov.

    If you have any questions on matters discussed in this report, please
    contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact
    points for our Offices of Congressional Relations and Public Affairs may
    be found on the last page of this report. GAO staff who made major
    contributions to this report are listed in appendix VI.

    David A. Powner
    Director, Information Technology Management Issues

    Appendix I: Objectives, Scope, and Methodology

    Our objectives were to (1) evaluate the National Polar-orbiting
    Operational Environmental Satellite System (NPOESS) program office's
    progress in restructuring the acquisition; (2) evaluate the program
    office's progress in establishing an effective management structure; (3)
    assess the reliability of the new life cycle cost estimate and proposed
    schedule; and (4) identify the status and key risks facing the program's
    major segments (the launch, space, data processing, and ground control
    segments) and evaluate the adequacy of the program's efforts to mitigate
    these risks.

    To evaluate the NPOESS program office's progress in restructuring the
    acquisition program, we reviewed the program's Nunn-McCurdy certification
    decision memo and program documentation including status briefings and
    milestone progress reports. We also interviewed program office officials
    and attended conferences and senior-level management program review
    meetings to obtain information on the program's acquisition restructuring.

    To evaluate the program office's progress in establishing an effective
    management structure, we reviewed the Nunn-McCurdy decision memo for the
    program, as well as program documentation and briefings. We assessed the
    status of efforts to implement recommendations regarding the program's
    management structure, including the work of the team responsible for
    reviewing the management structure under the Nunn-McCurdy review. We also
    analyzed the program office's organizational charts and position
    vacancies. Finally, we interviewed officials responsible for reviewing the
    management structure of the program under Nunn-McCurdy, attended
    senior-level management review meetings to obtain information related to
    the program's progress in establishing and staffing the new management
    structure, and interviewed program office officials responsible for human
    capital issues to obtain clarification on plans and goals for the new
    management structure.

    To assess the reliability of the new life cycle cost estimate and proposed
    schedule, we analyzed the Office of the Secretary of Defense's Cost
    Analysis Improvement Group's (cost analysis group) cost estimating
    methodology and the assumptions used to develop its independent cost
    estimate. Specifically, we assessed the cost estimating group's
    methodology against 12 best practices recognized by cost-estimating
    organizations within the federal government and industry for the
    development of reliable cost estimates. These best practices are also
    contained in a draft version of our cost guide, which is currently being
    developed by GAO cost experts. We also assessed cost- and schedule-related
    data, including the work breakdown structure and detailed schedule risk
    analyses to determine the reasonableness of the cost analysis group's
    assumptions. We also interviewed cost analysis group officials to obtain
    clarification on cost and schedule estimates and their underlying
    assumptions. Further, we interviewed program officials to identify any
    assumptions that may have changed.

    To identify the status and key risks facing the program's major segments
    (the launch, space, data processing, and ground control segments) and to
    evaluate the adequacy of the program's efforts to mitigate these risks, we
    reviewed the program's Nunn-McCurdy certification decision memo and other
    program documentation. We analyzed briefings and monthly program
    management documents to determine the status and risks of the key program
    segments. We also analyzed earned value management data obtained from the
    contractor to assess the contractor's performance to cost and schedule. We
    reviewed cost reports and program risk management documents and
    interviewed program officials to determine the program segments' risks
    that could negatively affect the program's ability to maintain the current
    schedule and cost estimates. We also interviewed agency officials from the
    National Aeronautics and Space Administration (NASA), the National Oceanic
    and Atmospheric Administration (NOAA), the Department of Defense (DOD),
    and the NPOESS program office to determine the status and risks of the key
    program segments. Finally, we observed senior-level management review
    meetings and attended conferences to obtain information on the status of
    the NPOESS program.

    We performed our work at the NPOESS Integrated Program Office and at DOD,
    NASA, and NOAA offices in the Washington, D.C., metropolitan area between
    July 2006 and April 2007 in accordance with generally accepted government
    auditing standards.

    Appendix II: Overview of Satellite Data Processing Algorithms and the
    Calibration and Validation Process

    Algorithms are sets of instructions, expressed mathematically, that
    translate satellite sensor measurements into usable information. In the
    NPOESS program, government contractors are responsible for algorithm
    development; the program office is responsible for independently
    validating the algorithms. Scientists develop these algorithms, which are
    then written as computer code to be incorporated into the interface data
    processing system (IDPS) operational system. The NPOESS ground system uses
    three primary types of algorithms:

               o Algorithms to develop raw data records "unpack" the digital
               packets received by the antennas/IDPS (the ones and zeros) and
               sent from the satellite, associate the data with the information
               about the satellite's location and, finally, translate it back
               into the data it was when it started at the sensor.
               o Algorithms used to develop sensor and temperature data records
               allow the on-ground users to understand what the sensor saw. It
               translates the information from the sensor into a measure of the
               various forms of energy (e.g., brightness, temperature, radiance).
               o Algorithms used to produce the weather products called
               environmental data records (EDR) are crosscutting. They combine
               various data records, as well as other data, in order to produce
               measures useful to scientists. Additionally, EDRs can be
               "chained"--that is, the output of one EDR algorithm will become an
               input into the next EDR algorithm. To illustrate, cloud
               detection/mask is an important "base" EDR because many EDRs, like
               sea surface temperature, are only calculated when clouds are not
               present. Figure 13 shows the flow of the data and algorithms.

    Figure 13: Satellite Data Processing Algorithms

    A corollary to algorithm development is the calibration and validation
    process. According to a senior algorithm scientist, in this process, once
    the satellite has been launched, scientists verify that the sensors
    accurately report what ground conditions are. For example, one EDR from
    the visible/infrared imager radiometer suite (VIIRS) is "ocean color."
    Once the sensor is in orbit, scientists can compare the results that the
    VIIRS sensor reports on ocean color with the known results from sensors on
    ocean buoys that also measure ocean color in select locations. Then, if
    the sensors do not accurately report the ground conditions, scientists can
    calibrate, or "tweak," the algorithms used to develop sensor, temperature,
    and environmental data records to report on ground conditions more
    accurately.

    According to an agency official, fully calibrating a simple sensor once it
    has been launched can take approximately a year. A more complicated sensor
    can take 18 months to 2 years (see fig. 14).

    Figure 14: Describes the High-Level Calibration and Validation Process

    Appendix III: Comments from the Department of Commerce

    Appendix IV: Comments from the Department of Defense

    Appendix V: Comments from the National Aeronautics and Space
    Administration

    Appendix VI: GAO Contact and Staff Acknowledgments

    GAO Contact

    David A. Powner, (202) 512-9286, or pownerd@gao.gov

    Staff Acknowledgments

    In addition to the contact named above, Colleen Phillips, Assistant
    Director; Carol Cha; Neil Doherty; Nancy Glover; Kathleen S. Lovett; Karen
    Richey; and Teresa Smith made key contributions to this report.

    (310821)

    GAO's Mission

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    Obtaining Copies of GAO Reports and Testimony

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
    newly released reports, testimony, and correspondence on its Web site. To
    have GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000
    TDD: (202) 512-2537
    Fax: (202) 512-6061

    To Report Fraud, Waste, and Abuse in Federal Programs

    Contact:

    Web site: www.gao.gov/fraudnet/fraudnet.htm
    E-mail: fraudnet@gao.gov
    Automated answering system: (800) 424-5454 or (202) 512-7470

    Congressional Relations

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Washington,
    D.C. 20548

    Public Affairs

    Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
    U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

    www.gao.gov/cgi-bin/getrpt?GAO-07-498 .

    To view the full product, including the scope
    and methodology, click on the link above.

    For more information, contact David A. Powner, (202) 512-9286, and
    pownerd@gao.gov.

    Highlights of [55]GAO-07-498 , a report to congressional requesters

    April 2007

    POLAR-ORBITING OPERATIONALENVIRONMENTAL SATELLITES

    Restructuring Is Under Way, but Technical Challenges and Risks Remain

    The National Polar-orbiting Operational Environmental Satellite System
    (NPOESS) is a tri-agency acquisition--managed by the Departments of
    Commerce and Defense and the National Aeronautics and Space
    Administration--which experienced escalating costs, schedule delays, and
    technical difficulties. These factors led to a June 2006 decision to
    restructure the program thereby decreasing the program's complexity,
    increasing its estimated cost to $12.5 billion, and delaying the first two
    satellites by 3 to 5 years. GAO was asked to (1) assess progress in
    restructuring the acquisition, (2) evaluate progress in establishing an
    effective management structure, (3) assess the reliability of the cost and
    schedule estimate, and (4) identify the status and key risks facing the
    program's major segments. To do so, GAO analyzed program and contractor
    data, attended program reviews, and interviewed program officials.

    [56]What GAO RecommendsGAO recommends that the appropriate executives
    approve key acquisition documents, the Secretary of Defense delay
    reassigning the Program Executive, and the Secretary of Commerce ensure
    that program authorities identify and address staffing needs. Agency
    officials agreed with all of the recommendations except delaying the
    Program Executive's reassignment. GAO believes that proceeding with this
    reassignment would increase program risks.

    The NPOESS program office has made progress in restructuring the
    acquisition by establishing and implementing interim program plans guiding
    the contractors' work activities in 2006 and 2007; however, important
    tasks leading up to finalizing contract changes remain to be completed.
    Executive approvals of key acquisition documents are about 6 months
    late--due in part to the complexity of navigating three agencies' approval
    processes. Delays in finalizing these documents could hinder plans to
    complete contract negotiations by July 2007 and could keep the program
    from moving forward in fiscal year 2008 with a new program baseline.

    The program office has also made progress in establishing an effective
    management structure by adopting a new organizational framework with
    increased oversight from program executives and by instituting more
    frequent and rigorous program reviews; however, plans to reassign the
    recently appointed Program Executive Officer will likely increase the
    program's risks. Additionally, the program lacks a process and plan for
    identifying and filling staffing shortages, which has led to delays in key
    activities such as cost estimating and contract revisions. Until this
    process is in place the NPOESS program faces increased risk of further
    delays.

    The methodology supporting a June 2006 independent cost estimate with the
    expectation of initial satellite launch in January 2013 was reliable, but
    recent events could increase program costs and delay schedules.
    Specifically, the program continues to experience technical problems on
    key sensors and program costs will likely be adjusted during upcoming
    negotiations on contract changes. A new baseline cost and schedule
    reflecting these factors is expected by July 2007.

    Development and testing of major NPOESS segments--including key sensors
    and ground systems--are under way, but significant risks remain. For
    example, while work continues on key sensors, two of them experienced
    significant problems and are considered high risk (see table).
    Additionally, while progress has been made in reducing delays in the data
    processing system, work remains in refining the algorithms needed to
    translate sensor observations into useable weather products. Given the
    tight time frames for completing this work, it will be important for
    program officials and executives to continue to provide close oversight of
    milestones and risks.

    Key NPOESS Components and Corresponding Risk Levels

    Source: GAO analysis of NPOESS Integrated Program Office data.

    References

    Visible links
      35. http://www.gao.gov/cgi-bin/getrpt?GAO-06-573T
      36. http://www.gao.gov/cgi-bin/getrpt?GAO-06-249T
      37. http://www.gao.gov/cgi-bin/getrpt?GAO-04-1054
      38. http://www.gao.gov/cgi-bin/getrpt?GAO-03-987T
      39. http://www.gao.gov/cgi-bin/getrpt?GAO-02-684T
      40. http://www.gao.gov/cgi-bin/getrpt?GAO-03-987T
      41. http://www.gao.gov/cgi-bin/getrpt?GAO-04-1054
      42. http://www.gao.gov/cgi-bin/getrpt?GAO-06-249T
      43. http://www.gao.gov/cgi-bin/getrpt?GAO-06-573T
      44. http://www.gao.gov/cgi-bin/getrpt?GAO-06-249T
      45. http://www.gao.gov/cgi-bin/getrpt?GAO-06-249T
      46. http://www.gao.gov/cgi-bin/getrpt?GAO-06-626T
      55. http://www.gao.gov/cgi-bin/getrpt?GAO-07-498
    *** End of document. ***