    FROM DOC_DB.DOCUMENT D, DOC_DB.REPORTS R, DOC_DB.BACKGROUND B
    						 *
    ERROR at line 2:
    ORA-00942: table or view does not exist

    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-792

    TITLE:     WEATHER FORECASTING: National Weather Service Is Planning
    to Improve Service and Gain Efficiency, but Impacts of Potential
    Changes Are Not Yet Known

    DATE:   07/14/2006

    -----------------------------------------------------------------

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-792

         * Results in Brief
         * Background
              * NWS Office Structure: An Overview
              * NWS Relies on Key Systems and Technologies to Fulfill Its Mi
              * Next Generation Weather Radar (NEXRAD)
              * Automated Surface Observing System (ASOS)
                   * Operational Environmental Satellites
                   * Advanced Weather Interactive Processing System (AWIPS)
                   * Numerical Models
                   * Supercomputers
              * Previous Reports Focused on NWS Modernization Systems Risks
              * NWS Established Performance Goals and Tracks Progress agains
         * NWS Is Positioning Itself to Provide Better Service through
              * NWS Has Plans for Upgrading Its Systems, Models, and Computa
                   * NEXRAD
                   * ASOS
                   * AWIPS
                   * Numerical Models
                   * Supercomputers
              * NWS Appropriately Links Its System and Technical Upgrades to
         * NWS's Training Is Expected to Result in Forecast Service Imp
         * Changing Concept of Operations Could Affect Nationwide Offic
              * NWS Is Evaluating Changes to Its Current Operations
              * Impacts of New Concept of Operations Have Yet to Be Determin
         * Conclusions
         * Recommendations for Executive Action
         * Agency Comments
         * GAO Contact
         * Staff Acknowledgments
         * GAO's Mission
         * Obtaining Copies of GAO Reports and Testimony
              * Order by Mail or Phone
         * To Report Fraud, Waste, and Abuse in Federal Programs
         * Congressional Relations
         * Public Affairs

    Report to the Subcommittee on Environment, Technology, and Standards,
    Committee on Science, House of Representatives

    United States Government Accountability Office

    GAO

    July 2006

    WEATHER FORECASTING

    National Weather Service Is Planning to Improve Service and Gain
    Efficiency, but Impacts of Potential Changes Are Not Yet Known

    Weather Forecasting Weather Forecasting Weather Forecasting Weather
    Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
    Weather Forecasting Weather Forecasting Weather Forecasting Weather
    Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
    Weather Forecasting Weather Forecasting Weather Forecasting Weather
    Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
    Weather Forecasting Weather Forecasting Weather Forecasting Weather
    Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
    Weather Forecasting Weather Forecasting Weather Forecasting Weather
    Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
    Weather Forecasting Weather Forecasting Weather Forecasting Weather
    Forecasting Weather Forecasting Weather Forecasting Weather Forecasting

    GAO-06-792

    Contents

    Letter 1

    Results in Brief 2
    Background 3
    NWS Is Positioning Itself to Provide Better Service through Upgrades to
    Its Systems and Technologies 20
    NWS's Training Is Expected to Result in Forecast Service Improvements, but
    the Training Selection Process Lacks Sufficient Oversight 27
    Changing Concept of Operations Could Affect Nationwide Office
    Configuration, but Impact on Forecast Services, Staffing, and Budget Is
    Not Yet Known 28
    Conclusions 31
    Recommendations for Executive Action 31
    Agency Comments 31
    Appendix I Objectives, Scope, and Methodology 33
    Appendix II NWS Performance Goals for Fiscal Years 2005 to 2011 34
    Appendix III NWS Previously Used A Stringent Process to Ensure Service Was
    Not Degraded 36
    Appendix IV Comments from the Department of Commerce 37
    Appendix V GAO Contact and Staff Acknowledgments 40

    Tables

    Table 1: NWS's Performance Measures, Goals, and Actual Performance for
    Fiscal Years (FY) 2005, 2006, and 2011 18
    Table 2: Ongoing and Planned NEXRAD Improvements (as of May 31, 2006) 22
    Table 3: Ongoing and Planned ASOS Improvements (as of May 31, 2006) 22
    Table 4: Ongoing and Planned AWIPS Improvements 23
    Table 5: System Upgrades Are Linked to Expected Performance Improvements
    26

    Figures

    Figure 1: NWS's 122 Weather Forecast Offices 5
    Figure 2: Overview of Key Systems and Technologies Supporting NWS
    Forecasts 6
    Figure 3: NEXRAD Radar Tower 8
    Figure 4: An ASOS System 9
    Figure 5: ASOS Sensors 10
    Figure 6: Approximate GOES Geographic Coverage 12
    Figure 7: Configuration of Operational Polar Satellites 13
    Figure 8: An AWIPS Workstation 14
    Figure 9: Weather Model Output Shown on an AWIPS Workstation 15

    Abbreviations

    ASOS Automated Surface Observing System AWIPS Advanced Weather Interactive
    Processing System DMSP Defense Meteorological Satellite Program DOD
    Department of Defense FAA Federal Aviation Administration GOES
    Geostationary Operational Environmental Satellites NEXRAD Next Generation
    Weather Radar NPOESS National Polar-orbiting Operational Environmental
    Satellite System  NWS National Weather Service NOAA National Oceanic and
    Atmospheric Administration OMB Office of Management and Budget POES
    Polar-orbiting Operational Environmental Satellites

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    United States Government Accountability Office

    Washington, DC 20548

    July 14, 2006

    The Honorable Vernon J. Ehlers Chairman The Honorable David Wu Ranking
    Minority Member Subcommittee on Environment, Technology, and Standards
    Committee on Science House of Representatives

    The National Weather Service's (NWS) ability to forecast the weather
    affects the life and property of every American. The agency's basic
    mission is to provide storm and flood warnings and weather forecasts for
    the United States, its territories, and adjacent oceans and waters-in
    order to protect life and property and to enhance the national economy.
    NWS operations also support other agencies by providing aviation and
    marine-related weather forecasts and warnings.

    To carry out its mission, NWS uses a variety of systems, technologies, and
    manual processes to collect, process, and disseminate weather data to and
    among its network of field offices and regional and national centers. In
    the 1980s and 1990s, NWS undertook a nationwide modernization program to
    upgrade its systems and technologies and to consolidate its field office
    structure. Today, with the modernization completed, NWS continues to seek
    ways to upgrade its systems with a goal of further improving its
    forecasting abilities. The agency is also considering changing how its
    nationwide office structure works in order to enhance its efficiency.

    Because of your interest in these plans for continued improvements, you
    asked us to (1) evaluate NWS's efforts to achieve improvements in the
    delivery of its services through the upgrades to its systems, models, and
    computational abilities; (2) assess agency plans to achieve improvements
    in the delivery of its services through the training and professional
    development of its employees; and (3) evaluate agency plans to revise its
    nationwide office configuration and the implications of these plans on
    local forecasting services, staffing, and budgets.

    To address these objectives, we reviewed NWS plans for system
    enhancements, technology improvements, and professional training, and
    assessed the extent to which these plans were tied to the agency's service
    improvement goals. We also interviewed officials from NWS and the National
    Oceanic and Atmospheric Administration to obtain clarification on agency
    plans and goals. To determine the status and potential impact of any plans
    to revise the national office configuration, we assessed NWS reports on
    ways to enhance its operations and interviewed key officials involved in
    these reports.

    We conducted our work at NWS headquarters in the Washington, D.C.,
    metropolitan area and at NWS offices in Denver, Tampa, and Miami. We
    performed our work from October 2005 to June 2006, in accordance with
    generally accepted government auditing standards. Additional details on
    our objectives, scope, and methodology are provided in appendix I.

                                    Results in Brief

    NWS is positioning itself to provide better service through over $315
    million in planned upgrades to its systems and technologies through 2011.
    In annual plans, the agency links expected improvements in its service
    performance measures with the technologies and systems expected to improve
    them. For example, NWS expects to reduce the average error in its
    forecasts of hurricane paths by approximately 20 nautical miles between
    2005 and 2011 through a combination of upgrades to observation systems,
    better hurricane forecast models, enhancements to the computer
    infrastructure, and research that will be transferred to forecast
    operations. Also, NWS expects to increase its lead time on tornado
    warnings from 13 to 15 minutes by the end of fiscal year 2008 after the
    agency completes an upgrade to its radar system and realizes benefits from
    software improvements to its forecaster workstations.

    NWS also provides training courses for its employees to help improve its
    forecast services, but the agency's process for selecting training lacks
    sufficient oversight. Program officials propose and justify training needs
    on the basis of up to eight different criteria-including whether a course
    is expected to improve NWS forecasting performance measures, support
    customer outreach, or increase scientific awareness. Many of these course
    justifications appropriately demonstrate support for improved forecasting
    performance. For example, training on how to more effectively use
    forecaster workstations is expected to help improve tornado, flash flood,
    and hurricane warnings. However, in justifying training courses, program
    officials routinely link courses to NWS forecasting performance measures.
    For example, in 2006, almost all training needs were linked to
    expectations for improved forecast performance-including training on
    cardiopulmonary resuscitation, spill prevention, and systems security. The
    training selection process did not validate or question how these courses
    could help improve weather forecasts. The overuse of this justification
    undermines the distinctions among different training courses and the
    credibility of the course selection process. Until it establishes a
    training selection process that uses reliable justifications, NWS risks
    selecting courses that do not most effectively support its training goals.

    To improve its efficiency, NWS plans to develop a prototype of a new
    concept of operations-an effort that could affect its national office
    configuration, including the location and functions of its offices
    nationwide. However, many details about the impact of any proposed changes
    on forecast services, staffing, and budget have yet to be determined.
    Further, NWS has not yet determined key activities, timelines, or measures
    for evaluating the prototype of the new office operational structure. As a
    result, it is not evident that NWS will collect the information it needs
    on the impact and benefits of any office restructuring in order to make
    sound and cost-effective decisions.

    We are making recommendations to the Secretary of Commerce to direct NWS
    to strengthen its training selection process; to establish key activities,
    timelines, and measures for evaluating the prototype of the new concept of
    operations; and to ensure that the plans for evaluating the new concept of
    operations address the impact of any changes on budget, staffing, and
    services.

    The Department of Commerce provided written comments on a draft of this
    report in which it agreed with our recommendations and identified planned
    steps for implementing them (see app. IV). The department also provided
    technical corrections, which we have incorporated in this report as
    appropriate.

                                       Background

    The mission of NWS-an agency within the Department of Commerce's National
    Oceanic and Atmospheric Administration (NOAA)-is to provide weather,
    water, and climate forecasts and warnings for the United States, its
    territories, and its adjacent waters and oceans, in order to protect life
    and property and to enhance the national economy. NWS is the official
    source of aviation- and marine-related weather forecasts and warnings, as
    well as warnings about life-threatening weather situations.

    In the 1980s and 1990s, NWS undertook a nationwide modernization program
    to develop new systems and technologies and to consolidate its field
    office structure. The goals of the modernization program were to achieve
    more uniform weather services across the nation, improve forecasts,
    provide more reliable detection and prediction of severe weather and
    flooding, permit more cost-effective operations, and achieve higher
    productivity. The weather observing systems (including radars, satellites,
    and ground-based sensors) and data processing systems that currently
    support NWS operations were developed and deployed under the modernization
    program. During this period, NWS consolidated over 250 large and small
    weather service offices into the office structure currently in use.

    NWS Office Structure: An Overview

    The coordinated activities of weather facilities throughout the United
    States allow NWS to deliver a broad spectrum of climate, weather, water,
    and space weather services. These facilities include weather forecast
    offices, river forecast centers, national centers, and aviation center
    weather service units. The functions of these facilities are described
    below.

               o  122 weather forecast offices are responsible for providing a
               wide variety of weather, water, and climate services for their
               local county warning areas, including advisories, warnings, and
               forecasts (see fig. 1 for the current location of weather forecast
               offices).

               o  13 river forecast centers provide river, stream, and reservoir
               information to a wide variety of government and commercial users
               as well as to local weather forecast offices for use in flood
               forecasts and warnings.

               o  9 national centers constitute the National Centers for
               Environmental Prediction, which provide nationwide computer model
               output and manual forecast information to all NWS field offices
               and to a wide variety of government and commercial users. These
               centers include the Environmental Modeling Center, Storm
               Prediction Center, Tropical Prediction Center, Climate Prediction
               Center, Aviation Weather Center, and Space Environment Center,
               among others.

               o  21 aviation center weather service units, which are co-located
               with key Federal Aviation Administration (FAA) air traffic control
               centers across the nation, provide meteorological support to air
               traffic controllers.

    Figure 1: NWS's 122 Weather Forecast Offices

    NWS Relies on Key Systems and Technologies to Fulfill Its Mission

    To fulfill its mission, NWS relies on a national infrastructure of systems
    and technologies to gather and process data from the land, sea, and air.
    NWS collects data from many sources, including ground-based Automated
    Surface Observing Systems (ASOS), Next Generation Weather Radars (NEXRAD),
    and operational environmental satellites. These data are integrated by
    advanced data processing workstations-called Advanced Weather Interactive
    Processing Systems (AWIPS)-used by meteorologists to issue local forecasts
    and warnings. The data are also fed into sophisticated computer models
    running on high-speed supercomputers, which are then used to help develop
    forecasts and warnings. Figure 2 depicts the integration of the various
    systems and technologies and is followed by a description of each.

    Figure 2: Overview of Key Systems and Technologies Supporting NWS
    Forecasts

    Next Generation Weather Radar (NEXRAD)

    NEXRAD is a Doppler radar system1 that detects, tracks, and determines the
    intensity of storms and other areas of precipitation, determines wind
    velocities in and around detected storm events, and generates data and
    imagery to help forecasters distinguish hazards such as severe
    thunderstorms and tornadoes. It also provides information about heavy
    precipitation that leads to warnings about flash floods and heavy snow.
    The NEXRAD network provides data to other government and commercial users
    and to the general public via the Internet.

    The NEXRAD network is made up of 158 operational radars and 8
    nonoperational radars that are used for training and testing. Of these,
    NWS operates 120 radars, the Air Force operates 26 radars, and the FAA
    operates 12 radars. These radars are located throughout the continental
    United States and in 17 locations outside the continental United States.
    Figure 3 shows a NEXRAD radar tower.

    1Doppler radar is used to determine the speed and direction of rain or
    snow particles, cloud droplets, or dust moving toward or away from the
    radar. The radar accomplishes this by sending out a pulse using a stable
    frequency and then measuring the changing frequencies as the distance
    between the radar and the object changes.

    Figure 3: NEXRAD Radar Tower

    Automated Surface Observing System (ASOS)

    ASOS is a system of sensors, computers, display units, and communications
    equipment that automates the ground-based observation and dissemination of
    weather information nationwide. This system collects data on temperature
    and dew point, visibility, wind direction and speed, pressure, cloud
    height and amount, and types and amounts of precipitation. ASOS supports
    weather forecast activities and aviation operations, as well as the needs
    of research communities that study weather, water, and climate. Figure 4
    is a picture of the system, while figure 5 depicts a configuration of ASOS
    sensors and describes their functions.

    There are currently 1,002 ASOS units deployed across the United States,
    with NWS, FAA, and the Department of Defense (DOD) operating 313, 571, and
    118 units, respectively.

    Figure 4: An ASOS System

    Figure 5: ASOS Sensors

      Operational Environmental Satellites

    Although NWS does not own or operate satellites, geostationary and
    polar-orbiting environmental satellite programs are key sources of data
    for its operations. NOAA manages the Geostationary Operational
    Environmental Satellite (GOES) system and the Polar-orbiting Operational
    Environmental Satellite (POES) system. In addition, DOD operates a
    different polar satellite program called the Defense Meteorological
    Satellite Program (DMSP). These satellite systems continuously collect
    environmental data about the Earth's atmosphere, surface, cloud cover, and
    electromagnetic environment. These data are used by meteorologists to
    develop weather forecasts and other services, and are critical to the
    early and reliable prediction of severe storms, such as tornadoes and
    hurricanes.

    Geostationary satellites orbit above the Earth's surface at the same speed
    as the Earth rotates, so that each satellite remains over the same
    location on Earth. NOAA operates GOES as a two-satellite system that is
    primarily focused on the United States (see fig. 6). To provide continuous
    satellite coverage, NOAA acquires several satellites at a time as part of
    a series and launches new satellites every few years.2 Three satellites,
    GOES-10, GOES-11, and GOES-12, are currently in orbit. Both GOES-10 and
    GOES-12 are operational satellites, while GOES-11 is in an on-orbit
    storage mode. It is a backup for the other two satellites should they
    experience any degradation in service. The first in the next series of
    satellites, GOES-13, was launched in May 2006, and the others in the
    series, GOES-O and GOES-P, are planned for launch over the next few
    years.3 In addition, NOAA is planning a future generation of satellites,
    known as the GOES-R series, which are planned for launch beginning in
    2014.

    2GOES has historically been a joint program between NOAA and the National
    Aeronautics and Space Administration (NASA), with NOAA funding and
    managing the program and NASA providing engineering and launch
    capabilities.

    3Satellites in a series are identified by letters of the alphabet when
    they are on the ground and by numbers once they are in orbit.

    Figure 6: Approximate GOES Geographic Coverage

    Unlike the GOES satellites, which maintain a fixed position above the
    earth, polar satellites constantly circle the Earth in an almost
    north-south orbit, providing global coverage of conditions that affect the
    weather and climate. Each satellite makes about 14 orbits a day. As the
    Earth rotates beneath it, each satellite views the entire Earth's surface
    twice a day. Currently, there are four operational polar-orbiting
    satellites-two are POES satellites and two are DMSP satellites. These
    satellites are positioned so that they can observe the Earth in early
    morning, morning, and afternoon polar orbits. Together, they ensure that
    for any region of the Earth, the data are generally no more than 6 hours
    old. Figure 7 illustrates the current configuration of operational polar
    satellites.

    Figure 7: Configuration of Operational Polar Satellites

    NOAA and DOD plan to continue to launch remaining satellites in the POES
    and DMSP programs, with final launches scheduled for 2007 and 2011,
    respectively. In addition, NOAA, DOD, and the National Aeronautics and
    Space Administration are planning to replace the POES and DMSP systems
    with a state-of-the-art environment monitoring satellite system called the
    National Polar-orbiting Operational Environmental Satellite System
    (NPOESS). In recent years, we reported on a variety of issues affecting
    this major system acquisition.4

    4GAO, Polar-orbiting Operational Environmental Satellites: Cost Increases
    Trigger Review and Place Program's Direction on Hold, GAO-06-573T
    (Washington, D.C.: Mar. 30, 2006); Polar-orbiting Operational
    Environmental Satellites: Technical Problems, Cost Increases, and Schedule
    Delays Trigger Need for Difficult Tradeoff Decisions, GAO-06-249T
    Washington, D.C.: Nov. 16, 2005); Polar-orbiting Environmental Satellites:
    Information on Program Cost and Schedule Changes, GAO-04-1054 (Washington,
    D.C.: Sept. 30, 2004); Polar-orbiting Environmental Satellites: Project
    Risks Could Affect Weather Data Needed by Civilian and Military Users,
    GAO-03-987T (Washington, D.C.: July 15, 2003); and Polar-orbiting
    Environmental Satellites: Status, Plans, and Future Data Management
    Challenges, GAO-02-684T (Washington, D.C.: July 24, 2002).

      Advanced Weather Interactive Processing System (AWIPS)

    AWIPS is a computer system that integrates and displays all
    hydrometeorological data at NWS field offices. This system integrates data
    from NEXRAD, ASOS, GOES, and other sources to produce rich graphical
    displays to aid forecaster analysis and decision making. AWIPS is used to
    disseminate weather information to the national centers, weather offices,
    the media, and other federal, state, and local government agencies. NWS
    deployed hardware and software for this system to weather forecast
    offices, river forecast centers, and national centers throughout the
    United States between 1996 and 1999. As a software-intensive system, AWIPS
    regularly receives software upgrades called "builds." The most recent
    build, called Operational Build 6, is currently being deployed. NWS
    officials estimated that the nationwide deployment of this build should be
    completed by July 2006. Figure 8 shows a standard AWIPS workstation.

    Figure 8: An AWIPS Workstation

      Numerical Models

    Numerical models are advanced software programs that assimilate data from
    satellites and ground-based observing systems and provide short- and
    long-term weather pattern predictions. Meteorologists typically use a
    combination of models and their own experience to develop local forecasts
    and warnings. Numerical weather models are also a critical source for
    forecasting weather up to 7 days in advance and forecasting long-term
    climate changes. One of NWS's National Centers for Environmental
    Prediction, the Environmental Modeling Center, is the primary developer of
    these models within NWS and is responsible for making new and improved
    models available to regional forecasters via the AWIPS system. Figure 9
    depicts model output as shown on an AWIPS workstation.

    Figure 9: Weather Model Output Shown on an AWIPS Workstation

      Supercomputers

    NWS leases high-performance supercomputers to execute numerical
    calculations supporting weather prediction and climate modeling. In 2002,
    NWS awarded a $227 million contract to lease high-performance
    supercomputers to run its environmental models from 2002 through September
    2011. Included in this contract are an operational supercomputer used to
    run numerical weather models, an identical backup supercomputer located at
    a different site, and a research and development supercomputer on which
    researchers can test out new analyses and models. The supercomputer lease
    contract allows NWS to exercise options to upgrade the processing
    capabilities of the operational supercomputer.

    Previous Reports Focused on NWS Modernization Systems Risks

    During the 1990s, we issued a series of reports on NWS modernization
    systems and made recommendations to improve them.5 For example, early in
    the AWIPS acquisition, we reported that the respective roles and
    responsibilities of the contractor and government were not clear and that
    a structured system development environment had not been established. We
    made recommendations to correct these shortfalls before the system design
    was approved. We also reported that the ASOS system was not meeting
    specifications or user needs, and recommended that NWS define and
    prioritize system corrections and enhancements. On NEXRAD, we reported
    that selected units were falling short of availability requirements and
    recommended that NWS analyze and monitor system availability on a
    site-specific basis and correct any shortfalls. Because of such concerns,
    we identified NWS modernization as a high-risk information technology
    initiative in 1995, 1997, and 1999.6

    NWS took a number of actions to address our recommendations and to resolve
    system risks. For example, NWS enhanced its AWIPS system development
    processes, prioritized its ASOS enhancements, and improved the
    availability of its NEXRAD systems. In 2001, because of NWS's progress in
    addressing key concerns and in deploying and using the AWIPS system-the
    final component of its modernization program-we removed the modernization
    from our high-risk list.

    5See, for example, GAO, Weather Forecasting: Improvements Needed in
    Laboratory Software Development Process, GAO/AIMD-95-24 (Washington, D.C.:
    Dec. 14, 1994); Weather Forecasting: Unmet Needs and Unknown Costs Warrant
    Reassessment of Observing System Plans, GAO/AIMD-95-81 (Washington, D.C.:
    Apr. 21, 1995); Weather Forecasting: Radar Availability Requirement Not
    Being Met, GAO/AIMD-95-132 (Washington, D.C.: May 31, 1995); Weather
    Forecasting: Radars Far Superior to Predecessors, but Location and
    Availability Questions Remain, GAO/T-AIMD-96-2 (Washington, D.C.: Oct. 17,
    1995); Weather Forecasting: New Processing System Faces Uncertainties and
    Risks, GAO/T-AIMD-96-47 (Washington, D.C.: Feb. 29, 1996); Weather
    Forecasting: Recommendations to Address New Weather Processing System
    Development Risks, GAO/AIMD-96-74 (May 13, 1996); and Weather Satellites:
    Planning for the Geostationary Satellite Program Needs More Attention,
    GAO/AIMD-97-37 (Washington, D.C.: Mar. 13, 1997).

    6GAO, High-Risk Series: An Overview, GAO/HR-95-1 (Washington, D.C.:
    February 1995); High-Risk Series: Information Management and Technology,
    GAO/HR-97-9 (Washington, D.C.: February 1997); High-Risk Series: An
    Update, GAO/HR-99-1 (Washington, D.C.: January 1999); High-Risk Series: An
    Update, GAO-01-263 (Washington, D.C.: January 2001).

    NWS Established Performance Goals and Tracks Progress against These Goals

    In accordance with federal legislation requiring federal managers to focus
    more directly on program results, NWS established short- and long-term
    performance goals and regularly tracks its actual performance in meeting
    these goals.7 Specifically, NWS established 14 different performance
    measures-such as lead time for flash floods and false-alarm rates for
    tornado warnings. It also established 5-year goals for improving its
    performance in each of the 14 performance measures through 2011. For
    example, the agency plans to increase its lead time on tornado warnings
    from 13 minutes in 2005 to 15 minutes in 2011. Table 1 identifies NWS's 14
    performance measures, selected goals, and performance against those goals,
    when available. Appendix II provides additional information on NWS's
    performance goals.

    7The Government Performance and Results Act of 1993 (Pub. L. 103-62) was
    intended to improve federal program effectiveness, accountability, and
    service delivery by requiring federal agencies to develop strategic plans
    with long-term, outcome-oriented goals and objectives; annual performance
    goals linked to the long-term goals; and annual reports on actual results.

    Table 1: NWS's Performance Measures, Goals, and Actual Performance for
    Fiscal Years (FY) 2005, 2006, and 2011

                                             FY05                FY06           FY11
                                                                Actual
      Performance                            Actual  Goal           to      On
      measure            Description   Goal (final)  met?  Goal   date target?  Goal
      Tornado warning    The             13      13   Yes    13    13a     Yes    15
      lead time          difference
      (minutes)          between the
                         time a
                         warning is
                         issued and
                         the time of
                         the first
                         report of a
                         tornado in a
                         given county
      Tornado warning    The             73      75   Yes    76    82a     Yes    76
      accuracy (percent) percentage of
                         time a
                         tornado
                         actually
                         occurred in
                         an area
                         covered by a
                         tornado
                         warning
      Tornado warning    The             73      77    No    75    76a      No    74
      false-alarm rate   percentage of
      (percent)          time a
                         tornado
                         warning was
                         issued but no
                         tornado event
                         was reported
      Flash flood        The             48      54   Yes    48    63a     Yes    49
      warning lead time  difference
      (minutes)          between the
                         time a
                         warning is
                         issued and
                         the time of
                         the first
                         report of a
                         flash flood
                         in a given
                         county
      Flash flood        The             89      88    No    89    93a     Yes    90
      warning accuracy   percentage of
      (percent)          time a flash
                         flood
                         actually
                         occurred in
                         an area
                         covered by a
                         flash flood
                         warning
      Marine wind speed  A measure of    57      57   Yes    58    56b      No    59
      forecast accuracy  the accuracy
      (percent)          of wind speed
                         forecasts
      Marine wave height A measure of    67      67   Yes    68    71b     Yes    69
      forecast accuracy  the accuracy
      (percent)          of wave
                         forecasts
      Aviation forecast  The             46      46   Yes    47    45b      No    59
      Instrument Flight  percentage of
      Rule               time
      ceiling/visibility Instrument
      accuracy (percent) Flight Rule
                         conditionse
                         are predicted
                         and occur
      Aviation forecast  The             68      63   Yes    65    61b     Yes    50
      Instrument Flight  percentage of
      Rule               time
      ceiling/visibility Instrument
      false-alarm rate   Flight Rule
      (percent)          conditionse
                         are predicted
                         but do not
                         occur
      Winter storm       The average     15      17   Yes    15    16c     Yes    17
      warning lead time  time from the
      (hours)            issuance of a
                         warning to
                         the time of
                         the first
                         report of a
                         winter storm
                         in a given
                         county
      Winter storm       The             90      91   Yes    90    91c     Yes    92
      warning accuracy   percentage of
      (percent)          verified
                         winter storm
                         events that
                         were covered
                         by winter
                         storm
                         warnings
      Precipitation      A score based   27      29   Yes    28    39d     Yes    30
      forecast day 1     on the
      threat (score)     agency's
                         accuracy in
                         forecasting
                         precipitation
      U.S. seasonal      A score based   18      19   Yes    18    24d     Yes    20
      temperature        on the
      forecast skill     agency's
      (score)            accuracy in
                         forecasting
                         temperature
      Hurricane track    A measure of   128     101   Yes   111   N/Af    N/Af   106
      forecast error at  the
      48 hours (nautical difference
      miles)             between the
                         projected
                         locations of
                         the center of
                         storms and
                         the actual
                         location in
                         nautical
                         miles for the
                         Atlantic
                         Basin

    Source: GAO analysis of NOAA and NWS reports.

    aMetric measured between October 2005 and January 2006.

    bMetric measured between October 2005 and February 2006.

    cMetric measured between October 2005 and December 2005.

    dMetric measured between October 2005 and March 2006.

    eInstrument Flight Rule conditions exist when ceilings and visibilities
    are less than 1,000 feet and/or 3 miles, respectively, and ceilings and
    visibilities are greater than, or equal to, 500 feet and/or 1 mile,
    respectively.

    fData for this metric are not available until the beginning of the next
    calendar year because of the timing of the hurricane season.

    NWS periodically adjusts its performance goals as its assumptions change.
    After reviewing actual results from previous fiscal years and its
    assumptions about the future, in January 2006, NWS adjusted eight of its
    5-year performance goals to make more realistic predictions for
    performance for the next several years. Specifically, NWS made six
    performance goals less stringent and two goals more stringent. The six
    goals that were made less stringent-and the reasons for the changes-are
    the following:

               o  Tornado warning lead time: NWS changed its 2011 goal from 17 to
               15 minutes of warning because of delays in deploying new
               technologies on NEXRAD radars and a lack of access to FAA radar
               data.

               o  Tornado warning false-alarm rate: NWS changed its 2011 goal
               from a 70 to 74 percent false-alarm rate for the same reasons
               listed above.

               o  Flash flood warning accuracy: NWS changed its 2011 goal from 91
               to 90 percent accuracy after delays on two different systems in
               2004, 2005, and 2006.

               o  Marine wind speed accuracy: NWS changed its 2011 goal from 67
               to 59 percent accuracy after experiencing the delay of marine
               models and datasets, a deficiency of shallow water wave guidance,
               and a reduction in funds for training.

               o  Marine wave height accuracy: NWS changed its 2011 goal from 77
               to 69 percent accuracy for the same reasons identified above for
               marine wind speed accuracy.

               o  Aviation instrument flight rule ceiling/visibility: NWS changed
               its goal from 48 to 47 percent accuracy in 2006 because of a
               system delay and a reduction in funds for training. Goals for 2007
               through 2011 remained the same.

               Additionally, the following two goals were made more stringent:

               o  Aviation instrument flight rule ceiling/visibility false-alarm
               rate: NWS reduced its expected false-alarm rate from 68 percent to
               65 percent for 2006 because of better than anticipated results
               from the AWIPS aviation forecast preparation system and an
               aviation learning training course. Goals for the remaining years
               in the 5-year plan, 2007 to 2011, remained the same.

               o  Hurricane track forecasts: NWS changed its 2011 hurricane track
               forecast goal from 123 to 106 nautical miles after trends in
               observed data from 1987 to 2004 showed that this measure was
               improving more quickly than expected.

               NWS is positioning itself to provide better service through system
               and technology upgrades. Over the next few years, the agency plans
               to upgrade and improve its systems, predictive weather models, and
               computational abilities, and it appropriately links these upgrades
               to its performance goals. For example, planned improvements in
               NEXRAD technology are expected to help improve the lead times for
               tornado warnings, while AWIPS software enhancements are expected
               to help improve the accuracy of marine weather forecasts. The
               agency anticipates continued steady improvement in its forecast
               accuracy as it obtains better observation data, as computational
               resources are increased, and as scientists are better able to
               implement advanced modeling and data assimilation techniques.

               Over the next few years, NWS has plans to spend over $315 million
               to upgrade its systems, models, and computational abilities.8 Some
               planned upgrades are to maintain the weather system infrastructure
               (either to replace obsolete and difficult-to-maintain parts or to
               refresh aging hardware and workstations), while others are to take
               advantage of new technologies. Often, the infrastructure upgrades
               allow NWS to take advantage of newer technologies. For example,
               the replacement of an aging and proprietary NEXRAD subsystem is
               expected to allow the agency to implement enhancements in image
               resolution. Key planned upgrades for each of NWS's major systems
               and technologies are listed below.

               NWS has initiated two major NEXRAD improvements. It is currently
               replacing an outdated subsystem-the radar data acquisition
               subsystem-with current hardware that is compliant with open system
               standards. This new hardware is expected to enable important
               software upgrades. In addition, NWS plans to add a new technology
               called dual polarization to this subsystem, which will provide
               more accurate rainfall estimates and differentiate various forms
               of precipitation. Table 2 shows the details of these two projects.

      NWS Is Positioning Itself to Provide Better Service through Upgrades to Its
                                Systems and Technologies

    NWS Has Plans for Upgrading Its Systems, Models, and Computational Abilities

      NEXRAD

    8This cost estimate includes the expected cost of key system upgrades, as
    well as estimated annual costs for improvements to AWIPS software and
    numerical models through the year 2011. It does not include the expected
    costs of supercomputer upgrades because NWS does not estimate what portion
    of its $26 million annual supercomputer budget is attributable to
    upgrades.

    Table 2: Ongoing and Planned NEXRAD Improvements (as of May 31, 2006)

                                                         Estimated    Estimated
                                             Current     acquisition  completion
    Improvement  Description              status      cost         date
    Radar data   A subsystem that         In process; $43.8        Estimated
    acquisition  transmits and receives   107 of 158  million (NWS to be
    subsystem    radar signals, controls  sites have  portion is   completed
    replacement  the radar antenna,       been        $22.6        in late
                    processes the received   installed.  million)     2006.
                    signal, and sends the
                    processed data to the
                    radar product generator;
                    replacement of this
                    subsystem will enable
                    software upgrades
                    including an enhancement
                    that will allow
                    operators to view more
                    detailed weather
                    features.
    Dual         A technology upgrade to  Acquisition $38 million  Expected
    polarization allow enhanced target    process is  (NWS portion contract
    technology   identification.          under way.  is $25       award at
    upgrade                                           million)     the end of
                                                                      2006.

                                                                      Deployment
                                                                      is expected
                                                                      to begin in
                                                                      fiscal year
                                                                      2009 and
                                                                      end in
                                                                      fiscal year
                                                                      2011.

    Source: NEXRAD Program Office.

      ASOS

    NWS has seven ongoing and planned improvements for its ASOS system (see
    table 3). Many of these improvements are to replace aging parts and are
    expected to make the system more reliable and maintainable. Key subsystem
    replacements-including the all-weather precipitation accumulation
    gauge-are also expected to result in more accurate measurements.

    Table 3: Ongoing and Planned ASOS Improvements (as of May 31, 2006)

                                                          Estimated or Estimated
                                                          actual       or actual
                                                          acquisition  completion
    Improvement     Description       Current status   cost         date
    Processor       Provides a more   962              $6.61        June 30,
    upgrade         robust processor  installations    million (NWS 2006
                       with increased    completed out of portion is
                       capacity, speed,  1002 total       $2.89
                       and memory.       planned sites    million)
                                         (312
                                         installations
                                         completed out of
                                         313 NWS sites)
    All-weather     Replaces existing 323 of 331       $7.10        June 30,
    precipitation   heated tipping    installed (303   million      2006
    accumulation    bucket rain gauge of 311 NWS)
    gauge           with a gauge that
                       measures
                       precipitation by
                       weight, resulting
                       in more accurate
                       measurements.
    Dewpoint sensor Replace existing  958 of 1002      $9.20        June 30,
                       sensor's chilled  installed (303   million (NWS 2006
                       mirror technology of 311 NWS)      portion is
                       with a humidity                    $3.14
                       sensitive                          million)
                       capacitor.
    Ice-free wind   Replaces the      231 of 1000      $7.53        November
    sensor          existing cup and  installed (60 of million (NWS 30, 2006
                       vane anemometer   311 NWS)         portion is
                       with a new                         $2.90
                       ultrasonic                         million)
                       sensor.
    Enhanced        Replaces sensor   Field            $10.14       March 31,
    precipitation   that only reports demonstration    million (NWS 2009
    identifier      rain and snow     testing to begin portion is
                       with one that is  July 2006        $3.55
                       to report rain,                    million)
                       snow, drizzle,
                       hail, and ice
                       pellets.
    Ceilometer      Replaces senor    Evaluation of    $33 million  September
    (cloud height)  that measures     commercial       (NWS portion 30, 2011
                       cloud heights up  sensors almost   is $12
                       to 12,000 feet    complete;        million)
                       with one that is  solicitation for
                       expected to       system
                       measure cloud     development
                       heights up to     expected to
                       40,000 feet.      begin by end of
                                         May 2006.
    Sunshine        Adds a new sensor Program on hold  $1.77        September
    duration sensor to measure solar  pending          million      30, 2011
                       radiation.        ceilometer       (this
                                         production; will upgrade
                                         be developed     affects only
                                         after the        NWS systems)
                                         ceilometer;
                                         planned to
                                         restart by 2010.

    Source: ASOS program office.

      AWIPS

    Selected AWIPS system components have become obsolete, and NWS is
    replacing these components. In 2001, NWS began to migrate the existing
    Unix-based systems to a Linux system to reduce its dependence on any
    particular hardware platform. NWS expects this project, combined with
    upgraded information technology, to delay the need for a major information
    technology replacement. Table 4 shows planned improvements for the AWIPS
    system.

    Table 4: Ongoing and Planned AWIPS Improvements

                                            Current  Estimated Timeline/estimated
    Improvement      Description         status   cost      completion date
    Linux migration  An effort to        In       $17.92    2002 to 2007
                        replace legacy      progress million
                        hardware and to
                        port approximately
                        4 million source
                        lines of code of
                        AWIPS software from
                        the original
                        proprietary
                        Hewlett-Packard
                        Unix operating
                        system to the open
                        source Linux
                        operating system.
    Architecture     An effort to refine In       $900,000  2004 to 2006
    analysis         AWIPS hardware and  progress
                        communications
                        architecture in
                        support of the
                        Linux migration and
                        to build an
                        advanced Linux
                        prototype system.
    Information      An initiative to    In       $3.22     2004 to 2006
    technology       replace obsolete    progress million
    security         routers and
                        firewalls
                        throughout the
                        system.
    Hardware refresh An initiative to    In       $53.21    2006 to 2015
                        keep the AWIPS      progress million
                        hardware baseline
                        fresh and
                        maintainable
                        through a
                        continuous
                        technology refresh.
                        NWS plans to
                        refresh hardware
                        components every 4
                        to 5 years after
                        the Linux migration
                        is completed.
    Software         An initiative to    In       $23       2006 to 2010
    re-architecture  reengineer the      progress million
                        AWIPS software
                        suite to a standard
                        service-oriented
                        architecture.
    Software         Includes efforts to In       About $10 Continuous
    upgrades         enhance advanced    progress million
                        precipitation                per year
                        algorithms for
                        estimating
                        rainfall; continue
                        enhancement of
                        advanced decision
                        assistance tools;
                        implement a
                        distributed
                        hydraulic model;
                        and enhance
                        forecasting and
                        evaluation of seas
                        and lakes to
                        provide a
                        prediction
                        capability tool for
                        marine forecasters.

    Source: NWS.

      Numerical Models

    NWS plans to continue to improve its modeling capabilities by (1) better
    assimilating data from improved observation systems such as ASOS, NEXRAD,
    and environmental satellites; (2) developing and implementing an advanced
    global forecasting model (called the Weather Research and Forecast model)
    to allow forecasters to look at a larger domain area; (3) implementing a
    hurricane weather research forecast model; and (4) improving ensemble
    modeling, which involves running a single model multiple times with slight
    variations on a variable to get a probability that a given forecast is
    likely to occur. NWS expects to spend approximately $12.7 million in
    fiscal year 2006 to improve its weather and real-time ocean models.

      Supercomputers

    NWS is planning to exercise an option within its existing supercomputer
    lease to upgrade its computing capabilities to allow more advanced
    numerical weather and climate prediction modeling.

    NWS Appropriately Links Its System and Technical Upgrades to Expected Service
    Improvements

    In accordance with federal legislation and policy, NWS's planned upgrades
    to its systems and technologies are expected to result in improved
    service. The Government Performance and Results Act calls for federal
    managers to develop strategic performance goals and to focus program
    activities on obtaining results.9 Also, the Office of Management and
    Budget (OMB) requires agencies to justify major investments by showing how
    they support performance goals.10 NOAA and NWS implement the act and OMB
    guidance by requiring project officials to describe how planned system and
    technology upgrades are linked to the agency's programmatic priorities and
    performance measures. Further, in its annual performance plans, NOAA
    reports on expected NWS service improvements and identifies the
    technologies and systems that are expected to help improve them.

    9Pub. L. 103-62, 107 Stat. 285 (1993).

    NWS service improvements are often expected through a combination of
    system and technology improvements. For example, NWS expects to reduce its
    average error in forecasting a hurricane's path by approximately 20
    nautical miles between 2005 and 2011 through a combination of upgrades to
    observation systems, better hurricane forecast models, enhancements to the
    computer infrastructure, and research that will be transferred to NWS
    forecast operations. Also, NWS expects tornado warning lead times to
    increase from 13 to 15 minutes by the end of fiscal year 2008 after NWS
    completes retrofits to the NEXRAD systems, realizes the benefits of AWIPS
    software enhancements, and implements new training techniques. Table 5
    provides a summary of how system upgrades are expected to result in
    service improvements.

    10OMB requires agencies to annually submit documentation, called an
    exhibit 300, justifying major information technology initiatives or
    improvements.

    Table 5: System Upgrades Are Linked to Expected Performance Improvements

                      Expected results of ongoing    Primary performance measures
    System         and planned system upgrades    affected
    NEXRAD         Replacement of the data        Tornado warnings lead time
                      acquisition subsystem is       Tornado warnings accuracy
                      expected to allow future       Tornado warnings false-alarm
                      software and hardware          rate Flash flood warning
                      enhancements. These            lead time Flash flood
                      enhancements are expected to   warning accuracy Winter
                      improve forecasting            storm warnings lead time
                      performance.                   Winter storm warnings
                                                     accuracy
    ASOS           Processor and sensor           Flash flood warning lead
                      replacements are expected to   time Flash flood warning
                      allow more reliable and        accuracy Aviation forecast
                      maintainable systems. Selected ceiling/visibility accuracy
                      system improvements-including  Aviation forecast
                      the deployment of an           ceiling/visibility
                      all-weather precipitation      false-alarm rate
                      gauge, an enhanced
                      precipitation identifier, and
                      a new ceilometer-are expected
                      to directly improve
                      forecasting performance.
    AWIPS          Infrastructure upgrades        Tornado warnings lead time
                      (including a software          Tornado warnings accuracy
                      migration and hardware         Tornado warnings false-alarm
                      refreshment) are expected to   rate Flash flood warning
                      allow major software           lead time Flash flood
                      enhancements that will result  warning accuracy Marine wind
                      in more accurate and timely    speed forecasts accuracy
                      forecasts.                     Marine wave height forecasts
                                                     accuracy Aviation forecast
                                                     ceiling/visibility accuracy
                                                     Aviation forecast
                                                     ceiling/visibility
                                                     false-alarm rate Winter
                                                     storm warnings lead time
    Supercomputers Increased computational        Winter storm warnings lead
                      capabilities are expected to   time Winter storm warnings
                      allow advanced modeling and    accuracy Precipitation
                      data assimilation-and to       forecast day 1 threat score
                      result in improved forecast    U.S. seasonal temperature
                      accuracy.                      forecast skill Hurricane
                                                     track forecasts at 48 hours
    Models         Modeling improvements, enabled Flash flood warning lead
                      by increased supercomputer     time Flash flood warning
                      capacity, are expected to      accuracy Marine wind speed
                      result in more accurate and    forecasts accuracy Marine
                      timely forecasts.              wave height forecasts
                                                     accuracy Aviation forecast
                                                     ceiling/visibility accuracy
                                                     Aviation forecast
                                                     ceiling/visibility
                                                     false-alarm rate Winter
                                                     storm warnings lead time
                                                     Winter storm warnings
                                                     accuracy Precipitation
                                                     forecast day 1 threat score
                                                     U.S. seasonal temperature
                                                     forecast skill Hurricane
                                                     track forecasts at 48 hours

    Source: GAO analysis of NWS data.

     NWS's Training Is Expected to Result in Forecast Service Improvements, but the
                 Training Selection Process Lacks Sufficient Oversight

    NWS provides employee training courses that are expected to help improve
    forecast service performance, but the agency's process for selecting this
    training lacks sufficient oversight. Each year, NWS identifies its
    training needs and develops this training in order to enhance its
    services. NWS develops an annual training and education plan identifying
    planned training, how this training supports key criteria, and associated
    costs for the upcoming year. To develop the annual plan, program area
    teams, with representatives from NWS headquarters and field offices,
    prioritize and submit training recommendations. Each submission identifies
    how the training will support up to eight different criteria-including the
    course's effect on NWS forecasting performance measures, NOAA strategic
    goals, ensuring operational continuity, and providing customer outreach.
    These submissions are screened by a training and education team, and
    depending on available resources, selected for development (if not
    pre-existing) and implementation. The planned training courses are then
    delivered through a variety of means, including courses at the NWS
    training center, online training, and training at local forecast offices.

    In its 2006 training process, 25 program area teams identified 134
    training needs, such as training on how to more effectively use AWIPS,
    training on an advanced weather simulator, and training on maintaining
    ASOS systems. Given an expected funding level of $6.1 million, the
    training and education team then selected 68 of these training needs for
    implementation. NWS later identified another 5 training needs and
    allocated an additional $1.25 million to its training budget. In total,
    NWS funded 73 of 139 training courses.

    The majority of planned training courses demonstrate a clear link to
    expected forecasting service improvements. For example, NWS developed a
    weather event simulator to help forecasters improve their tornado warning
    lead times. In addition, AWIPS-related training courses are expected to
    help improve each of the agency's 14 forecasting performance measures by
    teaching forecasters advanced techniques in using the integrated data
    processing workstations.

    However, NWS's process for selecting which training courses to implement
    lacks sufficient oversight. In justifying training courses, program
    officials routinely link proposed courses to NWS forecast performance
    measures. Specifically, in 2006, 131 of the 134 original training needs
    were linked to expectations for improved forecasting performance-including
    training on cardiopulmonary resuscitation, spill prevention, leadership,
    systems security, and equal employment opportunity/diversity. The training
    selection process did not validate or question that these courses would
    improve tornado warning lead times or hurricane warning accuracy. Although
    these courses are important and likely justifiable on other bases, the
    overuse of this justification undermines the distinctions among training
    courses and the credibility of the course selection process. Additionally,
    because the training selection process does not clearly distinguish among
    courses, it is difficult to determine whether sufficient funds are
    dedicated to the courses that are expected improve performance.

    NWS training officials acknowledged that some of the course justifications
    seem questionable and that more needs to be done to strengthen the
    training selection process to ensure oversight of the justification and
    prioritization process. They noted that the training division plans to
    improve the training selection process over the next few years by adding a
    more systematic worker-focused assessment of training needs, better
    prioritizing strategic and organizational needs, and initiating
    post-implementation reviews. However, until NWS establishes a training
    selection process that uses reliable justification and results in
    understandable decisions, NWS risks selecting courses that do not most
    effectively support its training goals.

    Changing Concept of Operations Could Affect Nationwide Office Configuration, but
           Impact on Forecast Services, Staffing, and Budget Is Not Yet Known

    NWS plans to develop a prototype of a new concept of operations-an effort
    that could affect its national office configuration,11 including the
    location and functions of its offices nationwide. However, NWS has yet to
    determine many details about the impact of any proposed changes on NWS
    forecast services, staffing, and budget. Further, NWS has not yet
    identified key activities, timelines, or measures for evaluating the
    concept of operations prototype. As a result, it is not evident that NWS
    will collect the information it needs on the impact and benefits of any
    office restructuring in order to make sound and cost-effective decisions.

    11Because there is no precise definition of the term "office
    configuration," we have defined it as NWS's current number of offices, the
    location of the offices, hours worked at each of the offices, and the
    services and functions provided at each of the offices.

    NWS Is Evaluating Changes to Its Current Operations

    According to agency officials, over the last several years, NWS's
    corporate board12 noted that the constrained budget, high labor costs,
    difficulty in training and developing its employees, and a lack of
    flexibility in how the agency was operating were making it more difficult
    for the agency to continue to perform its mission. In August 2005, the
    board chartered a working group to evaluate the roles, responsibilities,
    and functions of weather offices nationwide and to make a proposal for a
    new concept of operations. The group was given a set of guiding
    principles, including that the proposed concept should (1) be cost
    effective, (2) ensure that there would be no degradation of service, (3)
    ensure that weather services nationwide were equitable, and (4) not reduce
    the number of forecast offices nationwide. In addition, the working group
    was instructed not to address grade structure, staffing levels, office
    sizes, or overall organizational chart structure.

    The group gathered input from various agency stakeholders and other
    partners within NOAA and considered multiple alternatives. They dismissed
    all but one of the alternative concepts because they were not consistent
    with the guiding principles. In its December 2005 proposal, the working
    group proposed a "clustered peer" office plan designed to redistribute
    some functions among various offices, particularly when there is a
    high-intensity weather event. An agency official explained that each
    weather forecast office currently has a fixed geographic area for which it
    provides forecasts. If a severe weather event occurs, forecast offices ask
    their staff to work overtime so that there are enough personnel available
    to do both the normal forecasting work and the watches and warnings
    required by the severe event. If a local office becomes unable to provide
    forecast and warning functions, an adjacent office will temporarily assume
    those duties by calling in extra personnel to handle the workload of both
    offices.

    Alternatively, under a clustered peer office structure, several offices
    with the same type of weather and warning responsibilities, climate, and
    customers would be grouped in a cluster. Offices within a cluster would
    share the workload associated with routine services, such as 7-day
    forecasts. During a high-impact weather event-such as a severe storm,
    flood, or wildfire-the offices would redistribute the workload to allow
    the impacted office to focus solely on the event, while the other offices
    in the cluster would pick up the impacted office's routine services. In
    this way, peer offices could help supplement staffing needs and the
    workload across multiple offices could be more efficiently balanced.

    12NWS's Corporate Board is chaired by the Director of the National Weather
    Service, and made up of senior officials responsible for different aspects
    of the agency's mission, including the Chief Information Officer and the
    Directors of the Office of Climate, Water, and Weather Services; the
    National Centers for Environmental Prediction; and the NWS Regions. It
    meets at least twice annually to discuss the NWS budget and other
    strategic issues. It also holds special meetings, as needed, to focus on
    NWS issues such as postevent assessments of major weather services, such
    as an assessment of weather services during Hurricane Charley in 2004.

    After receiving this proposal, the NWS corporate board chartered another
    team to develop a prototype of the clustered peer idea to evaluate the
    benefits of this approach. The team plans to recommend the scope of the
    prototype and select several weather offices for the prototype
    demonstration by the end of September 2006. It also plans to conduct the
    prototype demonstration in fiscal years 2007 and 2008. Initial prototype
    results are due in fiscal year 2009.

    Impacts of New Concept of Operations Have Yet to Be Determined

    Many details about the impact of the changes on NWS forecast services,
    staffing, and budget have yet to be determined. Sound decision making on
    moving forward with a new concept of operations will require data on the
    relative costs, benefits, and impacts of such a change, but at this time
    the implications of NWS's revised concept of operations on staffing,
    budget, and forecasting services are unknown.

    The charter for the team developing the prototype for the new concept of
    operations calls for it to identify metrics for evaluating the prototype
    and to define mechanisms for obtaining customer feedback. However, the
    team has not yet established a plan or timeline for developing these
    metrics or mechanisms. Further, it is not yet evident that these metrics
    will include the relative costs, benefits, or impacts of this change or
    which customers will be offered the opportunity to provide feedback. This
    is not consistent with the last time NWS undertook a major change to its
    concept of operations-during its modernization in the mid-1990s. During
    that effort, the agency developed a detailed process for identifying
    impacts and ensuring that there would be no degradation of service (see
    app. III for a summary of this prior process).

    Until it establishes plans, timelines, and metrics for evaluating its
    prototype of a revised concept of operations, NWS is not able to ensure
    that it is on track to gather the information it needs to fully evaluate
    the merits of the revised concept of operations and to make sound and
    informed decisions on a new office configuration.

                                      Conclusions

    NWS is appropriately positioning itself to improve its forecasting
    services by upgrading its systems and technologies and by developing
    training to enhance the performance of its professional staff. Over the
    next few years, NWS expects to improve all of its 14 performance
    measures-ranging from seasonal temperature forecasts, to severe weather
    warnings, to specialized aviation and marine weather warnings. However, it
    is not clear that NWS is consistently choosing the best training courses
    to improve its performance because the training selection process does not
    rigorously review the training justifications.

    Recognizing that high labor costs, difficulty in training and developing
    its employees, and a constrained budget environment make it difficult to
    fulfill its mission, NWS is evaluating changes to its office structure and
    operations in order to achieve greater productivity and efficiency. It
    plans to develop a prototype of a new concept of operations that entails
    sharing responsibilities among a cluster of offices. Because it is early
    in the prototype process, the implications of these plans on staffing,
    budget, and forecasting services are unknown at this time. However, NWS
    does not yet have detailed plans, timelines, or measures for assessing the
    prototype. As a result, NWS risks not gathering the information it needs
    to make an informed decision in moving forward with a new office
    operational structure.

                          Recommendations for Executive Action

    To improve NWS's ability to achieve planned service improvements, we
    recommend that the Secretary of Commerce direct the Assistant
    Administrator for Weather Services to take the following three actions:

               o  require training officials to validate the accuracy of training
               justifications;

               o  establish key activities, timelines, and measures for
               evaluating the "clustered peer" office structure prototype before
               beginning the prototype; and

               o  ensure that plans for evaluating the prototype address the
               impact of any changes on budget, staffing, and services.

               We received written comments on a draft of this report from the
               Department of Commerce (see app. IV). In the department's
               response, the Deputy Secretary of Commerce agreed with our
               recommendations and identified plans for implementing them.
               Specifically, the department noted that it plans to revise its
               training process to ensure limited training resources continue to
               target improvements in NWS performance. The department also noted
               that the concept of operations working team is developing a plan
               for the prototype and stated that this plan will include the items
               we recommended.

               The department also provided technical corrections, which we have
               incorporated as appropriate.

               We are sending copies of this report to the Secretary of Commerce,
               the Director of the Office of Management and Budget, and other
               interested congressional committees. Copies will be made available
               to others on request. In addition, this report will be available
               at no charge on our Web site at www.gao.gov .

               If you have any questions about this report, please contact me at
               (202) 512-9286 or by e-mail at pownerd@gao.gov . Contact points
               for our Offices of Congressional Relations and Public Affairs may
               be found on the last page of this report. GAO staff who made major
               contributions to this report are listed in appendix V.

               David A. Powner Director, Information Technology Management Issues

               Our objectives were (1) to evaluate the National Weather Service's
               (NWS) efforts to achieve improvements in the delivery of its
               services through upgrades to its systems, models, and
               computational abilities; (2) to assess the agency's plans to
               achieve improvements in the delivery of its services through the
               training and professional development of its employees; and (3) to
               evaluate the agency's plans for revising its nationwide office
               configuration and the implications of these plans on local
               forecasting services, staffing, and budgets.

               To evaluate NWS's efforts to achieve service improvements through
               system and technology upgrades, we reviewed the agency's system
               development plans and discussed system-specific plans with NWS
               program officials. We assessed system-specific documentation
               justifying system upgrades to evaluate whether these upgrades were
               linked to anticipated improvements in performance goals. We also
               evaluated NWS performance goals and identified the extent to which
               anticipated service improvements were tied to system and
               technology upgrades. We interviewed National Oceanic and
               Atmospheric Administration (NOAA) and NWS officials to obtain
               clarification on agency plans and goals.

               To assess NWS's plans for achieving service improvements through
               the training and professional development of its employees, we
               reviewed NWS policies and plans for training and professional
               development. We reviewed the agency's service performance goals
               and assessed the link between those goals and planned and expected
               training and professional development activities. We also
               interviewed NWS officials responsible for training and
               professional development activities.

               To evaluate the status and potential impact of any plans to revise
               the national office configuration, we assessed studies of options
               for changing the NWS concept of operations. We also reviewed the
               charter for the prototype and interviewed key NWS officials to
               determine the possible effect of these plans on local forecasting
               services, staffing, and budgets and to identify plans for
               determining the implications of changing to a new concept of
               operations.

               We performed our work at NWS headquarters in the Washington, D.C.,
               metropolitan area, and at geographically diverse NOAA and NWS
               weather forecast offices in Denver and in Tampa, and at the NWS
               National Hurricane Center in Miami. We performed our work from
               October 2005 to June 2006 in accordance with generally accepted
               government auditing standards.

               Source: GAO analysis of NOAA and NWS reports.

               aMetric measured between October 2005 and January 2006.

               bMetric measured between October 2005 and February 2006.

               cMetric measured between October 2005 and December 2005.

               dMetric measured between October 2005 and March 2006.

               eInstrument Flight Rules take effect when ceilings and
               visibilities are less than 1,000 feet and/or 3 miles,
               respectively, and ceilings and visibilities are greater than, or
               equal to, 500 feet and/or 1 mile, respectively.

               fData for this metric are not available until the beginning of the
               next calendar year because of the timing of the hurricane season.

               In the 1980s, NWS began a nationwide modernization program to
               upgrade weather observing systems such as satellites and radars,
               to design and develop advanced computer workstations for
               forecasters, and to reorganize its field office structure. The
               goals of the modernization were to achieve more uniform weather
               services across the nation, improve forecasting, provide more
               reliable detection and prediction of severe weather and flooding,
               achieve higher productivity, and permit more cost-effective
               operations through staff and office reductions.

               NWS's plans for revising its office structure were governed by the
               Weather Service Modernization Act,1 which required that, prior to
               closing a field office, the Secretary of Commerce certify that
               there was no degradation of service. NWS developed a plan for
               complying with the law. To identify community concerns regarding
               modernization changes and to study the potential for degradation
               of service, the Department of Commerce published a notice in the
               Federal Register requesting comments on service areas where it was
               believed that services could be degraded by planned modernization
               changes. The department also contracted for an independent
               assessment by the National Research Council on whether weather
               services would be degraded by the proposed changes. As part of
               this assessment, the contractor developed criteria to identify
               whether service would be degraded in certain areas of concern. The
               department then applied these criteria to areas of concern to
               determine whether services would be degraded or not. Before
               closing any office, the Secretary of Commerce certified that
               services would not be degraded.

               David A. Powner, (202) 512-9286 or pownerd@gao.gov .

               In addition to the contact named above, William Carrigg, Barbara
               Collier, Neil Doherty, Kathleen S. Lovett, Colleen Phillips, Karen
               Talley, and Jessica Waselkow made key contributions to this
               report.

               The Government Accountability Office, the audit, evaluation and
               investigative arm of Congress, exists to support Congress in
               meeting its constitutional responsibilities and to help improve
               the performance and accountability of the federal government for
               the American people. GAO examines the use of public funds;
               evaluates federal programs and policies; and provides analyses,
               recommendations, and other assistance to help Congress make
               informed oversight, policy, and funding decisions. GAO's
               commitment to good government is reflected in its core values of
               accountability, integrity, and reliability.

               The fastest and easiest way to obtain copies of GAO documents at
               no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
               GAO posts newly released reports, testimony, and correspondence on
               its Web site. To have GAO e-mail you a list of newly posted
               products every afternoon, go to www.gao.gov and select "Subscribe
               to Updates."

               The first copy of each printed report is free. Additional copies
               are $2 each. A check or money order should be made out to the
               Superintendent of Documents. GAO also accepts VISA and Mastercard.
               Orders for 100 or more copies mailed to a single address are
               discounted 25 percent. Orders should be sent to:

               U.S. Government Accountability Office 441 G Street NW, Room LM
               Washington, D.C. 20548

               To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
               (202) 512-6061

               Contact:

               Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
               fraudnet@gao.gov Automated answering system: (800) 424-5454 or
               (202) 512-7470

               Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400
               U.S. Government Accountability Office, 441 G Street NW, Room 7125
               Washington, D.C. 20548

               Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
               512-4800 U.S. Government Accountability Office, 441 G Street NW,
               Room 7149 Washington, D.C. 20548

                                    Agency Comments

    Appendix I: Objectives, Scope, and Methodology Appendix I: Objectives,
    Scope, and Methodology

    Appendix II: NWS Performance Goals for Fiscal Years 2005 to 2011 Appendix
    II: NWS Performance Goals for Fiscal Years 2005 to 2011

                                        FY05        FY06     FY07 FY08 FY09 FY10 FY11
                                                      Actual
    Performance                            Final          to
    measure            Description   Goal actual Goal   date Goal Goal Goal Goal Goal
    Tornado warning    The             13     13   13    13a   14   15   15   15   15
    lead time          difference
    (minutes)          between the
                       time a
                       warning is
                       issued and
                       the time of
                       the first
                       report of a
                       tornado in a
                       given county
    Tornado warning    The             73     75   76    82a   76   76   76   76   76
    accuracy (percent) percentage of
                       time a
                       tornado
                       actually
                       occurred in
                       an area
                       covered by a
                       tornado
                       warning
    Tornado warning    The             73     77   75    76a   74   74   74   74   74
    false-alarm rate   percentage of
    (percent)          time a
                       tornado
                       warning was
                       issued but no
                       tornado event
                       was reported
    Flash flood        The             48     54   48    63a   49   49   49   49   49
    warning lead time  difference
    (minutes)          between the
                       time a
                       warning is
                       issued and
                       the time of
                       the first
                       report of a
                       flash flood
                       in a given
                       county
    Flash flood        The             89     88   89    93a   90   90   90   90   90
    warning accuracy   percentage of
    (percent)          time a flash
                       flood
                       actually
                       occurred in
                       an area
                       covered by a
                       flash flood
                       warning
    Marine wind speed  A measure of    57     57   58    56b   58   58   59   59   59
    forecast accuracy  the accuracy
    (percent)          of wind speed
                       forecasts
    Marine wave height A measure of    67     67   68    71b   68   68   69   69   69
    forecasts accuracy the accuracy
    (percent)          of wave
                       forecasts
    Aviation forecast  The             46     46   47    45b   48   51   52   53   59
    Instrument Flight  percentage of
    Rule               time
    ceiling/visibility Instrument
    accuracy (percent) Flight Rule
                       conditionse
                       are predicted
                       and occur
    Aviation forecast  The             68     63   65    61b   64   58   57   56   50
    Instrument Flight  percentage of
    Rule               time
    ceiling/visibility Instrument
    false-alarm rate   Flight Rule
    (percent)          conditionse
                       are predicted
                       but do not
                       occur
    Winter storm       The average     15     17   15    16c   15   15   16   17   17
    warning lead time  time from the
    (hours)            issuance of a
                       warning to
                       the time of
                       the first
                       report of a
                       winter storm
                       in a given
                       county
    Winter storm       The             90     91   90    91c   90   90   91   92   92
    warning accuracy   percentage of
    (percent)          verified
                       winter storm
                       events that
                       were covered
                       by winter
                       storm
                       warnings
    Precipitation      A score based   27     29   28    39d   29   29   29   30   30
    forecast day 1     on the
    threat (score)     agency's
                       accuracy in
                       forecasting
                       precipitation
    U.S. seasonal      A score based   18     19   18    24d   19   19   19   20   20
    temperature        on the
    forecast skill     agency's
    (score)            accuracy in
                       forecasting
                       temperature
    Hurricane track    A measure of   128    101  111   N/Af  110  109  108  107  106
    forecasts at 48    the
    hours (nautical    difference
    miles)             between the
                       projected
                       locations of
                       the center of
                       storms and
                       the actual
                       locations in
                       nautical
                       miles for the
                       Atlantic
                       Basin

    Appendix III: NWS Previously Used A Stringent Process to Ensure Service
    Was Not Degraded Appendix III: NWS Previously Used A Stringent Process to
    Ensure Service Was Not Degraded

    1Pub. L. 102-567 S: 706(b), 106 Stat. 4303, 4306 (1992).

    Appendix IV: Comments from the Department of Commerce Appendix IV:
    Comments from the Department of Commerce

    Appendix V: A Appendix V: GAO Contact and Staff Acknowledgments

                                      GAO Contact

                                 Staff Acknowledgments

    (310817)

    GAO's Mission

    Obtaining Copies of GAO Reports and Testimony

    Order by Mail or Phone

    To Report Fraud, Waste, and Abuse in Federal Programs

    Congressional Relations

    Public Affairs

    www.gao.gov/cgi-bin/getrpt? GAO-06-792 .

    To view the full product, including the scope

    and methodology, click on the link above.

    For more information, contact David Powner at (202) 512-9286 or
    pownerd@gao.gov.

    Highlights of GAO-06-792 , a report to the Subcommittee on Environment,
    Technology, and Standards, Committee on Science, House of Representatives

    July 2006

    WEATHER FORECASTING

    National Weather Service Is Planning to Improve Service and Gain
    Efficiency, but Impacts of Potential Changes Are Not Yet Known

    To provide accurate and timely weather forecasts, the National Weather
    Service (NWS) uses systems, technologies, and manual processes to collect,
    process, and disseminate weather data to its nationwide network of field
    offices and centers. After completing a major modernization program in the
    1990s, NWS is seeking to upgrade its systems with the goal of improving
    its forecasting abilities, and it is considering changing how its
    nationwide office structure operates in order to enhance efficiency. GAO
    was asked to (1) evaluate NWS's efforts to achieve improvements in the
    delivery of its services through system and technology upgrades, (2)
    assess agency plans to achieve service improvements through training its
    employees, and (3) evaluate agency plans to revise its nationwide office
    configuration and the implications of these plans on local forecasting
    services, staffing, and budgets.

    What GAO Recommends

    GAO is making recommendations to the Secretary of Commerce to direct NWS
    to strengthen its training selection process, and to establish key
    activities, timelines, and measures for evaluating the prototype of a new
    concept of operations before beginning the prototype. In written comments,
    the Department of Commerce agreed with the recommendations and identified
    plans for implementing them.

    NWSis positioning itself to provide better service through over
    $315millionin planned upgrades to its systems and technologies. In annual
    plans, the agency links expected improvements in its service performance
    measures with the technologies and systems expected to improve them. For
    example, NWS expects to reduce the average error in its forecasts of
    hurricane paths by approximately 20 nautical miles between 2005 and 2011
    through a combination of upgrades to observation systems, better hurricane
    forecast models, enhancements to the computer infrastructure, and research
    that will be transferred to forecast operations. Also, NWS expects to
    increase tornado warning lead times from 13 to 15 minutes by the end of
    fiscal year 2008 after the agency completes an upgrade to its radar system
    and realizes benefits from software improvements to its forecaster
    workstations.

    NWS also provides training courses for its employees to help improve its
    forecasting services, but the agency's process for selecting training
    lacks sufficient oversight. Program officials propose and justify training
    needs on the basis of up to eight different criteria-including whether a
    course is expected to improve NWS forecasting performance measures,
    support customer outreach, or increase scientific awareness. Many of these
    course justifications appropriately demonstrate support for improved
    forecasting performance. For example, training on how to more effectively
    use forecaster workstations is expected to help improve tornado and
    hurricane warnings. However, in justifying training courses, program
    officials routinely link courses to NWS forecasting performance measures.
    For example, in 2006, almost all training needs were linked to
    expectations for improved performance-including training on
    cardiopulmonary resuscitation, spill prevention, and systems security. The
    training selection process did not validate or question how these courses
    could help improve weather forecasts. Overuse of this justification
    undermines the distinctions among different training courses and the
    credibility of the course selection process. Additionally, because the
    training selection process does not clearly distinguish among courses, it
    is difficult to determine whether sufficient funds are dedicated to the
    courses that are expected to improve performance.

    To improve its efficiency, NWS plans to develop a prototype of a new
    concept of operations, an effort that could affect its national office
    configuration, including the location and functions of its offices
    nationwide. However, many details about the impact of any proposed changes
    on NWS forecast services, staffing, and budget have yet to be determined.
    Further, the agency has not yet determined key activities, timelines, or
    measures for evaluating the prototype of the new office operational
    structure. As a result, it is not evident that NWS will collect the
    information it needs on the impact and benefits of any office
    restructuring in order to make sound and cost-effective decisions.
    *** End of document. ***