Weather Forecasting: NWS Has Not Demonstrated That New Processing System
Will Improve Mission Effectiveness (Letter Report, 02/29/96,
GAO/AIMD-96-29).
This report focuses on the National Weather Service's (NWS) Advanced
Weather Interactive Processing System, which is designed to help local
weather forecasters obtain meteorological data from state-of-the-art
weather observing systems and national weather models, analyze trends,
and disseminate forecasts and warnings to the public. This $525 million
system is to be the centerpiece of NWS' massive $4.5 billion
modernization and restructuring program. GAO discusses whether NWS'
process for developing the system has shown that all proposed system
capabilities will contribute to promised modernization outcomes--better
forecasts, fewer weather offices, and reduced staffing. GAO summarized
this report in testimony before Congress; see: Weather Forecasting: New
Processing System Faces Uncertainties and Risks, by Jack L. Brock, Jr.,
Director of Information Resources Management Issues, before the
Subcommittee on Energy and the Environment, House Committee on Science.
GAO/T-AIMD-96-47, Feb. 29 (six pages).
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-96-29
TITLE: Weather Forecasting: NWS Has Not Demonstrated That New
Processing System Will Improve Mission Effectiveness
DATE: 02/29/96
SUBJECT: Weather forecasting
Systems conversions
Computer software verification and validation
Systems design
Requirements definition
Computer prototypes
Image data bases
Wide area networks
Information systems
IDENTIFIER: NWS AWIPS Forecast Preparation System
NWS Advanced Weather Interactive Processing System
NWS Interactive Computer Worded Forecast
NWS Next Generation Weather Radar
FAA Automated Terminal Doppler Radar Warning System
NWS Automation of Field Operations and Services System
NWS Denver AWIPS Risk Reduction and Requirements Evaluation
Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman, Committee on Science, House of
Representatives
February 1996
WEATHER FORECASTING - NWS HAS NOT
DEMONSTRATED THAT NEW PROCESSING
SYSTEM WILL IMPROVE MISSION
EFFECTIVENESS
GAO/AIMD-96-29
Weather Forecasting
(511399)
Abbreviations
=============================================================== ABBREV
AFPS - AWIPS Forecast Preparation System
AFOS - Automation of Field Operations and Services
ASOS - Automated Surface Observing System
AWIPS - Advanced Weather Interactive Processing System
DAR\3 E - Denver AWIPS Risk Reduction and Requirements Evaluation
GOES-Next - Next Generation Geostationary Operational Environmental
Satellites
ICWF - Interactive Computer Worded Forecast
NEXRAD - Next Generation Weather Radar
NOAA - National Oceanic and Atmospheric Administration
NWS - National Weather Service
OMB - Office of Management and Budget
PRC - Planning Research Corporation
PROTEUS - Prototype River Forecast Center Operational Test,
Evaluation, and User Simulation
Letter
=============================================================== LETTER
B-266117
February 29, 1996
The Honorable Robert S. Walker
Chairman, Committee on Science
House of Representatives
Dear Mr. Chairman:
This report responds to your request concerning the National Weather
Service's (NWS) Advanced Weather Interactive Processing System
(AWIPS). AWIPS is an information processing system to support local
weather forecasters in acquiring meteorological data from
state-of-the-art weather observing systems and national weather
models, analyzing these data, and disseminating the resulting
forecasts and warnings to the public. AWIPS, which NWS estimates
will cost about $525 million to develop and deploy, is to be the
centerpiece of NWS' massive, $4.5 billion modernization and
restructuring program.
As agreed with your office, our objective was to determine whether
NWS' process for developing AWIPS has demonstrated that all proposed
system capabilities will contribute to promised modernization
outcomes--improved forecasts, fewer weather offices, and reduced
staffing levels. Appendix I provides a detailed explanation of our
objective, scope, and methodology.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
NWS has yet to demonstrate that all envisioned AWIPS capabilities are
needed to meet NWS' three stated mission improvement goals of better
forecasts, fewer weather offices, and lower staffing levels. In
short, AWIPS' 22,000 requirements, such as "zooming-in" on displayed
images and vividly coloring areas of intense weather, are to provide
a collection of roughly 450 higher order capabilities, such as
viewing and manipulating images. However, NWS has not justified
these higher-level capabilities on the basis of mission impact. In
the absence of such evidence, NWS runs the risk of spending money for
AWIPS capabilities that do not contribute to accomplishing NWS'
mission more effectively, efficiently, or economically.
To NWS' credit, it has clearly shown that a replacement for the
obsolete information processing, display, and communication systems
currently used by NWS field offices is sorely needed. Additionally,
NWS has performed several valuable requirements definition and
validation activities. For example, NWS has effectively used system
prototyping to define users' needs and demonstrate the technological
feasibility of these needs. Further, NWS currently has a process
underway for validating that AWIPS' requirements are not duplicative
or technologically obsolete. However, these activities do not trace
the full complement of planned AWIPS capabilities to NWS' stated
mission improvement goals.
BACKGROUND
------------------------------------------------------------ Letter :2
Since the early 1980s, NWS has been modernizing its observing,
information processing, and communications systems to improve the
accuracy, timeliness, and efficiency of weather forecasts and
warnings. The
$4.5 billion modernization includes four major systems--AWIPS, the
Next Generation Geostationary Operational Environmental Satellites
(GOES-Next), the Next Generation Weather Radars (NEXRAD), and the
Automated Surface Observing System (ASOS).\1
GOES-Next, NEXRAD, and ASOS form the foundation of NWS' observing
infrastructure, watching the weather from earth and space. The two
orbiting GOES-Next satellites each provide as many as 1,200 digital
weather images daily to each forecast office, compared to less than
100 from their predecessors. The almost 150 NEXRADs are to blanket
the continental United States and using Doppler radar technology,
allow forecasters to see inside weather events to detect motion and
dynamics that were invisible to pre-NEXRAD radars. ASOS, which
consists of nearly 900 separate ground-based sensor sets, is to
provide a portfolio of weather readings, such as temperature and
visibility, from more locations and with greater frequency than has
been done by human observers.
--------------------
\1 According to NWS, the estimated project costs for these four
systems are approximately $525 million, $2.0 billion, $1.4 billion,
and $351 million, respectively. The remainder of the modernization's
cost is attributable to several much smaller projects.
AWIPS AT A GLANCE
---------------------------------------------------------- Letter :2.1
AWIPS is to function as the "central nervous system" of a modernized
NWS. That is, it is to be the information processing and display
system that forecasters use to integrate, analyze, and graphically
view the immense number of weather observations and products that
form the basis for each day's weather and river forecasts and
warnings. It is also to be the national communications
infrastructure for NWS' many forecasting offices and centers,\2
connecting them not only to each other but also linking them to the
many users of forecasts and warnings throughout the nation.
Using AWIPS' advanced processing, display, and communications
capabilities, NWS expects to fully capitalize on its new observing
systems for the first time. Without AWIPS, these observing systems
cannot be maximized because the current Automation of Field
Operations and Services (AFOS) computer and communications system and
the associated weather office display systems cannot accommodate the
mountainous data streams that these observing systems now provide.
For example, the current system can only accept two satellite images
an hour. With AWIPS, GOES-Next images are to be received in real
time, meaning that up to eight images can be received and displayed
each hour.
--------------------
\2 These sites include 119 Weather Forecast Offices, 13 River
Forecast Offices, the National Hurricane Center, and the National
Severe Storms Forecast Center.
OVERVIEW OF AWIPS' EXPECTED
IMPACT ON NWS' MISSION
PERFORMANCE
---------------------------------------------------------- Letter :2.2
According to NWS, AWIPS is to contribute to improvements in the
accuracy and timeliness of forecasts and warnings as well as
streamlining its operations and downsizing its organization. All
told, NWS expects that the combined pieces of the modernization will
result in the number of NWS field offices dropping from 254 to 119
and the number of NWS staff falling from 5,100 to 4,678. AWIPS is
expected to improve forecaster productivity by allowing forecasters
to view disparate data sets in an integrated fashion, perform an
assortment of scientific computations on these data sets, and
graphically display and interact with these data sets. Currently,
such activities are generally performed manually as data from the
various observing systems are displayed on multiple screens and
assimilated by the forecaster. For example, today, when forecasters
want to combine radar and satellite images to view weather pattern
movements, the images must be manually overlayed on transparencies.
With AWIPS, such integration is expected to occur automatically on
the AWIPS workstation with a few simple mouse clicks or keystroke
commands. In short, AWIPS is to result in forecasters spending less
time physically and mentally manipulating data and more time
practicing meteorology.
AWIPS HISTORY AND STATUS IN
BRIEF
---------------------------------------------------------- Letter :2.3
The National Oceanic and Atmospheric Administration (NOAA), which is
NWS' parent agency, began AWIPS in the mid-1980s to replace AFOS.
Since then, NWS has invested considerable time and effort in
analyzing and defining AWIPS' requirements, effectively involving
users, as we reported in 1993,\3 through hands-on experience with
prototypes. In 1992, NOAA awarded the AWIPS development contract to
the Planning Research Corporation (PRC). Because of the contractor's
failure to deliver an acceptable AWIPS design, NWS renegotiated the
contract in 1995, basically assuming responsibility for development
of all hydrology and meteorology application software\4 and assigning
the contractor responsibility for delivering the hardware and systems
software\5 and for integrating the entire system. To fulfill its
responsibility, NWS established joint NWS and contractor applications
software (for example, the software that executes atmospheric and
hydrological numerical and statistical models, manipulates satellite
and radar graphics, etc.) development teams.
NWS' current plans call for building and integrating AWIPS in seven
increments.\6
Thus far, NWS and the contractor have installed a very limited
version of AWIPS, the first increment, at three sites to gain some
experience in developing, testing, implementing, and operating a
limited capability AWIPS. This limited version is also intended to
validate selected AWIPS architectural features, such as satellite
broadcasts and the central communications and system monitoring
"hub." NWS has begun development of the second increment. NWS
estimates that AWIPS will cost $525 million to fully develop and
deploy.\7
Deployment of less than the full AWIPS capability to NWS field
offices and national centers is now scheduled to begin in 1996. Full
AWIPS deployment is scheduled to begin in 1999.
--------------------
\3 Weather Forecasting: Important Issues on Automated Weather
Processing System Need Resolution (GAO/IMTEC-93-12BR, January 6,
1993).
\4 Application software is the collection of computer programs that
allows a user to perform a specific job or task, such as creating a
table, writing a letter, playing a game, or in the case of AWIPS,
modeling the flood impact of precipitation data or generating
graphical images of weather movement and intensity.
\5 Systems software is the collection of computer programs that
manage the computer system's hardware components (for example,
central processing unit, disk drives, input and output devices) and
that allow the application software to interact with this hardware.
Examples of systems software are the operating system, the data base
management system, and the compiler.
\6 Incremental development employs a build-a-little, test-a-little
approach in which software products are developed in a series of
increments of increasing functional capability, that is, software is
partitioned into increments whose development is phased over the
total development cycle.
\7 We did not independently verify this estimate.
A HIERARCHICAL DESCRIPTION
OF AWIPS FUNCTIONS,
CAPABILITIES, AND
REQUIREMENTS
---------------------------------------------------------- Letter :2.4
AWIPS consists of about 22,000 requirements that have been grouped
into about 450 higher-level capabilities. These capabilities are
described in the AWIPS System/Segment Specification, commonly
referred to as the "A-Spec," which relates about three-fourths of the
capabilities to five broad functional areas. The five functional
areas are (1) communications, (2) monitoring and control, (3)
processing, (4) display and interaction, and (5) data management.
The first two functional areas constitute what NWS calls the AWIPS
network segment (that is, the national communications
infrastructure). The latter three are referred to by NWS as the
AWIPS site segment (that is, the functionality applicable to AWIPS
sites). Appendix II provides examples of AWIPS capabilities for each
functional area. The remaining one-fourth relate to such
capabilities as AWIPS' performance, security, availability, and
flexibility. Figure 1 depicts this AWIPS hierarchy.
Figure 1: AWIPS Hierarchy
(See figure in printed
edition.)
NWS CANNOT SHOW THAT ALL AWIPS'
PROPOSED CAPABILITIES ARE
RELATED TO MISSION IMPROVEMENTS
------------------------------------------------------------ Letter :3
Investments in information technology, like AWIPS, should be
justified on the basis of whether or not all planned system
capabilities will make a clear difference in advancing mission
efficiency and effectiveness (for example, improved forecasts and
warnings). NWS has not done this for AWIPS. According to NWS
officials, they have not explicitly linked either AWIPS requirements
or higher-level capabilities to mission improvements, and they have
no plans to do so because they claim that other requirements reviews,
analyses, and validation activities already provide implicit
justification for all AWIPS' proposed capabilities. We disagree. We
carefully reviewed these other activities and while we found them to
be valuable for different reasons, they were neither intended to nor
do they demonstrate that AWIPS' full array of capabilities will
improve NWS mission effectiveness. As a result, NWS risks
unnecessarily spending money on AWIPS capabilities that do not
satisfy any of its mission improvement goals--better forecasts, fewer
field offices, and fewer staff.
SYSTEM CAPABILITIES SHOULD
BE VALIDATED TO MISSION
IMPROVEMENTS
---------------------------------------------------------- Letter :3.1
Office of Management and Budget (OMB) Circular A-130, Management of
Federal Information Resources, requires agencies to create and
maintain management and technical frameworks that define linkages
between mission needs and information technology capabilities. OMB
Circular A-109, "Major System Acquisitions," expands on A-130,
requiring federal agencies to make system design decisions based on a
review of proposed system functional and performance capabilities
contributions to mission needs and program objectives. In effect,
agencies developing computer systems, like AWIPS, are to show that
proposed system capabilities will produce some mission effectiveness
or efficiency gain, like more reliable and timely forecasts or office
and staffing reductions.
These requirements are consistent with our recent findings on how
leading public and private organizations tie technology investments
to measurable mission improvements.\8 We found that successful
organizations' information system investment decisions are tied to
explicit and quantifiable mission improvements. By doing so, these
organizations know that investing in system requirements will make a
difference in mission outcomes.
Ensuring that proposed system capabilities are justified before
expensive software development begins requires validating (that is,
proving) that system requirements are anchored in user needs, which
in turn are grounded in positive mission impacts. To do less
increases the chances of spending money on capabilities that, even
though desired by users, will not advance the organization's
effectiveness or efficiency. Accordingly, software development
guidance advocates assuring traceability from derived system
requirements, designs, and implementations to both original user
needs and mission needs.\9 Approaches to validating proposed
capabilities include performance modeling and prototyping.
--------------------
\8 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology (GAO/AIMD-94-115, May 1994).
\9 Guidelines for Successful Acquisition and Management of Software
Intensive Systems, Department of the Air Force, Software Technology
Support Center, February 1995.
DESPITE PAST AND ONGOING
REQUIREMENTS REVIEWS, NWS
HAS YET TO JUSTIFY AWIPS'
CAPABILITIES ON THE BASIS OF
MISSION RESULTS
---------------------------------------------------------- Letter :3.2
According to NWS, planned AWIPS capabilities are necessary and will
contribute to NWS' goals of weather forecast and warning
improvements, field office consolidation, and staff reductions.
However, NWS officials were unable to produce any analysis or
associated documentation to validate this claim. Instead, they
presented the results of past AWIPS requirements analysis and
definition activities and discussed ongoing requirements validation
activities that, while useful in their own right, do not justify
AWIPS capabilities on the basis of mission improvements. Each of
these requirements review activities are discussed below.
The first AWIPS requirements review began in 1984, when NWS initiated
the Denver AWIPS Risk Reduction and Requirements Evaluation (DAR\3 E)
program, an extensive AWIPS prototyping effort to analyze and refine
meteorology requirements. Later, NWS augmented the DAR\3 E effort
with an equally impressive prototyping effort addressing hydrology
requirements, which NWS labeled the Prototype River Forecast Center
Operational Test, Evaluation, and User Simulation (PROTEUS). Through
these efforts, user observations, feedback, and suggestions
concerning such things as data integration, access, animation,
presentation, as well as workstation performance were obtained and
used to revise AWIPS requirements and specifications.
Without question, these prototyping efforts effectively involved
users in the process of refining requirements and contributed
immensely to establishing a set of AWIPS requirements that reflected
the wishes of NWS' forecasting community. However, AWIPS
requirements and capabilities, as currently planned, extend beyond
these prototypes. According to NWS officials, the DAR\3 E prototype
addressed roughly half of AWIPS' 22,000 requirements. The half that
were addressed equates to over three-fourths of AWIPS' total lines of
code. Moreover, even those AWIPS capabilities that were part of the
prototypes, with one limited exception, were not explicitly linked to
measurable improvements in NWS' mission effectiveness during these
prototyping activities. The one exception is an NWS analysis linking
DAR\3 E to increased warning lead times, more specific warning
locations, decreased warning durations, and reduced false alarms.
However, this same analysis attributed the lion's share of the
improvements to NEXRAD, which NWS officials also stated was the true
reason for the improvements.
On the basis of early AWIPS prototyping and the efforts of the AWIPS
Requirements Task Team, system requirements were again validated to
user needs in 1985. According to AWIPS requirements traceability
documents, the AWIPS System Requirements Specification was traced to
user-generated AWIPS requirements, known as the user language
specification, that resulted from the early prototyping and the task
team's efforts. Again, this validation activity was valuable in
ensuring that proposed AWIPS capabilities are anchored in user needs.
However, it says nothing with respect to whether they are rooted in
mission-based goals.
Another major review effort occurred in 1991 and 1992 when NWS
performed what it calls the "AWIPS rebaselining." Briefly, this
effort entailed prioritizing AWIPS' capabilities, and designating
which capabilities needed to be developed first and which could be
postponed to a later development phase. The rebaselining did not
attempt to justify pursuit of these capabilities on the basis of
contribution to mission performance gains.
Since 1991, follow-on DAR\3 E activities occurred at the Norman,
Oklahoma, weather office to emulate the operations of a future,
modernized weather office. According to NWS, the emulation examined
such things as the AWIPS user interface for displaying NEXRAD
products, AWIPS' integration of national weather products (e.g.,
satellite imagery) with local data (e.g., NEXRAD products), and
future weather office staffing levels. However, we reviewed the
results of these emulation activities and found no evidence
validating AWIPS-specific capabilities on the basis of stated NWS
mission goals of better forecasts, fewer weather offices, or less
staff. For example, one report concluded that the AWIPS prototype
provided capabilities for viewing NEXRAD data that forecasters found
"useful," but does not show the mission outcome of having the
capabilities. Another report that was AWIPS-specific concluded that
a proposed AWIPS capability known as Interactive Computer Worded
Forecast (ICWF), which is later to be replaced by the AWIPS Forecast
Preparation System (AFPS), actually decreased rather than increased
forecaster productivity and should not be deployed in its current
form.
In 1994, in response to the contractor's earlier mentioned failure to
produce an AWIPS design, NWS undertook what it refers to as a
"functional decomposition" of the 22,000 AWIPS requirements. In
effect, NWS placed these requirements into about 450 capability
categories. These categories are the foundation of the AWIPS
"A-Spec," which, as mentioned earlier, is the high-level system
specification that further combines most of the 450 capabilities into
five broad functional areas. (See figure 1.) Clearly, the
development of the "A-Spec" was a valuable undertaking in that it
translated the 22,000 AWIPS requirements into a smaller, simpler,
more understandable set of high-level functions. However, this
translation did not, nor was it intended to, link requirements or
capabilities to mission improvements.
Beginning in 1994 and continuing through today, NWS also has been
reviewing all AWIPS requirements to identify any that are, in its
words, "archaic," meaning that they are technologically obsolete,
duplicative of other AWIPS requirements, or in need of modification
to comply with AWIPS' system design. Thus far, NWS has reviewed
about 900 of the 1,000 requirements it dubbed as potentially
extraneous and chose to eliminate about 600. The remaining 100 still
need to be evaluated. Again, this requirements "scrub" has been and
continues to be worthwhile. However, NWS' above-cited criteria do
not address whether the proposed requirement or capability will
produce a measurable mission improvement.
At the same time NWS is completing its review of the aforementioned
"archaic" requirements, the joint NWS/contractor teams established to
develop AWIPS' application software are examining requirements one
last time before building each software module. According to the
AWIPS Software Development Plan, the required AWIPS scientific
applications in many instances were written several years ago and may
still contain ones that are obsolete. However, the NWS official
responsible for overseeing the joint development teams stated that
the teams' process for reviewing the requirements does not attempt to
validate requirements back to mission improvements. Also, the
official stated that the focus of the reviews is on reaching a common
understanding on how best to proceed in developing the software
module.
CONCLUSIONS
------------------------------------------------------------ Letter :4
NWS clearly needs a new, modern system to support its current
operations and allow it to take advantage of the vast data streams
now available through its new observing systems. However, whether
all of the 450 capabilities it plans for AWIPS are necessary to
accomplish this is unknown because the process it has followed in
developing AWIPS, while providing for traceability between proposed
system capabilities and user expressed needs, does not include
validating that these capabilities explicitly and measurably advance
NWS' mission efficiency and effectiveness, which NWS has defined in
terms of improved forecasts, fewer field offices, and reduced
staffing levels. Validating system capabilities to mission outcomes
is vital because it confirms prior to costly software development
that proposed system capabilities are grounded in and will contribute
to NWS' mission goals. Unless NWS expands ongoing AWIPS requirements
review activities to include demonstrating, either quantitatively or
qualitatively, that proposed capabilities measurably advance NWS'
mission abilities, it risks spending money to develop capabilities
that are not justified.
RECOMMENDATIONS
------------------------------------------------------------ Letter :5
We recommend that the Secretary of Commerce direct the NOAA Assistant
Administrator for Weather Services to (1) expand ongoing AWIPS
requirements review activities to include validation that proposed
capabilities are justified on the basis of mission impact and (2) not
implement any of those capabilities that are not validated. At a
minimum, such validation should include analyses of data and factual
accounts from past and ongoing AWIPS prototype experiences that link
those proposed capabilities to stated mission improvement goals.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :6
On January 24, 1996, we discussed a draft of this report with NOAA
and NWS officials, including the NOAA Associate Administrator for
Weather Services, the Deputy Assistant Administrator for Operations,
and the Deputy Assistant Administrator for Modernization. In
general, these officials did not agree with the report's conclusions
and recommendations, reiterating the NWS positions that were in our
draft report. In particular, they stated that extensive AWIPS
requirements validation activities have occurred and are ongoing.
They also stated that only AWIPS capabilities that are essential to
NWS' mission are being pursued, and that their inability to prove
that mission-based requirements validation activities were performed
is not sufficient to conclude that AWIPS is not needed. They
promised to provide additional documentation to show that AWIPS
proposed capabilities are grounded in mission impacts.
We agree that extensive validation activities have occurred and are
still ongoing, and we give NWS credit for these activities in the
report. Unfortunately, NWS' validation activities have only dealt
with a part of the validation equation and have not validated AWIPS
capabilities to its stated mission outcomes of better forecasts,
fewer field offices, and fewer staff. This is completely at odds
with our recent findings on how leading public and private sector
organizations base successful technology investments on whether they
produce meaningful improvements in the cost, quality, and timeliness
of product and service delivery.\10
Further, we neither state nor imply that AWIPS is not needed.
Rather, we are saying that NWS is spending hundreds of millions of
dollars without knowing whether all AWIPS capabilities will
contribute to its stated reasons for investing in the system
(improving forecasts and reducing field offices and staffing levels).
Restated, while we do not question the need to replace AFOS, we do
question whether AWIPS, with all the capabilities that NWS currently
envisions it providing, should be that replacement. Unless NWS
addresses this question, it risks spending money for capabilities
that do not advance its mission performance. Fortunately, NWS has
the opportunity to perform this validation activity as part of
already ongoing and planned requirements reviews. We strongly
encourage NWS to take advantage of this opportunity.
We reviewed the additional documentation that NWS officials provided
and have included it in the report as further evidence of NWS'
thorough validation of AWIPS capabilities to user needs. However,
this documentation does not show that AWIPS capabilities are anchored
in mission improvements. We have incorporated other comments made by
the officials in the report where appropriate.
We are sending copies of this report to the Secretary of Commerce,
the Director of the Office of Management and Budget, and interested
congressional committees. Copies will also be made available to
others upon request.
Please call me at (202) 512-6240 if you or your staff have any
questions concerning this report. Other major contributors are
listed in appendix III.
Sincerely yours,
Jack L. Brock, Jr.
Director, Information Resources
Management/Resources, Community, and
Economic Development
--------------------
\10 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology.
OBJECTIVE, SCOPE, AND METHODOLOGY
=========================================================== Appendix I
The objective of our review was to determine whether NWS' process for
developing AWIPS has demonstrated that all proposed system
capabilities will contribute to promised modernization
outcomes--improved forecasts, fewer weather offices, or reducing
staffing levels. To determine this, we interviewed program officials
and reviewed system development documentation to document past and
ongoing steps to validate AWIPS requirements. In particular, we
reviewed analyses of AWIPS' prototyping efforts, memoranda from the
1992 rebaselining of the AWIPS requirements, the System/Segment
Specification for the National Weather Service Advanced Weather
Interactive Processing System, and the Requirements Traceability
Document for the AWIPS Hydrometeorological Computer Software
Configuration Item. We also sought program officials explanations of
how AWIPS requirements are tied to and will result in improved
forecasts, weather office closings, and staff reductions.
Concerning ongoing AWIPS requirements reviews, we interviewed NWS
staff currently reviewing AWIPS requirements to determine the purpose
of the reviews and the criteria being used to assess the
requirements. In addition, we interviewed the NWS official
overseeing the NWS/contractor teams developing the AWIPS software
modules to learn what validation steps and criteria the teams are
employing.
We provided a draft of this report to the Department of Commerce for
comment. On January 24, 1996, we obtained oral comments from NOAA
and NWS officials. These comments have been incorporated in the
report as appropriate.
We performed our work at the AWIPS program office, and at NOAA and
NWS headquarters offices in Silver Spring, Maryland, from August 1995
through January 1996 in accordance with generally accepted government
auditing standards.
EXAMPLES OF AWIPS CAPABILITIES BY
FUNCTIONAL AREA
========================================================== Appendix II
This appendix provides examples of AWIPS capabilities for each of the
five functional areas.
Functional area Examples of capabilities
------------------ --------------------------------------------------
Communications --Acquire data automatically when permitted by the
observation systems.
--Acquire data through external interfaces at the
National Meteorological Center segment, including,
polar orbiter data, lightning data, and NEXRAD
base and derived products.
--Acquire data from network segment external
interfaces, including formatted GOES-Next
products, NEXRAD summary and winds data, surface
and upper air observations, and data from GOES-
Next data collection platforms.
--Distribute data among AWIPS sites.
--Request data (excluding NEXRAD products and
satellite imagery) from another AWIPS site.
--Specify the distribution control parameters,
including data destinations, data to be
distributed, data prioritization, and destinations
required to acknowledge receipt of a high-
priority product.
--Disseminate data when requested by an external
user.
--Disseminate hazardous weather products
designated by the user to external users
automatically.
Monitor and --Provide NWS with dedicated, electronic access to
control the network control facility.
--Monitor the integrity and timeliness of data and
products that are acquired or disseminated over a
central interface.
--Notify users when degradations and malfunctions
of site equipment and communications interfaces
are determined.
--Provide an orderly shutdown upon detection of a
system failure.
--Control the display of information on
workstations at other AWIPS sites.
--Provide the capability for users to remotely
install software at another site.
--Provide an interactive, graphical method to
allow the user to define two unique alert areas
for monitoring NEXRAD data.
Processing --Spatially transform a point and grid of points
from one map projection, coordinate system, and
grid definition to another by interpolation.
--Execute one-dimensional numerical cloud models
for forecasting cloud top heights, vertical
velocity, and hail sizes.
--Execute a numerical model to forecast icing
potential for aircraft.
--Execute a simplified dam break channel flow
model.
--Compute tide and water level heights and
departures.
--Compute extraterrestrial radiation parameters.
--Generate combined reflectivity/velocity products
from NEXRAD data.
--Produce three-dimensional image perspective
displays.
--Perform image inversions, on a pixel-by-pixel\a
basis.
--Perform image-sharpening and edge-enhancement on
images.
--Display the hydrometeorological field on cross-
section and time-section plots using contours,
plotted values, and wind symbols.
--Produce graphical pilot weather briefing
displays, including the depiction of the current
and forecast conditions along the flight route
plotted on a cross-section context background.
Display and --Simultaneously display up to at least eight data
interaction windows on each workstation monitor.
--Toggle between the components of a combined
image.
--Zoom in and out on displayed image and graphics
products with zoom ratios up to 8:1.
--Generate color slides, prints, and
transparencies of displayed data.
--Step frame-by-frame through an animation loop.
--Edit elements of a displayed graphic and its
attributes on a workstation image/graphics
monitor.
Data management --Retrieve data stored locally.
--Specify retrieval criteria, such as all
temperatures above a certain threshold value.
--Store and retrieve hydrometeorological (for
example, satellite data, observational data),
cartographic (for example, geopolitical
boundaries, topography), site management (for
example, region information, maintenance
activities), and event data (for example, systems
errors, performance parameters).
--Archive data at the network segment and all site
segments.
----------------------------------------------------------------------
\a Pixels, also known as picture elements, are the tiny dots that
collectively form a grid on a computer screen and that when turned on
in specific patterns form characters or drawings.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
Rona B. Stillman, Chief Scientist for Computers and Communications
Randolph C. Hite, Assistant Director
Keith A. Rhodes, Technical Assistant Director
David A. Powner, Evaluator-in-Charge
Robert C. Reining, Information Systems Analyst
Colleen M. Phillips, Information Systems Analyst
*** End of document. ***