National Test Facility: Civilian Agency Use of Supercomputers Not
Feasible (Letter Report, 12/09/94, GAO/AIMD-95-28).

On the basis of its interviews with the civilian agencies cited in the
House Armed Services Committee report accompanying the National Defense
Authorization Act for fiscal year 1995, GAO concludes that none of the
agencies would be able to make effective use of the National Test
Facility's (NTF) excess supercomputing capabilities.  These agencies
said that they could not use the resources mainly because (1) NTF's
supercomputers are older machines whose performance and costs cannot
match those of more advanced computers available from other sources and
(2) some agencies have not yet developed applications requiring
supercomputer capabilities or lack funding to support such activities.
In addition, future support for the hardware and the software at NTF is
uncertain, making any investment by an outside user risky.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  AIMD-95-28
     TITLE:  National Test Facility: Civilian Agency Use of 
             Supercomputers Not Feasible
      DATE:  12/09/94
   SUBJECT:  Supercomputers
             Computer modeling
             Command/control/communications/computer systems
             Interagency relations
             Systems development life cycle
             Information resources management
             Capacity management
             Information systems
IDENTIFIER:  Cray 2 Supercomputer
             Cray Y-MP C90 Supercomputer
             IBM RS/6000 590 Supercomputer
             Silicon Graphics TFP Supercomputer
             
*******************************************************************************
*   This file contains an ASCII representation of the text of a GAO           *
*   report.  Delineations within the text indicating chapter titles,          *
*   headings, and bullets are preserved.  Major divisions and subdivisions    *
*   of the text, such as Chapters, Sections, and Appendixes, are              *
*   identified by double and single lines.  The numbers on the right end      *
*   of these lines indicate the position of each of the subsections in the    *
*   document outline.  These numbers do NOT correspond with the page          *
*   numbers of the printed product.                                           *
*                                                                             *
*   No attempt has been made to display graphic images, although figure       *
*   captions are reproduced. Tables are included, but may not resemble        *
*   those in the printed version.                                             *
*                                                                             *
*   A printed copy of this report may be obtained from the GAO Document       *
*   Distribution Facility by calling (202) 512-6000, by faxing your           *
*   request to (301) 258-4066, or by writing to P.O. Box 6015,                *
*   Gaithersburg, MD 20884-6015. We are unable to accept electronic orders    *
*   for printed documents at this time.                                       *
*******************************************************************************


Cover
========================================================================== COVER


Report to Congressional Requesters

December 1994

NATIONAL TEST FACILITY - CIVILIAN
AGENCY USE OF SUPERCOMPUTERS NOT
FEASIBLE

GAO/AIMD-95-28

NTF Supercomputers


Abbreviations
========================================================================= ABBREV

  BMDO - Ballistic Missile Defense Organization
  DOD - Department of Defense
  DOE - Department of Energy
  FEMA - Federal Emergency Management Administration
  HHS - Department of Health and Human Services
  NOAA - National Oceanic and Atmospheric Administration
  NTF - National Test Facility
  SDIO - Strategic Defense Initiative Organization

Letter
========================================================================= LETTER


B-259277

December 9, 1994

The Honorable Ronald V.  Dellums
Chairman, Committee on Armed Services
House of Representatives

The Honorable John P.  Murtha
Chairman, Subcommittee on Defense
Committee on Appropriations
House of Representatives

The Honorable Sam Nunn
Chairman, Committee on Armed Services
United States Senate

The Honorable Daniel K.  Inouye
Chairman, Subcommittee on Defense
Committee on Appropriations
United States Senate

As directed by the House Committee on Armed Services in its report accompanying
the National Defense Authorization Act for fiscal year 1995, we reviewed
possible civilian agency use of the supercomputing capabilities at the
Department of Defense's (DOD) National Test Facility (NTF).  We were
specifically directed to include, but not limit our study to, using these
resources to support (1) the Department of Energy in monitoring oil pipeline
leakage, (2) the Department of Education in developing on-line educational
software, (3) the Department of Health and Human Services (HHS) in developing
an on-line medical imagery repository, (4) the Federal Emergency Management
Administration (FEMA) in developing innovative emergency reaction simulations,
and (5) the National Oceanic and Atmospheric Administration (NOAA) in
climatological modeling. 


   RESULTS IN BRIEF
---------------------------------------------------------------------- Letter :1

Based on our interviews with the civilian agencies cited in the House report,
none would be able to make effective use of NTF's excess supercomputing
capabilities.  These agencies stated they could not use the resources primarily
because (1) NTF's supercomputers are older machines whose performance and costs
cannot match those of more advanced computers available from other sources and
(2) some agencies have not yet developed applications requiring supercomputer
capabilities or do not have funding to support such activities.  In addition,
future support for the hardware and software at NTF is uncertain, making any
investment by an outside user risky. 


   BACKGROUND
---------------------------------------------------------------------- Letter :2

NTF was designed and constructed in the 1980s to support the Strategic Defense
Initiative Organization (SDIO) as the centerpiece for its National Test Bed, a
network of research facilities linked together to test and simulate software
and, to a lesser extent, hardware that might be used in defending the United
States against nuclear ballistic missile attack.  SDIO was later reorganized as
the Ballistic Missile Defense Organization (BMDO). 

Opened in 1990, NTF is a $123 million government-owned, contractor-operated
facility that features a sophisticated, reconfigurable design and extensive
network connectivity that give it the potential for conducting large-scale
computer operations.  Located near Colorado Springs, Colorado, NTF is also a
uniquely secure facility that meets DOD's top security requirements. 

Although NTF was originally built to house 21 Cray-2 supercomputers to support
SDIO activities, only 2 Cray-2s were acquired and installed.  The larger
supercomputer features four central processors and 512 million words of memory. 
It is currently running at capacity at the secure classified level, almost
totally in support of BMDO needs.  A smaller system, featuring two central
processors and 128 million words of memory, was turned off last year to save
operational and maintenance expenses.  When this machine is operating, its
maintenance costs up to $70,000 monthly. 

Changes in the world geopolitical situation and the diminished threat of a
deliberate Soviet nuclear attack have led to changes in NTF's mission and
funding level.  The volume of work associated with supporting simulation and
testing of missile defense hardware and software has diminished.  BMDO funding,
which accounted for 89 percent of NTF's budget in fiscal year 1994, has fallen
from $140 million in fiscal year 1993 to $76.5 million in fiscal year 1994. 
Fiscal year 1995 funding remains uncertain.  Based on the President's budget,
NTF anticipates receiving $52 million in BMDO funding this year. 

Faced with such reductions, NTF officials have begun looking for external users
to help cover operational costs.  Varied services have been offered, ranging
from simply leasing secure floor space at NTF to contracting for complete
software development and implementation projects.  NTF officials have also
proposed restarting the smaller Cray-2 system and selling processing time on it
to outside users.  Policies and procedures for this service have been
developed, but no customers have yet been found. 


   SCOPE AND METHODOLOGY
---------------------------------------------------------------------- Letter :3

To identify possible civilian agency use of NTF's supercomputing capabilities,
we spoke with responsible program officials at the Ballistic Missile Defense
Organization and visited NTF.  At NTF we assessed the supercomputer
capabilities of the facility and determined what excess capabilities are
available to outside users and the costs for using these capabilities.  We then
interviewed scientific researchers and officials at government laboratories
about these capabilities.  Finally, we spoke with officials from the relevant
programs at the specific civilian agencies cited in the act to ascertain their
interest in using NTF's supercomputer resources.  The results of our
discussions with these agency officials are summarized in appendix I. 

We conducted our review between August 1994 and October 1994, in accordance
with generally accepted government auditing standards.  As requested, we did
not obtain DOD's comments on this letter.  However, we provided a draft of this
letter to NTF's Assistant Deputy for Engineering, who generally agreed with the
facts as presented.  We have incorporated the Assistant Deputy's views as
appropriate. 


   THE CIVILIAN COMMUNITY CANNOT USE
   EXCESS NTF CAPABILITIES
---------------------------------------------------------------------- Letter :4

Officials from civilian agencies and laboratories we spoke with cited several
reasons they would be unable to use NTF's excess supercomputing resources. 
First, NTF's supercomputers are older machines whose performance and costs
cannot match those of more advanced computers available from other sources. 
Second, some agencies have not yet developed applications requiring
supercomputer capabilities or do not have funding to support such activities. 
Third, because future support for NTF hardware and software is uncertain, any
investment by an outside user could be risky. 


      NTF'S OLDER SUPERCOMPUTERS ARE NO
      LONGER COMPETITIVE
-------------------------------------------------------------------- Letter :4.1

Acquired in the late 1980s, the Cray-2 systems installed at NTF no longer
represent state-of-the-art supercomputer technology.  For example, the latest
Cray supercomputer, the Y-MP C90 system, equipped with
16 processors and 256 million words of memory, can perform at speeds 5 times
faster than the Cray-2.  Moreover, the latest generation of workstations, such
as the IBM RS/6000 590 or the Silicon Graphics TFP, are far more economical and
can almost match the sustained performance of one of the Cray-2's processors. 

Because of the Cray-2's processing speed, the $160 hourly rate NTF proposed for
using the system is not competitive with rates charged by other centers for
newer, faster supercomputers.  In fact, when the rates are expressed in
standard terms (that is, price per million instructions per second), the
proposed NTF Cray-2 rate turns out to be twice the price several of the
National Science Foundation sponsored supercomputing centers--such as the San
Diego Supercomputing Center or the National Center for Atmospheric
Research--charge for time on their newer and faster Crays. 


      SUPERCOMPUTER FUNDING AND
      APPLICABILITY ARE UNCLEAR FOR
      SOME AGENCIES
-------------------------------------------------------------------- Letter :4.2

Not all the agencies we contacted have the ability to use NTF supercomputer
resources, regardless of the desirability of those resources.  Agencies such as
the Department of Education and FEMA do not have budgets for developing
applications to take advantage of the power of a supercomputer.  Of all the
agencies we visited, we identified only one project, at HHS, that could
potentially need supercomputer time.  However, this project, the National
Library of Medicine's Visible Human Project, will not need access to a
supercomputer before 1996. 


      FUTURE SUPPORT AND MAINTENANCE
      FOR NTF SUPERCOMPUTERS IS
      UNCERTAIN
-------------------------------------------------------------------- Letter :4.3

Cray Research, Inc., stopped manufacturing the Cray-2 series supercomputer in
1989.  Although the company currently supports and maintains existing Cray-2
systems, there is no assurance that it will continue to do so.  Cray has
already informed its customers that after September 1996 no new software
upgrades will be provided.  In addition, hardware maintenance is likely to be
difficult or more expensive to obtain after that time. 

Future support for NTF's supercomputer resources is further jeopardized by the
uncertainty of funding to replace the Cray-2s and to retain the present cadre
of talented support staff.  The uncertainty surrounding future support and
maintenance for NTF supercomputers increases the risks for new users in buying
supercomputer time from NTF because they might not realize a full return on the
initial investment. 


-------------------------------------------------------------------- Letter :4.4

We are sending copies of this letter to the Ranking Minority Members of the
House and Senate Committees on Appropriations and the House and Senate
Committees on Armed Services, the Secretary of Defense, the Secretary of the
Air Force, the Director of BMDO, and other interested parties.  Copies will
also be made available to others upon request. 

If you or your staffs have questions or wish to discuss the issues in this
report further, please contact me at (202) 512-6240 or John de Ferrari,
Assistant Director, at (202) 512-6335.  Major contributors to this report are
listed in appendix II.




Jack L.  Brock, Jr.
Director, Information Resource
 Management/National Security and
 International Affairs


POTENTIAL CIVILIAN AGENCY USE OF EXCESS
NTF SUPERCOMPUTING RESOURCES
===================================================================== Appendix I

SUPPORT TO THE DEPARTMENT OF ENERGY IN
MONITORING OIL PIPELINE LEAKAGE

The Department of Energy (DOE) does not use supercomputers to monitor oil
pipeline leakage.  Only one program in which DOE participates within the Oil
and Gas Exploration and Production Division uses supercomputers--the Natural
Gas and Technology Partnership.  However, this program is not interested in
NTF's supercomputing resources because program officials do not consider the
Cray-2 powerful enough for their global climate monitoring simulations, which
they currently run on advanced parallel processing machines. 

SUPPORT TO THE DEPARTMENT OF EDUCATION
IN DEVELOPING ON-LINE EDUCATIONAL
SOFTWARE

The Department of Education has no funds to develop educational software to run
on supercomputers.  Moreover, the Department is just beginning to research how
supercomputers could be used as educational tools and has not yet determined
what directions are worthwhile pursuing.  Since the National Science Foundation
funds activities to develop educational applications for supercomputers, the
user community looks to them, rather than the Department of Education, for
support. 

SUPPORT TO THE DEPARTMENT OF HEALTH AND
HUMAN SERVICES IN DEVELOPING AN ON-LINE
MEDICAL IMAGERY REPOSITORY

The National Library of Medicine's Visible Human Project is the only HHS
project developing a large repository of medical imagery.  However, project
staff are just now acquiring their first data, which are being stored on a
conventional Unix workstation.  Supercomputer applications for this project
will not be needed until sometime in 1996 when the repository is scheduled to
come on-line and HHS is to begin developing algorithms for researchers to view
and manipulate the data.  At that time, HHS officials said they will first
consider using supercomputing resources already available at the National
Institutes of Health.  If these prove unavailable, they might be interested in
using NTF's resources.  As noted above, however, hardware and software support
to the Cray-2 at NTF will be uncertain after 1996. 

SUPPORT TO FEMA IN DEVELOPING
INNOVATIVE EMERGENCY REACTION
SIMULATION

FEMA has neither the funds nor the expertise to develop emergency reaction
simulation.  They currently contract with the Defense Nuclear Agency to develop
disaster models, which FEMA then runs in-house on high-end workstations.  The
models run effectively on these workstations and do not require a
supercomputer. 

SUPPORT TO NOAA IN CLIMATOLOGICAL
MODELING

Three NOAA laboratories--the Geophysical Fluid Dynamics Laboratory, the
National Meteorological Center, and the Forecasting Systems Laboratory--perform
climatological modeling on supercomputers.  However, none of these three is
interested in using NTF's supercomputers.  The Geophysical Fluid Dynamics
Laboratory did not find NTF's proposed price for leasing time on its Cray-2
attractive since the laboratory already has access to a newer, more
cost-effective supercomputer--the Cray Y-MP.  The National Meteorological Fluid
Dynamics Laboratory also found the hourly rate that NTF proposed to be limiting
when the costs of software conversion and communications were included.  The
Forecasting Systems Laboratory already has access to supercomputers and does
not have additional funding to lease time elsewhere.  Furthermore, laboratory
officials are not interested in the Cray-2 because it is an old technology. 
Instead, they said they would be interested in obtaining access to newer
generations of Cray machines, such as the Y-MP C90 or the massively parallel
T3D. 


MAJOR CONTRIBUTORS TO THIS REPORT
==================================================================== Appendix II

ACCOUNTING AND INFORMATION MANAGEMENT
DIVISION, WASHINGTON, D.C. 

John A.  de Ferrari, Assistant Director
Elizabeth L.  Johnston, Evaluator-in-Charge

DENVER REGIONAL OFFICE

Jamelyn Smith, Senior Information Systems Analyst

*** End of document. ***