Defense Computers: Air Force Needs to Strengthen Year 2000 Oversight
(Letter Report, 01/16/98, GAO/AIMD-98-35).

Pursuant to a congressional request, GAO reviewed the Air Force's
program for solving its year 2000 computer systems problem, focusing on
the: (1) status of the Air Force's efforts to oversee its Year 2000
program; and (2) appropriateness of the Air Force's strategy and actions
for ensuring that the problem will be successfully addressed.

GAO noted that: (1) as with other military services, the Air Force is
taking a decentralized approach to year 2000 correction--that is, it is
relying heavily on its components to identify and correct year 2000
problems affecting their own systems; (2) however, in providing
oversight for this effort, the Air Force must ensure that all of its
systems have been accounted for and that component actions are
successful; (3) it must also be well-positioned to make the resource
tradeoff decisions that are inevitable in any year 2000 effort and to
address conflicts between component approaches toward identifying and
correcting interfaces; (4) further, it must be able to provide
additional resources, such as testing facilities, that may be necessary
to correct and validate systems; (5) the Air Force has taken a number of
positive actions toward fulfilling its year 2000 oversight
responsibilities; (6) for example, it is taking inventory of its systems
and prioritizing them for conversion or replacement, and it has issued
extensive guidance on dealing with the year 2000 problem; (7) it has
also established a year 2000 working group comprised of focal points
from the components which aims to eliminate duplicative efforts, share
resources, and track component progress; (8) at the same time, the Air
Force has not yet adequately addressed several critical issues that
would ensure that it is well-positioned to deal with the later, and more
difficult, phases of year 2000 correction; (9) GAO's review revealed
that some components are failing to plan for the testing phase of their
year 2000 effort and develop contingency plans; (10) GAO also found that
some components are taking conflicting approaches toward determining the
actual impact or the program status of their system interfaces; (11) if
components and the Air Force do not promptly address and take consistent
action on these issues, they may well negate any success they may have
in making systems within their control year 2000 compliant; and (12)
while the Air Force has enlisted the services of the Air Force Audit
Agency to help address some of these concerns, this work needs to be
backed by comprehensive and continued Air Force oversight in order to
ensure that it can address unforeseen problems and delays in the next,
more difficult phases.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  AIMD-98-35
     TITLE:  Defense Computers: Air Force Needs to Strengthen Year 2000 
             Oversight
      DATE:  01/16/98
   SUBJECT:  Systems conversions
             Computer software verification and validation
             Strategic information systems planning
             Embedded computer systems
             Cost analysis
             Information resources management
             Systems compatibility
IDENTIFIER:  DOD Year 2000 Management Plan
             Air Force Year 2000 Program
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Acting Secretary of the
Air Force

January 1998

DEFENSE COMPUTERS - AIR FORCE
NEEDS TO STRENGTHEN YEAR 2000
OVERSIGHT

GAO/AIMD-98-35

Air Force Year 2000 Challenges

(511637)


Abbreviations
=============================================================== ABBREV

  AFCA - Air Force Communication Agency
  AFCIC - Air Force Communications and Information Center
  CIO - Chief Information Officer
  DOD - Department of Defense
  O&M - operational & maintenance
  OASD/C3I - Office of the Assistant Secretary of Defense for
     Command, Control, Communications and Intelligence

Letter
=============================================================== LETTER


B-278232

January 16, 1998

The Honorable F.  Whitten Peters
The Acting Secretary of the Air Force

Dear Mr.  Secretary: 

This report presents the results of our review to date of the Air
Force's program for solving its Year 2000 computer systems problem. 
If this problem is not corrected in time, Air Force computer systems
could malfunction or produce incorrect information.  The impact of
these failures could be widespread, costly, and debilitating to
important Air Force warfighting and military support missions. 

We performed this work as part of our review of the Department of
Defense's (DOD) Year 2000 computer systems efforts for the Chairman,
Senate Committee on Governmental Affairs; the Chairman and Ranking
Minority Member of the Subcommittee on Government Management,
Information and Technology, House Committee on Government Reform and
Oversight; and the Honorable Thomas M.  Davis, III, House of
Representatives.  Our objectives were to evaluate (1) the status of
the Air Force's efforts to oversee its Year 2000 program and (2) the
appropriateness of the Air Force's strategy and actions for ensuring
that the problem will be successfully addressed.  This letter
summarizes the concerns we raised and provides recommendations for
addressing these issues. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

As with the other military services, the Air Force is taking a
decentralized approach to Year 2000 correction--that is, it is
relying heavily on its components to identify and correct Year 2000
problems affecting their own systems.  However, in providing
oversight for this effort, the Air Force must ensure that all of its
systems have been accounted for and that component actions are
successful.  It must also be well-positioned to make the resource
trade-off decisions that are inevitable in any Year 2000 effort and
to address conflicts between component approaches toward identifying
and correcting interfaces.  Further, it must be able to provide
additional resources, such as testing facilities, that may be
necessary to correct and validate systems. 

The Air Force has taken a number of positive actions toward
fulfilling its Year 2000 oversight responsibilities.  For example, it
is taking inventory of its systems and prioritizing them for
conversion or replacement, and it has issued extensive guidance on
dealing with the Year 2000 problem.  It has also established a Year
2000 working group comprised of focal points from the components
which aims to eliminate duplicative efforts, share resources, and
track component progress. 

At the same time, the Air Force has not yet adequately addressed
several critical issues that would ensure that the Service is
well-positioned to deal with the later, and more difficult, phases of
Year 2000 correction.  For example, it has not refined its cost
estimate, using actual assessment data, so that it can make informed
resource trade-off decisions.  In addition, it has not ensured that
interfaces are properly accounted for or ensured that components are
developing contingency plans.  Finally, it has not ensured that
components have anticipated the need for testing resources or sought
to acquire such resources itself. 

Our review revealed that some components are failing to plan for the
testing phase of their Year 2000 effort and develop contingency
plans.  We also found that some components are taking conflicting
approaches toward determining the actual impact or program status of
their system interfaces.  If components and the Air Force do not
promptly address and take consistent action on these issues, they may
well negate any success they may have in making systems within their
control Year 2000 compliant.  While the Air Force has enlisted the
services of the Air Force Audit Agency to help address some of these
concerns, this work needs to be backed by comprehensive and continued
Air Force oversight in order to ensure that the Service can address
unforeseen problems and delays in the next, more difficult phases. 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :2

In conducting our review, we assessed the Air Force's Year 2000
efforts against our own Year 2000 Assessment Guide.\1 This guide
addresses common issues affecting most federal agencies and presents
a structured approach and a checklist to aid in planning, managing,
and evaluating Year 2000 programs.  The guidance, which is consistent
with DOD's Year 2000 Management Plan\2 and the Air Force's own Year
2000 management approach, describes five phases--supported by program
and project management activities--with each phase representing a
major Year 2000 program activity or segment.  The phases and a
description of each follows. 

  -- Awareness - Define the Year 2000 problem and gain
     executive-level support and sponsorship.  Establish a Year 2000
     program team and develop an overall strategy.  Ensure that
     everyone in the organization is fully aware of the issue. 

  -- Assessment - Assess the Year 2000 impact on the enterprise. 
     Identify core business areas and processes, inventory and
     analyze systems supporting the core business areas, and
     prioritize their conversion or replacement.  Develop contingency
     plans to handle data exchange issues, lack of data, and bad
     data.  Identify and secure the necessary resources. 

  -- Renovation - Convert, replace, or eliminate selected platforms,
     applications, databases, and utilities.  Modify interfaces. 

  -- Validation - Test, verify, and validate converted or replaced
     platforms, applications, databases, and utilities.  Test the
     performance, functionality, and integration of converted or
     replaced platforms, applications, databases, utilities, and
     interfaces in an operational environment. 

  -- Implementation - Implement converted or replaced platforms,
     applications, databases, utilities, and interfaces.  Implement
     data exchange contingency plans, if necessary. 

During our review, we concentrated primarily on the Air Force's
efforts to oversee its Year 2000 program during the awareness and
assessment phases.  We focused our review on Year 2000 work being
carried out by (1) DOD's Office of the Assistant Secretary of Defense
for Command, Control, Communications, and Intelligence
(OASD/C3I)--which is responsible for promulgating DOD guidance on
Year 2000 and providing assistance to Defense components, (2) Air
Force headquarters, including the Air Force Communications and
Information Center (AFCIC)--which is responsible for day-to-day
management and supervision and for issuing Air Force Year 2000 policy
and guidance, (3) Air Force Communication Agency, the designated Air
Force Year 2000 program office, and (4) selected program offices
managed by the Air Force Materiel Command's Aeronautical Systems
Center at Wright-Patterson Air Force Base, Ohio. 

To assess OASD/C3I efforts in providing Year 2000 support to the Air
Force, we met with the Acting Assistant Secretary of Defense for
Command, Control, Communications and Intelligence, the Principal
Director for Information Management, the Director for Information
Technology, and other senior staff responsible for Year 2000 issues. 
We reviewed the office's Year 2000 guidance and other documentation
on Year 2000 funding, reporting, and date format requirements. 

To assess Air Force headquarters efforts to manage and oversee the
Year 2000 computer problem, we (1) met with the Air Force
Communications and Information Center officials and Year 2000 focal
points, (2) obtained and analyzed documents issued by these offices
that describe organizational structure and responsibilities for
carrying out the Air Force Year 2000 program, and (3) reviewed the
Air Force's Year 2000 Guidance Package to assess the level of
guidance, roles, and responsibilities, and target milestone dates for
the Year 2000 effort. 

Further, we obtained and analyzed the Air Force Year 2000 inventory
data to determine (1) the number of systems owned and operated by Air
Force organizations and (2) the status of Air Force systems in their
Year 2000 efforts, proposed strategy, and the number of systems
reporting to be compliant.  We reviewed pertinent Year 2000 program
documentation such as Defense and Air Force guidance and management
directives, working group minutes, status reports, and cost and
schedule data. 

We performed our work primarily at the Air Force Materiel Command,
Wright-Patterson Air Force Base, Ohio; Headquarters Air Force at the
Pentagon, Washington, D.C.; and the Office of Assistant Secretary of
Defense for Command, Control, Communications and Intelligence at
Arlington, Virginia.  We conducted our work from July 1996 through
August 1997 in accordance with generally accepted government auditing
standards.  We received written comments on a draft of this report
from the Chief Information Officer for the Department of the Air
Force.  His comments are discussed in the "Agency Comments and Our
Evaluation" section and are reprinted in appendix II. 


--------------------
\1 Year 2000 Computing Crisis:  An Assessment Guide
(GAO/AIMD-10.1.14, September 1997). 

\2 Version 1.0, April 1997. 


   BACKGROUND
------------------------------------------------------------ Letter :3

Most of the Air Force's automated information systems and embedded
weapon systems are vulnerable to the Year 2000 problem, which is
rooted in the way dates are recorded and computed in automated
information systems.  For the past several decades, systems have
typically used two digits to represent the year, such as "97"
representing 1997, in order to conserve on electronic data storage
and reduce operating costs.  With this two-digit format, however, the
Year 2000 is indistinguishable from 1900, or 2001 from 1901, etc.  As
a result of this ambiguity, system or application programs that use
dates to perform calculations, comparisons, or sorting may generate
incorrect results when working with years after 1999. 

Should Air Force computer systems fail on the morning of the Year
2000, Air Force operations at all levels could be affected by the
incorrect processing of data, as well as corrupted databases, or even
massive system failures.  In turn, this could result in such problems
as delays in supply shipments, faulty inventory forecasts, unreliable
budget estimates, and erroneous personnel-related information. 
Moreover, the problem could adversely affect critical warfighting
functions such as combat, communications, command and control,
intelligence, surveillance, reconnaissance, and air traffic control. 

Like the other military services, the Air Force has adopted DOD's
Year 2000 management strategy, which calls for centralized oversight
with decentralized execution of Year 2000 correction.  In February
1995, the Air Force designated the Air Force Communication Agency
(AFCA)\3 as the focal point for Year 2000 efforts, with
responsibility for (1) coordinating Year 2000 efforts being carried
out by its 9 major commands, 36 field operating agencies, and 3
direct reporting units, (2) ensuring that components completed Year
2000-related tasks on time, (3) developing Year 2000 guidance, (4)
collecting and reporting progress and inventory-related data, and (5)
chairing the Air Force Year 2000 working group which is comprised of
representatives from components. 

In April 1997, the Air Force established a Year 2000 program office
at AFCA.  The program office is currently staffed with 24 full-time
personnel and it reports to the Air Force Communications and
Information Center (AFCIC).  AFCIC, which was established in April
1997 due to a Headquarters Air Force reorganization, was tasked with
responsibility for implementing Year 2000 policy and programmatic
changes across the Service.\4 AFCIC also reports to the Office of the
Chief Information Officer and it has assigned three full-time staff
members to oversee the Air Force's Year 2000 Program.  Appendix I
illustrates the Air Force's Year 2000 organizational structure and
describes the complexity involved in carrying out Year 2000 efforts
at the command level. 

Early in its Year 2000 effort, the Air Force introduced a five-phased
management approach for addressing the Year 2000 problem, which was
later adopted by DOD and the Federal Government CIO Council's Year
2000 Subcommittee.  According to Air Force officials, if properly
implemented, this phased approach will enable the Air Force to
achieve its goal of having every mission-critical system compliant by
December 1998.  The five phases and their supporting program and
project management activities are consistent with those identified in
our Year 2000 Assessment Guide, which draws heavily on the best
practices work of the CIO Council's Year 2000 Subcommittee. 

In addition to following the five-phase approach, our guidance
addresses common issues affecting most federal agencies and provides
a checklist to aid them in planning, managing, and evaluating their
year 2000 programs.  Also, because the Year 2000 is a massive and
complex management challenge, our guidance recommends that agencies
plan and manage a Year 2000 program as a single large information
system development effort and promulgate and enforce good management
practices on the program and project levels. 

To comply with DOD's current Year 2000 funding mandate, the Air Force
does not plan to provide system/program managers with any additional
funds to manage and fix the Year 2000 problem.  Rather,
system/program managers have been directed to reprioritize or
reprogram previously budgeted funds (primarily operational &
maintenance (O&M) funds) to fix Year 2000 problems. 


--------------------
\3 Formerly known as the Air Force Command and Control,
Communications, and Computer Agency.  AFCA is located at Scott Air
Force Base, Illinois. 

\4 Previously, the Air Force/SC (Command, Control, Communications,
and Computers) had this responsibility. 


   CURRENT STATUS OF AIR FORCE
   YEAR 2000 EFFORTS
------------------------------------------------------------ Letter :4

The Air Force estimates there are 2,944 automated information systems
and weapons embedded systems in its inventory and that the majority
of these systems will have to be either renovated, replaced, or
retired before January 1, 2000.  Of the 2,944 systems, 550 (about 19
percent) are considered to be mission-critical systems, that is, they
directly support wartime operations.  As of September 4, 1997, the
Air Force reported that all of its 2,944 systems completed the
awareness phase, 33 percent were in the assessment phase, 32 percent
in renovation, 17 percent in validation, 12 percent were in
implementation, and 6 percent will be decommissioned by December
1999.  As of September 1997, the Air Force estimated that it will
cost about $405 million to successfully complete its Year 2000
program.  Table 1 details the status of Air Force systems according
to their mission impact. 



                                     Table 1
                     
                     Status of Air Force Inventoried Systems
                                  by Criticality

         Group I--    Group II--   Group III--    Group IV--
Stat       Mission       Mission       Mission   Non-Mission       Other--  Tota
us        Critical     Essential      Impaired     Essential  Unassessable     l
----  ------------  ------------  ------------  ------------  ------------  ----
Awar             0             0             0             0             0     0
 ene
 ss
Asse           250           234           233           241             1   959
 ssm
 ent
Reno           156           163           432           201             3   955
 vat
 ion
Vali            81           145           173            87             6   492
 dat
 ion
Impl            59            71           125            98             0   353
 eme
 nta
 tio
 n
Deco             4            31            46           102             2   185
 mmi
 ssi
 one
 d
================================================================================
Tota           550           644         1,009           729            12  2,94
 l                                                                             4
--------------------------------------------------------------------------------
Note:  According to Air Force Manual 10-410, criticality is defined
as follows. 

(1) Mission Critical:  The loss of critical functions that would
cause immediate stoppage of direct support of wartime operations. 

(2) Mission Essential:  The loss or reduced capability due to loss of
equipment or parts.  If not corrected, degradation eventually causes
loss of mission capability. 

(3) Mission Impaired:  The loss of functions that would not have an
immediate effect on direct mission support of wartime operations. 

(4) Non-mission Essential:  The loss of functions that would not
effect mission operations. 

(5) Unassessable:  The effect on the mission cannot be judged and
falls into other groups when additional information becomes
available. 

Source:  AFCIC data as of September 4, 1997.  We did not
independently verify this information. 

The Air Force has taken a number of positive steps to ensure that its
personnel are fully aware of the impact should Air Force systems not
be compliant at the turn of the century.  For example, in November
1995, the Air Force established a Year 2000 working group comprised
of focal points from each major command, field operating agency, and
direct reporting unit.  This group has focused on such matters as
sharing lessons learned, eliminating duplicative efforts, sharing
resources, and tracking component progress.  In the same month, the
Air Force released an Air Force-wide impact assessment survey to all
major commands, field operating agencies, and direct reporting units
for the purpose of obtaining a rough order-of-magnitude of the Year
2000 problem throughout the Air Force.  The results of this survey
indicated that the Air Force Year 2000 problem would be significant
and that it required immediate and sustained management attention. 

The Air Force has also addressed a number of steps associated with
the assessment phase of Year 2000 correction, including the
following. 

  -- Developing a comprehensive Air Force-wide system inventory,
     which will include information on information systems, weapons
     systems, and infrastructure-related devices that could be
     affected by the Year 2000 problem. 

  -- Prioritizing systems for conversion or replacement according to
     their mission impact. 

  -- Tasking the Air Force Software Technology Support Center at Hill
     Air Force Base, Utah, to evaluate in-house and vendor tools and
     services that could be used to identify and fix Year 2000
     problems. 

  -- Creating a dedicated Year 2000 database, which contains system
     inventory-related information as well as information on
     component progress. 

  -- Issuing a Year 2000 Guidance Package for senior managers and
     Year 2000 points-of-contact, which (1) explains how to prepare
     individual project management plans and develop Year 2000
     strategies, (2) includes milestones and exit criteria for Year
     2000 tasks, (3) provides a flowchart illustrating the five-phase
     resolution process, and (4) provides cost estimating formulas. 
     This package is continually updated to reflect new managerial,
     technical, legal and other Year 2000-related developments. 

  -- Developing a checklist to assist system managers in ensuring
     that their systems are compliant for the Year 2000, which covers
     (1) the identification of systems and interfaces, (2) assessment
     of date usage by the systems, and (3) compliance testing, among
     other subjects. 

  -- Directing each major command and field operating agency to
     appoint Year 2000 certifiers to ensure that all systems
     belonging to the components have completed the necessary steps
     to become Year 2000 compliant. 


      THE ASSESSMENT PHASE IS
      RUNNING BEHIND SCHEDULE
---------------------------------------------------------- Letter :4.1

The Air Force originally anticipated that it would complete the
assessment phase of its Year 2000 effort in May 1997.  It
acknowledged that approximately 66 percent of its systems did not
meet this deadline and it subsequently revised the deadline to
October 1997.  However, as of September 4, 1997, about 33 percent of
its systems had still not been assessed.  With less than 26 months
remaining before the Year 2000 deadline, this will add pressure on
the Air Force to renovate, validate, and implement systems as quickly
as possible.  According to an industry expert, June 1997 is apt to be
the latest point in time to start fixing systems and to have a
reasonable probability of finishing before year 2000.\5 The Air
Force's Year 2000 guidance, as well as GAO and OMB's Year 2000
guidance, call for a similar completion date.  In addition, according
to the Gartner Group--an independent contractor hired by Defense to
provide Year 2000 technical support primarily in the areas of
scheduling and cost estimating--no more than 26 percent of an
organization's total Year 2000 effort should be spent in the
awareness and assessment phases.  Our analysis shows that the Air
Force has used nearly 46 percent of its available time to complete
these two phases.  While Air Force officials acknowledge that the
assessment phase is taking longer than expected, they do not believe
it will significantly affect their Year 2000 program because system
and program managers have already begun to fix systems identified
with Year 2000 problems. 

One reason for the delay in completing the assessment phase is that
it has taken longer than anticipated to develop a complete systems
inventory.  Before its Year 2000 effort, the Air Force did not have a
comprehensive servicewide system inventory.  As such, it could not
readily determine the magnitude (much less the cost to fix) of the
Year 2000 problem servicewide when it began the assessment phase. 
While its inventory now contains 2,944 systems, the Air Force is
still expanding it to include information on infrastructure-related
devices, such as elevators, traffic control and security devices,
telephone switching systems, and medical equipment.  These devices
rely on either microprocessors or microcontroller chips that may be
vulnerable to Year 2000 problems.  In addition, the Air Force is
contending with slow and incomplete reporting by system and program
managers.  As a result, it has revised reporting requirements to
facilitate better reporting on the part of its components. 

Furthermore, the Air Force must still resolve discrepancies between
its inventory and recent findings by the Air Force Audit Agency.  In
June 1997, the Audit Agency identified over 6,000 information systems
that were not included in the Air Force inventory (which contained
2,543 systems at the time this audit was conducted).  These
additional systems included 1,600 mission-critical systems.  The Air
Force is currently reconciling its database to the audit findings. 

The Air Force has recently enlisted the Air Force Audit Agency to
help evaluate component progress in completing the assessment phase. 
The agency will determine whether selected components have (1)
completed timely assessments, (2) addressed all system interfaces,
(3) accomplished mandatory system certifications, (4) prioritized and
scheduled required renovations, and (5) developed contingency plans. 


--------------------
\5 Caper Jones, The Global Economic Impact of the Year 2000 Software
Problem, Version 4, September 23, 1996, Software Productivity
Research, Inc.  Caper Jones is widely recognized as a leading Year
2000 expert. 


   KEY ISSUES NEED PROMPT
   ATTENTION FOR THE AIR FORCE TO
   SOLVE ITS YEAR 2000 PROBLEM
------------------------------------------------------------ Letter :5

Even though the Air Force is entering the next phases of its Year
2000 correction effort, it has yet to complete several critical
assessment steps, which are designed to ensure that it is
well-positioned to deal with the later, and more difficult, phases of
Year 2000 correction.  These include (1) recalculating its $405
million cost estimate, based on actual assessment data, so that it
can make informed choices about information technology priorities,
(2) ensuring that interfaces are properly accounted for, (3) ensuring
that components are developing contingency plans, and (4) ensuring
that components are adequately prepared for the testing phase.  The
Air Force Audit Agency audit should help the Air Force complete these
steps; however, this work will be carried out only at selected sites
and it will not provide the comprehensive and continued oversight
that is needed to ensure that the Air Force can handle unforeseen
problems and delays. 


      FULL COST OF YEAR 2000
      PROBLEM NEEDS TO BE
      DETERMINED
---------------------------------------------------------- Letter :5.1

As DOD's Year 2000 Management Plan and our Year 2000 Assessment Guide
state, the primary purpose of the assessment phase is to gather and
analyze the information in order to determine the size and scope of
the problem.  Among other things, this enables an agency to estimate
the cost of its Year 2000 effort in terms of dollars and work years,
and, in turn, to make informed choices about information technology
priorities and whether other system development efforts should be
deferred or canceled so that resources can be freed up to solve the
Year 2000 problem.  The Air Force, however, has not yet fully defined
the scope of its Year 2000 problem or refined cost estimates, using
actual assessment data, in order to gauge what resources are needed
for correction.  The need to take immediate action in this regard is
critical, given that some organizations are already discovering that
they do not have sufficient funding to correct their systems. 

Currently, the Air Force expects to spend about $405 million from
fiscal year 1997 through 1999 to fix its Year 2000 problem.  Table 2
breaks down the estimated cost by fiscal year. 



                                Table 2
                
                     Estimated Cost by Fiscal Year

                         (Dollars in millions)

Fiscal Year                                                       Cost
----------------------------------------  ----------------------------
1997                                                            $105.3
1998                                                             283.5
1999                                                              16.2
======================================================================
Total                                                           $405.0
----------------------------------------------------------------------
Source:  Air Force Communications and Information Center (as of
September 1997).  We did not independently verify this information. 

According to AFCIC officials, the cost estimate was calculated using
the Gartner cost formula, which recommends multiplying $1.10 by the
lines of code contained in the agency's automated information systems
and $8.00 by the lines of code for weapon systems.  The Gartner
method is helpful in developing a rough estimate of what it will cost
to resolve the problem early in the Year 2000 effort.  However,
according to a directive\6 from Defense's Chief Information Officer
as well as Year 2000 consultants, agencies should refine their cost
estimates as they progress through the assessment phase and into the
later Year 2000 phases to factor in the actual resources they believe
are needed to renovate and implement their systems.  According to
DOD's Year 2000 Management Plan, these can include: 

  -- The age of the systems being corrected.  Age can have a
     significant impact on the cost of correction since older code
     tends to be less structured and thus harder to understand and
     correct than newer code. 

  -- The Year 2000 strategy that the program is pursuing.  Strategies
     that involve keeping the two-digit code, for example, are much
     less expensive than those that involve changing the two-digit
     code to a four-digit code.\7

  -- The degree of documentation that is available on the system and
     its understandability and the availability of source code. 

  -- The skill and expertise of in-house programmers. 

  -- Projected engineering costs. 

  -- Labor hours required to fix systems. 

  -- Testing requirements. 

The September estimate still used the Gartner formula and did not
take into account other factors that can have a significant impact on
the cost of correction including those identified in DOD's Year 2000
Management Plan.  Air Force officials acknowledged that the $405
million estimate is a rough figure.  They planned to re-estimate
costs at some point after the assessment phase is completed.  Costs
should be continuously reestimated through the assessment and
subsequent Year 2000 phases.  By waiting to refine its cost
estimates, the Air Force will be delaying the availability of
information needed to make informed resource trade-off decisions. 

In fact, trade-off issues and other funding disputes, which call for
the need to develop more accurate cost estimates, have already
surfaced in some Air Force programs.  For example, one aircraft
weapon system program found that correcting the Year 2000 problem in
ground software equipment that is used to program the aircraft's
operational avionics software for navigation and weapons delivery
would cost $42 million more than what was budgeted for routine
maintenance of the aircraft.  In August 1997, the program office
reported that it fixed the problem for about $300,000 using a
temporary workaround.\8 However, according to a program office
official, because the existing equipment consists of old IBM
mainframes and outdated Jovial code\9 it will have to be replaced
eventually--and likely at a higher cost--in order to support future
planned aircraft enhancements such as Joint Direct Attack Munition
and Joint Standoff Weapon. 

In addition, the Air Force estimates that it will cost between $70
million and $90 million to fix telephone switches throughout the
Service.  This estimate is not included in the $405 million total Air
Force Year 2000 cost estimate.  The Air Force is currently in a
dispute with the contractor that supplied the switches over who is
responsible for Year 2000 correction.  At the same time, Air Force
components have not budgeted funds to fix their telephone switches. 
Since then, and according to AFCIC officials, the Air Force has begun
to address this funding issue through its normal corporate funding
process. 


--------------------
\6 Memorandum from the Assistant Secretary of Defense for Command,
Control, Communications and Intelligence, who is also Defense's Chief
Information Officer, dated January 14, 1997, on cost estimating
metrics for Year 2000. 

\7 DOD has identified three strategies--field expansion, procedural
code, and sliding windows--for purposes of renovating noncompliant
systems.  Field expansion increases the size of the date field
generally from a two-digit year to a four-digit year.  Procedural
code is code which derives the correct century based on the two-digit
year (e.g., any year smaller than year 50 is a 2000 date, and any
year 50 or larger is a 1900 date).  Sliding windows are similar to
procedural code in that they derive the correct century based on the
two-digit year, but the numeric constant used to determine the
century changes each year. 

\8 This would involve a 28-year backdate solution whereby the
computer interprets 2000 as 1972, 2001 as 1973, etc. 

\9 A programming language. 


      SYSTEM INTERFACES NEED MORE
      ATTENTION
---------------------------------------------------------- Letter :5.2

It is critically important during the Year 2000 effort that agencies
protect against the potential for introducing and propagating errors
from one organization to another and ensure that interfacing systems
have the ability to exchange data through the transition period. 
According to our Year 2000 Assessment Guide, to address the issue of
interfaces, agencies should (1) identify their internal and external
interfaces, (2) determine the need for data bridges and filters, (3)
notify outside data exchange partners of their interface plans, (4)
test their interface correction strategies, and (5) develop
contingency plans that address the possibility of failing to receive
data from an external source or receiving invalid data.  DOD's Year
2000 Management Plan places responsibility on component heads or
their designated Year 2000 points of contact to document and obtain
system interface agreements in the form of memorandums of agreement
or the equivalent. 

Since October 1996, the Air Force has participated in six high-level
DOD Year 2000 interface workshops, including finance, intelligence,
command and control, communications, logistics, and weapons systems. 
However, to date, the Air Force has not been tracking (1) how its
components are going about identifying their interfaces, (2) how they
plan to correct interfaces, and (3) whether they are instituting
memorandums of agreement in order to communicate their interface
plans to their data exchange partners.  It is important for the Air
Force to immediately begin tracking these issues since individual
components are embarking on varying--and possibly
conflicting--approaches to addressing interfaces.  Moreover, others
have not yet addressed the interface issue. 

For example, none of the five weapon system program offices we
surveyed had fully determined the actual impact or program status of
their system interfaces.  One program office told us that it did not
plan to do so until the Air Force prescribed a uniform approach to
interfaces.  In addition, we found other weapon system program
approaches to identifying their interfaces to be considerably
different.  For example, the F-22 weapon systems program formally
requested its development contractor, in writing, to assess the
impact of the Year 2000 problem on the aircraft.  This assessment
would include identification of interfaces and an evaluation on
whether they pose a Year 2000 problem.  By contrast, the F-16 program
office planned to informally contact its subcontractors to identify
the status of interfaces and Year 2000 issues for on-board components
of the aircraft that the program office does not directly manage. 
For components that the program office directly manages, it plans to
informally request that its contractor assess Year 2000 problems and
identify the status of interfaces.  However, that assessment will not
be documented as the F-22 program office's assessment will be. 
Clearly, the second approach will provide the Air Force with less
assurance that all interfaces have been accounted for than the first
approach. 

Without centralized oversight over the identification and correction
of interfaces, there is a chance that some systems and interfaces,
for which ownership is unclear, may not be identified and corrected. 
In addition, there is also a higher risk that conflicting interface
solutions will be implemented without the data bridges that are
necessary to ensure that information can still be transferred.\10

For example, one system manager may choose to fix a system by
expanding its date and year, while another may choose to keep the
two-digit format and use procedural code or sliding windows as a
strategy for becoming Year 2000 compliant.  According to current
Defense guidance, either fix is acceptable, but both parties need to
know of the potential conflict so that they can install the data
bridge. 

AFCIC plans to recommend that responsible system/program managers
prepare interface memorandums of agreement, which describe the method
of interface and assign responsibility for accommodating the exchange
of data.  If implemented, these agreements could ensure that
information can be transferred even when components take conflicting
approaches to their interfaces.  At the time of our review, however,
none of the five program offices we visited had prepared such
agreements, and the Air Force was not tracking whether these or
comparable agreements were being instituted. 


--------------------
\10 Bridging involves receiving information in one format, modifying
it, and outputting it in another format, such as receiving the year
in a two-digit format, adding century information through the use of
an algorithm, and writing the output with a four-digit year. 


      AIR FORCE IS NOT ENSURING
      COMPONENTS ARE PLANNING FOR
      TESTING
---------------------------------------------------------- Letter :5.3

Our Year 2000 Assessment Guide calls on agencies to develop
validation strategies and test plan, and to ensure that resources,
such as facilities and tools, are available to perform adequate
testing.  This planning should begin in the assessment phase since
agencies may need over a year to adequately validate and test
converted or replaced systems for Year 2000 compliance and since the
testing and validation process may consume over half of the Year 2000
program resources and budget. 

At the time of our review, however, the Air Force was not ensuring
that components were developing test plans.  It was also not
assessing the need for additional testing resources, even though it
acknowledged that these resources would be in demand.  Instead, AFCIC
officials told us that they are relying heavily on system/program
managers to organize, plan, and manage the necessary resources to
test Year 2000 fixes.  Our review showed that more attention is
needed in this area.  For example, none of the five program offices
we surveyed had completed a master Year 2000 test plan.\11

Due to the complexities and risks involved with testing, components
that are not currently planning their testing strategies run a high
risk of not completing the Year 2000 effort on time.  This is because
components must not only test the year 2000 compliance of individual
applications, but also the complex interactions between scores of
converted or replaced computer platforms, operating systems,
utilities, applications, databases, and interfaces.  Moreover, in
some instances, components may not be able to shut down their
production systems for testing and thus have to operate parallel
systems implemented on a year 2000 test facility.  Components may
also find that they need computer-aided software testing tools and
test scripts\12 to help prepare and manage test data, automate
comparisons of test results, and schedule tests.  AFCIC officials
themselves believe that there is a good chance that adequate test
facilities may not be available to conduct joint interoperability
testing involving systems that interface with one another.  For these
reasons, it is critical that Air Force headquarters ensure that
components are taking time now to assess their testing needs and that
the Air Force is well-positioned to provide components with
additional testing facilities and tools. 

In August 1997, the Air Force working group began to address this
testing issue in part by directing its components to identify and
develop an inventory of existing testing facilities that could
support Year 2000 testing of selective platforms such as Unisys and
IBM.  This effort is ongoing. 


--------------------
\11 The purpose of a test plan is basically to document (1) the
specific test processes to be followed, (2) the extent and timing
(schedule) of tests required for each system, (3) the testing
facilities, personnel, and financial resources needed to accomplish
the required tests, and (4) reporting requirements.  According to Air
Force officials, until testing is fully completed, a system cannot be
certified as Year 2000 compliant. 

\12 A set of detailed instructions for the set-up, execution, and
evaluation of results for a given test case. 


      REQUIRED CONTINGENCY PLANS
      NOT BEING PREPARED
---------------------------------------------------------- Letter :5.4

DOD's Year 2000 Management Plan and our Year 2000 Assessment Guide
call on agencies to develop realistic contingency plans during the
assessment phase for critical systems and activities to ensure the
continuity of their core business processes.  Contingency plans are
important because they identify the manual or other fallback
procedures to be employed should some critical systems miss their
Year 2000 deadline or fail unexpectedly even after they are found to
be compliant.  Contingency plans also establish a series of
checkpoints that allow the agency to identify performance problems
early enough to correct them. 

The Air Force itself has acknowledged that components need to develop
contingency plans and it has directed system/program managers to
prepare, at a minimum, contingency plans for all mission-critical
systems.  It has also incorporated this requirement into its
assessment phase exit criteria. 

However, the Air Force has not been tracking the extent to which
components have prepared plans for mission-critical
functions/systems.  Without greater oversight over the preparation of
such plans, some components may fail to adequately plan for
contingencies without the Air Force's knowledge.  In fact, at the
time of our review, none of the five system program offices we
surveyed had prepared contingency plans.  Officials from these
offices told us that contingency plans are not needed because they
believed that their systems did not require extensive Year 2000 work
and thus their corrections would be made before the Year 2000
deadline expired.  In addition, they did not believe that contingency
planning was cost-effective. 

All Air Force organizations need to be engaged in contingency
planning since there is no guarantee that the corrections they will
make will be completed on time or be free of unforeseen problems.  As
such, according to DOD's Year 2000 Management Plan, components, at a
minimum, need to (1) analyze the impact of a system failure, (2)
identify alternative activities--including manual or contract
procedures--to be employed should critical systems fail to meet their
Year 2000 deadline, and (3) identify procedures and responsibilities
for implementing such alternatives.  Furthermore, given the dangers
associated with not having contingency plans, we believe the Air
Force headquarters' oversight responsibility must involve ensuring
that all components are planning for contingencies for
mission-critical systems. 


   CONCLUSIONS
------------------------------------------------------------ Letter :6

To its credit, the Air Force has recognized that virtually every
computer system it operates is vulnerable to the Year 2000 problem,
it has raised the awareness of the Year 2000 problem among system
owners, and it has begun assessing the Year 2000 impact on Air Force
systems.  However, the Air Force is unnecessarily putting its Year
2000 program at risk of failure because it has not yet refined cost
estimates based on actual assessment data, fully examined resource
trade-offs, and ensured strong and continuous oversight for
interface, testing, and contingency planning issues.  Because these
steps are designed to ensure that organizations are well-positioned
to deal with the more difficult stages of Year 2000 correction,
neglecting any one of them can seriously endanger the Air Force's
ability to meet its Year 2000 deadline.  Given its role in national
security, and its interdependence with other military organizations,
the Air Force cannot afford this risk. 


   RECOMMENDATIONS
------------------------------------------------------------ Letter :7

We recommend that the Secretary of the Air Force immediately require
that the Air Force ensure its cost estimates factor in the actual
resources it believes are needed to renovate and implement systems so
that the Service can make informed resource trade-off decisions and
ensure that this estimate is periodically refined throughout the Year
2000 program. 

We also recommend that the Secretary ensure that an approach is
developed to continuously track how components are going about
identifying interfaces, how they plan to correct interfaces, and
whether they are instituting memorandums of agreement. 

In addition, we recommend that the Secretary ensure that components
are developing test plans and identifying the need for additional
testing resources and design an approach to obtain any needed testing
resources that are identified by Air Force components. 

Finally, we recommend that the Secretary act to ensure that
components have prepared contingency plans for their mission-critical
systems. 


   AGENCY COMMENTS AND OUR
   EVALUATION
------------------------------------------------------------ Letter :8

In written comments on a draft of this report, the Office of the Air
Force Chief Information Officer agreed with all of our
recommendations to improve the Air Force's Year 2000 program.  In
response to our recommendations, the Air Force agreed to update its
cost estimates as it progresses through the remaining Year 2000
phases and include actual resources needed to renovate and implement
systems so that it can make informed resource trade-off decisions. 
The Air Force also agreed to place greater management attention on
identifying system interfaces and improve reporting practices to
ensure that interface corrections are properly accounted for and can
be readily tracked.  In addition, the Air Force agreed to have major
commands and product centers outline and prioritize their test
requirements to ensure that testing resources will be available when
needed. 

The Air Force pointed out that it is working with components to
develop Year 2000 contingency plans as part of the renovation and
validation phases.  In addition, the Air Force plans to open
servicewide crisis response centers around August or September 1999
to deal with critical systems that will not be Year 2000 compliant by
January 1, 2000.  The Air Force is taking steps to ensure that
contingency plans will be prepared on each noncompliant system
identified and be made readily available to the crisis response
centers.  The full text of Air Force's comments is provided in
appendix II. 


---------------------------------------------------------- Letter :8.1

This report contains recommendations to you.  The head of a federal
agency is required by 31 U.S.C.  720 to submit a written statement on
actions taken on these recommendations to the Senate Committee on
Governmental Affairs and the House Committee on Government Reform and
Oversight not later than 60 days after the date of this report.  A
written statement also must be sent to the House and Senate
Committees on Appropriations with the agency's first request for
appropriations made more than 60 days after the date of this report. 

We appreciate the courtesy and cooperation extended to our audit team
by Air Force officials and staff.  We are providing copies of this
letter to the Chairman and Ranking Minority Members of the Senate
Committee on Governmental Affairs; the Subcommittee on Oversight of
Government Management, Restructuring and the District of Columbia,
Senate Committee on Governmental Affairs; the Subcommittee on
Defense, Senate Committee on Appropriations; the Senate Committee on
Armed Services; the Subcommittee on Government Management,
Information, and Technology, House Committee on Government Reform and
Oversight; the Subcommittee on National Security, House Committee on
Appropriations; and the House Committee on National Security.  We are
also sending copies to the Honorable Thomas M.  Davis, III, House of
Representatives; the Deputy Secretary of Defense; the Acting
Assistant Secretary of Defense for Command, Control, Communications
and Intelligence; the Air Force Chief Information Officer, Department
of Defense; and the Office of Management and Budget; and other
interested parties.  Copies will be made available to others on
request. 

If you have any questions on matters discussed in this letter, please
call me at (202) 512-6240 or John B.  Stephenson, Assistant Director
at (202) 512-6225.  Major contributors to this report are listed in
appendix III. 

Sincerely yours,

Jack L.  Brock, Jr.
Director, Information Management Issues


AIR FORCE YEAR 2000 ORGANIZATIONAL
STRUCTURE
=========================================================== Appendix I

As figure I.1 below indicates, the size and complexity of the Air
Force's organization structure will pose a significant management
challenge.  Year 2000 management and oversight efforts will have to
be coordinated among 9 major commands, each with complex and diverse
organizational structures of their own, 3 direct reporting units, and
36 field operating agencies. 

Figure I.2 provides an example of just one command's organizational
structure.  To understand the complexity involved in carrying out
Year 2000 efforts at the command level, consider the following: 

  -- the Air Force Materiel Command employs about 112,000 personnel;

  -- the command manages about 1,700 computer applications and
     embedded systems;

  -- 175 of these systems cover 21 various types of aircraft,
     including the F-22 and F-16 fighters, the B-1 and the B-2
     bombers, and C-17 cargo plans;

  -- 410 of these systems are business applications;

  -- 266 of these systems are applications covering command, control,
     communications and intelligence activities;

  -- 915 of these systems are base-level owned and operated
     applications, such as local area networks and medical systems;
     and

  -- the Air Force Materiel Command alone has about 50 Year 2000
     points-of-contact. 

   Figure I.1:  Air Force Year
   2000 Organizational Structure

   (See figure in printed
   edition.)

   Figure I.2:  The Air Force
   Materiel Command Organizational
   Structure

   (See figure in printed
   edition.)




(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
THE
AIR FORCE
=========================================================== Appendix I



(See figure in printed edition.)



(See figure in printed edition.)


MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III

ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C. 

Cristina T.  Chaplain, Communications Analyst
Alicia Loudis Sommers, Senior Information Systems Analyst

CHICAGO/DAYTON FIELD OFFICE

Robert P.  Kissel, Jr., Senior Evaluator
Steven M.  Hunter, Senior Evaluator
Robert G.  Preston, Senior Evaluator

*** End of document. ***