[Evaluation Report on Year 2000 Readiness of Automated Information Systems at the Bureau of Reclamation]
[From the U.S. Government Printing Office, www.gpo.gov]

Report No. 99-i-165

Title: Evaluation Report on Year 2000 Readiness of Automated Information
       Systems at the Bureau of Reclamation

Date:  January 8, 1999




                  **********DISCLAIMER**********

This file contains an ASCII representation of an OIG report.
No attempt has been made to display graphic images
or illustrations.
Some tables may be included, but may not resemble those
in the printed version.
A printed copy of this report may be obtained by referring
to the PDF
file or by calling the Office of Inspector General, Division of
Acquisition and Management Operations at (202) 208-4599.

                  ******************************






U.S. Department of the Interior
Office of Inspector General


EVALUATION REPORT


YEAR 2000 READINESS OF AUTOMATED INFORMATION
SYSTEMS AT THE BUREAU OF RECLAMATION

REPORT NO. 99-I-165
JANUARY 1999




MEMORANDUM


             TO:  The Secretary

           FROM:  Eljay B. Bowron
                  Inspector General

SUBJECT SUMMARY:  Final Evaluation Report - "Year 2000
                  Readiness of Automated Information Systems
                  at the Bureau of Reclamation" (No. 99-i-165)

Attached for your information is a copy of the subject  final
evaluation  report.  The objective of our review was to determine
whether the Bureau of Reclamation  inventoried its automated
information systems and identified those systems that were
mission critical and were not Y2K (Year 2000)  compliant and
whether the Bureau and its Denver Administrative Service Center
(1) developed auditable cost estimates for renovating systems to
be Y2K compliant; (2) identified, by name, individuals
responsible for ensuring that the Bureau is Y2K compliant; (3)
ensured that responsible individuals' personnel performance
evaluation plans included critical elements related to
identifying and remedying Y2K problems; (4) developed a credible
plan that included milestones and a critical path to ensure that
the Bureau is Y2K compliant; and (5) developed a contingency plan
that would address the failure of any part of the systems not
being Y2K ready.  This review was conducted at the request of the
Department of the Interior's Chief Information Officer.

Although the Bureau of Reclamation has taken actions to address
Y2K problems, we concluded that of the six areas the Chief
Information Officer requested us to evaluate, the Bureau had
completed actions for two areas and had not completed actions for
four areas.  Specifically, the Bureau had designated responsible
officials and had inventoried its automated information systems.
However, the Bureau reported only 16 of its 48 noncompliant
mission-critical systems with the approval of the Department's
Office of Information Resources Management.  Additionally, the
Bureau had not included critical elements related to identifying
and remedying Y2K problems in all responsible individuals'
personnel performance evaluation plans; had not included all
systems in its master plan, which had completion dates that were
inaccurate; had contingency plans that may not be adequate; and
had not developed auditable cost estimates.

Of the five areas that the Chief Information Officer requested us
to evaluate at the Service Center, the Service Center had
completed actions for three areas,  had not completed actions for
one area, and had determined that one area was not applicable.
Specifically, the Service Center had developed a credible plan,
including milestones; had designated responsible individuals; and
had updated the annual personnel performance evaluation plans.
However, the Center had  not completed action on developing a
contingency plan but had determined that the development of
auditable cost estimates was not applicable to the Federal
Personnel Payroll System.

The report did not contain any recommendations because the Bureau
said that it was taking actions which should enable it to meet
the required milestones for having Y2K compliant mission-critical
systems.

If you have any questions concerning this matter, please contact
me at (202) 208-5745 or Mr. Robert J. Williams, Assistant
Inspector General for Audits, at (202) 208-4252.


Attachment




                                              A-IN-BOR-001-98- R



EVALUATION REPORT


Memorandum

     To:  Commissioner, Bureau of Reclamation
          Director, Denver Administrative Service Center, Bureau
          of Reclamation

   From:  Robert J. Williams
          Assistant Inspector General for Audits

Subject:  Evaluation Report on Year 2000
          Readiness of Automated Information Systems
          at the Bureau of Reclamation (No. 99-i-165)

INTRODUCTION

This report presents the results of our evaluation of the year
2000 (Y2K) readiness of  automated information systems at the
Bureau of Reclamation and the Bureau's Denver Administrative
Service Center.  The objective of our review was to determine
whether the Bureau inventoried its automated information systems
and identified those systems that were mission critical and were
not Y2K compliant and whether the Bureau and the Service Center
(1) developed auditable cost estimates for renovating systems to
be Y2K compliant; (2) identified, by name, individuals
responsible for ensuring that the Bureau is Y2K compliant; (3)
ensured that responsible individuals' personnel performance
evaluation plans included critical elements related to
identifying and remedying Y2K problems; (4) developed a credible
plan that included milestones and a critical path to ensure that
the Bureau is Y2K compliant; and (5) developed a contingency plan
that would address the failure of any part of the systems not
being Y2K ready.  This review was conducted at the request of the
Department of the Interior's Chief Information Officer to assist
the Information Officer in monitoring the progress of
Departmental agencies in ensuring Y2K readiness, implementing Y2K
compliant systems, and validating the accuracy of the information
reported by the Departmental agencies to the Chief
Information Officer.

BACKGROUND

The "Y2K problem" is the term used to describe the potential
failure of information technology systems, applications, and
hardware related to the change to the year 2000.  Many
computer systems that use two digits to keep track of the date
will, on January 1, 2000, recognize "double zero" not as 2000
but as 1900.  This could cause computer systems to stop running
or to start generating erroneous data.  The problem has been
recognized as nationally significant by the President in
Executive Order 13073, issued in February 1998.  The Secretary
of the Interior, in a December 1997 memorandum, stated that the
Y2K problem was critical to the Department in meeting its
mission and that resolution of the problem was one of  his
highest  priorities.   Further, Office of Management and Budget
Memorandum 98-02, "Progress Reports on Fixing Year 2000
Difficulties," issued on January 20, 1998, requires all Federal
executive branch agencies to ensure that Federal Government
systems do not fail because of the change to the year 2000 and
to have all systems, applications, and hardware renovated by
September 1998, validated by January 1999, and implemented (that
is, "fixes to all systems--both mission critical and non-mission
critical") by March 1999.  The Office of Management and Budget,
in Memorandum 98-02, states that it is to provide "information
to the Congress and the public as part of its [Office of
Management and Budget's] quarterly summary rep orts on agency
progress . . . [and] to report on the status of agency
validation and contingency planning efforts and on progress in
fixing . . . equipment that is date sensitive."

The Department has developed the "Department of the Interior Year
2000 Management Plan," which focuses on the resolution of the Y2K
problem and provides an overall strategy for managing
Departmental mission-critical systems and infrastructure.  The
Department has a multitiered approach to managing the Y2K problem
that includes a top tier, which comprises the Secretary of the
Interior; the Information Technology Steering Committee, which
consists of the Chief of Staff and Assistant Secretaries; and the
Chief Information Officer, who is responsible for the
Department's Y2K issues.  This tier, which represents
senior-level Departmental managers, provides the Y2K project's
direction and resources and ensures accurate reporting to
external organizations, such as the Office of Management and
Budget and the Congress.  A Departmentwide Y2K project team,
which reports to the Chief Information Officer and comprises
representatives from each agency and the Office of the Secretary,
is tasked with developing the Department's "Year 2000 Management
Plan," refining inventory data on the Department's
mission-critical and information technology portfolio systems,[1]
and monitoring and reporting the progress of each conversion.

In addition, a Y2K Embedded Microchip[2] Coordinators Team has
been established to inventory and monitor embedded microchip
technology Y2K problems.  The team is led by the Office of
Managing Risk and Public Safety and comprises representatives of
the eight Departmental agencies, the Denver Administrative
Service Center, and various Departmental offices.

The Department's May 15, 1998, "Progress Report," which was
submitted to the Office of Management and Budget, reported that
the Department had 91 mission-critical systems, of which the
Bureau of Reclamation had 16 systems (see Appendix 1).  In
addition, the Federal Personnel Payroll System (FPPS), which was
developed and maintained by the Service Center, is 1 of the
Office of the Secretary's mission-critical systems and is 1 of
the Department's 13 information technology portfolio systems. To
address the Y2K problems, the Bureau established a Y2K project
management structure.  The structure included multiple
"Coordination Teams," which were headed by an executive Y2K
manager who is the Director of the Management Services Office,
and the teams included the Reclamation Information Resources
Management Coordinator, the Manager of the Policy and Program
Management Group, the Bureau's IT (Information Technology)
Security Manager, and Y2K coordinators at Bureau program and
regional offices.  The Service Center also
established a Y2K project management structure that includes a
Y2K executive, a Y2K program manager, a Y2K coordinator, and
three Y2K program element leaders.

SCOPE OF EVALUATION

To accomplish our objective, we reviewed the documentation
available that supported the Bureau's information submitted to
the Department's Chief Information Officer for the May  1998
"Progress Report" and the documentation available that supported
the information submitted by the Service Center to the Office of
the Secretary.  We performed our evaluation during April through
July 1998 at the Reclamation Service Center's Management
Services Office and the Denver Administrative Service Center,
located in Denver, Colorado, and the Bureau's Eastern Colorado
Area Office, located in Loveland, Colorado.  We interviewed
personnel responsible for project coordination to identify the
Bureau's and the Service Center's Y2K plans and progress.  We
also interviewed personnel involved in various aspects of the
Y2K project, including coordination, compliance identification,
software remediation, and project management.

The evaluation was conducted in accordance with the "Quality
Standards for Inspections," issued by the President's Council on
Integrity and Efficiency, and included such tests and inspection
procedures considered necessary to accomplish the objective.  Our
conclusions on the status of the progress made by the Bureau in
addressing and remediating Y2K problems were based on reviews
of documentation maintained by the Management Services Office and
discussions with the Y2K coordinators throughout the Bureau and
with individuals performing remediation or replacement of
noncompliant applications or hardware.  Also, our conclusions on
the status of the progress made by the Denver Administrative
Service Center were based on reviews of documentation and
discussions with the Y2K program manager, the Y2K coordinator,
the program element leaders, and individuals performing
remediation or replacement of noncompliant applications or
hardware.  As specifically agreed to in our discussions with the
Department's Chief Information Officer, we did not validate or
certify that the Bureau's or the Service Center's systems were
Y2K compliant.

RESULTS OF EVALUATION

Regarding the six areas that the Chief Information Officer
requested us to evaluate, we found that the Bureau had completed
actions for two areas, had partially completed actions for three
areas, and had not completed action for one area.  Specifically,
the Bureau had designated responsible officials and had
inventoried its automated information systems, but the
Department's Office of Information Resources Management  agreed
that the Bureau had to report only 16 of its 48 noncompliant
mission-critical systems.  Additionally, the Bureau had
not included critical elements related to identifying and
remedying Y2K problems in all responsible individuals' personnel
performance evaluation plans; had not included all systems in
its master plan, which had completion dates that were
inaccurate; and had contingency plans that may not be adequate.
For the remaining area, the Bureau had not developed auditable
cost estimates.  Because actions have not been completed for all
areas, we believe that there is a risk that the Bureau may not
meet the Office of Management and Budget's target date of March
1999 for having compliant Y2K systems implemented.

Additionally, of the five areas that the Chief Information
Officer had requested us to review, we found that the Denver
Administrative Service Center had completed actions for three
areas, had not completed actions for one area, and had determined
that one area was not applicable.  Specifically, the Service
Center had developed a credible plan, including milestones; had
designated responsible individuals; and had updated the annual
personnel performance evaluation plans. However, the Center had
not completed action on developing a contingency plan but had
determined that the development of auditable cost estimates was
not applicable to FPPS.  As a result of the progress being made
by Service Center Y2K project management, we believe that the
Service Center will meet the Office of Management and Budget's
target date of March 1999 for having compliant Y2K systems
implemented for FPPS if the Y2K project proceeds as scheduled.
If delays are encountered, development of contingency plans may
be necessary.

The specific actions taken by the Bureau of Reclamation and the
Service Center related to each area and other issues affecting
the Bureau's and the Service Center's Y2K progress are discussed
in the paragraphs that follow.

Automated Information Systems Inventory

Although the Bureau had performed an inventory of all of its
automated information systems except for international project
offices and identified 48 mission-critical systems, the Bureau's
May 1998 "Quarterly Report" to the Department's Chief
Information Officer showed only 16 noncompliant mission-critical
systems. We found that the Bureau was reporting only 16 systems
as mission critical because its Y2K project management did not
use the Department's criterion for reporting mission-critical
systems, which states that "those systems that when their
capabilities are degraded, the organization realizes a resulting
loss of a core capability or life or property are threatened."
Although Bureau Y2K project management did not use the
Department's criterion for reporting, the Bureau received
approval from the Department's Office of Information Resources
Management to report only those systems that were to be repaired
because of the large number of mission-critical systems that
would otherwise have to be reported and managed.  Therefore,
Bureau Y2K project management said that it tracked and reported
only systems that met all of the following Bureau criteria:  (1)
were mission critical, (2) were not Y2K compliant, (3) were date
dependent, and (4) were being redeveloped or repaired. However,
Office of Management and Budget Memorandum 98-02 requires that
executive agencies report the "total number of mission-critical
systems," as well as the "number compliant, number being
replaced, number being repaired, and number being retired." As a
result, the Bureau and  Departmental Y2K project management were
not tracking the replacement of Bureau mission-critical systems,
such as the Mid-Pacific Region's Sutron Database System and the
Centralized Water and Power System Control system, that were not
Y2K compliant and were not reporting these systems to the Office
of Management and Budget. However, in its September 23, 1998,
response (Appendix 2) to the draft report, the Bureau stated
that it is providing "high-level, ongoing man agement attention
to ensure that all mission-critical applications will be Y2K
compliant in sufficient time prior to January 2000."
Additionally, Y2K project management had not initially ensured
that systems which had been identified by Bureau regional
personnel as Y2K compliant were Y2K compliant.  However, Y2K
project management stated in exit conferences that a process was
in place as of July 1998 to ensure that systems which had been
identified by Bureau regional personnel as Y2K compliant are Y2K
compliant.

We also found that the Bureau's method of reporting
mission-critical systems to the Department's Chief Information
Officer did not focus on the Bureau's mission and ability to
perform its core capabilities and that each region was allowed
to define its mission-critical systems. Therefore, the reporting
of mission-critical systems was not consistent within the
Bureau. For example, the Bureau tracked and reported the Upper
Colorado Region's Colorado River Storage Project (CRSP) and the
Great Plains Region's Wyoming Area Office  Supervisory Control
and Data Acquisition (SCADA)[3] systems as mission critical.
However, there are SCADA systems in the other regions, such as
the Lower Colorado and the Mid-Pacific, that were identified as
mission critical and not Y2K compliant at the time the inventory
was completed but were not report ed.  By not ensuring that all
mission-critical systems which support core capabilities are Y2K
compliant before the year 2000, the risk is increased that some
of the Bureau's systems may fail and that the Bureau may not be
able to deliver water and hydroelectric power to its customers
without incurring significant personnel costs.

The Department's Chief Information Officer requested that we
determine the progress of the Bureau and the Service Center in
addressing embedded microchips in information systems and
facilities. We found that the Bureau and the Service Center, at
the time of our review, had begun to inventory embedded
microchips in information systems and facilities.

Auditable Cost Estimates

Of the 16 mission-critical systems reported to the Department's
Chief Information Officer, documentation was maintained for 2
systems:  the Technical Service Center's Data Acquisition and
Management System (DAMS) and the Upper Colorado Region's CRSP
SCADA system.  We found that the cost estimate of $9,000 for DAMS
was auditable and that the revised estimate of $285,300 for the
Upper Colorado Region's CRSP SCADA system also was auditable.

Although cost estimates reported for the remaining 14
mission-critical systems were not auditable, we attempted to
determine whether the methodologies used by Bureau personnel to
develop the cost estimates were reasonable.  Based on
information from regional personnel responsible for developing
the cost estimates, we found that the methodologies used varied.
For example, cost estimates for the Modsim, the PNOPER, and the
Hydromet PN1 systems in the Pacific Northwest Region were
initially based on total lines of source code[4] multiplied by
$1.50.  However, personnel in the Region said that because they
believed the results were too high, they lowered the amounts for
reporting purposes based on "best estimates."

In addition, the EM340 terminal emulator in the Pacific
Northwest Region was renovated to be Y2K compliant. However, we
found that the renovation costs were $35 rather than the $5,000
reported.  For the SCADA system in the Great Plains Region's
Wyoming Area Office, personnel in the Region said that the
amount was a "best estimate."  For the Hydromet Support and the
North Platte River Daily Water Accounting systems, which also
are in the Great Plains Region, personnel in the Region said
that the estimates were based on total lines of source code
multiplied by $1.50.  Additionally, renovation of the
Hydrological River Operations Study System (HYDROSS) at the
Technical Service Center was estimated to cost $5,680, which,
according to the Y2K coordinator, was based on an estimated 2
weeks of effort (80 hours) multiplied by $60 per hour.  However,
this formula would result in an estimate of $4,800.  In its
response to the draft report, the Bureau stated that
the costs were $500 to renovate the EM340 terminal emulator
and $5,680 for the HYDROSS.  However, because adequate
documentation to support these costs was not provided and
because the Bureau stated in its response that "in most
instances there have been no means to track Y2K
costs," we determined that the Bureau's reported estimated and
actual costs were not auditable.

In its response to the draft report, the Bureau stated that the
Office of Inspector General did not "expect consistent and
auditable cost estimates at this point."  However, we stated
that we did not expect the Bureau to expend resources in
correcting prior estimates but to ensure that future estimated
and actual costs were supported and auditable.

According to Service Center Y2K project management, the Service
Center had not developed any cost estimates related to FPPS
because the System was reported as compliant by design.  Project
management was aware that products and the mainframe operating
system which were used to develop and operate FPPS were
identified as not Y2K compliant. However, Service Center Y2K
project management said that they considered the upgrades to
these systems to be part of normal maintenance and not directly
related to Y2K. Consequently, the cost estimates related to Y2K
remediation were not applicable.
However, if costs specifically related to remediating Y2K
problems are identified, these costs should be reported to the
Department's Chief Information Officer.

Designation of Responsible Individuals

We found that the Bureau had specifically designated, by name,
the Y2K executive, the Bureau Y2K coordinator, and Y2K
coordinators in each of the Bureau's regional offices in its
"Year 2000 IT Comprehensive Plan."  In addition, the Service
Center had specifically named a Y2K executive, a Y2K program
manager, a Y2K coordinator, and three Y2K program element
leaders.  Therefore, the Bureau and the Service Center
had completed this requirement.

Annual Personnel Performance Evaluation Plans

The Secretary of the Interior's December 1997 memorandum
required that "a critical performance element for identifying
and remedying" the Y2K problem be included as part of each
responsible official's annual performance plan.  Responsible
officials are defined in the memorandum as agency directors,
agency Y2K executives, agency information resources management
coordinators, safety officials, and all others as determined by
the Y2K executives.  We found that 5 of the 13 Bureau Y2K
coordinators, the Bureau Information Resources Management
Coordinator, and the Bureau Y2K executive had elements
addressing Y2K objectives in their annual personnel performance
evaluation plans.  However, the remaining eight Bureau Y2K
coordinators did not have such elements included in their annual
personnel performance evaluation plans.

We found that all six members of the Denver Administrative
Service Center's Y2K project team had elements addressing Y2K
objectives in their annual personnel performance evaluation
plans.

Plan for Milestones

We found that the Bureau had provided a reasonable basis for
developing the master plan and critical paths for the systems
reported to the Department as part of the progress reports.
Specifically, the milestones established in the Bureau's master
plan were developed by system owners or other responsible
persons in each region who were knowledgeable of the systems,
and the Bureau Y2K coordinator used a project management
software tool to assist in developing the plan.
However, as discussed in the section "Automated Information
Systems Inventory" in this report, not all mission-critical
systems had been included in the master plan.  In addition,
although the initial milestones appear to be reasonable, some of
the completion dates reported in the Bureau's progress report
were inaccurate (see the section "Other Issues").

The Service Center had developed five action plans:  (1) the
"Year 2000 Conversion Project Action Plan," (2) the "DASC
Telecommunications Year 2000 Compliance Plan," (3) the "DASC
LAN/PC Systems Year 2000 Compliance Plan," (4) the "Y2K Embedded
Chip Project Plan," and (5) the "FPPS Year 2000 Conversion
Action Plan." All of these plans contain milestones and critical
paths to address Y2K compliance for each system.  Although these
plans were internal Service Center documents that were not
submitted to the Department's Chief Information Officer
and dates were revised as necessary, we believe that the plans
adequately identified milestones and critical areas.  As of May
1998, these action plans had completion dates of March 1999 or
earlier.

Contingency Plans

We found that the Bureau had contingency plans for 15 of the 16
reported mission-critical systems.  The Bureau did not develop
a contingency plan for the Pacific Northwest Region's EM340
system because Bureau management stated that this system was
compliant and a contingency plan was not necessary.  The
contingency plans for the 15 remaining systems were not
specifically related to Y2K but were existing plans for
disasters such as floods or earthquakes at the
project sites.  If the systems fail on January 1, 2000, the
Bureau's contingency plans are to perform the functions of these
systems manually based upon the procedures defined in the
disaster plans.  The disaster recovery plans may not be adequate
in the case of Y2K failures because of the possible length of
time required to remediate the affected systems and the ready
availability of people to repair lines of code or to perform the
manual operations. The disaster plans we reviewed were for
specific projects or clusters of projects, such as for Hoover,
Parker, and Davis Dams, and did not assess the impact on the
Bureau for the loss of systems such as the SCADA. Further, the
plans did not identify the number and experience level of
personnel required to operate the Bureau's facilities manually
and did not take into consideration the impact that embedded
microchips in cellular phones, telephones, radios, and
automobiles may have on the Bureau's ability to operate its
facilities. For example, we found the following:

   - Bureau management said that they believed "people will be
   available at the power plants to take over [operate the power
   plants manually]" if the Bureau's noncompliant SCADA systems,
   including hardware, software, and sensing devices, do not
   function after December 31, 1999.  In the Department's May
   1998 "Progress Report" to the Office of Management and Budget,
   the Bureau requested a waiver to the Dual Compensation Act[5]
   from the Office of Personnel Management to hire 10 power plant
   operators should the SCADA systems fail. In its response, the
   Bureau stated that two SCADA systems were not Y2K compliant
   and that "[m]ost operations managers do not expect a need for
   additional help."   Therefore, according to the Bureau, the
   request for 10 power plant operators was "Reclamation-wide in
   nature to cover unforseen contingencies." However, we continue
   to believe that if SCADA systems fail and the additional
   people required to operate water project gates and power
   plants manually are not readily available, the Bureau may not
   have control over water flow and its power plants.

   -   The North Platte River Daily Water Accounting system
   automates the daily accounting for stream flows, reservoir
   conditions, and ownership in the North Platte River Basin in
   Wyoming.  We found that if the system fails, the Wyoming Area
   Office will be prevented from performing the daily accounting
   of the North Platte River Basin. Wyoming Area Office officials
   stated that the Bureau has a legal requirement to supply
   information generated by the system.

In its response to the draft report, the Bureau stated that
although the Continuity of Operations Plans did not address the
failure of "cellular phones, telephones, radios, and
automobiles," the Bureau "cannot be held responsible for global
and common possibilities outside our scope or ability to
control."  However, the Bureau stated that it is developing a
contingency and management guide for power and water
facilities.  We believe that when contingency plans are
completed, the Bureau will be better able to ensure that its
water and power facilities will operate beyond the year 2000.

The Denver Administrative Service Center had not developed
contingency plans related to Y2K for the mainframe systems
because, according to Service Center Y2K project management, the
systems were to be Y2K compliant by the fall of 1998.  Service
Center Y2K project management stated that if this target date
was not met, a contingency plan would be developed.

Other Issues

We found other issues that affect the Bureau's and the Service
Center's Y2K readiness efforts which should be addressed as
follows:

       - Data Exchange.  The Department of the Interior and the
       Office of Management and Budget required that an inventory
       of all data exchanges with outside parties be completed by
       February 1, 1998, and that coordination with these parties
       to determine a transition plan occur by March 1, 1998.  We
       found that the Bureau had not met the Office of Management
       and Budget's target date in that only 22 of the 33 Bureau
       field offices responded to the Bureau Y2K coordinator's
       request for data exchange information as of May 1998.

The Service Center had inventoried its data exchanges, including
external and internal interfaces, and had contacted responsible
parties for all but 5 of the 48 interfaces identified in the
inventory that were in use.  According to Service Center Y2K
project management, two of the five interfaces will be retired
before the year 2000, and the remaining three interfaces were
determined to be the receivers' responsibility.  However,
documentation was not available to support that these parties
were contacted by Service Center project management to confirm
responsibilities.  - Independent Verification and Validation.
According to Bureau management,  the independent verification
and validation testing of mission-critical systems being
renovated was to be performed internally by Bureau staff because
of the expertise needed to test the systems and the cost
involved in having independent verification and validation
performed by outside contractors.  According to Service Center
Y2K project management, the Service Center, at the time of our
review, was the use of a contractor to assist in the independent
verifica tion and validation testing of FPPS for Y2K.  However,
we found that neither the Bureau nor the Service Center had
developed independent verification and validation test plans or
performed independent verification and validation testing of
mission-critical systems as of May 1998.

      - Compliance Reporting.  The Bureau had reported to the
      Department's Chief Information Officer that the Wyoming
      Area Office's SCADA system was compliant as of December
      1997 except for certification of the system.  However,
      based on information from Great Plains Regional personnel
      responsible for the system, we found, at the time of our
      review, that the repaired system had not been implemented.
      In its response to the draft report, the Bureau stated
      that the system had been implemented.

The Service Center's mainframe computer, the computer that
operates FPPS and the Department's Federal Financial System
(FFS), was reported to be compliant as of July 1997.  However,
we found that the Service Center's version of the mainframe
computer operating system was not Y2K compliant and would not be
compliant unless more than 100 program temporary fixes were
implemented or the system was upgraded to a newer version of the
operating system.  Service Center Y2K project management said
that they planned to upgrade to the newer version of the
operating system and to test the upgraded operating system for
Y2K compliance by August 1998.  In addition, the Service Center
reported that FPPS was compliant by design.  However, the
software products used to develop and operate FPPS were not Y2K
compliant (see the section "Auditable Cost Estimates").  These
issues were not addressed in the Service Center's information
submitted to the Department's Chief Information Officer for the
May 1998 "Progress Report."

      - Vendor Certifications.  We found that to determine the
      Y2K compliance of vendor-supplied hardware and software,
      Service Center Y2K project management relied on the
      vendors' written certifications. However, when the
      vendors' certifications could not be obtained, information
      contained in the vendors' Internet home pages was used.
      Although the Service Center had requested written
      certifications of Y2K compliance from its 25 mainframe
      software vendors, Service Center Y2K project management
      had received only letters certifying Y2K compliance for
      the software in use from 6 of the vendors.  As of May
      1998, the status of the requests made to the data
      communication vendors by the Service Center was as
      follows:

      - Responses had not been received from vendors on about 4
      percent of the Service Center's components.
      - Responses had been received from vendors stating that
      these components were Y2K compliant on about 18 percent of
      the components.       - Service Center Y2K project
      management was still       addressing the Y2K problems for
      35 percent of its data communication components through
      planned software upgrades, date independence, and end of
      life for hardware and software components (no longer
      needed).

      - Service Center Y2K project management relied on
      Internet site information for the remaining 43 percent of
      the Service Center's components to ensure Y2K compliance.

However, Service Center Y2K project management stated that all
data communication software and hardware would be tested where
possible.

- System Component Consolidation.  The Bureau reported each
component of the Pacific Northwest Region's Sutron Hydromet
system as an individual mission-critical system.  However,
personnel in that region responsible for the system said that
these components should not have been reported individually.  If
the components were combined into one mission-critical Hydromet
system, the number of mission-critical systems reported for the
Pacific Northwest Region would be 5 rather than 10.

- System Owners.  In the Bureau's Y2K master plan, regional
offices rather than personnel were identified as system owners.
Although the Bureau's master plan identified a contact (by name)
for each of the mission-critical systems reported, we believe
that the designation of regional offices as system owners did
not meet the intent of the General Accounting Office's "Year
2000 Computing Crisis: An Assessment Guide," which the
Department required bureaus to follow for Y2K project management
and for identifying system owners.

- The Federal Financial System.  During our review, we
found that Bureau Y2K project management as a customer and
Service Center Y2K project management as a service provider were
concerned that the FFS Y2K testing may not be completed as
scheduled.  FFS is used by six bureaus within the Department of
the Interior and by other Federal agencies.  FFS is being
renovated under the Office of the Secretary, and acceptance
testing is the responsibility of the U.S. Geological Survey's
Washington Administrative Service Center.  Further, the Geolo
gical Survey's mainframe computer, which is the computer where
FFS acceptance testing is performed and where FFS operates for
three bureaus, had the same operating system as the Denver
Administrative Service Center's mainframe computer (which is not
Y2K compliant and will not be compliant unless more than 100
program temporary fixes are implemented or the system is
upgraded to a newer version of the operating system).  Bureau
and Service Center Y2K project management said that they are not
certain that FFS can be fully tested at either the Denver or the
Washington Administrative Service Center until the upgraded
version of the mainframe operating system has been implemented
at each Center.  Additionally, the originally scheduled
completion date of June 1998 for implementation of the renovated
FFS was not feasible because the upgrade to the operating system
was not available for testing until July 1998, and the Bureau
and the Denver Administrative Service Center expressed concerns
that problems related to Y2K could be encountered during fiscal
year 1999.

On July 30, 1998, we held an exit conference with Y2K project
management of the Denver Administrative Service Center and on
August 6, 1998, with project management of the Bureau of
Reclamation and the Chief Information Officer.  Service Center
Y2K project management generally agreed with the conclusions
contained in this report. However, Bureau Y2K project management
expressed concerns regarding our conclusions on the Bureau's
reporting of mission-critical systems and adequacy of
contingency plans.  The Bureau provided additional information
in its response. Specifically, the Bureau generally concurred
with the report and said that it "will continue to focus on Y2K
project issues" and on meeting Departmental and Office of
Management and Budget milestone dates.  Although the Bureau
disagreed with our conclusion regarding the adequacy of its
contingency plans and stated that only one SCADA will not be Y2K
compliant by January 2000, it cited actions being taken which
should enable the Bureau to meet the required milestones for
having Y2K compliant mission-critical systems except for the
"Mid-Pacific's CVACS."  Based on discussions with Bureau Y2K
project management and on the response, we made changes to the
report as appropriate, and we have not made any recommendations.

Since this report does not contain any recommendations, a
response is not required.

The legislation, as amended, creating the Office of Inspector
General requires semiannual reporting to the Congress on all
audit reports issued, the monetary impact of audit findings,
actions taken to implement audit recommendations, and
identification of each significant recommendation on which
corrective action has not been taken.

We appreciate the assistance of personnel at the Bureau of
Reclamation's Management Services Office and regional offices
and the Denver Administrative Service Center in the conduct of
our review.


APPENDIX 1
Page  of 2

BUREAU OF RECLAMATION/DENVER ADMINISTRATIVE SERVICE CENTER
MISSION-CRITICAL SYSTEMS INVENTORY[6]

System Name or
Acronym

Description

Estimated Cost for
Compliance

Great Plains (GP) Hydromet Support

Provides Hydromet data reporting and maintenance functions and
supports the capture and upload of data into Hydromet from
outside sources.

$50,000

EM340

Digital 340 Terminal Emulation software for Hydromet interfaces.

5,000

Hydromet PN1

Yakima Hydromet System.  Collects and processes hydrologic and
meteorologic data on a near-real-time basis.

30,000

Inhouse Hydromet

Hydromet data analysis tools.  Program uses the information from
Hydromet to retrieve, compute, and convert data.  This allows
users to format reports and use the data for analysis and
display.

16,000

Sutron Hydromet

Hydromet data collection, translation, and storage.

75,000

North Platte River Daily Water Accounting (NPRDWA)

Automated daily accounting of stream flows, reservoir conditions,
and ownership in the North Platte River Basin in Wyoming.

60,000

Wyoming Area Office Supervisory Control and Data Acquisition
(SCADA WYAO)

Monitors and controls 14 power plants and 3 irrigation canals,
and controls flows in 5 river systems.

3,000

Inhouse Agricultural and Meteorology Data (AGRIMET)

Data analysis and formatting.

22,500

System Name or
Acronym

Description

Estimated Cost for
Compliance

Inhouse River Operations

Data analysis tools.  Program uses the information from Hydromet
to retrieve, compute, and convert data.  This allows users to
format reports and use the data for analysis and display.

37,500

Model Simulator (Modsim)

River System Operations Simulation.  River modeling and
simulation program.

20,000

Pacific Northwest Operations (PNOPER)

Real-time hydrologic and meteorologic data to support the
Bureau's water resource management mission.  Supported functional
areas include flood control, hydrologic and structural monitoring
related to dam safety, irrigation water supply, power generation,
dam operations, and water supply.

15,000

Umatilla Planning Module

River System Operations Simulation.  River modeling and
simulation program.

20,500

Yakima Planning Model

River Operations Simulation.  River modeling and simulation
program.

20,000

Data Acquisition and Management System (DAMS)

Instrumentation database.

9,000

Hydrological River Operations Study System (HYDROSS)

Water rights and supply accounting model used in river basin
planning studies.

5,680

Colorado River Storage Project Supervisory Control and Data
Acquisition (CRSP SCADA)

Remotely controls generation and water bypass for 8 hydroelectric
plants, with a total of 19 generating units.

285,300**

Total

$674,480

**This revised estimated cost is the amount reported by the
Department of the Interior to the Office of Management and Budget
as of May 1998 in the Department's "Quarterly Progress Report."
[1] The portfolio is an inventory listing of 13 crosscutting or
sensitive systems  that are receiving attention at the
Secretarial level.

[2] Embedded microchips are "integrated circuits (miniature
circuit boards)" that  control "electrical devices," which
include "elevators;  heating, ventilation, and air  conditioning
(HVAC) systems; water and gas flow controllers;  aircraft
navigational  systems; . . . medical equipment"; and office
devices such as  telephones, facsimile
machines, pagers, and cellular telephones.  (Department of the
Interior's Office of  Managing Risk and Public Safety "Year 2000
Embedded Microchip  Hazards" [Web site])

[3] SCADA systems are systems that interface within selected
water projects and are  used by the Bureau to monitor and control
water flow and  hydroelectric power.  [4] Lines of source code
are statements and instructions used  by the computer to  execute
the tasks of computer programs.  (Computer Desktop  Encyclopedia,
Version 9.4,  4th quarter, 1996)

[5] A waiver from the Act allows the Bureau to hire retired
Federal employees  without loss or reduction to the employees'
entitlements.

[6] Information is from the "Department of the Interior Year
2000 Management Plan,"  issued in February 1998, and the Bureau
of Reclamation's "Y2K  Software Application  Report," dated
February 1998.




ILLEGAL OR WASTEFUL ACTIVITIES SHOULD BE REPORTED

TO THE OFFICE OF INSPECTOR GENERAL BY:

Sending written documents to:



Within the Continental United States

U.S. Department of the Interior
Office of Inspector General
1849 C Street,N.W.
Mail Stop 5341
Washington, D.C. 20240

Calling:

Our 24 hour
Telephone HOTLINE
1-800-424-5081 or
(202) 208-5300

TDD for hearing impaired
(202) 208-2420 or
1-800-354-0996



Outside the Continental United States


Caribbean Region

U.S. Department of the Interior
Office of Inspector General
Eastern Division- Investigations
1550 Wilson Boulevard
Suite 410
Arlington, Virginia 22209

Calling:
(703) 235-9221


North Pacific Region

U.S. Department of the Interior
Office of Inspector General
North Pacific Region
238 Archbishop F.C. F'lores Street
Suite 807, PDN Building
Agana, Guam 96910


Calling:
(700) 550-7428 or
COMM 9-011-671-472-7279