Year 2000 Computing Crisis: Progress Made in Compliance of VA Systems,
But Concerns Remain (Letter Report, 08/21/1998, GAO/AIMD-98-237).
GAO has reported in the past that unless timely corrective action is
taken, the Department of Veterans Affairs (VA) could face widespread
computer system failures at the turn of the century because of incorrect
information processing involving dates. (See GAO/T-AIMD-97-174, Sept.
1997; GAO/T-AIMD-97-114, June 1997; GAO/AIMD-97-79, May 1997; and
GAO/AIMD-96-103, June 1996.) In many systems, the year 2000 is
indistinguishable from the year 1900, which could make veterans who are
due to receive benefits and medical care appear ineligible. The upshot
is that benefits and health care that veterans depend on could be
delayed or interrupted. This report assesses the Year 2000 programs of
the Veterans Benefits Administration and the Veterans Health
Administration.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-98-237
TITLE: Year 2000 Computing Crisis: Progress Made in Compliance of
VA Systems, But Concerns Remain
DATE: 08/21/1998
SUBJECT: Y2K
Computer software
Systems conversions
Information resources management
Veterans benefits
Computer software verification and validation
Commercial products
Strategic information systems planning
Information systems
Systems compatibility
IDENTIFIER: VA Year 2000 Strategy
VBA Year 2000 Program
VHA Year 2000 Program
VBA Beneficiary Identification and Record Locator
Subsystem
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
GAO/AIMD-98-237
Cover
================================================================ COVER
Report to the Chairman, Subcommittee on Oversight and Investigations,
Committee on Veterans' Affairs, House of Representatives
August 1998
YEAR 2000 COMPUTING CRISIS -
PROGRESS MADE IN COMPLIANCE OF VA
SYSTEMS, BUT CONCERNS REMAIN
GAO/AIMD-98-237
VA Year 2000 Compliance
(511228)
Abbreviations
=============================================================== ABBREV
CIO - chief information officer
COTS - commercial-off-the-shelf
FDA - Food and Drug Administration
OMB - Office of Management and Budget
VA - Department of Veterans Affairs
VBA - Veterans Benefits Administration
VHA - Veterans Health Administration
VISN - Veterans Integrated Service Network
VISTA - Veterans Health Information Systems Architecture
Letter
=============================================================== LETTER
B-278053
August 21, 1998
The Honorable Terry Everett
Chairman, Subcommittee on Oversight
and Investigations
Committee on Veterans' Affairs
House of Representatives
Dear Mr. Chairman:
We have reported in the past that unless timely corrective action is
taken, the Department of Veterans Affairs (VA) could face widespread
computer system failures at the turn of the century due to incorrect
information processing relating to dates.\1 The reason for this is
that in many systems, the year 2000 is indistinguishable from the
year 1900,\2 which could make veterans who are due to receive
benefits and medical care appear ineligible. If this happens, the
issuance of benefits and the provision of medical care that veterans
rely on could be delayed or interrupted.
As you requested, we have assessed the status of (1) the Veterans
Benefits Administration's (VBA) Year 2000 program and (2) the
Veterans Health Administration's (VHA) Year 2000 program. This
report provides the results of our review.
--------------------
\1 Veterans Affairs Computer Systems: Action Underway Yet Much Work
Remains to Resolve Year 2000 Crisis (GAO/T-AIMD-97-174, September 25,
1997); Veterans Benefits Computer Systems: Uninterrupted Delivery of
Benefits Depends on Timely Correction of Year-2000 Problems
(GAO/T-AIMD-97-114,
June 26, 1997); Veterans Benefits Computer Systems: Risks of VBA's
Year 2000 Efforts (GAO/AIMD-97-79, May 30, 1997); and Veterans
Benefits Modernization: Management and Technical Weaknesses Must Be
Overcome if Modernization Is to Succeed (GAO/AIMD-96-103, June 19,
1996).
\2 The Year 2000 problem is rooted in how dates are recorded and
computed. For the past several decades, many existing systems have
used a two-digit date field to represent the current year--such as
"97" for 1997. However, such a format does not distinguish between
2000 and 1900. Computer programs that are not corrected to
accommodate the 2000 date could process information incorrectly,
affecting the payment of benefits and provision of services.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
VBA has made progress in addressing the recommendations in our May
1997 report\3 and making its information systems Year 2000 compliant.
It has changed its Year 2000 strategy from developing new
applications to fixing the current ones and established a Year 2000
project office to oversee and coordinate all VBA Year 2000 projects.
It has also reportedly renovated 75 percent of its mission-critical
applications as of June 1998, and completed renovation of two
specific mission-critical systems--vocational rehabilitation and
insurance.
Despite this progress, concerns remain. For example, VBA has made
limited progress in renovating two key mission-critical software
applications--(1) compensation and pension online, which processes
claims benefits and updates benefit information, and (2) the
Beneficiary Identification and Record Locator Sub-System. VBA also
has to reassess its commercial-off-the-shelf (COTS) products because
one of its largest vendors, who initially informed VBA that its
products were Year 2000 compliant, recently informed VBA that some of
its products were not compliant and that others were being assessed
and tested. This problem is not unique to VBA--it applies to all
consumers of these products. Finally, except for its Insurance
Service, VBA has not developed Year 2000 business continuity and
contingency plans for its core business processes. These issues, if
not adequately addressed, could affect the timely processing of
benefits to veterans and their dependents.
VHA has also made progress in addressing the Year 2000 problem.
Since September 1997, it has reported having assessed all and
renovated the vast majority of its mission-critical information
systems and having completed 98 percent of its renovation by June
1998. However, concerns also remain. For example, VHA does not know
the full extent of its Year 2000 problem because it has not yet
completed its assessment of (1) locally developed software
applications or customized versions of national applications used by
its medical facilities, (2) COTS products, such as software
applications and operating systems, (3) facility systems, such as
heating, ventilating, and air conditioning equipment, and (4)
biomedical devices. VHA's efforts on several of theses issues are
complicated by the fact that it, like other consumers of these
products, has to receive compliance information from the
manufacturers, some of whom have been slow to respond to VHA's
requests for compliance information. Like VBA, VHA has not developed
Year 2000 business continuity and contingency plans. Failure to
adequately address these issues could result in disruptions in
patient care at VHA medical facilities.
--------------------
\3 GAO/AIMD-97-79, May 30, 1997.
BACKGROUND
------------------------------------------------------------ Letter :2
VA comprises three major components: VBA, which provides benefits to
veterans and their dependents; VHA, which provides health care
services through the nation's largest health care system; and the
National Cemetery System, which provides burial services in 115
national cemeteries.
In fiscal year 1997, VBA reported that it paid about $23 billion in
benefits to about 4.4 million veterans and their dependents. VBA
distributes these benefits through five programs: compensation and
pension (the largest), educational assistance, housing loan guaranty,
vocational rehabilitation and counseling, and insurance. VBA
administers its benefits programs through one or more of 58 regional
offices supported by three data processing centers.
During this same period, VHA reported that it spent $17 billion
providing medical care to about 3 million veterans. Such care is
managed through 22 Veterans Integrated Service Networks (VISN) which
are geographically dispersed throughout the country. VISNs manage
711 separate facilities: 172 VHA medical centers, 376 outpatient
clinics, 133 nursing homes, and 30 domiciliaries.
VA has 11 mission-critical system areas. As shown in table 1, VBA
has six of these areas and VHA has two. About two-thirds of the VBA
software applications--95 of the 151--are considered business
priorities, essential to the department's mission. Over one third of
VHA's applications--104 of the 283--are business priorities critical
to the department.
Table 1
VA's Mission-Critical Computer System
Areas and Their Applications
Number of
Component/office (Number of applicatio
systems) Systems ns
---------------------------- ---------------------------- ----------
Veterans Benefit Compensation and Pension 151
Administration (6) Education
Insurance
Loan Guaranty
Vocational
Rehabilitation
Administrative
Veterans Health Veterans Health 145
Administration (2) Information Systems and
Technology Architecture
Veterans Health 138
Administration Corporate
Systems
National Cemetery System Burial Operations Support 2
(1)\a System/Automated Monument
Application System
Reengineer
Office of Financial Personnel and Accounting 1
Management (2) Integrated Data
Financial Management 1
System\b
----------------------------------------------------------------------
\a This was the first VA system to become Year 2000 compliant.
\b This system became compliant in February 1998.
Source: VA. We have not independently verified this information.
OBJECTIVE, SCOPE, AND
METHODOLOGY
------------------------------------------------------------ Letter :3
The objective of this review was to assess the status of VBA's and
VHA's Year 2000 programs. In conducting this review, we applied
criteria from our Year 2000 Assessment Guide,\4 and our Year 2000
Business Continuity and Contingency Planning Guide.\5 We reviewed
Year 2000 documents developed by VA, including its June 1997 Year
2000 Solutions document; its January 13, 1997, Year 2000 Readiness
Review; and its quarterly Year 2000 reports to OMB for May 1997,
August 1997, November 1997, February 1998, and May 1998.
In assessing the status of VBA's Year 2000 program, we reviewed and
analyzed numerous VBA documents, including its October 1997 Year 2000
Risk Assessment; its January 1998 response to our May 1997 report,\6
and monthly Year 2000 progress reports to VA. We also reviewed and
analyzed VBA project plans, schedules, and progress reports for its
replacement and conversion initiatives. In addition, we met with
and/or interviewed project teams, including contractor support, in
Washington, D.C., and Roanoke, Virginia, as well as computer
personnel at VBA's data centers at Hines, Illinois, and Philadelphia,
Pennsylvania, and its Austin Systems Development Center to discuss
their Year 2000 activities. We also discussed VBA's Year 2000
efforts with VBA headquarters officials in Washington, D.C.; VBA
regional office officials in Baltimore, Maryland; Chicago, Illinois;
Roanoke, Virginia; Philadelphia, Pennsylvania; Waco, Texas; St.
Paul, Minnesota; St. Petersburg, Florida; and representatives from
VA's Office of Information Resources Management.
In assessing the status of VHA's Year 2000 program, we reviewed and
analyzed numerous VHA documents, including its April 30, 1997, Year
2000 Plan; its July 1997 Year 2000 Product Risk Program; its January
30, 1998, Assessment Phase Report; its strategic plan; and monthly
reports to VA. We also reviewed the Year 2000 plan prepared by each
of VHA's 22 VISNs. We also reviewed and analyzed the backup and
contingency plans prepared by the VISNs and VISN cost estimates. In
addition, we met with and/or interviewed VISN project team members in
Pittsburgh, Pennsylvania; Philadelphia, Pennsylvania; Wilmington,
Delaware; Washington, D.C.; Baltimore, Maryland; Martinsburg, West
Virginia; and Chicago, Illinois. We also met with VHA software
development staff at Silver Spring, Maryland, and Hines, Illinois, to
discuss progress and problems involved with their Year 2000
activities. We also discussed VHA's Year 2000 efforts with VHA
headquarters officials in Washington, D.C., and a representative from
VA's Office of Information Resources Management.
We performed our work from July 1997 through June 1998, in accordance
with generally accepted government auditing standards. We requested
comments on a draft of this report from the Secretary of Veterans
Affairs or his designee. The Assistant Secretary for Policy and
Planning provided us with written comments that are discussed in the
"Agency Comments and Our Evaluation" section and are reprinted in
appendix I.
--------------------
\4 Year 2000 Computing Crisis: An Assessment Guide
(GAO/AIMD-10.1.14, September 1997).
\5 Year 2000 Computing Crisis: Business Continuity and Contingency
Planning (GAO/AIMD-10.1.19, August 1998).
\6 GAO/AIMD-97-79, May 30, 1997.
PROGRESS MADE IN VBA YET
CONCERNS REMAIN REGARDING
APPLICATIONS, COTS PRODUCTS,
AND CONTINGENCY PLANNING
------------------------------------------------------------ Letter :4
VBA has made progress in addressing the recommendations in our May
1997 report\7 and in making its information systems Year 2000
compliant. However, concerns remain surrounding the renovation of
two key mission-critical applications, assessment of COTS software
products, and contingency planning.\8
Unless these issues are addressed, the issuance of benefits to
veterans and their dependents could be delayed or interrupted.
--------------------
\7 GAO/AIMD-97-79, May 30, 1997.
\8 Our Year 2000 Computing Crisis: Business Continuity and
Contingency Planning (GAO/AIMD-10.1.19, August 1998) describes a
planning process that safeguards an agency's ability to produce a
minimally acceptable level of services in the event of failures of
mission-critical systems.
PROGRESS MADE IN ADDRESSING
THE YEAR 2000 PROBLEM
---------------------------------------------------------- Letter :4.1
In May 1997, we made numerous recommendations designed to correct
weaknesses in VBA's Year 2000 efforts. These were in the areas of
program management, systems assessment and prioritization, completion
of inventories and development of plans for addressing internal and
external software interfaces, prioritizing information technology
projects to make the best use of limited resources, and developing
contingency plans for critical business processes.
Additionally, in September 1997,\9 we expressed concern about the
compressed schedules of key software developments and renovations,
especially for VBA's largest and most critical payment system,
compensation and pension. The schedule for this system was
compressed because computer analysts responsible for it were not
available and were working on legislatively mandated changes and
cost-of-living increases. In addition, VBA had not completed
assessments of its internal and external data interfaces.
In concurring with all our recommendations and addressing other
concerns, VBA has taken several actions. For example, it changed its
Year 2000 strategy from developing new systems to converting existing
ones. Second, a single VBA Year 2000 project office now oversees and
coordinates all VBA Year 2000 activities. Third, VBA has completed
inventories for its mission-critical systems, data interfaces, and
third-party products, and assessed the products in these inventories
for Year 2000 compliance. Finally, VBA has developed a schedule for
replacing and/or converting all noncompliant systems or products.
VBA has also made progress in renovating its noncompliant software
applications. Since July 1997, it has reported increasing from 50 to
75 the percentage of renovated mission-critical applications. It
also has reported completing renovation for its vocational
rehabilitation and insurance systems, and making significant progress
in its education system. VBA has also recently completed inventories
and assessed Year 2000 compliance for applications developed at its
regional offices.
In addition, to address our recommendations and concerns regarding
its data interfaces, VBA has developed Year 2000-related agreements
with its external trading partners for 278 of 287 interfaces.
According to VBA's Year 2000 Project Manager, VBA has developed a
bridge\10 for five of the nine interfaces remaining without
agreements so that information from these partners can be converted
into an acceptable format. VBA must still work out agreements with
the Department of Defense on the remaining four interfaces, which
deal with payment systems. In these cases, the external partner has
not determined the data format for the interfaces so VBA can ensure
that the data are acceptable to its systems. The Project Manager
informed us that VBA will continue to work with Defense in resolving
this problem.
--------------------
\9 GAO/T-AIMD-97-174, September 25, 1997.
\10 Bridging involves receiving information in one format, modifying
it, and outputting it in another format, such as receiving the year
in a two-digit format, adding century information through the use of
an algorithm, and writing the output with a four-digit year.
RISKS REMAIN REGARDING
RENOVATION OF
MISSION-CRITICAL
APPLICATIONS, COTS PRODUCTS,
AND CONTINGENCY PLANNING
---------------------------------------------------------- Letter :4.2
Despite VBA's progress in addressing the Year 2000 problem, areas of
risk remain. These include (1) limited progress in making two key
mission-critical applications compliant, (2) the unexpected need to
reassess its COTS software products that VBA was previously assured
were fully Year 2000 compliant, and (3) the lack of business
continuity and contingency plans for core business processes. If
these issues are not adequately addressed, benefits to veterans could
be delayed or interrupted.
MAKING MISSION-CRITICAL
APPLICATIONS COMPLIANT
-------------------------------------------------------- Letter :4.2.1
VBA has made limited progress in renovating its compensation and
pension online application.\11 Specifically, the level of reported
renovation has remained at 18 percent for several months. As we
pointed out in our September 1997 testimony,\12 one factor for this
slow pace has been that computer analysts responsible for Year 2000
renovations have been working on legislative mandates, special
projects, and yearly changes, such as cost-of-living increases and
clothing allowances.
To address this problem, VBA contracted for additional Year 2000
support. The contractor was tasked with analyzing the software code
using a software conversion tool, providing the tool's
recommendations to VBA analysts, and making renovations, once
approved by VBA analysts. However, the Project Manager informed us
that only 35 percent of the renovations could be made using the tool.
The remaining 65 percent must be made by the analysts because they
involve benefit calculations or other issues that the tool cannot
handle. These same analysts must also review and test all Year 2000
changes, including those made by the contractor, as well as continue
to work on legislative mandates,\13 special projects,\14 and other
changes before 2000. In commenting on a draft of this report, VA
stated that VBA's compensation and pension online application is now
59 percent compliant, up from the 18 percent noted in our report.
VBA's Year 2000 Project Manager told us that the progress was due to
use of a software development tool and that additional resources had
been assigned to this project, as we recommended. The milestones for
the compensation and pension online application include completion of
renovation by September 4, 1998, and implementation by October 5,
1998.
VBA is also having problems renovating the Beneficiary Identification
and Record Locator Sub-System,\15 a software application in its
mission-critical administrative system. VBA hired a contractor to
modernize and renovate the system to ensure Year 2000 compliance.
The contractor, however, had difficulty meeting its deliverable
dates. For example, it did not complete renovation in April 1998, as
originally planned, or in June 1998, as rescheduled. According to
VBA's Year 2000 Project Manager, the contractor did not assign
sufficient staff to the project and did not have the expertise to
perform the work on time. She informed us that the contractor and
VBA have agreed to a third schedule, with specific contract
deliverables scheduled in July and August 1998. If these dates are
not met, VBA plans to invoke a contingency plan to fix this system
using in-house resources currently working on non-Year 2000 work,
such as VBA's common security project.
--------------------
\11 This application is used to process new claims benefits and
update current claims benefit information. According to a 1996 VBA
estimate, if this application fails to work, about 10 percent or
300,000 VBA payments might be adversely affected.
\12 GAO/T-AIMD-97-174, September 25, 1997.
\13 According to VBA, legislative mandates include a minimum income
for widows benefit, which was previously provided by the Department
of Defense and is now the responsibility of VA.
\14 These include initiatives to develop databases to assist VBA
management in data analysis and an interface to the Department of the
Treasury's payment system for VBA's systems modernization effort
known as VETSNET.
\15 This system contains veterans' names, addresses, service
histories, and claims folder locations.
REASSESSING COTS PRODUCTS
-------------------------------------------------------- Letter :4.2.2
VBA, along with other consumers, will have to reassess some of its
COTS products. According to VBA's Year 2000 Project Manager, one of
its largest vendors, which initially informed VBA that its products
were Year 2000 compliant, recently told VBA that some of its products
were not compliant and announced that it was beginning to assess and
test its product line for Year 2000 compliance. Because this vendor
accounts for about 30 percent of VBA's COTS products--including a
database management system used by many VBA software developers and
VBA's electronic mail software, which is used to communicate between
its regional offices, VBA headquarters, and the data centers--VBA
will have to replace and/or upgrade these products. Because of this
reassessment by one vendor, VBA now plans to review and reassess all
of its COTS products.
DEVELOPING BUSINESS
CONTINUITY AND
CONTINGENCY PLANS
-------------------------------------------------------- Letter :4.2.3
In May 1997, we reported that VBA should develop contingency plans
for its core mission-critical business processes and assess the
impact of Year 2000 failures on its mission-critical program
services. To assist agencies with this, we have prepared a guide\16
that discusses the scope of the Year 2000 challenge and offers a
step-by-step approach for reviewing an agency's risks and threats as
well as how to develop backup strategies to minimize these risks.
The business continuity planning and contingency planning process
outlined in the guide safeguards the agency's ability to produce a
minimally acceptable output if internal or external mission-critical
information systems and services fail. The guide also states that an
agency should develop business continuity and contingency plans for
each core business process or program service and that the plans
should provide a description of resources, staff roles, procedures,
and timetables needed for implementation.
VBA has generally not developed business continuity and contingency
plans for its program services to ensure that they would continue to
operate if Year 2000 failures occur. Only VBA's Insurance Service
has developed such a plan. The insurance plan addresses potential
business impact, contingencies and alternatives, and triggers for
activating contingencies and contains a high-level plan for the
December 31, 1999, weekend.
According to the VBA Year 2000 Project Manager, VBA is aware of the
need for Year 2000 business continuity and contingency plans and has
hired a contractor to develop a framework for developing plans to
address all aspects of business operations, not just information
systems. This framework is to be used to assist VBA in preparing a
Year 2000 business continuity plan that would allow it to maintain a
minimal acceptable level of service. VBA currently does not have a
date for when this framework will be completed, but it plans to have
business continuity and contingency plans completed for each of its
program services by January 1, 1999.
--------------------
\16 GAO/AIMD-10.1.19, August 1998.
VHA HAS MADE PROGRESS IN
RENOVATING MISSION-CRITICAL
SYSTEMS, BUT CONCERNS REMAIN
------------------------------------------------------------ Letter :5
VHA has made progress in assessing and renovating its two
mission-critical systems for Year 2000 compliance. Since our
September testimony,\17 it has reportedly completed assessment and 98
percent of renovation. However, despite this progress, concerns
remain about the Year 2000 compliance status of VHA's locally
developed software applications, COTS products, facility systems, and
biomedical devices. Also, VHA's medical facilities have not
completed Year 2000 business continuity and contingency plans. Until
its local software applications, COTS products, facility systems, and
biomedical devices are determined to be Year 2000 compliant, VHA
lacks assurance that its delivery of medical care to veterans will
not be delayed or interrupted at the turn of the century.
--------------------
\17 GAO/T-AIMD-97-174, September 25, 1997.
REPORTED PROGRESS MADE IN
ASSESSING AND RENOVATING
MISSION-CRITICAL SYSTEMS
---------------------------------------------------------- Letter :5.1
VHA has progressed in assessing and renovating its mission-critical
systems. It reportedly completed the assessment of the Veterans
Health Information Systems Architecture (VISTA) and VHA corporate
systems\18 at the end of January 1998. VHA reported at that time
that about half of the 147 VISTA software applications would have to
be renovated, as would about 10 percent of the VHA corporate systems
applications. It also completed assessment of related data
interfaces and reported that 6 percent of these interfaces would have
to be renovated.
In addition, according to its June 1998 Year 2000 Status Report to
VA, VHA renovated 100 percent of its VISTA software applications and
95 percent of the VHA corporate systems by the end of May 1998.
Further, VHA reports that 93 percent of the VISTA and 90 percent of
the VHA corporate software applications have been validated. For the
VISTA software applications, validation includes (1) testing by
programmers and systems development groups, (2) testing by medical
facility staff, (3) quality assurance testing, and (4) final testing
by the customer services unit prior to release. Testing for the VHA
corporate systems is similar to the process for the VISTA
applications. VHA also plans to conduct end-to-end testing along
with VBA in July 1999.
The VHA Year 2000 Project Manager told us that as an added
precaution, VHA has decided to hire a contractor to conduct an
independent verification and validation of VHA's compliance process
for mission-critical systems, especially its corporate systems. VHA
will be developing the requirements and a statement of work for this
effort and expects to award a contract by the end of September 1998.
It also expects to have the independent verification and validation
completed by the end of December 1998.
--------------------
\18 These corporate systems are a set of computers and databases
intended to gather information from more than one field facility to
support the VHA CIO's office.
REMAINING CONCERNS ABOUT
VHA'S YEAR 2000 PROGRAM
---------------------------------------------------------- Letter :5.2
In our September 1997 testimony,\19 we expressed concern that some of
the medical facilities have customized\20 the national VISTA software
applications and/or purchased software add-ons to work with the
national applications, and these have not been assessed for Year 2000
compliance. Further, we expressed concerns that VHA had not
completed an inventory of its COTS products, facility-related systems
and equipment, and biomedical devices.
The medical facilities still have not completed their assessment of
locally developed and/or modified VISTA applications. The most
recent information shows that about 76 percent of these applications
had been assessed as of May 31, 1998. Of the applications assessed,
16 percent are noncompliant. All medical facilities were scheduled
to complete renovations by July 1998. According to the VHA Year 2000
Project Manager, the facilities did not meet this date. He also
added that the project office is not aware of any modifications by
the medical facilities to the VISTA applications involving
mission-critical functions. Recognizing that the medical facilities
are behind schedule, VHA's Year 2000 Project Manager has told us that
the medical facilities have been informed that if they cannot assess
or renovate their locally developed or modified VISTA software
applications in time--as yet unspecified--they will have to replace
them with unmodified national versions. This appears to be a viable
solution since VHA has reportedly renovated 100 percent of its VISTA
applications, and 93 percent of these have been validated.
A second area of concern relates to the Year 2000 compliance status
of VHA's COTS software. VHA has over 3,000 such products,\21
supplied by nearly 1,000 vendors, for use in its offices and medical
facilities nationwide. Its strategy for determining the Year 2000
compliance status of these products is to request the information
from the manufacturers. VHA, however, has had difficulty in
obtaining this information. Since November 1997, it has sent letters
to 566 manufacturers requesting compliance information; as of June
1998, 260 responses had been received, leaving over 300 outstanding.
The Year 2000 Project Manager informed us that they will continue to
send follow-up letters to nonresponding manufacturers.
The reliability of the information received from COTS vendors may
also be an issue. For example, one large COTS software manufacturer
that initially provided documentation stating that all its products
were compliant, recently published "more detailed" information
stating that some of its products were, in fact, not fully compliant.
VHA is now reviewing this additional information to determine what
needs to be done at the headquarters and field levels to ensure Year
2000 compliance.
A third area of risk involves the Year 2000 compliance status of
VHA's facility-related systems and equipment, such as heating,
ventilating, and air conditioning equipment. As with its Year 2000
strategy for COTS products, VHA relies on the manufacturers for
compliance information on their products. It sent a letter to over
200 facility systems/equipment manufacturers requesting Year 2000
compliance information. In March 1998, VHA sent follow-up letters to
127 nonrespondents. To date, it has received responses from 93 of
the over 200 manufacturers, and most of the 93 manufacturers have
indicated that their systems are compliant. The slow response rate
is a concern because of its potential impact on the ability of
facilities to ensure continued delivery of essential services. For
example, heating, ventilating and air conditioning equipment are
utilized by hospitals to ensure that contaminated air is confined to
a specified area of the facility, such as an isolation room or
patient ward. If computer systems used to maintain these systems
were to fail, any resulting climate fluctuations could adversely
affect patient safety.
The facility-related systems area is difficult to gauge because many
of these systems involve a combination of items made by different
manufacturers. Year 2000 compliance must be determined for each
element. Accordingly, the manufacturer must contact all of its
suppliers to determine whether each of the applicable parts of the
facility system is compliant. This process can take a long time to
complete, and VHA is concerned that it may not have sufficient lead
time to replace the product or have it made compliant. The VHA Year
2000 Project Manager informed us that VHA will continue to send
follow-up letters to nonresponding manufacturers, as well as work
with the Year 2000 Subcommittee on Facilities of the federal
government's CIO Council Committee to increase manufacturer response.
A fourth area of concern relates to the Year 2000 compliance status
of VHA's biomedical device inventory. Because VHA relies on the
manufacturers for compliance information on their products, it has
sent a series of letters to the manufacturers requesting information
on the compliance status of their devices. About 70 percent of the
manufacturers on its list of suppliers have responded with
information on their devices; but some high-profile manufacturers\22
have yet to respond. Until the remaining manufacturers respond, VHA
will not know the extent to which its current biomedical device
inventory is Year 2000 compliant and the cost associated with making
its devices compliant. Also, the Food and Drug Administration (FDA)
has sent letters to 16,000 biomedical equipment manufacturers. To
date, about 10 percent of these manufacturers have responded. The
detailed results of our review of VA's and FDA's Year 2000 biomedical
device programs will be provided in a separate report.
Lastly, although VHA medical facilities have prepared hospital
contingency plans, as required by the Joint Commission on
Accreditation of Healthcare Organizations, our review showed that
these plans do not specifically address Year 2000-related failures.
Because VHA medical facilities, like other health care facilities in
the public and private sectors, are highly dependent upon information
technology to carry on their business, Year 2000-induced failures of
one or more mission-critical systems can have a severe impact on
their ability to deliver patient care. This risk of failure is not
limited to the medical facilities' internal information systems, but
includes information and data provided by their business
partners--other federal, state, and local agencies as well as local
and international private entities--and services provided by the
public infrastructure, including power, water, transportation, and
voice and data communications companies. VHA also relies on
information provided by the manufacturers. VHA cannot, by itself,
ensure that its systems-related software applications and/or products
are Year 2000 compliant. Accordingly, it is critical that VHA
develop business continuity and contingency plans to address the
potential Year 2000 failures induced by its business partners and
infrastructure service providers, especially those manufacturers who
have not provided compliance information to VHA on their products.
VHA's Year 2000 Project Manager has acknowledged the need for Year
2000 business continuity and contingency plans. He told us that the
Year 2000 Project Office has informed VISNs and medical facilities
that they need to address Year 2000-induced failures in their
business continuity and contingency plans. Furthermore, VHA also
plans to develop a contingency planning guidebook that will assist
the medical facilities in preparing Year 2000 business continuity and
contingency plans to address all VHA systems and products that may
affect patient safety. However, he did not know when the guidebook
will be finalized and distributed to the medical facilities for
implementation.
--------------------
\19 GAO/T-AIMD-97-174, September 25, 1997.
\20 Such customization includes special purpose programs written by
local information resources management staff or other system users
on-site or exported from other VA medical facilities. These programs
generally meet a specific local need or extend the functionality of
nationally released software.
\21 These include software applications, operating systems, and
system software.
\22 A high-profile manufacturer is one whose products are (1) high
cost (more than $250,000 per unit), (2) high volume (multiple units
at each VHA medical center) or (3) used to provide critical care or
life support functions.
CONCLUSIONS
------------------------------------------------------------ Letter :6
VBA has made progress in responding to the Year 2000 computer crisis.
However, it faces risks in several areas. It has made limited
progress in making two key mission-critical applications compliant.
For example, computer analysts responsible for renovating the
compensation and pension online application have been working on
other information technology initiatives, including special projects.
VBA also has to reassess some of the COTS products that the vendor
previously stated were fully Year 2000 compliant. In addition to
these risks, except for the Insurance Service, VBA has not developed
business continuity and contingency plans for its program services to
ensure that they would continue to operate if Year 2000 failures
occur. Not adequately addressing these concerns could delay or
interrupt benefits to veterans.
VHA also has made progress on the Year 2000 problem. However, it
also faces several remaining risks because it has not completed Year
2000 assessments of software applications developed or modified by
its medical facilities. Furthermore, it still lacks a great deal of
compliance information from manufacturers that provide it with COTS
products, facility-related systems and equipment, and biomedical
devices. These uncertainties are all the more worrisome in light of
VHA's lack of Year 2000 business continuity and contingency plans.
Until these concerns are addressed, VHA lacks assurance that its
delivery of medical care to veterans will not be delayed or
interrupted by Year 2000 failures.
RECOMMENDATIONS
------------------------------------------------------------ Letter :7
To reduce the likelihood of delayed or interrupted benefits, we
recommend that the Secretary of Veterans Affairs, with support from
VBA's Chief Information Officer (CIO), ensure that VBA:
-- Reassesses its Year 2000 mission-critical efforts for the
compensation and pension online application and the Beneficiary
Identification and Record Location Sub-System, as well as other
information technology initiatives, such as special projects, to
ensure that the Year 2000 efforts have adequate resources,
including contract support, to achieve compliance in time.
-- Establishes a milestone for the contractor-developed business
continuity framework and subsequent critical dates for the
preparation of business continuity and contingency plans for
each core business process or program service so that
mission-critical functions affecting benefits delivery can be
carried out if software applications and COTS products fail.
These plans should provide a description of resources, staff
roles, procedures, and timetables needed for implementation.
We also recommend that the Secretary, with support from the VHA CIO,
ensure the rapid development of business continuity and contingency
plans for each medical facility so that mission-critical functions
affecting patient care can be carried out if software applications,
COTS products, and/or facility-related systems and equipment do not
function properly. These plans should address issues such as when to
invoke alternative solutions and/or options if the manufacturer, who
VHA depends on for compliance information, does not submit any. The
plans also should describe resources, staff roles, procedures, and
timetables needed for implementation.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :8
In commenting on a draft of this report, VA concurred with all three
of our recommendations. VA stated that it is committed to ensuring
that its benefit and health care services to veterans will not be
adversely affected by the Year 2000 problem and it has applied
dedicated resources to address these issues.
However, VA stated that although this report recognizes the progress
it has made in mitigating Year 2000 problems, VA does not believe
that statements in the report adequately reflect its efforts. VA was
concerned that statements in the report could be taken out of context
and unnecessarily alarm veterans. We believe, however, that the
report accurately reflects VA's Year 2000 efforts and our resulting
concerns. The report raises concerns surrounding the renovation of
two key VBA mission-critical applications, COTS software products,
and contingency planning, and the Year 2000 compliance status of
VHA's locally developed software applications, COTS products,
facility-related systems, and biomedical devices. Failure to address
these concerns may delay or interrupt the issuance of benefits and
the provision of medical care to veterans.
In addition, VA stated that it was concerned that the report
portrayed issues, such as COTS products, facility-related systems,
and biomedical equipment, as unique to VA. VA stated that, like any
other consumer of these products, it is dependent upon manufacturers'
disclosure of their Year 2000 compliance, and some manufacturers are
reluctant to supply compliance information on their products despite
VA's attempts to obtain it. We have revised the report to reflect
that these issues are not unique to VA.
Further, VA stated that since some manufacturers indicated that
compliance information will not be available until late 1998, it will
not spend what it termed unnecessary time developing continuity of
business plans based on unrealistic assumptions. VA also stated that
the expectation of its having compliance plans in place today is
unrealistic considering that we just issued our Business Continuity
and Contingency Planning Guide exposure draft in March 1998. We
disagree with VA on these issues. First, our February 1997 exposure
draft of the Year 2000 Assessment Guide called for agencies to
develop realistic contingency plans to ensure the continuity of their
core business processes. Second, our May 1997 report reiterates the
need for agencies, including VA, to develop contingency plans for
their major mission-critical business processes. Since our May 1997
report, only VA's Insurance Service has developed such a plan.
Third, because VA is dependent upon information from service
providers and/or manufacturers who have yet to report on the
compliance status of their services and/or products, it is critical
that VA develop business continuity plans and contingency plans to
address these services and/or products in the event that VA does not
receive the information on the date promised. Further, if service
providers and/or manufacturers provide assurance that their services
and/or products are compliant, VA still needs to develop business
continuity and contingency plans in the event that these services
and/or products do not operate or do not function properly when
processing data related to the Year 2000.
Lastly, VA described actions it has taken and planned to implement
our recommendations, as well as a number of technical suggestions to
this report. These comments have been incorporated into the report
as appropriate and are reprinted in appendix I.
---------------------------------------------------------- Letter :8.1
As agreed with your office, unless you publicly announce the contents
of this report earlier, we will not distribute it until 30 days from
its date. At that time, we will send copies to the Ranking Minority
Member of the Subcommittee on Oversight and Investigations, House
Committee on Veterans' Affairs, the Chairmen and Ranking Minority
Members of the Subcommittee on Benefits, House Committee on Veterans'
Affairs, and the Subcommittee on Health, House Committee on Veterans'
Affairs. We will also provide copies to the Chairmen and Ranking
Minority Members of the Senate and House Committees on Veterans'
Affairs and the Senate and House Committees on Appropriations, the
Secretary of Veterans Affairs, and the Director of the Office of
Management and Budget. Copies will also be made available to others
upon request.
Please contact me at (202) 512-6253 or by e-mail at
[email protected] if you have any questions concerning this
report. Major contributors to this report are listed in appendix II.
Sincerely yours,
Joel C. Willemssen
Director, Civil Agencies Information Systems
(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
VETERANS AFFAIRS
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on the Department of Veterans
Affairs' letter dated July 31, 1998.
GAO COMMENTS
1. Discussed in "Agency Comments and Our Evaluation" section of
report.
2. Our report now refers to "core business processes" as used in our
Business Continuity and Contingency Planning Guide.
3. As we stated in our report, the detailed results of our review of
VA's and FDA's Year 2000 biomedical devices will be provided in a
separate report.
4. We modified the report to reflect "limited progress" as
communicated during our briefing before the Senate Committee on
Veterans' Affairs staff.
5. Our report now refers to "limited progress" when discussing VBA's
efforts in renovating two of its key mission-critical software
applications--compensation and pension online, and the Beneficiary
Identification and Record Locator Sub-System. Regarding COTS
products, we have clarified the report to indicate that this problem
is not unique to VBA. We also replaced "announced" with "told" and
added information to point out that a particular vendor is still
assessing and testing its products for compliance.
6. We modified the report to delete the word "considerable." Our
statement that VHA has not completed its assessment and that it does
not know the full extent of the Year 2000 problem at its medical
facilities is accurate and consistent with the findings in the
report. For example, because VHA is relying on the manufacturers of
COTS products, facility-related systems, and biomedical devices to
determine the compliance status of their products, it is critical
that VHA obtain this information from the manufacturers. It should
also consider contacting the manufacturers of COTS products and
facility-related systems, who have yet to provide VHA with compliance
information, by telephone and/or meet with them. Moreover, given the
uncertainties surrounding the compliance status of its local software
applications, COTS products, facility-related systems, and biomedical
devices, VHA needs to develop business continuity and contingency
plans to ensure that core business processes can be carried out in
the event that its systems and products do not function properly on
and after January 1, 2000.
7. Report revised to reflect agency comments.
8. The report now states that our concerns relate to renovation of
two key mission-critical applications.
9. Report revised to say "application" and reflect milestones for
compensation and pension online application.
10. We have added information to clarify the role of the contractor
in renovating VBA's compensation and pension online application.
11. Report changed to reflect agency comments.
12. We added language explaining that according to VBA's Year 2000
Project Manager, one of VBA's largest vendors told VBA that some of
its products were not compliant and it was beginning to assess and
test its product line for Year 2000 compliance.
13. As stated in VA's comments to our draft report, 73 (about half)
of the 147 VISTA software applications require renovation to achieve
compliance, as would about 10 percent of the VHA corporate systems
applications.
14. We added information to explain VHA's plans to hire a contractor
to conduct an independent verification and validation of VHA's
compliance process for mission-critical systems, especially its
corporate systems.
15. Added the percentage of noncompliant applications within the
number of locally developed and/or modified VISTA applications
assessed as of May 31, 1998.
16. Report modified to include "with the exception of the Insurance
Service."
17. Report modified to include "Year 2000" in discussion of business
continuity and contingency plans.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
Helen Lew, Assistant Director
Tonia L. Johnson, Information Systems Analyst-in-Charge
J. Michael Resser, Business Process Analyst
*** End of document. ***