Medicare Computer Systems: Year 2000 Challenges Put Benefits and Services
in Jeopardy (Letter Report, 09/28/98, GAO/AIMD-98-284).
Pursuant to a congressional request, GAO reviewed the extent to which
the Health Care Financing Administration (HCFA) is: (1) making progress
in renovating its Medicare systems to be year 2000 (Y2K) compliant; (2)
directing and overseeing the Y2K effort; (3) ensuring that all data
exchanges necessary for processing Medicare claims are identified,
renovated, tested, and validated; and (4) developing and initiating
business continuity and contingency plans for key business processes.
GAO noted that: (1) HCFA and its contractors are severely behind
schedule in repairing, testing, and implementing the mission-critical
systems supporting Medicare; (2) HCFA has recently begun improving its
management of Y2K matters, including establishing a Y2K organization and
hiring independent contractors to assist in overseeing the Y2K work; (3)
in August HCFA reported that as of June 30, 1998, less than a third of
Medicare's 98 mission-critical systems had been fully renovated, and
none had been validated or implemented; (4) compounding this difficult
task is HCFA's lack of key management practices necessary to adequately
direct and monitor its Y2K project; (5) to date, HCFA has not: (a)
developed an adequate overall schedule and a critical path that
identifies and ranks Y2K tasks, and helps ensure that they can be
completed in a timely manner; (b) implemented risk management processes
necessary to highlight potential technical and managerial weaknesses
that could impair project success; and (c) planned for or scheduled
end-to-end testing to ensure that Medicare-wide renovations will work as
planned; (6) HCFA has also not been effectively managing the
identification and correction of its electronic data exchanges; (7)
neither has HCFA determined whether needed agreements with data exchange
partners have been made; (8) this increases the risk that Y2K errors
will be transferred through data exchanges from one organization's
computer systems to another's; (9) given the magnitude of the task and
risks ahead, and the limited time remaining, it is highly unlikely that
all of the Medicare systems will be compliant in time to ensure the
delivery of uninterrupted benefits and services into the year 2000; (10)
it is more critical than ever that HCFA have sound business continuity
and contingency plans in place, which can be implemented should systems
failures occur; (11) HCFA is late in establishing its business
continuity and contingency plans; (12) HCFA is relying on its Medicare
contractors to develop plans for their own systems; several contractors
told GAO they do not plan to begin developing their individual plans
until 1999; (13) also, HCFA has not yet developed a Medicare-wide
business continuity and contingency planning framework; and (14) HCFA
has only recently completed drafting a set of contingency planning
guidelines, and does not plan to have its Medicare-wide plan completed
and tested until June 20, 1999.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-98-284
TITLE: Medicare Computer Systems: Year 2000 Challenges Put
Benefits and Services in Jeopardy
DATE: 09/28/98
SUBJECT: Claims processing
Computer software verification and validation
Systems compatibility
Data integrity
Information resources management
Internal controls
Strategic information systems planning
Systems conversions
IDENTIFIER: HCFA Medicare Transaction System
Medicare Program
Y2K
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
September 1998
MEDICARE COMPUTER SYSTEMS - YEAR
2000 CHALLENGES PUT BENEFITS AND
SERVICES IN JEOPARDY
GAO/AIMD-98-284
Medicare Computer Systems
(511243)
Abbreviations
=============================================================== ABBREV
CIO -
CWF -
HCFA -
HHS -
IV&V -
MTS -
OMB -
Y2K -
Letter
=============================================================== LETTER
B-280607
September 28, 1998
The Honorable Charles E. Grassley
Chairman
The Honorable John Breaux
Ranking Minority Member
Special Committee on Aging
United States Senate
The Honorable Nancy L. Johnson
Chairwoman
Subcommittee on Oversight
The Honorable Bill Thomas
Chairman
Subcommittee on Health
Committee on Ways and Means
House of Representatives
This report responds to your request that we review the Health Care
Financing Administration's (HCFA) progress in addressing the Year
2000 issue for its Medicare claims processing systems. The Year 2000
problem, commonly referred to as Y2K, results from the inability of
computer systems to interpret the century correctly when only two
digits are used to indicate the year in recorded or calculated dates.
In many systems, the year 2000 is indistinguishable from 1900, and
those systems that support the processing of Medicare claims are no
exception. As you know, time is running out to correct Medicare
systems that could malfunction or produce incorrect information when
the year 2000 is encountered during automated data processing. If
uncorrected, this problem could result in serious disruption to
critical functions and services administered through HCFA's Medicare
program, as well as to all programs governmentwide. For this reason,
we included the Y2K computing problem in our high-risk series,
published in early 1997.\1
For a program as large and complex as Medicare--paying about $207
billion in benefits for about 39 million beneficiaries--addressing
the Y2K problem is a formidable task. The Medicare program uses 7
standard Medicare claims processing systems, over 70 private
contractors, and financial institutions nationwide to process about
800 million Medicare claims each year for about 1 million hospitals,
physicians, and medical equipment suppliers. Over 85 percent of
these Medicare claims are submitted and paid electronically, which
will require that electronic data exchanges also be assessed for Y2K
compliance.
In view of the impact this problem could have on millions of elderly
and disabled American citizens, you requested that we provide
information on HCFA's progress in addressing the Y2K issue for its
Medicare claims processing systems. During our review, we assessed
the extent to which HCFA is (1) making progress in renovating its
Medicare systems to be Y2K compliant, (2) directing and overseeing
the Y2K effort, (3) ensuring that all data exchanges necessary for
processing Medicare claims are identified, renovated, tested, and
validated, and (4) developing and initiating business continuity and
contingency plans for key business processes.
--------------------
\1 High-Risk Series: Information Management and Technology
(GAO/HR-9, February 1997).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
HCFA and its contractors are severely behind schedule in repairing,
testing, and implementing the mission-critical systems supporting
Medicare. HCFA has recently begun improving its management of Y2K
matters, including establishing a Y2K organization and hiring
independent contractors to assist in overseeing the Y2K work.
However, because of the complexity and magnitude of the problem and
HCFA's late start, its progress in repairing mission-critical
Medicare systems for the year 2000 is far behind schedule.
Specifically, in August HCFA reported that as of June 30, 1998, less
than a third of Medicare's 98 mission-critical systems had been fully
renovated, and none had been validated or implemented. The Office of
Management and Budget (OMB) has established target dates of September
30, 1998, for completion of agencies' Y2K renovations; January 31,
1999, for validation; and March 31, 1999, for implementation of
renovated and validated systems.
Compounding this difficult task is HCFA's lack of key management
practices necessary to adequately direct and monitor its Y2K project.
To date, HCFA has not
-- developed an adequate overall schedule and a critical path that
identifies and ranks Y2K tasks, and helps ensure that they can
be completed in a timely manner,
-- implemented risk management processes necessary to highlight
potential technical and managerial weaknesses that could impair
project success, and
-- planned for or scheduled end-to-end testing to ensure that
Medicare-wide renovations will work as planned.
HCFA has also not been effectively managing the identification and
correction of its electronic data exchanges. It appears to have
thousands of such exchanges, but HCFA does not know for sure because
it has not yet identified the actual number. Neither has it
determined whether needed agreements with data exchange partners have
been made. This increases the risk that Y2K errors will be
transferred through data exchanges from one organization's computer
systems to another's.
Given the magnitude of the task and risks ahead, and the limited time
remaining, it is highly unlikely that all of the Medicare systems
will be compliant in time to ensure the delivery of uninterrupted
benefits and services into the year 2000. Accordingly, it is more
critical than ever that HCFA have sound business continuity and
contingency plans in place, which can be implemented should systems
failures occur. However, HCFA is late in establishing its business
continuity and contingency plans. It is relying on its Medicare
contractors to develop plans for their own systems; several
contractors told us they do not plan to begin developing their
individual plans until 1999. Also, HCFA has not yet developed a
Medicare-wide business continuity and contingency planning framework.
It has only recently completed drafting a set of contingency planning
guidelines, and does not plan to have its Medicare-wide plan
completed and tested until June 20, 1999.
OBJECTIVES, SCOPE, AND
METHODOLOGY
------------------------------------------------------------ Letter :2
Our objectives were to determine the extent to which HCFA is (1)
making progress in renovating its Medicare systems to be Y2K
compliant, (2) directing and monitoring the Y2K program, (3) ensuring
that all data exchanges necessary for processing Medicare claims are
identified, renovated, tested, and validated, and (4) developing and
initiating business continuity and contingency plans for key business
processes.
In conducting our review, we compared HCFA's activities with the key
processes in our Year 2000 assessment and testing guides.\2 \3 The
assessment guide addresses common issues affecting most federal
agencies and presents a structured approach and a checklist to aid in
planning, managing, and evaluating Y2K programs. It describes five
phases--to be supported by program and project management
activities--with each phase representing a major Y2K program activity
or segment. The phases and a description of each follows.
-- Awareness - Define the Y2K problem and gain executive-level
support and sponsorship for it. Establish a Y2K program team
and develop an overall strategy. Ensure that everyone in the
organization is fully aware of the issue.
-- Assessment - Assess the Y2K impact on the enterprise. Identify
core business areas and processes, inventory and analyze the
systems supporting the core business areas, and prioritize their
conversion or replacement. Develop contingency plans to handle
data exchange issues, lack of data, and bad data. Identify and
secure the necessary resources.
-- Renovation - Convert, replace, or eliminate selected platforms,
applications, databases, and utilities. Modify interfaces.
-- Validation - Test, verify, and validate all converted or
replaced platforms, applications, databases, and utilities.
Test the performance, functionality, and integration of
converted or replaced platforms, applications, databases,
utilities, and interfaces in an environment that represents the
operational environment.
-- Implementation - Implement converted or replaced platforms,
applications, databases, utilities, and interfaces. Implement
data exchange contingency plans, if necessary.
The testing guide is intended to aid organizations in managing and
assessing their Y2K testing programs. It presents a Y2K test model
and sets forth five levels of test activity supported by continual
management oversight and control activities. The testing guide
phases and a description of each follows.
-- Testing infrastructure - Assign Y2K test management authority
and responsibility and define compliance criteria. Develop a
test and evaluation master plan, define and secure resources,
establish a test environment, and develop and issue test
guidance. Establish processes and information sources to
support testers, ensure Y2K compliance of vendor-supported
products and services, establish processes and metrics for test
reporting, and establish test tools.
-- Software unit testing - Schedule and plan software unit tests.
Prepare test procedures and data and define test exit criteria.
Execute tests and document test results. Correct defects and
ensure that test exit criteria are satisfied.
-- Software integration testing - Schedule and plan the software
integration test. Prepare test procedures and data and define
test exit criteria. Execute tests and document test results.
Correct defects and ensure test exit criteria are satisfied.
-- System acceptance testing - Schedule and plan system acceptance
tests, prepare test procedures and data, and define test exit
criteria. Confirm Y2K compliance of vendor-supported system
components. Execute tests, document test results, correct
defects, and ensure that test criteria are satisfied.
-- End-to-end testing - Define end-to-end test boundaries. Secure
commitments from the data exchange partners and establish an
end-to-end test team. Confirm the Y2K compliance of the
vendor-supported telecommunications infrastructure, schedule and
plan end-to-end tests, prepare test procedures and data, and
define test exit criteria. Execute tests and document test
results. Correct defects and ensure that test exit criteria are
satisfied.
We also analyzed documents which included HCFA and contractors' Y2K
plans and schedules, reports provided to HCFA by its independent
verification and validation (IV&V) contractor, and reports provided
to HCFA by its Medicare contractors and internal systems maintainers.
We interviewed agency officials, Medicare contractors, HCFA's IV&V
and independent testing contractors, and standard system
maintainers.\4 We used the resulting information to identify the
status of HCFA's renovation work and determine the extent of its
project direction and oversight. Specifically, we assessed HCFA's
actions to (1) ensure that its contractors prepare their Medicare
systems for the year 2000, (2) prepare its internal claims
processing-related systems for the year 2000, and (3) identify and
reach agreement on renovating data exchanges that support Medicare
claims.
We then assessed HCFA's business continuity and contingency planning
against our guide for such planning.\5 The guide provides agencies
with a structured approach to reducing the risk and potential impact
of Y2K-induced information systems failures on their core business
processes by implementing a rigorous business continuity planning
process. The guidance describes four phases--to be supported by
agency Y2K program management activities--with each phase
representing a major Y2K business continuity planning project,
activity, or segment. In reviewing HCFA's business continuity and
contingency plans, we compared the completeness of these plans
against our contingency planning guidance, and discussed these plans
and the time allowed to implement them with agency and contractor
officials. The phases in this guide and a description of each
follows.
-- Initiation - Establish a business continuity project workgroup
and develop a high-level business continuity planning strategy.
Develop a master schedule and milestones, and obtain executive
support.
-- Business impact analysis - Assess the potential impact of
mission-critical systems failures on an agency's core business
processes. Define Y2K failure scenarios, and perform a risk and
impact analysis for each core business process. Assess
infrastructure risks, and define the minimum acceptable levels
of outputs for each core business process.
-- Contingency planning - Identify and document contingency plans
and implementation modes. Define triggers for activating
contingency plans, and establish a business resumption team for
each core business process.
-- Testing - Validate the agency's business continuity strategy.
Develop and document contingency test plans. Prepare and
execute tests. Update disaster recovery plans and procedures.
We performed our work at HCFA headquarters in Baltimore, Maryland; at
claims processing contractors' sites and a common working file
(CWF)\6 host site in Richardson, Texas; at a standard part B system
maintainer's office in Plano, Texas; and at the CWF Y2K contractor's
office in Camp Hill, Pennsylvania. Our work was performed from
February 1998 through August 1998, in accordance with generally
accepted government auditing standards. HCFA provided comments on a
draft of this report. These comments are summarized and evaluated in
the "Agency Comments and Our Evaluation" section of this report, and
are reprinted as appendix I.
--------------------
\2 Year 2000 Computing Crisis: An Assessment Guide
(GAO/AIMD-10.1.14, September 1997).
\3 Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.2.21,
exposure draft, June 1998).
\4 Standard system maintainers are those contractors who provide
claims processing software to Medicare fiscal intermediaries and
carriers. Currently there are two part A standard systems
maintainers and five part B standard systems maintainers.
\5 Year 2000 Computing Crisis: Business Continuity and Contingency
Planning (GAO/AIMD-10.1.19, August 1998).
\6 CWF is a set of databases used by Medicare claims processing
contractors to provide beneficiary-related edit checks such as
whether Medicare premiums have been paid or whether duplicate claims
have been made to cover beneficiary conditions. Contractors use the
CWF edit checks to determine whether to approve, adjust, or deny
claims payments.
BACKGROUND
------------------------------------------------------------ Letter :3
HCFA, under the Department of Health and Human Services (HHS),
administers the Medicare program. Medicare is the nation's largest
health insurer, serving about 39 million Americans by providing
federal health insurance to individuals 65 or older and to many of
the nation's disabled. For fiscal year 1997, HCFA provided a
reported $207 billion in fee-for-service and managed care benefits,
and expects to process over 1 billion claims and pay $288 billion in
benefits annually by 2000.
MEDICARE FEE-FOR-SERVICE
CLAIMS PROCESS IS
COMPLICATED
---------------------------------------------------------- Letter :3.1
HCFA uses about 70 fiscal intermediaries and carriers to process
Medicare claims. These intermediaries are the contractors that
process part A claims (those submitted by hospitals, skilled nursing
facilities, hospices, home health agencies, and rehabilitation
agencies). Carrier contractors process part B claims (those
submitted by physicians, laboratories, durable medical equipment
suppliers, outpatient providers, and other practitioners).
In addition to the Medicare contractors, the process involves about
970,000 medical providers, numerous banks serving both contractors
and providers, the Federal Reserve System, nine CWF databases of
Medicare beneficiary information, systems maintained by HCFA that
provide information to Medicare contractors as part of their claims
processing activities, and hundreds of thousands of electronic data
exchanges that carry claims data throughout the process. Figure 1
depicts this intricate, complicated Medicare claims process. Many of
the claims processing data exchanges, such as those used with the
Medicaid program, are not shown in figure 1.
Figure 1: Overview of the
Medicare Fee for Service Claims
Process Showing the Complexity
of the Process and Data
Exchanges Supporting the
Medicare Program
(See figure in printed
edition.)
(See figure in printed
edition.)
Note: The claims process also includes a data exchange with the
Social Security Administration, which is used to determine
beneficiary eligibility.
Source: GAO, from HCFA documentation.
CLAIMS PROCESSING SYSTEMS
ARE COMPLEX
---------------------------------------------------------- Letter :3.2
The Medicare claims process involves four categories of
systems--contractors' standard external systems, the CWF, internal
HCFA systems and providers' systems. All interrelate and must be
modified and tested to ensure Y2K compliance.
The over 70 intermediaries and carriers use the standard external
systems (currently consisting of two part A and five part B systems)
to process claims that are submitted by the providers. Each
contractor's system also obtains data from the CWF and HCFA's
internal systems, and sends information to systems outside of HCFA's
control, such as those run by banks.
The CWF consists of nine databases that are processed at seven
different computer sites around the country. Each CWF database
contains beneficiary information for specific geographic regions. It
provides data to help contractors determine if claims are for
eligible individuals and for appropriate benefits. The CWF also
obtains information, such as enrollment data, from HCFA's internal
systems.
HCFA's internal systems include financial and accounting data. They
provide relevant information to the contractors and the CWF through
data exchanges. Figure 2 depicts the complexity of the Medicare
systems.
Figure 2: Overview - Structure
of Medicare Systems
(See figure in printed
edition.)
This complex system poses several challenges in achieving Y2K
compliance. For example, because the contractors modify the standard
systems they use to address local claims processing needs, each
contractor will have to renovate and test its modified system for Y2K
compliance. Also, because the standard systems, CWF, and internal
HCFA systems exchange data, they must be renovated and ready for Y2K
testing in proper sequence. Further, detailed planning and careful
project management will be required to manage the complex
relationships between almost 1 million providers and over 200,000
reported data exchanges with the contractor systems. Finally, HCFA
recently estimated that its internal and external Medicare systems
contain about 50 million lines of computer code that must be assessed
for Y2K compliance. On August 17, 1998, HHS reported that HCFA's
estimated cost for its Y2K effort will range between $917 million and
$1.3 billion.
At a July 16, 1998, hearing by the Subcommittee on Health, House Ways
and Means Committee, HCFA's Administrator said that Medicare claims
payments could be delayed if Medicare's systems are not made Y2K
compliant. According to HCFA, if these systems and data exchanges
are not renovated, providers could experience cash flow problems,
enrollment systems could malfunction, and beneficiaries could be
denied services because providers may not be able to confirm
eligibility.
ADDITIONAL Y2K WORK REQUIRED
BECAUSE OF FAILED MEDICARE
TRANSACTION SYSTEM
---------------------------------------------------------- Letter :3.3
In January 1994, HCFA entered into a contract to develop a claims
processing system that would have significantly reduced the amount of
work necessary for it to address the Medicare Y2K problem. Called
the Medicare Transaction System (MTS), the project was intended to be
a single government-owned system that would replace the existing, two
standard part A and five standard part B claims processing systems
currently being used by Medicare contractors. HCFA intended to
develop MTS as a Y2K-compliant system and have it in place and
operational before 2000. However, the MTS project encountered
problems from the very beginning. It was plagued with schedule
delays, cost overruns, and the lack of effective management and
oversight. We repeatedly reported that HCFA was not applying
effective investment management practices in its planning and
management and, as a result, had no assurance that the project would
be cost-effective, delivered within estimated time frames, or even
improve the processing of Medicare claims.\7
Given the magnitude of these problems, along with continually
increasing costs, HCFA terminated the MTS contract on August 15,
1997. The failure of MTS cost HCFA about $50 million for the
software development contract alone. While MTS provided HCFA with a
learning experience about the difficulty of acquiring such a large
system and a better understanding of the requirements for developing
a Medicare claims processing system, the project did not result in a
new, integrated, Y2K compliant claims processing system.
Because the Y2K compliant MTS was not successful, HCFA is continuing
to process Medicare claims with the two standard part A and five
standard part B claims processing systems. Consequently, additional
time and resources must now be spent to make these systems Y2K
compliant.
--------------------
\7 See Medicare Transaction System: Success Depends Upon Correcting
Critical Managerial and Technical Weaknesses (GAO/AIMD-97-78, May 16,
1997); Medicare Transaction System: Serious Managerial and Technical
Weaknesses Threaten Modernization (GAO/T-AIMD-97-91, May 16, 1997);
Medicare Transaction System: Strengthened Management and Sound
Development Approach Critical to Success (GAO/T-AIMD-96-12, November
16, 1995); and Medicare: New Claims Processing System Benefits and
Acquisition Risks (GAO/HEHS/AIMD-94-79, January 25, 1994).
HCFA HAS MOBILIZED FOR ACTION,
BUT ITS Y2K EFFORT IS SEVERELY
BEHIND SCHEDULE
------------------------------------------------------------ Letter :4
In our May 1997 report on MTS, we stated that unless timely and
effective Y2K changes are implemented, HCFA may be unable to process
Medicare claims.\8 We identified serious problems with HCFA's
oversight of its Medicare contractors' Y2K remediation efforts, as
well as problems with its own Y2K activities. For example, HCFA had
not planned to establish legal agreements with its contractors
specifying how or when the Y2K problem would be corrected, had no
plans to independently validate contractors' strategies and test
plans, had not approved their approaches for addressing data exchange
issues, and had not developed contingency plans in the event that the
Medicare systems fail.
Our May 1997 report made several recommendations to HCFA to improve
its contractors' Y2K remediation efforts. We recommended that HCFA
require its contractors to submit for review and approval their plans
for identifying and correcting potential Y2K problems, including a
certification that their planned changes would correct the problems,
as well as their plans for Y2K testing, and addressing interface and
data exchange issues. We also recommended that HCFA develop
contingency plans in the event of systems failures. To improve its
internal Y2K program, we further recommended that HCFA take action to
identify responsibilities for managing and monitoring the Y2K
project.
--------------------
\8 GAO/AIMD-97-78, May 16, 1997.
HCFA HAS TAKEN STEPS THAT
BETTER POSITION IT TO
PREPARE FOR 2000
---------------------------------------------------------- Letter :4.1
HCFA has taken several steps to respond to our recommendations.
First, HCFA prepared a contract amendment to help ensure that its
contractors make appropriate, timely Y2K remediations. This
amendment directed its contractors to develop and submit for review
their Y2K project and test plans, to use their best efforts to make
their systems Y2K compliant, and to certify that their
mission-critical systems would be millennium compliant no later than
December 31, 1998.
Further, in September 1997 HCFA awarded independent verification and
validation and testing contracts for its Y2K program. These
contractors are (1) assessing HCFA's process for certifying
millennium compliance, (2) assessing Medicare contractors' Y2K
project plans, test plans, test results, and methodologies, (3)
developing acceptance test plans, test cases, scenarios and
specifications, and (4) conducting acceptance testing of standard
part A and part B external systems. HCFA headquarters, regional
offices, and its IV&V contractor began conducting oversight visits to
claims processing contractors' locations in October 1997; those
visits are continuing.
HCFA also asked its contractors and internal systems maintainers to
complete agreements with all of its data exchange partners by May 1,
1998, and provided suggested wording for these agreements.\9 Further,
HCFA asked its contractors to develop and implement contingency plans
by October 1, 1998, showing their planned courses of action in event
of systems failures. It also revised its Medicare Carriers Manual to
require contractors to develop these plans.
In response to our recommendations to improve its internal Y2K
program, HCFA's administrator hired a chief information officer (CIO)
and directed this official to make Y2K his top priority. As part of
that effort, the CIO established a Y2K organization that includes
groups devoted to overseeing Medicare contractors' Y2K activities,
renovating HCFA's internal systems, and developing contingency plans.
Further, HCFA has issued surveys to all contractors to gather
additional information on their systems' status.
--------------------
\9 Internal systems maintainers are HCFA staff who are responsible
for the Y2K compliance of HCFA's internal systems.
HCFA'S Y2K WORK IS SEVERELY
BEHIND SCHEDULE
---------------------------------------------------------- Letter :4.2
Despite its actions to improve the direction and oversight of the Y2K
effort, HCFA's Y2K progress is significantly behind schedule. OMB is
also concerned about the status of HCFA's effort and, as a result,
has placed HHS on its list of agencies that are not making adequate
progress. HCFA's August 15, 1998, report to HHS showed that as of
June 30, 1998,
-- renovations that, according to OMB guidelines, should be
completed by September 1998 are only 16 percent complete for
HCFA's contractors, and 64 percent complete for its internal
systems, and
-- no systems have been validated as Y2K compliant, or implemented.
Table 1 summarizes HCFA's reported status of its Y2K effort.
Table 1
Reported Status of Medicare Mission-
Critical Systems (as of June 30, 1998)
External systems Internal systems
(73) (25)
-------------------- --------------------
Percentag Percentag
Phase completed Number e Number e
-------------------------- --------- --------- --------- ---------
Assessment 71\a 97 25 100
Renovation 12 16 16 64
Validation 0 0 0 0
Implementation 0 0 0 0
----------------------------------------------------------------------
\a HCFA reported that 2 of the 73 external mission-critical systems
will be phased out before 2000 and thus will not be assessed or
renovated.
Source: HCFA's August 15, 1998, quarterly report to HHS. We did not
independently verify this information.
HCFA's latest quarterly report to HHS, covering the period ending
June 30, 1998, reflects that fewer systems have been validated and
implemented than earlier reports. As shown in figures 3 and 4, HCFA
reported in November 1997 that four of its external systems and two
of its internal systems were implemented. By August 1998, HCFA had
revised its report to show that none of its systems had been
implemented. Since its November 1997 report, the data also show that
the numbers of validated and implemented external and internal
systems had declined.
Figure 3: Trend in HCFA
Reporting Progress of External
Systems
(See figure in printed
edition.)
Source: HCFA Quarterly Reports to HHS. We did not independently
verify this information.
Figure 4: Trend in HCFA
Reporting Progress of Internal
Systems
(See figure in printed
edition.)
Source: HCFA Quarterly Reports to HHS. We did not independently
verify this information.
HCFA provided two reasons for this change. Officials explained that
since its November 1997 report to HHS, the department has
strengthened its definition of Y2K compliance. For example, systems
that Medicare contractors reported as compliant are now not
considered compliant unless HCFA's IV&V contractor tests and
certifies that status. HCFA also explained that since its IV&V
contractor began making site visits, its Medicare contractors and
internal systems maintainers have a better understanding of the
renovation work required and are submitting more realistic reports.
For example, after one contractor completed its renovation work and
preliminary testing, and put the system into production, the IV&V
contractor conducted a 2-month evaluation of 4.6 percent of the
system's renovated code and found that 83 (14 percent) of 581
two-digit years had not been corrected. IV&V determined that the
renovation quality was low and the system was not ready for testing
until the errors were corrected. As of August 19, 1998, 2 months
after the independent test was done, the Medicare contractor still
had not completed the required follow-up renovation and testing.
Even though the status information in the most recent quarterly
reports to HHS may be more realistic, it is not being verified for
accuracy. HCFA officials acknowledge that the quality of the data
reported by Medicare contractors and internal systems maintainers is
an important issue. HCFA officials also told us that the IV&V
contractor does follow up on any unusual data reported by Medicare
contractors or internal systems maintainers and may schedule an
additional site visit or meeting to review the data reported. In
addition, the CIO told us that he and the HCFA administrator, to
reiterate the urgency of the situation and of the need to meet their
Y2K deadlines, have personally contacted several Medicare contractors
that have reported significant schedule problems. However, HCFA
officials said that, because of the demands of other Y2K-related
work, HCFA and its IV&V contractor do not have time to follow up on
most of the Medicare contractors' reports to verify the information
provided.
On September 1, 1998, HCFA gave us updated information on the status
of their work and stated that it had made significant progress since
its latest quarterly report to HHS, which provided the status as of
June 30, 1998. The updated data showed that HCFA had identified 9
additional external mission-critical systems, bringing its total from
73 to 82. It also showed that all 82 (100 percent) of these external
systems had completed assessment and that 30 (37 percent) had
completed renovation. The data also showed that 22 (88 percent) of
the 25 internal systems had completed renovation and that 3 (12
percent) had completed validation. However, no external systems had
completed validation and none of the external or internal systems had
completed implementation. The data had not yet been reported in
HCFA's quarterly reports to HHS, nor did we verify it.
DIRECTION AND OVERSIGHT COULD
BE IMPROVED
------------------------------------------------------------ Letter :5
HCFA is not fully implementing key practices to effectively direct
and oversee its Y2K program, as recommended in our assessment
guide.\10 Specifically, HCFA has not (1) developed an adequate
project schedule and a critical path--to ensure that all project
activities are completed in appropriate time frames and to assess the
project's progress,\11 and (2) implemented its own risk management
process--to surface potential technical and managerial weaknesses
that could impair project success. Without these program management
processes and tools in place, HCFA is increasing its risk of not
completing its Y2K work in time to ensure uninterrupted Medicare
claims processing beyond December 31, 1999.
Further, HCFA has not yet developed a plan and a schedule for
critical end-to-end testing. The purpose of end-to-end testing is to
verify that interrelated systems, which support an organization's
core business functions, interoperate as intended in a production
environment.\12 In planning an end-to-end test, it is critical to
analyze the organization's core business functions, the
interrelationships among systems supporting those functions, and
potential risk exposure due to date-induced system failures of any
system in the chain of support. However, HCFA has not yet completed
such plans.
--------------------
\10 GAO/AIMD-10.1.14, September 1997.
\11 According to Joseph G. Monks, in Theory and Problems of
Operations Management, 2nd Edition, McGraw-Hill, 1996, a project's
critical path is determined by (1) identifying and defining the
activities and tasks required to complete the project, (2)
identifying the relationships among all of those activities and
tasks, (3) developing an estimate of the time required to complete
each activity or task, (4) computing the time requirement for each
possible sequence of tasks required to complete the entire project,
and (5) designating the path with the longest estimated time as the
critical path.
\12 These interrelated systems include not only those owned and
managed by the organization, but also the external systems with which
they interface. For example, agencies that administer key federal
benefits programs, such as the Department of Veterans Affairs,
exchange data with the Department of the Treasury which, in turn,
interfaces with various financial institutions to ensure that
benefits checks are issued. Consequently, end-to-end testing of the
federal benefits payment function would include systems for all
entities involved, as well as their supporting telecommunications
infrastructures.
HCFA LACKS AN ADEQUATE
OVERALL Y2K PROGRAM SCHEDULE
AND CRITICAL PATH FOR ITS
MEDICARE MISSION-CRITICAL
SYSTEMS
---------------------------------------------------------- Letter :5.1
HCFA does not yet have an adequate overall schedule showing how all
Y2K tasks are interrelated and prioritized, or a critical path to
establish the sequence in which tasks must be completed to ensure
that this complex project can be finished on time. Without a
complete, overall project schedule, HCFA cannot effectively
prioritize its remaining work to accomplish the most within the time
remaining. Without a critical path, it cannot judge the likelihood
of completing its most critical remediation efforts before the year
2000.
HCFA's external system schedule is the most comprehensive Y2K
schedule that it has developed to date. However, it cannot be
considered an adequate overall project schedule because it lacks many
external systems' Y2K project tasks and internal, mission-critical
systems' remediation activities. For example, the schedule includes
start and end dates for some of the contractor's remediation efforts
as well as tasks related to testing the renovated CWF and installing
it at two CWF test sites. However, it does not include tasks
associated with testing HCFA's contractors' systems using the two CWF
test sites even though HCFA's contractors have reported that testing
with the CWF is a critical step in the validation of their claims
processing systems. Other key Y2K tasks not included in HCFA's
external systems schedule include planning and conducting system
certification testing for all systems, testing end-to-end, renovating
data exchanges, and developing and implementing business continuity
and contingency plans.
HCFA has also not yet developed a Y2K project schedule to control its
internal systems work, or incorporated internal systems' tasks into
its more comprehensive external schedule. Instead, HCFA monitors the
progress and status of its internal systems with tracking reports,
but these reports do not document all of the tasks necessary to
complete internal systems Y2K work. For example, one undocumented
critical task is the need to test several of the internal systems
with the CWF. According to HCFA, several of these systems must be
tested against the CWF before they can be certified as compliant.
However, the task and schedule details of testing these internal
systems against the CWF are not included in HCFA's tracking reports.
At the conclusion of our review, HCFA assured us that it is gathering
and validating data for an external project schedule, planned for
mid-September 1998, and also plans to have an internal systems
project plan in place by September 30, 1998. However, its plans to
develop separate external and internal systems' schedules will not
provide HCFA with the overall project information it needs to
adequately prioritize its remaining Y2K work. In addition, separate
external and internal project schedules will not enable HCFA to
develop a greatly needed critical path for its Y2K work.
A critical path based on an overall Y2K project schedule is important
because it shows the total time necessary to complete all key tasks
of a project, and it helps ensure that these tasks are addressed in
proper sequence and in time to be available for later critical tasks
that must rely on their being completed. A Y2K critical path would
help HCFA more effectively oversee and monitor the project's
progress, and better estimate program completion dates. Two examples
illustrate the value of such a critical path.
First, HCFA is not including time to conduct all key tasks of the
project, such as a complete system testing program and end-to-end
testing. According to HCFA and contractor officials, each contractor
will need 60 to 90 days to test its system with the CWF to certify
Y2K compliance. HCFA has scheduled time for contractors to use a
test version of the CWF starting in September 1998. However, HCFA
has not included in the schedule the 60 to 90 days required for
testing systems with the Y2K compliant production version of the CWF,
scheduled to start January 4, 1999. If HCFA had performed a critical
path analysis incorporating the Y2K compliant production version of
the CWF, and included reasonable amounts of time for testing, this
management tool would have shown that the Medicare Y2K project could
not be completed before March 1999 at the very earliest. Further,
none of HCFA's schedules include time to conduct end-to-end testing.
Including this as a critical task in the overall schedule would show
that even more time will be required to complete the project testing.
Second, HCFA has not ensured that all critical tasks will be
completed in a timely sequence. For example, 71 of the 73 Medicare
contractors have planned to complete individual system renovations
before the Y2K-compliant production version of the CWF is available
for contractor testing. According to current schedules, these
contractors will have to wait an average of 114 days to begin testing
against this CWF. HCFA recently identified this problem and acted to
speed CWF renovations so that a test version would be available for
contractor testing at an earlier date. HCFA officials assured us
that they are taking steps to minimize the number and complexity of
changes to the CWF that would require that Medicare contractors
extensively retest their systems when the production version of the
CWF is available. However, the production version of the CWF still
will not be available for contractor testing before January 4, 1999.
Early development of a critical path would have highlighted this task
dependency, allowing HCFA to address it in a more timely manner and
allowing contractors to better plan for their Y2K renovation,
testing, and certification work.
HCFA LACKS KEY RISK
MANAGEMENT PROCESS NECESSARY
FOR ENSURING THAT Y2K
PROGRAM WEAKNESSES ARE
ADDRESSED
---------------------------------------------------------- Letter :5.2
HCFA has no risk management process which enables it to track all
identified risks and ensure that they are mitigated. HCFA does not
track all risks that have been identified and allows risks it does
track to be prematurely closed. Also, a planned HCFA tracking system
will likely be no better than the IV&V system it currently relies on.
HCFA is not appropriately tracking and mitigating the risks that have
been identified. Of the risks reported by the IV&V contractor, HCFA
is only tracking those risks for which the IV&V contractor made
accompanying recommendations for mitigation. For example, during an
April 1998 site visit, the IV&V contractor identified 17 issues that
required attention by HCFA or its claims processing contractor, and
made recommendations for mitigating 3 of them. Although 17 risks
were identified as requiring attention, HCFA is only tracking the 3
risks that contained accompanying recommendations. One of the 17
risks not being tracked addressed the need to ensure that data
exchange software provided by the Medicare contractor to its provider
community had not been adequately addressed. If this software is
left unaddressed, it could affect about 22,000 data exchange
partners.
Also, HCFA is allowing the risks it is tracking to be prematurely
closed. The IV&V contractor who tracks and reports risks to HCFA
closes them as soon as a mitigation plan is established, rather than
when the risk is actually mitigated. For example, the IV&V
contractor had identified a risk associated with the potential
unavailability of sites for external systems acceptance testing along
with related issues associated with conducting the test, and using
configuration management and Y2K-specific test tools. The IV&V
contractor closed the risk and discontinued tracking it as soon as
HCFA developed a potential test site list, even though the other
issues had not been addressed.
HCFA is developing its own recommendation-tracking database.
However, according to recent plans it will be no better than the
reports from its IV&V contractor, which it now uses. HCFA's planned
database will only track recommendations made and reported by the
IV&V contractor, and will not function as a risk tracking system.
For example, it will not include (1) those risks for which HCFA's
IV&V contractor has not provided a recommended solution, (2)
established time frames for mitigating risks, or (3) a course of
action to mitigate individual risks.
HCFA officials told us they need their own recommendation-tracking
database because they do not have direct access to the IV&V tracking
system, only to the reports developed from it. While having its own
recommendation-tracking database or complete access to the IV&V
contractor's system would provide HCFA with more information than it
currently receives, it would not be adequate as long as the IV&V
contractor continues to close risks prematurely. HCFA said it does
not plan to develop a more comprehensive risk management system
because it is relying on its IV&V contractor to track risks
associated with its Y2K program, and because it does not want to slow
down its Y2K work by devoting resources to building a new risk
management system. On September 1, 1998, HCFA officials told us that
they are evaluating a commercial tool designed to help organizations
formalize their risk management activities. Without a more
comprehensive risk tracking system than the one that HCFA currently
uses, it cannot establish a comprehensive risk management process to
ensure adequate and timely mitigation of Y2K program risks.
HCFA LACKS AN ADEQUATE
END-TO-END TESTING PLAN FOR
ITS MEDICARE CLAIMS
PROCESSING SYSTEMS
---------------------------------------------------------- Letter :5.3
Our Year 2000 testing guide states that a Y2K testing program may
require between 50 and 70 percent of a project's time and
resources.\13 Because year 2000 conversions often involve numerous
and large interconnecting systems with many external interfaces,
testing should be approached in a structured and disciplined fashion.
First, an organizational infrastructure for testing should be
established. Second, software should be tested as units to isolate
Y2K errors. Then it should undergo software integration testing to
assure that subsystems work together properly. Third, a system
acceptance test should be carried out to ensure that the complete
system operates correctly in the future date environment of the
computer. After completing and passing all these tests, a major
system such as that used by Medicare needs to undergo an end-to-end
test that would include HCFA's systems, the part A and part B
systems, the CWF, the provider community, banks, and financial
institutions.
Our testing guide recommends that end-to-end testing be conducted
when one major system in a chain of systems is modified or replaced.
The Medicare system clearly fits that criterion. The purpose of
end-to-end testing is to verify that a defined set of interrelated
systems, which collectively support an organizational core business
area or function, interoperates as intended. HCFA's administrator
stated that she will not be assured that Medicare claims processing
systems have been renovated correctly until an end-to-end test has
been performed.
HCFA has not yet scheduled an end-to-end test which will require
coordinating over 100 separate external and internal Y2K projects to
ensure that all will be completed in time to conduct the test. A
group to define and plan an end-to-end test for the Medicare claims
processing systems was established in March 1998. On September 1,
1998, HCFA was continuing to define the parameters of an end-to-end
test. HCFA officials told us that they plan to require each Medicare
contractor to test its Medicare claims process, including testing the
receiving and processing of provider claims and the data exchanges
between the contractor and the provider's bank. In addition, HCFA
plans to use its independent testing contractor to test each of the
seven standard systems at a single contractor site.
However, HCFA has not yet established system boundaries for an
end-to-end test, obtained commitments from key data exchange partners
that will participate in the test, developed procedures and data for
the test, and developed a plan to execute the test. Also, although
HCFA officials told us that they have obtained a Y2K compliance
certification from their telecommunications vendor, they have not yet
confirmed that all telecommunications links to contractors and
providers that are necessary for such a test are Year 2000 compliant.
Little time remains for HCFA to plan and implement an end-to-end
test. As discussed earlier in this report, certification testing on
the individual contractors' systems cannot be completed, at the
earliest, before March 1999. Also, HCFA has to identify Y2K
certified participants for the test from among almost 1 million
providers, 73 Medicare contractors, 25 internal system maintainers,
and financial institutions and reach agreement on a date to conduct
the test.
--------------------
\13 GAO/AIMD-10.2.21, exposure draft, June 1998.
HCFA LACKS AN ACCURATE
INVENTORY AND ASSESSMENT OF ITS
MEDICARE DATA EXCHANGES
------------------------------------------------------------ Letter :6
HCFA's internal and external systems must exchange data between
themselves as well as with the CWF, other federal agencies, banks,
and providers. As a result, it is essential that HCFA ensure that
Y2K related errors will not be introduced into the Medicare program
through these data exchanges. Our assessment guide recommends that
data exchange issues be identified early in the Y2K process, during
the assessment phase.\14 HCFA has reported that over 200,000 data
exchanges exist, and required its contractors and internal systems
maintainers to sign data exchange agreements with exchange partners
by May 1, 1998; agreeing to the format that will be used to
consistently exchange data. As of September 1, 1998, HCFA had (1) no
assurance that all of its data exchanges related to its external or
internal systems had been identified, (2) not ensured that the
required data exchange agreements had been signed, and (3) increased
the difficulty of ensuring that exchanged data is Y2K compliant by
not requiring all contractors to use a consistent method for making
temporary modifications to their systems (called the windowing
conversion feature).
In July 1998, HCFA reported that over 200,000 data exchanges were
involved. It has not verified this data and has no assurance that
this number is accurate. Contractors, who report monthly on the
status of their data exchange work have reported a wide disparity in
the number of data exchanges, which indicates that some reports may
be inaccurate. For example, 13 part B contractors reported that they
had no data exchanges, yet many others reported that they had 1,000
or more; one even reported that it had about 50,000 data exchanges.
HCFA officials told us that the disparity in the number of reported
data exchanges resulted from differing interpretations of the
guidance provided by HCFA. They also said that due to the demands of
other Y2K work, they have not had time to verify the data exchange
information reported by the contractors or internal systems
maintainers. Recognizing that their data exchange reports have been
inconsistent, HCFA officials told us that they have directed their
staff to ensure that the quality of data exchange data that is
received from its Medicare contractors and internal systems
maintainers as well as the quality of reports that are provided to
HHS is improved.
HCFA is also not ensuring that the data exchange agreements are being
signed on schedule. HCFA required its external systems contractors
and internal systems maintainers to obtain signed data exchange
agreements by May 1, 1998. It relies on its IV&V contractor to
determine if data exchange agreements involving both external and
internal systems had been signed. However, as of July 1998, the IV&V
contractor's monthly reports made no mention about the status of data
exchange agreements. In August, HCFA officials told us that they had
received 54 signed data exchange agreements but had not yet begun to
review them, or to follow-up with the internal systems maintainers
who did not provide the signed copies as requested. At the
conclusion of our review, HCFA's CIO told us that HCFA is preparing
to conduct a more rigorous inventory of its Medicare contractors'
data exchange agreements.
Finally, HCFA is not requiring contractors and internal systems
maintainers to use a consistent pivot year when making temporary
modifications to their systems using the Y2K windowing conversion
feature. Windowing is used to make temporary corrections to the Y2K
problem and provide a "window of time" for permanent solutions. With
windowing, the computer is programmed to convert two-digit years into
four-digit years. This is accomplished by designating a pivot year,
such as 1917. The computer is then programmed to place a 20 in front
of two-digit years less than 17 (as in 2016) and a 19 in front of any
years equal to or greater than 17 (as in 1917).
This practice of using multiple pivot years will increase the
potential for errors in data exchanges and require additional
management oversight to avoid future systems' failures. When date
information is exchanged between systems that have been windowed with
different pivot years, the same date may be interpreted differently
by the systems. For example, consider two systems where one manager
chooses 17 as the pivot year and the other manager chooses 64 as the
pivot year. The system with 17 as the pivot year is programmed to
treat 50 as 1950, but the system with 64 as the pivot year is
programmed to convert 50 to 2050. In this example, such
misinterpretation could occur for any two-digit year that lies
between 17 and 63. HCFA officials told us that they would have
preferred to use a standard pivot year for all Medicare data exchange
partners, but instead are developing bridging software to accommodate
inconsistent use of two- and four-digit date formats. Had a single
pivot year been established and used by all Medicare data exchange
partners, these potential problems and additional efforts could have
been avoided. Further, the use of multiple pivot years will require
HCFA to manage additional, future renovations to ensure that they are
adequately planned and performed to avoid future systems failures.
--------------------
\14 GAO/AIMD-10.1.14, September 1997.
Y2K BUSINESS CONTINUITY AND
CONTINGENCY PLANNING NOT DONE
------------------------------------------------------------ Letter :7
Given the magnitude of HCFA's Y2K problem and its slow progress in
addressing this issue, it is highly unlikely that all Medicare
systems will be compliant by the year 2000. Thus, contingency plans
to ensure continuity of critical operations and business processes
are critical.
Our assessment guide recommends that agencies perform risk
assessments and begin developing realistic contingency plans during
the assessment phase. Business continuity and contingency plans are
vital because they identify the fall-back procedures to be employed
should systems miss their Y2K deadline or fail unexpectedly in
operation. These plans also define the specific conditions that will
cause their activation.
To ensure uninterrupted Medicare claims processing beyond 1999, it is
imperative that HCFA prepare and test business continuity and
contingency plans. However, HCFA has only recently begun
establishing guidelines for such plans, is relying on its contractors
to develop reliable plans for the external systems, and has yet to
develop a comprehensive business continuity and contingency plan for
the full Medicare program. HCFA completed a draft set of guidelines
for contingency planning in June 1998, and officials told us that
they expect an initial set of contingency plans will be completed by
the end of this calendar year. HCFA's guidelines, currently under
development, are primarily for internal systems' maintainers, and are
only suggested as guidance for contractors' external systems.
On July 10, 1998, HCFA requested that its seven standard system
maintainers provide copies of their contingency plans for review by
HCFA, and as of September 1, 1998, all but one had responded. HCFA
officials told us it is currently reviewing these plans to ensure
that they are adequate. HCFA's CIO told us that HCFA will require
the contractors that use these standard systems to adopt HCFA's
Medicare-wide contingency plan to ensure continuity of claims
payments, but will not require these contractors to submit
contingency plans for approval. HCFA told us it is relying on its
IV&V contractor to review these plans during its scheduled site
visits.
Several contractors told us they have not yet scheduled detailed
contingency planning activities as part of their efforts. They said
that their systems remediation tasks alone are so overwhelming that
they do not anticipate beginning to prepare and test their business
continuity and contingency plans until 1999.
Finally, HCFA does not intend to have its Medicare-wide claims
processing contingency plan developed and tested until June 20, 1999,
thus potentially leaving insufficient time for implementation. For
example, HCFA may include in its business continuity and contingency
plan an approach to transfer the workload of any contractor not Y2K
compliant by July 1, 1999, to a contractor certified as Y2K
compliant. However, both contractor and HCFA officials told us that,
at a minimum, it requires 6 months to a year to transfer the claims
processing workload from one contractor to another. At its current
rate of plan development, HCFA has no assurance that its plans will
be developed and tested before they may be needed on January 1, 2000.
CONCLUSIONS
------------------------------------------------------------ Letter :8
The size and complexity of Medicare systems and processes poses
significant challenges to HCFA as it works to prepare its Medicare
claims processing systems for the year 2000. Correcting the Y2K
problem is crucial if HCFA is to maintain its current level of
support for Medicare beneficiaries and providers. HCFA's leadership
has been working hard to catch up for its very late start, and has
taken steps to address the problem and prepare its Medicare claims
processing systems for the change of century.
HCFA, however, is severely behind schedule in repairing its
mission-critical Medicare systems for the year 2000; as of September
1, 1998, only 37 percent of its external systems had been reported as
being fully renovated and none of these systems has been validated or
implemented. Further, HCFA lacks several critical project management
practices and tools that could help ensure the success of its Y2K
efforts. Without an integrated Y2K project schedule, a critical
path, and an adequate risk mitigation process, HCFA risks inadequate
oversight of the Medicare systems Y2K programs and progress.
Further, without planning for and conducting end-to-end testing of
its Medicare claims processing systems, HCFA will lack adequate
assurance that it can avoid Y2K induced disruptions. Likewise, HCFA
has no assurance that all data exchanges critical to the functioning
of the Medicare claims process have been identified and will be
renovated and tested for Y2K compliance. Also, without identifying
and tracking the multiple pivot years used throughout its Medicare
systems, HCFA will not be able to adequately identify and manage them
to prevent future systems' malfunctions resulting from pivot year
failures.
In addition, the activities associated with project scheduling, using
a critical path, conducting end-to-end testing, renovating data
exchanges, and contingency planning are interrelated. Without
prioritizing its remaining work with an overall schedule and a
critical path in place, and without first identifying, renovating,
and testing its mission-critical data exchanges, HCFA cannot ensure
that adequate time for end-to-end testing of its mission-critical
systems remains. Similarly, it is critical that HCFA ensure that all
of these Y2K project activities are addressed so that it can more
effectively focus its efforts on the most mission-critical priorities
and problems identified during these activities, as well as to better
direct its contingency planning efforts.
Until HCFA supports its Y2K remediation efforts with key management
practices that will help it adequately direct and oversee its Y2K
program, Medicare benefits and services remain vulnerable to severe
disruption as a result of the Y2K problem.
RECOMMENDATIONS
------------------------------------------------------------ Letter :9
To minimize disruption of Medicare benefits and services, we
recommend that the Administrator of HCFA take the following actions.
-- Rank the remaining Y2K work on the basis of an integrated
project schedule that includes milestones for the renovation and
testing of all (1) Medicare contractors systems, (2) internal
mission-critical systems, (3) mission-critical data exchanges,
and (4) the CWF. This schedule should include time to conduct
end-to-end testing of the Y2K-compliant Medicare claims process,
and incorporate time frames to develop, implement, and test
business continuity and contingency plans.
-- Identify the critical path for the Y2K program on the basis of a
complete and integrated Y2K project schedule and use it to (1)
ensure that all critical tasks are prioritized and completed in
time to prevent unnecessary delays and (2) report a more
realistic completion date.
-- Define the scope of an end-to-end test of the Medicare claims
process, and develop plans and a schedule for conducting such a
test. This work should include developing testing procedures
and a plan for executing the test, obtaining commitments from
participating data exchange partners, confirming that
telecommunications infrastructures are Y2K compliant, and
reaching an agreement on dates for conducting the test.
-- Develop a risk management process that identifies all risks and
their interdependencies, assesses their impact on the Y2K
program, establishes time frames for mitigation and criteria for
determining when risks should be considered mitigated, and
follows this criteria to ensure that risks are indeed fully
mitigated.
-- Ensure that all external and internal systems' data exchanges
have been identified, and agreements signed between the data
exchange partners. Also, enhance management control over data
exchanges by developing a pivot year tracking system that
identifies each pivot year used, the expected date when the
pivot year will fail to function, and the contractor or
maintainer responsible for the system.
-- Accelerate the development of business continuity and
contingency plans for the Medicare program to allow time to
ensure that they are reliable and ready when they may be needed.
-- Ensure that HCFA is adequately assessing the scope of the
remaining Y2K work by independently verifying the reported
status of its claims processing contractors systems, internal
systems, and data exchanges--either internally or through its
IV&V contractor. All reports submitted to HHS should be
included as part of this independent verification.
AGENCY COMMENTS AND OUR
EVALUATION
----------------------------------------------------------- Letter :10
In her comments, the HCFA administrator said that we had raised
significant issues that were of concern and appreciated the
information and constructive suggestions contained in our report.
The administrator also said she would take immediate steps to address
our recommendations, and that, "HCFA will do whatever it takes, and
devote whatever resources necessary, to ensure there is no
interruption of services and claims payments for our thirty-eight
million beneficiaries and one million providers." Further, she
outlined a number of actions being taken to address our
recommendations, including preparing an integrated critical path,
improving the quality and completeness of its data exchange
information, and verifying the accuracy of its reported Y2K status.
While agreeing with our recommendations, and acknowledging the need
to intensify its efforts, the administrator stated that we did not
take into account actions completed or already underway on several
issues. She expressed her belief that HCFA is making significant
progress toward Y2K compliance. She pointed out that the reports
HCFA provided to OMB do not reflect renovations that are almost
completed, may contain information that is months old, and do not
reflect substantial steps that HCFA has taken over the past year.
Among the steps cited by the administrator were hiring independent
experts to provide assessments of progress; rehiring retired federal
programmers to assist in the Y2K effort; working within the
Administration and with the Congress to increase funding for Y2K
renovation activities; and establishing a Y2K organization to better
manage and oversee the Y2K effort. The administrator stated that
according to HCFA's current data, it is not severely behind schedule,
and that using multiple pivot years is not a problem and will not
cause an increased amount of oversight.
We disagree. The additional information that was provided to support
these comments does not change our position. Specifically, the
administrator commented that 85 percent of HCFA's external systems
renovations are now complete, based on lines of computer code,
subroutines, and similar measures. However, not one additional
external system has completed renovation beyond the 12 we reported,
nor has HCFA addressed the status of the nine additional mission
critical external systems it identified in its September 1, 1998,
update. Also, we stressed that use of multiple pivot years will
increase the potential for errors in data exchanges and require
additional management oversight. While using multiple pivot years is
not an insurmountable problem, it will require additional management
oversight that otherwise would not have been necessary.
As we concluded in our report, HCFA will substantially reduce the
risk to its Y2K effort by implementing all of our recommendations.
HCFA's comments in their entirety and our detailed evaluation of them
are in appendix I.
--------------------------------------------------------- Letter :10.1
As agreed with your offices, unless you publicly announce the
contents of this report earlier, we will not distribute it until 30
days from the date of this letter. At that time, we will provide
copies to the Secretary of Health and Human Services, the
Administrator of the Health Care Financing Administration, the
Director of the Office of Management and Budget, appropriate
congressional committees, and other interested parties. Copies will
also be made available to others upon request.
Please contact me at (202) 512-6253 or by e-mail at
[email protected], or Senior Assistant Director, Mark Heatwole
at (202) 512-6203 or by e-mail at [email protected] if you have
any questions concerning this report. Major contributors to this
report are listed in appendix II.
Joel C. Willemssen
Director, Civil Agencies Information Systems
(See figure in printed edition.)Appendix I
COMMENTS FROM THE HEALTH CARE
FINANCING ADMINISTRATION
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on the Health Care Financing
Administration's September 18, 1998, letter responding to a draft of
this report.
GAO COMMENTS
----------------------------------------------------------- Letter :11
1. We do not agree that the additional data HCFA provided supports
its position that the Y2K work is not severely behind schedule.
After several years of effort, according to HCFA's September 1, 1998,
update, only 37 percent of its external systems and 88 percent of its
internal systems had completed Y2K renovation. In its September 18,
1998, comments, HCFA did not (1) report progress in actually
completing the renovation of any external systems and (2) provide the
renovation status of the nine additional mission critical external
systems that it identified. Accordingly, it will be difficult for
HCFA to achieve OMB's September 30, 1998, target for completing Y2K
renovations.
2. HCFA noted that it agrees with the technical accuracy of our
report regarding the IV&V contractor's test of a system that had been
renovated and tested by its maintainer. It implied, however, that
the number of identified errors was low and that addressing this
issue 18 months before the year 2000 is evidence that the process is
working. We disagree. By assuming that the error rate of this
renovation effort is low compared to an average error rate found on
the Internet, HCFA is not taking into consideration that the error
rate of this renovation was determined from a partial test, and that
the IV&V contractor determined that the system was not ready for IV&V
testing. Further, the Internet average cited does not provide the
range of values or the number of systems evaluated in developing its
value.
The IV&V contractor's opinion that the renovation was not suitable
for testing was based on an in-depth understanding of these systems.
Therefore, it is a better gauge of this effort than the Internet
average, the basis of which is unknown. Also, because of the extent
of problems found during its partial test, the IV&V contractor
recommended that the system's entire portfolio of code ultimately
receive an IV&V evaluation. Even though the maintenance contractor
aggressively addressed the identified errors HCFA noted, it could
take years for the IV&V contractor to complete its evaluation; thus,
conducting this test 18 months before 2000 is not necessarily timely.
For example, it took 2 months to evaluate 200,000 lines of computer
code. At this rate of work, to evaluate the remaining 4.3 million
lines of computer code will take another 3-1/2 years--or until
2002--to complete the test. Further, during our audit, three
individuals (the Acting Director of Tracking and Reporting, the
Director of External Systems, and a Senior Technical Advisor) told us
that because of other Y2K demands, HCFA and its IV&V contractor did
not have time to follow up on and verify most of the Medicare
contractor reports. Yet, HCFA has demonstrated that it believes that
these activities are worthwhile by concurring with our recommendation
to independently verify the reported status of its claims processing
contractors' systems, internal systems, and data exchanges and
developing a plan to significantly increase its staff and related
activity.
3. HCFA concurs that end-to-end testing is critical and is procuring
additional support services to test standard systems and local
contractor Medicare systems not currently resident at one of the
existing HCFA Medicare shared system test sites. While end-to-end
tests of individual Medicare systems are important as a first step,
they will not provide sufficient assurance that Medicare is Y2K
compliant without a test of the combined Medicare system and its
components. HCFA's Director of Engineering concurs and told us that
conducting a series of individual overlapping tests does not meet the
standard for an end-to-end test. Further, HCFA has not received a
certification from its IV&V contractor that its planned approach to
end-to-end testing is complete and sufficient.
4. HCFA agrees that an overall schedule and a critical path are
important to managing its Y2K work. While we agree with HCFA that
preparing an overall project plan and a critical path is a daunting
task for a system as complex as that of Medicare, carrying out such a
complex project without these tools would be overwhelming. Even
though HCFA is working to gather and validate data for an overall
project schedule and is identifying dependencies to develop an
overall critical path for its Y2K work, its external and internal
schedules are being developed separately. As stated in our report,
HCFA's plans to develop two separate schedules will not provide the
overall project information it needs to adequately prioritize its
remaining work and will not enable it to develop a unified Y2K
critical path. In its summary, HCFA stated that it plans to
integrate its external systems project plan into a critical path
analysis for all of its systems, internal and external, before the
end of the year. However, integrating separately developed schedules
and critical path dependencies will require additional time and
resources. Accordingly, HCFA should start to combine these efforts
immediately and develop a single, overall Y2K schedule and critical
path for its external and internal systems. Further, we agree that
HCFA should remain highly proactive in monitoring the progress of its
shared system maintainers and its internal system renovation effort.
A fully integrated schedule with a critical path will help it achieve
this goal.
5. HCFA's comments about its efforts to renovate the CWF do not
change our position that a critical path analysis, incorporating the
production version 98.04 of the CWF (scheduled to be operational
after January 4, 1999) and including reasonable amounts of time for
testing, would have shown that the Medicare Y2K project could not be
completed before March 1999 at the very earliest. HCFA's comments
address a non-production version of the CWF. While HCFA is correct
in stating that this CWF version 98.02 has been renovated, it was not
installed until September 2, 1998. Also, even though it was
installed at four test sites so that shared systems maintainers and
contractors could self-certify that their systems are Y2K compliant,
this version cannot substitute for the production version. All
self-certification tests conducted with CWF version 98.02 will have
to be repeated using production version 98.04.
Further, it is highly unlikely that the self-certifying test against
version 98.02 will be completed by HCFA's December 31, 1998,
deadline. Thus, the testing process could slip even further. For
example, according to the HCFA Testing Project Officer, the first
three standard systems to complete renovation will not begin testing
against this version until the first week of October 1998. Given
that this testing generally requires 60 to 90 days to complete, as we
stated earlier in our report, only these three of HCFA's seven
standard systems will complete these tests by December 31, 1998, as
indicated by HCFA. Further, this assumes that no additional changes
to these systems will be required when they are tested against
production version CWF 98.04, which generally is not the case. HCFA
had previously estimated that these three systems would complete
their tests by September 1, October 28, and December 18, 1998.
Shared system maintainers and contractors will have to go through a
complete recertification with production version CWF 98.04 after it
is installed. If these tests take 60 to 90 days, as generally
required, the testing will not be completed until at least March
1999.
6. HCFA recognizes that its risk management and tracking processes
need improvement. Consequently, HCFA plans to work with its IV&V
contractor to improve these processes. Indeed, HCFA has taken a
proactive stance in addressing potential risks, such as suspending
many contractors transitions to the standard part A and part B claims
processing systems to ensure that resources would be available to
address Y2K remediation efforts. However, HCFA's comments do not
address several of the risk management problems we identified, such
as closing risks before they are mitigated and tracking only those
risks for which its IV&V contractor has made a recommendation. As
stated in our report, without a more comprehensive risk tracking
system than it now uses, HCFA cannot ensure adequate and timely
mitigation of Y2K program risks.
7. We agree with HCFA that pivot years are generally recognized and
accepted as a Y2K remediation technique throughout the United States.
However, as we reported, using multiple years increases the potential
for errors in data exchanges and requires additional management
oversight. Although HCFA disagreed with our assessment, its
guidelines recommended that all contractors use a single pivot year
in their Y2K renovations, and HCFA's Common Working File Technical
Advisor agreed that it would "make sense" to have one pivot year for
all HCFA systems. HCFA stated that (1) it does not matter to the
bridging software what the pivot year is as long as it is known and
(2) bridging software that accepts dates with two-digit years
requires senders to specify the pivot year used. While this is true,
we continue to maintain our position that multiple pivot years should
not be used because they require additional management oversight.
Additionally, even when bridging software is used, the same two-digit
year may be expanded into different four-digit years when a single
system uses multiple pivot years. The potential for date
misinterpretation exists when multiple pivot years are used without a
tracking mechanism. Now that HCFA has allowed the contractors and
internal system maintainers to use multiple pivot years, it should
track the pivot years and assess any potential problems due to their
use. This requires additional management oversight that would not
have been necessary if a single pivot year had been used. Our
recommendation on this subject calls for a pivot year tracking
system, which is needed to assist in this additional management
effort.
8. HCFA has taken steps to improve the quality and completeness of
the information on data exchanges between its Medicare contractors
and third parties. While HCFA did not identify those steps in its
comments, our report identified some of the actions that HCFA is
taking, such as ensuring that the quality of information on data
exchanges received from contractors is improved and preparing to
conduct a more rigorous inventory of contractor data exchange
agreements. We agree that these actions are important. However, we
are concerned that HCFA will not be able to complete its remaining
data exchange work in time to support its testing and contingency
planning. Without additional information, we cannot determine how
well the steps HCFA is taking will help it ensure that its end-to-end
testing is thorough and its contingency plans are complete as HCFA
states.
9. HCFA has provided more detail concerning its budget requests for
Y2K renovation projects. This additional information should be
useful to OMB and the Congress in analyzing HCFA's Y2K effort and
understanding its planned approach for the next few years. It should
be noted that OMB's most recent quarterly report on governmentwide
Year 2000 progress did not include HHS' recent estimate of Y2K costs
for fiscal year 2000. According to OMB, "approximately $550 million
in FY 2000 costs is still being reviewed by OMB. Almost all of these
costs are attributable to HCFA."
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION,
WASHINGTON, D.C.
-------------------------------------------------------- Appendix II:1
Mark E. Heatwole, Senior Assistant Director
Elizabeth A. Roach, Analyst-in-Charge
Robert C. Norris, PhD, Senior Information Systems Analyst
Nabajyoti Barkakati, PhD, Technical Assistant Director
Michael P. Fruitman, Communications Analyst
*** End of document. ***