VA Health Care Delivery: Top Management Leadership Critical to Success of
Decision Support System (Letter Report, 09/29/95, GAO/AIMD-95-182).
One of the main missions of the Department of Veterans Affairs (VA) is
to provide health care to veterans. VA's Veterans Health Administration
(VHA) runs the largest health care system in the nation, with an annual
budget of more than $16 billion. Despite its size and complexity,
however, VHA lacks detailed, reliable information on the operating costs
of its 172 hospitals. Consequently, as GAO reported in December 1992
(GAO/OCG-93-4TR), VHA could not determine which of its facilities were
working well and which procedures were cost-effective. This report
assesses VA's efforts to develop a medical decision support system, a
computer system that has provided hospitals in the private sector with
improved data on patterns of patient care and the cost of providing
health care services. As of July 1995, VA had started implementing its
decision support system at 38 hospitals. GAO examines (1) the benefits
that such a system can provide VA, (2) whether VA is pursuing the
comprehensive business strategy needed to achieve these benefits, and
(3) whether VA is establishing an adequate information infrastructure
for the system.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-95-182
TITLE: VA Health Care Delivery: Top Management Leadership Critical
to Success of Decision Support System
DATE: 09/29/95
SUBJECT: Health care cost control
Data integrity
Management information systems
Medical records
Strategic information systems planning
Cost effectiveness analysis
Veterans hospitals
Hospital care services
IDENTIFIER: VA Centralized Accounting for Local Management System
VA Personnel Accounting and Integrated Data System
VA Decentralized Hospital Computer Program
VA Decision Support System
Medical Care Cost Recovery Fund
VA Patient Treatment File
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to the Committee on Veterans' Affairs, House of
Representatives
September 1995
VA HEALTH CARE DELIVERY - TOP
MANAGEMENT LEADERSHIP CRITICAL TO
SUCCESS OF DECISION SUPPORT SYSTEM
GAO/AIMD-95-182
VA's Decision Support System
(511169)
Abbreviations
=============================================================== ABBREV
DHCP - Decentralized Hospital Computer Program
DSS - Decision Support System
GAO - General Accounting Office
IRM - Information Resources Management
MCCR - Medical Care Cost Recovery
MIRMO - Medical Information Resources Management Office
VA - Department of Veterans Affairs
VHA - Veterans Health Administration
Letter
=============================================================== LETTER
B-260377
September 29, 1995
The Honorable Bob Stump
Chairman
The Honorable G.V. (Sonny) Montgomery
Ranking Minority Member
Committee on Veterans' Affairs
House of Representatives
One of the primary missions of the Department of Veterans Affairs
(VA) is the delivery of health care services to eligible veterans.
The Veterans Health Administration (VHA), within VA, operates the
largest health care system in the nation, with an annual budget in
excess of $16 billion. Despite its size and complexity, however, VHA
lacks detailed, reliable information on the operating costs of its
172 hospitals. Consequently, as we previously reported,\1 it could
not determine which of its facilities were working well and where
procedures were or were not cost-effective.
This report responds to the former Chairman's June 6, 1994, request
and subsequent discussions with the Committee's staff that we assess
VHA's efforts to implement a medical decision support system. Such a
computer-based system has provided hospitals in the private sector
with improved data on patterns of patient care and the cost of
providing health care services. As of July 31, 1995, VA had started
implementing its decision support system (DSS) at 38 hospitals. Our
objectives were to assess (1) the kinds of benefits that such a
system can provide VA, (2) whether VA is pursuing the comprehensive
business strategy needed to achieve these benefits, and (3) whether
VA is establishing an adequate information infrastructure for DSS.
--------------------
\1 GAO Transition Series, Financial Management Issues
(GAO/OCG-93-4TR, December 1992).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
DSS has the potential to be an effective management tool for
improving the quality and cost-effectiveness of VHA health care
operations. This has already been demonstrated in the private
sector. VA, however, has not yet developed the comprehensive
business strategy necessary to achieve such potential benefits.
Business goals and a comprehensive implementation strategy have not
been formulated to clearly define how VA will use DSS-generated
information or prioritize its limited resources to implement DSS. VA
also has not established the information infrastructure needed to
support DSS. Some of the data provided to DSS from other VA
information systems are incomplete and inaccurate, limiting VA's
ability to rely on DSS-generated information to make sound business
decisions.
VA's Under Secretary for Health recently announced support of DSS and
strong commitment to making it a priority system within VHA. These
are steps in the right direction. Sustaining top management
leadership and commitment within VHA is critical to the successful
implementation and use of DSS. VHA recognizes that its day-to-day
management culture needs to be transformed to one with much greater
attention to cost-effectiveness and the need for adequate management
information to balance cost containment, quality of care, and
accountability. Greater top management involvement will help ensure
that the benefits offered by the much needed system are realized,
including an adequate return on a projected $132 million DSS
investment.
BACKGROUND
------------------------------------------------------------ Letter :2
Decision support systems provide managers with information on
business operations to assist decision-making. In the health care
industry, these systems can provide managers and clinicians with data
on patterns of patient care and patient health outcomes, which can
then be used to analyze resource utilization and the cost of
providing health care services. A number of vendors offer various
types of decision support systems for the health care industry.
Decision support systems can compute the costs of services provided
to each patient by combining patient-based information on services
provided during episodes of care with financial information on the
costs and revenue associated with those services. For example, a
private sector hospital performing cataract surgery collects
information on the services provided to each patient, including the
laboratory tests performed and the medications supplied, through its
billing system. The hospital then collects revenue and cost
information through its accounting systems, incorporating the
collections from the insurance companies and applicable parties, such
as Medicare, and expenditures for utilities and equipment.
Using a decision support system to combine the clinical and financial
information from the billing and accounting systems, the hospital
can, for example, (1) calculate the specific cost of providing
cataract surgery to a patient, (2) compare revenue received to costs
incurred to determine profitability for this type of service, (3)
compare costs incurred for different physicians and for surgery
performed at different locations, (4) evaluate patient outcomes, and
(5) perform analyses on ways to increase the quality of service,
reduce costs, or increase profitability. Decision support systems
can also support the comparison of patient care to predefined health
care standards.
VHA'S PLANS FOR A DECISION
SUPPORT SYSTEM
---------------------------------------------------------- Letter :2.1
In light of VHA's lack of cost information on its hospitals and at
the urging of your Committee, VHA conducted a study resulting in the
acquisition of a decision support system. In September 1993, it
awarded a contract to a commercial vendor to implement this system at
10 VA hospitals. VHA has since increased the total number of
hospitals/sites currently implementing DSS to 38. As shown in figure
1, VA's interest in acquiring DSS dates back to 1983.
Figure 1: History of DSS at VA
(See figure in printed
edition.)
VA believes that DSS can help it effectively manage the cost and
quality of health care provided to an estimated 2.5 million veterans
annually. It also expects that DSS can help it remain a viable
option in national health care delivery as the country moves towards
a managed care environment focusing on cost-effectiveness.
In implementing DSS, VA plans to use its existing information systems
as the primary source of clinical and financial information.
Although VA does not have a billing system analogous to the private
sector, VA's Decentralized Hospital Computer Program (DHCP) captures
clinical workload information. VA also has accounting systems, the
Personnel and Accounting Integrated Data, and Centralized Accounting
for Local Management systems, which capture financial information on
labor and supplies, respectively. The systems providing information
to DSS, as shown in figure 2, are sometimes referred to as feeder
systems. VA has also developed software to extract information from
the feeder systems for input to DSS. Standard cost accounting
information, such as allocations of indirect material and labor, are
entered directly into DSS by hospital personnel.
Figure 2: DSS Information Flow
(See figure in printed
edition.)
VA plans to implement DSS at 161 of its hospitals.\2 This is a major
undertaking for the vendor--the DSS project is the largest
implementation of the vendor's product to date. The vendor's next
largest implementation involved 20 private sector hospitals. As
shown in figure 3, the implementation was initially planned over a
3-year period from January 1994 through December 1996. The
implementation was recently slowed to allow VA to address critical
implementation issues.
Figure 3: DSS Implementation
Plan as of September 1994
(See figure in printed
edition.)
As of June 1, 1995, VA had started implementing DSS at the 32 sites
shown in figure 4. VA implemented another 6 hospitals in July. VA
estimates that the total cost of implementing DSS will be about $132
million.\3 Also, according to VA officials, as of July 20, 1995, they
had spent about $30 million on the DSS project.
Figure 4: Location of First 32
DSS Implementation Sites
(See figure in printed
edition.)
Operational responsibility for the DSS project lies with the DSS
Program Office in Kansas City, Missouri,\4 which reports to the VHA
Chief Financial Officer in Washington, D.C. The program office is
responsible for coordinating and directing the implementation of DSS
at the hospitals. In June 1995, the program office was headed by an
acting project director,\5 who was assisted by an acting deputy
director for operations and a deputy director for information
resource management. Assisting the program office on technical and
quality issues are the deputy directors for technical implementation,
data systems development, administration and resource management, and
quality management.
Under VHA's March 1995 restructuring plan, which is expected to begin
implementation on October 1, the program office will report to the
new position of VHA Chief Information Officer, instead of the Chief
Financial Officer. The Chief Information Officer will report to the
Under Secretary for Health. While it is unclear at this time what
role the Chief Financial Officer will have over DSS in the future, we
believe that the DSS project will benefit from having this individual
serve in an advisory capacity to the Under Secretary regarding DSS.
--------------------
\2 VA has stated that it has 172 hospitals; 11 of these are part of
other hospitals, leaving 161 hospitals at which to implement DSS.
\3 The overall implementation cost for DSS is comprised of vendor
costs and internal VA costs. Of the $132 million, $22 million is
estimated for the vendor costs, covering software, software
maintenance, and consulting services. VA's internal costs cover
personnel, facilities, hardware, training, and travel.
\4 In July 1995, the DSS Program Office relocated from Kansas City,
Missouri, to Washington, D.C.
\5 A new DSS Program Director was appointed in July 1995.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3
To determine the potential benefits to be gained from VA implementing
DSS and whether VA was pursuing a coordinated business strategy, we
discussed these issues with the Under Secretary for Health, the VHA
Chief Financial Officer, the DSS Project Director, the Director of
Medical Information Resources Management Office (MIRMO), and
representatives of private sector hospitals who use the vendor's
software. We also reviewed relevant VHA organizational plans and
related management documents.
To determine whether VA was establishing an adequate information
infrastructure for DSS, we interviewed key DSS program officials in
Washington, D.C.; Kansas City, Missouri; the National DSS Training
and Education Office in Cleveland, Ohio; and the Technical Office
located in Bedford, Massachusetts. We reviewed DHCP documentation,
DSS processing information, and extract software design information.
We had extensive discussions with MIRMO staff at the Information
System Center in Birmingham, Alabama, involved in developing DSS
extract software. Additionally, we met with staff at the Austin
Automation Center in Austin, Texas, involved in processing DSS and
DHCP information.
To determine whether VHA was implementing DSS in a manner likely to
maximize success, we visited VHA Medical Centers implementing DSS in
Brockton, Massachusetts; New York, New York; Oklahoma City, Oklahoma;
and Temple, Texas. We met with members of the DSS implementation
team at each location as well as with top management personnel. We
also compared VA's effort to implement DSS against the best practices
of leading private and public organizations for strategic information
management identified in our publication entitled, Executive Guide:
Improving Mission Performance Through Strategic Information
Management and Technology, (GAO/AIMD-94-115, May 1994). We met with
the vendor providing the DSS software and had discussions with other
vendors who market similar software. We also had discussions with
private sector health care providers who are using the vendor's DSS
software regarding their successes and problems in using DSS. We
reviewed VA's DSS implementation plans, the contract between VA and
the vendor, and other DSS implementation project documents.
In addition, we obtained oral comments on a draft of this report from
the VHA Chief Financial Officer. His comments are summarized in the
"Agency Comments and Our Evaluation" section of this report. We
conducted our work between June 1994 and June 1995 in accordance with
generally accepted government auditing standards.
DSS HOLDS PROMISE FOR VA
------------------------------------------------------------ Letter :4
VA believes that DSS can provide it with an opportunity to gain
control of its health care costs and increase the efficiency of
health care delivery. With DSS, VA can calculate the cost of its
health care services and use this information to assess its financial
competitiveness in changing health care markets and improve its
operations. For example, DSS can provide VA with a basis for
maximizing third-party reimbursements through the Medical Care Cost
Recovery (MCCR)\6 program, improving the quality of health care
delivery and allocating VHA resources on the basis of workload and
local efficiencies.
As we reported in December 1992, VA lacks information on the costs of
providing health care services at each of its 172 hospitals. The
availability of this information would be a major step toward
financial accountability at VA. DSS is expected to provide hospital
managers and health care providers with variance reports identifying
areas for reducing costs and improving patient outcomes and clinical
processes. Private sector hospitals already use decision support
systems to achieve these objectives. For example, a private sector
health care organization used information from its decision support
system to reduce the costs associated with surgical supply packs.
Staff there determined that the supply packs for a gall bladder
procedure varied greatly in price, yet the higher cost packs did not
improve patient outcomes. The organization was able to work with a
vendor to reduce the price of the packs, saving $600,000 annually.
According to representatives of another private sector health care
organization, the vendor's software enabled them to competitively
price medical services and win contracts for these medical services.
VA officials have also stated that DSS can help them collect more
MCCR revenue by providing them with itemized cost information on
which to base bills to third-party payers. An itemized bill would
identify the costs of all medical services and supplies provided to
the patient. Because VA currently lacks a cost accounting system, it
is unable to prepare itemized bills. VA currently bills third-party
payers on a flat-rate basis, regardless of the level of services
provided or the cost of these services. For example, these payers
are billed a flat rate of $1,350 per day for inpatient surgery,
regardless of the type of surgery performed. As such, VA may not be
billing third-party payers for all applicable costs associated with
the patient.
Aside from enhancing financial management, VA can use DSS to improve
the quality of its health care services. For example, a private
sector hospital used the vendor's software to conduct a pilot study,
comparing the treatment of heart failure patients with medical
treatment standards defined by hospital experts and identified some
treatment practices requiring modification by physicians. By
adopting these treatment modifications, the hospital reduced its
patient length of stay by an average of half a day and treatment
costs by $250,000. According to a hospital official, mortality rates
for these patients decreased by 2.6 percent, and readmissions
decreased by 3.3 percent.
When fully implemented, DSS should be able to provide valuable
information on the costs of medical services and patterns of patient
care and patient outcomes at the regional and national levels of VHA.
DSS also has the capability to "roll-up" information to the corporate
level. For example, a private sector organization with multiple
hospitals used the vendor's software to analyze the cost and
profitability of its cardiology services at different locations. The
decision support software enabled the manager to determine that one
of its hospitals was purchasing expensive catheterization lab
services, which reduced the profitability at that hospital.
Similarly, VHA can use DSS to assess the relative performance of
specific hospitals, both within and across its networks, and make
necessary adjustments, such as reallocation of personnel resources,
based on workload and local efficiencies. VHA can also use DSS,
which allows it to model the patient case mix, volume, resource cost,
and reimbursement changes, to assist in preparing its budget request.
--------------------
\6 The mission of VA's MCCR program is to maximize the recovery of
funds due VA for the provision of health care services to veterans,
dependents, and others using the VA system. VA is authorized to
submit claims to veterans' third-party insurance carriers and collect
co-payments from veterans for treatment and medications for
nonservice-connected conditions.
COMPREHENSIVE DSS BUSINESS
STRATEGY LACKING
------------------------------------------------------------ Letter :5
VHA has not developed a business strategy for effectively utilizing
DSS as a management tool. Top managers have not defined the business
goals to be achieved and measured using DSS, nor have they
historically assumed the leadership necessary to ensure that DSS is
successfully implemented. Lack of goals and leadership has put the
DSS project at risk. Correcting these problems will not be easy
because VA's culture has not traditionally focused on the
cost-effectiveness of hospital operations. The Under Secretary for
Health, however, has recently demonstrated a strong commitment to
DSS, and has taken initial steps to develop business goals and
address cultural issues.
BUSINESS GOALS NOT
ESTABLISHED
---------------------------------------------------------- Letter :5.1
Business goals are the foundation from which organizations develop
strategic plans and strategic information management plans. These
goals and associated plans guide the organization, determine how and
where resources will be used, and provide a framework for using
management tools such as DSS. Additionally, performance measures
based on clearly defined goals provide a mechanism for identifying
problems and assessing progress.
The Under Secretary for Health told us that VHA does not have
business goals. While he was unable to explain why VHA had not
established business goals earlier in the project,\7 the Under
Secretary acknowledged the importance of business goals and said that
they were a necessary prerequisite for developing performance
measures.
The lack of business goals for VHA has contributed to a lack of clear
goals for the DSS project. Without clear business goals for DSS, the
individuals involved with the project set their own personal
objectives for DSS. These varied and sometimes conflicted. For
example, the Project Director's goal was simply to implement DSS at
the 161 VA hospitals--how each hospital used DSS was up to each
hospital. The objective of the Deputy Director for Technical
Implementation was for DSS to accurately capture all clinical
episodes of care. The Deputy Director for Quality Management's goal
was to achieve health care delivery improvements. Clear business
goals could incorporate these objectives into a common framework to
enhance VHA health care delivery.
--------------------
\7 The current Under Secretary came to VHA during the fall of 1994.
IRM LEADERSHIP ESSENTIAL TO
DSS IMPLEMENTATION
---------------------------------------------------------- Letter :5.2
The senior information resource management (IRM) executive in an
organization should play a critical role in seeing that business and
information strategies are carefully coordinated to achieve
organizational goals. The VHA organizational structure currently
does not have an executive in a position to coordinate competing
priorities between DHCP and DSS and effectively allocate limited IRM
resources. For example, no one at VHA is setting priorities on the
critical data elements needed in DHCP to support the DSS information
infrastructure. As we discuss later, DSS requires some key data not
currently captured in DHCP. To obtain the data from DHCP would
require VHA top management to direct MIRMO, responsible for managing
DHCP and related projects, to work on DSS priorities. However, the
DSS Project Office and MIRMO report to different individuals. While
both offices are organizationally under the Deputy Under Secretary
for Health for Administration and Operations, this position has been
vacant since January 31, 1995.
VA CULTURE CONSTRAINS
PROGRESS
---------------------------------------------------------- Letter :5.3
As we have previously stated,\8 VHA does not operate as a centrally
managed health care system but as individual medical centers
competing with each other to provide as wide a range of services as
possible. Medical center directors' performances are generally
judged by what new facilities, services, and equipment they bring to
the medical centers. During the initial DSS test period, several
directors at one VA hospital did not see DSS as needed, were not
interested in using DSS, and did not attempt to understand it.
VHA is in the process of replacing its current regional system, which
is comprised of four regions, with 22 Veterans Integrated Service
Networks. VHA's vision, according to its March 1995 restructuring
plan, is to improve customer satisfaction, quality of care, access,
and cost-effectiveness. The plan also states that "VHA has instilled
certain behaviors and attitudes in its employees that are not
compatible with this new direction." The Under Secretary for Health
recognizes that this transformation will take time, and that it will
not be easy to change VHA's decades-old culture. He further stated
that if the veterans health care system is to remain viable it must
fundamentally change its approach to providing care.
--------------------
\8 VA Health Care: Challenges and Options for the Future
(GAO/T-HEHS-95-147, May 9, 1995).
RECENT DEVELOPMENTS
---------------------------------------------------------- Letter :5.4
We met with the Under Secretary for Health on March 10, 1995, and
expressed our concerns about the lack of a comprehensive business
plan for DSS, including a lack of leadership, goals, and performance
measures. In response to our concerns, the Under Secretary for
Health recently initiated steps to address the need for a coordinated
business strategy for DSS. In a May 18, 1995, memorandum, he stated
that DSS is one of VHA's top information systems priorities. In
addition, VHA plans to reorganize its IRM organizational structure.
Specifically, it plans to place DSS and clinical feeder systems such
as DHCP under the newly created position of VHA Chief Information
Officer, which reports to the Under Secretary. These actions should
help address the lack of leadership and competing IRM priorities.
Finally, to help address some of the cultural issues, the Under
Secretary for Health plans to implement a performance-based pay
system. According to VHA's restructuring plan, managers have
historically been evaluated on a variety of inconsistent, often
changing performance indicators that were frequently subjective. In
contrast, the performance-based system is expected to hold field
units and senior managers accountable for objective, measurable
achievements. However, VHA has not yet articulated clear business
goals or formulated a comprehensive business plan for DSS.
INFORMATION INFRASTRUCTURE
INADEQUATE
------------------------------------------------------------ Letter :6
Accurate and complete data from VA's feeder systems are also critical
to the success of DSS. Anything less will result in the �garbage
in-garbage out� analogy. If inaccurate and incomplete data are input
to DSS, DSS either will not be used because its data will not be
credible, or managers and health care providers relying on DSS will
make poor decisions based on incorrect data. We found that some of
the key clinical data in DHCP and other clinical feeder systems\9 are
being collected completely and provided to DSS. For example, general
laboratory test information is collected by DHCP's laboratory
software and provided to DSS. The lab software collects all needed
pieces of information to define a billable event. Radiology is
another clinical area in which DHCP collects all needed information
for input to DSS.
However, as shown in figure 5, we also found that some clinical data
are incomplete, inaccurate, or inconsistent. For analysis and
decision-making purposes, DSS must have information on all relevant
clinical events or clinical workload. This information is equivalent
to data describing the clinical services billed the payor in the
private sector. For VA, the following information is needed from
DHCP and other clinical feeder systems, to define a clinical billable
event:
patient identification;
provider identification--who ordered or provided the treatment;
time and date of treatment;
description of service provided, for example, type of x-ray or lab
test; and
location where the service was provided.
These data must be captured as needed to support the specific
management decisions to be made using DSS.
Our review showed that some clinical data provided to DSS from DHCP
and other clinical feeder systems are incomplete or inaccurate.
These problems stem from the fact that DHCP was not designed to
capture itemized clinical billing information and feed this
information to a billing or decision support system. Moreover, as we
discussed earlier, VHA management has not identified specific
decisions that DSS is to support, which is a critical factor in
determining the data needed for DSS.
Figure 5: Examples of Clinical
Data Problems
(See figure in printed
edition.)
Incomplete clinical data make it difficult to perform detailed
analysis of clinical costs and activities and make appropriate
improvements regarding cost-effectiveness and quality of care.
Inaccurate clinical data could cause decisions to be made on the
basis of erroneous information. Inconsistent clinical data make
efforts to consolidate data across VA medical centers for corporate
roll-up difficult.
In addition, VHA needs to properly record clinical events in the
correct time period and reconcile these events to ensure accuracy and
completeness of data--a process called close out. The use of DSS is
based on data flowing from the feeder systems to DSS on a monthly
basis. Implicit in this transfer is the availability of accurate and
complete information at the end of each month. To accomplish this,
private sector facilities reconcile, or close out, their clinical
workload records monthly. In contrast, VA closes out its records on
an annual basis only, at the end of each fiscal year. Timely monthly
close out would allow VA to know the cost of medical care provided
within discrete time frames. This would facilitate periodic cost
analyses, faster identification of trends and patterns, and more
timely adjustment of health care practices-- key DSS benefits.
Failure to close out in a timely manner can adversely affect the
usefulness of the data for decision-making and result in an
administrative burden in making necessary adjustments to clinical
workload records. For example, at VA's fiscal year 1994 annual close
out, it had to correct 8 million outpatient visits, out of a total of
23 million visits documented in its computerized outpatient clinic
file. These records would need to be accurate and complete at the
end of each month to support DSS. Adopting monthly close out will
require fundamental restructuring of administrative activities at VA
facilities.
Finally, VA recognizes deficiencies with its financial systems that
feed DSS. For example, the audits of VA's consolidated financial
statements for fiscal years 1994 and 1993, which were conducted by
the Office of the Inspector General, reported that real property,
plant, and equipment, and related depreciation account balances
captured in the Centralized Accounting for Local Management system
were unreliable because some accounting personnel at the VHA
hospitals lacked sufficient training and oversight. Additionally,
according to VA's 1994 and 1993 Federal Managers' Financial Integrity
Act reports, the Personnel Accounting Integrated Data System cannot
support mission-critical resource accounting functions necessary to
support initiatives such as the National Performance Review, MCCR,
and DSS.
Without accurate and complete financial information, VHA cannot
determine the cost of clinical events. VA is currently in the
process of replacing its Centralized Accounting for Local Management
system with a new system, known as the Financial Management System,
which is expected to be fully functional in October 1995.
During our March 10, 1995, meeting with the Under Secretary for
Health, we expressed concerns about the integrity of data being
provided to DSS, and the fact that VHA was going ahead with the
scheduled DSS implementations in light of these problems and others,
such as the lack of business goals and performance measures. We also
suggested that VHA consider selecting a small number of sites to
pilot the use of DSS by management before the system is implemented
throughout VA. By piloting DSS at selected sites, VHA can (1)
document the kinds of benefits that have been gained from using the
system and (2) identify the problems that have occurred at the pilot
test sites requiring top management's attention and resolution.
To address our concerns, the Under Secretary for Health took several
actions. Specifically, in his May 18, 1995, memorandum, the Under
Secretary reduced the number of additional hospitals scheduled for
July implementation from 30 to 6 and established a team to ensure
that some data elements are consistent across VA medical centers. In
addition, he told us VHA plans to have a system in place to collect
all billable outpatient care information by October 1996. While
these actions begin to address some of our concerns, VHA still does
not have a comprehensive plan to (1) identify what data are needed to
achieve its business goals, (2) correct known flaws in its data, or
(3) ensure that its feeder system software will collect the data
needed by DSS. In addition, VHA has not identified specific DSS
sites to pilot the use of the system as a management tool,
documenting the benefits gained and the problems encountered from
using DSS.
--------------------
\9 Other clinical feeder systems include the Patient Treatment File
and Outpatient Clinic File.
CONCLUSIONS
------------------------------------------------------------ Letter :7
Top management leadership is crucial if VHA is to effectively use DSS
as a management tool--and DSS is essential if health care costs,
quality, and reimbursement are to be effectively managed by VHA. A
comprehensive, proactive DSS strategy that establishes business
goals, leadership, and accountability would provide a framework
within which management could improve health care delivery and cost
recovery. This will not be easy and will take time. If VA is to
achieve the benefits associated with DSS, it must change a
decades-old culture in which business is conducted without enough
focus on delivering high quality health care at minimal cost.
In addition, for DSS to be useful for decision-making, it will
require a complete and accurate information infrastructure. We are
encouraged by the recent steps taken by the Under Secretary for
Health. He has demonstrated an understanding of the issues and a
willingness to respond. However, unless the Under Secretary's
actions are sustained and expanded to fully address the
organizational and information infrastructure issues identified,
including piloting DSS at a small number of sites, the millions of
dollars invested in DSS to date are at risk.
RECOMMENDATIONS
------------------------------------------------------------ Letter :8
To increase the likelihood of DSS' success, we recommend that the
Secretary of Veterans Affairs direct the Under Secretary for Health
to develop a comprehensive business strategy to
identify the specific business goals (for example, reduction of
cost in a specific area by a specific percentage), performance
measures, and key decisions that DSS will be required to
support;
give high priority, by allocating appropriate resources, to
establishing a complete, consistent, and accurate DSS
information infrastructure; and
identify data that are needed to support decision-making and ensure
that these data are complete, accurate, consistent, and
reconciled monthly.
We also recommend that VA not implement DSS at any site beyond the 38
already begun until (1) defined business goals and a supporting
information infrastructure supporting key decisions are in place and
(2) VA's capability to use DSS effectively as a management tool can
be demonstrated.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :9
The VHA Chief Financial Officer provided oral comments to our draft
report. He stated that the report was a fair, open, and honest
assessment of VA's efforts to implement DSS and that VA concurred
with most of the recommendations in the report. VA concurred with
our recommendation to establish a business strategy and specific
business goals and has already taken several actions in this regard.
The Under Secretary for Health recently established a work group on
performance measures that will be a key component to this effort. In
addition, VA recently appointed a new DSS Program Director, and his
first priority is to draft and implement a detailed DSS business
plan. The Under Secretary for Health also authorized establishing a
DSS Corporate Advisory Board to oversee implementation of major
systemwide policies and a Field Advisory Board to identify,
prioritize, track, and resolve issues that arise from pilot site
experience. VA also concurred with our recommendation to allocate
appropriate resources to support the DSS information infrastructure.
The new VHA Chief Information Officer will oversee both DHCP and DSS.
This individual and the VHA Chief Financial Officer will address
resource allocation needs relating to these systems.
VA concurred with our recommendation to identify data needed to
support decision-making and ensure that these data are complete,
accurate, and consistent. However, VA did not agree that monthly
reconciliations of clinical or workload records were necessary in
light of its future data improvement plans. Specifically, VA plans
to establish a national patient care database, which is expected to
be implemented in October 1996, that would provide the agency with
patient-unique encounter data so that individual changes can be
monitored and used in an automatic reconciliation process. The VHA
Chief Financial Officer stated that VA's efforts to establish the
database would be hampered if scarce resources were diverted to
performing monthly reconciliations.
To ensure accuracy and completeness of data, we believe that VA
should reconcile its clinical workload records on a monthly rather
than annual basis because VA plans to use DSS on a monthly basis. As
we pointed out in this report, timely monthly reconciliation or close
out would allow VA to know the cost of medical care provided within
discrete time frames. This would also facilitate periodic cost
analyses, faster identification of trends and patterns, and more
timely adjustment of health care practices. Failure to close out in
a timely manner can adversely affect the usefulness of data in DSS
for decision-making purposes and result in an administrative burden
in making necessary adjustments to clinical workload records at
fiscal year-end.
Furthermore, the VHA Chief Financial Officer did not clearly explain
how the national patient care database would eliminate VA's need to
perform monthly reconciliations. We believe that until this database
is implemented and providing complete and accurate data to DSS and
until the automated reconciliation process is defined and operating
effectively, VA should perform monthly reconciliations. Also, it is
crucial that as VA begins to develop this database, it ensures that
adequate internal control policies and procedures are in place so
that the database captures, maintains, and generates timely, accurate
data.
Lastly, the VHA Chief Financial Officer did not agree that DSS should
not be implemented beyond the 38 sites already begun until (1)
defined business goals and a supporting information infrastructure
are in place and (2) VA has demonstrated its ability to use DSS
effectively. He indicated that VA has made progress and is confident
that it will be able to effectively use DSS as a management tool. He
also indicated that private sector hospitals that use DSS did not
always have good, reliable data after 1 year and that expectations
for VA's implementation should be realistic. He felt that slowing
down the implementation of DSS could jeopardize its success.
While we agreed with the VHA Chief Financial Officer that private
sector hospitals implementing DSS may not necessarily have complete
and accurate data after 1 year, these hospitals generally have other
controls in place, such as billing systems, which provide them some
degree of financial accountability. VA, in contrast to the private
sector, does not have a billing system. Also, no private sector
hospital has implemented DSS at as many sites or as rapidly as VA
plans to do. For example, one private sector health care
organization told us that it implemented DSS at four sites over a
period of 18 months.
In addition, the likelihood of DSS's success will be jeopardized by
deploying it to 161 sites before a complete and accurate information
infrastructure and effective procedures for its use are in place. We
believe that a more appropriate course of action is to pilot DSS at a
small number of sites capable of such an undertaking, ensuring that
it is free from significant data integrity problems, that supporting
procedures and controls are in place, and that the system is useful
to management before it is deployed across 161 sites.
---------------------------------------------------------- Letter :9.1
We are sending copies of this report to the Chairman, Subcommittee on
Veterans Affairs, Housing and Urban Development, and Independent
Agencies, Senate Committee on Appropriations; the Secretary of
Veterans Affairs; the Director, Office of Management and Budget; and
other interested parties. Copies will also be made available to
others upon request. Please contact me at (202) 512-6252 if you or
your staffs have any questions concerning this report. Major
contributors to this report are listed in appendix I.
Frank W. Reilly
Director, Information Resources
Management/Health, Education,
and Human Services Issues
MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix I
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION,
WASHINGTON, D.C.
--------------------------------------------------------- Appendix I:1
Helen Lew, Assistant Director
Ira S. Sachs, Evaluator-in-Charge
KANSAS CITY REGIONAL OFFICE
--------------------------------------------------------- Appendix I:2
Janet M. Chapman, Senior Evaluator
*** End of document. ***