VA Information Technology: Improvements Needed to Implement Legislative
Reforms (Letter Report, 07/07/98, GAO/AIMD-98-154).
Pursuant to a congressional request, GAO reviewed how the Department of
Veterans Affairs (VA) has implemented specific provisions of the
Clinger-Cohen Act and other legislative reforms, including: (1)
reengineering business processes before acquiring information
technology; (2) completing an integrated information technology
architecture; (3) institutionalizing a disciplined information
technology investment decisionmaking process; and (4) appointing an
agency Chief Information Officer (CIO).
GAO noted that: (1) VA has not fully implemented critical provisions of
the Clinger-Cohen Act and other legislative reforms; (2) although VA has
taken some initial steps, it has not adequately implemented these
legislative reforms; (3) specifically, the Clinger-Cohen Act requires
agencies to analyze their mission-related and administrative processes,
and on the basis of this analysis, revise and improve these processes
before making significant investments in supporting information
technology; (4) although GAO's business process reengineering guide
states that agencies should have an overall business process improvement
strategy to accomplish reengineering, VA has not developed such a
strategy; (5) VA also has not yet defined the departmentwide integrated
information technology architecture needed to efficiently utilize
information systems across the department; (6) in addition, VA has not
institutionalized a disciplined process for selecting, controlling, and
evaluating information technology as investments as required by the
Clinger-Cohen Act; (7) specifically, VA decisionmakers did not have
current and complete information such as cost, benefit, schedule, risk,
and performance data at the project level, which is essential to making
sound investment decisions; (8) in addition, VA's process for
controlling and evaluating its investment portfolio is incomplete and,
as a result, decisionmakers do not have the information needed to: (a)
detect or avoid problems early; and (b) improve VA's investment process;
(9) as a consequence, the department does not know whether it is making
the right investments, how to control these investments effectively, or
whether these investments have provided mission-related benefits in
excess of their costs; (10) although the Clinger-Cohen Act requires
agencies' CIOs to have information management as their primary duty, the
responsibilities of VA's CIO are not limited primarily to information
management; (11) instead, the CIO also functions as the department's
Assistant Secretary for Management and Chief Financial Officer; and (12)
as a result, information technology issues are not addressed promptly.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-98-154
TITLE: VA Information Technology: Improvements Needed to Implement
Legislative Reforms
DATE: 07/07/98
SUBJECT: Federal agency reorganization
Information resources management
Reengineering (management)
Strategic information systems planning
Chief information officers
Systems conversions
Information technology
Systems design
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
July 1998
VA INFORMATION TECHNOLOGY -
IMPROVEMENTS NEEDED TO IMPLEMENT
LEGISLATIVE REFORMS
GAO/AIMD-98-154
VA Information Technology
(511233)
Abbreviations
=============================================================== ABBREV
CIO - Chief Information Officer
CFO - Chief Financial Officer
ITA - Veteran-Focused Information Technology Architecture Program
MVR - Master Veteran Record
NCS - National Cemetery System
NIST - National Institute of Standards and Technology
OMB - Office of Management and Budget
PIR - Post-implementation review
VA - Department of Veterans Affairs
VBA - Veterans Benefits Administration
VETSNET - Veterans Service Network
VHA - Veterans Health Administration
Letter
=============================================================== LETTER
B-278709
July 7, 1998
The Honorable Arlen Specter
Chairman, Committee on Veterans' Affairs
United States Senate
The Honorable John D. Rockefeller IV
Ranking Minority Member
Committee on Veterans' Affairs
United States Senate
The Honorable Bob Stump
Chairman, Committee on Veterans' Affairs
House of Representatives
The Honorable Lane Evans
Ranking Minority Member
Committee on Veterans' Affairs
House of Representatives
In your September 1997 letter, you requested that we review and
evaluate how well the Department of Veterans Affairs (VA) is
selecting and managing its information technology investments and
identify specific areas where improvements could be made.
Information technology represents about $1 billion of VA's fiscal
year 1999 budget request of $43 billion. This report provides the
results of our review.
The Clinger-Cohen Act and other related legislative reforms provide
guidance on how VA and other agencies should plan, manage, and
acquire information technology as part of their overall information
resources management responsibilities. They also require federal
agencies to appoint Chief Information Officers (CIOs) responsible for
providing leadership in the acquisition and management of information
resources. Accordingly, as agreed with your offices, our specific
objectives were to examine how VA has implemented the following
specific provisions of the Clinger-Cohen Act and other legislative
reforms: reengineering business processes before acquiring
information technology; completing an integrated information
technology architecture;\1 institutionalizing a disciplined
information technology investment decision-making process; and
appointing an agency CIO.
--------------------
\1 An integrated information technology architecture is a blueprint,
consisting of logical and technical components, to guide and
constrain the development and evolution of a collection of related
systems. At the logical level, the architecture provides a
high-level description of an organization's mission, the business
functions being performed and the relationships among the functions,
the information needed to perform the functions, and the flow of
information among functions. At the technical level, the
architecture provides the rules and standards needed to ensure that
the interrelated systems are built to be interoperable and
maintainable. These include specifications of critical aspects of
component systems' hardware, software, communication, data, security,
and performance characteristics.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
VA has not fully implemented critical provisions of the Clinger-Cohen
Act and other legislative reforms. Although VA has taken some
initial steps, it has not adequately implemented these legislative
reforms. Specifically, the Clinger-Cohen Act requires agencies to
analyze their mission-related and administrative processes, and on
the basis of this analysis, revise and improve these processes before
making significant investments in supporting information technology.
Although GAO's business process reengineering guide\2 states that
agencies should have an overall business process improvement strategy
to accomplish reengineering, VA has not developed such a strategy.
VA also has not yet defined the departmentwide integrated information
technology architecture needed to efficiently utilize information
systems across the department.
In addition, VA has not institutionalized a disciplined process for
selecting, controlling, and evaluating information technology as
investments as required by the Clinger-Cohen Act. Specifically, VA
decisionmakers did not have current and complete information such as
cost, benefit, schedule, risk, and performance data at the project
level, which is essential to making sound investment decisions. In
addition, VA's process for controlling and evaluating its investment
portfolio is incomplete and, as a result, decisionmakers do not have
the information needed to (1) detect or avoid problems early and (2)
improve VA's investment process. As a consequence, the department
does not know whether it is making the right investments, how to
control these investments effectively, or whether these investments
have provided mission-related benefits in excess of their costs.
Finally, although the Clinger-Cohen Act requires agencies' CIOs to
have information management as their primary duty, the
responsibilities of VA's CIO are not limited primarily to information
management. Instead, the CIO also functions as the department's
Assistant Secretary for Management and Chief Financial Officer (CFO).
As a result, information technology issues are not addressed
promptly.
In commenting on a draft of this report, VA concurred with all six
recommendations. VA stated, among other things, that it will
establish the position of Assistant Secretary to serve as the
department's CIO, reporting directly to the Secretary on all
information resources issues.
--------------------
\2 Business Process Reengineering Assessment Guide (GAO/AIMD-10.1.15,
April 1997, Version 3).
BACKGROUND
------------------------------------------------------------ Letter :2
VA comprises three major components: the Veterans Benefits
Administration (VBA), the Veterans Health Administration (VHA), and
the National Cemetery System (NCS).\3 VA's mission is "to administer
the laws providing benefits and other services to veterans and
dependents. . . ." The department's vision is to be a more
customer-focused organization, functioning as "One VA." This vision
stemmed from the recognition that veterans think of VA as a single
entity, but often encounter a confusing, bureaucratic maze of
uncoordinated programs that put them through repetitive and
frustrating administrative procedures and delays. The "One VA"
vision is to create versatile new ways for veterans to obtain
services and information by streamlining interactions with its
customers and integrating information technology resources to enable
VA employees to help customers more quickly and effectively. This
will require modifying or replacing separate information systems with
integrated systems using common standards to share information across
VA programs and with external partner organizations, such as the
Department of Defense.
Information technology accounted for approximately $1 billion of VA's
fiscal year 1999 budget request of $43 billion. Of the $1 billion,
about $847 million, $146 million, and $5 million were for VHA, VBA,
and NCS, respectively.
Over the past several years, we have identified weaknesses in VA's
efforts to modernize its operations and manage its information
technology resources. As we reported in 1992, VBA's procurement of
hardware was not supported by a defined information architecture,
thereby increasing the risk of developing systems that would not work
as intended.\4 In June 1996, we testified that VBA needed to develop
a much improved investment strategy for selecting and managing
information technology projects in a more disciplined, businesslike
manner.\5 In January 1998, we reported\6 that while VA made
significant progress in preparing a strategic plan, dated September
30, 1997, the plan needed improvement in four major areas: (1)
development of results-oriented goals, (2) descriptions of how the
goals are to be achieved, (3) discussion of external factors, and (4)
discussion of coordination efforts with other agencies. Finally, in
October 1997, we testified on the importance of having strong CIOs at
major federal agencies, such as VA, to bring about much-needed
reforms in the government's management of information technology.\7
In addition, a panel of the National Academy of Public Administration
reported in August 1997, that VBA lacks strategic planning and
management capabilities that are necessary for leadership to define
where the organization wants to be, enable development of specific
operational plans for getting there, and provide a set of
coordinating and integrating capacities for implementing planned
initiatives.\8
Recognizing the need to better manage information technology, recent
legislative reforms--the Clinger-Cohen Act of 1996,\9 the Paperwork
Reduction Act of 1995, and the Federal Acquisition Streamlining Act
of 1994--provide guidance to federal agencies on how to plan, manage,
and acquire information technology as part of their overall
information resources management responsibilities. These legislative
reforms highlight the need for business process reengineering,
integrated architectures, investment processes, and CIOs to help with
major information resource management responsibilities.
--------------------
\3 VBA provides nonmedical benefits to veterans and their dependents;
VHA provides services through the nation's largest health-care
system; and NCS provides burial services in 115 national cemeteries.
\4 Veterans Benefits: Acquisition of Information Resources for
Modernization Is Premature (GAO/IMTEC-93-6, November 4, 1992).
\5 Veterans Benefits Modernization: Management and Technical
Weaknesses Must Be Overcome If Modernization Is To Succeed
(GAO/T-AIMD-96-103, June 19, 1996).
\6 Managing for Results: Agencies' Annual Performance Plans Can Help
Address Strategic Planning Challenges (GAO/GGD-98-44, January 30,
1998), app. XV, "Observations on the Department of Veterans Affairs'
Strategic Plan."
\7 Chief Information Officers: Ensuring Strong Leadership and an
Effective Council (GAO/T-AIMD-98-22, October 27, 1997).
\8 Management of Compensation and Pension Benefits Claim Processes
for Veterans--A Report by a Panel of the National Academy of Public
Administration for Congress and the Department of Veterans Affairs,
August 1997.
\9 The Omnibus Consolidated Appropriations Act, 1997, renamed both
the Federal Acquisition Reform Act of 1996 and the Information
Technology Management Reform Act of 1996 as the Clinger-Cohen Act of
1996.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3
In assessing VA's implementation of the Clinger-Cohen Act and other
legislative reforms, we reviewed and analyzed numerous documents
pertaining to VA's business process reengineering, integrated
information technology architecture, information technology
investment decision-making process, and appointment of an agency CIO.
These documents include VA's draft entitled One VA: Vision of
Information Technology Enhanced Customer Service, dated January 22,
1998; OMB's Memorandum on Information Technology Architectures, dated
June 18, 1997; VA's draft FY 1999 Department Capital Plan, dated
October 1997; and VA's April 1997 Progress Report on the Department
of Veterans Affairs CIO Program.
We discussed VA's implementation of the Clinger-Cohen Act and other
legislative reforms with Office of Management and Budget (OMB)
officials and with various VA headquarters and component officials,
including the Offices of the CIO, Information Resources Management,
and Policy and Planning. We also interviewed a VA representative of
the contractor responsible for developing VA's information technology
vision document. We used our investment guide\10 to evaluate and
assess VA's information technology investment process.
We performed our work from October 1997 through April 1998, in
accordance with generally accepted government auditing standards. We
requested comments on a draft of this report from the Secretary of
Veterans Affairs and they are reprinted in appendix I. More details
of our objectives, scope, and methodology are included as appendix
II.
--------------------
\10 Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making (GAO/AIMD-10.1.13, February
1997).
VA LACKS OVERALL BUSINESS
PROCESS IMPROVEMENT STRATEGY
------------------------------------------------------------ Letter :4
The Clinger-Cohen Act requires agency heads to analyze the missions
of the agency and, on the basis of this analysis, revise and improve
the agency's mission-related and administrative processes before
making significant investments in supporting information technology.
Specifically, agencies should maximize the potential of technology to
improve performance, rather than simply automating inefficient
processes. According to our business process reengineering guide,\11
an agency should have an overall business process improvement
strategy that provides a means to coordinate and integrate the
various reengineering and improvement projects, set priorities, and
make appropriate budget decisions.
VA has not analyzed its business processes in terms of implementing
its "One VA" vision. In addition, it does not have a departmentwide
business process improvement strategy specifying what reengineering
and improvement projects are needed, how they are related, and how
they are prioritized. VA's Directive 6000 instructs administration
heads, assistant secretaries, and other key officials to apply sound
business process improvement or business reengineering methods to
enhance the benefits of information technology, but the directive
does not provide the guidance needed to accomplish this
departmentwide effort. Specifically, VA's strategy does not identify
needed reengineering and improvement projects, describe how they are
interrelated, determine the order in which they will be pursued, and
define specific goals, time frames, resource requirements, and key
participants for each.
In the absence of a departmentwide strategy, VA components are
proceeding with separate, uncoordinated efforts, which undermines the
department's "One VA" vision. For example, both VBA and VHA are
building information centers which enable callers, through the use of
an 800 number, to obtain information on general benefits and basic
services. However, these efforts are not currently coordinated. As
a result, both projects are unnecessarily providing the same
functionality. A senior VA official acknowledged that VA should not
be building two separate information center systems, and that doing
so is not consistent with the "One VA" vision.
Similarly, VA and its components have not adequately coordinated and
integrated their business process improvement efforts for the
development of a master veteran record (MVR). This departmentwide
project is intended to electronically link VHA, VBA, NCS, and Board
of Veterans' Appeals information systems and databases to share vital
information, such as death notification, change of address,
representation and family status about veterans. However, according
to the project manager, this project is experiencing difficulties
because VBA will not fund a segment of the project necessary to
establish a link between VBA's compensation and pension
program--VBA's largest program--and other VA components. As a
result, VBA is not in a position to obtain timely information, such
as death notifications, which can result in overpayments to veterans.
VA has acknowledged the need to develop a strategy to achieve the
"One VA" vision and has hired a contractor to analyze the
department's business plans and information technology projects to
determine how well they fit within the vision. However, to date, VA
has not committed to when it will have an overall business process
improvement strategy to accomplish reengineering.
--------------------
\11 GAO/AIMD-10.1.15, April 1997, Version 3.
VA LACKS AN INTEGRATED
INFORMATION TECHNOLOGY
ARCHITECTURE
------------------------------------------------------------ Letter :5
The Clinger-Cohen Act and recent OMB guidelines require agency CIOs
to implement an architecture to provide a framework for evolving or
maintaining existing information technology, and for acquiring new
information technology to achieve the agency's strategic and
information technology goals. Leading organizations both in the
private sector and in government use systems architectures to guide
mission-critical systems development and to ensure the appropriate
integration of information systems through common standards.\12
Despite the importance of doing so, VA and its components have yet to
define a departmentwide integrated architecture. For example, as we
reported in May 1997, VBA did not have a complete, integrated systems
architecture to help guide its new systems development activities.\13
We therefore recommended that VBA develop such an architecture,
including a security architecture and performance characteristics and
standards. VA concurred with our recommendation.
To formulate an approach for developing an integrated architecture,
VA in March 1997 established an architecture team consisting of
representatives from VA's Office of Information Resources Management,
VBA and VHA. This team issued a report to the VA CIO Council\14 in
May 1997 adopting the National Institute of Standards and Technology
(NIST)\15 five-layer model for its departmentwide information
technology architecture. The five layers--business processes,
information flows and relationships, applications processing, data
descriptions, and technology--provide a framework for defining an
information technology architecture. VA can use this model to help
it document the baseline architecture, identify a target
architecture, and develop a migration plan showing how the department
will make the necessary transition from its existing architecture to
the target architecture.
Despite the VA architecture team's efforts, VA does not yet have a
departmentwide target architecture and migration plan. While a
baseline architecture has been established, VA has not addressed key
aspects of the target architecture, such as information flows, data
descriptions, and common technical standards that would apply to VBA,
VHA, and NCS. According to VA's CIO, the department has not
addressed these aspects because it is waiting for the Strategic
Management Steering Committee\16 to take a position on the proposal
to develop a departmentwide target architecture and establish a
program office to implement the architecture and related efforts,
including business process reengineering and customer service
improvements.
--------------------
\12 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology--Learning From Leading
Organizations (GAO/AIMD-94-115, May 1994).
\13 Veterans Benefits Computer Systems: Risks of VBA's Year-2000
Efforts (GAO/AIMD-97-79, May 30, 1997).
\14 VA's CIO Council is comprised of VA's Assistant Secretary for
Management and his Deputy CIO, VA's Assistant Secretary for Policy
and Planning, VBA's CIO, VHA's CIO, NCS' Director of Operations
Support, and Board of Veterans' Appeals' Director of Management and
Administration.
\15 NIST Special Publication 500-167, "Information Management
Directions: The Information Challenge."
\16 The members of this committee include the Chief of Staff, the
Deputy Under Secretaries for Health and Benefits, the CIOs for VHA
and VBA, the CFOs for VHA and VBA, the Assistant Secretary for Policy
and Planning, and the NCS Directors for Field Operations and
Operations Support.
VA HAS NOT INSTITUTIONALIZED A
DISCIPLINED INFORMATION
TECHNOLOGY INVESTMENT PROCESS
------------------------------------------------------------ Letter :6
While information technology represents $625 million or 80 percent of
VA's proposed $786 million capital investment budget for fiscal year
1999, the department lacks an effective process for selecting,
controlling, and evaluating its information technology projects as
investments. VA's newly developed selection process, used for the
first time in the fiscal year 1999 budget cycle, is incomplete,
undisciplined, and does not satisfy the selection process
requirements specified in the Clinger-Cohen Act. For example,
decisionmakers did not have adequate information pertaining to
project cost, benefits, risk, and performance measures to make
well-informed decisions. Also, VA's process for monitoring and
controlling its investment portfolio was incomplete and provided
little information to VA decisionmakers reviewing ongoing projects.
Finally, VA's process for evaluating completed projects did not
include reviews to (1) determine the causes of major differences
between actual and expected results in terms of cost, schedule, and
performance, and (2) revise investment processes on the basis of
lessons learned. As a result of these weaknesses, the department
does not know whether it is making the right investments, how to
control these investments effectively, or whether these investments
have provided mission-related benefits in excess of their costs.
THE CLINGER-COHEN ACT
PRESCRIBES AN INVESTMENT
MANAGEMENT APPROACH
---------------------------------------------------------- Letter :6.1
The Clinger-Cohen Act requires agency heads to implement an approach
for maximizing the value and assessing and managing the risks of
information technology investments. It stipulates that this approach
should be integrated with the agency's budget, financial, and program
management processes.
According to our investment guide,\17 an information technology
investment process is an integrated approach that provides for
disciplined, data-driven identification, selection, control,
life-cycle management, and evaluation of information technology
investments. Information from one phase is used to support
activities in the other phases. When identifying information
technology investments to be managed at the department level, leading
organizations use criteria that include (1) high-dollar, high-risk
projects, (2) cross-functional projects (two or more organizational
units benefitting from the projects), and (3) common infrastructure
support, such as hardware and telecommunications.
Once selected, information technology projects in the investment
portfolio are consistently controlled and managed through progress
reviews at key milestones in a project's life cycle. Progress
reviews should include assessing deliverables, technical issues,
schedule, costs, and risks. Finally, once a project has been fully
implemented, a post-implementation review or evaluation should be
conducted, comparing actuals against estimates in order to assess
performance and identify areas where future decision-making can be
improved.
VA defined a new decision-making process for information technology
investments and conducted a "dry run" in formulating the fiscal year
1999 budget. As depicted in figure 1, this process began with VA's
staff offices and components submitting information packages about
their capital investment projects to an Information Technology
Strategic Planning Working Group. This group--composed of project
decisionmakers representing the VA Central Office's Office of
Information Resources Management, Office of Financial Management,
VBA, VHA, Board of Veterans' Appeals, Office of Planning and Policy,
and NCS--was created to assist VA's CIO Council in its review of
prospective projects for funding. The working group used risk and
return criteria, which included factors such as investment size,
project longevity, technical risk, business impact on mission, and
customer needs.
Figure 1: VA's New Investment
Decision-Making Process
(See figure in printed
edition.)
The working group then forwarded the scored and ranked projects to
VA's CIO Council, which is responsible for ensuring that the
information technology projects are well planned, completely
documented, support the strategic plan and corporate goals, and are
mission critical. After its review, the council forwarded the scored
and ranked projects to VA's Capital Investment Panel. This panel,
which is composed of project decisionmakers representing the VA CFO,
VA CIO, VHA, VBA, VA Office of Planning and Policy, and NCS, was
created to assist VA's Capital Investment Board. The panel reviewed
the projects from the CIO Council and recommended to the VA Capital
Investment Board that these projects be included in the department's
capital investment plan for the upcoming year.\18
The board is composed of the Deputy Secretary, Assistant Secretary
for Management, Assistant Secretary for Policy and Planning, Under
Secretaries for VHA and VBA, and the Director of NCS and is
responsible for making decisions on capital investment projects and
ensuring that the projects conform with VA mission, goals,
priorities, and strategies.
--------------------
\17 GAO/AIMD-10.1.13, February 1997.
\18 VA's Fiscal Year 1999 Department Capital Plan supports VA's
annual budget request and summarizes how each proposal addresses the
Capital Investment Board's criteria.
VA HAS NOT FOLLOWED A
DISCIPLINED PROCESS FOR
SELECTING PROJECTS
---------------------------------------------------------- Letter :6.2
Under the Clinger-Cohen Act, agencies need to compare and prioritize
projects using explicit quantitative and qualitative decision
criteria, such as data on hardware and software life-cycle costs,
technical risks, and mission-related benefits. In conducting their
selection processes, leading organizations assess and manage all
information technology projects, including mission-critical or
infrastructure projects, at all phases of their life cycles, in order
to create a complete strategic investment portfolio and help ensure
that the benefits of their investments will be realized.\19 By
continually scrutinizing and analyzing their entire information
technology investment portfolio, managers can examine the costs of
maintaining existing systems versus investing in new ones and, on the
basis of mission priorities, reach decisions on systems' overall
contributions to organizational goals.
As stated in our investment guide,\20 good decisions require good
data. To help make decisions on information technology investments,
leading organizations require all projects to have complete and
up-to-date project information. This information includes cost and
benefit data, risk assessments, implementation plans, and initial
performance measures. Further, this information allows senior
executives to rigorously evaluate each project, make project
comparisons across the organization, and establish project review
schedules for projects selected for funding in order to monitor and
track project cost, benefits, and risks.
VA's selection criteria requires that relevant reports (e.g.,
congressionally requested audits and studies, VA in-process and
post-implementation review findings), cost-benefit analyses, risk
analyses, and risk management plans, be provided to decisionmakers
reviewing projects for funding.
VA did not follow a disciplined process for selecting its information
technology projects. Specifically, the VA Capital Investment Board
was not provided sufficient data with which to make good funding
decisions. In our analysis of VA's selection process, we examined 7
of 16 projects approved by the board. These seven projects represent
about $223 million or 36 percent of VA's fiscal year 1999 information
technology capital investment budget of $625 million. As shown in
table 1, none of the seven projects we examined contained all the
required information. Further, despite the importance of VA's
Veteran-Focused Information Technology Architecture (ITA) program to
defining and achieving the "One VA" vision, none of the required
information for this project was provided to the board. Nonetheless,
the board decided to fund all seven projects. Further, the board did
not establish a schedule for conducting project reviews, at key
milestones, for each approved project.
Table 1
Summary of Supporting Analyses and
Documentation Presented to the VA
Information Technology Strategic
Planning Working Group
MVR\ HRLink C&P\ IDCU ITA\
ES\a IC\b c $\d e \f g
-------------------------- ---- ---- ---- ------ ---- ---- ----
Relevant reports\h X
Market research results X Z
Benefit-cost analysis X Z X X X
Formal risk analysis X Z X
Risk management plan X Z X
Acquisition strategy/plan X X
----------------------------------------------------------------------
Legend: X represents documents presented to VA's Information
Technology
Strategic Planning Working Group, which was created to assist VA's
CIO Council in its review of the projects.
Z represents a document provided to decisionmakers that contained
elements of these types of analyses and documents.
\a VHA's VA Medical Care Enrollment System.
\b VBA's Information Centers.
\c VA's Master Veteran Record.
\d VA's replacement human resources and payroll system (formerly
known as PAY-VA).
\e VBA's Replacement of the Compensation and Pension Payment System.
\f VA's Integrated Data Communications Utility Follow-On.
\g VA's Veteran-Focused Information Technology Architecture Program.
\h Relevant reports include congressionally requested audits and
studies as well as VA in-process and post-implementation review
findings.
Source: VA. We did not independently assess the quality and
validity of the analyses and documents provided.
Recognizing the weaknesses in the investment selection process, the
Deputy CIO, in an August 4, 1997, memorandum to VA's CIO, recommended
that this process be improved in the next budget cycle. For example,
she recommended that (1) adequate documentation be provided for all
information technology projects and (2) adequate time be provided for
thorough reviews of the documentation prior to scoring and ranking.
Seven months later, the CIO in a memorandum to VA's administration
heads, assistant secretaries, and other key officials, specified that
changes would be made to the department's capital investment process
for the fiscal year 2000 budget cycle to ensure the provision of
adequate documentation and adequate documentary review. In addition,
the memorandum stated that projects with incomplete documentation
will be returned to the originating office for the missing
information. The memorandum did not address what action will be
taken if the missing documentation is not provided.
--------------------
\19 GAO/AIMD-94-115, May 1994.
\20 GAO/AIMD-10.1.13, February 1997.
CONTROL PROCESS DOES NOT
PROVIDE FOR ADEQUATE
MONITORING AND MANAGEMENT OF
INVESTMENTS
---------------------------------------------------------- Letter :6.3
Leading organizations continue to manage their investments once
selection has occurred, maintaining a cycle of continual control and
monitoring.\21 Senior managers review the project at specific
milestones as the project moves through its life cycle and as the
dollar amounts spent on the project increase. At these milestones,
the executives compare the expected costs, risks, and benefits of
earlier phases with the actual costs incurred, risks encountered, and
performance benefits realized to date. This enables senior
executives to (1) identify and focus on managing high-potential or
high-risk projects, (2) reevaluate investment decisions early in a
project's life cycle if problems arise, (3) respond to changing
external and internal conditions in mission priorities and budgets,
and (4) learn from past successes and mistakes in order to make
better decisions in the future. During the control phase, senior
executives determine if projects should be functionally modified,
continued, accelerated, delayed, or terminated. As executives
responsible for implementing legislative reform, it is critical that
senior managers stay actively involved in the process for controlling
information technology projects and receive complete and up-to-date
information related to the projects under review.
To control and monitor its information technology projects, VA relies
on periodic project status reviews and formal in-process reviews.
Periodic project status reviews are conducted at the VA component
level. Formal in-process reviews are conducted at the department
level. According to VA's policy, formal in-process reviews are only
conducted ad hoc, such as when it becomes apparent that a project is
behind schedule, over-budget, not performing as planned, or when
oversight agencies raise issues.
VA's process for monitoring and managing its investment portfolio is
not timely and provides little information to VA decisionmakers.
First, VA does not conduct formal in-process reviews before
significant dollars are expended or substantial risks are
encountered. For example, VA had initially scheduled an in-process
review of a VBA project to replatform and redesign a system that
provides educational benefits to reservists. However, the in-process
review was canceled when the project ran into problems and the
project is now being reassessed. The problems associated with this
project might have been avoided had VA conducted proactive,
risk-based in-process reviews at all critical project milestones.
Second, to the extent that periodic project status reviews and formal
in-process reviews are conducted, the results of the reviews were not
provided to decisionmakers reviewing projects for funding. For
example, of the 15 major ongoing or maintenance projects\22 that the
VA investment board approved for funding, only one, VBA's Replacement
of the Compensation and Pension Payment System project, received a
formal in-process review during fiscal year 1997. However, as shown
in table 1, decisionmakers were not provided with the results of this
review. Therefore, they were not in a position to effectively
monitor and manage this project.
--------------------
\21 GAO/AIMD-94-115, May 1997.
\22 An ongoing project is a project that VA's Capital Investment
Board initially approved as a new project and which is now seeking
out-year funding; a maintenance project is a project that is fully
operational/implemented and in a maintenance mode.
EVALUATION PROCESS DOES NOT
IDENTIFY HOW TO IMPROVE THE
INVESTMENT PROCESS
---------------------------------------------------------- Letter :6.4
Once projects have been implemented and become operational, leading
organizations conduct post-implementation reviews (PIRs) to determine
whether they have achieved expected benefits, such as lowered cost,
reduced cycle time, increased quality, or increased speed of service
delivery.\23 Our information technology investment guide\24 points
out that each PIR should have a dual focus. First, it should provide
an assessment of the implemented project, including an evaluation of
customer/user satisfaction and mission/program impact in terms of
achieving the estimated cost, schedule, and mission-related benefits.
Second, it should provide lessons learned so that the investment
decision-making processes can be improved.
VA has developed a standard methodology for conducting PIRs. This
methodology focuses on elements, such as: (1) customer/user
satisfaction, (2) strategic impact and effectiveness, and (3) impact
on organization's internal operations including security, internal
controls, standards and compliance, and maintenance.
Our review identified deficiencies with VA's process for evaluating
completed projects. First, while the three PIRs VA performed during
fiscal years 1996 and 1997 gathered information on customer/user
satisfaction and discussed development and implementation challenges,
none of them compared actuals to estimates in terms of cost,
schedule, and mission-related benefits.\25 For example, while the
PIRs discuss cost savings, they do not provide information on whether
the projects met, exceeded, or fell short of expectations.
Second, VA did not identify lessons learned that can be used to
improve VA's investment process for selecting, controlling, and
evaluating information technology initiatives. Our review of the
three PIRs VA performed disclosed that the PIRs did identify some
project specific improvements. For example, based on a PIR of VHA's
Integrated Funds Distribution, Control Point Activity, Accounting and
Procurement system, VHA subsequently modified this system to ensure
appropriate security access. However, none of the PIRs assessed the
completed projects to identify improvements that could be made to
VA's information technology investment process.
--------------------
\23 GAO/AIMD-94-115, May 1994.
\24 GAO/AIMD-10.1.13, February 1997.
\25 The three reviews were of VHA's Automated Medical Information
Exchange; VHA's Integrated Funds Distribution, Control Point
Activity, Accounting and Procurement; and NCS' Burial Operations
Support System.
VA'S CIO RESPONSIBILITIES NOT
LIMITED PRIMARILY TO
INFORMATION TECHNOLOGY
MANAGEMENT
------------------------------------------------------------ Letter :7
The Paperwork Reduction Act and the Clinger-Cohen Act direct federal
agency heads to appoint CIOs to (1) promote improvements to the work
processes used by the agency to carry out its programs, (2) implement
an integrated agencywide systems or technology architecture, and (3)
help to establish a sound investment review process to select,
control, and evaluate spending for information technology. To help
ensure that these responsibilities are effectively executed, the
Clinger-Cohen Act also requires that the CIO's primary responsibility
be related to information management.
VA's CIO responsibilities are not limited primarily to information
management. The CIO also serves the department in a variety of top
management positions, including Assistant Secretary for Management,
CFO, and Deputy Assistant Secretary for Budget.
In an agency as decentralized as VA, its CIO is faced with many
significant information management responsibilities, such as ensuring
(1) that the department's operations will not be disrupted by the
Year 2000 problem, (2) that its systems developments are not
handicapped by incomplete architectures, and (3) that a sound
information management investment review process that provides a
systematic, data-driven means of selecting, controlling, and
evaluating information technology projects will be institutionalized.
As we testified in October 1997,\26 each of these responsibilities is
formidable. Taken together, they certainly constitute a full-time
job for any CIO.
We have raised concerns in the past about agencies that have vested
CIO and CFO responsibilities in one person.\27 Agencies face
challenges in improving both financial and information management.
In our opinion, each management area requires full-time leadership by
separate individuals with appropriate talent, skills, and experience
in these two areas. The Clinger-Cohen Act calls for CIOs to have
information resources management as their primary duty. We have
stressed the importance of this principle in testimony and in our
February 1997 high-risk report, in which we emphasized that the CIO's
duties should focus sharply on strategic information management
issues and not include other major responsibilities.\28
In a May 1997 report\29 to OMB, VA's Assistant Secretary for
Management acknowledged that he was the department's CIO as well as
its CFO. He indicated that the VA Secretary felt that assigning
multiple responsibilities to the department's CIO would establish
clear accountability for information resources management activities
at VA, where financial systems represent a substantial part of the
agency's information systems portfolio. However, officials familiar
with the current information management environment at VA and its
components told us that VA's CIO is unable to get involved in the
normal, day-to-day business of a CIO unless a problem arises that
absolutely demands his attention.
Moreover, VA's CIO told us that because he does not have a technical
background in information resources management, he relies on his
deputy. VA's Deputy CIO, however, told us that since she has not
been officially delegated the decision-making authority that the CIO
has, she can not make important information technology decisions
promptly. For example, the Deputy CIO recognized problems VBA was
having with the Veterans Service Network (VETSNET). Consequently,
she wrote a plan to correct the problems and briefed the CIO.
Despite the problems that VETSNET has encountered and the
significance of this project to VA, the CIO has not acted yet on this
plan beyond presenting the plan to the VBA CIO. The Deputy CIO
stated that she does not have the authority to ensure that this
corrective action plan is enacted. As a result, she added, such
issues, when left unaddressed, tend to evolve into different issues
or problems later.
The CIO recently told us that he would soon step down from his
Assistant Secretary for Management/CFO/CIO positions and assume the
position of VA's Deputy Assistant Secretary for Budget. It is not
known at this time whether the new Assistant Secretary will also hold
the CIO and CFO positions.
According to VA's Director of Information Resource Management Policy
and Standards Service, VA recently formed a working group\30 to
determine whether to separate the department's CIO and CFO positions.
The working group has submitted several options on this matter to
VA's Secretary for consideration.
--------------------
\26 GAO/T-AIMD-98-22, October 27, 1997.
\27 GAO/T-AIMD-98-22, October 27, 1997.
\28 Government Reform: Legislation Would Strengthen Federal
Management of Information and Technology (GAO/T-AIMD-95-205, July 25,
1995); Managing Technology: Best Practices Can Improve Performance
and Produce Results (GAO/T-AIMD-97-38, January 31, 1997); High-Risk
Series: Information Management and Technology (GAO/HR-97-9, February
1997); and GAO/T-AIMD-98-22, October 27, 1997.
\29 Progress Report on the Department of Veterans Affairs Chief
Information Officer Program, April 1997.
\30 The working group consists of VA's Chief of Staff, who is also
the chairperson of the working group; VA's Deputy to the Assistant
Secretary for Management; VA's Deputy Assistant Secretary for
Information Resources Management; VA's Deputy General Counsel; and
VA's Deputy Assistant Secretary for Human Resources Management.
CONCLUSIONS
------------------------------------------------------------ Letter :8
VA has not fully implemented critical provisions of the Clinger-Cohen
Act and other information technology legislative reforms to achieve
its "One VA" vision of becoming more customer-focused and delivering
seamless service to veterans. It lacks a departmentwide strategy for
reengineering and improving business processes. As a result,
business process reengineering efforts at the component levels are
uncoordinated, duplicative, and do not provide VA with opportunities
to share information.
Further, while VA recognizes the importance of defining a
departmentwide integrated information technology architecture, it has
not yet done so. Without an integrated architecture, VA will
continue to develop duplicative and redundant information systems and
will not accomplish its vision of "One VA."
In addition, VA has not institutionalized a disciplined investment
management process. Decisionmakers continue to make investment
decisions involving millions of dollars without reliable data on
expected and actual costs, benefits, and risks. Moreover, VA's
process for controlling information technology projects through
periodic status and in-process reviews does not adequately monitor
and manage its investments so as to detect or avoid problems early.
Further, VA's process for evaluating completed projects does not
modify and improve the investment process based on lessons learned.
Finally, given the size of VA's information technology budget and the
many serious information management issues its CIO must face, such as
ensuring that the department's operations will not be disrupted by
the Year 2000 problem, it is important that information resources
management be the CIO's primary duty. A full-time CIO would help
ensure adequate coverage of information management issues.
Information resources management is not the primary duty of VA's CIO.
He also serves as Assistant Secretary for Management, CFO, and Deputy
Assistant Secretary for Budget.
RECOMMENDATIONS
------------------------------------------------------------ Letter :9
We recommend that the Secretary of Veterans Affairs direct the
Assistant Secretary for Policy and Planning to develop a
departmentwide strategy that details how VA will reengineer its
business processes, including identifying and prioritizing process
improvement projects, and delineating their interrelationships.
To fulfill the requirements of the Clinger-Cohen Act and other
information technology legislative reforms, we also recommend that
the Secretary direct VA's CIO to
-- develop a detailed implementation plan with milestones for
completing an integrated, departmentwide information technology
architecture;
-- fully implement a disciplined process for selecting information
technology investments in which all decisions are based upon
complete and current project information including estimated
project costs, expected mission-related benefits, projected
schedule, and risks;
-- conduct formal in-process reviews at key milestones in a
project's life cycle, including comparing actual and estimated
project costs, benefits, schedule, and risks, and provide these
results, as well as the results of periodic project status
reviews performed by VA components, to decisionmakers who will
determine whether to continue, accelerate, or terminate
information technology projects; and
-- initiate post-implementation reviews for information technology
projects within 12 months of implementation, to compare
completed project cost, schedule, performance, and mission
improvement outcomes with original estimates, and provide the
results of these reviews to decisionmakers so that improvements
can be made to VA's information technology investment process.
In addition, we recommend that the Secretary appoint a CIO with
full-time responsibilities for information resources management.
AGENCY COMMENTS AND OUR
EVALUATION
----------------------------------------------------------- Letter :10
In commenting on a draft of this report, the Department of Veterans
Affairs concurred with all six of our recommendations. The
department also stated that it recognizes that its information
resources management challenges are broad and critical to the success
of the department's mission, and, therefore, established the position
of Assistant Secretary to serve as CIO reporting directly to the
Secretary on all information resources issues. This new Assistant
Secretary will be responsible for ensuring that all of the
department's information technology initiatives support the overall
"One VA" vision.
Finally, in concurring with our recommendation to complete an
integrated, departmentwide information technology architecture, the
department did not specify how and when it plans to do so. Until it
completes and implements an integrated architecture, VA will continue
to develop duplicative and redundant information systems and will not
accomplish its vision of "One VA."
--------------------------------------------------------- Letter :10.1
As agreed with your offices, we will not distribute this report until
5 days after its date. At that time, we will send copies to the
Chairman and Ranking Minority Member of the Subcommittee on Oversight
and Investigations, House Committee on Veterans' Affairs; and the
Chairman and Ranking Minority Member of the Subcommittee on Benefits,
House Committee on Veterans' Affairs. We will also provide copies to
the Chairmen and Ranking Minority Members of the Senate and House
Committees on Appropriations; the Secretary of Veterans Affairs; and
the Director of the Office of Management and Budget. Copies will
also be made available to others upon request.
Please contact me at (202) 512-6253 or by e-mail at
willemssenj.aimd@gao.gov if you have any questions concerning this
report. Major contributors to this report are listed in appendix
III.
Joel C. Willemssen
Director, Civil Agencies Information Systems
(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
VETERANS AFFAIRS
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
OBJECTIVES, SCOPE, AND METHODOLOGY
========================================================== Appendix II
Our objectives were to examine how VA has implemented the following
specific provisions of the Clinger-Cohen Act and other legislative
reforms: reengineering business processes before acquiring
information technology, completing an integrated information
technology architecture, institutionalizing a disciplined information
technology investment decision-making process; and appointing an
agency CIO.
In examining VA's reengineering of its business processes, we applied
GAO's guide for business process reengineering.\1 We also reviewed
VA's draft One VA: Vision of Information Technology Enhanced
Customer Service, dated January 22, 1998, and its Strategic
Plan--Fiscal Years 1998-2003, dated September 30, 1997. In addition,
we discussed VA business process revision activities with VA, VBA,
VHA, NCS, and OMB officials.
Regarding VA's information technology architecture, we applied OMB's
Memorandum on Information Technology Architecture, dated June 18,
1997, and the National Institute of Standards and Technology's
Special Publication 500-167, "Information Management Directions: The
Information Challenge." We also reviewed agency documents and
interviewed VA officials on the department's efforts to develop an
integrated information technology architecture.
To assess VA's information technology investment process, we applied
applicable requirements from the Clinger-Cohen Act of 1996, the
Paperwork Reduction Act of 1995, the Government Performance and
Results Act of 1993, the Federal Acquisition Streamlining Act of
1994, the Chief Financial Officers Act of 1990, OMB Circular A-130,
GAO's best practices report on strategic information management,\2
and OMB's guide Evaluating Information Technology Investments: A
Practical Guide. We reviewed and analyzed numerous documents
provided by VA, including its (1) Strategic Plan--Fiscal Years
1998-2003, dated September 30, 1997, (2) Information Technology
Strategic Plan--FY 1999-FY 2003, dated July 1997, (3) Office of
Information Resources Management--IRM Policy and Standards
Service--Information Technology Evaluation Process, dated November 4,
1997, (4) Directive 6000--VA Information Resources Management (IRM)
Framework, dated September 17, 1997, (5) draft Department of Veterans
Affairs FY 1999 Department Capital Plan, dated October 1997, and (6)
VA's Information Technology Strategic Planning, dated March 1997.
In addition, we compared VA's information technology investment plans
and process documents with selected criteria following GAO's guide
for evaluating and assessing federal agencies' selection and
management of information technology resources\3 as well as OMB's
Capital Programming Guide (Version 1.0) Supplement to OMB Circular
A-11, Part 3: Planning, Budgeting, and Acquisition of Capital
Assets, dated July 1997. We also applied criteria from our
investment guide to our review of seven VA information technology
projects approved for funding in VA's fiscal year 1999 budget cycle.
These seven projects were selected based on a variety of factors,
including some of interest to congressional oversight committees,
some that exhibited potential duplication of project functionality,
and the single highest cost departmentwide project. We interviewed
key VA, VBA, VHA, NCS, and OMB officials regarding the department's
information technology investment process.
Finally, to assess VA's implementation of the CIO provision of the
Clinger-Cohen Act, we analyzed (1) VA's April 1997 Progress Report on
the Department of Veterans Affairs CIO Program, (2) VA's strategic,
IRM, and information technology plans mentioned above, and (3) OMB
documentation regarding CIOs. We also interviewed key VA, VBA, VHA,
NCS, and OMB officials regarding the duties and responsibilities of
CIOs.
--------------------
\1 GAO/AIMD-10.1.15, April 1997, Version 3.
\2 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology--Learning From Leading
Organizations (GAO/AIMD-94-115, May 1994).
\3 GAO/AIMD-10.1.13, February 1997.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
Helen Lew, Assistant Director
Nabajyoti Barkakati, Technical Assistant Director
John T. Christian, Senior Business Process Analyst
Mary J. Dorsey, Information Systems Analyst-in-Charge
Michael P. Fruitman, Communications Analyst
Thomas M. McDonald, Senior Business Process Analyst
John P. Rehberger, Senior Information Systems Analyst
John A. Riley, Senior Business Process Analyst
*** End of document. ***