Veterans Benefits Modernization: Management and Technical Weaknesses Must
Be Overcome If Modernization Is To Succeed (Testimony, 06/19/96,
GAO/T-AIMD-96-103).
GAO discussed the Veterans Benefits Administration's (VBA) efforts to
modernize and streamline its business processes. GAO noted that: (1) VBA
is experiencing information technology acquisition problems and serious
management and technical weaknesses; (2) VBA needs to adopt a clearly
articulated business strategy to solve its service-delivery problems,
coordinate its reengineering efforts, and cope with constrained
resources; (3) VBA is developing strategic and information resources
management plans that it will use in preparing its fiscal year 1998
budget request; (4) the National Academy of Public Administration will
assess ways to make VBA claims processing more efficient; (5) VBA needs
to develop performance goals and measures aimed at improving the quality
of service; (6) VBA is reengineering its compensation and pension
business processes, but it must improve its management of its
information technology investments by developing critical cost, benefit,
and risk information; (7) VBA has acquired some computer equipment
before completing its reengineering analysis and may have to discard the
equipment; and (8) VBA is not following sound software and systems
development practices.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-AIMD-96-103
TITLE: Veterans Benefits Modernization: Management and Technical
Weaknesses Must Be Overcome If Modernization Is To
Succeed
DATE: 06/19/96
SUBJECT: Reengineering (management)
Claims processing
Veterans benefits
ADP procurement
Requirements definition
Customer service
Information resources management
Management information systems
Strategic information systems planning
Cost effectiveness analysis
IDENTIFIER: VA Benefit Delivery Network
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Before the Subcommittee on Compensation, Pension,
Insurance and Memorial Affairs
Committee on Veterans' Affairs
House of Representatives
For Release on Delivery
Expected at
10:00 a.m.
Wednesday,
June 19, 1996
VETERANS BENEFITS MODERNIZATION -
MANAGEMENT AND TECHNICAL
WEAKNESSES MUST BE OVERCOME IF
MODERNIZATION
IS TO SUCCEED
Statement of Gene L. Dodaro
Assistant Comptroller General
Accounting and Information Management Division
GAO/T-AIMD-96-103
GAO/AIMD-96-103T
(511204)
Abbreviations
=============================================================== ABBREV
============================================================ Chapter 0
Mr. Chairman and Members of the Subcommittee:
I am pleased to have this opportunity to discuss the Veterans
Benefits Administration's (VBA) efforts to modernize its operations
by streamlining its business processes and taking greater advantage
of information technology. Successfully meeting this challenge is
key to improving critical services to almost 27 million veterans and
their dependents and lowering VBA's operating costs.\1 VBA is,
however, experiencing many of the classic, fundamental management and
technical problems that have prevented federal agencies from
realizing the benefits expected from substantial investments in
information technology. While it is clear that dedicated employees
at VBA who care about veterans and service delivery are working to
overcome these difficulties, major, sustained improvements in
critical service delivery and operating performance have eluded VBA.
Unless VBA takes more aggressive action to effectively address its
serious management and technical weaknesses, its modernization
efforts will not succeed. My testimony today will focus on what VBA
needs to do in three major areas to increase the likelihood of
success. These are:
-- creating a credible business strategy and supporting information
resources management plan;
-- developing a much improved investment strategy for selecting and
managing its portfolio of information technology projects in a
more disciplined, businesslike manner; and
-- strengthening its technical capability to develop software
applications that will be critical to supporting efforts to
improve service to veterans and control costs.
--------------------
\1 VBA is one of three major components of the Department of Veterans
Affairs (VA), which also includes the Veterans Health Administration
and the National Cemetery System.
BUSINESS STRATEGY NEEDED TO
SOLVE ENTRENCHED
SERVICE-DELIVERY PROBLEMS
---------------------------------------------------------- Chapter 0:1
VBA's business environment encompasses many difficult challenges.
These include a backlog of disability claims, improving a number of
relationships with other organizations that affect how VBA does its
work, and responding to its customers who are frustrated about the
long-standing need to improve the accuracy and timeliness of
processing claims. To deal with these issues, as well as cope with
today's constrained budgetary climate, the agency is undertaking a
number of major initiatives, including beginning a business process
reengineering effort for its compensation and pension programs,\2
restructuring its regional office responsibilities, and consolidating
its data centers.
VBA has, however, been proceeding without an overall business
strategy clearly setting forth how it will improve its performance
and tackle entrenched service-delivery problems. For example, the
reported backlog of original and reopened disability claims increased
from 378,000 in fiscal year 1990 to a high of 571,000 at the end of
December 1993. This rise was due to several factors, including
increasing complexity in claims processing and the use of
inexperienced regional claims raters. VBA instituted several
conventional stopgap measures to deal with this backlog. It
authorized extensive overtime, shifted workloads among regional
offices, purchased information technology equipment, increased the
number of claims raters by about one third (from 667 to 897), and
relaxed some paperwork requirements, such as accepting photocopies of
certain documents. As a result the backlog has been reduced, but it
is now still about 380,000--similar to the 1990 level.
Similar trends have been experienced in the processing times for
original disability compensation claims, which rose from an average
of 151 days in fiscal year 1990 to 213 days in fiscal year 1994. The
stopgap measures used to decrease the backlog have also reduced the
average processing time in fiscal year 1995 to 161--10 days more than
the level in fiscal year 1990. VBA officials acknowledge that these
measures cannot be sustained over a prolonged period of time. VBA
must, therefore, find other solutions to achieve greater
service-delivery breakthroughs.
Other entities also affect the speed with which VBA processes claims
and the agency's overall direction. For example, VBA relies on the
Veterans Health Administration for most medical information needed to
substantiate a disability claim, and the Department of Defense for
information relating to a veteran's service time and conditions of
discharge, as well as medical information from the veteran's tour of
active duty. Delays by either of these organizations can have a
significant impact on the timeliness of VBA's claims processing.
Judicial review organizations also affect VBA's workload and backlog.
For example, the Board of Veterans' Appeals returns almost half of
its cases to VBA regional offices for additional development and
reconsideration each year. The Board itself also has a significant
and increasing backlog of cases; its appeals grew from about 19,500
in fiscal year 1990 to more than 50,000 in fiscal year 1995--an
increase of more than 150 percent. It takes the Board about 2 years
to render a decision from the date it receives an appeal.
In addition, VBA--like most federal agencies--must deal with
constrained resource levels and, at the same time, maintain existing
levels of service and operations. VBA is in the process of
restructuring its regional offices in an effort to cope with
declining resources. At the same time, funding for VBA's information
technology initiatives is discretionary and, as such, comes under
close budgetary scrutiny by the Congress and the Office of Management
and Budget (OMB).\3
A comprehensive business strategy is needed--one that includes
developing strategic and information resources management plans,
setting performance goals and measures, and incorporating the results
of major agency initiatives, such as business process reengineering.
VBA is moving in this direction; currently, however, it has no
clearly articulated business strategy.
Recent legislative changes provide the framework for VBA to develop
such a strategy and identify the tools needed to implement it. For
example, the Government Performance and Results Act of 1993\4
requires agency heads to submit to OMB and the Congress a strategic
plan for program activities, including a mission statement, goals and
objectives, and a description of how these will be achieved and what
key factors could affect their achievement. The act also requires
that agencies prepare annual performance plans for each
program--performance indicators that will allow measurement of
outputs and service levels. In addition, the Information Technology
Management Reform Act of 1996\5 requires agency heads to establish
goals for improving the efficiency and effectiveness of agency
operations and, as appropriate, the delivery of services to the
public, through more effective use of information technology and
business process reengineering.
VBA's weaknesses in planning have been well documented since 1987.
VBA's planning process has been cited by us and others for (1) not
having specific, measurable goals and objectives against which
progress can be assessed and (2) not analyzing the costs and benefits
of alternative approaches to modernization. According to VBA
officials, they are in the process of developing strategic and
information resources management plans and will have them ready to
use in preparing the agency's budget submission for fiscal year 1998.
Assistance in this area could come from the National Academy of
Public Administration, which has recently been commissioned by the
Senate Appropriations Committee. In the Committee's September 1995
report on the 1996 appropriations bill,\6
the Committee provided $1 million to the Academy for a comprehensive
assessment of VBA, with particular emphasis on the specific steps
required to make claims processing more efficient and less
time-consuming. The Academy will evaluate the modernization
initiative and its link to strategic goals and priorities, efforts to
reengineer VBA's claims-processing methodology, performance measures
for restructuring, and the roles of the Board of Veterans' Appeals
and the Court of Veterans Appeals. As of a few weeks ago, VBA was
still working out the details of this study with the Academy.
VBA also needs to develop a full set of performance goals or
measures. At present, processing timeliness is the primary
performance measure that VBA uses. Customer-focused goals, aimed at
improving the quality of service, are needed. For example, a VBA
survey of "stakeholders"\7 indicated that , in their view, an
emphasis on quality over productivity alone would be the key to
service excellence at VBA. These stakeholders defined quality as
making the correct award decision the first time, which would improve
the timeliness of claims processing and reduce the number of appeals
filed.
VBA's current goal for claims processing was set without the benefit
of any clear plan. For example, its goal is to reduce average
original compensation claims processing time to 106 days by 1998;\8
this goal was set as part of a 1993 agreement with OMB to establish
outcome-oriented performance goals. The performance goal is not
linked to a business strategy or plan that explains how the agency
intends to achieve this goal.
Reengineering is key to achieving major performance improvements that
VBA establishes as business goals. As our 1994 study pointed out,\9
organizations that successfully develop information systems do so
only after thoroughly analyzing and redesigning their current
business processes. Information system projects that do not first
consider business process redesign typically fail, or reach only a
fraction of their full potential.
In response to concerns raised by us and others over the past 3
years, VBA is preparing to reengineer its compensation and pension
claims-processing operations, and has taken several positive steps.
In November 1995 the agency established a Business Process
Reengineering Office, and subsequently adopted a business process
reengineering methodology. It also hired a consultant to assist with
reengineering. By the end of this month, a business process
reengineering team comprised of VBA staff and the consultant is
expected to have completed a key step in the process by developing a
proposal for changing the compensation and pension business
processes. This proposal will be submitted to VBA management for
review and approval before implementation. VBA also plans to begin a
different business analysis project each year for its other four
business areas. The next area planned for such an analysis is
educational assistance.
It is still too early to judge whether the current business process
reengineering effort will help VBA achieve its goals, but we continue
to have some concerns about VBA's current business process
reengineering focus and approach. For example, VBA has not yet set
quantifiable performance measures using the experiences and
performance of other leading claims-processing organizations. Also,
the scope of VBA's analysis and reengineering of its business
processes in the compensation and pension area does not address the
claims appeal process, which has a significant impact on the
timeliness and quality of some claims-processing decisions. Finally,
as I will discuss later, we are concerned that reengineering is not
the driver behind all of VBA's information technology initiatives.
To solve entrenched problems and sustain long-term improvements in
service delivery and operations, VBA must first know exactly what it
needs to pay attention to and where it wants to go. A business
strategy containing specific goals and performance measures is
absolutely essential. By effectively using the framework established
in recent legislation to develop the business strategy and complete
its strategic and information resources management plans, VBA will go
a long way toward setting out a clear path to be followed.
--------------------
\2 VBA has five major business areas: compensation and pension (the
largest), loan guarantee, vocational rehabilitation and counseling,
educational assistance, and insurance.
\3 The Budget Enforcement Act of 1990 discusses required reductions
in budget authority and outlays, while the Federal Workforce
Restructuring Act discusses limitations on personnel levels.
\4 Public Law 103-62; 5 U.S.C. 306 and 31 USC 1115; Aug. 3, 1993.
\5 Public Law 104-106, Division E, Feb. 10, 1996.
\6 Senate Report 104-140.
\7 Stakeholders included veterans, VA employees, top VA management,
congressional veterans' committees, agencies such as GAO and OMB,
veterans' service organizations, the Social Security Administration,
and components of the Department of Defense.
\8 According to the Secretary of Veterans Affairs, an interim goal is
117 days by 1997.
\9 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology--Learning From Leading
Organizations (GAO/AIMD-94-115, May 1994).
MANAGING INFORMATION TECHNOLOGY
IS ESSENTIAL TO VBA'S SUCCESS
---------------------------------------------------------- Chapter 0:2
VBA's investment in modernization activities has yielded some
improvement in hardware and software applications. However, it is
difficult to measure return on any of these investments.
As shown in attachment 1, between fiscal years 1986 and 1995, VBA
reported that it obligated about $688 million for information
technology, of which about $284 million, or about 40 percent, was for
systems modernization. In December 1992 VBA awarded the first
contract in its planned three-stage procurement. During stage I, VBA
acquired a number of personal computers, local area networks,
minicomputers, and commercial off-the-shelf software for its 58
regional offices; during stage II, VBA procured imaging equipment and
associated software. Stage III was suspended in 1994; during this
stage, VBA was to procure mainframe computers for its data centers in
Hines, IL, and Philadelphia.
VBA has also realized some limited benefits from the development of
several short-term, targeted software applications that are being
used on equipment acquired during stage I. These projects include
the following:
-- Control of Veterans Records--used to track the location of
veterans' claims folders containing application-related
information;
-- Rating Board Automation--used to generate letters to veterans
regarding award decisions; and
-- Personal Computer-Generated Letters--used to prepare general
letters to disability claimants.
To help manage its information technology investments in a way that
will lead to major returns, VBA must now meet the challenges of new
information technology legislation that has been modeled after the
best practices of leading private and public organizations. For
example, the Information Technology Management and Reform Act and the
Paperwork Reduction Act require agency heads to
-- analyze the agency's mission and, on the basis of this analysis,
revise business processes as appropriate;
-- design and implement a process for maximizing the value and
assessing and managing the risks of information technology
acquisitions;
-- integrate budgetary, financial, and program management decisions
in this process; and
-- use this process to select, control, and evaluate the results of
information technology initiatives.
VBA needs to make major improvements in the way it manages its
information technology investments to meet these legislative
requirements. Our analysis of past and current VBA information
technology initiatives shows that VBA lacks the critical cost,
benefit, and risk information necessary to determine whether it has
made worthwhile investments. Our analysis also shows that these
initiatives preceded VBA's business process reengineering effort,
which increases the risk that they may need to be substantially
changed or abandoned once reengineering results become available.
For example:
-- Between fiscal years 1993 and 1995, VBA purchased 24
minicomputers without having a clear understanding of the
software applications to be placed on the equipment or the
benefits to be derived from this investment. Although VBA
expected to use these minicomputers in processing claims, they
were not put into use until recently, when VBA began testing its
software application to track claims folders. This was done at
four sites: Baltimore; St. Petersburg; San Juan, Puerto Rico;
and Winston-Salem, NC.
-- At VBA's educational assistance processing sites in Atlanta and
St. Louis, the agency has acquired and is in the process of
installing imaging equipment to scan all documents in the
chapter 30\10 education claims folders, which contain an average
of 30 documents each. VBA has not, however, performed any
reengineering analysis for the educational assistance area to
assess how the imaging equipment could be used to improve
education claims processing. In addition, while VBA has begun
to collect baseline information to compare against
post-implementation data in order to determine what impact the
equipment will have on its operations at the Atlanta site, such
information has not been collected for St. Louis, which has
been using such equipment since 1987.
Also, this past March VBA embarked on a 2-year effort at its St.
Petersburg regional office to replace its current benefits payment
system. The objectives of this replacement system were to (1) permit
more timely updating of master benefit files through on-line access,
(2) provide national access to service organizations that must
respond to veterans' questions about the status of their claims, and
(3) address the potential effects of processing benefits payments and
other critical information after the turn of the century.
This recent project has several inherent risks that must be assessed
before VBA can determine if this initiative will be worth the
investment. First, the project team, comprised of VBA staff and
contractor personnel, will be using a new software development
language\11 and a rapid application development methodology.\12 While
this methodology is used more frequently in the private sector, it
has not been previously used at VBA. When it is used, highly skilled
and experienced people are a necessity. Given both VBA's and the
contractor's unfamiliarity with using this methodology, the staff and
contractor must learn the new tools and become proficient with them
so as not to jeopardize the implementation of the replacement payment
system, scheduled for 1998.
We believe that this initiative is high risk because the payment
replacement system timetable was based on unrealistic assumptions
about the productivity and skills of newly-trained, inexperienced
people, and the level of complexity of the task. Further, as I will
discuss in more detail in a few moments, although VBA is in the
process of developing software for its replacement system, our
evaluation found that VBA is very weak in its ability to develop
software and manage software-development contracts. This factor
substantially increases the risks associated with this project.
Another risk is that this project was not following sound
systems-development practices. For example, VBA's system development
guidelines--policies and procedures used to design and develop
computer software and systems--call for verification and validation
of the system requirements before proceeding from one phase of system
development to the next phase. VBA's implementation of the standard
systems-development process consists of four phases: planning,
analysis, design, and construction. It has been demonstrated that
proceeding to a subsequent phase without reviewing the work done in
the current phase for correctness, consistency, and completeness will
almost always adversely impact on the project's cost, its
performance, and the delivery schedule. VBA directed the project
team to proceed into the system design phase, however, without
completing this important first step. Further, the data model\13
that is being used to develop the replacement payment system has not
been completed, although this should have been done prior to
proceeding into the system design phase. The incomplete requirements
verification and validation and incomplete data model increases the
risk that the system will be designed incorrectly. Also, VBA does
not have cost-benefit information with which to assess its return on
this investment. For example, it has not estimated the total amount
of software that must be developed, or its cost.
In addition to lacking the information to determine whether or not
specific projects will pay off, VBA also lacks a process that ranks
and prioritizes its investments in information technology as a
consolidated portfolio. VBA is undertaking several projects
simultaneously, without a full consideration of the resources
required, costs, risks, and potential impact on agency operations.
Current system-development activities--including addressing the
year-2000 issue, data-center consolidation and related software
conversion, and replacement of the benefits payment system--are all
examples of investments that have not been ranked or prioritized.
Year 2000. Like all other federal agencies--and private
businesses--VBA must address the effects of processing information in
light of the change of century. Most of the computer software in use
today employs 2-digit date fields. Consequently, at the turn of the
century, computer software will be unable to distinguish between the
years 1900 and 2000, since both would be designated "00." Industry
and government experts have already gone on record saying that the
effort to correct this problem will become extremely costly and
time-consuming, and requires early and detailed planning. If the
year-2000 problem is not addressed, it will render the vast majority
of date-sensitive computer information unusable or obsolete. For
example, calculations based on incorrect dates in service could
result in errors in processing benefit checks in the compensation and
pension programs. In VBA's educational assistance program, VBA could
-- send threatening debt-collection letters to veterans who do not
actually owe money;
-- charge incorrect interest rates to veterans or charge interest
to veterans who do not owe money; or
-- send debtor information to the Internal Revenue Service for
refund withholding, to the federal government for wage
garnishment, or to private credit firms to go on a veteran's
credit report.
In our opinion, the year-2000 issue is an absolutely critical
challenge that VBA faces over the next 2-3 years. Some of the
computer code was developed more than 20 years ago, using nonstandard
coding techniques. In some cases, the software documentation may be
incomplete or nonexistent.
It is essential that VBA develop and implement a strategy to address
the inherent risks that accompany the year-2000 change. First, a
sufficient number of experienced staff must be devoted to this task,
especially since VBA must maintain its current software and service
levels at the same time that it is correcting date-sensitive code.
Second, it will need to complete the programming by 1998, since
industry experts recommend that 1999 be reserved for thoroughly
testing the year-2000 changes. Third, VBA must have a contingency
plan that outlines alternatives for processing claims if systems are
not corrected.
Data-Center Consolidation and Related Software Conversion. In
response to a request from OMB,\14 VA and VBA are in the process of
developing a strategy paper to reduce operational costs by
consolidating their data centers. However, critical information in
terms of costs and benefits is missing--information needed to
determine how and when this should be done and how this effort ranks
in terms of priority with competing demands, such as the year-2000
activities.
Currently, VA's data center is in Austin, Texas, and uses IBM
computer equipment to process the Department's accounting and
financial management information related to administrative
operations. VBA's two data centers--Hines and Philadelphia--use
mostly Honeywell equipment; the Hines facility primarily processes
disability (compensation and pension) claims, while Philadelphia
processes insurance claims. The joint VA/VBA data-center
consolidation strategy paper is due to OMB in July.
Because the data-center consolidation approach must also consider
converting the current software to run on more modern computer
equipment, added risks must be considered. Specifically, VBA is
considering converting the Benefits Delivery Network\15
software--currently in use at Hines--to more modern computer
equipment. The cost and time frames for this conversion will depend
upon which of the three data centers is chosen as the site for
Benefits Delivery Network processing. To date, two studies have been
commissioned to evaluate the software conversion. The first,
commissioned by VA, estimated the cost and time frames for moving the
current Benefits Delivery Network to IBM equipment; the second,
commissioned by VBA, assessed the feasibility of converting the
Benefits Delivery Network software. The finding was that such a
conversion is feasible, and could likely take 2-3 years to complete.
Neither study, in our view, provides enough information on all three
sites to adequately assess the investment needed, nor do they fully
address General Services Administration (GSA) criteria\16 for making
software conversion decisions. Neither contains an analysis of
alternative approaches or a full description of the cost, benefits,
and risks of conversion. We have discussed our analysis with VA and
VBA officials, and they agree with our assessment of these studies.
VA has since hired another consultant to analyze the costs and
benefits and to develop a strategy for data-center consolidation.
Until the results of this study are available, VBA will not be able
to identify the best approach to take.
The conversion of the Benefits Delivery Network software must be
carried out correctly in order to realize the potential benefits of
data-center consolidation. This conversion will require much work
and a dedicated staff with in-depth knowledge of the existing network
software. In-depth knowledge of the Benefits Delivery Network
software currently resides at VBA's Hines data center. It will also
be necessary, despite limitations on personnel and funding, to
maintain the current network software and service level of operations
while converting the software. The conversion risk will be further
compounded by VBA's need to address the year-2000 issue.
Replacement of the Payment System. In addition to the previously
mentioned risks associated with the replacement of the payment
system, we believe that VBA did not adequately consider alternative
approaches for achieving the reliability and additional functionality
expected in the replacement. The Federal Information Resources
Management Regulations require that agencies use their systems
requirements as the basis for analyzing alternatives, commensurate
with the size and complexity of the agency's business needs. The
regulation stipulates that agencies should calculate the total
estimated cost of each feasible alternative, and assess the risks.
Further, VBA recently acquired excess computing equipment from GSA to
replace some of the equipment at Hines and Philadelphia. According
to staff at both centers, the excess equipment is more reliable, has
greater capacity, and is less expensive to maintain. This newer
equipment allows VBA more time to analyze and assess alternatives
because it makes the computing environment more stable.
Lastly, critical to VBA's ability to identify the true return on any
of these information technology initiatives is the need for accurate
and reliable cost information. Our analysis of VBA's modernization
obligations to date shows that the cost of these activities may be
understated because VBA lacks a managerial cost-accounting system to
track payroll benefits and indirect costs associated with
modernization. VBA also appears to have miscategorized some items in
its information technology budget as nonmodernization items when, in
our opinion, they were modernization-related and should have been
categorized in that way. In addition, VBA has not updated its
modernization life-cycle cost estimate of $478 million in over 3
years. Therefore, precisely how much VBA's systems modernization
effort will ultimately cost taxpayers remains uncertain. VBA's chief
financial officer is currently in the process of developing guidance
for implementing a cost-accounting methodology.
Our work indicates that VBA has much to do to develop an investment
strategy that can assure the Congress that scarce information
technology dollars are being spent on the highest priority projects
with the greatest potential for a substantial return on investment.
The recent acquisition of excess equipment now provides VBA with an
opportunity to effectively develop this kind of approach. VBA must
-- expeditiously develop an effective investment process for
selecting, controlling, and evaluating information technology
initiatives in terms of cost, capability of the system to meet
requirements, risk, timeliness, and quality;
-- give top priority to addressing the year-2000 problem; and
-- improve its accounting of obligations and costs associated with
the modernization.
--------------------
\10 Chapter 30 relates to the Montgomery GI bill which provides
education benefits for veterans on active duty after July 1, 1985.
\11 Called JAM7, this commercial, off-the-shelf, 4th-generation
software development tool is designed to be used with relational
database management and transaction processing systems. Relational
database management refers to a method of organizing data elements so
that a specific, defined relationship exists among those
elements--such as an individual's social security number being
related to only one specific name.
\12 An application development methodology that emphasizes
prototyping and the use of advanced tools. A critical element of
this methodology is rigid adherence to a schedule.
\13 A graphical representation of data and its interrelationships.
Data models are used to specify database requirements.
\14 OMB Bulletin No. 96-02 (Consolidation of Agency Data Centers),
October 4, 1995.
\15 VBA's existing computer and software infrastructure for
processing claims.
\16 GSA's Preparing Software Conversion Studies, OIT/FCSC-84/001,
January 1984, updated December 1989.
SOFTWARE DEVELOPMENT FOUNDATION
TO EXECUTE TECHNOLOGY
INVESTMENT NEEDS URGENT
ATTENTION
---------------------------------------------------------- Chapter 0:3
Once technology investment processes have identified the most
beneficial information technology projects in terms of cost, benefit,
and return, the focus then shifts to the technical capabilities
necessary to make the projects a reality. The agency must be able to
quickly determine if it has the necessary in-house capability to
develop the software for the new system or whether this development
should be performed by an experienced contractor.
In order to mitigate any risk of not being able to deliver
high-quality software within schedule and budget, agencies must have
a disciplined and consistent software-development process. Software
development has been identified by many experts as one of the most
risky and costly components of systems development.
To evaluate VBA's software development processes, we applied the
Software Engineering Institute's\17 software capability evaluation
methodology to those projects identified by VBA as using the best
development processes. This evaluation compares agencies' and
contractors' software development processes against the Institute's
five-level software capability maturity model, with 5 being the
highest level of maturity and 1 being the lowest. As shown in
attachment 2, these levels--and the key process areas described
within each--define an organization's ability to develop software,
and can be used to measure improvements in this area.
On the basis of our analysis, we determined that VBA is operating at
a level-1 capability, defined as ad hoc and chaotic. At this level,
VBA cannot reliably develop and maintain high-quality software on any
major project within existing cost and schedule constraints, placing
VBA modernization at significant risk. In this context, VBA relies
solely on the various capabilities of individuals rather than on an
institutional process that will yield repeatable, or level-2,
results. VBA does not satisfy any of the criteria for a level-2
capability, the minimum level necessary to be able to significantly
improve productivity and return on investment. For example, VBA is
weak in the requirements management, software project planning, and
software subcontract management areas, with no identifiable strengths
or planned improvement activities. However, VBA can build upon its
strengths in the software configuration-management and software
quality-assurance areas.
Our report on this matter is being issued soon and will contain
recommendations to better position VBA to develop and maintain its
software successfully and to protect its software investments.
Specifically, we recommend in that report that VBA
-- obtain expert advice to improve its ability to develop
high-quality software and expeditiously implement a plan that
describes a strategy for reaching the repeatable (i.e., level 2)
level of process maturity,
-- delay any major investment in new software development--beyond
what is needed to sustain critical day-to-day operations--until
the repeatable level of process maturity is attained, and
-- ensure that any future contracts for software development
require the contractor to have a software development capability
of at least a level 2.
VBA agreed with all but one recommendation. VBA agreed that a
repeatable level of process maturity is a goal that must be attained,
but disagreed that "all software development beyond that which is
day-to-day critical must be curtailed." VBA stated that the payment
system replacement projects and other activities to address the
change of century must continue. We agree that the software
conversion and development activities required to address issues such
as the year 2000 must continue; we would, in fact, characterize these
as sustaining critical day-to-day operations. However,
systems-development initiatives in support of major new projects,
such as the replacement of the payment system, should be reassessed
for the risk of potential delays, cost overruns, and shortfalls in
anticipated system functions and features. We are pleased to see
that VBA is already initiating positive actions relating to our other
recommendations, including acquiring expert advice to assist it in
improving its ability to develop high-quality software, consistent
with criteria set forth by the Software Engineering Institute.
--------------------
\17 This is a nationally recognized, federally funded research and
development center established at Carnegie Mellon University in
Pittsburgh, to address software development issues.
CONCLUSIONS
---------------------------------------------------------- Chapter 0:4
The business and operational problems facing VBA are complex and not
easy to resolve. VBA has begun to take action to improve agency
operations and service delivery, but it has not yet implemented
enough of the right kinds of actions--actions that involve developing
a sound business strategy and the supporting plans, approaches, and
measures to guide them into the next century. The need for more
rigorous management and technical methods is critical if VBA is to
successfully develop modern, efficient, and cost-effective business
processes and computer systems that will allow them to deliver truly
improved services to veterans.
Mr. Chairman, this completes my testimony this morning. I would be
pleased to respond to any questions you or other members of the
Subcommittee may have at this time.
ATTACHMENTS
============================================================ Chapter 1
(See figure in printed
edition.)
*** End of document. ***