Software Capability Evaluation: VA's Software Development Process Is
Immature (Letter Report, 06/19/96, GAO/AIMD-96-90).
GAO reviewed the development of computer software at the Veterans
Benefits Administration (VBA) and at the Department of Veterans Affairs'
Austin Automation Center. Neither VBA nor the Austin Automation Center
had satisfied any of the key process areas required for a level 2
capability, as described by the Software Engineering Institute's
five-level software capability model. Under a level 2 capability, basic
project management processes are established to track cost, schedule,
and functionality. Moreover, the necessary process discipline is in
place to repeat earlier successes on projects with similar applications.
The upshot is that the Department of Veterans Affairs (VA) has little
assurance that (1) investments in new software development will achieve
their objectives or (2) software will be delivered consistent with cost
and schedule estimates. GAO recommends that VA delay any major
investment in software development beyond what is needed to sustain
critical day-to-day operations until the repeatable level of process
maturity is achieved. VA should also obtain expert advice on how to
improve the ability of VBA and the Austin Automation Center to develop
high-quality software, develop a strategy to reach the repeatable level
of process maturity, implement that strategy quickly, and ensure that
any future contracts for software development require that the
contractor have a software development capability of at least level 2.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-96-90
TITLE: Software Capability Evaluation: VA's Software Development
Process Is Immature
DATE: 06/19/96
SUBJECT: Research and development contracts
Veterans benefits
Computer networks
Application software
Contract administration
Cost control
Requirements definition
Information systems
IDENTIFIER: VA Veterans Integrated Service Network
VA Benefit Delivery Network
Software Capability Maturity Model
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman, Subcommittee on Compensation, Pension,
Insurance and Memorial Affairs, Committee on Veterans Affairs, House
of Representatives
June 1996
SOFTWARE CAPABILITY EVALUATION -
VA'S SOFTWARE DEVELOPMENT PROCESS
IS IMMATURE
GAO/AIMD-96-90
VA's Software Capability Evaluation
(511191)
Abbreviations
=============================================================== ABBREV
AAC - Austin Automation Center
CMM - Capability Maturity Model
KPA - key process area
SCE - software capability evaluation
SEI - Software Engineering Institute
SIS - Systems Integration Service
SQA - software quality assurance
SQ&C - software quality and control
VA - Department of Veterans Affairs
VBA - Veterans Benefits Administration
VETSNET - Veterans Services Network
Letter
=============================================================== LETTER
B-261652
June 19, 1996
The Honorable Terry Everett
Chairman, Subcommittee on Compensation,
Pension, Insurance and Memorial Affairs
Committee on Veterans Affairs
House of Representatives
Dear Mr. Chairman:
You requested that we review the Department of Veterans Affairs' (VA)
capability for developing and maintaining software for its
computer-based systems. The Department depends on these systems to
support program operations and assist in managing the achievement of
critical mission objectives. Our objective was to review software
development processes at the Veterans Benefits Administration (VBA)
and VA's Office of Information Resources Management's Austin
Automation Center (AAC). The sites and projects were selected by VBA
and VA, respectively, as those that represent their best software
development processes and practices.
BACKGROUND
------------------------------------------------------------ Letter :1
VBA is in the process of modernizing many of its older, inefficient
systems and has reportedly spent an estimated $294 million on these
activities between October 1, 1986 and February 29, 1996. The
modernization program can have a major impact on the efficiency and
accuracy with which over $20 billion in benefits and other services
is paid to our nation's veterans and their dependents. However, in
the last 6 years some aspects of VBA's service to the veterans have
not improved. For example, in the past 6 years, VBA's reported
processing time for an original compensation claim rose from 151 days
in fiscal year 1990 to 212 days in fiscal year 1994. In March 1996
the average time was 156 days.
Software development is a critical component of this major
modernization initiative. VBA, with the assistance of contractors,
will be developing software for the Veterans Services Network
(VETSNET) initiative, a replacement for the existing Benefit Delivery
Network. For efforts like VETSNET to succeed, it is crucial that VBA
have in place a disciplined set of software development processes to
produce high quality software within budget and on schedule.
VBA relies upon its own staff and contractors to develop and maintain
software that is crucial to its overall operations. In fiscal year
1995, VBA had 314 full-time equivalents, with payroll expenses of
$20.8 million, devoted to developing and maintaining software
throughout the organization. It also spent $17.7 million in contract
services in these areas.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :2
To evaluate VA's software development capability, version 2.0 of the
Software Engineering Institute's (SEI)\1 software capability
evaluation (SCE) method\2
was used by an SEI-trained team of GAO specialists. The SCE is a
method for evaluating agencies' and contractors' software development
processes against SEI's five-level software Capability Maturity Model
(CMM), as shown in table 1. These levels and the key process areas
(KPAs) described within each level define an organization's ability
to develop software, and can be used to improve its software
development processes. The findings generated from an SCE identify
(1) process strengths that mitigate risks, (2) process weaknesses
that increase risks, and (3) improvement activities that indicate
potential mitigation of risks.
Table 1
CMM Levels and Descriptions
Level Name Description
---------- -------------- ------------------------------------------
5 Optimizing Continuous process improvement is enabled
by quantitative feedback from the process
and from piloting innovative ideas and
technologies.
4 Managed Detailed measures of the software process
and product quality are collected. Both
the software process and products are
quantitatively understood and controlled.
3 Defined The software process for both management
and engineering activities is documented,
standardized, and integrated into a
standard software process for the
organization. All projects use an
approved, tailored version of the
organization's standard software process
for developing and maintaining software.
2 Repeatable Basic project management processes are
established to track cost, schedule, and
functionality. The necessary process
discipline is in place to repeat earlier
successes on projects with similar
applications.
1 Initial The software process is characterized as
ad hoc, and occasionally even chaotic. Few
processes are defined, and success depends
on individual effort.
----------------------------------------------------------------------
Source: Capability Maturity Model for Software, Version 1.1,
(Technical Report CMU/SEI-93-TR-24, February 1993).
We requested that VA identify for our evaluation those projects using
the best software development processes implemented within VBA and
AAC. VBA and AAC identified the following sites and projects.
-- VBA
Hines, IL
--Compensation & Pension/Financial Management System
--Claims Processing System
--Rating Board Automation
Philadelphia, PA
--Variable Loan Rate
Washington, DC
--Veterans Services Network
-- AAC
Austin, TX
--On-Line Data Entry
--Personnel Accounting Integrated Data
We evaluated the software development processes used on these
projects, focusing on KPAs necessary to achieve a repeatable
capability. Organizations that have a repeatable software
development process have been able to significantly improve their
productivity and return on investment. In contrast, organizations
that have not developed the process discipline necessary to better
manage and control their projects at the repeatable level incur
greater risk of schedule delay, cost overruns, and poor quality
software.\4 These organizations rely solely upon the variable
capabilities of individuals, rather than on institutionalized
processes considered basic to software development.
According to SEI,\5 processes for a repeatable capability (i.e., CMM
level 2) are considered the most basic in establishing discipline and
control in software development and are crucial steps for any project
to mitigate risks associated with cost, schedule, and quality. As
shown in table 2, these processes include (1) requirements
management, (2) software project planning, (3) software project
tracking and oversight, (4) software subcontract management, (5)
software quality assurance, and (6) software configuration
management.
Table 2
CMM Level 2 "Repeatable" Key Process
Area Descriptions
CMM Level 2 KPAs Description
---------------------------------------- ----------------------------
Requirements management Defining, validating, and
prioritizing requirements,
such as functions,
performance, and delivery
dates.
Project planning Developing estimates for the
work to be performed,
establishing the necessary
commitments, and defining
the plan to perform the
work.
Project tracking and oversight Tracking and reviewing
software accomplishments and
results against documented
estimates, commitments, and
plans and adjusting these
based on the actual
accomplishments and results.
Software subcontract management Selecting qualified
contractors and managing
them effectively.
Software quality assurance Reviewing and auditing the
software products and
activities to ensure that
they comply with the
applicable processes,
standards, and procedures
and providing the staff and
managers with the results of
their reviews and audits.
Software configuration management Selecting project baseline
items, such as
specifications;
systematically controlling
these items and changes to
them; and recording and
reporting status and change
activity for these items.
----------------------------------------------------------------------
We conducted our review between August 1995 and February 1996, in
accordance with generally accepted government auditing standards.
--------------------
\1 SEI is a nationally recognized, federally funded research and
development center established at Carnegie Mellon University in
Pittsburgh, Pennsylvania, to address software development issues. In
the late 1980's, the Software Engineering Institute, with assistance
from the Mitre Corporation, developed a process maturity framework
that would help organizations improve their software process. In
general, software process maturity serves as an indicator of the
likely range of cost, schedule, and quality results to be achieved by
projects within a software organization.
\2 Version 2.0 of the SCE method is based on SEI's Capability
Maturity Model, Version 1.1.
\3 According to an SEI study ( i.e., Moving on Up: Data and
Experience Doing CMM-Based Process Improvement, Technical Report
CMU/SEI-95-TR-008, August 1995) of 48 organizations that implemented
software process improvement programs, the time required to increase
process maturity from level 1 to level 2 took an average of 30
months, with a range of 11 months to 58 months.
\4 Capability Maturity Model for Software, Version 1.1 (Technical
Report CMU/SEI-93-TR-24, February 1993).
\5 Software Capability Evaluation, Version 2.0, Method Description
(CMU/SEI-94-TR-06, June 1994).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :3
VA does not satisfy the criteria for a level 2 (i.e., repeatable)
software development capability on any of the seven projects they
identified as using their most mature processes. None of the
projects reviewed satisfy any of the criteria (i.e., KPAs) that the
Software Engineering Institute's CMM requires for a repeatable
process. For example, VBA is extremely weak in the requirements
management, software project planning, and software subcontract
management KPAs with no identifiable strengths or improvement
activities. As a level 1 organization, VBA cannot reliably develop
and maintain high-quality software on any major project within
existing cost and schedule constraints, placing the VBA modernization
program at risk.
Similarly, AAC has a level 1 software development capability. It
does not satisfy any of the criteria (i.e., KPAs) for a level 2
(i.e., repeatable) process. Austin's weakest KPA is software
subcontract management with no identifiable strengths or improvement
activities.
However, there are some strengths and improvement activities that
both VBA and AAC can build upon to improve their respective software
development processes. For example, VBA can build upon its strengths
in the software quality assurance KPA, and its improvement activities
in the software configuration management KPA. Similarly, AAC can
build upon its strengths in the software quality assurance and
software configuration management KPAS. Activities of this nature
will help both VBA and AAC move toward a level 2 (i.e., repeatable)
software development capability.
VETERANS BENEFITS
ADMINISTRATION SOFTWARE
PRACTICES
------------------------------------------------------------ Letter :4
Highlights of our evaluation of VBA's software practices using the
SEI criteria outlined in appendix II follow.
-- Requirements Management - The purpose of requirements management
is to establish a common understanding between the customer and
the software project of the customer's requirements that will be
addressed by the software project. The first goal within this
KPA states that, "system requirements allocated to software are
controlled to establish a baseline for software engineering and
management use." VBA does not manage and control system
requirements as required by this goal. Moreover, members of
software-related groups are not trained in requirements
management activities. Also, changes made to software plans,
work products, and activities resulting from changes to the
software requirements are not assessed for risk.
-- Software Project Planning - The purpose of software project
planning is to establish reasonable plans for performing the
software engineering and for managing the software project. VBA
projects do not have software development plans, estimates for
software project costs are not derived using conventional
industry methods and tools, and VBA is unable to show the
derivation of the estimates for the size (or changes to the
size) of the software work products. Also, individuals involved
in the software project planning are not trained in estimating
and planning procedures applicable to their area of
responsibility.
-- Software Project Tracking and Oversight - The purpose of
software project tracking and oversight is to provide adequate
visibility into actual progress so that management can take
effective actions when the software project's performance
deviates significantly from software plans. VBA does track
software project schedules against major milestones; however, as
mentioned previously, these schedules and milestones are not
derived using conventional industry methods nor is there a
comprehensive software plan against which to track activities.
Moreover, the size of software work products (or the size of
changes to software work products) are not tracked, and the
software risks associated with cost, resource, schedule, and
technical aspects of the project are not tracked.
-- Software Subcontract Management - The purpose of software
subcontract management is to select qualified software
subcontractors and manage them effectively. VBA does not have a
written organizational policy that describes the process for
managing software contracts. Additionally, the software work to
be contracted is neither defined nor planned according to a
documented procedure. Finally, software managers and other
individuals who are involved in developing, negotiating, and
managing a software contract are not trained to perform these
activities.
-- Software Quality Assurance - The purpose of software quality
assurance is to provide management with appropriate visibility
into the process being used by the software project and of the
products being built. VBA has a software quality and control
(SQ&C) group that has a reporting channel to senior management,
independent of the project managers. The SQ&C group also
performs testing of the software code. However, the SQ&C group
does not participate in other software quality assurance (SQA)
functions, such as the preparation, review, and audit of
projects' software development plans, standards, procedures, and
other work products. Also, projects do not have SQA plans.
-- Software Configuration Management - The purpose of software
configuration management is to establish and maintain the
integrity of products of the software project throughout the
project's software life cycle. VBA has provided formal training
to its staff in defining software processes. However, VBA
cannot effectively control the integrity of its software work
products because it has no software configuration control board,
it does not identify software work products to be placed under
configuration management, and it has no configuration management
library system to serve as a repository for software work
products. VBA has begun improvement activities in this area by
(1) establishing a software configuration management group and
(2) drafting a software configuration management procedure.
Following a presentation of GAO's SCE results to the Chief
Information Officer of VBA, the Director of VBA's Office of
Information Systems forwarded a letter to GAO citing a number of
initiatives that are currently underway to address some of the stated
deficiencies. Initiatives cited by the VBA include:
-- development and distribution of interim configuration management
procedures;
-- identification of a library structure to hold all of the work
products from the development process; and
-- initiation of several meetings with SEI to discuss the Software
CMM.
AUSTIN AUTOMATION CENTER
SOFTWARE PRACTICES
------------------------------------------------------------ Letter :5
Similar to VBA, we compared the CMM criteria in appendix II to the
software development practices at AAC. Summary results of this
evaluation follow.
-- Requirements Management - AAC does not create or control a
requirements baseline for software engineering. Also, AAC does
not manage or control requirements. AAC does have a process for
negotiating periodic contractual arrangements with customers,
but this process does not include baselining and controlling
software requirements.
-- Software Project Planning - Although AAC documents its schedule
estimates for software development projects, there is (1) no
defined methodology in use for estimating software costs, size,
or schedule, (2) no derivation of estimates for the size (or
changes to the size) of software products, and (3) no derivation
of the estimates for software project costs. Similarly, AAC
uses a project planning tool called "MultiTrak". However,
projects do not have software development plans.
-- Software Project Tracking and Oversight - AAC performs schedule
tracking at major milestones. However, the goals for this KPA
call for (1) the tracking of actual results and performances
against software plans, (2) the management of corrective actions
when deviations from the software plan occur, and (3) the
affected parties to mutually agree to changes in commitments.
AAC does not conform to these goals. For example, AAC does not
track (1) the software risks associated with cost, resource,
schedule, and technical aspects of the project and (2) the size
of software work products (or size of changes to software work
products).
-- Software Subcontract Management - Although the goals for this
KPA emphasize the selection of qualified software subcontractors
and managing them effectively, AAC does not (1) have a
documented procedure that explains how the work to be contracted
should be defined and planned and (2) ensure that software
managers and other individuals who are involved in establishing
a software contract are trained to perform this activity.
-- Software Quality Assurance - The goals within this KPA emphasize
(1) the verification of the adherence of software products and
activities to applicable standards, procedures, and requirements
and (2) the reporting of noncompliance issues that cannot be
resolved within the project to senior management. AAC has an
automated data processing system integrity guideline and a
systems integration service (SIS) group that has a reporting
channel to senior management and is independent of the project
managers. However, projects do not have SQA plans; the SIS
group does not participate in certain SQA functions, such as the
preparation, review, and audit of projects' software development
plans, standards, and procedures; and members of the SIS group
are not trained to perform their SQA activities.
-- Software Configuration Management - AAC performs software (i.e.,
code only) change control using a tool called "ENDEVOR," and its
employees are trained in the use of this tool. However, the
scope of the goals within this KPA cover all products in the
entire software life cycle and not just the software code. AAC
has not identified software work products (with the exception of
software code) that need to be placed under configuration
management, established a configuration management library
system that can be used as a repository for software work
products, or established a software configuration control board.
Unless both VBA and AAC initiate improvement activities within the
various KPAs and accelerate those already underway, they are unlikely
to produce and maintain high-quality software on time and within
budget.
CONCLUSIONS AND RECOMMENDATIONS
------------------------------------------------------------ Letter :6
Because VBA and AAC do not satisfy any of the KPAs required for a
level 2 (i.e., repeatable) capability, there is no assurance that (1)
investments made in new software development will achieve their
operational improvement objectives or (2) software will be delivered
consistent with cost and schedule estimates. To better position VBA
and AAC to develop and maintain their software successfully and to
protect their software investments, we recommend that the Secretary
of Veterans Affairs take the following actions:
-- Delay any major investment in software development beyond that
which is needed to sustain critical day-to-day operations until
the repeatable level of process maturity is attained.
-- Obtain expert advice to assist VBA and AAC in improving their
ability to develop high-quality software, consistent with
criteria promulgated by SEI.
-- Develop an action plan, within 6 months from the date of this
letter, that describes a strategy to reach the repeatable level
of process maturity.
-- Implement the action plan expeditiously.
-- Ensure that any future contracts for software development
require the contractor have a software development capability of
at least CMM level 2.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :7
VBA comments responded to its SCE results, and VA comments responded
to the SCE results for AAC.
VBA COMMENTS ON GAO
RECOMMENDATIONS
---------------------------------------------------------- Letter :7.1
In commenting on a draft of this report, the Veterans Benefits
Administration (VBA) agreed with four of our recommendations and
disagreed with one recommendation. VBA stated that while it agreed
that a repeatable (i.e., level 2) level of process maturity is a goal
that must be attained, it disagreed that "...all software development
beyond that which is day-to-day critical must be curtailed..." VBA
further stated that the payment system replacement projects, the
migration of legacy systems, and other activities to address the
change of century must continue. While we agree that the software
conversion or development activities required to address issues such
as the change of century or changes to legislation must continue, we
would characterize these as sustaining critical day-to-day
operations. However, major system development initiatives in support
of major projects such as the system modernization effort, which
involves several system replacement projects and the migration of
legacy systems, and VETSNET, which includes several payment system
replacement projects, should be reassessed for risk of potential
schedule slippage, cost overrun, and shortfall in anticipated system
functions and features. Shortcomings such as these are more likely
from organizations with a software development maturity rating below
level 2 (i.e., the repeatable level). Therefore, to minimize
software development risks, we continue to believe that VBA should
delay any major investment in software development unless it is
required to sustain day-to-day operations, until a maturity rating of
level 2 is reached.
Regarding the remaining four recommendations, we are pleased to see
that VBA is already initiating positive actions, including acquiring
the assistance of the Software Engineering Institute.
VA COMMENTS ON SCE RESULTS
FOR AAC
---------------------------------------------------------- Letter :7.2
VA stated that we did not demonstrate a willingness or flexibility in
relating AAC documentation products, activities, and terms to the SEI
terms. We reviewed all documentation provided to us by VA including
the documents listed in their comments on our draft report. As
called for by the SCE methodology, we carefully compared all this
documentation to the SEI CMM criteria. As stated throughout our
report, we found some strengths but in many cases, VA's documentation
was not commensurate with that called for by the SCE methodology.
Our comments on the specific key process areas follow.
REQUIREMENTS MANAGEMENT
-------------------------------------------------------- Letter :7.2.1
The VA comments stated that the OFM/IRM Business Agreement, dated
September 1994, contains guidelines which mandate the management of
software requirements. However, in our review of the documentation
listed under requirements management (Enclosure 1: Documents
Addressing Key Process Area), we found no evidence that these
documents addressed any of the goals of this KPA. For example, (1)
the allocated requirements are neither managed, controlled, nor
baselined, and (2) no software development plans were developed based
on the allocated requirements.
SOFTWARE PROJECT PLANNING
-------------------------------------------------------- Letter :7.2.2
VA feels that the AAC Business Agreement and the negotiated quarterly
contract satisfies this KPA; however, we found that AAC does not
perform a majority of the activities required to meet the goals
within this KPA. For example, AAC was not able to submit evidence
for estimating software size and cost, nor did AAC demonstrate any
methodology used for estimating schedules.
SOFTWARE PROJECT TRACKING
AND OVERSIGHT
-------------------------------------------------------- Letter :7.2.3
VA stated that project size and risk remain consistent throughout the
development/implementation cycle. However, AAC did not provide our
SCE team with any evidence validating this assertion and, as
discussed on page 8, AAC does not track this information.
SOFTWARE SUBCONTRACT
MANAGEMENT
-------------------------------------------------------- Letter :7.2.4
VA claims that specific written policies and procedures are followed
when managing software contracts; however, AAC staff interviewed were
unable to provide us with any specific policies or procedures used
for software contracting. The AAC staff acknowledged that they do
not track (1) software contractor performance at the coding level
(i.e., track functionality only) or (2) contractor produced software
documentation. Regarding training for software contract management,
VA stated that its COTRs receive training in procurement, project
management, and evaluating contractor performance. However, there is
no indication that these courses are specific to software
contracting. In addition, other individuals involved in establishing
the software contract for the projects reviewed had not received
contract management training related to software.
SOFTWARE QUALITY ASURANCE
-------------------------------------------------------- Letter :7.2.5
VA states that its ADP System Integrity Guide, dated September 1994,
contains detailed procedures directing the SIS group in specific SQA
functions. Although this is a good first step, the AAC is still
deficient because it does not have project specific software quality
assurance plans that are implemented for individual projects, as
requuired by this KPA within the CMM. Furthermore, we were not
provided with any evidence showing that the ADP System Integrity
Guide has been officially issued or whether its use will be mandatory
or discretionary.
SOFTWARE CONFIGURATION
MANAGEMENT
-------------------------------------------------------- Letter :7.2.6
The VA comments do not present any additional evidence that would
help to satisfy the criteria for this KPA. Specifically,
communication between the SIS, AAC staff, and customer do not
substitute for the rigor and discipline of a software configuration
control board, which VA acknowledged they do not have. Furthermore,
the placement of software code under configuration management is not
sufficient to satisfy this KPA because other software work
products--such as system design specifications, database
specifications, and computer program specifications--are also
required. Finally, although the AAC does maintain a library of those
software work products that it does produce, the products are not
maintained under a formal software configuration management
discipline, which would include version control and rigorous
requirements traceability.
---------------------------------------------------------- Letter :7.3
We are sending copies of this report to the Chairmen and Ranking
Minority Members of the House and Senate Committees on Veterans
Affairs and the House and Senate Committees on Appropriations; the
Secretary of Veterans Affairs; and the Director, Office of Management
and Budget. Copies will also be made available to other interested
parties upon request.
This work was performed under the direction of William S. Franklin,
Director, Information Systems Methodology and Support, who can be
reached at (202) 512-6234. Other major contributors are listed in
appendix IV.
Sincerely yours,
Gene L. Dodaro
Assistant Comptroller General
(See figure in printed edition.)Appendix I
COMMENTS FROM THE VETERANS
BENEFITS ADMINISTRATION
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
VETERANS AFFAIRS
============================================================== Letter
text appears at the end of this appendix.
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following is GAO's comment on the Department of Veterans Affairs'
May 24, 1996, letter.
GAO COMMENT
1. This issue is not addressed in our report.
LEVEL 2 KEY PROCESS AREA GOALS
========================================================= Appendix III
Level 2 KPA Purpose Goals
------------------ ----------------------------- -----------------------------
Requirements To establish a common Goal 1
management understanding between the System requirements allocated
customer and the software to software are controlled to
project of the customer's establish a baseline for
requirements that will be software engineering and
addressed by the software management use.
project.
Goal 2
Software plans, products, and
activities are kept
consistent with the system
requirements allocated to
software.
Software project To establish reasonable plans Goal 1
planning for performing the software Software estimates are
engineering and for managing documented for use in
the software project. planning and tracking the
software project.
Goal 2
Software project activities
and commitments are planned
and documented.
Goal 3
Affected groups and
individuals agree to their
commitments related to the
software project.
Software project To provide adequate Goal 1
tracking and visibility into actual Actual results and
oversight progress so that management performances are tracked
can take effective actions against the software plans.
and when the software
project's performance Goal 2
deviates significantly from Corrective actions are taken
software plans. and managed to closure when
actual results and
performance deviate
significantly from the
software plans.
Goal 3
Changes to software
commitments are agreed to by
the affected groups and
individuals.
Software To select qualified software Goal 1
subcontract subcontractors and manage The organization selects
management them effectively. qualified software
subcontractors.
Goal 2
The organization and the
software subcontractor agree
to their commitments to each
other.
Goal 3
The organization and the
software subcontractor
maintain ongoing
communications.
Goal 4
The organization tracks the
software subcontractors'
actual results and
performance against its
commitments.
Software quality To provide management with Goal 1
assurance appropriate visibility into Software quality assurance
the process being used by the activities are planned.
software project and of the
products being built. Goal 2
Adherence of software
products and activities to
the applicable standards,
procedures, and requirements
is verified objectively.
Goal 3
Affected groups and
individuals are informed of
software quality assurance
activities and results.
Goal 4
Noncompliance issues that
cannot be resolved within the
software project are
addressed by senior
management.
Software To establish and maintain the Goal 1
configuration integrity of products of the Software configuration
management software project throughout management activities are
the project's software life planned.
cycle.
Goal 2
Selected software work
products are identified,
controlled, and available.
Goal 3
Changes to identified
software work products are
controlled.
Goal 4
Affected groups and
individuals are informed of
the status and content of
software baselines.
--------------------------------------------------------------------------------
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix IV
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
David Chao, SCE Team Leader
Gary R. Austin, SCE Team Member
K. Alan Merrill, SCE Team Member
Madhav S. Panwar, SCE Team Member
Keith A. Rhodes, SCE Team Member
Paul Silverman, SCE Team Member
*** End of document. ***