Defense Management: Selection of Depot Maintenance Standard System Not
Based on Sufficient Analyses (Letter Report, 07/13/95, GAO/AIMD-95-110).
Pursuant to a congressional request, GAO reviewed the Department of
Defense's (DOD) justification for developing and deploying its Depot
Maintenance Standard System (DMSS), focusing on whether DOD has: (1)
based its DMSS selection on costs and benefit analyses as well as
economic and technical risks; and (2) selected a strategy that would
dramatically improve depot maintenance operations.
GAO found that: (1) DOD has not based its DMSS decisions on sufficient
cost and benefit analyses or detailed assessments of economic and
technical risks; (2) DOD may not achieve the marginal improvements
envisioned, since it has failed to obtain project milestone reviews or
approvals for DMSS that would ensure that system development and
implementation decisions are consistent with sound business practices;
(3) DMSS will not dramatically improve DOD depot maintenance or produce
significant cost savings, since DOD has not reengineered its business
practices; and (4) DOD may have made future reengineering efforts more
difficult by entrenching inefficient and ineffective work processes.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-95-110
TITLE: Defense Management: Selection of Depot Maintenance Standard
System Not Based on Sufficient Analyses
DATE: 07/13/95
SUBJECT: Defense procurement
ADP procurement
Management information systems
Military inventories
Cost effectiveness analysis
Systems analysis
Systems conversions
Equipment maintenance
Inventory control systems
Logistics
IDENTIFIER: DOD Depot Maintenance Standard System
DOD Depot Maintenance Resource Planning System
DOD Programmed Depot Maintenance Scheduling System
DOD Baseline Advanced Industrial Management System
Air Force Depot Maintenance Management Information System
DOD Corporate Information Management Initiative
CIM
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to the Chairman, Subcommittee on Military Readiness, Committee
on National Security, House of Representatives
July 1995
DEFENSE MANAGEMENT - SELECTION OF
DEPOT MAINTENANCE STANDARD SYSTEM
NOT BASED ON SUFFICIENT ANALYSES
GAO/AIMD-95-110
Depot Maintenance Standard System
Abbreviations
=============================================================== ABBREV
AIS - automated information system
CFI&I - Center For Integration & Interoperability
CIM - Corporate Information Management
DMMIS - Depot Maintenance Management Information System
DMRP - Depot Maintenance Repair Planning System
DMSS - Depot Maintenance Standard System
DOD - Department of Defense
DUSD(L) - Deputy Under Secretary of Defense for Logistics
FEA - functional economic analysis
IM - information management
IPR - in-process review
JLSC - Joint Logistics System Center
LCM - life-cycle management
MAISRC - Major Automated Information System Review Council
MDA - Milestone Decision Authority
OSD - Office of the Secretary of Defense
PSA - Principal Staff Assistant
SDP - system decision paper
Letter
=============================================================== LETTER
B-259429
July 13, 1995
The Honorable Herbert H. Bateman
Chairman, Subcommittee on Military Readiness
Committee on National Security
House of Representatives
Dear Mr. Chairman:
This report responds to the Committee's request that we evaluate the
Department of Defense's (DOD) justification for developing and
deploying the Depot Maintenance Standard System (DMSS). DOD is
developing DMSS to support its efforts to streamline depot
maintenance operations and manage resources more efficiently at its
repair depots. Annually, the Department spends about $13 billion to
manufacture, overhaul, and repair equipment, such as airplanes,
ships, and tanks and reparable parts of this equipment, such as
radios and engines.
DMSS is being developed and deployed as a migratory information
system\1 under the Department's Corporate Information Management
(CIM) initiative. DOD expects to spend more than $1 billion to
develop and deploy DMSS over 10 years from fiscal years 1993 through
2003. In its report on the National Defense Authorization Act for
Fiscal Year 1995, the Committee expressed concern that despite
spending billions of dollars for information technology over the last
few years, DOD had not achieved significant quality improvements,
cost savings, and productivity gains in service operations. The
Committee directed the Secretary of Defense to conduct a study to
determine the best prototype depot maintenance system and directed
GAO to assess the soundness of the study's conclusions.
The Department, however, has not done a study in response to this
requirement. DOD officials stated that, in their opinion, two
studies done by the Logistics Management Institute during 1994
comparing the services depot maintenance automated systems would meet
the legislative requirement. These studies, however, focused on only
one of the eight system components of DMSS and did not address
reengineering alternatives for improving depot maintenance
operations. Therefore, in response to the Committee's concerns, our
objectives were to determine whether DOD had (1) based its selection
of DMSS on convincing analyses of costs and benefits as well as
economic and technical risks and (2) selected a strategy that would
dramatically improve depot maintenance operations.
--------------------
\1 A migratory system is an existing information system or systems
designated to replace each of the many service-unique systems now
supporting similar business functions.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Defense has little real assurance that it can achieve even the
relatively modest projected improvements and cost savings in depot
maintenance operations from DMSS. The Department did not base its
decision to develop and implement DMSS on sufficient analyses of
costs and benefits or on detailed assessments of economic and
technical risks. Also, Defense did not obtain project milestone
reviews by Defense's Major Automated Information System Review
Council (MAISRC) and approvals from the Milestone Decision Authority
(MDA). These reviews and approvals are designed to ensure that
system development and implementation decisions are consistent with
sound business practices and to better manage risks inherent in large
information system projects.
Even if successfully implemented as envisioned, DMSS will not
dramatically improve depot maintenance in DOD. Defense did not
reengineer its business processes, which could provide dramatic
improvement and cost savings. Instead, it is currently making a
major investment, totaling more than $1 billion, to develop and
deploy DMSS, designed to incrementally improve depot maintenance
processes while migrating from service-unique supporting information
systems toward a DOD-wide integrated corporate system. These
improvements are intended to reduce depot maintenance operational
costs by $2.6 billion or less than 2.3 percent over a 10-year period.
Defense has achieved incremental improvements through initial
implementation of DMSS system components. However, by focusing first
on developing and deploying a standard depot maintenance information
system designed to incrementally improve depot maintenance processes,
DOD will not achieve any immediate dramatic cost reductions and may
make future reengineering efforts more difficult by entrenching
inefficient and ineffective work processes.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :2
We based our review on an assessment of DOD's implementation of its
own directives and instructions for new automated information systems
or the selection and implementation of standard migratory systems
under the CIM initiative, as these projects relate to the depot
maintenance business area. These directives, referred to as
Life-Cycle Management,\2 contain the same steps and milestones as
GAO's own methodology for reviewing large automated information
systems/projects.
Our audit was performed between April 1994 and March 1995 in
accordance with generally accepted government auditing standards. We
performed our work primarily at the offices of the Deputy Under
Secretary of Defense for Logistics in Washington, D.C., and the Joint
Logistics Systems Center, Wright-Patterson Air Force Base, Ohio.
Appendix I details our scope and methodology. The Deputy Under
Secretary of Defense for Logistics provided written comments on a
draft of this report. These comments are discussed at the end of
this report and presented, along with our evaluation, in appendix II.
--------------------
\2 Defense Directive 8000.1, Defense Information Management (IM)
Program; Defense Directive 8120.1, Life-Cycle Management (LCM) of
Automated Information Systems (AISs); and Defense Manual 8020.1-M,
Functional Process Improvement (Functional Management Process for
Implementing the Information Management Program of the Department of
Defense).
SIGNIFICANT PROBLEMS IN DOD
DEPOT MAINTENANCE
------------------------------------------------------------ Letter :3
Each year DOD spends about $13 billion to manufacture, overhaul, and
repair more than 2 million items at its more than 27 maintenance
depots. The depots have primary responsibility for the maintenance,
overhaul, and repair of large items, such as tanks, ships, and
airplanes, and small and intricate ones, such as communications and
electronic components. Depot maintenance consists of three basic
business processes: project management (maintenance of major-end
items, such as airplanes, ships, and tanks), reparables management
(maintenance of items, such as engines, transmissions, and radios),
and specialized support (various individual functions, such as
tracking hazardous materials, tools, and test samples).
For years, GAO and DOD have reported on major problems facing the
depot maintenance area, principally that DOD's depot management
structure has not resulted in substantial competition,
interservicing, or reduction of excess capacity and duplication of
effort. For example:
In 1983, GAO testified\3 that DOD had not moved quickly to
eliminate duplicate capability and excess capacity within depot
maintenance because of (1) parochial interests, (2) lack of
central authority, and (3) absence of DOD-wide planning.
In 1993, the Joint Chiefs of Staff reported that closing a
significant number of depots was needed to reduce excess
capacity and that significant savings could come from
consolidating depot workload across service boundaries. In May
1993, we testified,\4 that the Joint Chiefs of Staff identified
25 to 50 percent more depot capacity than will be needed in the
future and that this problem had been exacerbated by (1) the end
of the cold war, (2) reduction of defense systems and equipment,
(3) retirement of less reliable and more maintenance-intensive
systems, and (4) the private sector's push for a greater share
of the depot maintenance workload.
In 1993,\5 we reported that internal controls at Army depots did
not adequately safeguard millions of dollars of weapons and
equipment during the maintenance processes. Specifically, we
reported that poor storage practices increased maintenance
costs, depot inventory records were not accurate, and the Army's
depot cost accounting system did not capture actual job costs.
In 1995, DOD reported to the Congress that its financial systems
and databases were inadequate to provide the type of information
to determine the cost-effectiveness of greater public-private
competition for providing depot maintenance services.
Over the last several years, DOD has taken a number of actions to
correct these problems. One of these actions is its Corporate
Information Management initiative, which was established to prepare
DOD for future budget reductions and post-cold war readiness
requirements through (1) streamlining business processes, (2)
integrating essential data systems, and (3) eliminating duplicate or
redundant information systems across the Department. The DMSS
project was undertaken as part of this effort.
--------------------
\3 Depot Maintenance: Issues in Management and Restructuring To
Support a Downsized Military (GAO/T-NSIAD-93-13, May 6, 1993).
\4 See footnote 3.
\5 Financial Management: Poor Internal Control Has Led to Increased
Maintenance Costs and Deterioration of Equipment (GAO/AFMD-93-8,
January 25, 1993).
STRATEGY FOR ADDRESSING
DEPOT MAINTENANCE PROBLEMS
---------------------------------------------------------- Letter :3.1
To improve its depot maintenance operations and manage its resources
more efficiently, the Principal Staff Assistant (PSA)\6 for
logistics,\7 in November 1991, established the Joint Logistics System
Center (JLSC). JLSC is to facilitate the improvement of depot
maintenance processes by identifying business process improvements
and managing the development and deployment of a standard depot
maintenance system to replace service-unique systems currently used.
In January 1994, JLSC prepared an economic analysis recommending
development and deployment of a standard depot maintenance
information system--called the Depot Maintenance Resource Planning
(DMRP) system, which consisted of four system components.\8
Currently, the standard information system consists of eight
components and is called DMSS. The following table identifies the
core depot maintenance business processes and the eight system
components selected to support them.
Table 1
Depot Maintenance Core Business
Processes and Their Supporting DMSS
System Components
Business Process Supporting System Component
----------------------------- -----------------------------
Project management Baseline Advanced Industrial
planning and allocating Management System
labor, material, and capital
resources for repairing
major-end items, such as
airplanes, ships, and tanks
Reparables management Depot Maintenance Management
activities for making labor Information System
and equipment more productive Interservice Material
on the shop floor Accounting and Control System
Specialized support Enterprise Information
various individual functions, System
such as tracking hazardous Facilities and Equipment
materials, tools, and test Maintenance
samples Depot Maintenance Hazardous
Material Maintenance System
Laboratory Information
Management System
Tool Inventory Management
Application
------------------------------------------------------------
Note: See appendix III for brief descriptions of each of these eight
system components.
By implementing DMRP, DOD expected a return on its investment of $2.6
billion through business process improvements and savings derived
from replacing more than 60 service-unique automated depot
maintenance information systems. Specifically, these benefits are to
be derived from (1) reduced direct and indirect labor costs, (2)
reduced direct and indirect material costs, (3) reduced costs
associated with shutting down old information technology (legacy)
systems, (4) shorter cycle time for certain types of maintenance and
inspections, and (5) automation of many currently paper-based work
processes.
Our concerns with this strategy are twofold. First, DOD did not base
its decision to develop and deploy DMSS on convincing analyses of
expected system development and deployment costs or detailed
assessments of DMSS's economic and technical risks. Further, Defense
did not obtain the independent reviews by the MAISRC and approvals by
the MDA of the project's milestones, which are designed to ensure the
decision was consistent with sound business practice. Second, we
believe that DOD needs to consider reengineering entire processes
before implementing system changes if it is to achieve the dramatic
reductions in operational support costs called for by CIM.
--------------------
\6 PSAs include the Under Secretaries, Assistant Secretaries, General
Counsel, Inspector General, Comptroller, Assistants to the Secretary
of Defense, and the Office of the Secretary of Defense Directors or
equivalents, including the Chairman of the Joint Chiefs of Staff, who
report directly to the Secretary or Deputy Secretary of Defense.
\7 When JLSC was created, the PSA for logistics was the Assistant
Secretary of Defense for Production and Logistics. Following a
reorganization in the Office of the Secretary of Defense, the PSA for
logistics is now the Deputy Under Secretary of Defense for Logistics.
\8 One DMRP system component, the Programmed Depot Maintenance
Scheduling System, is a module of the DMSS system component called
Baseline Advanced Industrial Management System.
DUSD(L) DID NOT USE SUFFICIENT
ANALYSES IN SELECTING DMSS
------------------------------------------------------------ Letter :4
In selecting DMSS as DOD's initial step toward improving defense
maintenance depot operations, the Deputy Under Secretary of Defense
for Logistics (DUSD(L)) did not base its decision on sufficient
analyses of expected system development and deployment costs or
detailed assessments of DMSS's economic and technical risks.
Further, DUSD(L) did not obtain independent milestone reviews and
approvals which are designed to ensure (1) decisions are consistent
with sound business principles and (2) risks inherent in large
information systems projects are adequately managed. Thus, even the
marginal improvements Defense expects from DMSS may never be
achieved.
Defense directives\9 require that decisions to develop and deploy
information systems be based on convincing, well-supported estimates
of project costs, benefits, and risks. These directives establish a
disciplined process for selecting the best projects based on
comparisons of competing alternatives.
Defense's principal means for comparing various alternatives is a
functional economic analysis. For each alternative, it identifies
resource, schedule, and other critical project characteristics and
presents estimates of the costs, benefits, and risks. The Office of
the Assistant Secretary of Defense for Program Analysis and
Evaluation is required to validate these estimates to help ensure
that the economic analysis presents compelling quantitative data for
each of the alternatives being evaluated. Once an alternative is
chosen, the analysis becomes the basis for project approval. Any
significant change in the project's expected costs, benefits, or
risks requires that the project selection and direction be
reevaluated.
Also, DOD directives established the Major Automated Information
System Review Council (MAISRC) to provide oversight of individual
major information system projects.\10 At each development
milestone\11 for proposed information system projects, MAISRC reviews
these projects to determine if they are consistent with DOD policies
and directives. MAISRC then recommends continuation, redirection, or
termination of each project to the project's Milestone Decision
Authority (MDA). DOD's current policy is to ensure that funds are
not obligated for any automated information system until the MAISRC
milestone review and MDA approval are complete.
In January 1994, following the logistics CIM migration strategy, the
JLSC evaluated three alternatives for improving the core Defense
depot maintenance functions. The alternatives considered involved
(1) maintaining status quo by allowing each service to continue to
operate its own information system with some new development under
JLSC's purview, (2) choosing a corporate information system from
among the services and establishing it as the DOD-wide standard
system--deploying it either immediately and then enhancing it over a
3-year period or deploying it after enhancements, and (3) developing
a new system.
In selecting an alternative, DUSD(L) did not evaluate sufficiently
accurate cost data and detailed assessment of risks, nor did it
obtain milestone reviews and approvals designed to ensure automated
information systems are selected consistent with sound business
practices.
--------------------
\9 See footnote 2.
\10 Major information system projects are those with estimated
development and deployment costs in excess of $25 million in any 1
year, $100 million in total, or are designated as being of special
interest.
\11 Under DOD's life-cycle management process, system development
projects are divided into five phases with corresponding decision
points, called milestones, where project progress is assessed and
documented.
DUSD(L) SELECTED DMSS
WITHOUT SUFFICIENT COST DATA
---------------------------------------------------------- Letter :4.1
DUSD(L) selected DMSS without analyzing the system's full development
and deployment costs. Instead, it relied on a functional economic
analysis of a previously proposed project--the Depot Maintenance
Resource Planning (DMRP) system. This analysis significantly
understated DMSS costs by including costs for only some components,
and it understated costs for the components it did include. In early
1994, the JLSC Commander recognized that the DMRP economic analysis
did not reflect DMSS as defined.
According to JLSC officials, DUSD(L) used the DMRP functional
economic analysis as a basis for selecting DMSS because it was the
best available at the time. The DMRP analysis estimated project
costs at $988 million--$582 million to develop and deploy and $406
million to operate and support over a 10-year period. These
officials stated that the DMRP analysis fairly represented the DMSS
project. However, the Office of the Assistant Secretary of Defense
for Program Analysis and Evaluation reviewed this analysis and found
its level of detail insufficient to validate either cost or benefit
estimates.
Although we also found insufficient details supporting cost and
benefit estimates, we believe that DMSS will cost significantly more
than the DMRP. As shown in table 2, the DMRP economic analysis
included costs for only three of the eight DMSS system components.
Therefore, the analysis understated DMSS costs by the amount
necessary to develop and deploy the five additional system
components. As of February 1995, JLSC had not completed a cost
estimate for these five additional components.
Table 2
Comparison of DMRP and DMSS System
Components
System Components DMRP\ DMSS
---------------------------------------- -------- --------
Baseline Advanced Industrial Management �
Depot Maintenance Hazardous Material � �
Management System
Depot Maintenance Management Information � �
System
Enterprise Information System �
Facilities and Equipment Maintenance �
Interservice Material Accounting and �
Control System
Laboratory Information Management System �
Programmed Depot Maintenance Scheduling � �\a
System
Tool Inventory Management Application � �
------------------------------------------------------------
\a Under DMSS, this system is a module of the Baseline Advanced
Industrial Management System.
In addition, the DMRP economic analysis underestimated costs for
system components common to both DMRP and DMSS projects.
Specifically, it underestimated licensing costs for using
commercially owned software, costs to exchange data with other
information systems, and costs to install the system.
One example of underestimated licensing costs is in a key DMSS
component--the Air Force's Depot Maintenance Management Information
System (DMMIS). Over the last 10 years, the Air Force spent over
$200 million to develop DMMIS for use in its maintenance depots.
Originally designed around a core of commercially available
application and database software, the Air Force chose to extensively
modify this proprietary software to better meet its unique depot
maintenance requirements. However, all software versions remain the
sole property of the commercial developers. As a result, to use the
DMMIS system, DOD will have to pay license fees to several commercial
software developers.
Although the DMRP economic analysis did not specify DMMIS license fee
costs, JLSC officials stated that $1.6 million per site was included
in the deployment cost totals. In February 1995, JLSC estimated that
DMMIS license fees for just the development facility and two
operational sites would exceed $13 million, including a one-time
payment of over $5 million and nearly $850,000 each year over the
system's life. As of April 1995, JLSC expected to run DMMIS at three
additional sites. Licensing agreements had yet to be negotiated for
these sites.
The DMRP analysis also underestimated costs to develop interfaces
needed to allow system components to exchange data with the
information systems currently used by the services to accomplish
their missions. While the analysis recognized that system components
must interface with other systems, it did not include the full cost
of these interfaces. According to JLSC officials, some costs to
interface the DMMIS and Programmed Depot Maintenance Scheduling
System were included in the $37.7 million estimate for developing the
system's software applications. However, they did not specify these
costs.
Although JLSC has yet to identify them, DMSS will require numerous
system interfaces if it is to be the corporate depot maintenance
system. For example, prior work done by the Air Force to deploy
DMMIS, before it was selected as a DMSS component, identified 73
required interfaces just to meet Air Force requirements. As a DMSS
system component, additional DMMIS interfaces will be needed to meet
Army, Navy, and Marine Corps requirements. Further, interfaces for
the remaining seven DMSS system components must be identified and
developed. In February 1995, JLSC's Deputy Director for Depot
Maintenance estimated that $70 million not included in the DMRP
economic analysis would be needed to develop the DMSS interfaces.
Finally, the DMRP economic analysis underestimated costs for
deploying the system. The analysis estimates $497 million for system
deployment. This estimate includes nonrecurring costs of $17 million
to install the system at each operational site. Since DMSS was
initiated, JLSC has identified that an additional $60 million would
be needed to deploy the system.
In May 1994, the JLSC Commander told the DOD Comptroller about the
DMRP economic analysis. The Commander stated that the economic
analysis briefed to DUSD(L) in December 1993 and submitted for the
DOD Comptroller's review in early 1994 did not reflect DMSS as it was
then defined. Further, he stated that to accommodate changes
requested by the Comptroller and the office of Program Analysis and
Evaluation and to reflect the current DMSS, JLSC was developing a new
analysis. According to JLSC officials, the final economic analysis
is expected to be completed in July 1995. However, by this time
Defense will have spent more than $200 million to develop and deploy
DMSS.
DUSD(L) SELECTED DMSS
WITHOUT FULLY ASSESSING
RISKS
---------------------------------------------------------- Letter :4.2
Although any large automated information system development project
is inherently a high-risk venture, DUSD(L) decided to develop and
deploy DMSS without first fully assessing the risks to the project's
success. Without a detailed risk assessment, DOD has no assurance
that DUSD(L) selected the best information system alternative for
improving defense depot maintenance operations, nor can it plan
actions designed to avoid or lessen the potential for project delay,
overspending, or failure.
DOD has long recognized that project success relies on its ability to
manage risk. The Defense Systems Management College guide on risk
management states that, as a minimum, a prudent manager should
attempt to understand system specific risks and quantify their
potential impact for each alternative. While the earlier DMRP
analysis identified several potential risks associated with each
alternative being considered, it did not quantitatively or
qualitatively compare these risks. Additionally, it did not contain
any plans to mitigate potential project risks.
After DUSD(L) selected DMSS, JLSC convened a customer advisory team
in April 1994 to identify and generate ideas on how to mitigate DMSS
risks. This team, with membership from all the military services,
identified a number of risks facing DMSS, such as (1) incomplete
design and testing of the two core DMSS systems-- Depot Maintenance
Management Information System and the Baseline Advanced Industrial
Management System, (2) not enough personnel to implement and maintain
the system, (3) inability to obtain service cooperation needed to
successfully build and deploy the system, (4) numerous external and
internal interface issues, and (5) depot maintenance workers'
reluctance to work with an entirely new system.
JLSC requested another high-level risk analysis of the depot
maintenance standard system strategy from the Defense Information
Systems Agency's Center For Integration & Interoperability (CFI&I).
In a July 1994 briefing to JLSC, CFI&I said that program management
posed the greatest risks to DMSS success. CFI&I said the project
lacked (1) integrated detailed planning specifying the activities and
milestones to be achieved at each depot and (2) coordination of
events necessary to implement the system, and that, as a result,
there was no assurance that DMSS could meet cost, schedule, and
performance expectations.
In addition, CFI&I identified a number of technical risks to DMSS
implementation, including (1) no encompassing data migration
strategy, (2) incomplete and inadequate understanding of the
requirement to interface DMSS with other current service systems, (3)
difficulties associated with maintaining modified commercially owned
software, and (4) incomplete development and testing of two of the
system components.
In October 1994, JLSC began an iterative detailed assessment of DMSS
to quantify risks, identify possible mitigation or avoidance steps,
and develop a risk management plan. As of April 1995, JLSC was
continuing this assessment.
PROJECT MILESTONE REVIEWS
AND APPROVALS NOT OBTAINED
---------------------------------------------------------- Letter :4.3
Although Defense directives establish MAISRC review and MDA approval
procedures to ensure that decisions to develop major automated
information systems are based on sound business principles, as of
February 1995, DUSD(L) had not scheduled a date for an initial
milestone review of the entire DMSS project. Under MAISRC
guidelines, a project should be reviewed and approved at each of five
decision milestones\12 before substantial funds are obligated.
Despite this DOD policy, DUSD(L) spent nearly $180 million in fiscal
years 1993 and 1994 on DMSS, and budgeted $111.2 million in fiscal
year 1995 and $95.1 million in fiscal year 1996. These budgeted
amounts are for the development and deployment of DMSS and do not
include amounts to maintain and operate the current systems.
According to the director of logistics systems development within
DUSD(L), DMSS will be submitted for MAISRC review and MDA approval
during 1995. However, we found that as of February 1995, DMSS was on
the MAISRC review schedule for 1995 but no date for the review had
been established. The director also indicated that continued
implementation of DMSS at selected prototype sites is justified based
on past MAISRC reviews and MDA approvals of the DMMIS component of
the project. However, Defense directives\13 require programs which
consist of a number of component systems to be reviewed by MAISRC and
approved by MDA as a single project. Without these reviews and
approvals, DOD has less assurance that the decision to select DMSS
was consistent with sound business practices. Also, DOD did not have
an opportunity afforded by the MAISRC review and MDA approval to
redirect or terminate DMSS before investing significant amounts of
money.
--------------------
\12 DOD Instruction 8120.2, Automated Information System (AIS)
Life-Cycle Management (LCM) Process, Review, and Milestone Approval
Procedures, describes five milestone decision points: Concept
Studies Decision, Concept Demonstration Decision, Development
Decision, Production Decision, and Major Modification Decision.
\13 Defense Directive 8120.1, Life-Cycle Management (LCM) of
Automated Information Systems (AISs) and Defense Instruction 8120.2
Automated Information System (AIS) Life-Cycle Management (LCM)
Process, Review, and Milestone Approval Procedures.
DUSD(L) DID NOT CONSIDER
REENGINEERING DEPOT MAINTENANCE
PROCESSES BEFORE SELECTING DMSS
------------------------------------------------------------ Letter :5
In evaluating alternatives to improve depot maintenance operations,
DUSD(L) did not consider reengineering alternatives which offer
opportunities to dramatically improve depot maintenance business
processes and greatly reduce the costs of operations. Even if
successful, DOD's strategy to develop and deploy an information
system designed to incrementally improve depot maintenance processes
will only provide marginal cost reductions and productivity increases
rather than the fundamental and dramatic changes needed to meet the
challenges of maintaining military readiness in the 1990s.
REENGINEERING OF BUSINESS
PROCESSES CAN OFFER DRAMATIC
IMPROVEMENT
---------------------------------------------------------- Letter :5.1
DOD recognizes that business process reengineering holds the greatest
potential for meeting the demands of a changing Defense environment
with dramatic increases in effectiveness and efficiency. In the DOD
Enterprise Model,\14 the Assistant Secretary of Defense for Command,
Control, Communications, and Intelligence stated:
The defense community must make fundamental changes in the way
it performs its activities if it is to provide the nation with
the defense it requires and demands. ... Incremental
improvements...will not shift the Department to a higher plateau
of performance. Breakthrough innovation and change--a new
paradigm for defense activities--is needed to meet the
challenges of the 1990's.
In January 1991, the Deputy Secretary of Defense endorsed a CIM
implementation plan in which DOD would "reengineer," or thoroughly
study and redesign, its business processes before it standardized its
information systems. The Deputy Secretary understood that DOD would
have to improve the way it does business to achieve dramatic cost
reductions and productivity increases and that it could not merely
standardize old, inefficient processes and systems. Simply stated,
doing the same thing faster will not provide dramatic improvement.
Though reengineering efforts in DOD have been limited in scope and
represent a small portion of operations, significant improvements
have been achieved through reengineering specific logistics business
areas. For example,\15 in 1980, the Defense Construction Supply
Center established a contractor-operated parts depot program that
reduced order and delivery time from 70 to 35 days--a 50-percent
reduction.
In addition, the private sector, which also has major industrial
centers that use similar maintenance and repair supplies for
regularly scheduled maintenance of equipment, has undergone
successful reengineering efforts when faced with increasing costs
associated with acquiring supplies, spare parts, and raw materials.
For example,\16 since 1986, through customized agreements with
suppliers and the use of new inventory management practices, an Ohio
steel firm, Timken Company, reduced maintenance and repair
inventories by $4 million (32 percent). The company also eliminated
six inventory storerooms, improved inventory availability, and
increased the accuracy of physical inventories.
We have also reported that by adopting certain commercial practices,
Defense could similarly dramatically improve depot maintenance. In
1993, for example, we found\17 that a number of private firms provide
third-party logistics transportation services, such as freight bill
processing, pre-auditing, verifying, and generating management
reports with freight payment. Two of these firms proposed to perform
transportation services for DOD at a cost ranging to $.75 to $1.25
per government bill of lading. DOD spends about $5.70 per freight
bill to provide these same services. If DOD used these firms or
changed its process to obtain similar performance, it could reduce
costs for these services by more than 75 percent.
--------------------
\14 The DOD Enterprise Model, Volume II: Using the DOD Enterprise
Model - A Strategic View of Change In DOD - A White Paper, January
1994, Office of the Secretary of Defense, Assistant Secretary of
Defense for Command, Control, Communications, and Intelligence.
\15 Commercial Practices: DOD Could Save Millions by Reducing
Maintenance and Repair Inventories (GAO/NSIAD-93-155, June 7, 1993).
\16 See footnote 12.
\17 Defense Transportation: Commercial Practices Offer Improvement
Opportunities (GAO/NSIAD-94-26, November 26, 1993).
REENGINEERING NOT CONSIDERED
FOR IMPROVING DEPOT
MAINTENANCE
---------------------------------------------------------- Letter :5.2
Instead of first considering opportunities to reengineer business
processes, DUSD(L) chose a strategy that focuses on the development
and deployment of a DOD standard depot maintenance information
system. Under this strategy, business processes are to be
incrementally improved as DMSS is deployed. Reengineering of these
processes will be considered only after system deployment.
Currently, DMSS deployment is expected to be completed by fiscal year
1999. Accordingly, fundamental and dramatic changes to the depot
maintenance processes will be delayed for years.
According to DOD officials, the vast number of different logistics
processes and supporting information systems across the Department
must be reduced before significant improvements can be made. These
officials further stated that, once fully deployed, the Defense
standard information systems will form the foundation upon which
significant improvements to current depot maintenance practices can
be made. This foundation will eliminate the need to implement major
changes across a multitude of information systems and business
processes that exist throughout the services.
Additionally, JLSC officials emphasized that improvements are being
made to depot maintenance processes as DMSS is being deployed.
According to these officials, benefits being achieved from these
improvements include (1) cost reductions of $7 million in shop floor
material recovered at the Air Logistics Center in Ogden, Utah, and a
$8 million reduction in purchase of hazardous material at Hill Air
Force Base and (2) performance increases from a 30-percent reduction
in labor hours for overhauls of the Los Angeles class submarine, and
two additional B-1 bombers processed through the Oklahoma City Air
Logistics Center.
While these examples show that incremental improvements are being
made, JLSC estimated that, overall, the DMSS project would reduce
depot operational costs by $2.6 billion over a 10-year period ending
in fiscal year 2003 from $112.9 billion to $110.3 billion over this
period--a net cost reduction of about 2.3 percent.
We believe that standardizing existing information systems and
incrementally improving business processes will not position DOD for
reengineering its processes or dramatically improve their operations.
Government and private industry have learned that initial focus on
information system deployment may make future reengineering efforts
more difficult by entrenching inefficient and ineffective work
processes.
Accomplishing order-of-magnitude improvements in both government and
private organizations requires reengineering--fundamental
redesign--of critical work processes. Information system initiatives
that do not first reengineer business processes typically fail or
attain only a fraction of their potential. In addition, case studies
of private organizations presented in Reengineering The Corporation -
A Manifesto For Business Revolution,\18 revealed that companies often
commit a fundamental error in viewing automation as the answer to
enhancing or streamlining their business operations. They spend
billions of dollars to automate existing processes so they can
perform the same work faster. Companies that initially focused on
information technology managed only to entrench inefficient processes
and made future change to these processes more difficult.
Additionally, our case studies of government organizations show that
simply automating existing processes will not likely provide
significant cost reductions or productivity increases. For example,
in February 1995, the Comptroller General of the United States
testified\19 that this has been a hard lesson for federal agencies to
learn. Specifically addressing the efforts of the Department of
Defense, the Comptroller General also stated:
...Defense has focused on trying to pick the best of its
hundreds of existing automated systems and standardizing their
use across the military components without thoroughly analyzing
the technical, cost, and performance risks of this approach. As
a result, Defense may lock itself into automated ways of doing
business that do not service its goals for the future and cannot
provide promised benefits and cost savings.
Our review of DUSD(L)'s depot maintenance standard system strategy
confirms this. The benefits it expects from implementing DMSS are
relatively meager when compared with results other organizations are
achieving through reengineering.
--------------------
\18 Reengineering The Corporation - A Manifesto For Business
Revolution, Michael Hammer and James Champy, 1994.
\19 Testimony before the Senate Committee on Governmental Affairs,
Government Reform: Using Reengineering and Technology to Improve
Government Performance (GAO/T-OGC-95-2, February 2, 1995).
CONCLUSION
------------------------------------------------------------ Letter :6
We agree with DOD's concern over depot maintenance operations.
Further, we agree that accurate information on depot operations and
costs is critical to improving this important readiness-related
support process. However, the decision to develop DMSS was based on
insufficient cost data and with little consideration of identified
risks. Efficient, cost-effective depot maintenance operations are
important to supporting the Department's military operations. Major
investment decisions--such as DMSS--represent significant
opportunities to make dramatic improvements in core business
processes. Further, DOD's proposed solution was made without due
consideration of reengineering alternatives which offer dramatic
improvements and greatly reduce costs of depot operations. DOD's
failure to consider reengineering alternatives and to fully consider
the costs and risks associated with DMSS will likely limit those
opportunities.
RECOMMENDATIONS
------------------------------------------------------------ Letter :7
To achieve the dramatic improvements in effectiveness and efficiency
of its depot maintenance operations that Defense has stated are
critical to meet the challenges of the 1990s and beyond, we recommend
that the Secretary of Defense direct the Deputy Under Secretary of
Defense for Logistics to complete the following actions.
Prepare a full set of project documentation that describes the
project and validates that it is the best alternative for
improving depot operations. At a minimum, this documentation
should include the following.
A final functional economic analysis containing a comprehensive
evaluation of information system alternatives. This analysis should
formulate and compare estimates of the total costs and benefits of
each alternative.
Identification of economic and technical risks associated with
success of each project alternative and development of a plan to
avoid or mitigate these risks.
A comprehensive implementation plan that identifies actions to be
taken, schedules, and milestones for these actions, and performance
measures to be used to manage the system deployment.
Obtain the Major Automated Information Systems Review Council
review and Milestone Decision Authority approval of the project
documentation prior to spending any fiscal year 1996 funds on
DMSS development and deployment.
Conduct a thorough study of opportunities to reengineer the depot
maintenance business processes. Reengineering alternatives
identified by this study should be analyzed as part of the final
functional economic analysis and submitted for MAISRC review and
MDA approval.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :8
The Department of Defense provided written comments on a draft of
this report. The Deputy Under Secretary of Defense for Logistics
generally disagreed with our findings, but partially concurred with
our recommendations. Defense's specific comments are summarized
below and presented, along with our rebuttals, in appendix II.
In its comments, Defense took the following positions.
The DMSS functional economic analyses of March 1993 and January
1994 provided sufficient cost, benefit, and risk information to
select the best alternative for improving depot maintenance
business processes.
DMSS is not one system requiring MAISRC oversight and that the
individual system components meeting MAISRC oversight criteria
have been reviewed and approved.
Process reengineering is being accomplished concurrently with DMSS
development and deployment. Defense asserts that by following
this strategy, it has achieved substantial depot maintenance
improvements yielding significant cost reductions. Defense
expects even more dramatic improvements and savings in the
future.
We disagree with Defense's positions on these matters. Specifically:
The March 1993 and January 1994 FEAs were insufficient because they
did not include cost and benefit estimates for the DMSS,
contained cost estimates of questionable accuracy, and did not
include cost and benefit estimates for five of the DMSS system
components.
Defense CIM guidance specifically directs that information system
projects be reviewed and approved in accordance with Defense
life-cycle management directives. Under these directives DMSS
is required to be reviewed by MAISRC and approved by the MDA at
five milestone decision points before any funds are spent to
develop the system. Further, these directives state that
projects consisting of several components shall be considered as
a single automated system.
Defense's approach to improving depot maintenance business
processes focuses on the selection of the best currently
operating information systems and implementation of these
selected systems across all Defense components. While this
approach may improve overall DOD business processes and may
provide incremental benefits, it cannot be construed as
reengineering. DMSS is designed to provide only incremental
improvements to existing business processes and it is clear from
Defense's own benefit projections that it will not result in the
dramatic improvements that are possible by considering
reengineering-based solutions. While it claims that DMSS has
improved depot maintenance processes and resulted in some
reductions in operational costs, DUSD(L)'s focus on information
system selection and implementation may inhibit reengineering
efforts by entrenching current work processes.
Although Defense disagreed with our findings, it agreed with our
recommendation to prepare a full set of project documentation that
describes DMSS and validates that it is the best alternative to
improve depot maintenance. It partially concurred with our
recommendation on obtaining MAISRC review but specifically disagreed
with our recommendation concerning thoroughly studying opportunities
to reengineer depot maintenance business processes. Our
recommendation for MAISRC review is consistent with review
requirements established in Defense life-cycle management directives.
Further, because reengineering offers order-of-magnitude improvement
and cost reductions, Defense cannot afford to deploy DMSS beyond the
first five prototype sites until it has fully assessed reengineering
alternatives.
---------------------------------------------------------- Letter :8.1
We are sending copies of this report to the Ranking Minority Member
of the Subcommittee, the Chairmen and Ranking Minority Members of the
Senate and House Committees on Appropriations, the Senate Committee
on Armed Services, the Senate Committee on Governmental Affairs, and
the House Committee on Government Reform and Oversight; the
Secretaries of Defense, the Army, the Navy, and the Air Force; the
Director of the Office of Management and Budget; and other interested
parties. Copies will be made available to others on request.
If you have any questions about this report, please call me at (202)
512-6240 or Carl M. Urie, Assistant Director, at (202) 512-6231.
Major contributors to this report are listed in appendix IV.
Sincerely yours,
Jack L. Brock, Jr.
Director, Information Resources Management/
National Security and International Affairs
OBJECTIVES, SCOPE, AND METHODOLOGY
=========================================================== Appendix I
We based our review on an assessment of DOD's implementation of its
own directives and instructions for new automated information systems
or the selection and implementation of standard migratory systems
under the CIM initiative, as such projects relate to the depot
maintenance business area. These directives, referred to as Life
Cycle Management, contain the same steps and milestones as GAO's own
methodology for reviewing large automated information
systems/projects.
Specifically, to determine whether the Department based its selection
of DMSS on convincing analyses of costs and benefits, we reviewed
policies, procedures, directives, and memoranda establishing criteria
for the successful acquisition of automated information systems under
the CIM initiative. We compared the Department's actions and plans
for selecting and implementing DMSS with these criteria. To further
assess the adequacy of the selection, we examined the cost and
benefit data available to senior Defense officials responsible for
selecting DMSS. Because the level of detail was insufficient, we did
not evaluate these cost and benefit data. Also, we interviewed
Defense logistics officials to obtain the rationale behind the DMSS
selection. To identify expected DMSS costs and benefits, we analyzed
available functional economic analyses (FEA). We did not validate
the costs and benefits presented in the FEA used to justify DMSS
since (1) our objective was to examine DOD's decision given the cost
and benefit information available to it at the time and (2) the FEA
was based on a different project--the DMRP. We interviewed JLSC
officials to determine changes made to project scope, costs, or
benefits occurring since early 1994 and any additional analyses
currently being done. We also met with numerous program and
functional officials, including JLSC managers responsible for
implementing the eight DMSS system components, and depot officials at
the Air Force's repair depot in Ogden, Utah, and the Army's depot in
Tobyhanna, Pennsylvania.
To determine whether the Department had fully assessed economic and
technical risks threatening the successful implementation of DMSS and
identified actions to avoid or mitigate these risks, we reviewed risk
assessments available when DUSD(L) decided to develop and deploy
DMSS. Additionally, we examined risk analyses conducted by the Joint
Logistics Systems Center, other Defense organizations, and industry
experts completed since the DMSS selection was made. We interviewed
program and technical officials to obtain opinions on the potential
impact of risks identified by these analyses on project success and
to identify actions for avoiding or mitigating those risks most
likely to result in project failure, delay, and overspending.
To determine whether the Department selected a strategy that would
dramatically improve depot maintenance processes, we reviewed DOD
documents detailing challenges of meeting the defense mission in the
post-cold war environment, CIM goals and objectives to meet these
challenges, and the plans and strategies for implementing CIM across
the Department. We compared these DOD strategies and plans to the
Logistics Migration Approach established to implement the CIM
initiative in the logistics business area. We then compared the
level of improvement expected from a standard depot maintenance
information system to the DOD stated requirement to meet the
challenges of the future defense environment. To identify
alternatives to information system approaches, we reviewed private
industry studies and past GAO reports of lessons learned by private
and public organizations that have successfully improved their
business processes. We compared these lessons learned and case
studies with the approach being implemented through the development
and deployment of DMSS.
Our work was performed between April 1994 and March 1995 primarily at
the offices of the Deputy Under Secretary of Defense for Logistics in
the Pentagon, Washington D.C., and the Joint Logistics Systems
Center, Wright-Patterson Air Force Base, Ohio. We also performed
work at the offices of the Assistant Secretary of Defense for
Command, Control, Communications, and Intelligence in Washington
D.C.; the Center for Integration & Interoperability, Defense
Information Systems Agency, Blacklick, Ohio; the Air Force Air
Logistics Center, Hill Air Force Base, Utah; and the Tobyhanna Army
Depot, Tobyhanna, Pennsylvania.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
=========================================================== Appendix I
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on the Department of Defense's
letter dated May 30, 1995.
GAO COMMENTS
1. The March 1993 Phase I Functional Economic Analysis (FEA) and
January 1994 Phase II FEA for the DMRP processes did not provide
well-supported estimates of project costs, benefits, and risks upon
which to approve the Depot Maintenance Standard System (DMSS). As
stated in our report, there are three reasons why these FEAs did not
sufficiently support the DMSS selection.
First, these FEAs did not include cost and benefit estimates for the
DMSS. The DMSS was not defined as a project until March 1994--more
than a year after the Phase I FEA was completed and 2 months after
the Phase II FEA was submitted for review. The Commander of Joint
Logistics Systems Center told the DOD Comptroller that the functional
economic analysis briefed to DUSD(L) and submitted for the
Comptroller's review in early 1994 did not include DMSS. The
Commander also stated that JLSC was developing a new economic
analysis to (1) accommodate changes requested by the Comptroller and
the Office of Program Analysis and Evaluation and (2) reflect the
current DMSS.
Secondly, the analyses contained estimates of questionable accuracy.
The Office of the Assistant Secretary of Defense for Program Analysis
and Evaluation, which is required to validate automated information
system project estimates to help ensure that the economic analyses
present compelling quantitative data for project selection, found
that the level of detail was insufficient to validate either cost or
benefit estimates. Even with this limitation, we determined that the
Phase II FEA underestimated the cost to develop and deploy the three
system components that later became part of DMSS by at least $100
million.
Finally, the Phase I and II FEAs were incomplete. They included cost
and benefit estimates for only three of the eight system components
forming the DMSS. DUSD(L)'s contention that the larger DMSS does not
represent a significantly larger cost than the smaller suite of
systems identified in the Phase I FEA is unfounded. The scope of the
projects differ significantly. The DUSD(L) improperly equates
expected costs and benefits of the DMRP project to develop and deploy
a suite of three system components at 27 maintenance depots over a
7-year period to the DMSS project to develop and deploy eight system
components at no more than 20 depots over a 3-year period, including
5 prototype sites over the next 2 years.
2. Contrary to DUSD(L)'s position, the Major Automated Information
System Review Council (MAISRC) is chartered to approve selection of
migratory systems, but has not reviewed the DMSS initiative from a
life-cycle management and implementation strategy perspective. DOD
8020.1-M, Functional Management Process for Implementing the
Information Management Program of the Department of Defense, which
DUSD(L) states it is following to implement its DMSS initiative,
states:
Information system options and opportunities that support the
functional management strategy and process improvement efforts
are evaluated based on technical feasibility, cost, schedule,
performance, risk, and conformance to architectural guidelines
and standards. Information system development/modernization
must comply with life cycle management policy...The SDP [System
Decision Paper] will be part of the approval decision package
supporting the designation of the AIS [Automated Information
System] as a migration system by the OSD Principal Staff
Assistant. The SDP will also support an in-process review (IPR)
or milestone review, as appropriate, by the designated milestone
decision authority (MDA)...When AIS changes are part of the
process improvement alternative(s) selected for more detailed
analysis, the Functional Activity Program Manager's evaluation
decision is a filter that precedes other reviews required by
DoDD 8120.1...The Functional Activity Program Manager is
responsible for ensuring that the AIS-related aspects of the
process improvement proposal are reviewed and approved in
accordance with DoDD 8120.1, in addition to being reviewed and
approved by the OSD Principal Staff Assistant as part of the
complete process improvement proposal.
As stated in our report, Defense Directive 8120.1, Life-Cycle
Management (LCM) of Automated Information Systems (AISs); and Defense
Instruction, 8120.2, Automated Information System (AIS) Life-Cycle
Management (LCM) Process, Review, and Milestone Approval Procedures,
establish MAISRC review and MDA approval procedures to ensure that
decisions to develop or modernize major automated information systems
are based on sound business principles. Under these procedures, a
project should be reviewed and approved at each of five decision
milestones before substantial funds are obligated. Despite this
policy, DUSD(L) spent over $200 million to implement the DMSS
initiative without receiving approval for even the initial milestone
decision point.
Also, DUSD(L)'s claim that the MAISRC did review the DMSS initiative
on March 16, 1993, is not accurate. On this date the MAISRC
completed an In-Process Review (IPR) of the overall Logistics CIM
strategy. An IPR is defined as "An LCM review between LCM milestones
to determine the current program status, progress since the last LCM
review, program risks and risk-reduction measures, and potential
program problems." Further, as admitted by DUSD(L), the DMSS
initiative was not approved until early 1994--more than a year after
this review.
Finally, DUSD(L)'s contention that the MAISRC milestone reviews of
the Depot Maintenance Management Information System (DMMIS) or any
other single system component of the DMSS initiative satisfies the
Defense life-cycle management review and approval requirements is
inaccurate. Defense's life-cycle management directives define an
automated information system program as "A directed and funded AIS
effort, to include all migration systems, that is designated to
provide a new or improved capability in response to a validated
need." Further, the directives state:
For the purpose of determining whether an AIS is major, the
separate AISs that constitute a multi-element program, or that
make up an evolutionary or incremental development program, or
make up a multi-component AIS program, shall be aggregated and
considered a single AIS.
Based on these directives, DUSD(L) is required to obtain MAISRC
review and approval for the entire DMSS initiative at each of five
milestone decision points before any additional funds are spent.
3. DUSD(L) officials contend that reengineering of depot maintenance
processes is occurring concurrently with the deployment of the DMSS.
Further, they assert that these reengineering efforts will provide
dramatic economic benefits, and cite cost savings and productivity
increases accrued from initial implementation of four DMSS system
components as support. DUSD(L)'s approach focuses on the selection
of the best currently operating information systems and
implementation of these selected system across all Defense
components. While this approach may improve overall DOD business
processes and may provide incremental benefits, it is not the
fundamental rethinking and radical redesign of depot maintenance
processes and will not provide the dramatic cost reductions and
productivity gains available from process reengineering. At best, it
will allow DOD to accomplish current depot maintenance processes
faster and more efficiently. At worst, DUSD(L)'s focus on
information system selection and implementation will make future
reengineering efforts more difficult by entrenching current work
processes.
In January 1991, the Deputy Secretary of Defense endorsed a Corporate
Information Management initiative implementation plan that directed
business processes be reengineered before information systems are
standardized. However, DUSD(L) did not consider reengineering
opportunities as alternatives to the DMSS initiative. As discussed
in the report, the functional economic analysis used by DUSD(L) to
approve the DMSS initiative compared only three alternatives. All
three of these alternatives focused on using automated information
systems to improve current depot maintenance functions.
Further, as stated in DMSS documentation, the initiative is designed
to provide only incremental improvements to existing business
processes. It is clear from Defense's own benefit projections that
DMSS will not result in dramatic improvements possible from
consideration of reengineering-based solutions. DUSD(L) projected
DMSS would reduce the costs to DOD depot maintenance operational
costs over a 10-year period from $112.9 billion to $110.3 billion. A
cost reduction of $2.6 billion or only 2.3 percent over this period
does not constitute a dramatic increase in efficiency.
In late 1994, the Office of the Secretary of Defense for Command,
Control, Communications, and Intelligence, responsible for CIM
initiatives across DOD, found major flaws in the overall
implementation. It concluded that, as opposed to the private sector
which uses a very different approach, "DOD has virtually no chance of
making high impact/quantum changes using the current approach."
Further, the Commission on Roles and Missions of the Armed Forces,
charged by the Congress to provide an independent review of the roles
and missions of the armed services, has found that "[r]ather than
reengineering its processes, DOD has spent its energies in closing
excess capacity (base and facilities) and in standardizing its
management information systems" and concluded that DOD will achieve a
more compact, more standardized version of its traditional logistics
approach.\1 The Commission confirmed that DOD must radically
reengineer its logistics processes to achieve meaningful
improvements.
4. While information technology is critical to any reengineering
effort because it provides a tool for breaking old rules and creating
new ways of working, it should not be the driver of the reengineering
effort. Such an approach may make future reengineering efforts more
difficult by entrenching inefficient and ineffective work processes.
Reengineering offers order-of-magnitude improvement compared to the
incremental improvements DMSS is designed to provide. DUSD(L) can
not afford to deploy DMSS departmentwide beyond the first five
prototype sites until it has first determined which old rules need to
be broken and what new ways of accomplishing depot maintenance are
most efficient and effective. The Commission on Roles and Missions
of the Armed Forces has identified a number of alternatives for
changing the way DOD conducts its depot maintenance. These
alternatives could serve as a starting point for a thorough study by
DUSD(L) of its reengineering opportunities.
--------------------
\1 Logistics Issues Case Studies for the Roles and Missions
Commission of the Armed Forces, February 8, 1995.
DESCRIPTION OF DMSS COMPONENT
SYSTEMS
========================================================= Appendix III
This appendix provides brief descriptions of the eight information
systems selected as Depot Maintenance Standard System components to
support the DOD-wide depot maintenance function.
Baseline Advanced Industrial Management System: Supports allocation
decisions on resource application, schedules, and job management of
maintenance projects. It allows timely review of cost and schedule
performance at any level of the work breakdown structure. One of
this system's major modules, Programmed Depot Maintenance Scheduling
System, provides project schedules of individual maintenance
operations and critical path of work requirements for maintenance of
major end items.
Depot Maintenance Management Information System (DMMIS): Provides
depot maintenance managers with an automated capability to forecast
workloads; schedule repair activities; track and control inventories;
program staffing, materials, and other resources; and track and
manage production costs.
Enterprise Information System: Provides the ability to interface to
existing data sources, extract relevant data, and package the
information to support decisionmakers with timely summary
information.
Facilities and Equipment Maintenance: Provides an integrated
tracking and control system for equipment and facility maintenance,
preventive maintenance, and calibration of precision measurement
equipment.
Depot Maintenance Hazardous Materiel Maintenance System: Records the
receipt and issue of all hazardous material within a maintenance
depot. Provides inventory visibility of all hazardous material to
control the issue of hazardous material to authorized users.
Interservice Material Accounting and Control System: Provides the
tracking of Depot Maintenance Interservice Support agreements and
visibility and control for interservice workloads.
Laboratory Information Management System: Provides the monitoring
and control of laboratory data such as sample order status, order
tracking, backlog, scheduling, location tracking, workload
prediction, pricing, and invoicing. Automates tracking and archiving
for depot material samples and test results.
Tool Inventory Management Application: Provides total inventory
tracking and accountability of both hard and perishable (consumable)
tools and tooling assets. Tracks issues and receipts of assets to
both individuals and in tool kits.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix IV
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
Carl M. Urie, Assistant Director
David B. Shumate, Staff Evaluator
Cristina T. Chaplain, Communications Analyst
CINCINNATI REGIONAL OFFICE
Sanford F. Reigle, Evaluator-In-Charge
Roberto Rivera, Staff Evaluator
Thomas C. Hewlett, Staff Evaluator
RELATED GAO REPORTS
============================================================ Chapter 0
Government Reform: Using Reengineering and Technology to Improve
Government Performance (GAO/T-OGC-95-2, Feb. 2, 1995).
Defense Management: Impediments Jeopardize Logistics Corporate
Information Management (GAO/NSIAD-95-28, Oct. 21, 1994).
Commercial Practices: DOD Could Reduce Electronic Inventories by
Using Private Sector Techniques (GAO/NSIAD-94-110, Jun. 29, 1994).
Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology (GAO/AIMD-94-115, May 1994).
Commercial Practices: Leading Edge Practices Can Help DOD Better
Manage Clothing and Textile Stocks (GAO/NSIAD-94-64, Apr. 13, 1994).
Defense Management: Stronger Support Needed for Corporate
Information Management Initiative To Succeed (GAO/AIMD/NSIAD-94-101,
Apr. 12, 1994).
Defense IRM: Business Strategy Needed for Electronic Data
Interchange Program (GAO/AIMD-94-17, Dec. 9, 1993).
Defense Transportation: Commercial Practices Offer Improvement
Opportunities (GAO/NSIAD-94-26, Nov. 26, 1993).
Defense Inventory: Applying Commercial Purchasing Practices Should
Help Reduce Supply Costs (GAO/NSIAD-93-112, Aug. 6, 1993).
Commercial Practices: DOD Could Save Millions by Reducing
Maintenance and Repair Inventories (GAO/NSIAD-93-155, Jun. 7, 1993).
DOD Food Inventory: Using Private Sector Practices Can Reduce Costs
and Eliminate Problems (GAO/NSIAD-93-110, Jun. 4, 1993).
Defense ADP: Corporate Information Management Must Overcome Major
Problems (GAO/IMTEC-92-77, Sep. 14, 1992).
DOD Medical Inventory: Reductions Can Be Made Through the Use of
Commercial Practices (GAO/NSIAD-92-58, Dec. 5, 1991).
Commercial Practices: Opportunities Exist to Reduce Aircraft Engine
Support Costs (GAO/NSIAD-91-240, Jun. 28, 1991).
Defense Logistics: Observations on Private Sector Efforts to Improve
Operations (GAO/NSIAD-91-240, Jun. 13, 1991).