DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments (17-MAY-04, GAO-04-731R).
The Department of Defense's (DOD) long-standing business systems
problems adversely affect the economy, effectiveness, and
efficiency of its business operations and have resulted in a lack
of adequate transparency and appropriate accountability across
all of its major business areas. In July 2001, DOD initiated a
program to, among other things, develop a DOD business enterprise
architecture (architecture). This effort is an essential part of
the Secretary of Defense's broad initiative to "transform the way
the department works and what it works on." Because DOD is one of
the largest and most complex organizations in the world,
overhauling its business operations and supporting systems
represents a huge management challenge. In fiscal year 2003, DOD
reported that its operations involved over $1 trillion in assets,
nearly $1.6 trillion in liabilities, approximately 3.3 million
military and civilian personnel, and disbursements of over $416
billion. To support its business operations, DOD reported that it
relies on about 2,300 business systems, including accounting,
acquisition, logistics, and personnel systems. The department
requested about $19 billion--about $4.8 billion for business
systems modernization and about $14 billion for operation and
maintenance of these systems--in fiscal year 2004. Recognizing
the importance of DOD's efforts to transform its business
operations and systems through the use of an enterprise
architecture, the Congress included provisions in the National
Defense Authorization Act for Fiscal Year 2003 that were aimed at
developing and effectively implementing a well-defined
architecture. Specifically, section 1004 of this act required
that DOD (1) develop, by May 1, 2003, a financial management
enterprise architecture and a transition plan for implementing
the architecture that meets certain requirements and (2) review
financial system improvements with proposed obligations of funds
in amounts exceeding $1 million to determine if those system
improvements meet specific conditions that are called for in the
act. The act also directed us to assess actions that DOD has
taken to comply with these requirements. In July and September
2003, we reported on DOD's actions and made a number of
recommendations to assist DOD in its efforts to effectively
develop and implement an architecture and to guide and constrain
its business systems investments. The act further requires that
the Secretary of Defense submit an annual report not later than
March 15 of each year from 2004 through 2007 to congressional
defense committees on its progress in implementing the
architecture, including the transition plan. Additionally, the
act directs us to submit to congressional defense committees,
within 60 days of DOD's report submission, an assessment of DOD's
actions taken to comply with these requirements. DOD submitted
its first annual report on March 15, 2004. This report is our
assessment of DOD's March 15, 2004 report. We determined (1) the
actions DOD has taken to address our previous recommendations
regarding the development and implementation of the architecture
and (2) the actions DOD is taking to ensure its ongoing and
planned investments will be consistent with its evolving
architecture.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-731R
ACCNO: A10112
TITLE: DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments
DATE: 05/17/2004
SUBJECT: Defense cost control
Federal agency accounting systems
Financial management systems
Information resources management
Information technology
Management information systems
Strategic information systems planning
Systems conversions
Systems design
Systems management
Best practices
Investment planning
Performance measures
Accountability
Enterprise architecture
DOD Business Management Modernization
Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-731R
United States General Accounting Office Washington, DC 20548
May 17, 2004
Congressional Defense Committees
Subject: DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments
The Department of Defense's (DOD) long-standing business systems problems
adversely affect the economy, effectiveness, and efficiency of its
business operations and have resulted in a lack of adequate transparency
and appropriate accountability across all of its major business areas. To
help the department transform its operations, we recommended1 in 2001 that
DOD develop an enterprise architecture to guide and constrain its almost
$20 billion annual investment in business systems and that it establish
the investment controls needed to implement this architecture. In July
2001, DOD initiated a program2 to, among other things, develop a DOD
business enterprise architecture (architecture). This effort is an
essential part of the Secretary of Defense's broad initiative to
"transform the way the department works and what it works on."
Because DOD is one of the largest and most complex organizations in the
world, overhauling its business operations and supporting systems
represents a huge management challenge. In fiscal year 2003, DOD reported
that its operations involved over $1 trillion in assets, nearly $1.6
trillion in liabilities, approximately 3.3 million military and civilian
personnel, and disbursements of over $416 billion. To support its business
operations, DOD reported that it relies on about 2,300 business systems,
including accounting, acquisition, logistics, and personnel systems. The
3
department requested about $19 billion-about $4.8 billion for business
systems
1U.S. General Accounting Office, Information Technology: Architecture
Needed to Guide Modernization of DOD's Financial Operations, GAO-01-525
(Washington, D.C.: May 17, 2001).
2The Business Management Modernization Program is the department's
business transformation initiative; it encompasses defense policies,
processes, people, and systems that guide, perform, or support all aspects
of business management-including development and implementation of the
business enterprise architecture. The Under Secretary of Defense
(Comptroller) established a DODwide program management office called
Business Modernization and Systems Integration (BMSI), to oversee and
manage the program.
3Business systems include financial and nonfinancial systems, such as
civilian personnel, finance, health, logistics, military personnel,
procurement, and transportation, with the common element being the
generation or use of financial data to support DOD's business operations.
modernization and about $14 billion for operation and maintenance of these
systems-in fiscal year 2004.
Recognizing the importance of DOD's efforts to transform its business
operations and systems through the use of an enterprise architecture, the
Congress included provisions in the National Defense Authorization Act for
Fiscal Year 20034 that were aimed at developing and effectively
implementing a well-defined architecture. Specifically, section 1004 of
this act required that DOD (1) develop, by May 1, 2003, a financial
management enterprise architecture5 and a transition plan for implementing
the architecture that meets certain requirements and (2) review financial
system improvements with proposed obligations of funds in amounts
exceeding $1 million to determine if those system improvements meet
specific conditions that are called for in the act. The act also directed
us to assess actions that DOD has taken to comply with these requirements.
In July6 and September 2003,7 we reported on DOD's actions and made a
number of recommendations to assist DOD in its efforts to effectively
develop and implement an architecture and to guide and constrain its
business systems investments.
The act further requires that the Secretary of Defense submit an annual
report not later than March 15 of each year from 2004 through 2007 to
congressional defense committees on its progress in implementing the
architecture, including the transition plan. Additionally, the act directs
us to submit to congressional defense committees, within 60 days of DOD's
report submission, an assessment of DOD's actions taken to comply with
these requirements. (See enc. I for a copy of section 1004 of the act.)
DOD submitted its first annual report on March 15, 2004. This report is
our assessment of DOD's March 15, 2004 report. As agreed with your
offices, we determined (1) the actions DOD has taken to address our
previous recommendations regarding the development and implementation of
the architecture and (2) the actions DOD is taking to ensure its ongoing
and planned investments will be consistent with its evolving architecture.
4Bob Stump National Defense Authorization Act for Fiscal Year 2003, Pub.
L. No. 107-314, S: 1004, 116 Stat. 2458, 2629 (Dec. 2, 2002).
5In May 2003, the DOD Comptroller changed the architecture name from the
Financial Management Enterprise Architecture to the Business Enterprise
Architecture to reflect the transformation of departmentwide business
operations and supporting systems, including accounting and finance,
budget formulation, acquisition, inventory management, logistics,
personnel, and property management systems.
6U.S. General Accounting Office, DOD Business Systems Modernization:
Summary of GAO's Assessment of the Department of Defense's Initial
Business Enterprise Architecture, GAO-03-877R (Washington, D.C.: July 7,
2003).
7U.S. General Accounting Office, DOD Business Systems Modernization:
Important Progress Made to Develop Business Enterprise Architecture, but
Much Work Remains, GAO-03-1018 (Washington, D.C.: Sept. 19, 2003).
We performed our work from December 2003 through April 2004 in accordance
with U.S. generally accepted government auditing standards. Details on our
scope and methodology are in enclosure II.
Results in Brief
Since our last review-and after 3 years of effort and over $203 million in
obligations-we have not seen any significant change in the content of
DOD's architecture or in DOD's approach to investing billions of dollars
annually in existing and new systems. Few actions have been taken to
address the recommendations we made in our September 20038 report, which
were aimed at improving DOD's plans for developing the next version of the
architecture and implementing the institutional means for selecting and
controlling both planned and ongoing business systems investments. To
DOD's credit, it has established, for example, a group under the Business
Management Modernization Program (program) steering committee9 to
facilitate communication and coordination across the domains10 for
modernization program activities, including extending and evolving the
architecture. However, DOD has not yet adopted key architecture management
best practices11 and has not added the scope and detail to its
architecture that we previously identified as missing. Further, DOD has
not yet implemented an effective management structure and processes to
provide adequate control and accountability over its $5 billion annual
investment in business systems modernization. Additionally, the department
does not have reasonable assurance that it is in compliance with the
National Defense Authorization Act for Fiscal Year 2003, which requires
DOD's Comptroller to review all system improvements with obligations
exceeding $1 million.
Regarding architecture management best practices, DOD has not yet
implemented key elements, such as assigning accountability and
responsibility for directing, overseeing, and approving the architecture.
In addition, it has not explicitly defined performance metrics to evaluate
the architecture's quality, content, and utility. With
8GAO-03-1018.
9The steering committee, made up of senior leaders from across the
department, is advisory in nature and is not accountable for directing,
overseeing, and approving the architecture.
10DOD has six departmental domains, which are (1) accounting and finance,
(2) acquisition, (3) human resources management, (4) installations and
environment, (5) logistics, and (6) strategic planning and budgeting. It
also has one enterprise information environment mission area. The domains
and the mission area, comprised of the Under Secretaries of Defense and
the Assistant Secretary of Defense for Networks and Information
Integration/DOD Chief Information Officer, have authority, responsibility,
and accountability within their domains for business transformation,
implementation of the architecture, development and execution of the
transition plan, portfolio management, and establishment of a structure to
ensure representation of the DOD components and the appropriate federal
agencies.
11U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003).
respect to the architectural content, DOD's latest version of the
architecture does not include many of the key elements of a well-defined
"As Is," "To Be," and transition plan that we previously reported as not
being satisfied.12 For example, the "To Be" environment does not provide
sufficient descriptive content related to future business operations and
supporting technology to permit effective acquisition and implementation
of system solutions and associated operational change. Similarly, DOD's
verification and validation contractor has concluded that this latest
version of the architecture retains most of the limitations that the
initial version had. Moreover, DOD has not yet defined specific plans,
including milestones, detailing how it intends to extend and evolve the
architecture to incorporate this missing content. For example, the
program's first objective for increment one-to enable asset
accountability, total force visibility, and an unqualified audit opinion
on DOD's fiscal year 2007 consolidated financial statements-is not
supported by a DOD-wide plan of action, and the individual component plans
for accomplishing this objective may not be linked to the program's
activities, including the architecture. According to DOD, it will not have
the plans on how the capabilities for increment one will be achieved until
August 2004.
DOD has also made limited progress in addressing our recommendations aimed
at establishing and implementing effective investment management processes
and, therefore, continues to lack effective management oversight and
control over ongoing business systems modernization investments. While DOD
has recently issued a policy that assigns investment management
responsibilities to the domains, the policy has not yet been implemented,
and DOD has not clearly defined the roles and responsibilities of the
domains, established common investment criteria, and conducted a
comprehensive review of its existing business systems to ensure that they
are consistent with the architecture. Further, each of the DOD components
continues to receive its own funding and make its own parochial investment
decisions. Moreover, DOD has not yet established and implemented an
effective process for ensuring that system improvements with obligations
exceeding $1 million are submitted to DOD's Comptroller for review and to
determine whether they are consistent with the architecture, as required
by the National Defense Authorization Act for Fiscal Year 2003. Based upon
a comparison of limited information provided by the military services and
defense agencies, we identified a total of $863 million in obligations for
improvements that exceeded $1 million each but had not been submitted to
DOD's Comptroller for review and determination.
The department acknowledges that it still has much more to do, including
developing the architecture to a necessary level of detail, defining
specific performance metrics, and clarifying the roles and
responsibilities associated with managing the domains' portfolios of
business systems and ensuring that these systems comply with the
architecture. The limited progress that DOD has made is due, in part, to
the lack of clearly assigned, accountable, and sustained program
leadership and to changes in the program direction and priorities. For
example, from May 2003 to February 2004, there was no program manager to
identify, direct, and execute program activities.
12GAO-03-1018.
Our experience in reviewing other challenged architecture efforts shows
that these efforts have suffered from limited senior management
understanding of and commitment to an architecture and from cultural
resistance to having and using one. Cultural resistance to change,
organization component parochialism, and stovepiped operations are all
evident at DOD.
Because many of our prior recommendations-for managing architecture
development, maintenance, and implementation and for controlling ongoing
and planned investments in business systems-remain open, we are not making
any new recommendations in this report, but we are reiterating the 22 open
recommendations that we made in our May 2001,13 February 2003,14 and
September 200315 reports. (Enc. III contains details on the status of all
of our prior recommendations, including our assessment of DOD's actions.)
It is imperative that DOD act swiftly to implement these recommendations.
If it does not, the prognosis for this program is bleak, which in turn
puts the department's business transformation efforts in jeopardy.
In written comments, which are reprinted in enclosure IV, DOD agreed with
our assessment that the department's long-standing business systems
problems have resulted in a lack of adequate transparency and appropriate
accountability across all major business areas and that development and
use of an architecture is necessary to transforming its business
operations and supporting systems. In response to our characterization
that progress has been limited, DOD stated that, over the past 3 years,
progress has been slower than either GAO or DOD would prefer but that it
has been significant, despite appearances to the contrary. In support of
its view, DOD provided an extensive package of information. We considered
this information, most of which we had previously evaluated. The
willingness of DOD's leadership to tackle the department's decades-old
problems with its business operations and supporting systems represents a
major step forward. However, we remain convinced that it is fair to
characterize DOD's progress in developing a well-defined architecture and
implementing effective management oversight and control over its business
systems as limited. As a result, we did not make any changes to this
report.
13GAO-01-525.
14U.S. General Accounting Office, DOD Business Systems Modernization:
Improvements to Enterprise Architecture Development and Implementation
Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28, 2003).
15GAO-03-1018.
Background
Prior Reviews of DOD's Architecture Efforts Have Identified Challenges and
Weaknesses
Over the last 3 years, we have reported16 that one of the key elements to
successfully meeting DOD's financial and related business management
challenges is establishing and implementing an enterprise architecture, or
modernization blueprint. An enterprise architecture provides a clear and
comprehensive picture of an entity, whether it is an organization (e.g.,
federal department or agency) or a functional or mission area that cuts
across more than one organization (e.g., financial management). This
picture consists of snapshots of both the enterprise's current or "As Is"
operational and technological environment and its target or "To Be"
environment, as well as a capital investment road map for transitioning
from the current to the target environment. These snapshots further
consist of "views," which are basically one or more architecture products
that provide conceptual or logical representations of the enterprise. We
have made numerous recommendations to assist DOD in successfully
developing the architecture and using it to gain control over its ongoing
business systems investments. Enclosure III contains details on the status
of all of our prior recommendations, including our assessment of DOD's
actions.
In May 200117, we reported that the department did not have an
architecture for its financial and financial-related business operations,
nor the management structures, processes, and controls in place to
effectively develop and implement one. We reported that if the department
continued to spend billions of dollars on new and modified systems,
independently from one another and outside the context of an architecture,
this would result in more processes and systems that are duplicative, not
interoperable, and unnecessarily costly to maintain and interface. We made
eight recommendations aimed at providing the means for effectively
developing and implementing an architecture. The Secretary of Defense
established a program in July 2001 to develop and implement an
architecture. In April 2002, DOD entered into an agreement with
International Business Machines (IBM) pursuant to a governmentwide General
Services Administration contract, under which DOD issued task orders for
services to begin developing the architecture.
During the first year of DOD's architecture development, in 2002, we
reviewed the department's efforts and recognized that it was undertaking a
challenging and
16GAO-01-525; U.S. General Accounting Office, DOD Financial Management:
Important Steps Underway But Reform Will Require a Long-term Commitment,
GAO-02-784T (Washington, D.C.: June 4, 2002); and U.S. General Accounting
Office, Department of Defense: Further Actions Needed to Establish and
Implement a Framework for Successful Business Transformation, GAO-04-626T
(Washington, D.C.: Mar. 31, 2004).
17GAO-01-525.
ambitious task and following some architecture best practices and
information technology (IT) investment management processes and controls.
We also identified challenges and weaknesses in DOD's architecture
efforts. For example, in a February 2003 report,18 we pointed out that DOD
had not yet (1) established a governance structure and process controls
needed to ensure ownership of and accountability for the architecture
across the department, (2) clearly communicated to intended stakeholders
its purpose, scope, and approach for developing the architecture, and (3)
defined and implemented an independent quality assurance process. We also
reported that DOD had yet to establish the necessary departmental
investment governance structure and process controls needed to adequately
align ongoing investments with its architectural goals and direction. We
made six recommendations aimed at enhancing DOD's ability to further
develop its architecture and guide and constrain its business systems
modernization investments.
Our March 200319 report noted that the draft version of the architecture
did not include a number of items recommended by relevant architectural
guidance and that DOD's plans would not fully satisfy the requirements of
the National Defense Authorization Act for Fiscal Year 2003. For example,
the draft architecture did not include a "To Be" security view, which
defines the security requirements, including relevant standards to be
applied in implementing security policies, procedures, and controls. DOD
officials agreed with our preliminary assessment of the architecture and
stated that subsequent versions of the architecture would provide these
missing details.
In July and September 2003,20 we reported that although DOD had expended
tremendous effort and resources in complying with statutory requirements
for developing and implementing a well-defined architecture, the initial
version of its architecture, including the transition plan, did not
adequately address the statutory requirements and other relevant
architectural requirements. For example, the "As Is" environment did not
include descriptions of the current business operations in terms of
entities and people who perform the functions, processes, and activities
and the locations where these are performed. The "To Be" environment did
not include descriptions of actual systems to be developed or acquired to
support future business operations and the physical infrastructure that
would be needed to support the business systems. The transition plan did
not include time frames for phasing out existing systems within DOD's
reported current inventory of about 2,300 business systems. Overall, the
department's initial version of the architecture did not contain
sufficient scope and detail either to satisfy the act's requirements or to
effectively guide and constrain the departmentwide business transformation
and systems
18GAO-03-458.
19U.S. General Accounting Office, Information Technology: Observations on
Department of Defense's Draft Enterprise Architecture, GAO-03-571R
(Washington, D.C.: Mar. 28, 2003).
20GAO-03-877R and GAO-03-1018.
modernization. In our September 2003 report21, we reiterated the open
recommendations that we had made in our May 200122 and February 200323
reports, and we made 10 new recommendations aimed at improving DOD's plans
for developing the next version of the architecture and implementing the
institutional means for selecting and controlling both planned and ongoing
business systems investments. To date, DOD has yet to address 22 of our
recommendations.
Funding Status of Architecture Program
As of March 12, 2004, about $258 million had been appropriated to support
the program, of which the department reported having obligated over $203
million and having made disbursements of $111 million since the program
began in 2002. Table 1 shows the reported status of program funding by
appropriation and fiscal year.
21 GAO-03-1018.
22GAO-01-525.
23GAO-03-458.
Table 1: Reported Funding Status of DOD's Business Management
Modernization
Program as of March 12, 2004
(Dollars in millions)
Appropriation Appropriated Obligated Unobligateda Disbursed Unliquidatedb
Fiscal year 2002/2003
Research,
Development, Test,
and Evaluation
(RDT&E) -
Defensewide (DW)c $94.5 $94.5 $0.0 $83.0 $11.5
Fiscal year 2003/2004
RDT&E -DWd 67.2 67.2 0.0 11.9 55.3
Fiscal year 2004/2005
RDT&E - DWe 45.1 8.5 36.6 0.0
Fiscal year 2003
Operations and
Maintenance (O&M) -
DWf 24.9 24.9 0.0 16.1
Fiscal year 2004 O&M-
DWg 26.1 8.3 17.8 0.0
Total $257.8 $203.4 $54.4 $111.0 $92.4
Source: Unaudited funding information contained in DOD's March 15, 2004
report to congressional defense committees. We did not validate the
accuracy and reliability of the funding information reported.
a
Unobligated balances are the differences between funds appropriated and
obligated; they represent funds that have not been obligated for
expenditure. GAO's calculations are based on DOD's reported data.
b Unliquidated balances are the differences between the funds obligated
and disbursed; they represent funds that have been obligated but have not
yet been paid. GAO's calculations are based on DOD's reported data.
c The fiscal year 2002/2003 RDT&E funds supported the initial delivery of
the architecture, transition plan, and change management and
communications initiatives.
d The fiscal year 2003/2004 RDT&E funds were used to refine the
architecture through business process modeling/reengineering for the
program's first increment and to fund test and evaluation activities,
verification and validation efforts, and engineering support.
e The fiscal year 2004/2005 RDT&E funds will be used to complete the
program's first increment and begin work on the second increment and to
fund the integration of the architecture, engineering support, and test
and evaluation activities.
f The fiscal year 2003 O&M funds were used for salaries, facilities,
supplies, and program management support contracts.
g The fiscal year 2004 O&M funds are being used for salaries, facilities,
supplies, and program management support contracts.
DOD Has Taken Few Actions to Address Our Previous Recommendations
DOD has taken some steps since our last review, but it has not yet (1)
implemented key architecture management best practices,24 such as
assigning accountability and responsibility for directing, overseeing, and
approving the architecture to a committee or group comprised of
representatives from across DOD's major organization components, (2)
revised the architecture products to include the missing scope and detail
needed to guide and constrain the implementation of system solutions, and
(3) clearly defined near-term and long-term plans for guiding its
transformation activities. Until the department addresses these key
elements to fully satisfy relevant architectural guidance, it will be
challenged in its ability to produce an architecture that is sufficient to
guide and constrain its business operations and systems modernization
efforts. In response to our recommendations, DOD has taken some actions,
including establishing a group to facilitate communication and
coordination across the domains for program activities; beginning to
establish a configuration change management process; recently issuing a
policy governing the development, maintenance, and implementation of the
architecture; and updating the initial version of the architecture.
However, these actions are not sufficient and do not fully address our
concerns.
DOD Has Not Yet Implemented Key Architecture Management Best Practices
DOD's actions have not been adequate to address architecture management
best practices or our previous recommendations. Since our last report, the
department has taken some actions in response to our recommendations
concerning the need to implement an effective architecture management
program. First, in September 2003, it formally established the Domain
Owners Integration Team (DO/IT), which reports to the steering committee.
The DO/IT is comprised of the various senior executives from each domain
and the Business Modernization and Systems Integration office. The DO/IT
is responsible for facilitating communication and coordination across the
domains for program activities, including extending and evolving the
architecture. According to program officials, the DO/IT's role is
continuing to evolve. In particular, the specific actions or tasks it
needs to perform to carry out this responsibility have not been defined in
detail.
Second, DOD has taken steps to establish and implement a configuration
management process to ensure that all changes to the architecture products
are justified and are accounted for in a manner that maintains
documentation integrity. Specifically, the department has established a
configuration control board (CCB), developed a charter, and written
procedures governing this process. However, the CCB has been tasked with
reviewing changes to only some, but not all, of the architecture products.
For example, the CCB is not responsible for tracking changes that are
being made to the transition plan. In addition, both the charter and the
procedures governing this process are still in draft. According to DOD
officials, the
24GAO-03-584G.
charter is to be approved in June 2004, but no time frame has been
provided for final approval of the procedures.
Third, DOD has recently issued an IT portfolio management policy governing
the development, maintenance, and implementation of the architecture. This
policy addresses, among other things, the roles, responsibilities, and
relationships of key players and program participants, the value of an
architecture, and the scope of the architecture. However, the policy does
not address accountability for and approval of updates to the
architecture, as called for by best practices, nor does it address the
issuance of waivers in those instances when exceptions to the architecture
are justified on the basis of documented analysis.
Finally, DOD has developed high-level performance measures that are to be
used to develop the more specific results-oriented performance metrics
needed to enable it to evaluate program progress and benefits. This means
that DOD has yet to establish measurable, results-oriented goals to
evaluate and track, on an ongoing basis, specific program progress,
outcomes, and results. For example, it has not explicitly defined
performance measures to evaluate the quality, content, and utility of
subsequent major updates to its initial architecture. Given that DOD has
reported obligations of over $203 million since architecture development
efforts began 3 years ago, this is a serious performance management
weakness. It is critical that the department establishes meaningful,
tangible, and measurable program goals and objectives- short-term and
long-term-to enable the department to determine what value it is receiving
for its investment.
In addition, the department has yet to address other prior recommendations
related to architecture management best practices. Two examples are
provided below.
o DOD has not assigned accountability and responsibility for directing,
overseeing, and approving the architecture to a committee or group
comprised of representatives from across the department. Although it
previously established an executive and a steering committee, these
committees are advisory in nature, and they are currently not accountable
or responsible for directing, overseeing, and approving the architecture.
According to the department, it expects to revise the committees' charters
in June 2004 to include these responsibilities.
o DOD does not have an independent verification and validation function
to review the architecture products and management processes. The
department's current verification and validation contractor is not
independent25 and is responsible for reviewing the architecture products
and only selected program deliverables. According to program officials,
the department is planning to acquire the services of a new contractor to
perform this work, who will be tasked with reviewing all
25Best practices recommend that the verification and validation function
be independent of the architecture program and report directly to the
steering committee.
the architecture products and all other program deliverables. However, DOD
officials also said that no decision has been made to change the reporting
structure of this function to make it independent or to have this
contractor review architecture management processes. In addition, no
milestones have been set for any of these planned actions.
Several factors have contributed to the department's limited progress in
implementing an effective architecture management program, such as changes
to and a lack of accountability in program leadership, direction, and
priorities. As we previously stated, DOD has not yet formalized
responsibility for directing, overseeing, and approving the architecture.
Further, from May 2003 to February 2004, there was no program manager to
identify, direct, and execute program activities. DOD hired a new program
manager in February 2004. Initially, DOD had planned to develop and
implement its architecture in 1 year, but later it adopted an incremental
approach to developing the architecture. About 1 year after adopting this
approach, DOD has assigned labels to these increments, but it has not yet
defined the purpose of these increments and plans of action to implement
them.
Further, our research of successful organizations and experience in
reviewing other challenged enterprise architecture efforts show that
senior management's understanding of and commitment to an enterprise
architecture and overcoming cultural resistance to having and using one
are critical success factors. Cultural resistance to change, military
service parochialism, and stovepiped operations have all contributed
significantly to the failure of previous attempts to implement broadbased
management reforms at DOD. Until such barriers are addressed, and
effective architecture management structures and processes are
established, it is unlikely that DOD will be able to produce and maintain
a complete and enforceable architecture or implement modernized systems in
a way that minimizes overlap and duplication and maximizes integration and
mission support.
DOD's Architecture Products Remain Incomplete, with Minimal Content Change
Since our last review, DOD has not made significant changes to the content
of its architecture. The department has an updated version of the
architecture (version 2.0), which is currently being reviewed, but it has
not been approved. According to DOD, this version of the architecture
differs from the initial architecture (version 1.0) in that it
incorporates integrated "baseline reference business process models,"
which, according to DOD, constitute a high-level process framework
consisting of processes that provide a business perspective on the
department's operations (e.g., execute budget and performance plans) that
cut across functions and organizations. DOD also stated that version 2.0
includes the previously excluded requirements (e.g., revenue requirements)
for the Joint Financial Management Improvement Program (JFMIP)26 and
partially addresses 21 of the 62 missing architecture content elements
26JFMIP requirements arise from various public laws, regulations,
bulletins, circulars, federal accounting standards, and leading practices
and are applicable governmentwide.
that we recommended be added. According to DOD, the domains are currently
validating all of the requirements-including the JFMIP requirements-that
are contained in the architecture and, as a result, requirements may be
added or deleted.
Further, with regard to the 21 missing elements, documentation provided by
DOD shows that it is either just beginning to initiate activities or was
planning to develop plans to address these missing elements. This
documentation also did not specify either how or when any of the 21
elements would be fully addressed. Instead, it showed that the department,
to date, has made only cosmetic changes to the architecture, such as
correcting typographical (i.e., misspelled words) and grammatical errors,
as well as deleting unnecessary text from various architecture products.
DOD has yet to incorporate the missing scope and detail that we previously
identified. For example, the "To Be" environment does not provide
sufficient descriptive content related to future business operations and
supporting technology to permit effective acquisition and implementation
of system solutions and the associated operational change.
Program officials also stated that version 2.0 addresses some of the
recommendations made by its verification and validation contractor.
However, DOD did not provide documentation showing that it had addressed
any of these recommendations. For example, the verification and validation
contractor had previously reported that the "As Is" information was
insufficient to support realistic transition planning. In April 2004, this
contractor further reported that, while there have been some improvements
in both the overall completeness of definitions and the structure of the
architecture products, version 2.027 retains most of the critical problems
previously cited for version 1.0. Some examples are below.
o The utility of the architecture to stakeholders and the ability of the
architecture to support acquisition and portfolio investment management
decisions remain unclear.
o Supporting documentation that describes the analysis and rationale for
architecture choices represented within many of the architecture products
remains incomplete in many areas and, when available, is poorly linked or
referenced.
o The "As Is" environment, including business processes and existing
business application systems and supporting technology, is inadequate,
making it difficult for DOD to perform a gap analysis to support
development of a transition plan.
o The information assurance (security) representation remains spotty and
difficult to find even at the enterprise level, and it has some
unnecessary deviations from the department's accepted practices.
27The focus of this review was limited to the "As Is" and "To Be"
architecture products, because the transition plan has not been updated.
In addition, the verification and validation contractor stated that the
reference business process models included in version 2.0, which DOD cited
as the major difference from version 1.0, lacked the underlying data and
linkage to make it useful in reengineering existing processes. The
contractor also stated that these models would likely need to be reworked
based on the results of efforts that the domains currently have under way.
According to DOD and contractor officials responsible for updating the
architecture products, the changes made to date have primarily affected
architecture management processes and not the architecture's content; and
this is the result of the change in the program's focus and priorities, as
discussed above. However, these officials also stated that the
Architecture Integration Teams (AIT), comprised of representatives from
the domains, are currently focused on developing needed architectural
content-primarily business processes, business rules, and regulations for
increment one.
Until the architecture is sufficiently complete and includes the missing
scope and detail, the department remains at risk for not achieving its
intended business transformation goals and of not having an architecture
that can be used to guide and constrain ongoing and planned business
systems investments to prevent duplicative and noninteroperable systems.
DOD Has Yet To Explicitly Define Near-term and Long-term Plans for Guiding
Its Transformation Activities
DOD has not yet developed either near-term or long-term plans for
developing the architecture that explicitly identify and establish a
baseline for the actions to be taken, milestones to be achieved, cost
estimates to be met, and targeted outcomes to be achieved. As we
previously stated, DOD has adopted an incremental approach to developing
the architecture, including the transition plan, and plans to refine and
extend the architecture in three increments. The increments, as defined by
DOD, are:
o Increment one. To enable asset accountability, total force visibility,
and an unqualified audit opinion of DOD's consolidated fiscal year 2007
financial statements.
o Increment two. To focus on reducing acquisition cycle time and
streamlining the Planning, Programming, Budgeting, and Execution System
process between fiscal years 2004 and 2009.
o Increment three. To focus on providing total asset visibility and
total force management between fiscal years 2004 and 2010.
However, it is unclear what the increments individually or collectively
mean, and what they will provide or allow DOD to achieve in the near-term
and long-term, because DOD does not have detailed plans that include
performance measures for
the quality, content, and utility of the architecture. Although the three
increments were identified in November 2003, program officials do not
expect to have a plan for increment one until the next version of the
transition plan is completed in August 2004. According to program
officials, the goals and scope for the second and third increments were
only recently approved by the steering committee and, therefore, detailed
plans of action and milestones for developing these plans do not yet
exist.
Currently, DOD has three initiatives under way to support increment one.
First, the program office is developing a plan of action for increment one
and intends to complete the plan by August 2004. Second, the accounting
and finance domain is conducting workshops to develop needed business
rules and requirements for extending and evolving version 2.0 of the
architecture. This domain also has two ongoing pilot initiatives to
acquire and implement production accounting systems by the end of 2005 at
an estimated cost of $135 million, which are intended to provide
JFMIP-compliant financial management systems to support the department's
general and working capital fund activities. Last, DOD components are
developing individual plans detailing their respective efforts for
supporting increment one. However, there is no evidence that the program
office is coordinating with the components and that the components are
coordinating amongst themselves. According to a Defense Logistics Agency
(DLA) official, there is currently no direct link between DLA's plan and
modernization program activities. Because there are not yet detailed plans
guiding the program's activities, it is unclear whether and how these
activities support each other and whether they support the department's
goal of achieving an unqualified audit opinion in 2007.
DOD recognizes that it needs to develop detailed plans and establish
performance metrics to measure and track program progress in order to
determine (1) what it wants to accomplish by a certain point in time, (2)
what it has actually accomplished by that point in time, and (3) how much
it has spent. Changes in the direction of the program and a lack of
sustained leadership have hindered DOD's ability to do so. In its March
15, 2004 progress report, DOD reported that it plans to establish an
initial approved program metrics baseline to evaluate the cost, schedule,
and performance of the program and that, beginning with the fourth quarter
of fiscal year 2004, it plans to begin formal tracking and reporting of
specific program goals, objectives, and measures.
The lack of explicitly defined program plans also has made it difficult to
define measurable tasks that are or will be assigned to DOD employees and
the 289 program contractor employees who are involved in architecture
development, domain support, and program support functions. In reviewing
the contractor's work statements for architecture development, we found
that many of the tasks lacked the specificity necessary to use them
effectively to monitor the contractor's progress. For example, tasks
included such broad statements as "maintain, extend, and integrate" the
architecture. Since DOD is acquiring services on the basis of direct labor
hours at specified fixed hourly rates and cost of materials, it is
essential that the department use efficient methods and effective cost
controls to measure the work
being performed.28 It is also essential because the proposed work
statement that DOD plans to approve, at an estimated cost of $43 million,
explicitly states that work is considered complete when the hours allotted
have been expended, rather than defining completion as the contractor's
delivery of products that meet predefined quality standards. According to
DOD, to enable it to better monitor the contractor's progress, it plans to
reduce future tasks to 15 staff-day increments (from 6 week increments)
and to require the contractor to provide more frequent, informal products
to program officials so they can monitor and track progress more
rigorously.
Without an explicitly defined program baseline, detailed plans, and
performance measures, it is difficult to validate or justify the $122
million that DOD has requested for fiscal year 2005 and the $494 million
the department estimates it will need for fiscal years 2006 through 2009.
In fact, DOD has been unable to show measurable progress and meaningful
utility of the architecture products to DOD stakeholders for the $203.4
million that had been obligated for this program as of March 2004.
DOD Has Taken Few Actions to Control Ongoing and Planned Investments in
Business Systems
DOD continues to lack effective investment management oversight and
control over its numerous investments in business systems. While the
domains have been designated to oversee business systems investments, the
actual funding continues to be spread among DOD's components, which
continue to make their own parochial decisions regarding those
investments, including obligations in excess of $1 million for system
improvements, without having received the scrutiny of DOD's Comptroller as
required by the National Defense Authorization Act for Fiscal Year 2003.29
Because DOD lacks a departmentwide focus and effective management
oversight and control of its business systems investments, it continues to
invest billions of dollars in systems that potentially will fail to
deliver the integrated business systems outcomes
28We also found that in issuing task orders to International Business
Machines (IBM) pursuant to the General Services Administration contract,
DOD did not execute a Determination and Finding to support its use of
Time-and-Materials task orders, as required by Federal Acquisition
Regulation 16.601, 48 C.F.R. S: 16.601 (2003). This Determination and
Finding requirement guides agencies in evaluating the cost and performance
risks assumed by the government in awarding Time-and-Materials contracts
compared to other types of contracts that may provide increased incentives
for the contractor to control costs and efficiency.
29 Subsection 1004(d) of the Bob Stump National Defense Authorization Act
for Fiscal Year 2003, Pub.
L. No. 107-314, 116 Stat. 2630, (Dec. 2, 2002), provides that any amount
in excess of $1 million may be obligated for Defense financial system
improvements before approval of its enterprise architecture and a
supporting transition plan only if the DOD Comptroller makes a
determination that the improvement is necessary for (1) critical national
security capability or critical safety and security requirements or (2)
prevention of significant adverse effect on a project that is needed to
achieve an essential capability. The act further provides that after the
architecture and transition plan are approved, the DOD Comptroller must
determine, before making obligations that exceed $1 million for system
improvements, that such improvements are consistent with the enterprise
architecture and the transition plan.
that are needed to provide DOD management with timely and reliable
financial information.
We previously recommended30 that DOD establish investment review boards to
better control its business systems investments (with each board comprised
of representatives from across the department) and that the boards,
consistent with recognized best practices, use a standard set of
investment review and decisionmaking criteria to ensure compliance and
consistency with the architecture. DOD agreed with our recommendations
and, in response, it issued in March 2004 an IT portfolio management
policy that assigns the domains responsibility for IT portfolio
management. However, the procedures to be followed to implement the policy
are still under development and no time frames for completion have been
provided. According to the IT portfolio management policy, the department
has 180 days (until mid-September 2004) to develop these procedures. In
addition, the department has yet to formalize specific roles and
responsibilities of the domains, develop standard criteria for performing
the system reviews, and assign explicit authority for fulfilling roles and
responsibilities. DOD recognizes the need to clarify the roles and
responsibilities associated with managing the domains' portfolios of
business systems and ensure compliance with the architecture. However, it
has yet to establish time frames for completing these activities.
DOD has not yet implemented an effective investment management structure
and processes for controlling ongoing and planned business systems
investments, including one that meets the act's requirements for ensuring
that system improvements with obligations in excess of $1 million are
consistent with the architecture. In an attempt to substantiate that the
obligations for business systems modernization were in accordance with the
National Defense Authorization Act for Fiscal Year 2003, we requested that
DOD agencies provide us with a list of obligations greater than $1 million
for fiscal year 200331 and fiscal year 2004, as of December 2003. To
ascertain whether DOD's Comptroller had made the determination required by
the act, we compared a list of systems approvals provided by the program
office with the obligational data (by system) provided by the DOD
activities. As we previously testified,32 we identified $479 million in
obligations for business systems modernization reported by the DOD
military services that were not submitted to DOD's Comptroller for review.
Subsequently, we requested information from the defense agencies and found
$384 million in reported obligations for business systems modernization
that had not been submitted to DOD's Comptroller for review. Examples of
DOD system improvements with obligations in excess of $1 million that were
not submitted include the following:
30GAO-01-525 and GAO-03-458.
31We requested the obligational data for fiscal year 2003 for the period
December 2, 2002, the date of enactment of the act, through September
2003.
32GAO-04-626T.
o The Defense Finance and Accounting Service (DFAS) obligated about $19
million in fiscal year 2003 for the DFAS Corporate Database/DFAS Corporate
Warehouse. We previously reported33 that DOD had yet to provide economic
justification that its investment in this system would result in tangible
improvements to DOD's financial management operations. As of April 2004,
an economic analysis has yet to be approved but, according to DOD
officials, about $129 million had been spent on the program through
January 2004.
o The Defense Information Systems Agency obligated about $7 million in
fiscal year 2003 and about $2 million in fiscal year 2004 for the Wide
Area Workflow.34
o DFAS obligated about $7 million in fiscal year 2003 for the Defense
Standard Disbursing System before it was terminated in December 2003 after
approximately 7 years of effort and a reported investment of about $53
million. We previously reported35 that continued investment in this system
had not been justified, because an economic analysis had not been updated
to reflect significant schedule delays. DFAS noted that this system was
terminated because a valid business case for continuing the effort could
not be made.
o DLA obligated about $5 million in fiscal year 2003 and about $2
million in fiscal year 2004 for the Defense Medical Logistics Standard
Support Program.
These and other examples demonstrate that DOD does not have reasonable
assurance that it is in compliance with the National Defense Authorization
Act for Fiscal Year 2003, which provides that obligations in excess of $1
million for system improvements may not be made unless DOD Comptroller
makes a determination that the improvement is in accordance with the
criteria specified in the act. The act places limitations on the legal
authority of individual program and government contracting officials to
obligate funds in support of the systems for which they are responsible,
but DOD has yet to proactively manage its investments to avoid violations
of the limitations and to review and approve investments in any meaningful
way in order to comply with these statutory limitations.
DOD acknowledges that the department does not have a systematic means to
identify and determine which system improvements should be submitted to
DOD's Comptroller for review and, in essence, is dependent upon system
owners coming forward to the domains and requesting approval. Also, as we
have testified,36 DOD components continue to receive direct funding for
their business systems and continue to make their own parochial decisions
regarding investments without the
33U.S. General Accounting Office, DOD Business Systems Modernization:
Continued Investment in Key Accounting Systems Needs to be Justified,
GAO-03-465 (Washington, D.C.: Mar. 28, 2003).
34Wide Area Workflow is a secure web-based system for electronic
invoicing, receipt, and acceptance.
35GAO-03-465.
36GAO-04-626T.
scrutiny of DOD's Comptroller that is required by the act. The current
funding process has contributed to the proliferation of duplicative,
nonintegrated, and stovepiped business systems that are highly prone to
error and are unable to provide DOD management with timely, reliable
financial information. In March 2004,37 we offered a suggestion for
legislative action to address this issue, consistent with our open
recommendations to DOD that funds for its business systems be appropriated
directly to the domains in order to provide for accountability,
transparency, and the ability to prevent the continued parochial approach
to systems investments that exists today. The legislation we
envisioned-for which we will make a legislative proposal in a related
report to be issued soon-would define the scope of the domains and
establish functional responsibility for managing the portfolio of business
systems to the domains. The domains would establish business systems
investment review boards with DOD-wide representation, including the
military services and defense agencies.
Until DOD strengthens its process for selecting and controlling business
systems investments and sufficiently develops a well-defined architecture
that can be used to guide and constrain investment decisions, it will
likely continue to spend billions of dollars on duplicative,
nonintegrated, stovepiped systems that do not optimize mission performance
and accountability and, therefore, do not support the department's
transformation goals.
Conclusions
Having and effectively using a well-defined architecture is essential for
guiding and constraining DOD's business transformation efforts and moving
the department away from nonintegrated business systems development
efforts. DOD's efforts to date to address our prior recommendations aimed
at accomplishing this have not been sufficient. Specifically, despite 3
years of effort and over $203 million in reported obligations, DOD's
architecture remains insufficiently defined, and the way in which the
department makes business systems investments decisions remains largely
unchanged. As a result, billions of dollars continue to be at risk of
being spent on more systems that are duplicative, are not interoperable,
cost more to maintain than necessary, and do not optimize mission
performance and accountability.
The future of DOD's architecture development and implementation activities
is at risk; this in turn places the department's business transformation
effort in jeopardy of failing as other efforts by the department have in
the past. Therefore, it is imperative that DOD move swiftly to implement
our 22 open recommendations, which are aimed at strengthening architecture
management activities, adding missing content to architecture products,
and implementing investment oversight and control measures.
37U.S. General Accounting Office, Department of Defense: Further Actions
Needed to Establish and Implement a Framework for Successful Financial and
Business Management Transformation,
GAO-04-551T (Washington, D.C.: Mar. 23, 2004) and GAO-04-626T.
Agency Comments and Our Evaluation
In commenting on a draft of this report (reprinted in enc. IV), DOD agreed
with our assessment that its long-standing business systems problems have
resulted in a lack of adequate transparency and appropriate accountability
across all major business areas. It confirmed that development and use of
an architecture is necessary to transforming its business operations and
supporting systems. However, in response to our characterization that
progress has been limited, DOD stated that, over the past 3 years,
progress has been slower than either GAO or DOD would prefer but that it
has been significant, despite appearances to the contrary. The willingness
of DOD's leadership to tackle the department's decades-old problems with
its business operations and supporting systems represents a major step
forward. However, we continue to believe that our characterization of
DOD's progress in developing a welldefined architecture and implementing
effective management oversight and control over its business systems as
limited is fair. Therefore, we are not making any changes to our report.
DOD stated that we did not adequately consider or incorporate in our
report important completed and ongoing activities, plans, and
documentation, and provided an extensive package of information. We
disagree that we did not adequately consider or incorporate in our report
important completed and ongoing activities, plans, and documentation.
While our assessment focused primarily on the department's completed,
ongoing, and planned activities that it reported to Congress on March 15,
2004, we also considered and incorporated various activities, plans, and
documentation through April 27, 2004, which included the vast majority of
the information in the package.
In addition, we acknowledged in this report most, if not all, of the
actions DOD represents in their comments as having been completed such as
issuing the IT portfolio management policy and establishing the DO/IT. We
also recognized many of DOD's ongoing activities, such as the workshops
being held by the accounting and finance domain and the department's
establishment of the AIT, which are currently focused on developing needed
architectural content for increment one. To gain an understanding of the
specific AIT activities, which began in January 2004, we met with the
domains during our review and discussed various collaborative efforts,
such as their participation in validating business rules and requirements
to support the development of end-to-end business processes. Regarding
planned activities, we recognized DOD's intent to revise the executive and
steering committees' charters to include responsibilities for directing,
overseeing, and approving the architecture, and to develop the procedures
for implementing the IT portfolio management policy. In its response, DOD
stated that it plans to complete these actions in June and December 2004,
respectively.
Because we received information about certain activities after we
completed our fieldwork-or not at all-these activities are not included in
our evaluation. For example, DOD stated that the department plans to
release a management initiative during May 2004; establish an architecture
and program management maturity plan
and supporting metrics to address the quality, content, and utility of the
architecture; and establish a balanced scorecard and supporting metrics
focused on the organization's internal management processes. While we
inquired about the management initiative during our review, program
officials stated that they were unable to provide us a copy because DOD
does not allow the release of such draft documents outside the department.
In addition, program officials did not provide a characterization of the
focus or scope of the management initiative; therefore we were unable to
include it in our report. We made inquiries about other initiatives during
the course of our review, but were not advised of the maturity plan and
balanced scorecard. However, it is important to note that carrying out
activities does not necessarily guarantee results, and thus progress,
unless these activities are based on an approved program plan that can be
used to measure progress.
In addition, we disagree with DOD's other comments. Specifically, DOD
stated that the Comptroller is accountable for approval of updates to the
architecture as called for by best practices. As we previously reported38
and reiterated in this report, best practices recommend that the
accountability and responsibility for directing, overseeing, and approving
the architecture be assigned to a committee or group comprised of
representatives responsible for the core business areas across the
department-not a single individual. The purpose of assigning the
accountability and responsibility to such a committee is to obtain and
ensure continued enterprisewide support of the architecture effort,
thereby, increasing the department's chances for success.
In addition, DOD stated that its verification and validation function is
independent because the contractor responsible for these activities
currently reports to the Deputy Director, Program Support, who is not
directly involved with developing the architecture. DOD also stated that
to ensure this independence, reports prepared by this contractor are
provided directly to GAO and the steering committee. We disagree that this
function is sufficiently independent. According to best practices, the
contractor performing this role should report the results of its work
directly to the steering committee, as well as the program office. Since
the Deputy Director, Program Support reports directly to the program
management office and not the steering committee, the function is not
independent. In addition, the verification and validation contractor
agreed with us that its reports are not provided directly to GAO or the
steering committee. The contractor also concurred with our statement that,
in the one instance in which it briefed the steering committee, the
program office modified the content of the presentation.
DOD disagreed with our prior position that funds for DOD business systems
be appropriated directly to the domains. We are reaffirming that position
as stated by the Comptroller General in his March 2004 testimony39 before
the Subcommittee on Readiness and Management Support, Senate Armed
Services Committee. This matter
38 GAO-03-458.
39 GAO-04-551T.
for congressional consideration and related changes in responsibility will
be discussed in additional detail in a report to be issued shortly.
We are sending copies of this report to interested congressional
committees; the Director, Office of Management and Budget; the Secretary
of Defense; the Under Secretary of Defense (Comptroller); the Assistant
Secretary of Defense (Networks and Information Integration)/DOD Chief
Information Officer; the Under Secretary of Defense (Acquisition,
Technology, and Logistics); the Under Secretary of Defense (Personnel and
Readiness); and the Director, Defense Finance and Accounting Service. This
report will also be available at no charge on our Web site at
http://www.gao.gov.
If you have any questions concerning this information, please contact
Gregory Kutz at (202) 512-9095 or kutzg@gao.gov or Randolph Hite at (202)
512-3439 or hiter@gao.gov. GAO contacts and key contributors to this
report are listed in enclosure V.
Gregory D. Kutz
Director, Financial Management and Assurance
Randolph C. Hite
Director, Information Technology Architecture
and Systems Issues
Enclosures
List of Committees
The Honorable John W. Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate
The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate
The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives
The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives
Enclosure I
SEC. 1004. [of Public Law 107-314] DEVELOPMENT AND IMPLEMENTATION OF FINANCIAL
MANAGEMENT ENTERPRISE ARCHITECTURE
(a) REQUIREMENT FOR ENTERPRISE ARCHITECTURE AND FOR TRANSITION
PLAN-
Not later than May 1, 2003, the Secretary of Defense shall develop-
(1) a financial management enterprise architecture for all budgetary,
accounting,
finance, enterprise resource planning, and mixed information systems of
the
Department of Defense; and
(2) a transition plan for implementing that financial management
enterprise
architecture.
(b) COMPOSITION OF ENTERPRISE ARCHITECTURE-
(1) The financial management enterprise architecture developed under
subsection (a)(1) shall describe an information infrastructure that, at a
minimum,
would enable the Department of Defense to-
(A) comply with all Federal accounting, financial management, and
reporting
requirements;
(B) routinely produce timely, accurate, and reliable financial information
for
management purposes;
(C) integrate budget, accounting, and program information and systems; and
(D) provide for the systematic measurement of performance, including the
ability
to produce timely, relevant, and reliable cost information.
(2) That enterprise architecture shall also include policies, procedures,
data
standards, and system interface requirements that are to apply uniformly
throughout
the Department of Defense.
(c) COMPOSITION OF TRANSITION PLAN-The transition plan developed under
subsection (a)(2) shall include the following:
(1) The acquisition strategy for the enterprise architecture, including
specific time
phased milestones, performance metrics, and financial and nonfinancial
resource
needs.
(2) A listing of the mission critical or mission essential operational and
developmental financial and nonfinancial management systems of the
Department of
Defense, as defined by the Under Secretary of Defense (Comptroller),
consistent with
budget justification documentation, together with-
(A) the costs to operate and maintain each of those systems during fiscal
year
2002; and
(B) the estimated cost to operate and maintain each of those systems
during
fiscal year 2003.
(3) A listing of the operational and developmental financial management
systems of the Department of Defense as of the date of the enactment of
this Act (known as `legacy systems') that will not be part of the
objective financial and nonfinancial management system, together with the
schedule for terminating those legacy systems that provides for reducing
the use of those legacy systems in phases.
(d) CONDITIONS FOR OBLIGATION OF SIGNIFICANT AMOUNTS FOR FINANCIAL SYSTEM
IMPROVEMENTS-An amount in excess of $1,000,000 may be obligated for a
defense financial system improvement only if the Under Secretary of
Defense (Comptroller) makes a determination regarding that improvement as
follows: (1) Before the date of an approval specified in paragraph (2), a
determination that the defense financial system improvement is necessary
for either of the following reasons:
(A) To achieve a critical national security capability or address a
critical
requirement in an area such as safety or security.
(B) To prevent a significant adverse effect (in terms of a technical
matter, cost, or
schedule) on a project that is needed to achieve an essential capability,
taking into
consideration in the determination the alternative solutions for
preventing the
adverse effect.
(2) On and after the date of any approval by the Secretary of Defense of a
financial
management enterprise architecture and a transition plan that satisfy the
requirements of this section, a determination that the defense financial
system
improvement is consistent with both the enterprise architecture and the
transition
plan.
(e) CONGRESSIONAL REPORTS-Not later than March 15 of each year from 2004
through 2007, the Secretary of Defense shall submit to the congressional
defense
committees a report on the progress of the Department of Defense in
implementing
the enterprise architecture and transition plan required by this section.
Each report
shall include, at a minimum-
(1) a description of the actions taken during the preceding fiscal year to
implement
the enterprise architecture and transition plan (together with the
estimated costs of
such actions);
(2) an explanation of any action planned in the enterprise architecture
and transition
plan to be taken during the preceding fiscal year that was not taken
during that fiscal
year;
(3) a description of the actions taken and planned to be taken during the
current
fiscal year to implement the enterprise architecture and transition plan
(together with
the estimated costs of such actions); and
(4) a description of the actions taken and planned to be taken during the
next fiscal
year to implement the enterprise architecture and transition plan
(together with the
estimated costs of such actions).
(f) COMPTROLLER GENERAL REVIEW-Not later than 60 days after the approval
of
an enterprise architecture and transition plan in accordance with the
requirements of
subsection (a), and not later than 60 days after the submission of an
annual report
required by subsection (e), the Comptroller General shall submit to the
congressional
defense committees an assessment of the extent to which the actions taken
by the
Department comply with the requirements of this section.
(g) DEFINITIONS-In this section:
(1) The term `defense financial system improvement' means the acquisition
of a new
budgetary, accounting, finance, enterprise resource planning, or mixed
information
system for the Department of Defense or a modification of an existing
budgetary,
accounting, finance, enterprise resource planning, or mixed information
system of
the Department of Defense. Such term does not include routine maintenance
and
operation of any such system.
(2) The term `mixed information system' means an information system that
supports
financial and non-financial functions of the Federal Government as defined
in Office
of Management and Budget Circular A-127 (Financial Management Systems).
(h) REPEAL-(1) Section 2222 of title 10, United States Code, is repealed.
The table
of sections at the beginning of chapter 131 of such title is amended by
striking the
item relating to such section.
(2) Section 185(d) of such title is amended by striking `has the meaning
given that
term in section 2222(c)(2) of this title' and inserting `means an
automated or manual
system from which information is derived for a financial management system
or an
accounting system'.
Enclosure II
Scope and Methodology
Consistent with the act and as agreed with congressional defense
committees' staffs, our review focused on (1) the actions the Department
of Defense (DOD) has taken to address our previous recommendations
regarding the development and implementation of the architecture and (2)
the actions DOD is taking to ensure its ongoing and planned investments
will be consistent with its evolving architecture.
To determine what actions DOD has taken to address our previous
recommendations that relate to the development and implementation of the
architecture, we met with DOD and contractor officials to obtain an update
on the status of our prior recommendations. We used our Enterprise
Architecture Management Maturity Framework,40 which describes the stages
of management maturity, to update the status of key elements of
architecture management best practices that DOD had not adopted. To make
this determination, we reviewed program documentation, such as program
policies and procedures, communications plan, and charters for the
governance bodies; and we compared them to the elements in the framework.
We reviewed program documentation, such as requests documenting proposed
and actual changes to the architecture and external41 requirements
extracted from the updated architecture. We also reviewed contractor task
orders to determine the work performed and planned by the prime contractor
associated with extending and evolving the architecture.
To determine what actions DOD is taking to ensure its ongoing and planned
business systems investments are consistent with its evolving
architecture, we met with DOD officials to obtain an update on the status
of our prior recommendations. We met with appropriate officials from the
offices of the Under Secretary of Defense (Comptroller) and the Assistant
Secretary of Defense (Networks and Information Integration)/DOD Chief
Information Officer, and with representatives from the domains to discuss
the status of various draft policies and guidance that are aimed at
improving the department's control of and accountability for business
systems investments. We also reviewed and analyzed DOD's budget request
for fiscal year 2004 to identify the business systems investments that
could be subject to the requirements of the National Defense Authorization
Act for Fiscal Year 2003, which requires that all financial system
improvements with obligations in excess of $1 million be reviewed by DOD's
Comptroller, who must make a determination on whether the system
improvements are in accordance with criteria specified in the act. To
assess DOD's compliance with the act, we also obtained and reviewed
departmental guidance, memorandums, DOD Comptroller review decisions, and
40U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003).
41External requirements are those that are obtained from authoritative
sources, such as the Joint Financial Management Improvement Program, and
constrain various aspects of the architecture.
other documentation provided by the program office. Additionally, we
requested that DOD provide us with data on obligations in excess of $1
million for business systems modernization for fiscal years 2003 and 2004,
as of December 2003. We also received obligational data from some of the
defense agencies. We then compared the obligational data with the
information we had received from the program office to ascertain if the
systems modernization had been reviewed.
To augment our document reviews and analyses, we interviewed officials
from various DOD organizations and contractors, including the Office of
the Under Secretary of Defense (Comptroller); Office of the Under
Secretary of Defense (Acquisition, Technology, and Logistics); Office of
the Under Secretary of Defense (Personnel and Readiness); International
Business Machines; and MITRE Corporation.
We did not validate the accuracy and reliability of the funding
information contained in DOD's March 15, 2004, report. Also, the
unliquidated and unobligated calculations that we made were based on DOD
reported data. Further, we did not validate the accuracy and reliability
of the obligational data provided by the military services and the defense
agencies for systems modernization.
We requested comments on a draft of this report from the Secretary of
Defense or his designee. We received written comments from the Acting
Under Secretary of Defense (Comptroller), which we printed in enclosure
IV. We considered, but did not reprint the voluminous attachments that DOD
provided with its written comments.
We conducted our work primarily at DOD headquarters offices in Washington,
D.C., and Arlington, Virginia, from December 2003 through April 2004, in
accordance with U.S. generally accepted government auditing standards.
Enclosure III
Status of Prior Recommendations on DOD's Business Enterprise Architecture
(BEA)
Page 29
GAO-04-731R DOD Business Systems Modernization
The
DOD recently department
issued its had
Information previously
Technology (IT) established
Portfolio the
Management executive
(1) The policy. This and
Secretary of policy, in steering
Defense conjunction with committees,
immediately the overarching which we
designate DOD (2) The Global reported
financial Secretary Information were
management immediately Grida policy, advisory in
modernization issue a DOD assigns nature. The
a policy that responsibilities (3) The department
departmental directs the for the Secretary recently
Status of Implemented priority and X development, X development, immediately X established
recommendation accordingly implementation, implementation, modify the the Domain
direct the and maintenance and maintenance Senior Owners
Deputy of a financial of the BEA.b Financial Integration
Secretary of management However, it does Management Teamc and
Defense to enterprise not provide for Oversight stated that
lead an architecture. having Council's these three
integrated accountability charter to o bodies are
GAO-01-525: program for and approval designate responsible
Information across the of updates to the Deputy for
Technology: department the BEA, Secretary of governing
Architecture for processes for Defense as the
Needed to modernizing BEA oversight the Council program.
Guide and and control, and Chair and the However,
Modernization optimizing review and Under these three
DOD's of DOD's financial validation of Secretary of groups have
comments Financial management the BEA. Defense not been
and our Partially Not Operations. operations (Comptroller) assigned
Recommendation assessment implemented implemented May 17, 2001. and systems. as the
Page 30
GAO-04-731R DOD Business Systems Modernization
the Council
vice-Chair; o accountability
empower the and
Council to responsibility
serve as DOD's for directing,
financial overseeing, and
management approving the
enterprise BEA. According to
architecture DOD, it expects
steering to revise and
committee, finalize the
giving it the executive and
responsibility steering
and authority committees'
to ensure that charters to
a DOD include these
financial responsibilities
management in June 2004. DOD
enterprise issued an IT
architecture portfolio
is developed management policy
and maintained (March 2004) that
in accordance assigns the
with the DOD domains d
C4ISR responsibility
Architecture for IT portfolio
Framework; o management.
empower the However, the
Council to procedures to be
serve as DOD's followed to
financial implement the
management policy are still
investment under development
Status of review board, and no time
recommendation Implemented giving it the frames for
responsibility completion have
and authority been provided.
to (1) select According to the
and control IT portfolio
all DOD management
financial policy, the
management department has
investments 180 days (until
and (2) ensure mid-September
that its 2004) to develop
investment these procedures.
decisions In addition, the
treat department has
compliance yet to formalize
with the specific roles
financial and
management responsibilities
enterprise of the domains,
architecture develop standard
as an explicit criteria for
condition for performing the
investment system reviews,
approval that and assign
can be waived explicit
only if authority for
justified by a fulfilling roles
compelling and
written responsibilities.
analysis; and DOD has yet to
o expand the establish time
DOD's role of the frames for
comments Council's completing these
and our Partially Not System activities.
Recommendation assessment implemented implemented Compliance
Page 31
GAO-04-731R DOD Business Systems Modernization
(4) The
Secretary
immediately
make the
Assistant
Secretary of
Defense
(Command,
Control, The Assistant
Communications Secretary of
& Defense
Intelligence), (Networks and
in Information
collaboration Integration)/DOD
with the Under Chief
Secretary of Information
Defense Officer is a
(Comptroller), member of the
accountable to executive and
the Senior steering
Financial committees;
Management however, as
Oversight discussed
Council for previously,
developing and members' roles
maintaining a and
DOD financial responsibilities
management are advisory in
enterprise nature. DOD
Status of architecture. established a
recommendation Implemented In fulfilling X Financial
this Management
responsibility, Modernization
the Assistant Program Office
Secretary in July 2001.
appoint a Chief DOD has also
Architect for appointed a
DOD financial chief architect
Working management and, according
Group to modernization to the
include and establish department, it
supporting and adequately has adequate
the Council staff and fund program funding
in an enterprise and staff for
determining architecture developing and
the program office maintaining its
compliance that is BEA. However,
of each responsible for DOD has not yet
system developing and defined the
investment maintaining a roles and
with the DODwide responsibilities
enterprise financial for the Chief
architecture management Architect.
at key enterprise
decision architecture in
points in a manner that
the system's is consistent
DOD's development with the
comments or framework
and our Partially Not acquisition defined in the
Recommendation assessment implemented implemented life cycle. CIO Council's
Page 32
GAO-04-731R DOD Business Systems Modernization
published
guide for
managing
enterprise
architectures.
In particular,
the Assistant
Secretary
should take
appropriate The Assistant
steps to Secretary of
ensure that (5) The Defense
the Chief Assistant (Networks and
Architect o Secretary of Information
obtains Defense Integration)/DOD
executive (Command, Chief
buy-in and Control, Information
support; o Communications Officer is a
establishes & member of the
architecture Intelligence) steering
management reports at committee, which
structure and least is briefed
Status of controls; o quarterly to monthly by the
recommendation Implemented defines the the Senior X program office X
architecture Financial on various
process and Management program
approach; o Oversight activities; but
develops the Council on the not necessarily
baseline Chief on the
architecture, Architect's development and
the target progress in maintenance of
architecture, developing a the BEA's
and the financial content. The
sequencing management Chief Architect
plan; o enterprise is a member of
facilitates architecture, the program
the use of the including the office.
architecture Chief
to guide Architect's
financial adherence to
management enterprise
modernization architecture
projects and policy and
DOD's investments; guidance from The
comments and o OMB, the CIO (6) The Deputy
and our Partially Not maintains the Council, and Senior Chief
Recommendation assessment implemented implemented architecture. DOD. Financial Financial
Page 33
GAO-04-731R DOD Business Systems Modernization
DOD has not yet
defined and
implemented an
approach for
selecting and
controlling
business systems
investments. In
March 2004, the
Deputy Secretary
of Defense signed
the IT portfolio
management policy
and assigned
responsibility to
(8) Until a the domains for
financial IT portfolio
management management.
enterprise However, the
architecture is procedures to be
(7) The developed and followed to
Secretary the Council is implement the
reports every positioned to policy are still
Officer 6 months to Senate serve as DOD's under development
briefed the the Report financial and no time
Secretary of congressional 107-213 management frames for
Defense in defense directs investment completion have
November authorizing that the review board as been provided.
Status of 2003 on and department recommended According to the
recommendation Implemented behalf of appropriating X report above, the X IT portfolio
DOD's committees on every 6 Secretary of management
Comptroller, progress in months on Defense limit policy, the
who chairs developing the status DOD components' department has
the and of the BEA financial 180 days (until
executive implementing effort. DOD management mid-September
committee. a financial submitted investments to 2004) to develop
management status o deployment of these procedures.
enterprise reports on systems that In addition, the
architecture. January 31 have already department has
and July been fully yet to formalize
31, 2003 tested and specific roles
and on involve no and
January 31, additional responsibilities
2004. development or of the domains,
Management However, acquisition develop standard
Oversight these cost; o criteria for
Council reports stay-in-business performing the
report to the were maintenance system reviews,
Secretary of submitted needed to keep and assign
Defense every by DOD's existing systems explicit
6 months on Comptroller operational; o authority for
progress in and were management fulfilling roles
developing not signed controls needed and
and by the to effectively responsibilities.
implementing members of invest in DOD has yet to
DOD's a financial the modernized establish time
comments management executive systems; and o frames for
and our Partially Not enterprise or steering new systems or completing these
Recommendation assessment implemented implemented architecture. committees. existing system activities.
Page 34
GAO-04-731R DOD Business Systems Modernization
The department
had previously
established the
executive and
steering
committees,
which we
reported were
(1) The advisory in
Secretary of nature. The According to
Defense department DOD, the
ensure that recently quality
the established the assurance
enterprise Domain Owners function
architecture Integration Team DOD has a does not yet
changes that executive and stated that strategic address
are committee these three communications process
congressionally members are bodies are plan; however, standards
directed or are singularly responsible for the plan has and program
relatively and governing the not yet been performance,
Status of small, cost collectively program. executed. nor is it
recommendation Implemented effective, and made X However, these X According to (3) The X yet an
low risk and explicitly groups have not DOD, the plan Secretary of independent
can be accountable been assigned is scheduled Defense function.
delivered in a to the the to be executed ensure that Further, DOD
relatively Secretary for accountability between June the quality subject
short time the delivery and and December assurance matter
frame. of the responsibility 2004. function o experts
enterprise for directing, includes the continue to
GAO-03-458: architecture, overseeing, and review of be involved
DOD Business including approving the (2) The adherence to in the
Systems approval of BEA. According Secretary of process quality
Modernization: each version to DOD, it Defense standards assurance
Improvements of the expects to ensure that and function.
to Enterprise architecture. revise and the reliability
Architecture finalize the enterprise of reported
Development executive and architecture program
and steering program is performance,
Implementation committees' supported by o is made
DOD's Efforts charters to a proactive independent
comments Needed. include these marketing and of the
and our Partially Not February 28, responsibilities communication program
Recommendation assessment implemented implemented 2003. in June 2004. program. management
Page 35
GAO-04-731R DOD Business Systems Modernization
DOD has not yet
defined and
implemented an
approach for
selecting and
controlling
business systems
investments. In
March 2004, the
Deputy Secretary
of Defense signed
the IT portfolio
management policy
and assigned
responsibility to
the domains for
(4) The IT portfolio
Secretary gain management.
control over However, the
ongoing IT procedures to be
investments by followed to
establishing a implement the DOD has not
hierarchy of policy are still developed
investment under development standard
review boards, and no time criteria
each responsible frames for for
and accountable completion have evaluating
for selecting been provided. business
Status of and controlling According to the systems
recommendation Implemented investments that X IT portfolio X investments
meet defined management and has not
threshold policy, the established
criteria, and department has time frames
each composed of 180 days (until for
the appropriate mid-September (5) The completing
level of 2004) to develop Secretary gain this
executive these procedures. control over activity.
representatives, In addition, the ongoing IT
depending on the department has investments by
threshold yet to formalize establishing a
criteria, from specific roles standard set of
across the and criteria to
department. responsibilities include (a)
of the domains, alignment and
develop standard consistency
criteria for with the DOD
function, performing the enterprise
and o is system reviews, architecture
not and assign and (b) our
performed by explicit open
subject authority for recommendations
matter fulfilling roles governing
experts and limitations in
involved in responsibilities. business
the DOD has yet to systems
DOD's development establish time investments
comments of key frames for pending
and our Partially Not architecture completing these development of
Recommendation assessment implemented implemented products. activities. the
Page 36
GAO-04-731R DOD Business Systems Modernization
In March 2004,
the Deputy
Secretary of
Defense signed
the IT portfolio
management policy
and assigned
responsibility to
the domains for
IT portfolio
management.
However, the
procedures to be
followed to
implement the
policy are still
(6) The under development
Secretary and no time
gain control frames for
over ongoing completion have As discussed
IT been provided. in this
investments According to the report, DOD
by directing IT portfolio has not yet
these boards management defined and
to policy, the implemented
immediately department has an effective
apply these 180 days (until investment
criteria in mid-September management
completing 2004) to develop process to
reviews of these procedures. proactively
Status of all ongoing In addition, the (1) The identify and
recommendation Implemented architecture. IT X department has Secretary of X control
investments, yet to formalize Defense or system
and to not specific roles his improvements
fund and appropriate exceeding $1
investments responsibilities designee million in
that do not of the domains, define and obligations.
meet these develop standard implement an DOD
criteria criteria for effective officials
unless they performing the investment have
are system reviews, management acknowledged
otherwise and assign process to that the
justified by explicit proactively department
explicit authority for identify, does not
criteria fulfilling roles control, and have a
waivers. and obtain DOD systematic
responsibilities. Comptroller means to
DOD has yet to review and identify and
establish time approval of determine
frames for GAO-03-1018: expenditures which system
completing these DOD Business for new and improvements
activities. Systems ongoing should be
Further, the Modernization: business submitted to
policy also does Important systems the
not address the Progress Made investments Comptroller
issuance of to Develop exceeding $1 for review
waivers in those Business million and, are
instances when Enterprise while the dependent
exceptions to the Architecture, architecture upon system
DOD's BEA are justified but Much Work is being owners
comments on the basis of Remains. developed coming
and our Partially Not documented September 19, and after it forward to
Recommendation assessment implemented implemented analysis. 2003. is the
Page 37
GAO-04-731R DOD Business Systems Modernization
DOD has taken some
actions, but these
actions are not
sufficient to fully
address our
previous concerns.
For example, DOD
has o established
the Domain Owners
Integration Team;
o begun to
establish a
configuration
(2) The management process;
Secretary of o recently issued
Defense or an IT portfolio
his management policy
appropriate governing the
designee development,
implement maintenance, and
the core implementation of
elements in the BEA; and o
our developed
Enterprise high-level
Architecture performance
Framework measures that are
for to be used to
Assessing develop the more
and specific
completed, and Improving resultsoriented
which includes Enterprise performance metrics
clearly defined Architecture that are needed to
domain owners' domains Management enable it to
Status of roles and and that we evaluate program
recommendation Implemented responsibilities requesting identify in X progress and
for selecting approval. this report benefits. However,
and controlling as not as discussed
ongoing and satisfied, previously,
planned system including accountability and
investments. ensuring responsibility have
that minutes not been assigned
of the to a committee or
meetings of group comprised of
the representatives
executive from across the DOD
body charged component
with organizations-which
directing, would be
overseeing, responsible for
and directing,
approving overseeing, and
the approving the BEA.
architecture DOD plans to update
are prepared the executive and
and steering
maintained. committees'
charters to include
these
responsibilities in
June 2004. Also,
the configuration
control board has
been tasked with
reviewing changes
to only some, not
DOD's all BEA products
comments (e.g., the board is
and our Partially Not not responsible for
Recommendation assessment implemented implemented tracking
Page 38
GAO-04-731R DOD Business Systems Modernization
changes that
are being made
to the
transition
plan as part
of the BEA
effort). In
addition, both
the charter
and the
procedures
governing the
configuration
management
process are
still in
draft.
According to
DOD, the
charter is to
be approved in
June 2004, but (3) The
no time frame Secretary of (4) The
has been Defense or Secretary of
provided for his Defense or
final approval appropriate his
of the designee appropriate
procedures. update designee
The portfolio version 1.0 update
management of the version 1.0
policy does architecture of the
not address to include architecture
Status of accountability the 340 to include
recommendation Implemented for and Joint X the 29 key X Of the 29
approval of Financial elements elements,
updates to the Management governing the DOD stated
BEA-as called Improvement "As Is" that it
for by best Program architectural plans to
practices. In requirements content that address 26,
addition, the that our our report but it did
policy does report identified as not have
not address identified not being detailed
the issuance as omitted fully plans on how
of waivers in or not fully satisfied. and when
those addressed. they would
instances when be fully
exceptions to addressed.
the BEA are According to
justified on DOD, it is
the basis of currently
documented addressing 2
analysis. The of the
department's 26-system
current and process
verification inventories-
and validation which will
contractor is be part of
not the
independent transition
and is plan. With
responsible respect to
for reviewing the system
the BEA inventory,
DOD's products and DOD's
comments only selected Comptroller
and our Partially Not program recently
Recommendation assessment implemented implemented deliverables. testified
Page 39
GAO-04-731R DOD Business Systems Modernization
According
to DOD, it
has
that the partially
actual number addressed
of DOD 21
systems could elements,
be twice as but it
many as is (5) The plans to
currently Secretary of address all
reported; Defense or 30.
2,274. For his However,
the 3 appropriate DOD did not
elements that designee provide any According to
DOD did not update detailed DOD, the next
consider to version 1.0 plans DOD did not transition plan
be relevant of the showing how (6) The have will be
to the business and when it Secretary of information (7) The available in
program, it enterprise would fully Defense or his readily Secretary of August 2004.
Status of did not architecture address appropriate available to Defense or However, DOD did
recommendation Implemented provide any to include X these designee update X show that it his X not have
explicit the 30 key elements. version 1.0 to had addressed appropriate information
analysis as elements In ensure that "To inconsistencies designee readily
to why the 3 governing the addition, Be" identified in update available to
would be "To Be" the architecture our report. version 1.0 show how or when
irrelevant. architectural information artifacts are of the it would fully
Until DOD content that provided internally architecture address our
provides such our report for the 21 consistent, to to include recommendations.
an analysis, identified as elements include (a) the 3
we consider not being shows that addressing the key elements
all 29 fully it had not inconsistencies governing
elements to satisfied. addressed described in the
be critical them but this report, as transition
for the "As was well as plan content
Is" planning to including user that our
architectural address instructions or report
content. them or had guidance for identified
DOD's just begun easier as not being
comments to take architecture fully
and our Partially Not needed navigation and satisfied
Recommendation assessment implemented implemented actions. use. and (b)
Page 40
GAO-04-731R DOD Business Systems Modernization
According to
DOD, it has
addressed
some of its
verification
and
validation
contractor's
comments and
plans to
address the
majority of As The
the others. (9) The discussed accounting
(8) The However, DOD Secretary of in this and finance
Secretary of did not have Defense or his report, DOD (10) The domain has
Defense or documentation appropriate has not Secretary two ongoing
his readily designee develop developed of Defense pilot
appropriate available to a well-defined welldefined or his initiatives
designee show that it near-term plan near-term appropriate to acquire
update had addressed for extending or designee and
Status of version 1.0 some of the and evolving the long-term limit the implement
recommendation Implemented of the X contractor's architecture and X plans to be pilot X production
architecture comments, and ensure that this used to projects to accounting
to address was unable to plan includes guide small, systems by
comments provide plans addressing our dayto-day low-cost, the end of
made by the detailing how recommendations, program lowrisk 2005-at an
verification and when it defining roles activities prototype estimated
and would fully and to enable investments cost of
validation address the responsibilities it to that are $135
contractor. remaining of all evaluate million.
comments. stakeholders its This domain
those system According to involved in progress. is
investments DOD, the extending and
that will not verification evolving the
become part and architecture,
of the "To validation explaining
Be" contractor's dependencies
architecture, comments among planned
including would be activities, and
DOD's time frames addressed in defining
comments for phasing BEA updates measures of
and our Partially Not out those after version activity
Recommendation assessment implemented implemented systems. 2.0. progress.
Status of recommendation
Implemented Partially Not DOD's comments and
Recommendation implemented implemented our assessment
also currently
conducting workshops
to develop needed
business rules and
requirements for
extending and
intended to evolving the BEA in
provide knowledge support of DOD's
needed to extend achieving objective
and evolve the of increment one,
architecture, and which includes
are not to achieving an
acquire and unqualified audit
implement opinion on DOD's
production fiscal year 2007
version system consolidated
solutions or to financial
deploy an statements. However,
operational the department did
system not provide detailed
capability. plans that showed
how or if the
workshop results
would serve as the
basis for acquiring
these production
systems.
Source: GAO analysis of DOD data.
a DOD defines the Global Information Grid as the globally interconnected,
end-to-end set of information, capabilities, associated processes, and
personnel for collecting, processing, storing, disseminating, and managing
information on demand to warfighters, policymakers, and support personnel.
b In May 2003, the DOD Comptroller changed the architecture name from the
Financial Management Enterprise Architecture to the Business Enterprise
Architecture to reflect the transformation of departmentwide business
operations and supporting systems, including accounting and finance,
budget formulation, acquisition, inventory management, logistics,
personnel, and property management systems.
c In September 2003, DOD established the Domain Owners Integration Team,
comprised of domain representatives, to facilitate communication and
coordination across the domains.
d DOD has six departmental domains, which are (1) accounting and finance,
(2) acquisition, (3) human resources management, (4) installations and
environment, (5) logistics, and (6) strategic planning and budgeting. It
also has one enterprise information environment mission area. The domains
and the mission area, comprised of the Under Secretaries of Defense and
the Assistant Secretary of Defense for Networks and Information
Integration/DOD Chief Information Officer, have authority, responsibility,
and accountability within their domains for business transformation,
implementation of the BEA, development and execution of the transition
plan, portfolio management, and establishment of a structure to ensure
representation of the DOD components and the appropriate federal agencies.
Enclosure IV
Comments from the Department of Defense
See comment 2.
See comment 1.
See comment 1. See comment 1.
See comment 1. See comment 3. See comment 1.
See comment 4.
See comment 5. See comment 3.
See comment 1.
See comment 6.
See comment 7. See comment 8. See comment 1.
See comment 9.
See comment 1.
See comment 1.
See comment 1. See comment 10.
See comment 11.
See comment 12. See comment 13.
See comment 13. See comment 14.
The following are GAO's comments on the Department of Defense's (DOD)
letter dated May 10, 2004.
GAO Comments
1. See the "Agency Comments and Our Evaluation" section of this report.
2. We have reviewed multiple versions of the architecture development
methodology, including the latest version, which is currently draft. Based
on our review of the latest draft, we concluded that this methodology does
not provide a documented approach for performing activities in a coherent,
consistent, accountable, and repeatable manner. Key DOD stakeholders also
expressed concerns about the methodology. For example, these stakeholders
stated that it is an "after-the-fact" description of work products rather
than a prescriptive document that details the approach to be followed to
develop the architecture.
3. The descriptions of increments 2 and 3 in DOD's comment letter are
different than the descriptions the program manager provided to us on
April 27, 2004, as reflected in this report. Because of the
inconsistencies in the responses, it remains unclear what the department's
focus is for increments 2 and 3. DOD's response also stated that the
executive committee approved all three increments; however, DOD did not
provide evidence of this approval.
4. As stated in our report, DOD's objective for achieving an unqualified
audit opinion is not supported by a DOD-wide plan of action nor are the
individual component plans linked to program activities.
5. As stated in our report, DOD does not expect to complete the next
version of its transition plan until August 2004. In addition, as we
previously reported,42 its initial transition plan was basically a plan to
develop a transition plan and did not possess the attributes needed to
guide its transformation efforts.
6. Attachment 11 does not list the deliverables the verification and
validation contractor has reviewed since April 2003.
7. The matrix does not show the anticipated disposition (e.g., date and
architecture version) of the verification and validation contractor's
comments.
8. This information was provided after fieldwork was completed.
9. These briefings were presentations given by each domain describing its
approach for conducting portfolio management reviews. The steering
42 GAO-03-1018.
committee did not review actual systems investments nor was there detailed
discussion about the domains' approaches.
10. The scope of our review did not include an evaluation of the
realignment of funds made by the logistics domain to determine an
effective demonstration of portfolio management.
11. DOD's characterization of its current process for oversight and
control of modernization investments is inaccurate. We along with the DOD
Inspector General continue to report on significant weaknesses in the
department's requirements, acquisition, and budgeting processes for
business systems modernization projects. For example, we reported43 that
estimated costs for the Defense Procurement Payment System had increased
by $274 million and the schedule had slipped by almost 4 years after
having already invested 7 years and over $126 million. DOD Comptroller's
noted that the project was being terminated due to poor program
performance and increased costs.
12. We look forward to receiving DOD's forthcoming guidance to contracting
and program officials that describes the process by which these officials
obtain DOD Comptroller's statutorily required determination before making
obligations exceeding $1 million for a financial system improvement.
However, given the obligation data provided to us by DOD and included in
our report, we are concerned that DOD plans no action to review previous
obligations for system improvements to ensure that there have been no such
violations since the enactment of the limitation.
13. We continue to believe that contractor tasks have not been clearly
defined and that, because DOD is acquiring services on the basis of time
and materials, it is essential that the department use efficient methods
and effective cost controls to measure the work being performed. While we
did not take issue with DOD on its use of a time and materials contract,
we did state that DOD's failure to execute the required Determination and
Finding to evaluate the cost and performance risks assumed by the
government illustrates an overall lack of clear architecture development
plans.
14. According to a program official, DOD did not begin requiring
acceptance criteria prior to the contractor beginning work until August
2003. Further, DOD has paid the contractor under the time and materials
contract even when the contractor did not meet all of the criteria for the
deliverable. For example, based on reports provided by DOD, the department
paid the contractor for delivery of version 2.0 of the architecture even
though this version did not meet 50 percent of the acceptance criteria.
43 GAO-03-465.
Enclosure V
GAO Contacts and Staff Acknowledgments
GAO Contacts Jenniffer Wilson, (202) 512-9192 Cynthia Jackson, (202)
512-5086
Acknowledgments In addition to the individuals named above, key
contributors to this report included Beatrice Alff, Francine DelVecchio,
Abe Dymond, Joanne Fiorino, Neelaxi Lakhmani, J. Christopher Martin, Mai
Nguyen, Michael Peacock, David Plocher, Katherine Schirano, Darby Smith,
and Bernard Trescavage.
(192114)
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: fraudnet@gao.gov
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
*** End of document. ***