DOD Business Systems Modernization: Progress Continues to Be Made
in Establishing Corporate Management Controls, but Further Steps
Are Needed (14-MAY-07, GAO-07-733).
In 1995, GAO first designated the Department of Defense's (DOD)
business systems modernization program as "high risk," and GAO
continues to do so today. To assist in addressing this high-risk
area, the Fiscal Year 2005 National Defense Authorization Act
contains provisions that are consistent with prior GAO
recommendations. Further, the act requires the department to
submit annual reports to its congressional committees on its
compliance with these provisions and it directs GAO to review
each report. In response, GAO assessed DOD's actions to address
(1) requirements in the act and (2) GAO's recommendations that it
reported as open in its prior annual report under the act. In
doing so, GAO reviewed documentation and interviewed officials
relative to the act and related guidance.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-733
ACCNO: A69585
TITLE: DOD Business Systems Modernization: Progress Continues to
Be Made in Establishing Corporate Management Controls, but
Further Steps Are Needed
DATE: 05/14/2007
SUBJECT: Accountability
Enterprise architecture
Information technology
IT investment management
Policy evaluation
Reporting requirements
Strategic information systems planning
Strategic planning
Systems conversions
GAO High Risk Series
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-733
* [1]Results in Brief
* [2]Background
* [3]Enterprise Architecture and IT Investment Management Control
* [4]Enterprise Architecture: A Brief Description
* [5]IT Investment Management: A Brief Description
* [6]DOD's Institutional Approach to Business Systems Modernizati
* [7]Tiered Accountability
* [8]Summary of Fiscal Year 2005 National Defense Authorization A
* [9]Summary of Recent GAO Reviews of DOD's Business Systems Mode
* [10]DOD Is Continuing to Improve Its Approach to Modernizing Bus
* [11]DOD Continues to Improve Its Corporate BEA, but Component Ar
* [12]DOD Continues to Expand and Update Its Enterprise Transition
* [13]DOD's Fiscal Year 2008 Budget Submission Includes Key Inform
* [14]DOD Has Largely Established Key Investment Management Struct
* [15]Investment Management Structures Have Been Largely Establish
* [16]Policies and Procedures Have Been Defined for Some, but Not
* [17]DOD Continues to Approve and Review Business Systems, but Mi
* [18]Air Force
* [19]Army
* [20]Navy
* [21]DOD Continues to Implement Our Prior Recommendations
* [22]Conclusions
* [23]Recommendation for Executive Action
* [24]Agency Comments
* [25]Appendix I: Objectives, Scope, and Methodology
* [26]Appendix II: Status of Prior Recommendations Identified as O
* [27]Appendix III: Other Open Recommendations on Business Archite
* [28]Appendix IV: Comments from the Department of Defense
* [29]Appendix V: GAO Contacts and Staff Acknowledgments
* [30]GAO Contacts
* [31]Staff Acknowledgments
* [32]Order by Mail or Phone
Report to Congressional Committees
United States Government Accountability Office
GAO
May 2007
DOD BUSINESS SYSTEMS MODERNIZATION
Progress Continues to Be Made in Establishing Corporate Management
Controls, but Further Steps Are Needed
[33]GAO-07-733
Contents
Letter 1
Results in Brief 3
Background 6
DOD Is Continuing to Improve Its Approach to Modernizing Business Systems
20
DOD Continues to Implement Our Prior Recommendations 38
Conclusions 39
Recommendation for Executive Action 40
Agency Comments 41
Appendix I Objectives, Scope, and Methodology 44
Appendix II Status of Prior Recommendations Identified as Open in GAO's
Prior Annual Report under the Act 47
Appendix III Other Open Recommendations on Business Architectures,
Federation Strategy, and Investment Management 52
Appendix IV Comments from the Department of Defense 54
Appendix V GAO Contacts and Staff Acknowledgments 56
Table
Table 1: DOD Business Systems Modernization Governance Entities' Roles,
Responsibilities, and Composition 15
Figures
Figure 1: Simplified DOD Organizational Structure 7
Figure 2: The Five ITIM Stages of Maturity with Critical Processes 13
Figure 3: Simplified Diagram of DOD's Business Mission Area Federated
Architecture 25
Abbreviations
ASD(NII)/CIO Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer
BEA business enterprise architecture
BEP business enterprise priority
BTA Business Transformation Agency
CIO chief information officer
DBSMC Defense Business Systems Management Committee
DOD Department of Defense
ETP enterprise transition plan
IRB Investment Review Board
IT information technology
ITIM Information Technology Investment Management framework
NCES Net-Centric Enterprise Services
OMB Office of Management and Budget
SOA service-oriented architecture
USD(AT&L) Under Secretary of Defense (Acquisition, Technology, and Logistics)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
May 14, 2007
Congressional Committees
For decades, the Department of Defense (DOD) has been challenged in
modernizing its timeworn business systems.^1 In 1995, we designated DOD's
business systems modernization program as high risk, and we continue to
designate it as such today.^2 As our research on public and private sector
organizations shows, two essential ingredients to a successful systems
modernization program are having a well-defined enterprise
architecture^3123 and an effective institutional approach to managing
information technology (IT) investments. For decades, the Department of
Defense (DOD) has been challenged in modernizing its timeworn business
systems. In 1995, we designated DOD's business systems modernization
program as high risk, and we continue to designate it as such today. As
our research on public and private sector organizations shows, two
essential ingredients to a successful systems modernization program are
having a well-defined enterprise architecture and an effective
institutional approach to managing information technology (IT)
investments.
Accordingly, we made recommendations to the Secretary of Defense in May
2001 that included the means for effectively developing an enterprise
architecture and establishing a corporate approach to investment control
and decision making.^4 Between 2001 and 2005, we reported that the
department's business systems modernization program continued to lack both
of these, concluding in 2005 that hundreds of millions of dollars had been
spent on a business enterprise architecture (BEA) and investment
Accordingly, we made recommendations to the Secretary of Defense in May
2001 that included the means for effectively developing an enterprise
architecture and establishing a corporate approach to investment control
and decision making.^45 Between 2001 and 2005, we reported that the
department's business systems modernization program continued to lack both
of these, concluding in 2005 that hundreds of millions of dollars had been
spent on a business enterprise architecture (BEA) and investment
management structures that had limited use. Accordingly, we made more
explicit architecture and investment-related recommendations.
^1Business systems support DOD's business operations, such as civilian
personnel, finance, health, logistics, military personnel, procurement,
and transportation.
^2GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: January
2007).
^3An enterprise architecture, or modernization blueprint, provides a clear
and comprehensive picture of an entity, whether it is an organization
(e.g., federal department or agency) or a functional or mission area that
cuts across more than one organization (e.g., financial management). This
picture consists of snapshots of the enterprise's current "As Is"
operational and technological environment and its target or "To Be"
environment, and contains a capital investment road map for transitioning
from the current to the target environment. These snapshots consist of
"views," which are basically one or more architecture products that
provide conceptual or logical representations of the enterprise.
^4GAO, Information Technology: Architecture Needed to Guide Modernization
of DOD's Financial Operations, GAO-01-525 (Washington, D.C.: May 17,
2001).
To assist DOD in addressing these modernization management challenges,
Congress included provisions in the Ronald W. Reagan National Defense
Authorization Act for Fiscal Year 2005^6 that were consistent with our
recommendations. More specifically, the act required the department to,
among other things, (1) develop a BEA, (2) develop a transition plan to
implement the architecture, (3) include systems information in its annual
budget submission, (4) establish a system investment approval and
accountability structure, (5) establish an investment review process, and
(6) approve and certify any system modernizations costing in excess of $1
million. The act further requires that the Secretary of Defense submit an
annual report to congressional defense committees on DOD's compliance with
certain requirements of the act not later than March 15 of each year from
2005 through 2009. Additionally, the act directs us to submit--within 60
days of DOD's report submission--to congressional defense committees an
assessment of the actions taken to comply with these requirements.
As agreed with your offices, the objectives of our review were to (1)
assess the actions taken by DOD to comply with requirements of section
2222 of Title 10, U.S. Code, and (2) determine the extent DOD has
addressed our prior open recommendations for institutionalizing key
business system modernization management controls. To accomplish this, we
used our prior annual report under the act^7 as a baseline, analyzing
whether the department had taken actions to comply with those provisions
of the act, related guidance, and the prior recommendations that we had
identified in our prior annual report as not yet addressed. In doing this,
we also relied on the results of relevant reports that we have issued
since our prior annual report.^8 We performed our work at DOD headquarters
in Arlington, Virginia, from March through May 2007 in accordance with
generally accepted government auditing standards. Details on our
objectives, scope, and methodology are contained in appendix I.
^5See, for example, GAO, Defense Business Transformation: A Comprehensive
Plan, Integrated Efforts, and Sustained Leadership Are Needed to Assure
Success, GAO-07-229T (Washington, D.C.: Nov. 16, 2006); Business Systems
Modernization: DOD Continues to Improve Institutional Approach, but
Further Steps Needed, GAO-06-658 (Washington, D.C.: May 15, 2006); DOD
Business Systems Modernization: Long-standing Weaknesses in Enterprise
Architecture Development Need to Be Addressed, GAO-05-702 (Washington,
D.C.: July 22, 2005); DOD Business Systems Modernization: Billions Being
Invested without Adequate Oversight, GAO-05-381 (Washington, D.C.: Apr.
29, 2005); DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments, GAO-04-731R (Washington, D.C.: May
17, 2004); DOD Business Systems Modernization: Important Progress Made to
Develop Business Enterprise Architecture, but Much Work Remains,
GAO-03-1018 (Washington, D.C.: Sept. 19, 2003); Business Systems
Modernization: Summary of GAO's Assessment of the Department of Defense's
Initial Business Enterprise Architecture, GAO-03-877R (Washington, D.C.:
July 7, 2003); Information Technology: Observations on Department of
Defense's Draft Enterprise Architecture, GAO-03-571R (Washington, D.C.:
Mar. 28, 2003); DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed,
GAO-03-458 (Washington, D.C.: Feb. 28, 2003); and GAO-01-525.
^6Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. S 2222).
Results in Brief
DOD continues to take steps to comply with legislative requirements and
related guidance pertaining to its business systems modernization high
risk area. In particular, on March 15, 2007, DOD released a new version of
its BEA, developed an updated enterprise transition plan, and issued its
annual report to Congress describing steps taken and planned relative to
the act's requirements, among other things. The steps address several of
the missing elements that we previously identified relative to the
legislative provisions and related best practices concerning the BEA,
enterprise transition plan, budgetary disclosure, investment management,
and reviews of systems costing in excess of $1 million. However,
additional steps are needed to fully comply with the act and relevant
guidance. For example:
o The latest version of the BEA now contains information about the
department's "As Is" corporate environment for some enterprise
priority areas (e.g., Financial Visibility), which is important to
support the business capability gap analyses needed for transition
planning; however, it does not do this for all priority areas
(e.g., Acquisition Visibility). Moreover, while the latest
version's focus on DOD-wide, corporate policies, capabilities,
rules, and standards is an essential element to meeting the act's
requirements, this version has yet to be augmented by the DOD
component organizations' subsidiary architectures that are also
essential to meeting the act's requirements and the department's
goal of having a federated family of architectures. Compounding
this are our recent reports showing the military departments'
architecture programs are not mature and the strategy that the
department has developed for federating its BEA needs more
definition to be executable.^9 To address these limitations, our
recent reports contain additional recommendations. Once these
limitations are addressed, the architecture should provide a more
sufficient frame of reference to optimally guide and constrain
DOD-wide system investments.
^7GAO-06-658.
^8GAO, Business Systems Modernization: DOD Needs to Fully Define Policies
and Procedures for Institutionally Managing Investments, GAO-07-538
(Washington, D.C.: May 11, 2007); and Business Systems Modernization:
Strategy for Evolving DOD's Business Enterprise Architecture Offers
Conceptual Approach but Execution Details Needed, GAO-07-451 (Washington,
D.C.: Apr. 16, 2007).
^9GAO-07-451 and Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006).
o The updated transition plan continues to identify more systems
and initiatives that are to fill business capability gaps and
address DOD-wide and component business priorities and continues
to provide a range of information for each system and initiative
in the plan (e.g., budget information, performance metrics, and
milestones). Further, the updated plan also identifies legacy
systems that will not be part of its target environment. However,
this latest transition plan still does not include system
investment information for all the defense agencies and combatant
commands. Moreover, the plan does not sequence the planned
investments based on a range of relevant factors, such as
technology opportunities, marketplace trends, institutional system
development and acquisition capabilities, legacy and new system
dependencies and life expectancies, and the projected value of
competing investments. According to DOD officials, they intend to
address such limitations in future versions of the transition
plan. We have an existing recommendation to the department to
formalize its plans for incrementally evolving the transition
plan. Once these limitations in the department's transition
plan(s) are addressed, it will be better positioned to effectively
and efficiently migrate to a more modernized systems environment.
o The department's fiscal year 2008 budget submission provides a
range of information on business systems, including types of
information cited in the act, such as system name, designated
approval authority, and funding to be used for
development/modernization versus operations/maintenance.
o While the department has established and begun implementing the
investment review structures and processes that are consistent
with the act, it has yet to do so in a manner that is consistent
with relevant guidance. As we recently reported,^10 the department
has yet to fully define the related policies and procedures needed
to effectively execute both project-level and portfolio-based IT
investment management practices. For example, DOD had established
an enterprisewide IT investment board responsible for defining and
implementing its business system investment governance process,
but it had not fully defined the policies and procedures needed
for oversight of and visibility into operations and maintenance
investments. To address these investment management weaknesses,
our recent report contains additional recommendations. Once these
policies and procedures are fully defined, the risk of projects
and portfolios of projects being inconsistently and improperly
selected and controlled will be reduced, thus increasing the
chances of investments meeting mission needs in the most
cost-effective manner.
o The department continues to review and approve business systems
as directed by the act. As of March 2007, the department reported
that its highest investment review body had approved 285 systems.
However, the military departments' review and approval processes
are still evolving, according to Air Force, Army, and Navy
officials, and additional work is needed to mature them. Because
of the importance of the military departments' investment
management structures and processes, we have ongoing work to
determine the extent to which the Air Force and the Navy are
employing relevant investment management guidance.
In concert with the department's efforts to comply with the act,
it has also largely implemented, or our recommendations in recent
reports have otherwise subsumed, 10 of the 14 recommendations that
we identified as open in our prior annual report under the act.
For example, DOD has implemented our recommendation aimed at
effectively using the results of the BEA independent verification
and validation contractor on prior versions of the architecture.
Use of an independent verification and validation agent is an
architecture management best practice for identifying architecture
strengths and weaknesses and disclosing to department and
congressional oversight bodies the information they need to better
ensure that DOD's family of architectures and associated
transition plan(s) satisfy key quality parameters. According to
department officials, they are committed to addressing all of our
open recommendations, and have actions under way and plans in
place to address the remaining 4.
^10GAO-07-538.
To facilitate congressional oversight and promote departmental
accountability, we are recommending that the department include in
its future annual reports under the act the results of its
independent verification and validation agent's assessment of the
extent to which the department's federated family of its corporate
and component architectures, including the related transition
plan(s), are complete, consistent, understandable, and usable. The
department has not included such information in its annual
reports. In written comments on a draft of this report, signed by
the Deputy Under Secretary of Defense (Business Transformation)
and reprinted in appendix IV, the department agreed with our
recommendation.
Background
DOD is a massive and complex organization. To illustrate, the
department reported that its fiscal year 2006 operations involved
approximately $1.4 trillion in assets and $2.0 trillion in
liabilities; more than 2.9 million in military and civilian
personnel; and $581 billion in net cost of operations. To date,
for fiscal year 2007, the department received appropriations of
about $501 billion. Organizationally, the department includes the
Office of the Secretary of Defense, the Chairman of the Joint
Chiefs of Staff, the military departments, numerous defense
agencies and field activities; and various unified combatant
commands that are either responsible for specific geographic
regions or specific functions. (See fig. 1 for a simplified
depiction of DOD's organizational structure.)
Figure 1: Simplified DOD Organizational Structure
aThe Chairman of the Joint Chiefs of Staff serves as the spokesman for the
commanders of the combatant commands, especially on the administrative
requirements of the commands.
In support of its military operations, the department performs an
assortment of interrelated and interdependent business functions,
including logistics management, procurement, health care management, and
financial management. As we have previously reported,^11 the DOD systems
environment that supports these business functions is overly complex and
error prone, and is characterized by (1) little standardization across the
department, (2) multiple systems performing the same tasks, (3) the same
data stored in multiple systems, and (4) the need for data to be entered
manually into multiple systems. Moreover, DOD recently reported that this
systems environment is comprised of approximately 3,100 separate business
systems. For fiscal year 2007, Congress appropriated approximately $15.7
billion to DOD, and for fiscal year 2008, DOD has requested about $15.9
billion in appropriated funds to operate, maintain, and modernize these
business systems and associated infrastructure.
^11GAO-06-658.
As we have previously reported,^12 the department's nonintegrated and
duplicative systems impair DOD's ability to combat fraud, waste, and
abuse. In fact, DOD currently bears responsibility, in whole or in part,
for 15 of our 27 high-risk areas.^13 Eight of these areas are specific to
DOD^14 and the department shares responsibility for 7 other governmentwide
high-risk areas.^15 DOD's business systems modernization is one of the
high-risk areas, and it is an essential enabler to addressing many of the
department's other high-risk areas. For example, modernized business
systems are integral to the department's efforts to address its financial,
supply chain, and information security management high-risk areas.
Enterprise Architecture and IT Investment Management Controls Are Critical to
Achieving Successful Systems Modernization
Effective use of an enterprise architecture--a modernization blueprint--is
a hallmark of successful public and private organizations. For more than a
decade, we have promoted the use of architectures to guide and constrain
systems modernization, recognizing them as a crucial means to this
challenging goal: optimally defined operational and technological
environments. Congress, the Office of Management and Budget (OMB), and the
federal Chief Information Officer's (CIO) Council have also recognized the
importance of an architecture-centric approach to modernization. The
Clinger-Cohen Act of 1996^16 mandates that an agency's CIO develop,
maintain, and facilitate the implementation of an information technology
architecture. Further, the E-Government Act of 2002^17 requires OMB to
oversee the development of enterprise architectures within and across
agencies. In addition, we, OMB, and the CIO Council have issued guidance
that emphasizes the need for system investments to be consistent with
these architectures.^18
12See, for example, GAO, DOD Travel Cards: Control Weaknesses Resulted in
Millions of Dollars of Improper Payments, GAO-04-576 (Washington, D.C.:
June 9, 2004); Military Pay: Army National Guard Personnel Mobilized to
Active Duty Experienced Significant Pay Problems, GAO-04-89 (Washington,
D.C.: Nov. 13, 2003); and Defense Inventory: Opportunities Exist to
Improve Spare Parts Support Aboard Deployed Navy Ships, GAO-03-887
(Washington, D.C.: Aug. 29, 2003).
^13GAO-07-310.
^14These 8 high-risk areas include DOD's overall approach to business
transformation, business systems modernization, financial management, the
personnel security clearance program, supply chain management, support
infrastructure management, weapon systems acquisition, and contract
management.
^15The 7 governmentwide high-risk areas are (1) disability programs, (2)
ensuring the effective protection of technologies critical to U.S.
national security interests, (3) interagency contracting, (4) information
systems and critical infrastructure, (5) information-sharing for homeland
security, (6) human capital, and (7) real property.
^16The Clinger-Cohen Act of 1996, 40 U.S.C. S 11315(b)(2).
^17The E-Government Act of 2002, Pub. L. No. 107-347 (Dec. 17, 2002).
A corporate approach to IT investment management is characteristic of
successful public and private organizations. Recognizing this, Congress
enacted the Clinger-Cohen Act of 1996,^19 which requires OMB to establish
processes to analyze, track, and evaluate the risks and results of major
capital investments in IT systems made by executive agencies.^20 In
response to the Clinger-Cohen Act and other statutes, OMB has developed
policy and issued guidance for planning, budgeting, acquisition, and
management of federal capital assets.^21 We have also issued guidance in
this area,^22 which defines institutional structures, such as Investment
Review Boards (IRB), processes for developing information on investments
(such as costs and benefits), and practices to inform management decisions
(such as whether a given investment is aligned with an enterprise
architecture).
Enterprise Architecture: A Brief Description
An enterprise architecture provides a clear and comprehensive picture of
an entity, whether it is an organization (e.g., a federal department) or a
functional or mission area that cuts across more than one organization
(e.g., financial management). This picture consists of snapshots of both
the enterprise's current ("As Is") environment and its target ("To Be")
environment. These snapshots consist of "views," which are one or more
interdependent and interrelated architecture products (e.g., models,
diagrams, matrices, and text) that provide logical or technical
representations of the enterprise. The architecture also includes a
transition or sequencing plan, which is based on an analysis of the gaps
between the "As Is" and "To Be" environments; this plan provides a
temporal road map for moving between the two environments and incorporates
such considerations as technology opportunities, marketplace trends,
fiscal and budgetary constraints, institutional system development and
acquisition capabilities, legacy and new system dependencies and life
expectancies, and the projected value of competing investments.
^18GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington, D.C.:
March 2004); OMB, Capital Programming Guide, Version 1.0 (July 1997); and
CIO Council, A Practical Guide to Federal Enterprise Architecture, Version
1.0 (February 2001).
^19The Clinger-Cohen Act of 1996, 40 U.S.C. SS 11101-11704. This act
expanded the responsibilities of OMB and the agencies that had been set
under the Paperwork Reduction Act with regard to IT management. See 44
U.S.C. 3504(a)(1)(B)(vi) (OMB); and 44 U.S.C. 3506(h)(5) (agencies).
^20We have made recommendations to improve OMB's process for monitoring
high-risk IT investments; see GAO, Information Technology: OMB Can Make
More Effective Use of Its Investment Reviews, GAO-05-276 (Washington,
D.C.: Apr. 15, 2005).
^21This policy is set forth and guidance is provided in OMB Circular No.
A-11 (Nov. 2, 2005) (section 300) and in OMB's Capital Programming Guide,
which directs agencies to develop, implement, and use a capital
programming process to build their capital asset portfolios.
^22See, for example, GAO-04-394G; Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003); and Assessing Risks and
Returns: A Guide for Evaluating Federal Agencies' IT Investment
Decision-making, GAO/AIMD-10.1.13 (Washington, D.C.: February 1997).
The suite of products produced for a given entity's enterprise
architecture, including its structure and content, is largely governed by
the framework used to develop the architecture. Since the 1980s, various
architecture frameworks have been developed, such as John A. Zachman's "A
Framework for Information Systems Architecture"^23 and the DOD
Architecture Framework.^24
The importance of developing, implementing, and maintaining an enterprise
architecture is a basic tenet of both organizational transformation and
systems modernization. Managed properly, an enterprise architecture can
clarify and help optimize the interdependencies and relationships among an
organization's business operations (and the underlying IT infrastructure
and applications) that support these operations. Moreover, when an
enterprise architecture is employed in concert with other important
management controls, such as portfolio-based capital planning and
investment control practices, architectures can greatly increase the
chances that an organization's operational and IT environments will be
configured to optimize mission performance. Our experience with federal
agencies has shown that investing in IT without defining these investments
in the context of an architecture often results in systems that are
duplicative, not well integrated, and unnecessarily costly to maintain and
interface.^25
23J.A. Zachman, "A Framework for Information Systems Architecture," IBM
Systems Journal 26, no. 3 (1987).
^24DOD, Department of Defense Architecture Framework, Version 1.0, Volume
1 (August 2003) and Volume 2 (February 2004).
One approach to structuring an enterprise architecture is referred to as a
federated enterprise architecture. Such a structure treats the
architecture as a family of coherent but distinct member architectures
that conform to an overarching architectural view and rule set. This
approach recognizes that each member of the federation has unique goals
and needs as well as common roles and responsibilities with the levels
above and below it. Under a federated approach, member architectures are
substantially autonomous, although they also inherit certain rules,
policies, procedures, and services from higher-level architectures. As
such, a federated architecture enables component organization autonomy
while ensuring enterprisewide linkages and alignment where appropriate.
Where commonality among components exists, there are also opportunities
for identifying and leveraging shared services.
A service-oriented architecture (SOA) is an approach for sharing business
capabilities across the enterprise by designing functions and applications
as discrete, reusable, and business-oriented services. As such, service
orientation permits sharing capabilities that may be under the control of
different component organizations. As we have previously reported,^26 such
capabilities or services need to be, among other things, (1)
self-contained, meaning that they do not depend on any other functions or
applications to execute a discrete unit of work; (2) published and exposed
as self-describing business capabilities that can be accessed and used;
and (3) subscribed to via well-defined and standardized interfaces. A SOA
approach is thus not only intended to reduce redundancy and increase
integration, but also to provide the kind of flexibility needed to support
a quicker response to changing and evolving business requirements and
emerging conditions.
^25See, for example, GAO, Homeland Security: Efforts Under Way to Develop
Enterprise Architecture, but Much Work Remains, GAO-04-777 (Washington,
D.C.: Aug. 6, 2004); GAO-04-731R; Information Technology: Architecture
Needed to Guide NASA's Financial Management Modernization, GAO-04-43
(Washington, D.C.: Nov. 21, 2003); GAO-03-1018; GAO-03-877R; Information
Technology: DLA Should Strengthen Business Systems Modernization
Architecture and Investment Activities, GAO-01-631 (Washington, D.C.: June
29, 2001); and Information Technology: INS Needs to Better Manage the
Development of Its Enterprise Architecture, GAO/AIMD-00-212 (Washington,
D.C.: Aug. 1, 2000).
^26GAO, Information Technology: FBI Has Largely Staffed Key Modernization
Program, but Strategic Approach to Managing Program's Human Capital Is
Needed, GAO-07-19 (Washington, D.C.: Oct. 16, 2006).
IT Investment Management: A Brief Description
IT investment management is a process for linking IT investment decisions
to an organization's strategic objectives and business plans that focuses
on selecting, controlling, and evaluating investments in a manner that
minimize risks while maximizing the return of investment.^27
o During the selection phase, the organization (1) identifies and
analyzes each project's risks and returns before committing
significant funds to any project and (2) selects those IT projects
that will best support its mission needs.
o During the control phase, the organization ensures that, as
projects develop and investment expenditures continue, they
continue to meet mission needs at the expected levels of cost and
risk. If the project is not meeting expectations or if problems
arise, steps are quickly taken to address the deficiencies.
o During the evaluation phase, actual versus expected results are
compared once a project has been fully implemented. This is done
to (1) assess the project's impact on mission performance, (2)
identify any changes or modifications to the project that may be
needed, and (3) revise the investment management process based on
lessons learned.
Consistent with this guidance, our IT Investment Management
framework (ITIM)^28 consists of five progressive stages of
maturity for any given agency relative to selecting, controlling,
and evaluating its investment management capabilities. (See fig. 2
for the five ITIM stages of maturity.) Stage 2 critical processes
lay the foundation by establishing successful, predictable, and
repeatable investment control processes at the project level.
Stage 3 is where the agency moves from project-centric processes
to portfolio-based processes and evaluates potential investments
according to how well they support the agency's missions,
strategies, and goals. Organizations implementing these Stages 2
and 3 practices have in place selection, control, and evaluation
processes that are consistent with the Clinger-Cohen Act.^29
Stages 4 and 5 require the use of evaluation techniques to
continuously improve both investment processes and portfolios in
order to better achieve strategic outcomes.
^27GAO-04-394G; GAO, GAO/AIMD-10.1.13; Executive Guide: Improving Mission
Performance Through Strategic Information Management and Technology,
GAO/AIMD-94-115 (Washington, D.C.: May 1994); and Office of Management and
Budget, Evaluating Information Technology Investments, A Practical Guide
(Washington, D.C.: November 1995).
^28GAO-04-394G.
^2940 U.S.C. SS 11311-11313.
Figure 2: The Five ITIM Stages of Maturity with Critical Processes
The overriding purpose of the framework is to encourage investment
selection, control, and evaluate processes that promote business
value and mission performance, reduce risk, and increase
accountability and transparency. We have used the framework in
several of our evaluations,^30 and a number of agencies have
adopted it. With the exception of the first stage, each maturity
stage is composed of "critical processes" that must be implemented
and institutionalized in order for the organization to achieve
that stage. Each ITIM critical process consists of "key
practices"--to include organizational structures, policies, and
procedures--that must be executed to implement the critical
process. Our research shows that agency efforts to improve
investment management capabilities should focus on implementing
all lower stage practices before addressing higher stage
practices.
^30GAO, Information Technology: Centers for Medicare & Medicaid Services
Needs to Establish Critical Investment Management Capabilities, GAO-06-12
(Washington, D.C.: Oct. 28, 2005); Information Technology: HHS Has Several
Investment Management Capabilities in Place, but Needs to Address Key
Weaknesses, GAO-06-11 (Washington, D.C.: Oct. 28, 2005); Information
Technology: FAA Has Many Investment Management Capabilities in Place, but
More Oversight of Operational Systems Is Needed, GAO-04-822 (Washington,
D.C.: Aug. 20, 2004); Information Technology: Departmental Leadership
Crucial to Success of Investment Reforms at Interior, GAO-03-1028
(Washington, D.C.: Sept. 12, 2003); Bureau of Land Management: Plan Needed
to Sustain Progress in Establishing IT Investment Management Capabilities,
GAO-03-1025 (Washington, D.C.: Sept. 12, 2003); United States Postal
Service: Opportunities to Strengthen IT Investment Management
Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002); and Information
Technology: DLA Needs to Strengthen Its Investment Management Capability,
GAO-02-314 (Washington, D.C.: Mar. 15, 2002).
DOD's Institutional Approach to Business Systems Modernization
In 2005, the department reassigned responsibility for providing
executive leadership for the direction, oversight, and execution
of its business systems modernization efforts to several entities.
These entities and their responsibilities include the Defense
Business Systems Management Committee (DBSMC), which serves as the
highest ranking governance body for business systems modernization
activities; the Principal Staff Assistants, who serve as the
certification authorities for business system modernizations in
their respective core business missions; the IRBs, which form the
review and decision-making bodies for business system investments
in their respective areas of responsibility; and the Business
Transformation Agency (BTA), which is responsible for leading and
coordinating business transformation efforts across the
department. The BTA is organized into seven directorates, one of
which is the Defense Business Systems Acquisition Executive--the
component acquisition executive for DOD enterprise-level
(DOD-wide) business systems and initiatives. This office is
responsible for developing, coordinating, and integrating
enterprise-level projects, programs, systems and initiatives,
including managing resources such as fiscal, personnel, and
contracts for assigned systems and programs.
Table 1 lists these entities and provides greater detail on their
roles, responsibilities, and composition.
Table 1: DOD Business Systems Modernization Governance Entities' Roles,
Responsibilities, and Composition
Roles and
Entity responsibilities Composition
DBSMC o Provides strategic Chaired by the Deputy
direction and plans Secretary of Defense;
for the business Vice Chair is the Under
mission area^a in Secretary of Defense for
coordination with Acquisition, Technology,
the warfighting and and Logistics
enterprise (USD(AT&L)). Includes
information senior leadership in the
environment mission Office of the Secretary
areas. of Defense, the military
o Recommends departments' secretaries,
policies and and defense agencies'
procedures required heads, such as the
to integrate DOD Assistant Secretary of
business Defense (Networks and
transformation and Information
attain Integration)/Chief
cross-department, Information Officer
end-to-end (ASD(NII)/CIO), the Vice
interoperability of Chairman of the Joint
business systems and Chiefs of Staff, and the
processes. Commanders of the U.S.
o Serves as Transportation Command
approving authority and Joint Forces Command.
for business system
modernization.
o Establishes
policies and
approves the
business mission
area strategic plan,
the enterprise
transition plan for
implementation for
business systems
modernization, the
transformation
program baseline,
and the BEA.
Principal Staff o Support the Under Secretaries of
Assistants/Certification DBSMC's management Defense for Acquisition,
Authorities of enterprise Technology, and
business IT Logistics; Comptroller;
investments. and Personnel and
o Serve as the Readiness.
certification
authorities
accountable for the
obligation of funds
for respective
business system
modernizations
within designated
core business
missions.^b
o Provide the DBSMC
with recommendations
for system
investment approval.
IRBs o Serve as the Includes the Principal
oversight and Staff Assistants; Joint
investment Staff; ASD(NII)/CIO; core
decision-making business mission area
bodies for those representatives; military
business departments; defense
capabilities that agencies; and combatant
support activities commands.
under their
designated areas of
responsibility.
o Recommend
certification for
all business systems
investments costing
more than $1 million
that are integrated
and compliant with
the BEA.
Component o Ensures Includes the Chief
Pre-Certification component-level Information Officer from
Authority investment review the Air Force, the
processes integrate Principal Director of
with the Investment Governance, Acquisition,
Management system. and Chief Knowledge
o Identifies those Office from the Army, the
component systems Chief Information Officer
that require IRB from the Navy, and
certification and comparable
prepare, review, representatives from
approve, validate, other defense agencies.
and transfer
investment
documentation as
required.
o Assesses and
precertifies
architecture
compliance of
component systems
submitted for
certification and
annual review.
o Acts as the
component's
principal point of
contact for
communication with
the IRBs.
BTA o Operates under the Comprised of seven
authority of the directorates (Defense
USD(AT&L) under the Business Systems
direction of the Acquisition Executive,
Deputy Under Enterprise Integration,
Secretary of Defense Transformation Planning
for Business and Performance,
Transformation and Transformation Priorities
the Deputy Under and Requirements,
Secretary of Defense Investment Management,
for Financial Warfighter Support
Management. Office, and Chief of
o Maintains and Staff).
updates the
department's BEA and
enterprise
transition plan.
o Ensures that
functional
priorities and
requirements of
various defense
components, such as
the Army and Defense
Logistics Agency are
reflected in the
architecture.
o Ensures adoption
of DOD-wide
information and
process standards as
defined in the
architecture.
o Serves as the
day-to-day
management entity of
the business
transformation
effort at the DOD
enterprise level.
o Provides support
to the DBSMC and
IRBs.
Source: DOD.
aAccording to DOD, the business mission area is responsible for ensuring
that capabilities, resources, and materiel are reliably delivered to the
warfighter. Specifically, the BMA addresses areas such as real property
and human resources management.
bDOD has five core business missions: Human Resources Management, Weapon
System Lifecycle Management, Materiel Supply and Service Management, Real
Property and Installations Lifecycle Management, and Financial Management.
Tiered Accountability
In 2005, DOD reported that it had adopted a tiered accountability approach
to business transformation. Under this approach, responsibility and
accountability for business architectures and systems investment
management are assigned to different levels in the organization. For
example, the BTA is responsible for developing the corporate BEA, which
provides the thin layer of corporate policies, capabilities, standards,
and rules. The components are responsible for defining a component-level
architecture and transition plans associated with their own tier of
responsibility and for doing so in a manner that is aligned with (i.e.,
does not violate) the corporate BEA's policies, capabilities, standards,
and rules. Similarly, program managers are responsible for developing
program-level architectures and plans and ensuring alignment with the
architectures and transition plans above them. As such, this concept
allows for autonomy while also ensuring linkages and alignment from the
program level through the component level to the enterprise level.
For business investment management, responsibility and accountability is
also tiered, meaning that it is allocated between the DOD corporate level
(i.e., Office of the Secretary of Defense) and the components based on the
amount of development/modernization funding involved and the investment's
designated "tier." More specifically, DOD corporate is responsible for
ensuring that all business systems with a development/modernization
investment in excess of $1 million are reviewed by the IRBs for compliance
with the BEA, certified by the Principal Staff Assistants, and approved by
the DBSMC. Components are responsible for certifying
development/modernization investments with total costs of $1 million or
less. All DOD development and modernization efforts are also assigned a
"tier" based on acquisition category and/or the size of the financial
investment.^31
Summary of Fiscal Year 2005 National Defense Authorization Act Requirements
Congress included six provisions in the act^32 that are aimed at ensuring
DOD's development of a well-defined BEA and associated enterprise
transition plan (ETP), as well as the establishment and implementation of
effective investment management structures and processes. The requirements
are as follows:
1. Develop a BEA that:
o includes an information infrastructure that, at a minimum, would
enable DOD to:
o comply with all federal accounting, financial
management, and reporting requirements;
o routinely produce timely, accurate, and reliable
financial information for management purposes;
o integrate budget, accounting, and program
information and systems;
^31As defined in the department's Investment Review Board Concept of
Operations and its Investment Certification and Annual Review Process User
Guidance, there are four tiers of business systems. Tier 1 systems include
all systems that are classified as a "major automated information system"
or a "major defense acquisition program;" tier 2 systems include those
with modernization efforts of $10 million or greater but that are not
designated as a major automated information system or a major defense
acquisition program, or programs that have been designated as IRB interest
programs because of their impact on DOD transformation objectives; tier 3
systems include those with modernization efforts that have anticipated
costs greater than $1 million but less than $10 million; and tier 4
systems are those with modernization efforts that have anticipated costs
of up to $1 million.
^32Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. S 2222).
o provide for the systematic measurement of
performance, including the ability to produce timely,
relevant, and reliable cost information;
o includes policies, procedures, data standards, and
system interface requirements that are to be applied
uniformly throughout the department; and
o is consistent with OMB policies and procedures.
2. Develop a transition plan for
implementing the architecture that
includes:
o an acquisition strategy for new systems needed to complete the
enterprise architecture;
o a list and schedule of legacy business systems to be terminated;
o a list and strategy of modifications to legacy business systems;
and
o time-phased milestones, performance metrics, and a statement of
financial and non-financial resource needs.
3. Identify each business system proposed for funding
in DOD's fiscal year budget submissions and include:
o information on each business system proposed for funding in that
budget;
o funds for current services and for business systems
modernization; and
o the designated approval authority for each business system.
4. Delegate the responsibility for business systems
to designated approval authorities within the Office
of the Secretary of Defense.
5. Require each approval authority to establish
investment review structures and processes, including
a hierarchy of IRBs--each with appropriate
representation from across the department. The review
process must cover:
o review and approval of each business system by an IRB before
funds are obligated;
o at least an annual review of every business system investment;
o use of threshold criteria to ensure an appropriate level of
review and accountability;
o use of procedures for making architecture compliance
certifications;
o use of procedures consistent with DOD guidance; and
o incorporation of common decision criteria.
6. Effective October 1, 2005, DOD may not obligate
appropriated funds for a defense business system
modernization with a total cost of more than $1
million unless, the approval authority certifies that
the business system modernization:
o complies with the BEA and
o is necessary to achieve a critical national security capability
or address a critical requirement in an area such as safety or
security; or is necessary to prevent a significant adverse effect
on an essential project in consideration of alternative solutions,
and the certification is approved by the DBSMC.
Summary of Recent GAO Reviews of DOD's Business Systems
Modernization and Business Transformation Efforts
In November 2005^33 and in May 2006,^34 we reported that DOD had
partially satisfied four of the six business system modernization
requirements in the fiscal year 2005 National Defense
Authorization Act^35 relative to architecture development,
transition plan development, budgetary disclosure, and investment
review; it had fully satisfied the requirement concerning
designated approval authorities; and it was in the process of
satisfying the last requirement for certification and approval of
modernizations costing in excess of $1 million. As a result, we
concluded that the department had made important progress in
defining and beginning to implement institutional management
controls (i.e., processes, structures, and tools), but much
remained to be accomplished relative to the act's requirements and
relevant guidance, including developing component architectures
that are aligned with the corporate BEA and ensuring that
investment review and approval processes are fully developed and
institutionally implemented across all organizational levels.
^33GAO, DOD Business Systems Modernization: Important Progress Made in
Establishing Foundational Architecture Products and Investment Management
Practices, but Much Work Remains, GAO-06-219 (Washington, D.C.: Nov. 23,
2005).
^34GAO-06-658.
^35Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. S 2222).
Notwithstanding this progress on business systems modernization,
we also testified in November 2006^36 that DOD continued to lack a
comprehensive, enterprisewide approach to its overall business
transformation effort. We noted that while DOD's planning and
management continued to evolve, it had yet to develop a
comprehensive, integrated, and enterprisewide plan that covered
all key business functions and contained results-oriented goals,
measures, and expectations that link organizational, unit, and
individual performance goals while also being clearly linked to
DOD's overall investment plans. We concluded that because of the
complexity and long-term nature of business transformation, the
department continued to need a chief management official with
significant authority, experience, and tenure to provide sustained
leadership and integrate its overall business transformation
effort. We also concluded that without formally designating
responsibility and accountability for results, reconciling
competing priorities in investments will be difficult and could
impede DOD's progress in its transformation efforts. We are
currently assessing the department's business transformation
efforts, including an analysis of the various proposals for a
chief management officer and its response to these proposals, and
plan to report our results in the near future.
DOD Is Continuing to Improve Its Approach to Modernizing Business Systems
DOD continues to take steps to comply with the requirements of the
act and to satisfy relevant systems modernization management
guidance. In particular, on March 15, 2007, DOD released an update
to its BEA (version 4.1), developed an updated ETP, and issued its
annual report to Congress describing steps taken and planned
relative to the act's requirements, among other things.
Collectively, these steps address several legislative provisions
and best practices concerning the corporate architecture,
transition plan, budgetary disclosure, and investment review of
systems costing in excess of $1 million that we previously
reported as missing. However, additional steps are needed to fully
comply with the act and relevant guidance. Specifically, the
department has yet to extend and evolve its corporate BEA to the
department's component organizations' (military departments and
defense agencies) architectures, fully define its IT investment
management policies and procedures, and officially establish one
of the five legislatively mandated IRBs. BTA officials agree that
additional steps are needed to fully implement the act's
requirements and related system modernization management best
practices. According to BTA officials, DOD leadership is committed
to fully addressing these areas and efforts are planned and under
way to do so.
^36GAO-07-229T.
DOD Continues to Improve Its Corporate BEA, but Component
Architectures Remain a Challenge
Among other things, the act requires DOD to develop a BEA that
would cover all defense business systems and the functions and
activities supported by defense business systems and enable the
entire department to (1) comply with all federal accounting,
financial management, and reporting requirements; (2) routinely
produce timely, accurate, and reliable financial information for
management purposes; and (3) include policies, procedures, data
standards, and system interface requirements that are to be
applied throughout the department.
In 2006,^37 we reported that the then current version of the BEA
(version 3.1) addressed several of the missing elements we had
previously identified relative to the act's requirements and
relevant guidance. However, we also reported that additional steps
were needed. On March 15, 2007, DOD released an update to its BEA
(version 4.1), which resolves several of the architecture gaps
associated with the prior version and adds content proposed by DOD
stakeholders.^38 For example, version 4.1 improves the Financial
Visibility business enterprise priority (BEP) area by including
the Standard Financial Information Structure data elements and
business rules to support cost accounting and reporting. This
version also addresses, to varying degrees, missing elements,
inconsistencies, and usability issues that we previously
identified.^39 Examples of these improvements and remaining issues
are summarized in the following text:
^37GAO-06-658.
^38According to DOD, the BEA stakeholders include the core business
mission areas through the Business Enterprise Priorities, which comprises
Personnel Visibility, Acquisition Visibility, Common Supplier Engagement,
Materiel Visibility, Real Property Accountability, and Financial
Visibility. The department added that as the BEA evolves, the stakeholders
will include components that must federate their architectures to the BEA,
program managers who must comply with the BEA, IRBs who use the BEA to
guide and constrain investments, and systems designers and integrators who
must build and configure their systems to comply with the BEA.
^39GAO-06-658.
o This latest version contains enterprise-level information about
DOD's "As Is" architectural environment to support business
capability gap analyses. As we previously reported,^40 such gap
analyses between the "As Is" and the "To Be" environments are
essential for the development of a well-defined transition plan.
However, such gap analyses were not previously provided for in
prior versions of the BEA. To DOD's credit, the architecture now
includes "As Is" information (e.g., problems that enterprise
priorities are to address and the root causes of each problem) for
five of the six BEPs. For example, this version identifies the
"inability to record or report funds distribution at the
transaction level" as a problem for the Financial Visibility
priority area, and "stove-pipe systems" and "non-standard forms"
as the root causes. Moreover, it includes "As Is" information
about related enterprise systems, such as the Wide-area Workflow
system. However, the current version does not provide "As Is"
information for the Acquisition Visibility priority area.
o The latest version includes performance metrics for the business
capabilities within enterprise priority areas, including actual
performance relative to performance targets that are to be met.
For example, currently 26 percent of DOD assets are reported by
using the Department of the Treasury's United States Standard
General Ledger^41 compliant formats, as compared to a target of
100 percent. However, the architecture does not describe the
actual baseline performance for operational activities, such as
for the "Manage Audit and Oversight of Contractor" operational
activity. As we have previously reported,^42 performance models
are an essential part of any architecture and having defined
performance baselines to measure actual performance against
provides the means for knowing whether the intended mission value
to be delivered by each business process is actually being
realized.
o The latest version identifies activities performed at each
location/organization and indicates which organization(s) are or
will be involved in each activity. We previously reported that
prior versions did not address the locations where specified
activities are to occur and that doing so is important because the
cost and performance of implemented business operations and
technology solutions are affected by the location and therefore
need to be examined, assessed, and decided on in an enterprise
context rather than in a piecemeal, systems-specific fashion.^43
To DOD's credit, the latest version includes some of this
information. For example, it indicates that the Defense Contract
Management Agency is involved in the "Conduct Acquisition
Assessment" operational activity. However, not all operational
activities, such as "Authorize Return or Disposal" activity are
assigned to a location/organization. In addition, the latest
version does not include the roles and responsibilities of
organizations performing the same operational activities, which is
important to avoid duplication and inconsistency in how functions
and activities are implemented.
^40GAO-06-219.
^41The United States Standard General Ledger provides a uniform chart of
accounts and technical guidance used in standardizing federal agency
accounting.
^42GAO-04-777 and GAO-03-584G.
o The latest version includes common policies (e.g., "IRBs approve
only those system investments that are aligned with enterprise
transformation objectives and standards") and procedures (e.g.,
"Components and programs use the Architecture Compliance and
Requirements Traceability tool to illustrate how their system
investments map to applicable activities, business rules, and data
in the BEA"). It also includes business rules (e.g., "each request
for commercial export of DOD technology must be processed within
30 days upon receipt of request from the Department of State or
the Department of Commerce") to facilitate consistent
implementation of the policies and procedures.^44 However, the
architecture does not identify enterprise business rules for all
business processes. For example, there are no business rules for
the Common Supplier Engagement business process "Perform
Acceptance Procedures for Other Goods and Services." Moreover, the
latest version continues to provide inconsistent levels of detail
for some business rules. For example, some business rules are
defined at the conceptual level (e.g., "ENT_Cost_Reporting") while
others are defined at a more operational level (e.g.,
"ENT_DOD_Obligations_Against"). Without well-defined business
rules, it is likely that policies and procedures will be
implemented inconsistently because they will be uniquely
interpreted.
o The latest version provides information flows among some
organizational units, business operations, and system elements.
These information flows are intended to show what information is
needed and where and how the information moves and is shared to
support mission functions. For example, the "Financial Management
Detail" operational node connectivity diagram is a graphical
depiction of the operational nodes (or organizations) with
"needlines" that indicate a need to exchange information and
identify information exchange requirements among the financial
management organizational units (e.g., between the accounting
office and commercial entitlement office operational nodes).
However, detailed operational node connectivity diagrams similar
to the "Financial Management Detail" diagram have not yet been
developed for the other core business mission areas, such as Human
Resources Management. Such information is critical for defining
business service interactions and establishing interfaces between
users and systems. Moreover, the BEA does not include information
flows between the enterprise and DOD components. Such information
is important for developing a common understanding of the semantic
meaning of information exchanges among DOD organizations.
^43GAO-06-658.
^44Business rules are important because they explicitly translate business
policies and procedures into specific, unambiguous rules that govern what
can and cannot be done.
o The latest version continues to represent the thin layer of
DOD-wide corporate architectural policies, capabilities, rules,
and standards. Having this layer is essential to a well-defined
federated architecture, but it alone does not provide the total
federated family of DOD parent and subsidiary architectures for
the business mission area that are needed to comply with the act.
As we recently reported, well-defined architectures do not yet
exist for the military departments,^45 which constitute the
largest members of the federation. In particular, we reported that
none of the three military departments had fully developed
architecture products that describe their respective target
architectural environments and developed transition plans for
migrating to a target environment, and none were employing the
full range of architecture management structures, processes, and
controls provided for in relevant guidance. Accordingly, we made
recommendations aimed at improving the management and content of
the military departments' respective architectures, which the
department agreed with.^46 (See app. III for the specific
recommendations.)
Recognizing the need to address its component architecture
challenge, the BTA released its business mission area federation
strategy and road map in September 2006 to address how the
corporate BEA would be extended to the military departments and
defense agencies. We recently reported^47 that this strategy
provides a foundation on which to build and align DOD's parent
business architecture with the subsidiary architectures of the
military departments and defense agencies (see fig. 3). In
particular, we noted that the strategy (1) states the department's
federated architecture goals; (2) describes federation concepts
that are to be applied; and (3) includes high-level activities,
capabilities, products, and services intended to facilitate
implementation of the concepts.
^45GAO-06-831.
^46GAO-06-831.
^47GAO-07-451.
Figure 3: Simplified Diagram of DOD's Business Mission Area Federated
Architecture
However, we also reported that the strategy does not adequately define the
tasks needed to achieve the strategy's goals, including those associated
with executing high-level activities and providing related capabilities,
products, and services. Specifically, it does not adequately address how
strategy execution will be governed, including assignment of roles and
responsibilities, measurement of progress and results, and provision of
resources. Also, the strategy does not address, among other things, how
the component architectures will be aligned with the latest version of the
BEA and how it will identify and provide for reuse of common applications
and systems across the department. Accordingly, we made recommendations
aimed at better defining the department's architecture federation plans,
which the department largely disagreed with.^48 (See app. III for the
specific recommendations.)
According to DOD, the corporate BEA focuses on providing tangible outcomes
for a limited set of enterprise-level (DOD-wide) priorities, and the
components are responsible under the department's tiered accountability
approach for defining their respective component-level architectures that
are aligned with the corporate BEA. According to DOD, subsequent releases
of the BEA will continue to reflect this federated approach and will
define enforceable interfaces to ensure interoperability and information
flow to support decision making at the appropriate level. To help ensure
this, the BTA plans to have its BEA independent verification and
validation contractor examine architecture federation when evaluating
subsequent BEA releases. Use of an independent verification and validation
agent is an architecture management best practice for identifying
architecture strengths and weaknesses. Through the use of such an agent,
department and congressional oversight bodies can gain information that
they need to better ensure that DOD's family of architectures and
associated transition plan(s) satisfy key quality parameters, such as
completeness, consistency, understandability, and usability, which the
department's annual reports have yet to include.
Until DOD has a well-defined family of architectures for its business
mission area, it will not fully satisfy the requirements of the act and it
will remain challenged in its ability to effectively manage its business
system modernization efforts.
DOD Continues to Expand and Update Its Enterprise Transition Plan, but Important
Elements Are Still Missing
Among other things, the act requires DOD to develop an ETP for
implementing its BEA that includes listings of the legacy systems that
will and will not be part of the target business systems environment and
specific time-phased milestones and performance metrics.
In 2006,^49 we reported that the prior version of the ETP addressed
several of the missing elements that we previously identified relative to
the act's requirements and relevant guidance. However, we also reported
that additional steps were needed. On March 15, 2007, DOD released an
updated version of its ETP, which provides information on 106 of what it
refers to as transformational programs (systems and initiatives) and
relates these to key transformational objectives. For example, it includes
specific time-phased milestones^50 for about 86 business system
investments and initiatives and performance metrics for about 84 systems
and initiatives. Further, the ETP discusses progress made on business
system investments over the last 6 months--including key accomplishments
and milestones attained, as well as new information on near-term
activities (i.e., activities to occur during the next 6 months). This
version also addresses, to varying degrees, missing elements that we
identified in our prior report.^51 Examples of these improvements and
remaining issues are summarized in the following text:
^48GAO-07-451.
^49GAO-06-658.
o The latest version of the ETP documents the results of ongoing
and planned analyses of gaps between its "As Is" and "To Be"
architectural environments, in which capability and performance
shortfalls are described and investments (such as transformation
initiatives and systems) that are to address these shortfalls are
clearly identified. For example, it aligns the Defense Integrated
Military Human Resources System with the Personnel Visibility
priority area and states that it will provide business capability
improvements that include providing accurate and timely pay
benefits for military service members and their families anytime
and anywhere. However, the gap analysis is not yet completed for
all the current BEPs. In particular, the gap analysis did not
include the Acquisition Visibility priority area. Without
identifying how business capability gaps between the baseline and
target architecture are to be addressed for all BEPs, the
department's transition plan cannot be considered sufficiently
complete, and thus its ability to support informed investment
selection and control decisions is limited.
o The latest version of the ETP provides a range of information
for the 106 systems and initiatives identified, such as 3 years of
budget information for 64 of these systems and initiatives.
However, the plan has yet to address our prior finding for
including system and budget information for investments by 13 of
its 15 defense agencies^52 and for 8 of its 9 combatant
commands.^53 BTA officials told us that information for these
defense agencies and combatant commands is not included because
the ETP focused on the largest business-related organizations in
DOD (i.e., those having the majority of the tier 1 and 2 business
investments), and the majority of the defense agencies and
commands do not have investments that meet this threshold
criteria. Nevertheless, they said that they plan to include all
component tier 1 and 2 systems over the next 3 years.
^50The time-phased milestones refer to milestones, such as initial
operating capability, full operating capability, technology development
phase, and system development and demonstration phase.
^51GAO-06-658.
o The latest version also provides performance measures for the
enterprise and component transformation programs, including key
milestones (e.g., Initial Operating Capability). However, the ETP
does not include other important information needed to understand
the sequencing of these business investments. In particular, the
planned investments in the transition plan are not sequenced based
on a range of activities that are critical to developing an
effective transition plan. More specifically, we previously
reported^54 that the plan is largely based on a bottom-up planning
process in which ongoing programs were examined and categorized in
the plan around BEPs and capabilities, including a determination
as to which programs would be designated and managed as DOD-wide,
enterprise programs versus component programs. This bottom-up
approach to developing the plan does not explicitly reflect
transition planning key practices cited in federal guidance, such
as consideration of technology opportunities, marketplace trends,
fiscal and budgetary constraints, institutional system development
and acquisition capabilities, and new and legacy system
dependencies and life expectancies, and the projected value of
competing investments.^55 For example, many of these investments
are dependent on Net-Centric Enterprise Services (NCES)^56 for its
core services, and as such the plans and milestones for each
should reflect the incremental capability deployment of NCES.
According to the BTA official responsible for the ETP, the
transition plan investments have not been sequenced based on any
of these considerations other than fiscal year budgetary
constraints. However, DOD officials reported that the BTA intends
to depict the dependencies in the ETP, especially
program-to-program dependencies associated with adoption of a
service-oriented architecture approach. BTA officials also said
that each technology-based sequencing decision will be governed by
DOD's tiered accountability approach to investment decision making
and architecture federation.
^52DOD included system and budget information for the Defense Financial
and Accounting Service and Defense Logistics Agency in the transition
plan. DOD did not include this information for the following defense
agencies: (1) Missile Defense Agency, (2) Defense Advanced Research
Projects Agency, (3) Defense Commissary Agency, (4) Defense Contract Audit
Agency, (5) Defense Contract Management Agency, (6) Defense Information
Systems Agency, (7) Defense Intelligence Agency, (8) Defense Legal
Services Agency, (9) Defense Security Cooperation Agency, (10) Defense
Security Service, (11) Defense Threat Reduction Agency, (12) National
Geospatial-Intelligence Agency, and (13) National Security Agency.
^53DOD included system and budget information for the Transportation
Command in the transition plan. DOD did not include this information for
the (1) Central Command, (2) Joint Forces Command, (3) Pacific Command,
(4) Southern Command, (5) Space Command, (6) Special Operations Command,
(7) European Command, and (8) Strategic Command.
^54GAO-06-219.
^55GAO-03-584G and CIO Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001).
o The latest version of the ETP includes a listing of the legacy
systems that will not be part of the "To Be" environment and the
termination dates for many of these systems. We previously
reported^57 that the prior version did not include a complete
listing of the legacy systems and that the termination dates for
many legacy systems, including the Personnel Records Management
System, Defense Departmental Reporting System, and Base Accounts
Receivable System, were not known, making it unclear whether or
not they will be part of the target environment. To DOD's credit,
the ETP now reflects all decisions recorded to date on these
legacy system terminations. According to the department, this list
will continue to evolve as components and IRBs make investment
decisions in the future. In addition, it provides information on
legacy system migration and retirement as a result of implementing
each target system. According to DOD, the annual report lists over
700 systems targeted for elimination as a result of the
implementation of targeted business systems, with specific
termination dates identified for over 93 percent of these systems.
^56NCES is intended to provide capabilities that are key to enabling
ubiquitous access to reliable decision-quality information. NCES
capabilities can be packaged into four product lines: service-oriented
architecture foundation (e.g., security and information assurance),
collaboration (e.g., application sharing), content discovery and delivery
(e.g., delivering information across the enterprise), and portal (e.g.,
user-defined Web-based presentation).
^57GAO-06-658.
o The latest version of the ETP also includes for the first time a
discussion of how the department plans to use enterprise
application integration,^58 including plans, methods, and tools
for reusing applications that already exist while also adding new
applications and databases. However, this discussion is
nevertheless still notional and thus lacks specifics on which
investments will reuse which applications.
According to BTA officials, a number of actions are envisioned to
address the above cited areas and further improve the ETP, such as
adding the results of capability gap analyses for all business
priorities, including tier 1 and 2 programs for all components,
and recognizing dependencies among investments. Until the ETP, or
a federated family of such plans, either directly or by reference
includes relevant information on the full inventory of investments
across the department, (and does so in a manner that reflects
consideration of the range of variables associated with a
well-defined transition plan, such as timing dependencies among
investments and the department's capability to manage them) it
will not have a sufficient basis for informed investment decision
making regarding disposition of the department's existing
inventory of systems or for sequencing the introduction of
modernized systems. To ensure that the above discussed
shortcomings with the department transition plan(s) are made, we
have previously made recommendations that the department is still
in the process of addressing aimed at formalizing its plans for
incrementally improving its transition plan. (See app. II for
these recommendations.)
DOD's Fiscal Year 2008 Budget Submission Includes Key Information
on Business Systems
Among other things, the act requires DOD's annual IT budget
submission to include key information on each business system for
which funding is being requested, such as the system's designated
approval authority and the appropriation type and amount of funds
associated with development/modernization and current services (to
operate and maintain the system).
The department's fiscal year 2008/2009 budget submission includes
a range of information for business system investments requesting
funding, such as the system's (1) name, (2) approval authority,
(3) approved funding for fiscal year 2007, and (4) requested
funding for fiscal year 2008. The submission also identifies the
amount of the fiscal year 2008 request that is for
development/modernization versus operations/maintenance (i.e.,
current services). For example, the Army's General Fund Enterprise
Business System, the amount of modernization funds related to
"Other Procurement, Army" and "Research, Development, Testing and
Evaluation, Army" are identified. For systems in excess of $1
million in modernization funding, the submission also cites the
DBSMC approval date, where applicable.
^58Enterprise application integration software is a commercial software
product, commonly referred to as middleware, to permit two or more
incompatible systems to exchange data from different databases.
DOD Has Largely Established Key Investment Management Structures,
but Related Policies and Procedures Are Missing
The act requires DOD to establish business system investment
review structures, including the previously mentioned DBSMC and
five IRBs, and processes that are consistent with the investment
management provisions of the Clinger-Cohen Act.^59 As noted
earlier, our ITIM framework provides five progressive stages of
maturity for any given agency relative to selecting, controlling,
and evaluating its IT investments. Organizations implementing
Stages 2 and 3 practices have in place capabilities that assist in
establishing selection, control, and evaluation structures,
policies, procedures, and practices that are required by the
investment management provisions of the Clinger-Cohen Act.
In 2006, we reported that DOD had established the DBSMC and four
of the five IRBs defined in the act and that it had developed a
range of processes governing how business system investments are
to be reviewed and approved.^60 More recently, we reported on the
extent to which the department's corporate approach to business
system investment management comports with the stages in our ITIM
framework that are associated with investment management
provisions of the Clinger-Cohen Act.^61 In summary, we found that
DOD had established important management structures needed to
manage its business system investments, but it had not fully
defined many of related policies and procedures that our framework
identified as needed to effectively manage its business
investments as individual projects (Stage 2) and as portfolios of
projects (Stage 3).
^5940 U.S.C. S 11312.
^60GAO-06-658.
^61GAO-07-538.
Investment Management Structures Have Been Largely Established
DOD has largely established the organizational structures that are
associated with Stages 2 and 3 of our framework. Specifically, it
has established an enterprisewide investment board and subordinate
boards that are responsible for business systems investment
governance, including conducting investment certification and
approval reviews and annual reviews as provided for in the act.
The enterprisewide board--the DBSMC--is composed of senior
executives, including the Deputy Secretary of Defense and the
ASD(NII)/CIO, as provided for in the act. Among other things, the
DBSMC is responsible for establishing and implementing policies
governing the organization's investment process and approving
lower-level investment board processes and procedures. The
subordinate boards include four IRBs that are composed of
representatives from their respective core business mission, as
well as representatives from the combatant commands, defense
agencies, military departments, and Joint Chiefs of Staff. Among
other things, they are responsible and accountable for overseeing
and controlling certain business system investments, including
ensuring compliance and consistency with the BEA. The department
has also assigned responsibility to the USD(AT&L) for managing
business system portfolio selection criteria.
Moreover, since we reported in 2006^62 that the department has
established four of the five IRBs mandated by the act, efforts
have begun to establish the fifth. Specifically, ASD(NII)/CIO
officials told us that they are now in the process of establishing
the Enterprise Information Environment Mission Area^63 IRB to
support IT infrastructure and information assurance activities, as
required by the act. According to these officials, the draft
concept of operations for this IRB is being revised and will
subsequently be approved by the ASD(NII)/CIO. While the IRB has
not been officially established, the officials stated that it has
been in effect for about a year and added that the chair is the
DOD Deputy CIO, and its membership includes representatives from
the Defense Information Systems Agency, the DOD mission areas, and
the military departments. They also said that the Under Secretary
of Defense (Comptroller) and the Joint Chiefs of Staff are
operating in an advisory role.
^62GAO-06-658.
^63The Enterprise Information Environment Mission Area enables the
functions of the other mission areas (e.g., Warfighting Mission Area,
Business Mission Area, and Defense Intelligence Mission Area) and
encompasses communications, computing, and core enterprise service
systems, equipment, or software that provide a common information
capability or service for enterprise use.
Policies and Procedures Have Been Defined for Some, but Not All,
Project-Level and Portfolio-Based Investment Management Activities
As we recently reported,^64 DOD has defined policies and
procedures relative to several key practices in our ITIM framework
that are associated with project-level investment management
(Stage 2). To its credit, the department has, for example,
documented policies and procedures for ensuring that systems
support ongoing and future business needs through alignment with
the BEA; developed procedures for identifying and collecting
information about these systems to support DBSMC and IRB
investment decision making; and assigned responsibility for
ensuring that the information collected about projects meets the
needs of DOD's investment review structures and processes.
However, we reported that it had not developed the full range of
project-level policies and procedures needed for effective
investment management. In commenting on our report, DOD stated
that under DOD's tiered accountability, these are performed at the
component level, and that departmental policies and procedures
established for overseeing execution of these practices by
components are sufficient. We do not agree. Examples of the
limitations in the department's project-level policies and
procedures are summarized next, along with their significance.
o Policies and procedures do not address how business system
investments that are past the development/modernization stage
(i.e., in operations and maintenance) are to be governed or
considered by the DBSMC or the IRBs. Given that DOD invests
billions of dollars annually in operating and maintaining business
systems, this is significant. While DOD officials stated that
component-level policies and procedures address systems that are
outside of development/modernization, best practices emphasize
that the corporate investment boards should continue to review
investment cost and performance baselines throughout their life
cycles.
o Policies and procedures do not outline how the DBSMC and IRB
certification and annual review processes are to be coordinated
with other decision-support processes used at DOD, such as the
Joint Capabilities Integration and Development System; the
Planning, Programming, Budgeting, and Execution system; and the
Defense Acquisition System.^65 Without clear linkages among these
processes, inconsistent and uninformed decision making may result.
^64GAO-07-538.
^65The Joint Capabilities Integration and Development System is a
need-driven management system used to identify future capabilities for
DOD; the Planning, Programming, Budgeting, and Execution process is a
calendar-driven management system for allocating resources and is
comprised of four phases--planning, programming, budgeting, and
executing--that define how budgets for each DOD component and the
department as a whole are created, vetted, and executed; and the Defense
Acquisition System is an event-driven system for managing product
development and procurement and guides the acquisition process for DOD.
o Procedures do not specify how the full range of cost, schedule,
and benefit data is to be used by the IRBs in certification
decisions. Without documenting how such boards are to consider
cost, schedule, and benefits factors when making these decisions,
the department cannot ensure that the boards and the DBSMC
consistently and objectively select proposals that best meet the
department's needs and priorities.
o Policies and procedures do not exist that provide for sufficient
oversight and visibility into component-level investment
management activities, including component reviews of systems in
operations and maintenance and tier 4 investments. According to
DOD officials, such oversight is accomplished through the
department's tiered accountability approach. However, the
department did not provide policies and procedures defining how
the DBSMC and IRBs ensure visibility into these component
processes. This is particularly important because, according to
DOD, only 285 of about 3,100 total business systems have completed
the IRB certification process and have been approved by the DBSMC.
Moreover, they said that the remaining business systems have not
been through the certification process and have not been given a
tier designation. Without policies and procedures defining how the
DBSMC and IRBs have visibility into and oversight of all business
system investments, DOD risks components continuing to invest in
systems that are duplicative, stovepiped, non-integrated, and
unnecessarily costly to manage, maintain, and operate.
DOD's policies and procedures relative to portfolio-based business
system investment management (Stage 3) are even less defined that
than those for project-level investment management. As we recently
reported,^66 DOD has not defined any of the policies and
procedures that our ITIM framework identifies as needed for
effective portfolio management. For example, the business mission
area does not have documented policies and procedures for defining
the criteria to be used for making portfolio selection decisions,
creating the portfolio of business system investments, evaluating
the performance of portfolio investments, and conducting
postimplementation reviews of these investments. According to our
ITIM framework, the development and use of portfolio selection
criteria focuses on the synergistic benefits to be found among an
agency's entire collection of investments, rather than just from
the sum of the individual investments. Moreover, adequately
documenting both the policies and the associated procedures that
provide predictable, repeatable, and reliable investment selection
and control and govern how an organization manages its IT
investment portfolio(s) is important because doing so reduces
investment risk of failure and provides the basis for having
rigor, discipline, and repeatability in how investments are
selected and controlled across the entire organization. In
commenting on our recent report, DOD stated that it intends to
improve departmental policies and procedures for business system
investments by, for example, establishing a single governance
structure, but plans or time frames for doing so had not been
established.
^66GAO-07-538.
Until DOD fully defines departmentwide policies and procedures for
both individual projects and portfolios of projects, it risks
selecting and controlling these business system investments in an
inconsistent, incomplete, and ad hoc manner, which in turn reduces
the chances that these investments will meet mission needs in the
most cost-effective manner. Accordingly, our recent report made a
series of recommendations to the department for strengthening both
its project- and portfolio-level business system investment
management policies and procedures.^67
^67GAO-07-538.
DOD Continues to Approve and Review Business Systems, but Military
Departments Processes for Doing So Are Still Evolving
The act specifies two basic requirements that took effect October
1, 2005, relative to DOD's obligation of funds for business system
modernizations costing more than $1 million. First, it requires
that these modernizations be certified by a designated approval
authority^68 as meeting specific criteria.^69 Second, it requires
that the DBSMC approve each of these certifications. The act also
states that failure to do so before the obligation of funds for
any such modernization constitutes a violation of the
Anti-deficiency Act.^70 In March 2006, the department reported
that the DBSMC had approved 226 business system modernizations,
and as of March 2007, it reported that the committee approved an
additional 59 systems, for a total of 285 approved systems.
A key element of the department's approach to reviewing and
approving business systems investments is the use of "tiered
accountability," in which investment review begins at the
component level and proceeds through a hierarchy of review and
approval authorities, depending on the size and significance of
the investment. Air Force, Army, and Navy officials told us that
the success of the process depends on thorough analysis of each
business system before it is submitted for higher-level review and
approval. However, they added that their respective processes for
reviewing investments are still evolving. A brief summary of each
military department's investment review activities is provided in
the following text.
Air Force
Air Force officials report that their department is following a
phased approach to conducting reviews of about 930 business
systems in accordance with the requirements of the act. In fiscal
year 2007, it is to review all tiers 1 through 4 business systems,
as well as tier 5 business systems^71 that have operating costs,
not simply development and modernization funding, greater than $1
million. During fiscal year 2008, the Air Force plans to review
all business systems in tiers 1 through 4 and all tier 5 systems
that have operating costs greater than $500,000. For fiscal year
2009, all business systems are to be reviewed. According to Air
Force officials, implementing a phased approach allows time to
adopt the investment management guidance set forth in our ITIM
framework.^72 While not specifically required by the act, Air
Force officials told us that the investment management practices
that it intends to put in place for its business systems will also
be leveraged for non-business system investments (e.g.,
warfighting systems). We currently have ongoing work to review the
extent to which the Air Force's business systems investment
structures and processes comport with our ITIM framework.
^68Approval authorities (the USD(AT&L); the Under Secretary of Defense
(Comptroller); the Under Secretary of Defense for Personnel and Readiness;
the ASD(NII)/CIO; and the Deputy Secretary of Defense or an Under
Secretary of Defense, as designated by the Secretary of Defense) are
responsible for the review, approval, and oversight of business systems
and must establish investment review processes for systems under their
cognizance.
^69A key condition identified in the act includes certification by
designated approval authorities that the defense business system
modernization is (1) in compliance with the enterprise architecture; (2)
necessary to achieve critical national security capability or address a
critical requirement in an area such as safety or security; or (3)
necessary to prevent a significant adverse effect on a project that is
needed to achieve an essential capability, taking into consideration the
alternative solutions for preventing such an adverse effect.
^7031 U.S.C. S 1341(a)(1)(A); see 10 U.S.C. S 2222(b).
Army
Army officials report that their department's primary emphasis has
been on reviewing its business system investments with funding in
excess of $1 million (i.e., tiers 1 through 3 business systems).
However, officials told us that they intend to develop a list of
all business systems that require annual reviews through January
2008 to guide future efforts. Currently, the Army reports an
inventory of 873 business systems, of which 108 are systems with
development/modernization funding in excess of $1 million, and
another 765 business system investments with funding below $1
million, including 62 with no development/modernization funding.
Navy
Navy officials report that their department is in the process of
conducting reviews of its 697 business systems in accordance with
the requirements of the act, although the processes being used are
still evolving. For example, Navy officials stated that the focus
of the reviews has thus far been on those systems with
development/modernization funding over $1 million. According to
DOD, for fiscal years 2006 and 2007, 54 business systems were
certified by the IRBs and approved by the DBSMC. Further, they
said that greater coordination with DOD functional areas (e.g.,
logistics) and ASD(NII)/CIO is needed to improve the control and
accountability over its business system investments. We currently
have ongoing work to review the extent to which the Navy's
business systems investment structures and processes comport with
our ITIM framework.
^71According to Air Force officials, tier 5 systems only spend current
service funds.
^72GAO-04-394G.
DOD Continues to Implement Our Prior Recommendations
The act's requirements concerning the architecture, transition
plan, budgetary disclosure, and investment management structures
and processes--as discussed earlier--are consistent with the 35
recommendations that we have made since 2001, to assist the
department in developing a well-defined and useful BEA and using
it to gain control over its ongoing business system investments.
To its credit, DOD largely agreed with these recommendations and
stated its commitment to implement them. In May 2006, we reported
that the department had taken steps to fully implement 21 of the
recommendations, while 14 had yet to be fully implemented.^73
Since then, 10 of the 14 have either been largely implemented or
have been subsumed by our more recent recommendations and thus we
are considering them closed. (See app. II for details on the
status of these 14 recommendations; see app. III for a detailed
listing of the additional recommendations that we have made since
our last annual report under the act.) For example, DOD has
addressed the core elements in our Enterprise Architecture
Management Maturity Framework^74 relative to its corporate BEA. In
particular, it has established a chief architect who is
responsible for developing the corporate BEA and ensuring that the
BEA depicts the "As Is" and "To Be" environments in terms of
business, performance, information/data, application/service,
technology, and security. As another example, the department has
taken steps to make effective use of the results of its BEA
independent verification and validation contractor on prior
versions of the architecture. As we have previously reported,
using an independent verification and validation agent is a
recognized best practice because it provides internal and external
oversight bodies important information on architecture and
transition plan quality and governance. By having and using an
independent verification and validation agent, organizations can
disclose to oversight bodies independent assessments of
architecture and transition plan quality, to include completeness,
consistency, understandability, and usability, which the
department has yet to provide in its annual reports.
With respect to the remaining 4 of the 14 recommendations, actions
are under way that are intended to implement them. For example, in
response to our recommendation to develop a BEA program management
plan^75 that defines what the department's incremental
improvements to the architecture and transition plan will be, and
how and when they will be accomplished, the BTA has developed the
Business Transformation Guidance, which describes the high-level
process by which incremental improvements are identified and
eventually incorporated into the architecture. In addition, BTA
officials stated that they are developing a BEA Concept of
Operations, which is to describe high-level milestones for the
BEA's use.
^73GAO-06-658.
^74GAO-03-584G.
As another example, the BTA has established a communications team
that is responsible for achieving strategic communications
objectives and promoting external awareness of the department's
vision, mission, and progress, and BTA officials told us that this
team is in the process of developing a communications plan.
According to the officials, these efforts will address our
recommendation for the BEA program to be supported by a proactive
marketing and communication program.^76
According to the Deputy Under Secretary of Defense (Business
Transformation), the department is committed to addressing all of
our open recommendations. It is important that the department move
swiftly in doing so because these recommendations are aimed at
strengthening architecture (and transition planning) management
activities and controlling ongoing and planned business system
investments. Until it does, the department will be challenged in
its ability to effectively guide and constrain the billions of
dollars it invests annually in thousands of business system
investments.
Conclusions
Since our last legislatively mandated report on DOD's compliance
with section 332 of the National Defense Authorization Act for
Fiscal Year 2005, DOD has continued to make important progress in
defining and implementing institutional modernization management
controls and business systems budgetary disclosure, but much
remains to be accomplished. In particular, the department has yet
to extend and evolve its corporate BEA through the development of
aligned subordinate architectures for each of its component
organizations, and while it has developed a strategy for
federating the BEA in this manner, this strategy lacks the detail
needed for it to be effectively implemented. Compounding this
situation is the known immaturity of the military service
architecture efforts, as well as DOD's corporate approach to
business system investment management not being governed by the
range of defined policies and procedures that are associated with
effective investment selection, control, and evaluation. Moreover,
the military departments' investment review processes are still
evolving. These architecture and investment management limitations
continue to put the billions of dollars that DOD spends each year
on its thousands of business system investments at risk.
^75GAO-06-658.
^76GAO-03-458.
The recommendations that we have made since we issued our last
annual report under the act are aimed at addressing these
architecture and investment management challenges. Given the
demonstrated commitment of DOD leadership to improving its
business systems modernization efforts and its recent
responsiveness to our prior recommendations, we are optimistic
concerning the likelihood that the department will continue to
make progress on these fronts.
Development of a well-defined federated architecture for the
business mission area and the definition of effective business
system investment management policies and procedures across all
levels of the department are critically important in addressing
the DOD business system modernization high-risk area. However, the
more formidable challenge facing the department is how well it
actually implements the architecture and investment management
controls over the years ahead on each and every business system
investment. While not a guarantee, development of a federated BEA,
including a transition plan(s), and effective institutional
business system investment management processes can go a long way
in addressing this longer-term challenge. In this regard, it is
important for the department to keep congressional defense
committees fully informed about its progress in federating the DOD
corporate BEA, to include the maturity of component organization
architecture efforts and the related transition plan(s).
Recommendation for Executive Action
To facilitate congressional oversight and promote departmental
accountability, we recommend that the Secretary of Defense direct
the Deputy Secretary of Defense, as the chair of the DBSMC, to
include in DOD's annual report to Congress on compliance with the
section 332 of Fiscal Year 2005 National Defense Authorization
Act, the results of assessments by its BEA independent
verification and validation contractor of the completeness,
consistency, understandability, and usability of its federated
family business mission area architectures, including the
associated transition plan(s).
Agency Comments
In written comments on a draft of this report, signed by the
Deputy Under Secretary of Defense (Business Transformation) and
reprinted in appendix IV, the department agreed with our
recommendation.
We are sending copies of this report to interested congressional
committees; the Director, Office of Management and Budget; the
Secretary of Defense; the Deputy Secretary of Defense; the Under
Secretary of Defense for Acquisition, Technology, and Logistics;
the Under Secretary of Defense (Comptroller); the Assistant
Secretary of Defense (Networks and Information Integration)/Chief
Information Officer; the Under Secretary of Defense (Personnel and
Readiness); and the Director, Defense Finance and Accounting
Service. Copies of this report will be made available to other
interested parties upon request. This report will also be
available at no charge on our Web site at http://www.gao.gov .
If you or your staffs have any questions on matters discussed in
this report, please contact me at (202) 512-3439 or [email protected],
or McCoy Williams at (202) 512-9095 or [email protected]. Contact
points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff
who made major contributions to this report are listed in appendix
V.
Randolph C. Hite
Director
Information Technology Architecture and Systems Issues
McCoy Williams
Director
Financial Management Assurance
List of Committees
The Honorable Carl Levin
Chairman
The Honorable John McCain
Ranking Member
Committee on Armed Services
United States Senate
The Honorable Daniel Inouye
Chairman
The Honorable Ted Stevens
Ranking Member
Committee on Appropriations
United States Senate
The Honorable Ike Skelton
Chairman
The Honorable Duncan Hunter
Ranking Member
Committee on Armed Services
House of Representatives
The Honorable John P. Murtha
Chairman
The Honorable C.W. Bill Young
Ranking Member
Committee on Appropriations
House of Representatives
Appendix I: Objectives, Scope, and Methodology
Our objectives were to (1) assess the actions by the Department of
Defense (DOD) to comply with the requirements of section 2222 of
Title 10, U.S. Code,^1 and (2) determine the extent to which DOD
has addressed our prior open recommendations for
institutionalizing key business system modernization management
controls.
For our first objective, we focused on five of the six
requirements in section 2222, and related best practices contained
in federal guidance, that we identified in our last annual report
under the act as not being fully satisfied.^2 Generally, these
five requirements are (1) development of a business enterprise
architecture (BEA), (2) development of a transition plan for
implementing the BEA, (3) inclusion of business systems
information in DOD's budget submission, (4) establishment of
business systems investment review processes and structures, and
(5) approval of defense business systems investments with
obligations in excess of $1 million. (See the Background section
of this report for additional information on the act's
requirements.) We did not include the sixth requirement because
our last annual report under the act shows that it had been
satisfied. Our methodology relative to each of the five
requirements is as follows.
o To determine whether the BEA addressed the requirements
specified in the act, and related guidance, we analyzed version
4.1 of the BEA, which was released on March 15, 2007, relative to
the act's specific architectural requirements and related guidance
that our last annual report under the act identified as not being
met. We also reviewed version 4.1 to confirm whether statements
made in DOD's March 15, 2007, annual report about the BEA's
content were accurate. Also, we reviewed and leveraged the
applicable results contained in our recent reports on major
departments' and agencies' enterprise architecture programs and on
DOD's BEA federation strategy.^3
o To determine whether the enterprise transition plan (ETP)
addressed the requirements specified in the act, we reviewed the
updated version of the ETP, which was released on March 15, 2007,
relative to the act's specific transition plan requirements and
related guidance that our last annual report under the act
identified as not being met. We also reviewed the ETP to confirm
that statements in DOD's March 15, 2007, annual report about the
content of the ETP were accurate.
^1Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 2004)
(codified in part at 10 U.S.C. S 2222).
^2GAO, Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, GAO-06-658 (Washington,
D.C.: May 15, 2006).
^3GAO, Business Systems Modernization: Strategy for Evolving DOD's
Business Enterprise Architecture Offers Conceptual Approach, but Execution
Details Needed, GAO-07-451 (Washington, D.C.: Apr. 16, 2007); and
Enterprise Architecture: Leadership Remains Key to Establishing and
Leveraging Architectures for Organizational Transformation, GAO-06-831
(Washington, D.C.: Aug. 14, 2006).
o To determine whether DOD's fiscal year 2008 information
technology budget submission was prepared in accordance with the
criteria set forth in the act, we reviewed and analyzed the
department report entitled Report on Defense Business System
Modernization FY 2005 National Defense Authorization Act, Section
332, prepared in February 2007 and compared the information
obtained to the specific requirements in the act.
o To determine whether DOD has established investment review
structures and processes, we focused the act's requirements that
our last annual report under the act identified as not being met,
obtaining documentation and interviewing cognizant DOD officials
about efforts to establish the one Investment Review Board (IRB)
specified in the act that had yet to be established. We also
reviewed and leveraged our recent report that assessed DOD's
corporate investment approach to managing business system
investments against relevant federal guidance.^4
o To determine whether the department was reviewing and approving
business system investments exceeding $1 million, we obtained the
list of business system investments certified by the IRBs and
approved by the Defense Business Systems Management Committee from
the Business Transformation Agency (BTA). We then compared the
detailed information provided with the summary information
contained in the department's March 15, 2007, report to the
congressional defense committees to identify any anomalies. We
also met with representatives from the Air Force, the Army, and
the Navy to ascertain the specific actions that were taken (or
planned to be taken) in order to perform the annual systems
reviews as required by the act.
To determine the extent to which DOD has addressed our prior open
recommendations, we focused on the 14 recommendations that we
identified in our last annual report under the act as not being
implemented. We did not examine the recommendations for
establishing and implementing key business system modernization
management controls that we made since this last annual report
because sufficient time had yet to elapse for the department to
have addressed them. (See app. III for a list of the
recommendations made since our last annual report under the act.)
In reviewing the 14 recommendations, we obtained and analyzed
documentation relative to corrective actions taken and planned.
Documentation that we reviewed included the DOD's March 15, 2007,
annual report, updated transition plan, and BEA version 4.1. We
also compared a range of other program documentation, such as
program policies and procedures and configuration plan, to
relevant elements in our Enterprise Architecture Management
Maturity Framework.^5 Further, we reviewed documentation regarding
DOD verification and validation contractor activities and the
BTA's human capital strategy. In addition, we reviewed the
guidance establishing the IRBs and describing the investment
review, certification, and approval process.
^4GAO, Business Systems Modernization: DOD Needs to Fully Define Policies
and Procedures for Institutionally Managing Investments, GAO-07-538
(Washington, D.C.: May 11, 2007).
^5GAO, Information Technology: A Framework for Assessing and Improving
Enterprise Architecture Management (Version 1.1), GAO-03-584G
(Washington, D.C.: April 2003).
We did not independently validate the reliability of the cost and
budget figures provided by DOD because the specific amounts were
not relevant to our findings. We conducted our work at DOD
headquarters in Arlington, Virginia, from March through May 2007
in accordance with generally accepted government auditing
standards.
Appendix II: Status of Prior Recommendations Identified as Open inGAO's
Prior Annual Report under the Act
Implemented/
Closed
GAO report information
and recommendation Yes In process GAO assessment
[43]GAO-01-525 :
Information Technology:
Architecture Needed to
Guide Modernization of
DOD's Financial
Operations, May 17,
2001.
(1) Until an enterprise X This recommendation has been
architecture is subsumed by more recent
developed and the recommendations concerning the
Council is positioned to department's efforts to federate
serve as Department of the corporate business
Defense's (DOD) enterprise architecture (BEA),
financial management mature DOD component
investment review board organization architectures, and
as recommended, the establish policies and
Secretary of Defense procedures for effective
limit DOD components' corporate business system
financial management investment management. (See app.
investments to the III for these more recent
deployment of systems recommendations.)
that have already been
fully tested and involve
no additional
development or
acquisition costs;
stay-in-business
maintenance needed to
keep existing systems
operational; management
controls needed to
effectively invest in
modernized systems; and
new systems or existing
system changes that are
congressionally directed
or are relatively small,
cost-effective, and low
risk and can be
delivered in a
relatively short time
frame.
[44]GAO-03-458 : DOD
Business Systems
Modernization:
Improvements to
Enterprise Architecture
Development and
Implementation Efforts
Needed, February 28,
2003.
(1) The Secretary of X The Business Transformation
Defense ensure that the Agency (BTA) has established a
enterprise architecture communications team that is
program is supported by responsible for achieving
a proactive marketing strategic communications
and communication objectives and promoting
program. external awareness of the
department's vision, mission,
and progress. However, the
department has yet to develop a
communication plan that adheres
to criteria set forth by the
best practices, to include an
explanation of roles and
responsibilities and details
regarding evaluation, metrics,
and feedback. BTA officials told
us that such a plan is currently
in development.
[45]GAO-03-1018 : DOD
Business Systems
Modernization: Important
Progress Made to Develop
Business Enterprise
Architecture, but Much
Work Remains, September
19, 2003.
(1) The Secretary of X The BTA has largely addressed
Defense or his the 31 core elements in our
appropriate designee Enterprise Architecture
implement the core Management Maturity Framework in
elements in our its corporate BEA, which is the
Enterprise Architecture intended focus of the
Framework for Assessing recommendation. For example, the
and Improving Enterprise BTA has established a chief
Architecture Management architect who is responsible for
that we identify in this developing and maintaining the
report as not satisfied, corporate BEA and the version
including ensuring that 4.1 of the BEA largely provides
minutes of the meetings a depiction of both the "As Is"
of the executive body and "To Be" environments in
charged with directing, terms of business, performance,
overseeing, and information/data,
approving the application/service, technology,
architecture are and security. (See app. III for
prepared and maintained. recent recommendations aimed at
having the military departments
address these core elements.)
(2) The Secretary of X The BTA has largely addressed
Defense or his these 29 key elements relative
appropriate designee to its corporate BEA, which is
update version 1.0 of the intended focus of the
the architecture to recommendation. For example,
include the 29 key version 4.1 of the BEA contains
elements governing the enterprise-level "As Is"
"As Is" architectural information to support business
content that our report capability gap analyses. In
identified as not being addition, the architecture
fully satisfied. includes "As Is" information for
five of the six business
enterprise priorities and "As
Is" information for enterprise
systems, such as the Wide-area
Workflow system. (See app. III
for recent recommendations aimed
at effectively federating the
corporate BEA to DOD component
organizations.)
(3) The Secretary of X The BTA has largely addressed
Defense or his these 30 key elements relative
appropriate designee to its corporate BEA, which is
update version 1.0 of the intended focus of the
the architecture to recommendation. For example,
include the 30 key version 4.1 of the BEA
elements governing the identifies activities performed
"To Be" architectural at each location/organization
content that our report and indicates which
identified as not being organization(s) is or will be
fully satisfied. involved in each activity.
Furthermore, it includes common
business rules (e.g., "each
request for commercial export of
DOD technology must be processed
within 30 days upon receipt of
request from the Department of
State or the Department of
Commerce") to facilitate
consistent implementation of the
architecture. (See app. III for
recent recommendations aimed at
effectively federating the
corporate BEA to DOD component
organizations.)
(4) The Secretary of X The BTA has largely addressed
Defense or his this recommendation for its
appropriate designee corporate or enterprise
update version 1.0 of transition plan, which is the
the architecture to intended focus of the
include (a) the 3 key recommendation. For example, the
elements governing the latest version of the transition
transition plan content plan now documents how BEA
that our report elements (e.g., specific
identified as not being business capability
fully satisfied and (b) improvements) provide solutions
those system investments to significant DOD issues or
that will not become business capability gaps (e.g.,
part of the "To Be" mission needs, materiel
architecture, including weaknesses). It also provides
time frames for phasing performance information of DOD
out those systems. transformation at both the
enterprise level and component
level, including performance
metrics and milestones. (See
app. III for recent
recommendations aimed at
effectively federating the
corporate BEA, to include the
transition plan, to DOD
component organizations.)
(5) The Secretary of X The verification and validation
Defense or his contractor reports that all of
appropriate designee these comments on versions 3.0
update version 1.0 of and prior versions have been
the architecture to addressed.
address comments made by
the verification and
validation contractor.
(6) The Secretary of X This recommendation has been
Defense or his subsumed by a later
appropriate designee recommendation in [46]GAO-06-658
develop a well-defined, .
near-term plan for
extending and evolving
the architecture and
ensure that this plan
includes addressing our
recommendations,
defining roles and
responsibilities of all
stakeholders involved in
extending and evolving
the architecture,
explaining dependencies
among planned
activities, and defining
measures of activity
progress.
(7) The Secretary of X According to BTA officials, the
Defense or his department is continuing to
appropriate designee assess and clarify the role of
limit the pilot projects pilot projects and a policy is
to small, low-cost, to be developed relative to
low-risk prototype them. However, they did not
investments that are provide specific plans and time
intended to provide frames for developing and
knowledge needed to implementing this policy.
extend and evolve the
architecture, and are
not to acquire and
implement production
version system solutions
or to deploy an
operational system
capability.
[47]GAO-05-381 : DOD
Business Systems
Modernization: Billions
Being Invested without
Adequate Oversight,
April 29, 2005.
(1) The Secretary of X DOD's March 15, 2007, annual
Defense direct that the report to the congressional
DBSMC develop a defense committees identifies
comprehensive plan that specific actions the department
addresses implementation is taking to address our open
of our previous recommendations. The March
recommendations related report noted that BTA has
to the BEA and the overall responsibility for
control and ensuring that remaining open
accountability over recommendations are adequately
business systems addressed.
investments (at a
minimum, the plan should
assign responsibility
and estimated time
frames for completion).
(2) The Secretary of X DOD's March 15, 2006, and March
Defense direct that the 15, 2007, reports to
comprehensive plan we congressional committees
recommend be included steps that DOD is
incorporated into the taking or plans to take to
department's second address our open
annual report due March recommendations.
15, 2006, to the defense
congressional
committees, as required
by the Fiscal Year 2005
Defense Authorization
act to help facilitate
congressional oversight.
[48]GAO-05-702 : DOD
Business Systems
Modernization:
Long-standing Weaknesses
in Enterprise
Architecture Development
Need to Be Addressed,
July 22, 2005.
(1) The Secretary of X BTA and BEA program
Defense should direct documentation reflects
the Deputy Secretary of activities and steps taken or
Defense, as the chair of planned to address our
the DBSMC and in recommendations relative to BEA
collaboration with DBSMC content and management.
members, to ensure that Furthermore, the department has
each of our stated its commitment to doing
recommendations related so in its annual reports to the
to the BEA management congressional defense
and content are committees.
reflected in the plans
and commitments.
(2) The Secretary of X On March 21, 2007, the BTA
Defense should direct released its Human Capital
the Deputy Secretary of Strategic Plan 2007-2009, which
Defense, as the chair of identifies BTA's goals for human
the DBSMC and in capital development and
collaboration with DBSMC workforce planning. This
members, to ensure that strategy provides an overview of
plans and commitments the current workforce status in
provide for effective relation to those goals and
BEA workforce planning, identifies several key
including assessing activities for how to proceed in
workforce knowledge and order to achieve the goals. In
skills needs, addition, the strategy includes
determining existing an initial implementation
workforce capabilities, roadmap with timelines for key
identifying gaps, and activities. According to BTA
filling these gaps. officials, the detailed plans
for accomplishing key activities
will be contained in BTA's Human
Capital Implementation Plan,
which has yet to be released.
[49]GAO-06-658 :
Business Systems
Modernization: DOD
Continues to Improve
Institutional Approach,
but Further Steps
Needed, May 15, 2006.
(1) The Secretary of X BTA has developed several
Defense direct the documents that are intended to
Deputy Secretary of begin addressing this
Defense, as the chair of recommendation. For example, it
the DBSMC, to submit an has developed the Business
enterprise architecture Transformation Guidance, which
program management plan describes the high-level process
to defense congressional by which incremental
committees that defines improvements are identified and
what the department's eventually incorporated into the
incremental improvements BEA. In addition, BTA officials
to the architecture and told us that they are developing
transition plan will be, a BEA Concept of Operations,
and how and when they which is to describe high-level
will be accomplished, milestones required to address
including what (and the architecture's use (e.g.,
when) architecture and investment management, strategic
transition plan scope decision making, oversight,
and content and system implementation, and
architecture compliance business case development).
criteria will be added Notwithstanding these steps, the
into which versions; the department has yet to develop an
plan should also include architecture program management
an explicit purpose and plan that we have recommended.
scope for each version (See app. III for a more recent
of the architecture, recommendation that augments
along with milestones, this recommendation.)
resource needs, and
performance measures for
each planned version.
Source: GAO.
Note: See GAO, Business Systems Modernization: DOD Continues to
Improve Institutional Approach, but Further Steps Needed,
GAO-06-658 (Washington, D.C.: May 15, 2006).
Appendix III: Other Open Recommendations on Business Architectures,
Federation Strategy, and Investment Management
GAO report information and recommendation
[50]GAO-06-831 : Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation, August 14, 2006.
1. The Secretary of Defense ensure that the Department of Defense (DOD)
- Global Information Grid enterprise architecture program develops and
implements plans for fully satisfying each of the conditions in our
enterprise architecture management maturity framework.
2. The Secretary of Defense ensure that the Department of the Air Force
enterprise architecture program develops and implements plans for fully
satisfying each of the conditions in our enterprise architecture
management maturity framework.
3. The Secretary of Defense ensure that the Department of the Army
enterprise architecture program develops and implements plans for fully
satisfying each of the conditions in our enterprise architecture
management maturity framework.
4. The Secretary of Defense ensure that the Department of the Navy
enterprise architecture program develops and implements plans for fully
satisfying each of the conditions in our enterprise architecture
management maturity framework.
[51]GAO-07-451 : Business Systems Modernization: Strategy for Evolving
DOD's Business Enterprise Architecture Offers a Conceptual Approach, but
Execution Details Are Needed, April 16, 2007.
1. The Secretary of Defense direct the Deputy Secretary of Defense, as
the chair of the Defense Business Systems Management Committee (DBSMC),
to ensure that the appropriate DOD organizations submit a business
enterprise architecture (BEA) development management plan that
describes, at a minimum, how the business mission area architecture
federation will be governed; how the business mission area federation
strategy alignment with the DOD enterprise architecture federation
strategy will be achieved; how component business architectures'
alignment with incremental versions of the BEA will be achieved; how
shared services will be identified, exposed, and subscribed to; and
what milestones will be used to measure progress and results.
[52]GAO-07-538 : Business Systems Modernization: DOD Needs to Fully Define
Policies and Procedures for Institutionally Managing Investments, May 11,
2007.
1. The Secretary of Defense should direct the Deputy Secretary of
Defense, as the chair of the DBSMC, to ensure that well-defined and
disciplined business system investment management policies and
procedures are developed and issued. At a minimum, this should include
project-level management policies and procedures that address the
following five areas:
o instituting the investment boards, including assigning the
investment boards responsibility, authority, and accountability
for programs throughout the investment life cycle and
specifying how the business investment management system is
coordinated with the Joint Capabilities Integration and
Development System, the Planning, Programming, Budgeting, and
Execution system, and the Defense Acquisition System;
o selecting new investments, including specifying how cost,
schedule, and benefit data are to be used in making
certification decisions; defining the criteria used to select
investments as enterprisewide; and establishing consistent and
effective guidance for BEA compliance;
o reselecting ongoing investments, including specifying how
cost, schedule, and performance data are to be used in the
annual review process and providing for the reselection of
investments that are in operations and maintenance;
o integrating funding with the process of selecting an
investment, including specifying how the DBSMC and the
investment review boards use funding information in carrying
out decisions on system certification and approvals; and
o overseeing IT projects and systems, including providing
sufficient oversight and visibility into component-level
investment management activities.
2. The Secretary of Defense should direct the Deputy Secretary of
Defense, as the chair of the DBSMC, to ensure that well-defined and
disciplined business system investment management policies and
procedures are developed and issued. These policies and procedures
should also include portfolio-level management policies and procedures
that address the following four areas:
o creating and modifying information technology portfolio
selection criteria for business system investments;
o analyzing, selecting, and maintaining business system
investment portfolios;
o reviewing, evaluating, and improving the performance of its
portfolio(s) by using project indicators such as cost,
schedule, and risk; and
o conducting postimplementation reviews for all investment
tiers and directing the investment boards who are accountable
for corporate business system investments, to consider the
information gathered and to develop lessons learned from these
reviews.
Source: GAO.
Appendix IV: Comments from the Department of Defense
Appendix V: GAO Contacts and Staff Acknowledgments
GAO Contacts
Randolph C. Hite (202) 512-3439 or [email protected] McCoy
Williams (202) 512-9095 or [email protected]
Staff Acknowledgments
In addition to the contact persons named above, key contributors
to this report were Beatrice Alff, Karl Essig, Nancy Glover,
Michael Holland, Neelaxi Lakhmani (Assistant Director), Anh Le,
Evelyn Logue, Jacqueline Mai, John Martin, Darby Smith (Assistant
Director), Debra Rucker, and Jennifer Stavros-Turner.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of
newly posted products every afternoon, go to www.gao.gov and
select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW,
Room 7125 Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
(310643)
www.gao.gov/cgi-bin/getrpt?GAO-07-733 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Randolph C. Hite at (202) 512-3439 or
[email protected].
Highlights of [54]GAO-07-733 , a report to congressional committees
May 2007
DOD BUSINESS SYSTEMS MODERNIZATION
Progress Continues to Be Made in Establishing Corporate Management
Controls, but Further Steps Are Needed
In 1995, GAO first designated the Department of Defense's (DOD) business
systems modernization program as "high risk," and GAO continues to do so
today. To assist in addressing this high-risk area, the Fiscal Year 2005
National Defense Authorization Act contains provisions that are consistent
with prior GAO recommendations. Further, the act requires the department
to submit annual reports to its congressional committees on its compliance
with these provisions and it directs GAO to review each report. In
response, GAO assessed DOD's actions to address (1) requirements in the
act and (2) GAO's recommendations that it reported as open in its prior
annual report under the act. In doing so, GAO reviewed documentation and
interviewed officials relative to the act and related guidance.
[55]What GAO Recommends
GAO is recommending that future DOD annual reports include an assessment
by its independent verification and validation agent of the quality of the
department's federated family of architectures, including the associated
transition plan(s). In written comments, DOD agreed with GAO's
recommendation.
As part of DOD's recent efforts to strengthen management of its business
systems modernization program, it has taken steps over the last year to
build on past efforts and further comply with the act's requirements and
relevant guidance. However, additional steps are needed. For example,
o The latest version of DOD's business enterprise architecture now
contains information about the department's "As Is" corporate
environment, which is important for effective transition planning.
Further, this version represents a major step in building the
family of architectures that are needed to fully satisfy the act
and effectively guide and constrain thousands of system
investments across all DOD component organizations. Nevertheless,
GAO's reports since its last annual report under the act show that
the strategy for extending the business enterprise architecture to
defense components needs further definition to make it executable
and the maturity of key components' architecture programs is
limited. GAO has recently made recommendations to address these
challenges.
o The updated enterprise transition plan, which is an essential
component of an enterprise architecture, continues to identify
systems and initiatives that are to fill business capability gaps
and address DOD-wide and component business priorities contained
in the business enterprise architecture. However, it does not
include investments for all components and does not reflect key
factors associated with properly sequencing planned investments,
such as dependencies among investments and the capability to
execute the plan, which GAO's existing recommendations provide for
addressing.
o DOD has established and begun implementing the investment review
structures and processes that are consistent with the act.
However, it has yet to do so in a manner that is consistent with
relevant guidance. In particular, it has yet to fully define the
related policies and procedures needed to effectively execute both
project-level and portfolio-based information technology
investment management practices. GAO has recently made
recommendations to address these shortcomings.
DOD also continues to make progress in implementing GAO recommendations
aimed at strengthening business systems modernization management. In
particular, of the 14 open recommendations that GAO identified in its
prior annual report under the act, 10 have either been largely implemented
or subsumed by the more recent recommendations cited above. For example,
DOD has implemented GAO's recommendations aimed at effectively using the
assessments that have been performed by DOD's independent verification and
validation contractor. Such assessments provide important information for
department and congressional oversight bodies to use to better ensure the
definition and institutionalization of the corporate management controls
that GAO has cited as essential to addressing the DOD business systems
modernization high-risk area. The department's annual reports have not
included such assessments.
References
Visible links
33. http://www.gao.gov/cgi-bin/getrpt?GAO-07-733
43. http://www.gao.gov/cgi-bin/getrpt?GAO-01-525
44. http://www.gao.gov/cgi-bin/getrpt?GAO-03-458
45. http://www.gao.gov/cgi-bin/getrpt?GAO-03-1018
46. http://www.gao.gov/cgi-bin/getrpt?GAO-06-658
47. http://www.gao.gov/cgi-bin/getrpt?GAO-05-381
48. http://www.gao.gov/cgi-bin/getrpt?GAO-05-702
49. http://www.gao.gov/cgi-bin/getrpt?GAO-06-658
50. http://www.gao.gov/cgi-bin/getrpt?GAO-06-831
51. http://www.gao.gov/cgi-bin/getrpt?GAO-07-451
52. http://www.gao.gov/cgi-bin/getrpt?GAO-07-538
54. http://www.gao.gov/cgi-bin/getrpt?GAO-07-733
*** End of document. ***