DOD Business Systems Modernization: Progress Continues to Be Made
in Establishing Corporate Management Controls, but Further Steps 
Are Needed (14-MAY-07, GAO-07-733).				 
                                                                 
In 1995, GAO first designated the Department of Defense's (DOD)  
business systems modernization program as "high risk," and GAO	 
continues to do so today. To assist in addressing this high-risk 
area, the Fiscal Year 2005 National Defense Authorization Act	 
contains provisions that are consistent with prior GAO		 
recommendations. Further, the act requires the department to	 
submit annual reports to its congressional committees on its	 
compliance with these provisions and it directs GAO to review	 
each report. In response, GAO assessed DOD's actions to address  
(1) requirements in the act and (2) GAO's recommendations that it
reported as open in its prior annual report under the act. In	 
doing so, GAO reviewed documentation and interviewed officials	 
relative to the act and related guidance.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-733 					        
    ACCNO:   A69585						        
  TITLE:     DOD Business Systems Modernization: Progress Continues to
Be Made in Establishing Corporate Management Controls, but	 
Further Steps Are Needed					 
     DATE:   05/14/2007 
  SUBJECT:   Accountability					 
	     Enterprise architecture				 
	     Information technology				 
	     IT investment management				 
	     Policy evaluation					 
	     Reporting requirements				 
	     Strategic information systems planning		 
	     Strategic planning 				 
	     Systems conversions				 
	     GAO High Risk Series				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-733

   

     * [1]Results in Brief
     * [2]Background

          * [3]Enterprise Architecture and IT Investment Management Control

               * [4]Enterprise Architecture: A Brief Description
               * [5]IT Investment Management: A Brief Description

          * [6]DOD's Institutional Approach to Business Systems Modernizati

               * [7]Tiered Accountability

          * [8]Summary of Fiscal Year 2005 National Defense Authorization A
          * [9]Summary of Recent GAO Reviews of DOD's Business Systems Mode

     * [10]DOD Is Continuing to Improve Its Approach to Modernizing Bus

          * [11]DOD Continues to Improve Its Corporate BEA, but Component Ar
          * [12]DOD Continues to Expand and Update Its Enterprise Transition
          * [13]DOD's Fiscal Year 2008 Budget Submission Includes Key Inform
          * [14]DOD Has Largely Established Key Investment Management Struct
          * [15]Investment Management Structures Have Been Largely Establish
          * [16]Policies and Procedures Have Been Defined for Some, but Not
          * [17]DOD Continues to Approve and Review Business Systems, but Mi

               * [18]Air Force
               * [19]Army
               * [20]Navy

     * [21]DOD Continues to Implement Our Prior Recommendations
     * [22]Conclusions
     * [23]Recommendation for Executive Action
     * [24]Agency Comments
     * [25]Appendix I: Objectives, Scope, and Methodology
     * [26]Appendix II: Status of Prior Recommendations Identified as O
     * [27]Appendix III: Other Open Recommendations on Business Archite
     * [28]Appendix IV: Comments from the Department of Defense
     * [29]Appendix V: GAO Contacts and Staff Acknowledgments

          * [30]GAO Contacts
          * [31]Staff Acknowledgments

               * [32]Order by Mail or Phone

Report to Congressional Committees

United States Government Accountability Office

GAO

May 2007

DOD BUSINESS SYSTEMS MODERNIZATION

Progress Continues to Be Made in Establishing Corporate Management
Controls, but Further Steps Are Needed

[33]GAO-07-733

Contents

Letter 1

Results in Brief 3
Background 6
DOD Is Continuing to Improve Its Approach to Modernizing Business Systems
20
DOD Continues to Implement Our Prior Recommendations 38
Conclusions 39
Recommendation for Executive Action 40
Agency Comments 41
Appendix I Objectives, Scope, and Methodology 44
Appendix II Status of Prior Recommendations Identified as Open in GAO's
Prior Annual Report under the Act 47
Appendix III Other Open Recommendations on Business Architectures,
Federation Strategy, and Investment Management 52
Appendix IV Comments from the Department of Defense 54
Appendix V GAO Contacts and Staff Acknowledgments 56

Table

Table 1: DOD Business Systems Modernization Governance Entities' Roles,
Responsibilities, and Composition 15

Figures

Figure 1: Simplified DOD Organizational Structure 7
Figure 2: The Five ITIM Stages of Maturity with Critical Processes 13
Figure 3: Simplified Diagram of DOD's Business Mission Area Federated
Architecture 25

Abbreviations

ASD(NII)/CIO Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer
BEA business enterprise architecture
BEP business enterprise priority
BTA Business Transformation Agency
CIO chief information officer
DBSMC Defense Business Systems Management Committee
DOD Department of Defense
ETP enterprise transition plan
IRB Investment Review Board
IT information technology
ITIM Information Technology Investment Management framework
NCES Net-Centric Enterprise Services
OMB Office of Management and Budget
SOA service-oriented architecture
USD(AT&L) Under Secretary of Defense (Acquisition, Technology, and Logistics)

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office
Washington, DC 20548

May 14, 2007 

Congressional Committees

For decades, the Department of Defense (DOD) has been challenged in
modernizing its timeworn business systems.^1 In 1995, we designated DOD's
business systems modernization program as high risk, and we continue to
designate it as such today.^2 As our research on public and private sector
organizations shows, two essential ingredients to a successful systems
modernization program are having a well-defined enterprise
architecture^3123 and an effective institutional approach to managing
information technology (IT) investments. For decades, the Department of
Defense (DOD) has been challenged in modernizing its timeworn business
systems. In 1995, we designated DOD's business systems modernization
program as high risk, and we continue to designate it as such today. As
our research on public and private sector organizations shows, two
essential ingredients to a successful systems modernization program are
having a well-defined enterprise architecture and an effective
institutional approach to managing information technology (IT)
investments.

Accordingly, we made recommendations to the Secretary of Defense in May
2001 that included the means for effectively developing an enterprise
architecture and establishing a corporate approach to investment control
and decision making.^4 Between 2001 and 2005, we reported that the
department's business systems modernization program continued to lack both
of these, concluding in 2005 that hundreds of millions of dollars had been
spent on a business enterprise architecture (BEA) and investment
Accordingly, we made recommendations to the Secretary of Defense in May
2001 that included the means for effectively developing an enterprise
architecture and establishing a corporate approach to investment control
and decision making.^45 Between 2001 and 2005, we reported that the
department's business systems modernization program continued to lack both
of these, concluding in 2005 that hundreds of millions of dollars had been
spent on a business enterprise architecture (BEA) and investment
management structures that had limited use. Accordingly, we made more
explicit architecture and investment-related recommendations.

^1Business systems support DOD's business operations, such as civilian
personnel, finance, health, logistics, military personnel, procurement,
and transportation.

^2GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: January
2007).

^3An enterprise architecture, or modernization blueprint, provides a clear
and comprehensive picture of an entity, whether it is an organization
(e.g., federal department or agency) or a functional or mission area that
cuts across more than one organization (e.g., financial management). This
picture consists of snapshots of the enterprise's current "As Is"
operational and technological environment and its target or "To Be"
environment, and contains a capital investment road map for transitioning
from the current to the target environment. These snapshots consist of
"views," which are basically one or more architecture products that
provide conceptual or logical representations of the enterprise.

^4GAO, Information Technology: Architecture Needed to Guide Modernization
of DOD's Financial Operations, GAO-01-525 (Washington, D.C.: May 17,
2001).

To assist DOD in addressing these modernization management challenges,
Congress included provisions in the Ronald W. Reagan National Defense
Authorization Act for Fiscal Year 2005^6 that were consistent with our
recommendations. More specifically, the act required the department to,
among other things, (1) develop a BEA, (2) develop a transition plan to
implement the architecture, (3) include systems information in its annual
budget submission, (4) establish a system investment approval and
accountability structure, (5) establish an investment review process, and
(6) approve and certify any system modernizations costing in excess of $1
million. The act further requires that the Secretary of Defense submit an
annual report to congressional defense committees on DOD's compliance with
certain requirements of the act not later than March 15 of each year from
2005 through 2009. Additionally, the act directs us to submit--within 60
days of DOD's report submission--to congressional defense committees an
assessment of the actions taken to comply with these requirements.

As agreed with your offices, the objectives of our review were to (1)
assess the actions taken by DOD to comply with requirements of section
2222 of Title 10, U.S. Code, and (2) determine the extent DOD has
addressed our prior open recommendations for institutionalizing key
business system modernization management controls. To accomplish this, we
used our prior annual report under the act^7 as a baseline, analyzing
whether the department had taken actions to comply with those provisions
of the act, related guidance, and the prior recommendations that we had
identified in our prior annual report as not yet addressed. In doing this,
we also relied on the results of relevant reports that we have issued
since our prior annual report.^8 We performed our work at DOD headquarters
in Arlington, Virginia, from March through May 2007 in accordance with
generally accepted government auditing standards. Details on our
objectives, scope, and methodology are contained in appendix I.

^5See, for example, GAO, Defense Business Transformation: A Comprehensive
Plan, Integrated Efforts, and Sustained Leadership Are Needed to Assure
Success,  GAO-07-229T (Washington, D.C.: Nov. 16, 2006); Business Systems
Modernization: DOD Continues to Improve Institutional Approach, but
Further Steps Needed, GAO-06-658 (Washington, D.C.: May 15, 2006); DOD
Business Systems Modernization: Long-standing Weaknesses in Enterprise
Architecture Development Need to Be Addressed, GAO-05-702 (Washington,
D.C.: July 22, 2005); DOD Business Systems Modernization: Billions Being
Invested without Adequate Oversight, GAO-05-381 (Washington, D.C.: Apr.
29, 2005); DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments,  GAO-04-731R (Washington, D.C.: May
17, 2004); DOD Business Systems Modernization: Important Progress Made to
Develop Business Enterprise Architecture, but Much Work Remains,
GAO-03-1018 (Washington, D.C.: Sept. 19, 2003); Business Systems
Modernization: Summary of GAO's Assessment of the Department of Defense's
Initial Business Enterprise Architecture, GAO-03-877R (Washington, D.C.:
July 7, 2003); Information Technology: Observations on Department of
Defense's Draft Enterprise Architecture, GAO-03-571R (Washington, D.C.:
Mar. 28, 2003); DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed,
GAO-03-458 (Washington, D.C.: Feb. 28, 2003); and GAO-01-525.

^6Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. S 2222).

Results in Brief

DOD continues to take steps to comply with legislative requirements and
related guidance pertaining to its business systems modernization high
risk area. In particular, on March 15, 2007, DOD released a new version of
its BEA, developed an updated enterprise transition plan, and issued its
annual report to Congress describing steps taken and planned relative to
the act's requirements, among other things. The steps address several of
the missing elements that we previously identified relative to the
legislative provisions and related best practices concerning the BEA,
enterprise transition plan, budgetary disclosure, investment management,
and reviews of systems costing in excess of $1 million. However,
additional steps are needed to fully comply with the act and relevant
guidance. For example:

           o The latest version of the BEA now contains information about the
           department's "As Is" corporate environment for some enterprise
           priority areas (e.g., Financial Visibility), which is important to
           support the business capability gap analyses needed for transition
           planning; however, it does not do this for all priority areas
           (e.g., Acquisition Visibility). Moreover, while the latest
           version's focus on DOD-wide, corporate policies, capabilities,
           rules, and standards is an essential element to meeting the act's
           requirements, this version has yet to be augmented by the DOD
           component organizations' subsidiary architectures that are also
           essential to meeting the act's requirements and the department's
           goal of having a federated family of architectures. Compounding
           this are our recent reports showing the military departments'
           architecture programs are not mature and the strategy that the
           department has developed for federating its BEA needs more
           definition to be executable.^9 To address these limitations, our
           recent reports contain additional recommendations. Once these
           limitations are addressed, the architecture should provide a more
           sufficient frame of reference to optimally guide and constrain
           DOD-wide system investments.
		   
^7GAO-06-658.

^8GAO, Business Systems Modernization: DOD Needs to Fully Define Policies
and Procedures for Institutionally Managing Investments, GAO-07-538
(Washington, D.C.: May 11, 2007); and Business Systems Modernization:
Strategy for Evolving DOD's Business Enterprise Architecture Offers
Conceptual Approach but Execution Details Needed, GAO-07-451 (Washington,
D.C.: Apr. 16, 2007).

^9GAO-07-451 and Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006).


           o The updated transition plan continues to identify more systems
           and initiatives that are to fill business capability gaps and
           address DOD-wide and component business priorities and continues
           to provide a range of information for each system and initiative
           in the plan (e.g., budget information, performance metrics, and
           milestones). Further, the updated plan also identifies legacy
           systems that will not be part of its target environment. However,
           this latest transition plan still does not include system
           investment information for all the defense agencies and combatant
           commands. Moreover, the plan does not sequence the planned
           investments based on a range of relevant factors, such as
           technology opportunities, marketplace trends, institutional system
           development and acquisition capabilities, legacy and new system
           dependencies and life expectancies, and the projected value of
           competing investments. According to DOD officials, they intend to
           address such limitations in future versions of the transition
           plan. We have an existing recommendation to the department to
           formalize its plans for incrementally evolving the transition
           plan. Once these limitations in the department's transition
           plan(s) are addressed, it will be better positioned to effectively
           and efficiently migrate to a more modernized systems environment.

           o The department's fiscal year 2008 budget submission provides a
           range of information on business systems, including types of
           information cited in the act, such as system name, designated
           approval authority, and funding to be used for
           development/modernization versus operations/maintenance.

           o While the department has established and begun implementing the
           investment review structures and processes that are consistent
           with the act, it has yet to do so in a manner that is consistent
           with relevant guidance. As we recently reported,^10 the department
           has yet to fully define the related policies and procedures needed
           to effectively execute both project-level and portfolio-based IT
           investment management practices. For example, DOD had established
           an enterprisewide IT investment board responsible for defining and
           implementing its business system investment governance process,
           but it had not fully defined the policies and procedures needed
           for oversight of and visibility into operations and maintenance
           investments. To address these investment management weaknesses,
           our recent report contains additional recommendations. Once these
           policies and procedures are fully defined, the risk of projects
           and portfolios of projects being inconsistently and improperly
           selected and controlled will be reduced, thus increasing the
           chances of investments meeting mission needs in the most
           cost-effective manner.

           o The department continues to review and approve business systems
           as directed by the act. As of March 2007, the department reported
           that its highest investment review body had approved 285 systems.
           However, the military departments' review and approval processes
           are still evolving, according to Air Force, Army, and Navy
           officials, and additional work is needed to mature them. Because
           of the importance of the military departments' investment
           management structures and processes, we have ongoing work to
           determine the extent to which the Air Force and the Navy are
           employing relevant investment management guidance.

           In concert with the department's efforts to comply with the act,
           it has also largely implemented, or our recommendations in recent
           reports have otherwise subsumed, 10 of the 14 recommendations that
           we identified as open in our prior annual report under the act.
           For example, DOD has implemented our recommendation aimed at
           effectively using the results of the BEA independent verification
           and validation contractor on prior versions of the architecture.
           Use of an independent verification and validation agent is an
           architecture management best practice for identifying architecture
           strengths and weaknesses and disclosing to department and
           congressional oversight bodies the information they need to better
           ensure that DOD's family of architectures and associated
           transition plan(s) satisfy key quality parameters. According to
           department officials, they are committed to addressing all of our
           open recommendations, and have actions under way and plans in
           place to address the remaining 4.
		   
^10GAO-07-538.

           To facilitate congressional oversight and promote departmental
           accountability, we are recommending that the department include in
           its future annual reports under the act the results of its
           independent verification and validation agent's assessment of the
           extent to which the department's federated family of its corporate
           and component architectures, including the related transition
           plan(s), are complete, consistent, understandable, and usable. The
           department has not included such information in its annual
           reports. In written comments on a draft of this report, signed by
           the Deputy Under Secretary of Defense (Business Transformation)
           and reprinted in appendix IV, the department agreed with our
           recommendation.
		   
		   Background

           DOD is a massive and complex organization. To illustrate, the
           department reported that its fiscal year 2006 operations involved
           approximately $1.4 trillion in assets and $2.0 trillion in
           liabilities; more than 2.9 million in military and civilian
           personnel; and $581 billion in net cost of operations. To date,
           for fiscal year 2007, the department received appropriations of
           about $501 billion. Organizationally, the department includes the
           Office of the Secretary of Defense, the Chairman of the Joint
           Chiefs of Staff, the military departments, numerous defense
           agencies and field activities; and various unified combatant
           commands that are either responsible for specific geographic
           regions or specific functions. (See fig. 1 for a simplified
           depiction of DOD's organizational structure.)

Figure 1: Simplified DOD Organizational Structure

aThe Chairman of the Joint Chiefs of Staff serves as the spokesman for the
commanders of the combatant commands, especially on the administrative
requirements of the commands.

In support of its military operations, the department performs an
assortment of interrelated and interdependent business functions,
including logistics management, procurement, health care management, and
financial management. As we have previously reported,^11 the DOD systems
environment that supports these business functions is overly complex and
error prone, and is characterized by (1) little standardization across the
department, (2) multiple systems performing the same tasks, (3) the same
data stored in multiple systems, and (4) the need for data to be entered
manually into multiple systems. Moreover, DOD recently reported that this
systems environment is comprised of approximately 3,100 separate business
systems. For fiscal year 2007, Congress appropriated approximately $15.7
billion to DOD, and for fiscal year 2008, DOD has requested about $15.9
billion in appropriated funds to operate, maintain, and modernize these
business systems and associated infrastructure.

^11GAO-06-658.

As we have previously reported,^12 the department's nonintegrated and
duplicative systems impair DOD's ability to combat fraud, waste, and
abuse. In fact, DOD currently bears responsibility, in whole or in part,
for 15 of our 27 high-risk areas.^13 Eight of these areas are specific to
DOD^14 and the department shares responsibility for 7 other governmentwide
high-risk areas.^15 DOD's business systems modernization is one of the
high-risk areas, and it is an essential enabler to addressing many of the
department's other high-risk areas. For example, modernized business
systems are integral to the department's efforts to address its financial,
supply chain, and information security management high-risk areas.

Enterprise Architecture and IT Investment Management Controls Are Critical to
Achieving Successful Systems Modernization

Effective use of an enterprise architecture--a modernization blueprint--is
a hallmark of successful public and private organizations. For more than a
decade, we have promoted the use of architectures to guide and constrain
systems modernization, recognizing them as a crucial means to this
challenging goal: optimally defined operational and technological
environments. Congress, the Office of Management and Budget (OMB), and the
federal Chief Information Officer's (CIO) Council have also recognized the
importance of an architecture-centric approach to modernization. The
Clinger-Cohen Act of 1996^16 mandates that an agency's CIO develop,
maintain, and facilitate the implementation of an information technology
architecture. Further, the E-Government Act of 2002^17 requires OMB to
oversee the development of enterprise architectures within and across
agencies. In addition, we, OMB, and the CIO Council have issued guidance
that emphasizes the need for system investments to be consistent with
these architectures.^18

12See, for example, GAO, DOD Travel Cards: Control Weaknesses Resulted in
Millions of Dollars of Improper Payments,  GAO-04-576 (Washington, D.C.:
June 9, 2004); Military Pay: Army National Guard Personnel Mobilized to
Active Duty Experienced Significant Pay Problems, GAO-04-89 (Washington,
D.C.: Nov. 13, 2003); and Defense Inventory: Opportunities Exist to
Improve Spare Parts Support Aboard Deployed Navy Ships, GAO-03-887
(Washington, D.C.: Aug. 29, 2003).

^13GAO-07-310.

^14These 8 high-risk areas include DOD's overall approach to business
transformation, business systems modernization, financial management, the
personnel security clearance program, supply chain management, support
infrastructure management, weapon systems acquisition, and contract
management.

^15The 7 governmentwide high-risk areas are (1) disability programs, (2)
ensuring the effective protection of technologies critical to U.S.
national security interests, (3) interagency contracting, (4) information
systems and critical infrastructure, (5) information-sharing for homeland
security, (6) human capital, and (7) real property.

^16The Clinger-Cohen Act of 1996, 40 U.S.C. S 11315(b)(2).

^17The E-Government Act of 2002, Pub. L. No. 107-347 (Dec. 17, 2002).

A corporate approach to IT investment management is characteristic of
successful public and private organizations. Recognizing this, Congress
enacted the Clinger-Cohen Act of 1996,^19 which requires OMB to establish
processes to analyze, track, and evaluate the risks and results of major
capital investments in IT systems made by executive agencies.^20 In
response to the Clinger-Cohen Act and other statutes, OMB has developed
policy and issued guidance for planning, budgeting, acquisition, and
management of federal capital assets.^21 We have also issued guidance in
this area,^22 which defines institutional structures, such as Investment
Review Boards (IRB), processes for developing information on investments
(such as costs and benefits), and practices to inform management decisions
(such as whether a given investment is aligned with an enterprise
architecture).

  Enterprise Architecture: A Brief Description

An enterprise architecture provides a clear and comprehensive picture of
an entity, whether it is an organization (e.g., a federal department) or a
functional or mission area that cuts across more than one organization
(e.g., financial management). This picture consists of snapshots of both
the enterprise's current ("As Is") environment and its target ("To Be")
environment. These snapshots consist of "views," which are one or more
interdependent and interrelated architecture products (e.g., models,
diagrams, matrices, and text) that provide logical or technical
representations of the enterprise. The architecture also includes a
transition or sequencing plan, which is based on an analysis of the gaps
between the "As Is" and "To Be" environments; this plan provides a
temporal road map for moving between the two environments and incorporates
such considerations as technology opportunities, marketplace trends,
fiscal and budgetary constraints, institutional system development and
acquisition capabilities, legacy and new system dependencies and life
expectancies, and the projected value of competing investments.

^18GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity,  GAO-04-394G (Washington, D.C.:
March 2004); OMB, Capital Programming Guide, Version 1.0 (July 1997); and
CIO Council, A Practical Guide to Federal Enterprise Architecture, Version
1.0 (February 2001).

^19The Clinger-Cohen Act of 1996, 40 U.S.C. SS 11101-11704. This act
expanded the responsibilities of OMB and the agencies that had been set
under the Paperwork Reduction Act with regard to IT management. See 44
U.S.C. 3504(a)(1)(B)(vi) (OMB); and 44 U.S.C. 3506(h)(5) (agencies).

^20We have made recommendations to improve OMB's process for monitoring
high-risk IT investments; see GAO, Information Technology: OMB Can Make
More Effective Use of Its Investment Reviews, GAO-05-276 (Washington,
D.C.: Apr. 15, 2005).

^21This policy is set forth and guidance is provided in OMB Circular No.
A-11 (Nov. 2, 2005) (section 300) and in OMB's Capital Programming Guide,
which directs agencies to develop, implement, and use a capital
programming process to build their capital asset portfolios.

^22See, for example, GAO-04-394G; Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003); and Assessing Risks and
Returns: A Guide for Evaluating Federal Agencies' IT Investment
Decision-making, GAO/AIMD-10.1.13 (Washington, D.C.: February 1997).

The suite of products produced for a given entity's enterprise
architecture, including its structure and content, is largely governed by
the framework used to develop the architecture. Since the 1980s, various
architecture frameworks have been developed, such as John A. Zachman's "A
Framework for Information Systems Architecture"^23 and the DOD
Architecture Framework.^24

The importance of developing, implementing, and maintaining an enterprise
architecture is a basic tenet of both organizational transformation and
systems modernization. Managed properly, an enterprise architecture can
clarify and help optimize the interdependencies and relationships among an
organization's business operations (and the underlying IT infrastructure
and applications) that support these operations. Moreover, when an
enterprise architecture is employed in concert with other important
management controls, such as portfolio-based capital planning and
investment control practices, architectures can greatly increase the
chances that an organization's operational and IT environments will be
configured to optimize mission performance. Our experience with federal
agencies has shown that investing in IT without defining these investments
in the context of an architecture often results in systems that are
duplicative, not well integrated, and unnecessarily costly to maintain and
interface.^25

23J.A. Zachman, "A Framework for Information Systems Architecture," IBM
Systems Journal 26, no. 3 (1987).

^24DOD, Department of Defense Architecture Framework, Version 1.0, Volume
1 (August 2003) and Volume 2 (February 2004).

One approach to structuring an enterprise architecture is referred to as a
federated enterprise architecture. Such a structure treats the
architecture as a family of coherent but distinct member architectures
that conform to an overarching architectural view and rule set. This
approach recognizes that each member of the federation has unique goals
and needs as well as common roles and responsibilities with the levels
above and below it. Under a federated approach, member architectures are
substantially autonomous, although they also inherit certain rules,
policies, procedures, and services from higher-level architectures. As
such, a federated architecture enables component organization autonomy
while ensuring enterprisewide linkages and alignment where appropriate.
Where commonality among components exists, there are also opportunities
for identifying and leveraging shared services.

A service-oriented architecture (SOA) is an approach for sharing business
capabilities across the enterprise by designing functions and applications
as discrete, reusable, and business-oriented services. As such, service
orientation permits sharing capabilities that may be under the control of
different component organizations. As we have previously reported,^26 such
capabilities or services need to be, among other things, (1)
self-contained, meaning that they do not depend on any other functions or
applications to execute a discrete unit of work; (2) published and exposed
as self-describing business capabilities that can be accessed and used;
and (3) subscribed to via well-defined and standardized interfaces. A SOA
approach is thus not only intended to reduce redundancy and increase
integration, but also to provide the kind of flexibility needed to support
a quicker response to changing and evolving business requirements and
emerging conditions.

^25See, for example, GAO, Homeland Security: Efforts Under Way to Develop
Enterprise Architecture, but Much Work Remains, GAO-04-777 (Washington,
D.C.: Aug. 6, 2004); GAO-04-731R; Information Technology: Architecture
Needed to Guide NASA's Financial Management Modernization, GAO-04-43
(Washington, D.C.: Nov. 21, 2003); GAO-03-1018; GAO-03-877R; Information
Technology: DLA Should Strengthen Business Systems Modernization
Architecture and Investment Activities, GAO-01-631 (Washington, D.C.: June
29, 2001); and Information Technology: INS Needs to Better Manage the
Development of Its Enterprise Architecture, GAO/AIMD-00-212 (Washington,
D.C.: Aug. 1, 2000).

^26GAO, Information Technology: FBI Has Largely Staffed Key Modernization
Program, but Strategic Approach to Managing Program's Human Capital Is
Needed, GAO-07-19 (Washington, D.C.: Oct. 16, 2006).

  IT Investment Management: A Brief Description

IT investment management is a process for linking IT investment decisions
to an organization's strategic objectives and business plans that focuses
on selecting, controlling, and evaluating investments in a manner that
minimize risks while maximizing the return of investment.^27

           o During the selection phase, the organization (1) identifies and
           analyzes each project's risks and returns before committing
           significant funds to any project and (2) selects those IT projects
           that will best support its mission needs.
           o During the control phase, the organization ensures that, as
           projects develop and investment expenditures continue, they
           continue to meet mission needs at the expected levels of cost and
           risk. If the project is not meeting expectations or if problems
           arise, steps are quickly taken to address the deficiencies.
           o During the evaluation phase, actual versus expected results are
           compared once a project has been fully implemented. This is done
           to (1) assess the project's impact on mission performance, (2)
           identify any changes or modifications to the project that may be
           needed, and (3) revise the investment management process based on
           lessons learned.

           Consistent with this guidance, our IT Investment Management
           framework (ITIM)^28 consists of five progressive stages of
           maturity for any given agency relative to selecting, controlling,
           and evaluating its investment management capabilities. (See fig. 2
           for the five ITIM stages of maturity.) Stage 2 critical processes
           lay the foundation by establishing successful, predictable, and
           repeatable investment control processes at the project level.
           Stage 3 is where the agency moves from project-centric processes
           to portfolio-based processes and evaluates potential investments
           according to how well they support the agency's missions,
           strategies, and goals. Organizations implementing these Stages 2
           and 3 practices have in place selection, control, and evaluation
           processes that are consistent with the Clinger-Cohen Act.^29
           Stages 4 and 5 require the use of evaluation techniques to
           continuously improve both investment processes and portfolios in
           order to better achieve strategic outcomes.
		   
^27GAO-04-394G; GAO, GAO/AIMD-10.1.13; Executive Guide: Improving Mission
Performance Through Strategic Information Management and Technology,
GAO/AIMD-94-115 (Washington, D.C.: May 1994); and Office of Management and
Budget, Evaluating Information Technology Investments, A Practical Guide 
(Washington, D.C.: November 1995).

^28GAO-04-394G.

^2940 U.S.C. SS 11311-11313.

           Figure 2: The Five ITIM Stages of Maturity with Critical Processes

           The overriding purpose of the framework is to encourage investment
           selection, control, and evaluate processes that promote business
           value and mission performance, reduce risk, and increase
           accountability and transparency. We have used the framework in
           several of our evaluations,^30 and a number of agencies have
           adopted it. With the exception of the first stage, each maturity
           stage is composed of "critical processes" that must be implemented
           and institutionalized in order for the organization to achieve
           that stage. Each ITIM critical process consists of "key
           practices"--to include organizational structures, policies, and
           procedures--that must be executed to implement the critical
           process. Our research shows that agency efforts to improve
           investment management capabilities should focus on implementing
           all lower stage practices before addressing higher stage
           practices.
		   
^30GAO, Information Technology: Centers for Medicare & Medicaid Services
Needs to Establish Critical Investment Management Capabilities,  GAO-06-12
(Washington, D.C.: Oct. 28, 2005); Information Technology: HHS Has Several
Investment Management Capabilities in Place, but Needs to Address Key
Weaknesses,  GAO-06-11 (Washington, D.C.: Oct. 28, 2005); Information
Technology: FAA Has Many Investment Management Capabilities in Place, but
More Oversight of Operational Systems Is Needed, GAO-04-822 (Washington,
D.C.: Aug. 20, 2004); Information Technology: Departmental Leadership
Crucial to Success of Investment Reforms at Interior, GAO-03-1028
(Washington, D.C.: Sept. 12, 2003); Bureau of Land Management: Plan Needed
to Sustain Progress in Establishing IT Investment Management Capabilities,
GAO-03-1025 (Washington, D.C.: Sept. 12, 2003); United States Postal
Service: Opportunities to Strengthen IT Investment Management
Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002); and Information
Technology: DLA Needs to Strengthen Its Investment Management Capability,
GAO-02-314 (Washington, D.C.: Mar. 15, 2002).

           DOD's Institutional Approach to Business Systems Modernization

           In 2005, the department reassigned responsibility for providing
           executive leadership for the direction, oversight, and execution
           of its business systems modernization efforts to several entities.
           These entities and their responsibilities include the Defense
           Business Systems Management Committee (DBSMC), which serves as the
           highest ranking governance body for business systems modernization
           activities; the Principal Staff Assistants, who serve as the
           certification authorities for business system modernizations in
           their respective core business missions; the IRBs, which form the
           review and decision-making bodies for business system investments
           in their respective areas of responsibility; and the Business
           Transformation Agency (BTA), which is responsible for leading and
           coordinating business transformation efforts across the
           department. The BTA is organized into seven directorates, one of
           which is the Defense Business Systems Acquisition Executive--the
           component acquisition executive for DOD enterprise-level
           (DOD-wide) business systems and initiatives. This office is
           responsible for developing, coordinating, and integrating
           enterprise-level projects, programs, systems and initiatives,
           including managing resources such as fiscal, personnel, and
           contracts for assigned systems and programs.

           Table 1 lists these entities and provides greater detail on their
           roles, responsibilities, and composition.

Table 1: DOD Business Systems Modernization Governance Entities' Roles,
Responsibilities, and Composition

                            Roles and                                         
Entity                   responsibilities        Composition               
DBSMC                       o Provides strategic Chaired by the Deputy     
                               direction and plans  Secretary of Defense;     
                               for the business     Vice Chair is the Under   
                               mission area^a in    Secretary of Defense for  
                               coordination with    Acquisition, Technology,  
                               the warfighting and  and Logistics             
                               enterprise           (USD(AT&L)). Includes     
                               information          senior leadership in the  
                               environment mission  Office of the Secretary   
                               areas.               of Defense, the military  
                               o Recommends         departments' secretaries, 
                               policies and         and defense agencies'     
                               procedures required  heads, such as the        
                               to integrate DOD     Assistant Secretary of    
                               business             Defense (Networks and     
                               transformation and   Information               
                               attain               Integration)/Chief        
                               cross-department,    Information Officer       
                               end-to-end           (ASD(NII)/CIO), the Vice  
                               interoperability of  Chairman of the Joint     
                               business systems and Chiefs of Staff, and the  
                               processes.           Commanders of the U.S.    
                               o Serves as          Transportation Command    
                               approving authority  and Joint Forces Command. 
                               for business system                            
                               modernization.                                 
                               o Establishes                                  
                               policies and                                   
                               approves the                                   
                               business mission                               
                               area strategic plan,                           
                               the enterprise                                 
                               transition plan for                            
                               implementation for                             
                               business systems                               
                               modernization, the                             
                               transformation                                 
                               program baseline,                              
                               and the BEA.                                   
Principal Staff             o Support the        Under Secretaries of      
Assistants/Certification    DBSMC's management   Defense for Acquisition,  
Authorities                 of enterprise        Technology, and           
                               business IT          Logistics; Comptroller;   
                               investments.         and Personnel and         
                               o Serve as the       Readiness.                
                               certification                                  
                               authorities                                    
                               accountable for the                            
                               obligation of funds                            
                               for respective                                 
                               business system                                
                               modernizations                                 
                               within designated                              
                               core business                                  
                               missions.^b                                    
                               o Provide the DBSMC                            
                               with recommendations                           
                               for system                                     
                               investment approval.                           
IRBs                        o Serve as the       Includes the Principal    
                               oversight and        Staff Assistants; Joint   
                               investment           Staff; ASD(NII)/CIO; core 
                               decision-making      business mission area     
                               bodies for those     representatives; military 
                               business             departments; defense      
                               capabilities that    agencies; and combatant   
                               support activities   commands.                 
                               under their                                    
                               designated areas of                            
                               responsibility.                                
                               o Recommend                                    
                               certification for                              
                               all business systems                           
                               investments costing                            
                               more than $1 million                           
                               that are integrated                            
                               and compliant with                             
                               the BEA.                                       
Component                   o Ensures            Includes the Chief        
Pre-Certification           component-level      Information Officer from  
Authority                   investment review    the Air Force, the        
                               processes integrate  Principal Director of     
                               with the Investment  Governance, Acquisition,  
                               Management system.   and Chief Knowledge       
                               o Identifies those   Office from the Army, the 
                               component systems    Chief Information Officer 
                               that require IRB     from the Navy, and        
                               certification and    comparable                
                               prepare, review,     representatives from      
                               approve, validate,   other defense agencies.   
                               and transfer                                   
                               investment                                     
                               documentation as                               
                               required.                                      
                               o Assesses and                                 
                               precertifies                                   
                               architecture                                   
                               compliance of                                  
                               component systems                              
                               submitted for                                  
                               certification and                              
                               annual review.                                 
                               o Acts as the                                  
                               component's                                    
                               principal point of                             
                               contact for                                    
                               communication with                             
                               the IRBs.                                      
BTA                         o Operates under the Comprised of seven        
                               authority of the     directorates (Defense     
                               USD(AT&L) under the  Business Systems          
                               direction of the     Acquisition Executive,    
                               Deputy Under         Enterprise Integration,   
                               Secretary of Defense Transformation Planning   
                               for Business         and Performance,          
                               Transformation and   Transformation Priorities 
                               the Deputy Under     and Requirements,         
                               Secretary of Defense Investment Management,    
                               for Financial        Warfighter Support        
                               Management.          Office, and Chief of      
                               o Maintains and      Staff).                   
                               updates the                                    
                               department's BEA and                           
                               enterprise                                     
                               transition plan.                               
                               o Ensures that                                 
                               functional                                     
                               priorities and                                 
                               requirements of                                
                               various defense                                
                               components, such as                            
                               the Army and Defense                           
                               Logistics Agency are                           
                               reflected in the                               
                               architecture.                                  
                               o Ensures adoption                             
                               of DOD-wide                                    
                               information and                                
                               process standards as                           
                               defined in the                                 
                               architecture.                                  
                               o Serves as the                                
                               day-to-day                                     
                               management entity of                           
                               the business                                   
                               transformation                                 
                               effort at the DOD                              
                               enterprise level.                              
                               o Provides support                             
                               to the DBSMC and                               
                               IRBs.                                          

Source: DOD.

aAccording to DOD, the business mission area is responsible for ensuring
that capabilities, resources, and materiel are reliably delivered to the
warfighter. Specifically, the BMA addresses areas such as real property
and human resources management.

bDOD has five core business missions: Human Resources Management, Weapon
System Lifecycle Management, Materiel Supply and Service Management, Real
Property and Installations Lifecycle Management, and Financial Management.

  Tiered Accountability

In 2005, DOD reported that it had adopted a tiered accountability approach
to business transformation. Under this approach, responsibility and
accountability for business architectures and systems investment
management are assigned to different levels in the organization. For
example, the BTA is responsible for developing the corporate BEA, which
provides the thin layer of corporate policies, capabilities, standards,
and rules. The components are responsible for defining a component-level
architecture and transition plans associated with their own tier of
responsibility and for doing so in a manner that is aligned with (i.e.,
does not violate) the corporate BEA's policies, capabilities, standards,
and rules. Similarly, program managers are responsible for developing
program-level architectures and plans and ensuring alignment with the
architectures and transition plans above them. As such, this concept
allows for autonomy while also ensuring linkages and alignment from the
program level through the component level to the enterprise level.

For business investment management, responsibility and accountability is
also tiered, meaning that it is allocated between the DOD corporate level
(i.e., Office of the Secretary of Defense) and the components based on the
amount of development/modernization funding involved and the investment's
designated "tier." More specifically, DOD corporate is responsible for
ensuring that all business systems with a development/modernization
investment in excess of $1 million are reviewed by the IRBs for compliance
with the BEA, certified by the Principal Staff Assistants, and approved by
the DBSMC. Components are responsible for certifying
development/modernization investments with total costs of $1 million or
less. All DOD development and modernization efforts are also assigned a
"tier" based on acquisition category and/or the size of the financial
investment.^31

Summary of Fiscal Year 2005 National Defense Authorization Act Requirements

Congress included six provisions in the act^32 that are aimed at ensuring
DOD's development of a well-defined BEA and associated enterprise
transition plan (ETP), as well as the establishment and implementation of
effective investment management structures and processes. The requirements
are as follows:

           1. Develop a BEA that:

           o includes an information infrastructure that, at a minimum, would
           enable DOD to:

                        o comply with all federal accounting, financial
                        management, and reporting requirements;
                        o routinely produce timely, accurate, and reliable
                        financial information for management purposes;
                        o integrate budget, accounting, and program
                        information and systems;
						
^31As defined in the department's Investment Review Board Concept of
Operations and its Investment Certification and Annual Review Process User
Guidance, there are four tiers of business systems. Tier 1 systems include
all systems that are classified as a "major automated information system"
or a "major defense acquisition program;" tier 2 systems include those
with modernization efforts of $10 million or greater but that are not
designated as a major automated information system or a major defense
acquisition program, or programs that have been designated as IRB interest
programs because of their impact on DOD transformation objectives; tier 3
systems include those with modernization efforts that have anticipated
costs greater than $1 million but less than $10 million; and tier 4
systems are those with modernization efforts that have anticipated costs
of up to $1 million.

^32Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. S 2222).

                        o provide for the systematic measurement of
                        performance, including the ability to produce timely,
                        relevant, and reliable cost information;

                        o includes policies, procedures, data standards, and
                        system interface requirements that are to be applied
                        uniformly throughout the department; and

                        o is consistent with OMB policies and procedures.

                                     2. Develop a transition plan for
                                     implementing the architecture that
                                     includes:

           o an acquisition strategy for new systems needed to complete the
           enterprise architecture;
           o a list and schedule of legacy business systems to be terminated;
           o a list and strategy of modifications to legacy business systems;
           and
           o time-phased milestones, performance metrics, and a statement of
           financial and non-financial resource needs.

                        3. Identify each business system proposed for funding
                        in DOD's fiscal year budget submissions and include:

           o information on each business system proposed for funding in that
           budget;
           o funds for current services and for business systems
           modernization; and
           o the designated approval authority for each business system.

                        4. Delegate the responsibility for business systems
                        to designated approval authorities within the Office
                        of the Secretary of Defense.
                        5. Require each approval authority to establish
                        investment review structures and processes, including
                        a hierarchy of IRBs--each with appropriate
                        representation from across the department. The review
                        process must cover:

           o review and approval of each business system by an IRB before
           funds are obligated;
           o at least an annual review of every business system investment;
           o use of threshold criteria to ensure an appropriate level of
           review and accountability;
           o use of procedures for making architecture compliance
           certifications;
           o use of procedures consistent with DOD guidance; and
           o incorporation of common decision criteria.

                        6. Effective October 1, 2005, DOD may not obligate
                        appropriated funds for a defense business system
                        modernization with a total cost of more than $1
                        million unless, the approval authority certifies that
                        the business system modernization:

           o complies with the BEA and

           o is necessary to achieve a critical national security capability
           or address a critical requirement in an area such as safety or
           security; or is necessary to prevent a significant adverse effect
           on an essential project in consideration of alternative solutions,
           and the certification is approved by the DBSMC.
		   
		   Summary of Recent GAO Reviews of DOD's Business Systems
		   Modernization and Business Transformation Efforts

           In November 2005^33 and in May 2006,^34 we reported that DOD had
           partially satisfied four of the six business system modernization
           requirements in the fiscal year 2005 National Defense
           Authorization Act^35 relative to architecture development,
           transition plan development, budgetary disclosure, and investment
           review; it had fully satisfied the requirement concerning
           designated approval authorities; and it was in the process of
           satisfying the last requirement for certification and approval of
           modernizations costing in excess of $1 million. As a result, we
           concluded that the department had made important progress in
           defining and beginning to implement institutional management
           controls (i.e., processes, structures, and tools), but much
           remained to be accomplished relative to the act's requirements and
           relevant guidance, including developing component architectures
           that are aligned with the corporate BEA and ensuring that
           investment review and approval processes are fully developed and
           institutionally implemented across all organizational levels.
		   
^33GAO, DOD Business Systems Modernization: Important Progress Made in
Establishing Foundational Architecture Products and Investment Management
Practices, but Much Work Remains, GAO-06-219 (Washington, D.C.: Nov. 23,
2005).

^34GAO-06-658.

^35Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. S 2222).

           Notwithstanding this progress on business systems modernization,
           we also testified in November 2006^36 that DOD continued to lack a
           comprehensive, enterprisewide approach to its overall business
           transformation effort. We noted that while DOD's planning and
           management continued to evolve, it had yet to develop a
           comprehensive, integrated, and enterprisewide plan that covered
           all key business functions and contained results-oriented goals,
           measures, and expectations that link organizational, unit, and
           individual performance goals while also being clearly linked to
           DOD's overall investment plans. We concluded that because of the
           complexity and long-term nature of business transformation, the
           department continued to need a chief management official with
           significant authority, experience, and tenure to provide sustained
           leadership and integrate its overall business transformation
           effort. We also concluded that without formally designating
           responsibility and accountability for results, reconciling
           competing priorities in investments will be difficult and could
           impede DOD's progress in its transformation efforts. We are
           currently assessing the department's business transformation
           efforts, including an analysis of the various proposals for a
           chief management officer and its response to these proposals, and
           plan to report our results in the near future.
		   
		   DOD Is Continuing to Improve Its Approach to Modernizing Business Systems

           DOD continues to take steps to comply with the requirements of the
           act and to satisfy relevant systems modernization management
           guidance. In particular, on March 15, 2007, DOD released an update
           to its BEA (version 4.1), developed an updated ETP, and issued its
           annual report to Congress describing steps taken and planned
           relative to the act's requirements, among other things.
           Collectively, these steps address several legislative provisions
           and best practices concerning the corporate architecture,
           transition plan, budgetary disclosure, and investment review of
           systems costing in excess of $1 million that we previously
           reported as missing. However, additional steps are needed to fully
           comply with the act and relevant guidance. Specifically, the
           department has yet to extend and evolve its corporate BEA to the
           department's component organizations' (military departments and
           defense agencies) architectures, fully define its IT investment
           management policies and procedures, and officially establish one
           of the five legislatively mandated IRBs. BTA officials agree that
           additional steps are needed to fully implement the act's
           requirements and related system modernization management best
           practices. According to BTA officials, DOD leadership is committed
           to fully addressing these areas and efforts are planned and under
           way to do so.
		   
^36GAO-07-229T.	

           DOD Continues to Improve Its Corporate BEA, but Component
		   Architectures Remain a Challenge	   

           Among other things, the act requires DOD to develop a BEA that
           would cover all defense business systems and the functions and
           activities supported by defense business systems and enable the
           entire department to (1) comply with all federal accounting,
           financial management, and reporting requirements; (2) routinely
           produce timely, accurate, and reliable financial information for
           management purposes; and (3) include policies, procedures, data
           standards, and system interface requirements that are to be
           applied throughout the department.

           In 2006,^37 we reported that the then current version of the BEA
           (version 3.1) addressed several of the missing elements we had
           previously identified relative to the act's requirements and
           relevant guidance. However, we also reported that additional steps
           were needed. On March 15, 2007, DOD released an update to its BEA
           (version 4.1), which resolves several of the architecture gaps
           associated with the prior version and adds content proposed by DOD
           stakeholders.^38 For example, version 4.1 improves the Financial
           Visibility business enterprise priority (BEP) area by including
           the Standard Financial Information Structure data elements and
           business rules to support cost accounting and reporting. This
           version also addresses, to varying degrees, missing elements,
           inconsistencies, and usability issues that we previously
           identified.^39 Examples of these improvements and remaining issues
           are summarized in the following text:
		   
^37GAO-06-658.

^38According to DOD, the BEA stakeholders include the core business
mission areas through the Business Enterprise Priorities, which comprises
Personnel Visibility, Acquisition Visibility, Common Supplier Engagement,
Materiel Visibility, Real Property Accountability, and Financial
Visibility. The department added that as the BEA evolves, the stakeholders
will include components that must federate their architectures to the BEA,
program managers who must comply with the BEA, IRBs who use the BEA to
guide and constrain investments, and systems designers and integrators who
must build and configure their systems to comply with the BEA.

^39GAO-06-658.

           o This latest version contains enterprise-level information about
           DOD's "As Is" architectural environment to support business
           capability gap analyses. As we previously reported,^40 such gap
           analyses between the "As Is" and the "To Be" environments are
           essential for the development of a well-defined transition plan.
           However, such gap analyses were not previously provided for in
           prior versions of the BEA. To DOD's credit, the architecture now
           includes "As Is" information (e.g., problems that enterprise
           priorities are to address and the root causes of each problem) for
           five of the six BEPs. For example, this version identifies the
           "inability to record or report funds distribution at the
           transaction level" as a problem for the Financial Visibility
           priority area, and "stove-pipe systems" and "non-standard forms"
           as the root causes. Moreover, it includes "As Is" information
           about related enterprise systems, such as the Wide-area Workflow
           system. However, the current version does not provide "As Is"
           information for the Acquisition Visibility priority area.
           o The latest version includes performance metrics for the business
           capabilities within enterprise priority areas, including actual
           performance relative to performance targets that are to be met.
           For example, currently 26 percent of DOD assets are reported by
           using the Department of the Treasury's United States Standard
           General Ledger^41 compliant formats, as compared to a target of
           100 percent. However, the architecture does not describe the
           actual baseline performance for operational activities, such as
           for the "Manage Audit and Oversight of Contractor" operational
           activity. As we have previously reported,^42 performance models
           are an essential part of any architecture and having defined
           performance baselines to measure actual performance against
           provides the means for knowing whether the intended mission value
           to be delivered by each business process is actually being
           realized.

           o The latest version identifies activities performed at each
           location/organization and indicates which organization(s) are or
           will be involved in each activity. We previously reported that
           prior versions did not address the locations where specified
           activities are to occur and that doing so is important because the
           cost and performance of implemented business operations and
           technology solutions are affected by the location and therefore
           need to be examined, assessed, and decided on in an enterprise
           context rather than in a piecemeal, systems-specific fashion.^43
           To DOD's credit, the latest version includes some of this
           information. For example, it indicates that the Defense Contract
           Management Agency is involved in the "Conduct Acquisition
           Assessment" operational activity. However, not all operational
           activities, such as "Authorize Return or Disposal" activity are
           assigned to a location/organization. In addition, the latest
           version does not include the roles and responsibilities of
           organizations performing the same operational activities, which is
           important to avoid duplication and inconsistency in how functions
           and activities are implemented.
		   
^40GAO-06-219.

^41The United States Standard General Ledger provides a uniform chart of
accounts and technical guidance used in standardizing federal agency
accounting.

^42GAO-04-777 and GAO-03-584G.		   

           o The latest version includes common policies (e.g., "IRBs approve
           only those system investments that are aligned with enterprise
           transformation objectives and standards") and procedures (e.g.,
           "Components and programs use the Architecture Compliance and
           Requirements Traceability tool to illustrate how their system
           investments map to applicable activities, business rules, and data
           in the BEA"). It also includes business rules (e.g., "each request
           for commercial export of DOD technology must be processed within
           30 days upon receipt of request from the Department of State or
           the Department of Commerce") to facilitate consistent
           implementation of the policies and procedures.^44 However, the
           architecture does not identify enterprise business rules for all
           business processes. For example, there are no business rules for
           the Common Supplier Engagement business process "Perform
           Acceptance Procedures for Other Goods and Services." Moreover, the
           latest version continues to provide inconsistent levels of detail
           for some business rules. For example, some business rules are
           defined at the conceptual level (e.g., "ENT_Cost_Reporting") while
           others are defined at a more operational level (e.g.,
           "ENT_DOD_Obligations_Against"). Without well-defined business
           rules, it is likely that policies and procedures will be
           implemented inconsistently because they will be uniquely
           interpreted.

           o The latest version provides information flows among some
           organizational units, business operations, and system elements.
           These information flows are intended to show what information is
           needed and where and how the information moves and is shared to
           support mission functions. For example, the "Financial Management
           Detail" operational node connectivity diagram is a graphical
           depiction of the operational nodes (or organizations) with
           "needlines" that indicate a need to exchange information and
           identify information exchange requirements among the financial
           management organizational units (e.g., between the accounting
           office and commercial entitlement office operational nodes).
           However, detailed operational node connectivity diagrams similar
           to the "Financial Management Detail" diagram have not yet been
           developed for the other core business mission areas, such as Human
           Resources Management. Such information is critical for defining
           business service interactions and establishing interfaces between
           users and systems. Moreover, the BEA does not include information
           flows between the enterprise and DOD components. Such information
           is important for developing a common understanding of the semantic
           meaning of information exchanges among DOD organizations.

^43GAO-06-658.

^44Business rules are important because they explicitly translate business
policies and procedures into specific, unambiguous rules that govern what
can and cannot be done.

           o The latest version continues to represent the thin layer of
           DOD-wide corporate architectural policies, capabilities, rules,
           and standards. Having this layer is essential to a well-defined
           federated architecture, but it alone does not provide the total
           federated family of DOD parent and subsidiary architectures for
           the business mission area that are needed to comply with the act.
           As we recently reported, well-defined architectures do not yet
           exist for the military departments,^45 which constitute the
           largest members of the federation. In particular, we reported that
           none of the three military departments had fully developed
           architecture products that describe their respective target
           architectural environments and developed transition plans for
           migrating to a target environment, and none were employing the
           full range of architecture management structures, processes, and
           controls provided for in relevant guidance. Accordingly, we made
           recommendations aimed at improving the management and content of
           the military departments' respective architectures, which the
           department agreed with.^46 (See app. III for the specific
           recommendations.)

           Recognizing the need to address its component architecture
           challenge, the BTA released its business mission area federation
           strategy and road map in September 2006 to address how the
           corporate BEA would be extended to the military departments and
           defense agencies. We recently reported^47 that this strategy
           provides a foundation on which to build and align DOD's parent
           business architecture with the subsidiary architectures of the
           military departments and defense agencies (see fig. 3). In
           particular, we noted that the strategy (1) states the department's
           federated architecture goals; (2) describes federation concepts
           that are to be applied; and (3) includes high-level activities,
           capabilities, products, and services intended to facilitate
           implementation of the concepts.
		   
^45GAO-06-831.

^46GAO-06-831.

^47GAO-07-451.

Figure 3: Simplified Diagram of DOD's Business Mission Area Federated
Architecture

However, we also reported that the strategy does not adequately define the
tasks needed to achieve the strategy's goals, including those associated
with executing high-level activities and providing related capabilities,
products, and services. Specifically, it does not adequately address how
strategy execution will be governed, including assignment of roles and
responsibilities, measurement of progress and results, and provision of
resources. Also, the strategy does not address, among other things, how
the component architectures will be aligned with the latest version of the
BEA and how it will identify and provide for reuse of common applications
and systems across the department. Accordingly, we made recommendations
aimed at better defining the department's architecture federation plans,
which the department largely disagreed with.^48 (See app. III for the
specific recommendations.)

According to DOD, the corporate BEA focuses on providing tangible outcomes
for a limited set of enterprise-level (DOD-wide) priorities, and the
components are responsible under the department's tiered accountability
approach for defining their respective component-level architectures that
are aligned with the corporate BEA. According to DOD, subsequent releases
of the BEA will continue to reflect this federated approach and will
define enforceable interfaces to ensure interoperability and information
flow to support decision making at the appropriate level. To help ensure
this, the BTA plans to have its BEA independent verification and
validation contractor examine architecture federation when evaluating
subsequent BEA releases. Use of an independent verification and validation
agent is an architecture management best practice for identifying
architecture strengths and weaknesses. Through the use of such an agent,
department and congressional oversight bodies can gain information that
they need to better ensure that DOD's family of architectures and
associated transition plan(s) satisfy key quality parameters, such as
completeness, consistency, understandability, and usability, which the
department's annual reports have yet to include.

Until DOD has a well-defined family of architectures for its business
mission area, it will not fully satisfy the requirements of the act and it
will remain challenged in its ability to effectively manage its business
system modernization efforts.

DOD Continues to Expand and Update Its Enterprise Transition Plan, but Important
Elements Are Still Missing

Among other things, the act requires DOD to develop an ETP for
implementing its BEA that includes listings of the legacy systems that
will and will not be part of the target business systems environment and
specific time-phased milestones and performance metrics.

In 2006,^49 we reported that the prior version of the ETP addressed
several of the missing elements that we previously identified relative to
the act's requirements and relevant guidance. However, we also reported
that additional steps were needed. On March 15, 2007, DOD released an
updated version of its ETP, which provides information on 106 of what it
refers to as transformational programs (systems and initiatives) and
relates these to key transformational objectives. For example, it includes
specific time-phased milestones^50 for about 86 business system
investments and initiatives and performance metrics for about 84 systems
and initiatives. Further, the ETP discusses progress made on business
system investments over the last 6 months--including key accomplishments
and milestones attained, as well as new information on near-term
activities (i.e., activities to occur during the next 6 months). This
version also addresses, to varying degrees, missing elements that we
identified in our prior report.^51 Examples of these improvements and
remaining issues are summarized in the following text:

^48GAO-07-451.

^49GAO-06-658.

           o The latest version of the ETP documents the results of ongoing
           and planned analyses of gaps between its "As Is" and "To Be"
           architectural environments, in which capability and performance
           shortfalls are described and investments (such as transformation
           initiatives and systems) that are to address these shortfalls are
           clearly identified. For example, it aligns the Defense Integrated
           Military Human Resources System with the Personnel Visibility
           priority area and states that it will provide business capability
           improvements that include providing accurate and timely pay
           benefits for military service members and their families anytime
           and anywhere. However, the gap analysis is not yet completed for
           all the current BEPs. In particular, the gap analysis did not
           include the Acquisition Visibility priority area. Without
           identifying how business capability gaps between the baseline and
           target architecture are to be addressed for all BEPs, the
           department's transition plan cannot be considered sufficiently
           complete, and thus its ability to support informed investment
           selection and control decisions is limited.

           o The latest version of the ETP provides a range of information
           for the 106 systems and initiatives identified, such as 3 years of
           budget information for 64 of these systems and initiatives.
           However, the plan has yet to address our prior finding for
           including system and budget information for investments by 13 of
           its 15 defense agencies^52 and for 8 of its 9 combatant
           commands.^53 BTA officials told us that information for these
           defense agencies and combatant commands is not included because
           the ETP focused on the largest business-related organizations in
           DOD (i.e., those having the majority of the tier 1 and 2 business
           investments), and the majority of the defense agencies and
           commands do not have investments that meet this threshold
           criteria. Nevertheless, they said that they plan to include all
           component tier 1 and 2 systems over the next 3 years.
		   
^50The time-phased milestones refer to milestones, such as initial
operating capability, full operating capability, technology development
phase, and system development and demonstration phase.

^51GAO-06-658.

           o The latest version also provides performance measures for the
           enterprise and component transformation programs, including key
           milestones (e.g., Initial Operating Capability). However, the ETP
           does not include other important information needed to understand
           the sequencing of these business investments. In particular, the
           planned investments in the transition plan are not sequenced based
           on a range of activities that are critical to developing an
           effective transition plan. More specifically, we previously
           reported^54 that the plan is largely based on a bottom-up planning
           process in which ongoing programs were examined and categorized in
           the plan around BEPs and capabilities, including a determination
           as to which programs would be designated and managed as DOD-wide,
           enterprise programs versus component programs. This bottom-up
           approach to developing the plan does not explicitly reflect
           transition planning key practices cited in federal guidance, such
           as consideration of technology opportunities, marketplace trends,
           fiscal and budgetary constraints, institutional system development
           and acquisition capabilities, and new and legacy system
           dependencies and life expectancies, and the projected value of
           competing investments.^55 For example, many of these investments
           are dependent on Net-Centric Enterprise Services (NCES)^56 for its
           core services, and as such the plans and milestones for each
           should reflect the incremental capability deployment of NCES.
           According to the BTA official responsible for the ETP, the
           transition plan investments have not been sequenced based on any
           of these considerations other than fiscal year budgetary
           constraints. However, DOD officials reported that the BTA intends
           to depict the dependencies in the ETP, especially
           program-to-program dependencies associated with adoption of a
           service-oriented architecture approach. BTA officials also said
           that each technology-based sequencing decision will be governed by
           DOD's tiered accountability approach to investment decision making
           and architecture federation.
		   
^52DOD included system and budget information for the Defense Financial
and Accounting Service and Defense Logistics Agency in the transition
plan. DOD did not include this information for the following defense
agencies: (1) Missile Defense Agency, (2) Defense Advanced Research
Projects Agency, (3) Defense Commissary Agency, (4) Defense Contract Audit
Agency, (5) Defense Contract Management Agency, (6) Defense Information
Systems Agency, (7) Defense Intelligence Agency, (8) Defense Legal
Services Agency, (9) Defense Security Cooperation Agency, (10) Defense
Security Service, (11) Defense Threat Reduction Agency, (12) National
Geospatial-Intelligence Agency, and (13) National Security Agency.

^53DOD included system and budget information for the Transportation
Command in the transition plan. DOD did not include this information for
the (1) Central Command, (2) Joint Forces Command, (3) Pacific Command,
(4) Southern Command, (5) Space Command, (6) Special Operations Command,
(7) European Command, and (8) Strategic Command.

^54GAO-06-219.

^55GAO-03-584G and CIO Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001).		   

           o The latest version of the ETP includes a listing of the legacy
           systems that will not be part of the "To Be" environment and the
           termination dates for many of these systems. We previously
           reported^57 that the prior version did not include a complete
           listing of the legacy systems and that the termination dates for
           many legacy systems, including the Personnel Records Management
           System, Defense Departmental Reporting System, and Base Accounts
           Receivable System, were not known, making it unclear whether or
           not they will be part of the target environment. To DOD's credit,
           the ETP now reflects all decisions recorded to date on these
           legacy system terminations. According to the department, this list
           will continue to evolve as components and IRBs make investment
           decisions in the future. In addition, it provides information on
           legacy system migration and retirement as a result of implementing
           each target system. According to DOD, the annual report lists over
           700 systems targeted for elimination as a result of the
           implementation of targeted business systems, with specific
           termination dates identified for over 93 percent of these systems.
		   
^56NCES is intended to provide capabilities that are key to enabling
ubiquitous access to reliable decision-quality information. NCES
capabilities can be packaged into four product lines: service-oriented
architecture foundation (e.g., security and information assurance),
collaboration (e.g., application sharing), content discovery and delivery
(e.g., delivering information across the enterprise), and portal (e.g.,
user-defined Web-based presentation).

^57GAO-06-658.		   

           o The latest version of the ETP also includes for the first time a
           discussion of how the department plans to use enterprise
           application integration,^58 including plans, methods, and tools
           for reusing applications that already exist while also adding new
           applications and databases. However, this discussion is
           nevertheless still notional and thus lacks specifics on which
           investments will reuse which applications.

           According to BTA officials, a number of actions are envisioned to
           address the above cited areas and further improve the ETP, such as
           adding the results of capability gap analyses for all business
           priorities, including tier 1 and 2 programs for all components,
           and recognizing dependencies among investments. Until the ETP, or
           a federated family of such plans, either directly or by reference
           includes relevant information on the full inventory of investments
           across the department, (and does so in a manner that reflects
           consideration of the range of variables associated with a
           well-defined transition plan, such as timing dependencies among
           investments and the department's capability to manage them) it
           will not have a sufficient basis for informed investment decision
           making regarding disposition of the department's existing
           inventory of systems or for sequencing the introduction of
           modernized systems. To ensure that the above discussed
           shortcomings with the department transition plan(s) are made, we
           have previously made recommendations that the department is still
           in the process of addressing aimed at formalizing its plans for
           incrementally improving its transition plan. (See app. II for
           these recommendations.)
		   
		   DOD's Fiscal Year 2008 Budget Submission Includes Key Information
		   on Business Systems

           Among other things, the act requires DOD's annual IT budget
           submission to include key information on each business system for
           which funding is being requested, such as the system's designated
           approval authority and the appropriation type and amount of funds
           associated with development/modernization and current services (to
           operate and maintain the system).

           The department's fiscal year 2008/2009 budget submission includes
           a range of information for business system investments requesting
           funding, such as the system's (1) name, (2) approval authority,
           (3) approved funding for fiscal year 2007, and (4) requested
           funding for fiscal year 2008. The submission also identifies the
           amount of the fiscal year 2008 request that is for
           development/modernization versus operations/maintenance (i.e.,
           current services). For example, the Army's General Fund Enterprise
           Business System, the amount of modernization funds related to
           "Other Procurement, Army" and "Research, Development, Testing and
           Evaluation, Army" are identified. For systems in excess of $1
           million in modernization funding, the submission also cites the
           DBSMC approval date, where applicable.
		   
^58Enterprise application integration software is a commercial software
product, commonly referred to as middleware, to permit two or more
incompatible systems to exchange data from different databases.

           DOD Has Largely Established Key Investment Management Structures,
		   but Related Policies and Procedures Are Missing

           The act requires DOD to establish business system investment
           review structures, including the previously mentioned DBSMC and
           five IRBs, and processes that are consistent with the investment
           management provisions of the Clinger-Cohen Act.^59 As noted
           earlier, our ITIM framework provides five progressive stages of
           maturity for any given agency relative to selecting, controlling,
           and evaluating its IT investments. Organizations implementing
           Stages 2 and 3 practices have in place capabilities that assist in
           establishing selection, control, and evaluation structures,
           policies, procedures, and practices that are required by the
           investment management provisions of the Clinger-Cohen Act.

           In 2006, we reported that DOD had established the DBSMC and four
           of the five IRBs defined in the act and that it had developed a
           range of processes governing how business system investments are
           to be reviewed and approved.^60 More recently, we reported on the
           extent to which the department's corporate approach to business
           system investment management comports with the stages in our ITIM
           framework that are associated with investment management
           provisions of the Clinger-Cohen Act.^61 In summary, we found that
           DOD had established important management structures needed to
           manage its business system investments, but it had not fully
           defined many of related policies and procedures that our framework
           identified as needed to effectively manage its business
           investments as individual projects (Stage 2) and as portfolios of
           projects (Stage 3).
		   
^5940 U.S.C. S 11312.

^60GAO-06-658.

^61GAO-07-538.

           Investment Management Structures Have Been Largely Established
		   
           DOD has largely established the organizational structures that are
           associated with Stages 2 and 3 of our framework. Specifically, it
           has established an enterprisewide investment board and subordinate
           boards that are responsible for business systems investment
           governance, including conducting investment certification and
           approval reviews and annual reviews as provided for in the act.
           The enterprisewide board--the DBSMC--is composed of senior
           executives, including the Deputy Secretary of Defense and the
           ASD(NII)/CIO, as provided for in the act. Among other things, the
           DBSMC is responsible for establishing and implementing policies
           governing the organization's investment process and approving
           lower-level investment board processes and procedures. The
           subordinate boards include four IRBs that are composed of
           representatives from their respective core business mission, as
           well as representatives from the combatant commands, defense
           agencies, military departments, and Joint Chiefs of Staff. Among
           other things, they are responsible and accountable for overseeing
           and controlling certain business system investments, including
           ensuring compliance and consistency with the BEA. The department
           has also assigned responsibility to the USD(AT&L) for managing
           business system portfolio selection criteria.

           Moreover, since we reported in 2006^62 that the department has
           established four of the five IRBs mandated by the act, efforts
           have begun to establish the fifth. Specifically, ASD(NII)/CIO
           officials told us that they are now in the process of establishing
           the Enterprise Information Environment Mission Area^63 IRB to
           support IT infrastructure and information assurance activities, as
           required by the act. According to these officials, the draft
           concept of operations for this IRB is being revised and will
           subsequently be approved by the ASD(NII)/CIO. While the IRB has
           not been officially established, the officials stated that it has
           been in effect for about a year and added that the chair is the
           DOD Deputy CIO, and its membership includes representatives from
           the Defense Information Systems Agency, the DOD mission areas, and
           the military departments. They also said that the Under Secretary
           of Defense (Comptroller) and the Joint Chiefs of Staff are
           operating in an advisory role.
		   
^62GAO-06-658.

^63The Enterprise Information Environment Mission Area enables the
functions of the other mission areas (e.g., Warfighting Mission Area,
Business Mission Area, and Defense Intelligence Mission Area) and
encompasses communications, computing, and core enterprise service
systems, equipment, or software that provide a common information
capability or service for enterprise use.

           Policies and Procedures Have Been Defined for Some, but Not All,
		   Project-Level and Portfolio-Based Investment Management Activities

           As we recently reported,^64 DOD has defined policies and
           procedures relative to several key practices in our ITIM framework
           that are associated with project-level investment management
           (Stage 2). To its credit, the department has, for example,
           documented policies and procedures for ensuring that systems
           support ongoing and future business needs through alignment with
           the BEA; developed procedures for identifying and collecting
           information about these systems to support DBSMC and IRB
           investment decision making; and assigned responsibility for
           ensuring that the information collected about projects meets the
           needs of DOD's investment review structures and processes.
           However, we reported that it had not developed the full range of
           project-level policies and procedures needed for effective
           investment management. In commenting on our report, DOD stated
           that under DOD's tiered accountability, these are performed at the
           component level, and that departmental policies and procedures
           established for overseeing execution of these practices by
           components are sufficient. We do not agree. Examples of the
           limitations in the department's project-level policies and
           procedures are summarized next, along with their significance.

           o Policies and procedures do not address how business system
           investments that are past the development/modernization stage
           (i.e., in operations and maintenance) are to be governed or
           considered by the DBSMC or the IRBs. Given that DOD invests
           billions of dollars annually in operating and maintaining business
           systems, this is significant. While DOD officials stated that
           component-level policies and procedures address systems that are
           outside of development/modernization, best practices emphasize
           that the corporate investment boards should continue to review
           investment cost and performance baselines throughout their life
           cycles.

           o Policies and procedures do not outline how the DBSMC and IRB
           certification and annual review processes are to be coordinated
           with other decision-support processes used at DOD, such as the
           Joint Capabilities Integration and Development System; the
           Planning, Programming, Budgeting, and Execution system; and the
           Defense Acquisition System.^65 Without clear linkages among these
           processes, inconsistent and uninformed decision making may result.
		   
^64GAO-07-538.

^65The Joint Capabilities Integration and Development System is a
need-driven management system used to identify future capabilities for
DOD; the Planning, Programming, Budgeting, and Execution process is a
calendar-driven management system for allocating resources and is
comprised of four phases--planning, programming, budgeting, and
executing--that define how budgets for each DOD component and the
department as a whole are created, vetted, and executed; and the Defense
Acquisition System is an event-driven system for managing product
development and procurement and guides the acquisition process for DOD.		   

           o Procedures do not specify how the full range of cost, schedule,
           and benefit data is to be used by the IRBs in certification
           decisions. Without documenting how such boards are to consider
           cost, schedule, and benefits factors when making these decisions,
           the department cannot ensure that the boards and the DBSMC
           consistently and objectively select proposals that best meet the
           department's needs and priorities.

           o Policies and procedures do not exist that provide for sufficient
           oversight and visibility into component-level investment
           management activities, including component reviews of systems in
           operations and maintenance and tier 4 investments. According to
           DOD officials, such oversight is accomplished through the
           department's tiered accountability approach. However, the
           department did not provide policies and procedures defining how
           the DBSMC and IRBs ensure visibility into these component
           processes. This is particularly important because, according to
           DOD, only 285 of about 3,100 total business systems have completed
           the IRB certification process and have been approved by the DBSMC.
           Moreover, they said that the remaining business systems have not
           been through the certification process and have not been given a
           tier designation. Without policies and procedures defining how the
           DBSMC and IRBs have visibility into and oversight of all business
           system investments, DOD risks components continuing to invest in
           systems that are duplicative, stovepiped, non-integrated, and
           unnecessarily costly to manage, maintain, and operate.

           DOD's policies and procedures relative to portfolio-based business
           system investment management (Stage 3) are even less defined that
           than those for project-level investment management. As we recently
           reported,^66 DOD has not defined any of the policies and
           procedures that our ITIM framework identifies as needed for
           effective portfolio management. For example, the business mission
           area does not have documented policies and procedures for defining
           the criteria to be used for making portfolio selection decisions,
           creating the portfolio of business system investments, evaluating
           the performance of portfolio investments, and conducting
           postimplementation reviews of these investments. According to our
           ITIM framework, the development and use of portfolio selection
           criteria focuses on the synergistic benefits to be found among an
           agency's entire collection of investments, rather than just from
           the sum of the individual investments. Moreover, adequately
           documenting both the policies and the associated procedures that
           provide predictable, repeatable, and reliable investment selection
           and control and govern how an organization manages its IT
           investment portfolio(s) is important because doing so reduces
           investment risk of failure and provides the basis for having
           rigor, discipline, and repeatability in how investments are
           selected and controlled across the entire organization. In
           commenting on our recent report, DOD stated that it intends to
           improve departmental policies and procedures for business system
           investments by, for example, establishing a single governance
           structure, but plans or time frames for doing so had not been
           established.

^66GAO-07-538.

           Until DOD fully defines departmentwide policies and procedures for
           both individual projects and portfolios of projects, it risks
           selecting and controlling these business system investments in an
           inconsistent, incomplete, and ad hoc manner, which in turn reduces
           the chances that these investments will meet mission needs in the
           most cost-effective manner. Accordingly, our recent report made a
           series of recommendations to the department for strengthening both
           its project- and portfolio-level business system investment
           management policies and procedures.^67
		   
^67GAO-07-538.

           DOD Continues to Approve and Review Business Systems, but Military
		   Departments Processes for Doing So Are Still Evolving

           The act specifies two basic requirements that took effect October
           1, 2005, relative to DOD's obligation of funds for business system
           modernizations costing more than $1 million. First, it requires
           that these modernizations be certified by a designated approval
           authority^68 as meeting specific criteria.^69 Second, it requires
           that the DBSMC approve each of these certifications. The act also
           states that failure to do so before the obligation of funds for
           any such modernization constitutes a violation of the
           Anti-deficiency Act.^70 In March 2006, the department reported
           that the DBSMC had approved 226 business system modernizations,
           and as of March 2007, it reported that the committee approved an
           additional 59 systems, for a total of 285 approved systems.

           A key element of the department's approach to reviewing and
           approving business systems investments is the use of "tiered
           accountability," in which investment review begins at the
           component level and proceeds through a hierarchy of review and
           approval authorities, depending on the size and significance of
           the investment. Air Force, Army, and Navy officials told us that
           the success of the process depends on thorough analysis of each
           business system before it is submitted for higher-level review and
           approval. However, they added that their respective processes for
           reviewing investments are still evolving. A brief summary of each
           military department's investment review activities is provided in
           the following text.
		   
		     Air Force

           Air Force officials report that their department is following a
           phased approach to conducting reviews of about 930 business
           systems in accordance with the requirements of the act. In fiscal
           year 2007, it is to review all tiers 1 through 4 business systems,
           as well as tier 5 business systems^71 that have operating costs,
           not simply development and modernization funding, greater than $1
           million. During fiscal year 2008, the Air Force plans to review
           all business systems in tiers 1 through 4 and all tier 5 systems
           that have operating costs greater than $500,000. For fiscal year
           2009, all business systems are to be reviewed. According to Air
           Force officials, implementing a phased approach allows time to
           adopt the investment management guidance set forth in our ITIM
           framework.^72 While not specifically required by the act, Air
           Force officials told us that the investment management practices
           that it intends to put in place for its business systems will also
           be leveraged for non-business system investments (e.g.,
           warfighting systems). We currently have ongoing work to review the
           extent to which the Air Force's business systems investment
           structures and processes comport with our ITIM framework.
		   
^68Approval authorities (the USD(AT&L); the Under Secretary of Defense
(Comptroller); the Under Secretary of Defense for Personnel and Readiness;
the ASD(NII)/CIO; and the Deputy Secretary of Defense or an Under
Secretary of Defense, as designated by the Secretary of Defense) are
responsible for the review, approval, and oversight of business systems
and must establish investment review processes for systems under their
cognizance.

^69A key condition identified in the act includes certification by
designated approval authorities that the defense business system
modernization is (1) in compliance with the enterprise architecture; (2)
necessary to achieve critical national security capability or address a
critical requirement in an area such as safety or security; or (3)
necessary to prevent a significant adverse effect on a project that is
needed to achieve an essential capability, taking into consideration the
alternative solutions for preventing such an adverse effect.

^7031 U.S.C. S 1341(a)(1)(A); see 10 U.S.C. S 2222(b).	

             Army	   

           Army officials report that their department's primary emphasis has
           been on reviewing its business system investments with funding in
           excess of $1 million (i.e., tiers 1 through 3 business systems).
           However, officials told us that they intend to develop a list of
           all business systems that require annual reviews through January
           2008 to guide future efforts. Currently, the Army reports an
           inventory of 873 business systems, of which 108 are systems with
           development/modernization funding in excess of $1 million, and
           another 765 business system investments with funding below $1
           million, including 62 with no development/modernization funding.
		   
		     Navy

           Navy officials report that their department is in the process of
           conducting reviews of its 697 business systems in accordance with
           the requirements of the act, although the processes being used are
           still evolving. For example, Navy officials stated that the focus
           of the reviews has thus far been on those systems with
           development/modernization funding over $1 million. According to
           DOD, for fiscal years 2006 and 2007, 54 business systems were
           certified by the IRBs and approved by the DBSMC. Further, they
           said that greater coordination with DOD functional areas (e.g.,
           logistics) and ASD(NII)/CIO is needed to improve the control and
           accountability over its business system investments. We currently
           have ongoing work to review the extent to which the Navy's
           business systems investment structures and processes comport with
           our ITIM framework.
		   
^71According to Air Force officials, tier 5 systems only spend current
service funds.

^72GAO-04-394G.

           DOD Continues to Implement Our Prior Recommendations

           The act's requirements concerning the architecture, transition
           plan, budgetary disclosure, and investment management structures
           and processes--as discussed earlier--are consistent with the 35
           recommendations that we have made since 2001, to assist the
           department in developing a well-defined and useful BEA and using
           it to gain control over its ongoing business system investments.
           To its credit, DOD largely agreed with these recommendations and
           stated its commitment to implement them. In May 2006, we reported
           that the department had taken steps to fully implement 21 of the
           recommendations, while 14 had yet to be fully implemented.^73

           Since then, 10 of the 14 have either been largely implemented or
           have been subsumed by our more recent recommendations and thus we
           are considering them closed. (See app. II for details on the
           status of these 14 recommendations; see app. III for a detailed
           listing of the additional recommendations that we have made since
           our last annual report under the act.) For example, DOD has
           addressed the core elements in our Enterprise Architecture
           Management Maturity Framework^74 relative to its corporate BEA. In
           particular, it has established a chief architect who is
           responsible for developing the corporate BEA and ensuring that the
           BEA depicts the "As Is" and "To Be" environments in terms of
           business, performance, information/data, application/service,
           technology, and security. As another example, the department has
           taken steps to make effective use of the results of its BEA
           independent verification and validation contractor on prior
           versions of the architecture. As we have previously reported,
           using an independent verification and validation agent is a
           recognized best practice because it provides internal and external
           oversight bodies important information on architecture and
           transition plan quality and governance. By having and using an
           independent verification and validation agent, organizations can
           disclose to oversight bodies independent assessments of
           architecture and transition plan quality, to include completeness,
           consistency, understandability, and usability, which the
           department has yet to provide in its annual reports.

           With respect to the remaining 4 of the 14 recommendations, actions
           are under way that are intended to implement them. For example, in
           response to our recommendation to develop a BEA program management
           plan^75 that defines what the department's incremental
           improvements to the architecture and transition plan will be, and
           how and when they will be accomplished, the BTA has developed the
           Business Transformation Guidance, which describes the high-level
           process by which incremental improvements are identified and
           eventually incorporated into the architecture. In addition, BTA
           officials stated that they are developing a BEA Concept of
           Operations, which is to describe high-level milestones for the
           BEA's use.
		   
^73GAO-06-658.

^74GAO-03-584G.

           As another example, the BTA has established a communications team
           that is responsible for achieving strategic communications
           objectives and promoting external awareness of the department's
           vision, mission, and progress, and BTA officials told us that this
           team is in the process of developing a communications plan.
           According to the officials, these efforts will address our
           recommendation for the BEA program to be supported by a proactive
           marketing and communication program.^76

           According to the Deputy Under Secretary of Defense (Business
           Transformation), the department is committed to addressing all of
           our open recommendations. It is important that the department move
           swiftly in doing so because these recommendations are aimed at
           strengthening architecture (and transition planning) management
           activities and controlling ongoing and planned business system
           investments. Until it does, the department will be challenged in
           its ability to effectively guide and constrain the billions of
           dollars it invests annually in thousands of business system
           investments.
		   
           Conclusions

           Since our last legislatively mandated report on DOD's compliance
           with section 332 of the National Defense Authorization Act for
           Fiscal Year 2005, DOD has continued to make important progress in
           defining and implementing institutional modernization management
           controls and business systems budgetary disclosure, but much
           remains to be accomplished. In particular, the department has yet
           to extend and evolve its corporate BEA through the development of
           aligned subordinate architectures for each of its component
           organizations, and while it has developed a strategy for
           federating the BEA in this manner, this strategy lacks the detail
           needed for it to be effectively implemented. Compounding this
           situation is the known immaturity of the military service
           architecture efforts, as well as DOD's corporate approach to
           business system investment management not being governed by the
           range of defined policies and procedures that are associated with
           effective investment selection, control, and evaluation. Moreover,
           the military departments' investment review processes are still
           evolving. These architecture and investment management limitations
           continue to put the billions of dollars that DOD spends each year
           on its thousands of business system investments at risk.
		   
^75GAO-06-658.

^76GAO-03-458.

           The recommendations that we have made since we issued our last
           annual report under the act are aimed at addressing these
           architecture and investment management challenges. Given the
           demonstrated commitment of DOD leadership to improving its
           business systems modernization efforts and its recent
           responsiveness to our prior recommendations, we are optimistic
           concerning the likelihood that the department will continue to
           make progress on these fronts.

           Development of a well-defined federated architecture for the
           business mission area and the definition of effective business
           system investment management policies and procedures across all
           levels of the department are critically important in addressing
           the DOD business system modernization high-risk area. However, the
           more formidable challenge facing the department is how well it
           actually implements the architecture and investment management
           controls over the years ahead on each and every business system
           investment. While not a guarantee, development of a federated BEA,
           including a transition plan(s), and effective institutional
           business system investment management processes can go a long way
           in addressing this longer-term challenge. In this regard, it is
           important for the department to keep congressional defense
           committees fully informed about its progress in federating the DOD
           corporate BEA, to include the maturity of component organization
           architecture efforts and the related transition plan(s).
		   
		   Recommendation for Executive Action

           To facilitate congressional oversight and promote departmental
           accountability, we recommend that the Secretary of Defense direct
           the Deputy Secretary of Defense, as the chair of the DBSMC, to
           include in DOD's annual report to Congress on compliance with the
           section 332 of Fiscal Year 2005 National Defense Authorization
           Act, the results of assessments by its BEA independent
           verification and validation contractor of the completeness,
           consistency, understandability, and usability of its federated
           family business mission area architectures, including the
           associated transition plan(s).
		   
		   Agency Comments

           In written comments on a draft of this report, signed by the
           Deputy Under Secretary of Defense (Business Transformation) and
           reprinted in appendix IV, the department agreed with our
           recommendation.

           We are sending copies of this report to interested congressional
           committees; the Director, Office of Management and Budget; the
           Secretary of Defense; the Deputy Secretary of Defense; the Under
           Secretary of Defense for Acquisition, Technology, and Logistics;
           the Under Secretary of Defense (Comptroller); the Assistant
           Secretary of Defense (Networks and Information Integration)/Chief
           Information Officer; the Under Secretary of Defense (Personnel and
           Readiness); and the Director, Defense Finance and Accounting
           Service. Copies of this report will be made available to other
           interested parties upon request. This report will also be
           available at no charge on our Web site at http://www.gao.gov .

           If you or your staffs have any questions on matters discussed in
           this report, please contact me at (202) 512-3439 or [email protected],
           or McCoy Williams at (202) 512-9095 or [email protected]. Contact
           points for our Offices of Congressional Relations and Public
           Affairs may be found on the last page of this report. GAO staff
           who made major contributions to this report are listed in appendix
           V.

           Randolph C. Hite
		   Director
		   Information Technology Architecture and Systems Issues

           McCoy Williams
		   Director
		   Financial Management Assurance

           List of Committees

           The Honorable Carl Levin
		   Chairman
		   The Honorable John McCain
           Ranking Member
		   Committee on Armed Services
		   United States Senate

           The Honorable Daniel Inouye
		   Chairman
		   The Honorable Ted Stevens
           Ranking Member
		   Committee on Appropriations
		   United States Senate

           The Honorable Ike Skelton
		   Chairman
		   The Honorable Duncan Hunter
           Ranking Member
		   Committee on Armed Services
		   House of Representatives

           The Honorable John P. Murtha
		   Chairman
		   The Honorable C.W. Bill Young
		   Ranking Member
		   Committee on Appropriations
		   House of Representatives
		   
		   Appendix I: Objectives, Scope, and Methodology

           Our objectives were to (1) assess the actions by the Department of
           Defense (DOD) to comply with the requirements of section 2222 of
           Title 10, U.S. Code,^1 and (2) determine the extent to which DOD
           has addressed our prior open recommendations for
           institutionalizing key business system modernization management
           controls.

           For our first objective, we focused on five of the six
           requirements in section 2222, and related best practices contained
           in federal guidance, that we identified in our last annual report
           under the act as not being fully satisfied.^2 Generally, these
           five requirements are (1) development of a business enterprise
           architecture (BEA), (2) development of a transition plan for
           implementing the BEA, (3) inclusion of business systems
           information in DOD's budget submission, (4) establishment of
           business systems investment review processes and structures, and
           (5) approval of defense business systems investments with
           obligations in excess of $1 million. (See the Background section
           of this report for additional information on the act's
           requirements.) We did not include the sixth requirement because
           our last annual report under the act shows that it had been
           satisfied. Our methodology relative to each of the five
           requirements is as follows.

           o To determine whether the BEA addressed the requirements
           specified in the act, and related guidance, we analyzed version
           4.1 of the BEA, which was released on March 15, 2007, relative to
           the act's specific architectural requirements and related guidance
           that our last annual report under the act identified as not being
           met. We also reviewed version 4.1 to confirm whether statements
           made in DOD's March 15, 2007, annual report about the BEA's
           content were accurate. Also, we reviewed and leveraged the
           applicable results contained in our recent reports on major
           departments' and agencies' enterprise architecture programs and on
           DOD's BEA federation strategy.^3 
           o To determine whether the enterprise transition plan (ETP)
           addressed the requirements specified in the act, we reviewed the
           updated version of the ETP, which was released on March 15, 2007,
           relative to the act's specific transition plan requirements and
           related guidance that our last annual report under the act
           identified as not being met. We also reviewed the ETP to confirm
           that statements in DOD's March 15, 2007, annual report about the
           content of the ETP were accurate.

^1Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. 108-375, S 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 2004)
(codified in part at 10 U.S.C. S 2222).

^2GAO, Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, GAO-06-658 (Washington,
D.C.: May 15, 2006).

^3GAO, Business Systems Modernization: Strategy for Evolving DOD's
Business Enterprise Architecture Offers Conceptual Approach, but Execution
Details Needed, GAO-07-451 (Washington, D.C.: Apr. 16, 2007); and
Enterprise Architecture: Leadership Remains Key to Establishing and
Leveraging Architectures for Organizational Transformation, GAO-06-831
(Washington, D.C.: Aug. 14, 2006).

           o To determine whether DOD's fiscal year 2008 information
           technology budget submission was prepared in accordance with the
           criteria set forth in the act, we reviewed and analyzed the
           department report entitled Report on Defense Business System
           Modernization FY 2005 National Defense Authorization Act, Section
           332, prepared in February 2007 and compared the information
           obtained to the specific requirements in the act.

           o To determine whether DOD has established investment review
           structures and processes, we focused the act's requirements that
           our last annual report under the act identified as not being met,
           obtaining documentation and interviewing cognizant DOD officials
           about efforts to establish the one Investment Review Board (IRB)
           specified in the act that had yet to be established. We also
           reviewed and leveraged our recent report that assessed DOD's
           corporate investment approach to managing business system
           investments against relevant federal guidance.^4

           o To determine whether the department was reviewing and approving
           business system investments exceeding $1 million, we obtained the
           list of business system investments certified by the IRBs and
           approved by the Defense Business Systems Management Committee from
           the Business Transformation Agency (BTA). We then compared the
           detailed information provided with the summary information
           contained in the department's March 15, 2007, report to the
           congressional defense committees to identify any anomalies. We
           also met with representatives from the Air Force, the Army, and
           the Navy to ascertain the specific actions that were taken (or
           planned to be taken) in order to perform the annual systems
           reviews as required by the act.

           To determine the extent to which DOD has addressed our prior open
           recommendations, we focused on the 14 recommendations that we
           identified in our last annual report under the act as not being
           implemented. We did not examine the recommendations for
           establishing and implementing key business system modernization
           management controls that we made since this last annual report
           because sufficient time had yet to elapse for the department to
           have addressed them. (See app. III for a list of the
           recommendations made since our last annual report under the act.)
           In reviewing the 14 recommendations, we obtained and analyzed
           documentation relative to corrective actions taken and planned.
           Documentation that we reviewed included the DOD's March 15, 2007,
           annual report, updated transition plan, and BEA version 4.1. We
           also compared a range of other program documentation, such as
           program policies and procedures and configuration plan, to
           relevant elements in our Enterprise Architecture Management
           Maturity Framework.^5 Further, we reviewed documentation regarding
           DOD verification and validation contractor activities and the
           BTA's human capital strategy. In addition, we reviewed the
           guidance establishing the IRBs and describing the investment
           review, certification, and approval process.
		   
^4GAO, Business Systems Modernization: DOD Needs to Fully Define Policies
and Procedures for Institutionally Managing Investments, GAO-07-538
(Washington, D.C.: May 11, 2007).

^5GAO, Information Technology: A Framework for Assessing and Improving
Enterprise Architecture Management (Version 1.1),  GAO-03-584G
(Washington, D.C.: April 2003).

           We did not independently validate the reliability of the cost and
           budget figures provided by DOD because the specific amounts were
           not relevant to our findings. We conducted our work at DOD
           headquarters in Arlington, Virginia, from March through May 2007
           in accordance with generally accepted government auditing
           standards.
		   
Appendix II: Status of Prior Recommendations Identified as Open inGAO's
Prior Annual Report under the Act

                              Implemented/                                    
                                 Closed                                       
GAO report information                                                     
and recommendation        Yes In process  GAO assessment                   

[43]GAO-01-525 :                                                           
Information Technology:                                                    
Architecture Needed to                                                     
Guide Modernization of                                                     
DOD's Financial                                                            
Operations,  May 17,                                                       
2001.                                                                      

(1) Until an enterprise   X               This recommendation has been     
architecture is                           subsumed by more recent          
developed and the                         recommendations concerning the   
Council is positioned to                  department's efforts to federate 
serve as Department of                    the corporate business           
Defense's (DOD)                           enterprise architecture (BEA),   
financial management                      mature DOD component             
investment review board                   organization architectures, and  
as recommended, the                       establish policies and           
Secretary of Defense                      procedures for effective         
limit DOD components'                     corporate business system        
financial management                      investment management. (See app. 
investments to the                        III for these more recent        
deployment of systems                     recommendations.)                
that have already been                                                     
fully tested and involve                                                   
no additional                                                              
development or                                                             
acquisition costs;                                                         
stay-in-business                                                           
maintenance needed to                                                      
keep existing systems                                                      
operational; management                                                    
controls needed to                                                         
effectively invest in                                                      
modernized systems; and                                                    
new systems or existing                                                    
system changes that are                                                    
congressionally directed                                                   
or are relatively small,                                                   
cost-effective, and low                                                    
risk and can be                                                            
delivered in a                                                             
relatively short time                                                      
frame.                                                                     

[44]GAO-03-458 : DOD                                                       
Business Systems                                                           
Modernization:                                                             
Improvements to                                                            
Enterprise Architecture                                                    
Development and                                                            
Implementation Efforts                                                     
Needed,  February 28,                                                      
2003.                                                                      

(1) The Secretary of          X           The Business Transformation      
Defense ensure that the                   Agency (BTA) has established a   
enterprise architecture                   communications team that is      
program is supported by                   responsible for achieving        
a proactive marketing                     strategic communications         
and communication                         objectives and promoting         
program.                                  external awareness of the        
                                             department's vision, mission,    
                                             and progress. However, the       
                                             department has yet to develop a  
                                             communication plan that adheres  
                                             to criteria set forth by the     
                                             best practices, to include an    
                                             explanation of roles and         
                                             responsibilities and details     
                                             regarding evaluation, metrics,   
                                             and feedback. BTA officials told 
                                             us that such a plan is currently 
                                             in development.                  

[45]GAO-03-1018 : DOD                                                      
Business Systems                                                           
Modernization: Important                                                   
Progress Made to Develop                                                   
Business Enterprise                                                        
Architecture, but Much                                                     
Work Remains, September                                                    
19, 2003.                                                                  

(1) The Secretary of      X               The BTA has largely addressed    
Defense or his                            the 31 core elements in our      
appropriate designee                      Enterprise Architecture          
implement the core                        Management Maturity Framework in 
elements in our                           its corporate BEA, which is the  
Enterprise Architecture                   intended focus of the            
Framework for Assessing                   recommendation. For example, the 
and Improving Enterprise                  BTA has established a chief      
Architecture Management                   architect who is responsible for 
that we identify in this                  developing and maintaining the   
report as not satisfied,                  corporate BEA and the version    
including ensuring that                   4.1 of the BEA largely provides  
minutes of the meetings                   a depiction of both the "As Is"  
of the executive body                     and "To Be" environments in      
charged with directing,                   terms of business, performance,  
overseeing, and                           information/data,                
approving the                             application/service, technology, 
architecture are                          and security. (See app. III for  
prepared and maintained.                  recent recommendations aimed at  
                                             having the military departments  
                                             address these core elements.)    

(2) The Secretary of      X               The BTA has largely addressed    
Defense or his                            these 29 key elements relative   
appropriate designee                      to its corporate BEA, which is   
update version 1.0 of                     the intended focus of the        
the architecture to                       recommendation. For example,     
include the 29 key                        version 4.1 of the BEA contains  
elements governing the                    enterprise-level "As Is"         
"As Is" architectural                     information to support business  
content that our report                   capability gap analyses. In      
identified as not being                   addition, the architecture       
fully satisfied.                          includes "As Is" information for 
                                             five of the six business         
                                             enterprise priorities and "As    
                                             Is" information for enterprise   
                                             systems, such as the Wide-area   
                                             Workflow system. (See app. III   
                                             for recent recommendations aimed 
                                             at effectively federating the    
                                             corporate BEA to DOD component   
                                             organizations.)                  

(3) The Secretary of      X               The BTA has largely addressed    
Defense or his                            these 30 key elements relative   
appropriate designee                      to its corporate BEA, which is   
update version 1.0 of                     the intended focus of the        
the architecture to                       recommendation. For example,     
include the 30 key                        version 4.1 of the BEA           
elements governing the                    identifies activities performed  
"To Be" architectural                     at each location/organization    
content that our report                   and indicates which              
identified as not being                   organization(s) is or will be    
fully satisfied.                          involved in each activity.       
                                             Furthermore, it includes common  
                                             business rules (e.g., "each      
                                             request for commercial export of 
                                             DOD technology must be processed 
                                             within 30 days upon receipt of   
                                             request from the Department of   
                                             State or the Department of       
                                             Commerce") to facilitate         
                                             consistent implementation of the 
                                             architecture. (See app. III for  
                                             recent recommendations aimed at  
                                             effectively federating the       
                                             corporate BEA to DOD component   
                                             organizations.)                  

(4) The Secretary of      X               The BTA has largely addressed    
Defense or his                            this recommendation for its      
appropriate designee                      corporate or enterprise          
update version 1.0 of                     transition plan, which is the    
the architecture to                       intended focus of the            
include (a) the 3 key                     recommendation. For example, the 
elements governing the                    latest version of the transition 
transition plan content                   plan now documents how BEA       
that our report                           elements (e.g., specific         
identified as not being                   business capability              
fully satisfied and (b)                   improvements) provide solutions  
those system investments                  to significant DOD issues or     
that will not become                      business capability gaps (e.g.,  
part of the "To Be"                       mission needs, materiel          
architecture, including                   weaknesses). It also provides    
time frames for phasing                   performance information of DOD   
out those systems.                        transformation at both the       
                                             enterprise level and component   
                                             level, including performance     
                                             metrics and milestones. (See     
                                             app. III for recent              
                                             recommendations aimed at         
                                             effectively federating the       
                                             corporate BEA, to include the    
                                             transition plan, to DOD          
                                             component organizations.)        

(5) The Secretary of      X               The verification and validation  
Defense or his                            contractor reports that all of   
appropriate designee                      these comments on versions 3.0   
update version 1.0 of                     and prior versions have been     
the architecture to                       addressed.                       
address comments made by                                                   
the verification and                                                       
validation contractor.                                                     

(6) The Secretary of      X               This recommendation has been     
Defense or his                            subsumed by a later              
appropriate designee                      recommendation in [46]GAO-06-658 
develop a well-defined,                   .                                
near-term plan for                                                         
extending and evolving                                                     
the architecture and                                                       
ensure that this plan                                                      
includes addressing our                                                    
recommendations,                                                           
defining roles and                                                         
responsibilities of all                                                    
stakeholders involved in                                                   
extending and evolving                                                     
the architecture,                                                          
explaining dependencies                                                    
among planned                                                              
activities, and defining                                                   
measures of activity                                                       
progress.                                                                  

(7) The Secretary of          X           According to BTA officials, the  
Defense or his                            department is continuing to      
appropriate designee                      assess and clarify the role of   
limit the pilot projects                  pilot projects and a policy is   
to small, low-cost,                       to be developed relative to      
low-risk prototype                        them. However, they did not      
investments that are                      provide specific plans and time  
intended to provide                       frames for developing and        
knowledge needed to                       implementing this policy.        
extend and evolve the                                                      
architecture, and are                                                      
not to acquire and                                                         
implement production                                                       
version system solutions                                                   
or to deploy an                                                            
operational system                                                         
capability.                                                                

[47]GAO-05-381 : DOD                                                       
Business Systems                                                           
Modernization: Billions                                                    
Being Invested without                                                     
Adequate Oversight,                                                        
April 29, 2005.                                                            

(1) The Secretary of      X               DOD's March 15, 2007, annual     
Defense direct that the                   report to the congressional      
DBSMC develop a                           defense committees identifies    
comprehensive plan that                   specific actions the department  
addresses implementation                  is taking to address our open    
of our previous                           recommendations. The March       
recommendations related                   report noted that BTA has        
to the BEA and the                        overall responsibility for       
control and                               ensuring that remaining open     
accountability over                       recommendations are adequately   
business systems                          addressed.                       
investments (at a                                                          
minimum, the plan should                                                   
assign responsibility                                                      
and estimated time                                                         
frames for completion).                                                    

(2) The Secretary of      X               DOD's March 15, 2006, and March  
Defense direct that the                   15, 2007, reports to             
comprehensive plan we                     congressional committees         
recommend be                              included steps that DOD is       
incorporated into the                     taking or plans to take to       
department's second                       address our open                 
annual report due March                   recommendations.                 
15, 2006, to the defense                                                   
congressional                                                              
committees, as required                                                    
by the Fiscal Year 2005                                                    
Defense Authorization                                                      
act to help facilitate                                                     
congressional oversight.                                                   

[48]GAO-05-702 : DOD                                                       
Business Systems                                                           
Modernization:                                                             
Long-standing Weaknesses                                                   
in Enterprise                                                              
Architecture Development                                                   
Need to Be Addressed,                                                      
July 22, 2005.                                                             

(1) The Secretary of      X               BTA and BEA program              
Defense should direct                     documentation reflects           
the Deputy Secretary of                   activities and steps taken or    
Defense, as the chair of                  planned to address our           
the DBSMC and in                          recommendations relative to BEA  
collaboration with DBSMC                  content and management.          
members, to ensure that                   Furthermore, the department has  
each of our                               stated its commitment to doing   
recommendations related                   so in its annual reports to the  
to the BEA management                     congressional defense            
and content are                           committees.                      
reflected in the plans                                                     
and commitments.                                                           

(2) The Secretary of          X           On March 21, 2007, the BTA       
Defense should direct                     released its Human Capital       
the Deputy Secretary of                   Strategic Plan 2007-2009, which  
Defense, as the chair of                  identifies BTA's goals for human 
the DBSMC and in                          capital development and          
collaboration with DBSMC                  workforce planning. This         
members, to ensure that                   strategy provides an overview of 
plans and commitments                     the current workforce status in  
provide for effective                     relation to those goals and      
BEA workforce planning,                   identifies several key           
including assessing                       activities for how to proceed in 
workforce knowledge and                   order to achieve the goals. In   
skills needs,                             addition, the strategy includes  
determining existing                      an initial implementation        
workforce capabilities,                   roadmap with timelines for key   
identifying gaps, and                     activities. According to BTA     
filling these gaps.                       officials, the detailed plans    
                                             for accomplishing key activities 
                                             will be contained in BTA's Human 
                                             Capital Implementation Plan,     
                                             which has yet to be released.    

[49]GAO-06-658 :                                                           
Business Systems                                                           
Modernization: DOD                                                         
Continues to Improve                                                       
Institutional Approach,                                                    
but Further Steps                                                          
Needed, May 15, 2006.                                                      

(1) The Secretary of          X           BTA has developed several        
Defense direct the                        documents that are intended to   
Deputy Secretary of                       begin addressing this            
Defense, as the chair of                  recommendation. For example, it  
the DBSMC, to submit an                   has developed the Business       
enterprise architecture                   Transformation Guidance, which   
program management plan                   describes the high-level process 
to defense congressional                  by which incremental             
committees that defines                   improvements are identified and  
what the department's                     eventually incorporated into the 
incremental improvements                  BEA. In addition, BTA officials  
to the architecture and                   told us that they are developing 
transition plan will be,                  a BEA Concept of Operations,     
and how and when they                     which is to describe high-level  
will be accomplished,                     milestones required to address   
including what (and                       the architecture's use (e.g.,    
when) architecture and                    investment management, strategic 
transition plan scope                     decision making, oversight,      
and content and                           system implementation, and       
architecture compliance                   business case development).      
criteria will be added                    Notwithstanding these steps, the 
into which versions; the                  department has yet to develop an 
plan should also include                  architecture program management  
an explicit purpose and                   plan that we have recommended.   
scope for each version                    (See app. III for a more recent  
of the architecture,                      recommendation that augments     
along with milestones,                    this recommendation.)            
resource needs, and                                                        
performance measures for                                                   
each planned version.                                                      

           Source: GAO.

           Note: See GAO, Business Systems Modernization: DOD Continues to
           Improve Institutional Approach, but Further Steps Needed,
           GAO-06-658 (Washington, D.C.: May 15, 2006).
		   
Appendix III: Other Open Recommendations on Business Architectures,
Federation Strategy, and Investment Management

GAO report information and recommendation                                  

[50]GAO-06-831 : Enterprise Architecture: Leadership Remains Key to        
Establishing and Leveraging Architectures for Organizational               
Transformation, August 14, 2006.                                           

      1. The Secretary of Defense ensure that the Department of Defense (DOD) 
      - Global Information Grid enterprise architecture program develops and  
      implements plans for fully satisfying each of the conditions in our     
      enterprise architecture management maturity framework.                  

      2. The Secretary of Defense ensure that the Department of the Air Force 
      enterprise architecture program develops and implements plans for fully 
      satisfying each of the conditions in our enterprise architecture        
      management maturity framework.                                          

      3. The Secretary of Defense ensure that the Department of the Army      
      enterprise architecture program develops and implements plans for fully 
      satisfying each of the conditions in our enterprise architecture        
      management maturity framework.                                          

      4. The Secretary of Defense ensure that the Department of the Navy      
      enterprise architecture program develops and implements plans for fully 
      satisfying each of the conditions in our enterprise architecture        
      management maturity framework.                                          

[51]GAO-07-451 : Business Systems Modernization: Strategy for Evolving     
DOD's Business Enterprise Architecture Offers a Conceptual Approach, but   
Execution Details Are Needed, April 16, 2007.                              

      1. The Secretary of Defense direct the Deputy Secretary of Defense, as  
      the chair of the Defense Business Systems Management Committee (DBSMC), 
      to ensure that the appropriate DOD organizations submit a business      
      enterprise architecture (BEA) development management plan that          
      describes, at a minimum, how the business mission area architecture     
      federation will be governed; how the business mission area federation   
      strategy alignment with the DOD enterprise architecture federation      
      strategy will be achieved; how component business architectures'        
      alignment with incremental versions of the BEA will be achieved; how    
      shared services will be identified, exposed, and subscribed to; and     
      what milestones will be used to measure progress and results.           

[52]GAO-07-538 : Business Systems Modernization: DOD Needs to Fully Define 
Policies and Procedures for Institutionally Managing Investments, May 11,  
2007.                                                                      

      1. The Secretary of Defense should direct the Deputy Secretary of       
      Defense, as the chair of the DBSMC, to ensure that well-defined and     
      disciplined business system investment management policies and          
      procedures are developed and issued. At a minimum, this should include  
      project-level management policies and procedures that address the       
      following five areas:                                                   
                                                                              
              o instituting the investment boards, including assigning the    
              investment boards responsibility, authority, and accountability 
              for programs throughout the investment life cycle and           
              specifying how the business investment management system is     
              coordinated with the Joint Capabilities Integration and         
              Development System, the Planning, Programming, Budgeting, and   
              Execution system, and the Defense Acquisition System;           
                                                                              
              o selecting new investments, including specifying how cost,     
              schedule, and benefit data are to be used in making             
              certification decisions; defining the criteria used to select   
              investments as enterprisewide; and establishing consistent and  
              effective guidance for BEA compliance;                          
                                                                              
              o reselecting ongoing investments, including specifying how     
              cost, schedule, and performance data are to be used in the      
              annual review process and providing for the reselection of      
              investments that are in operations and maintenance;             
                                                                              
              o integrating funding with the process of selecting an          
              investment, including specifying how the DBSMC and the          
              investment review boards use funding information in carrying    
              out decisions on system certification and approvals; and        
                                                                              
              o overseeing IT projects and systems, including providing       
              sufficient oversight and visibility into component-level        
              investment management activities.                               

      2. The Secretary of Defense should direct the Deputy Secretary of       
      Defense, as the chair of the DBSMC, to ensure that well-defined and     
      disciplined business system investment management policies and          
      procedures are developed and issued. These policies and procedures      
      should also include portfolio-level management policies and procedures  
      that address the following four areas:                                  
                                                                              
              o creating and modifying information technology portfolio       
              selection criteria for business system investments;             
                                                                              
              o analyzing, selecting, and maintaining business system         
              investment portfolios;                                          
                                                                              
              o reviewing, evaluating, and improving the performance of its   
              portfolio(s) by using project indicators such as cost,          
              schedule, and risk; and                                         
                                                                              
              o conducting postimplementation reviews for all investment      
              tiers and directing the investment boards who are accountable   
              for corporate business system investments, to consider the      
              information gathered and to develop lessons learned from these  
              reviews.                                                        

           Source: GAO.
		   
		   Appendix IV: Comments from the Department of Defense
		   
		   Appendix V: GAO Contacts and Staff Acknowledgments
		   
		   GAO Contacts

           Randolph C. Hite (202) 512-3439 or [email protected] McCoy
           Williams (202) 512-9095 or [email protected]
		   
		   Staff Acknowledgments

           In addition to the contact persons named above, key contributors
           to this report were Beatrice Alff, Karl Essig, Nancy Glover,
           Michael Holland, Neelaxi Lakhmani (Assistant Director), Anh Le,
           Evelyn Logue, Jacqueline Mai, John Martin, Darby Smith (Assistant
           Director), Debra Rucker, and Jennifer Stavros-Turner.
		   
		   GAO's Mission

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.
		   
		   Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each
           weekday, GAO posts newly released reports, testimony, and
           correspondence on its Web site. To have GAO e-mail you a list of
           newly posted products every afternoon, go to www.gao.gov and
           select "Subscribe to Updates."
		   
		   Order by Mail or Phone

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000
		   TDD: (202) 512-2537
		   Fax: (202) 512-6061
		   
		   To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm
		   E-mail: [email protected]
		   Automated answering system: (800) 424-5454 or (202) 512-7470
		   
		   Congressional Relations

           Gloria Jarmon, Managing Director, [email protected] (202)
           512-4400 U.S. Government Accountability Office, 441 G Street NW,
           Room 7125 Washington, D.C. 20548
		   
		   Public Affairs

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

(310643)

www.gao.gov/cgi-bin/getrpt?GAO-07-733 .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact Randolph C. Hite at (202) 512-3439 or
[email protected].

Highlights of [54]GAO-07-733 , a report to congressional committees

May 2007

DOD BUSINESS SYSTEMS MODERNIZATION

Progress Continues to Be Made in Establishing Corporate Management
Controls, but Further Steps Are Needed

In 1995, GAO first designated the Department of Defense's (DOD) business
systems modernization program as "high risk," and GAO continues to do so
today. To assist in addressing this high-risk area, the Fiscal Year 2005
National Defense Authorization Act contains provisions that are consistent
with prior GAO recommendations. Further, the act requires the department
to submit annual reports to its congressional committees on its compliance
with these provisions and it directs GAO to review each report. In
response, GAO assessed DOD's actions to address (1) requirements in the
act and (2) GAO's recommendations that it reported as open in its prior
annual report under the act. In doing so, GAO reviewed documentation and
interviewed officials relative to the act and related guidance.

[55]What GAO Recommends

GAO is recommending that future DOD annual reports include an assessment
by its independent verification and validation agent of the quality of the
department's federated family of architectures, including the associated
transition plan(s). In written comments, DOD agreed with GAO's
recommendation.

As part of DOD's recent efforts to strengthen management of its business
systems modernization program, it has taken steps over the last year to
build on past efforts and further comply with the act's requirements and
relevant guidance. However, additional steps are needed. For example,

           o The latest version of DOD's business enterprise architecture now
           contains information about the department's "As Is" corporate
           environment, which is important for effective transition planning.
           Further, this version represents a major step in building the
           family of architectures that are needed to fully satisfy the act
           and effectively guide and constrain thousands of system
           investments across all DOD component organizations. Nevertheless,
           GAO's reports since its last annual report under the act show that
           the strategy for extending the business enterprise architecture to
           defense components needs further definition to make it executable
           and the maturity of key components' architecture programs is
           limited. GAO has recently made recommendations to address these
           challenges.
           o The updated enterprise transition plan, which is an essential
           component of an enterprise architecture, continues to identify
           systems and initiatives that are to fill business capability gaps
           and address DOD-wide and component business priorities contained
           in the business enterprise architecture. However, it does not
           include investments for all components and does not reflect key
           factors associated with properly sequencing planned investments,
           such as dependencies among investments and the capability to
           execute the plan, which GAO's existing recommendations provide for
           addressing.
           o DOD has established and begun implementing the investment review
           structures and processes that are consistent with the act.
           However, it has yet to do so in a manner that is consistent with
           relevant guidance. In particular, it has yet to fully define the
           related policies and procedures needed to effectively execute both
           project-level and portfolio-based information technology
           investment management practices. GAO has recently made
           recommendations to address these shortcomings.

DOD also continues to make progress in implementing GAO recommendations
aimed at strengthening business systems modernization management. In
particular, of the 14 open recommendations that GAO identified in its
prior annual report under the act, 10 have either been largely implemented
or subsumed by the more recent recommendations cited above. For example,
DOD has implemented GAO's recommendations aimed at effectively using the
assessments that have been performed by DOD's independent verification and
validation contractor. Such assessments provide important information for
department and congressional oversight bodies to use to better ensure the
definition and institutionalization of the corporate management controls
that GAO has cited as essential to addressing the DOD business systems
modernization high-risk area. The department's annual reports have not
included such assessments.

References

Visible links
  33. http://www.gao.gov/cgi-bin/getrpt?GAO-07-733
  43. http://www.gao.gov/cgi-bin/getrpt?GAO-01-525
  44. http://www.gao.gov/cgi-bin/getrpt?GAO-03-458
  45. http://www.gao.gov/cgi-bin/getrpt?GAO-03-1018
  46. http://www.gao.gov/cgi-bin/getrpt?GAO-06-658
  47. http://www.gao.gov/cgi-bin/getrpt?GAO-05-381
  48. http://www.gao.gov/cgi-bin/getrpt?GAO-05-702
  49. http://www.gao.gov/cgi-bin/getrpt?GAO-06-658
  50. http://www.gao.gov/cgi-bin/getrpt?GAO-06-831
  51. http://www.gao.gov/cgi-bin/getrpt?GAO-07-451
  52. http://www.gao.gov/cgi-bin/getrpt?GAO-07-538
  54. http://www.gao.gov/cgi-bin/getrpt?GAO-07-733
*** End of document. ***