DOD Business Systems Modernization: Military Departments Need to 
Strengthen Management of Enterprise Architecture Programs	 
(12-MAY-08, GAO-08-519).					 
                                                                 
In 1995, GAO designated Department of Defense (DOD) business	 
systems modernization as a high-risk program, and the program	 
remains on the high-risk list today. A key to successful systems 
modernization is having and using an enterprise architecture as  
an authoritative frame of reference, or blueprint, for system	 
investment decisions. To assist DOD in modernizing its business  
systems, Congress passed legislation consistent with prior GAO	 
recommendations for DOD to develop and implement a business	 
enterprise architecture (BEA). In response, DOD developed a	 
corporate BEA that it intends to federate, or extend, to the	 
military departments and defense agencies. To support GAO's	 
legislative mandate to review DOD's BEA, GAO evaluated the status
of the Air Force, Navy, and Army architecture programs. To	 
accomplish this, GAO used its Enterprise Architecture Management 
Maturity Framework and associated evaluation method.		 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-519 					        
    ACCNO:   A82090						        
  TITLE:     DOD Business Systems Modernization: Military Departments 
Need to Strengthen Management of Enterprise Architecture Programs
     DATE:   05/12/2008 
  SUBJECT:   Defense capabilities				 
	     Defense cost control				 
	     Defense operations 				 
	     Enterprise architecture				 
	     Evaluation criteria				 
	     Federal agency reorganization			 
	     Information technology				 
	     Internal controls					 
	     Military forces					 
	     Program evaluation 				 
	     Program management 				 
	     Strategic planning 				 
	     Systems analysis					 
	     Systems conversions				 
	     Systems evaluation 				 
	     Systems integration				 
	     Systems management 				 
	     Business planning					 
	     Business transformation				 
	     GAO High Risk Series				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-519

   

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 
GAO: 

May 2008: 

DOD Business Systems Modernization: 

Military Departments Need to Strengthen Management of Enterprise 
Architecture Programs: 

GAO-08-519: 

GAO Highlights: 

Highlights of GAO-08-519, a report to congressional committees. 

Why GAO Did This Study: 

In 1995, GAO designated Department of Defense (DOD) business systems 
modernization as a high-risk program, and the program remains on the 
high-risk list today. A key to successful systems modernization is 
having and using an enterprise architecture as an authoritative frame 
of reference, or blueprint, for system investment decisions. To assist 
DOD in modernizing its business systems, Congress passed legislation 
consistent with prior GAO recommendations for DOD to develop and 
implement a business enterprise architecture (BEA). In response, DOD 
developed a corporate BEA that it intends to federate, or extend, to 
the military departments and defense agencies. 

To support GAOï¿½s legislative mandate to review DODï¿½s BEA, GAO evaluated 
the status of the Air Force, Navy, and Army architecture programs. To 
accomplish this, GAO used its Enterprise Architecture Management 
Maturity Framework and associated evaluation method. 

What GAO Found: 

The enterprise architecture programs within the Departments of the Air 
Force, Navy, and Army have yet to advance to a level that can be 
considered fully mature. Specifically, all three departments are at the 
initial stage of GAOï¿½s architecture maturity framework. This means that 
they have not fully satisfied all the core elements associated with the 
frameworkï¿½s second stage (establishing the management foundation for 
developing, maintaining, and using the architecture) or any of the 
frameworkï¿½s higher stages: Stage 3 (developing the architecture), Stage 
4 (completing the architecture), and Stage 5 (leveraging the 
architecture for organizational change). An organization generally 
needs to have achieved the fifth stage in the framework for it to have 
an effective architecture program because, at this stage, the 
management controls and structures are in place for using an approved 
architecture to guide and constrain system investments in a way that 
produces institutional results. 

Although each department is at Stage 1, the status of the programs vary 
considerably. Specifically, the Air Force far exceeds both the Navy and 
the Army, while the Navy generally exceeds the Army, in terms of the 
total percentage of core elements that are fully satisfied. Moreover, 
of the core elements that have not been fully satisfied by at least one 
of the three departments, most relate to architecture content, use, and 
measurement. Even though none of the departments have fully satisfied 
sufficient core elements to advance beyond Stage 1, the Air Force has 
at least partially satisfied all the core elements associated with 
Stages 2 and 3, as well as all but three core elements across all 
stages, and the Navy has at least partially satisfied all the core 
elements for Stage 2. In addition, the Air Force has made important 
progress in the last 2 years in maturing its architecture program, 
while the Navyï¿½s progress has been mixed, and the Army has not made any 
progress. 

Collectively, this means that DOD, as a whole, is not as well 
positioned as it should be to realize the significant benefits that a 
well-managed federation of architectures can afford its business 
systems modernization efforts. Individually, it means that the Air 
Force has a solid architectural foundation on which to continue 
building, while the Navy and, even more so, the Army has much to 
accomplish. According to Air Force officials, its progress owes largely 
to the architecture-related focus, commitment, and leadership of senior 
department executives, including the Secretary. 

Table: Percent of Framework Elements Fully Satisfied by Framework 
Maturity Stage: 

Military department: Air Force; 
All Stages: 61%; 
Stage 2: 78%; 
Stage 3: 83%; 
Stage 4: 38%; 
Stage 5: 50%. 

Military department: Navy; 
All Stages: 13%; 
Stage 2: 22%; 
Stage 3: 17%; 
Stage 4: 0%; 
Stage 5: 13%. 

Military department: Army; 
All Stages: 3%; 
Stage 2: 11%; 
Stage 3: 0%; 
Stage 4: 0%; 
Stage 5: 0%. 

Source: GAO analysis of agency data. 

[End of table] 

What GAO Recommends: 

Given the relative status and progress of the military departmentsï¿½ 
architecture programs, and GAOï¿½s existing recommendations for improving 
their maturity, GAO reiterates these existing recommendations and 
recommends that the Navy and Army reach out to the Air Force to learn 
from and apply its architecture-related lessons learned and 
experiences. In written comments, DOD agreed with GAOï¿½s recommendation. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-519]. For more 
information, contact Randolph C. Hite at (202) 512-3439 or 
[email protected]. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Maturity of Military Department Enterprise Architecture Programs 
Continues to Vary: 

Conclusions: 

Recommendation for Executive Action: 

Agency Comments: 

Appendix I: Objective, Scope, and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: Brief History of Architecture Frameworks and Management 
Guidance: 

Appendix IV: Department of the Air Force Assessment: 

Appendix V: Department of the Navy Assessment: 

Appendix VI: Department of the Army Assessment: 

Appendix VII: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Summary of EAMMF Version 1.1 Core Elements Categorized by 
Group: 

Table 2: Percent of Framework Elements Fully, Partially, and Not 
Satisfied by the Military Departments in 2006: 

Table 3: Percent of Framework Elements Fully Satisfied, by Maturity 
Stage: 

Table 4: Percent of Architecture Framework Core Elements Satisfied, by 
Group: 

Table 5: Percent of Framework Elements at Least Partially Satisfied, by 
Stage: 

Table 6: Net Change in Percent of Framework Elements Satisfied from 
2006 to 2008: 

Table 7: Stage 2 Evaluation Criteria: 

Table 8: Stage 3 Evaluation Criteria: 

Table 9: Stage 4 Evaluation Criteria: 

Table 10: Stage 5 Evaluation Criteria: 

Table 11: Federal Enterprise Architecture Reference Models: 

Table 12: OMB EA Assessment Framework Capability Areas: 

Figures: 

Figure 1: Summary of EAMMF Version 1.1: Maturity Stages, Critical 
Success Attributes, and Core Elements: 

Figure 2: Simplified Conceptual Depiction of the DOD EA Federation 
Strategy: 

Abbreviations: 

ASD(NII)/CIO: Assistant Secretary of Defense (Networks and Information 
Integration)/Chief Information Officer: 

BEA: business enterprise architecture: 

BMA: Business Mission Area: 

CIO: Chief Information Officer: 

DBSMC: Defense Business Systems Management Committee: 

DOD: Department of Defense: 

EA: enterprise architecture: 

EAMMF: Enterprise Architecture Management Maturity Framework; 

GIG: Global Information Grid: 

IT: information technology: 

OMB: Office of Management and Budget: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

May 12, 2008: 

Congressional Committees: 

For decades, the Department of Defense (DOD) has been challenged in 
modernizing its thousands of business systems. In 1995, we first 
designated DOD's business systems modernization program as high risk, 
and we continue to designate it as such today. As our research on 
public and private sector organizations shows, one key ingredient to a 
successful systems modernization program is having and using a well- 
defined enterprise architecture. Accordingly, we made recommendations 
to the Secretary of Defense in 2001 that included the means for 
effectively developing and implementing an enterprise architecture. 
[Footnote 1] Between 2001 and 2005, we reported on challenges that the 
department faced in its efforts to develop a business enterprise 
architecture (BEA) and made additional recommendations.[Footnote 2] 

To require DOD to address these and other modernization management 
challenges, Congress included provisions in the Ronald W. Reagan 
National Defense Authorization Act for Fiscal Year 2005[Footnote 3] 
that were consistent with our recommendations. In response, DOD adopted 
an incremental, federated approach to developing its BEA. We 
subsequently reported that this approach was consistent with best 
practices and that the initial version of the architecture provided a 
foundation on which to build and align the department's BEA with 
subsidiary architectures (i.e., military department and defense agency 
component-and individual program-level architectures).[Footnote 4] 

In light of the critical role that military department architectures 
play in DOD's federated BEA construct, you asked us to assess the 
status of the Departments of the Army, Navy, and Air Force enterprise 
architecture programs. To accomplish this, we used a standard data and 
document collection instrument to obtain key information about each 
department's architecture governance, content, use, and measurement. On 
the basis of the military departments' responses and supporting 
documentation, we analyzed the extent to which each satisfied the 31 
core elements in our architecture maturity framework.[Footnote 5] We 
also compared the current status of each military department's program 
against the status that we reported in 2006.[Footnote 6] We performed 
our work in the metropolitan area of Washington, D.C., from September 
2007 through March 2008 in accordance with generally accepted 
government auditing standards. Those standards require that we plan and 
perform the audit to obtain sufficient, appropriate evidence to provide 
a reasonable basis for our findings and conclusions based on our audit 
objectives. Details on our objectives, scope, and methodology are 
provided in appendix I. 

Results in Brief: 

The military departments' respective enterprise architecture programs 
have yet to advance to a level that can be considered mature. To 
effectively establish and leverage enterprise architectures as 
instruments of organizational transformation, research by us and others 
show that architecture programs should be founded upon both an 
institutional commitment and a measured and verified organizational 
capability to properly develop, maintain, and use the architecture to 
affect operational and technological change. Our framework for managing 
and evaluating the status of architecture programs consists of 31 core 
elements related to architecture governance, content, use, and 
measurement that are associated with five stages of maturity. In 2006, 
we reported that the Departments of the Air Force, Navy, and Army were 
in the initial stage of our framework, and they remain so today. This 
means that they have not fully satisfied all the core elements 
associated with the framework's second stage (establishing the 
management foundation for developing, maintaining, and using the 
architecture); nor have they fully satisfied the core elements 
associated with Stage 3 (developing the architecture), 4 (completing 
the architecture), and 5 (leveraging the architecture for 
organizational change). As we have previously reported, an organization 
generally needs to have achieved Stage 5 in our framework for it to 
have an effective architecture program because, at this stage, the full 
complement of architecture products and supporting management controls 
and structures are in place to guide and constrain information 
technology (IT) investments in a way that produces institutional 
results. 

Although each of the departments remain at Stage 1 of our maturity 
framework, the current status of their architecture programs are not 
identical. Also, while each department has not fully satisfied a number 
of core elements, some of them are at least partially satisfied. Most 
of the core elements that have not been fully or partially satisfied 
relate to developing sufficient architecture content (sufficient scope, 
depth, understanding, integrity, and consistency of products). In 
addition, even though all three departments remain at Stage 1 of our 
framework, one department has made important progress since 2006. More 
specifically: 

* The Air Force's architecture program is more mature than the Navy's, 
which is more mature than the Army's. The three military departments 
have fully satisfied 61, 13, and 3 percent, and partially satisfied 29, 
52, and 13 percent, of our framework's core elements, respectively. In 
addition, the Air Force and the Navy have at least partially satisfied 
the core elements associated with the framework's second stage 
(establishing the architecture management foundation), and the Air 
Force has partially satisfied the elements in the third stage 
(developing the architecture). 

* The Air Force has at least partially satisfied nearly all of the 
governance, content, use, and measurement-related core elements in our 
framework, while the Navy has at least partially satisfied most of the 
governance and content-related core elements. In contrast, the Army has 
only partially satisfied a few of the governance and use-related core 
elements and has not satisfied any of the content and measurement core 
elements. 

* The Air Force has made the most progress in the last 2 years in 
addressing our prior recommendations for satisfying our framework's 
core elements. For example, it has increased the percentage of core 
elements that are fully satisfied from 45 to 61 percent. In contrast, 
the Army has made the least progress, with its percentage of fully 
satisfied core elements remaining unchanged. 

As we have previously reported, the key to having a mature architecture 
program, and thereby realizing the benefits of an architecture-centric 
approach to IT investment decision making, is sustained executive 
leadership. This is because virtually all of the barriers to 
effectively developing and using architectures, such as parochialism, 
cultural resistance, adequate resources, and top management 
understanding, can be addressed through such leadership. In this 
regard, Air Force officials attributed their progress to the direct 
involvement and commitment of the Secretary of the Air Force. 

Because we have outstanding recommendations to the Secretary of Defense 
aimed at, among other things, having each of the military departments 
fully satisfy the core elements in our architecture framework, we are 
not making additional recommendations relative to our framework at this 
time. However, given the uneven status and progress of the respective 
military departments to date, opportunities exist for the Army and Navy 
to learn from the Air Force's experiences in maturing its architecture 
program. Therefore, we reiterate our outstanding recommendations and 
further recommend that the Secretary of Defense direct the Secretaries 
of the Navy and Army to ensure that their departments reach out to the 
Department of the Air Force to learn from and apply the lessons and 
experiences that have allowed the Air Force to make the progress it has 
in maturing its architecture program. In written comments on a draft of 
this report, signed by the Deputy Under Secretary of Defense (Business 
Transformation), and reprinted in appendix II, the department agreed 
with our recommendation. 

Background: 

DOD is a massive and complex organization. To illustrate, the 
department reported that its fiscal year 2007 operations involved 
approximately $1.5 trillion in assets and $2.1 trillion in liabilities, 
more than 2.9 million military and civilian personnel, and $544 billion 
in net cost of operations. 

In support of its military operations, the department performs an 
assortment of interrelated and interdependent business functions-- 
using thousands of business systems--related to major business areas 
such as weapon systems management, supply chain management, 
procurement, health care management, and financial management. The 
ability of these systems to operate as intended affects the lives of 
our warfighters both on and off the battlefield. As we have previously 
reported,[Footnote 7] the DOD systems environment that supports these 
business functions is overly complex; error-prone; and characterized by 
little standardization across the department, multiple systems 
performing the same tasks, the same data stored in multiple systems, 
and the need for data to be entered manually into multiple systems. 
Moreover, DOD recently reported that this systems environment is 
comprised of approximately 3,000 separate business systems. 

For fiscal year 2007, Congress appropriated approximately $15.7 billion 
to DOD; for fiscal year 2008, DOD has requested about $15.9 billion in 
appropriated funds to operate, maintain, and modernize these business 
systems and the associated infrastructures, of which approximately $11 
billion was requested for the military departments. 

DOD's pervasive business system and related financial management 
deficiencies adversely affect its ability to assess resource 
requirements; control costs; ensure basic accountability; anticipate 
future costs and claims on the budget; measure performance; maintain 
funds control; prevent and detect fraud, waste, and abuse; and address 
pressing management issues. In fact, DOD currently bears 
responsibility, in whole or in part, for 15 of the 27 federal 
government's program areas that we have designated as high risk. 
[Footnote 8] Eight of these areas are specific to DOD[Footnote 9] and 
the department shares responsibility for 7 other governmentwide high-
risk areas.[Footnote 10] DOD's business systems modernization is one of 
the high-risk areas, and it is an essential component for addressing 
many of the department's other high-risk areas. For example, modernized 
business systems are integral to the department's efforts to address 
its financial, supply chain, and information security management high-
risk areas. A well-defined and effectively implemented enterprise 
architecture is, in turn, integral to the successful modernization of 
DOD's business systems. 

Enterprise Architecture Is Critical to Achieving Successful Systems 
Modernization: 

An enterprise architecture (EA) is a blueprint that describes an 
organization's or a functional area's current and desired state in both 
logical and technical terms, as well as a plan for transitioning 
between the two states. As such, it is a recognized tenet of 
organizational transformation and IT management in public and private 
organizations. Without an EA, it is unlikely that an organization will 
be able to transform business processes and modernize supporting 
systems to minimize overlap and maximize interoperability. For more 
than a decade, we have conducted work to improve agency architecture 
efforts. To this end, we developed the Enterprise Architecture 
Management Maturity Framework (EAMMF) that provides federal agencies 
with a common benchmarking tool for assessing the management of their 
EA efforts and developing improvement plans. 

Enterprise Architecture Description and Importance: 

An enterprise can be viewed as either a single organization or a 
functional area that transcends more than one organization (e.g., 
financial management or homeland security). An architecture can be 
viewed as the structure (or structural description) of any activity. 
Thus, EAs are systematically derived and captured descriptions depicted 
in models, diagrams, and narratives. 

More specifically, an architecture describes the enterprise in logical 
terms (such as interrelated business processes and business rules, 
information needs and flows, and work locations and users) as well as 
in technical terms (such as hardware, software, data, communications, 
security attributes, and performance standards). It provides these 
perspectives both for the enterprise's current, or "as-is," 
environment, and for its target, or "to-be," environment, and it 
provides a transition plan for moving from the "as-is" to the "to-be" 
environment. 

The importance of EAs is a basic tenet of both organizational 
transformation and IT management, and their effective use is a 
recognized hallmark of successful public and private organizations. For 
over a decade, we have promoted the use of architectures, recognizing 
them as a crucial means to a challenging end: optimized agency 
operations and performance. The alternative, as our work has shown, is 
the perpetuation of the kinds of operational environments that saddle 
many agencies today, in which the lack of integration among business 
operations and the IT resources that support them leads to systems that 
are duplicative, not well integrated, and unnecessarily costly to 
maintain and interface.[Footnote 11] Employed in concert with other 
important IT management controls (such as portfolio-based capital 
planning and investment control practices), architectures can greatly 
increase the chances that organizations' operational and IT 
environments will be configured to optimize mission performance. 

The concept of EAs originated in the mid-1980s; various frameworks for 
defining the content of these architectures have been published by 
government agencies and the Office of Management and Budget (OMB). 
Moreover, legislation and federal guidance requires agencies to develop 
and use architectures. (See appendix III for a brief description of 
architecture frameworks and related legislation and management 
guidance.) 

GAO's Enterprise Architecture Management Maturity Framework: 

In 2002, we developed version 1.0 of the EAMMF to provide federal 
agencies with a common benchmarking tool for planning and measuring 
their efforts to improve enterprise architecture management, as well as 
to provide OMB with a means for doing the same governmentwide. We 
issued an update of the framework (version 1.1) in 2003.[Footnote 12] 
This framework is an extension of A Practical Guide to Federal 
Enterprise Architecture, Version 1.0, published by the Chief 
Information Officers Council.[Footnote 13] Version 1.1 of the framework 
arranges 31 core elements (practices or conditions that are needed for 
effective enterprise architecture management) into a matrix of five 
hierarchical maturity stages and four critical success attributes that 
apply to each stage. Within a given stage, each critical success 
attribute includes between one and four core elements. Based on the 
implicit dependencies among the core elements, the EAMMF associates 
each element with one of five maturity stages (see fig. 1). The core 
elements can be further categorized by four groups: architecture 
governance, content, use, and measurement. 

EAMMF Stages: 

Stage 1: Creating enterprise architecture awareness. At Stage 1, either 
an organization does not have plans to develop and use an architecture, 
or it has plans that do not demonstrate an awareness of the value of 
having and using an architecture. While Stage 1 agencies may have 
initiated some enterprise architecture activity, these agencies' 
efforts are ad hoc and unstructured, lack institutional leadership and 
direction, and do not provide the management foundation necessary for 
successful enterprise architecture development as defined in Stage 2. 

Stage 2: Building the enterprise architecture management foundation. An 
organization at Stage 2 recognizes that the EA is a corporate asset by 
vesting accountability for it in an executive body that represents the 
entire enterprise. At this stage, an organization assigns architecture 
management roles and responsibilities and establishes plans for 
developing architecture products and for measuring program progress and 
product quality; it also commits the resources necessary for developing 
an architecture--people, processes, and tools. Specifically, a Stage 2 
organization has designated a chief architect and established and 
staffed a program office responsible for EA development and 
maintenance. Further, it has established a committee or group that has 
responsibility for governance (i.e., directing, overseeing, and 
approving architecture development and maintenance). This committee or 
group membership has enterprisewide representation. At Stage 2, the 
organization either has plans for developing or has started developing 
at least some architecture products, and it has developed an 
enterprisewide awareness of the value of enterprise architecture and 
its intended use in managing its IT investments. The organization has 
also selected a framework and a methodology that will be the basis for 
developing the architecture products and has selected a tool for 
automating these activities. 

Stage 3: Developing the enterprise architecture. An organization at 
Stage 3 focuses on developing architecture products according to the 
selected framework, methodology, tool, and established management 
plans. Roles and responsibilities assigned in the previous stage are in 
place, and resources are being applied to develop actual products. At 
this stage, the scope of the architecture has been defined to encompass 
the entire enterprise, whether organization-based or function-based. 
Although the products may not be complete, they are intended to 
describe the organization in terms of business, performance, 
information/data, service/application, and technology (including 
security explicitly in each), as provided for in the framework, 
methodology, tool, and management plans.[Footnote 14] Further, the 
products are to describe the current ("as-is") and future ("to-be") 
states and the plan for transitioning from the current to the future 
state (the sequencing plan). As the products are developed and evolve, 
they are subject to configuration management. Further, through the 
established enterprise architecture management foundation, the 
organization is tracking and measuring its progress against plans, 
identifying and addressing variances, as appropriate, and then 
reporting on its progress. 

Stage 4: Completing the enterprise architecture. An organization at 
Stage 4 has completed its architecture products, meaning that the 
products have been approved by the EA steering committee (established 
in Stage 2) or an investment review board and by the Chief Information 
Officer (CIO). The completed products collectively describe the 
enterprise in terms of business, performance, information/data, 
service/application, and technology for both its current and future 
operating states; the products also include a plan for transitioning 
from the current to the future state. Further, an independent agent has 
assessed the quality (i.e., completeness and accuracy) of the 
architecture products. Additionally, evolution of the approved products 
is governed by a written EA maintenance policy approved by the head of 
the organization. 

Stage 5: Leveraging the enterprise architecture to manage change. An 
organization at Stage 5 has secured senior leadership approval of the 
architecture products and a written institutional policy stating that 
IT investments must comply with the architecture, unless granted an 
explicit compliance waiver. Further, decision makers are using the 
architecture to identify and address ongoing and proposed IT 
investments that are conflicting, overlapping, not strategically 
linked, or redundant. As a result, Stage 5 entities avoid unwarranted 
overlap across investments and ensure maximum systems interoperability. 
Maximum interoperability, in turn, ensures the selection and funding of 
IT investments with manageable risks and returns. Also, at Stage 5, the 
organization tracks and measures EA benefits or return on investment, 
and adjustments are continuously made to both the architecture 
management process and the enterprise architecture products. 

EAMMF Attributes: 

Attribute 1: Demonstrates commitment. Because the EA is a corporate 
asset for systematically managing institutional change, the support and 
sponsorship of the head of the enterprise are essential to the success 
of the architecture effort. An approved enterprise policy statement 
provides such support and sponsorship by promoting institutional buy-in 
and encouraging resource commitment from participating components. 
Equally important in demonstrating commitment is vesting ownership of 
the architecture in an executive body that collectively owns the 
enterprise. 

Attribute 2: Provides capability to meet commitment. The success of the 
EA effort depends largely on the organization's capacity to develop, 
maintain, and implement the enterprise architecture. Consistent with 
any large IT project, these capabilities include providing adequate 
resources (i.e., people, processes, and technology), defining clear 
roles and responsibilities, and defining and implementing 
organizational structures and process management controls that promote 
accountability and effective project execution. 

Attribute 3: Demonstrates satisfaction of commitment. Satisfaction of 
the organization's commitment to develop, maintain, and implement an EA 
is demonstrated by the production of artifacts (e.g., the plans and 
products). Such artifacts demonstrate follow through--actual EA 
production. Satisfaction of commitment is further demonstrated by 
senior leadership approval of enterprise architecture documents and 
artifacts; such approval communicates institutional endorsement and 
ownership of the architecture and the change that it is intended to 
drive. 

Attribute 4: Verifies satisfaction of commitment. This attribute 
focuses on measuring and disclosing the extent to which efforts to 
develop, maintain, and implement the EA have fulfilled stated goals or 
commitments of the enterprise architecture. Measuring such performance 
allows for tracking progress that has been made toward stated goals, 
allows appropriate actions to be taken when performance deviates 
significantly from goals, and creates incentives to influence both 
institutional and individual behaviors. Figure 1 illustrates the 
EAMMF's maturity stages, attributes, and core elements. 

Figure 1: Summary of EAMMF Version 1.1: Maturity Stages, Critical 
Success Attributes, and Core Elements: 

[See PDF for image] 

This figure is a table containing a summary of EAMMF Version 1.1: 
Maturity Stages, Critical Success Attributes, and Core Elements. 
Maturation increases from stage one through stage five. The following 
information is depicted: 

Attribute 1: Demonstrates commitment; 
Stage 1: Creating EA awareness: [Empty]; 
Stage 2: Adequate resources exist. Committee or group representing the 
enterprise is responsible for directing, overseeing, and approving EA. 
Stage 3: Developing EA products: Building the EA management foundation: 
Written and approved organization policy exists for EA development. 
Stage 4: Completing EA products: Written and approved organization 
policy exists for EA maintenance. 
Stage 5: Leveraging the EA to manage change: Written and approved 
organization policy exists for IT investment compliance with EA. 

Attribute 2: Provides capability to meet commitment; 
Stage 1: Creating EA awareness: [Empty]; 
Stage 2: Building the EA management foundation: Program office 
responsible for EA development and maintenance exists. Chief architect 
exists. EA is being developed using a framework, methodology, and 
automated tool. 
Stage 3: Developing EA products: EA products are under configuration 
management. 
Stage 4: Completing EA products: EA products and management processes 
undergo independent verification and validation. 
Stage 5: Leveraging the EA to manage change: Process exists to formally 
manage EA change.EA is integral component of IT investment management 
process. 

Attribute 3: Demonstrates satisfaction of commitment; 
Stage 1: Creating EA awareness: [Empty]; 
Stage 2: Building the EA management foundation: EA plans call for 
describing both the ï¿½as-isï¿½ and the ï¿½to-beï¿½ environments of the 
enterprise, as well as a sequencing plan for transitioning from the ï¿½as-
isï¿½ to the ï¿½to-be.ï¿½ EA plans call for describing both the ï¿½as-isï¿½ and 
the ï¿½to-beï¿½ environments in terms of business, performance, 
information/data, application/service, and technology. EA plans call 
for business, performance, information/data, application/service, and 
technology descriptions to address security.
Stage 3: Developing EA products: EA products describe or will describe 
both the ï¿½as-isï¿½ and the ï¿½to-beï¿½ environments of the enterprise, as 
well as a sequencing plan for transitioning from the ï¿½as-isï¿½ to the ï¿½to-
be.ï¿½ Both the ï¿½as-isï¿½ and the ï¿½to-beï¿½ environments are described or 
will be described in terms of business, performance, information/data, 
application/service, and technology. Business, performance, 
information/data, application/service, and technology descriptions 
address or will address security. 
Stage 4: Completing EA products: EA products describe both the ï¿½as-isï¿½ 
and the ï¿½to-beï¿½ environments of the enterprise, as well as a sequencing 
plan for transitioning from the ï¿½as-isï¿½ to the ï¿½to-be.ï¿½ Both the ï¿½as-
isï¿½ and the ï¿½to-beï¿½ environments are described in terms of business, 
performance, information/data, application/service, and technology. 
Business, performance, information/data, application/service, and 
technology descriptions address security.Organization CIO has approved 
current version of EA. Committee or group representing the enterprise 
or the investment review board has approved current version of EA. 
Stage 5: Leveraging the EA to manage change: EA products are 
periodically updated. IT investments comply with EA.Organization head 
has approved current version of EA. 

Attribute 4: Verifies satisfaction of commitment; 
Stage 1: Creating EA awareness: [Empty]; 
Stage 2: Building the EA management foundation: EA plans call for 
developing metrics for measuring EA progress, quality, compliance, and 
return on investment. 
Stage 3: Developing EA products: Progress against EA plans is measured 
and reported. 
Stage 4: Completing EA products: Quality of EA products is measured and 
reported. 
Stage 5: Leveraging the EA to manage change: Return on EA investment is 
measured and reported. Compliance with EA is measured and reported. 

Source: GAO. 

Note: Each stage includes all elements of previous stages. 

[End of figure] 

EAMMF Groups: 

The framework's 31 core elements can also be placed in one of four 
groups of architecture-related activities, processes, products, events 
and structures. The groups are architecture governance, content, use, 
and measurement. Each is defined below. 

* Governance refers to core elements that provide the management 
structures and processes needed to guide and direct the architecture 
program. 

* Content refers to core elements that provide for the scope, depth, 
integrity, understanding, and consistency of products and artifacts 
that make up the architecture. 

* Use refers to core elements that provide for an architecture-centric 
approach to IT investment management (i.e., treating architecture as 
the authoritative frame of reference in guiding and constraining IT 
investments). 

* Measurement refers to core elements that provide for determining and 
disclosing progress in developing, maintaining, and using the 
architecture, including measurement against plans, process standards, 
and product quality standards. 

These groups are generally consistent with the capability area 
descriptions in the OMB EA assessment tool.[Footnote 15] For example, 
OMB's completion capability area addresses ensuring that architecture 
products describe the agency in terms of processes, services, data, 
technology, and performance and that the agency has developed a 
transition strategy. Similarly, our content group includes developing 
and completing these same EA products. In addition, OMB's results 
capability area addresses performance measurement, as does our 
measurement group, and OMB's use capability area addresses many of the 
same elements in our governance and use groups. Table 1 lists the core 
elements according to EAMMF group. 

Table 1: Summary of EAMMF Version 1.1 Core Elements Categorized by 
Group: 

Group: Governance; 
Core element: Adequate resources exist (Stage 2). 

Group: Governance; 
Core element: Committee or group representing the enterprise is 
responsible for directing, overseeing, and approving EA (Stage 2). 

Group: Governance; 
Core element: Program office responsible for EA development and 
maintenance exists (Stage 2). 

Group: Governance; 
Core element: Chief architect exists (Stage 2). 

Group: Governance; 
Core element: EA being developed using a framework, methodology, and 
automated tool (Stage 2). 

Group: Governance; 
Core element: EA plans call for describing "as-is" environment, "to-be" 
environment, and sequencing plan (Stage 2). 

Group: Governance; 
Core element: EA plans call for describing enterprise in terms of 
business, performance, information/data, application/service, and 
technology (Stage 2). 

Group: Governance; 
Core element: EA plans call for business, performance, 
information/data, application/service, and technology to address 
security (Stage 2). 

Group: Governance; 
Core element: Written and approved policy exists for EA development 
(Stage 3). 

Group: Governance; 
Core element: Written and approved policy exists for EA maintenance 
(Stage 4). 

Group: Governance; 
Core element: Organization CIO has approved EA (Stage 4). 

Group: Governance; 
Core element: Committee or group representing the enterprise or the 
investment review board has approved current version of EA (Stage 4). 

Group: Governance; 
Core element: Written and approved organization policy exists for IT 
investment compliance with EA (Stage 5). 

Group: Governance; 
Core element: Organization head has approved current version of EA 
(Stage 5). 

Group: Content; 
Core element: EA products are under configuration management (Stage 3). 

Group: Content; 
Core element: EA products describe or will describe "as-is" 
environment, "to-be" environment, and sequencing plan (Stage 3). 

Group: Content; 
Core element: Both "as-is" and "to-be" environments are described or 
will be described in terms given in Stage 2 (Stage 3). 

Group: Content; 
Core element: These descriptions address or will address security 
(Stage 3). 

Group: Content; 
Core element: EA products and management processes undergo independent 
verification and validation (Stage 4). 

Group: Content; 
Core element: EA products describe "as-is" environment, "to-be" 
environment, and sequencing plan (Stage 4). 

Group: Content; 
Core element: Both "as-is" and "to-be" environments are described in 
terms given in Stage 2 (Stage 4). 

Group: Content; 
Core element: These descriptions address security (Stage 4). 

Group: Content; 
Core element: Process exists to formally manage EA change (Stage 5). 

Group: Content; 
Core element: EA products are periodically updated (Stage 5). 

Group: Use; 
Core element: EA is integral component of IT investment management 
process (Stage 5). 

Group: Use; 
Core element: IT investments comply with EA (Stage 5). 

Group: Measurement; 
Core element: EA plans call for developing metrics to measure EA 
progress, quality, compliance, and return on investment (Stage 2). 

Group: Measurement; 
Core element: Progress against EA plans is measured and reported (Stage 
3). 

Group: Measurement; 
Core element: Quality of EA products is measured and reported (Stage 
4). 

Group: Measurement; 
Core element: Return on EA investment is measured and reported (Stage 
5). 

Group: Measurement; 
Core element: Compliance with EA is measured and reported (Stage 5). 

Source: GAO. 

[End of table] 

DOD's Efforts to Adopt a Federated Approach to Architecting All of Its 
Mission Areas: 

DOD is pursing a federated strategy to develop and implement the many 
and varied architectures across the department's four mission areas-- 
Warfighting,[Footnote 16] Business,[Footnote 17] DOD Intelligence, 
[Footnote 18] and Enterprise Information Environment.[Footnote 19] 
According to officials in the Office of the Assistant Secretary of 
Defense (Networks and Information Integration)/Chief Information 
Officer (ASD(NII)/CIO), they have issued a strategy for evolving DOD's 
Global Information Grid (GIG)[Footnote 20] architecture that is to 
provide a comprehensive architectural description of the entire DOD 
enterprise, including all mission areas and the relationships between 
and among all levels of the enterprise (e.g., mission areas, 
components, and programs). Figure 2 provides a simplified, conceptual 
depiction of DOD's EA federation strategy. 

Figure 2: Simplified, Conceptual Depiction of the DOD EA Federation 
Strategy: 

[See PDF for image] 

This figure is a Simplified, Conceptual Depiction of the DOD EA 
Federation organizational chart, as follows: 

Department Layer: 
Global Information Grid. 

Mission Area Layer (fed by Department Layer): 
* Warfighting Mission Area; 
- Warfighting EA. 
* Business Mission Area; 
- Business EA. 
* Intelligence Mission Area; 
- Intelligence EA. 
* Enterprise Information Environment Mission Area; 
- Enterprise Information Environment EA. 

Component Layer (Fed by Mission Area Layer): 
* Service; 
* Agency; 
* Combat Command. 

Program Layer (fed by Component Layer): 
* Individual programs. 

Source: GAO analysis of DOD data. 

[End of figure] 

ASD(NII)/CIO officials stated that the goal of this strategy is to 
improve the ability of DOD's mission areas, components, and programs to 
share architectural information. In this regard, officials stated that 
the DOD EA federation strategy will define federation and integration 
concepts, alignment (i.e., linking and mapping) processes, and shared 
services. 

The first Business Mission Area (BMA) federation strategy was released 
in September 2006, according to ASD(NII)/CIO officials. Its purpose is 
to expand on the DOD EA federation strategy and provide details on how 
various aspects of the federation will be applied within the 
department's BMA. In this regard, the BMA strategy cites the following 
four goals: 

* establish a capability to search for data in member architectures 
that may be relevant for analysis, reference, or reuse; 

* develop a consistent set of standards for architecture configuration 
management that will enable users to determine the development status 
and quality of data in various architectures; 

* establish a standard methodology for specifying linkages among 
existing component architectures that were developed using different 
tools and that are maintained in independent repositories; and: 

* develop a standard methodology to reuse capabilities described by 
various architectures. 

To assist in accomplishing these goals, the strategy described three 
concepts that are to be applied: 

1. Tiered accountability provides for architecture development at each 
of the department's organizational levels. 

2. Net-centricity provides for seamless and timely accessibility to 
information where and when needed via the department's interconnected 
network environment. 

3. Federating DOD architectures provides for linking or aligning 
subordinate and parent architectures via the mapping of common 
architectural information. This concept advocates subordinate 
architecture alignment to the parent architecture. 

DOD Roles and Responsibilities for Business Enterprise Architecture 
Development and Use: 

In 2005, DOD reassigned responsibility for directing, overseeing, and 
executing its business transformation and systems modernization efforts 
to the Defense Business Systems Management Committee (DBSMC) and the 
Business Transformation Agency. The DBSMC is chaired by the Deputy 
Secretary of Defense and serves as the highest-ranking governance body 
for business systems modernization activities. According to its 
charter, the DBSMC provides strategic direction and plans for the BMA 
in coordination with the Warfighting and the Enterprise Information 
Environment Mission Areas. The DBSMC is also responsible for reviewing 
and approving the BEA and the Enterprise Transition Plan. 

The Business Transformation Agency operates under the authority, 
direction, and control of the DBSMC and reports to the Under Secretary 
of Defense for Acquisition, Technology, and Logistics in the 
incumbent's capacity as the vice chair of the DBSMC. Oversight for this 
agency is provided by the Deputy Under Secretary of Defense for 
Business Transformation, and day-to-day management is provided by the 
director. The Business Transformation Agency's primary responsibility 
is to lead and coordinate business transformation efforts across the 
department. In particular, it is responsible for (1) maintaining and 
updating the department's architecture, (2) ensuring that functional 
priorities and requirements of various defense component organizations 
are reflected in the architecture, and (3) ensuring the adoption of DOD-
wide information and process standards as defined in the architecture. 

Under DOD's tiered accountability approach to systems modernization, 
components are responsible for defining their respective component 
architectures and transition plans. Similarly, program managers are 
responsible for developing program-level architectures and transition 
plans and ensuring integration with the architectures and transition 
plans developed and executed at the enterprise and component levels. 

Summary of GAO's Prior Reviews on Business Enterprise Architecture and 
Military Department Architectures: 

Between May 2001 and July 2005, we reported on DOD's efforts to develop 
an architecture and identified serious problems and concerns with the 
department's architecture program, including the lack of specific plans 
outlining how DOD would extend and evolve the architecture to include 
the missing scope and detail.[Footnote 21] To address these concerns, 
in September 2003,[Footnote 22] we recommended that DOD develop a well- 
defined, near-term plan for extending and evolving the architecture and 
ensure that this plan would address our recommendations: defining roles 
and responsibilities of all stakeholders involved in extending and 
evolving the architecture, explaining dependencies among planned 
activities, and defining measures of progress for the activities. 

In response to our recommendations, in 2005, DOD adopted a 6-month 
incremental approach to developing its architecture and released 
version 3.0 of the BEA and the associated transition plan in September 
2005, describing them as the initial baselines. Since then, DOD has 
released three updated versions of both--version 3.1, released on March 
15, 2006; version 4.0, released on September 28, 2006; and version 4.1, 
released on March 15, 2007. As we have previously reported,[Footnote 
23] these incremental versions have provided additional content and 
clarity and resolved limitations that we identified in the prior 
versions. For example, version 4.1 improved the Financial Visibility 
business enterprise priority area by including the Standard Financial 
Information Structure data elements and business rules to support cost 
accounting and reporting. In addition, version 4.1 addressed, to 
varying degrees, missing elements, inconsistencies, and usability 
issues that we previously identified. 

In August 2006,[Footnote 24] we reported that the Departments of the 
Air Force, Navy, and Army had fully satisfied about 45, 32, and 3 
percent, and partially satisfied 26, 39, and 42 percent, respectively, 
of the 31 core elements in our architecture maturity framework (see 
table 2). 

Table 2: Percent of Framework Elements Fully, Partially, and Not 
Satisfied by the Military Departments in 2006: 

Military departments: Air Force; 
Fully satisfied: 45%; 
Partially satisfied: 26%; 
Not satisfied: 29%. 

Military departments: Navy; 
Fully satisfied: 32; 
Partially satisfied: 39; 
Not satisfied: 29. 

Military departments: Army; 
Fully satisfied: 3; 
Partially satisfied: 42; 
Not satisfied: 55. 

Source: GAO analysis of agency data. 

[End of table] 

By comparison, other major federal departments and agencies that we 
reviewed had, as a whole, fully satisfied about 67 percent of the 
framework's core elements. Among the key elements that all three 
military departments had not fully satisfied were developing 
architecture products that describe their respective target 
architectural environments and developing transition plans for 
migrating to a target environment. Furthermore, while the military 
departments had partially satisfied between 26 and 42 percent of the 
core elements in our framework, we reported that partially satisfied 
elements were not necessarily easy to satisfy fully, such as those that 
address architecture content; and thus, partials can have serious 
implications for the quality and usability of an architecture. To 
assist the military departments in addressing their EA challenges and 
managing their programs, we recommended that they develop and implement 
plans for fully satisfying each of the conditions in our framework. DOD 
generally agreed with our findings and recommendations. 

In April 2007,[Footnote 25] we reported that DOD's BMA federation 
strategy provided a foundation on which to build and align DOD's parent 
BEA with its subordinate architectures (i.e., component-and program- 
level architectures). In particular, we found that this strategy (1) 
stated the department's federated architecture goals; (2) described 
federation concepts that are to be applied; and (3) included high-level 
activities, capabilities, products, and services that are intended to 
facilitate implementation of the concepts. However, we also reported 
that DOD had yet to define the details needed to execute the strategy, 
such as how the architecture federation was to be governed; how 
alignment with the DOD federation strategy and other potential mission- 
area federation strategies was to be achieved; how component 
architectures' alignment with incremental versions of the BEA was to be 
achieved; how shared services would be identified, exposed, and 
subscribed to; and what milestones would be used to measure progress 
and results. As a result, we concluded that much remained to be decided 
and accomplished before DOD would have in place the means to create a 
federated architecture and thus be able to fully satisfy both our prior 
recommendations and legislative requirements aimed at adopting an 
architecture-centric approach to departmentwide business systems 
investment management. 

In May 2007,[Footnote 26] we concluded that while DOD has made progress 
in developing the BEA, much remained to be accomplished. In particular, 
we reported that DOD had yet to extend and evolve its corporate BEA 
through the development of aligned, subordinate architectures for each 
of its component organizations, and while it had developed a strategy 
for federating the BEA in this manner, this strategy lacked the detail 
needed for it to be fully effective. We also reported that this 
situation was compounded by the known immaturity of the military 
departments' architecture efforts. Accordingly, we recommended that DOD 
include in its annual report to Congress on compliance with section 332 
of the Fiscal Year 2005 National Defense Authorization Act the results 
of assessments by its BEA independent verification and validation 
contractor on the completeness, consistency, understandability, and 
usability of its federated family of business mission architectures, 
including the associated transition plans. DOD agreed with this 
recommendation. 

Maturity of Military Department Enterprise Architecture Programs 
Continues to Vary: 

Each of the military departments' enterprise architecture programs is 
at the initial stage of our maturity framework, meaning that each has 
not fully satisfied all of the core elements associated with the 
framework's second stage (establishing the management foundation for 
developing, maintaining, and using the architecture). Also, none have 
fully satisfied the core elements associated with Stage 3 (developing 
the architecture), 4 (completing the architecture), and 5 (leveraging 
the architecture for organizational change). As a result, none have yet 
to advance to a state that can be considered fully mature and 
effective. Although all departments are at Stage 1, the status of the 
three vary considerably. Specifically, the Air Force far exceeds the 
Navy, which generally exceeds the Army, in terms of the total number of 
core elements that are fully satisfied. Further, even though all three 
are at Stage 1, the Air Force has at least partially satisfied all of 
the core elements associated with Stage 3 and has partially satisfied 
all but three core elements across all stages. The Navy has at least 
partially satisfied all of the core elements associated with Stage 2 
and all but one of the Stage 3 core elements. Moreover, the Air Force 
has made important progress in maturing its EA program over the last 2 
years, while the Navy has made mixed progress, and the Army has not 
made progress. 

As our research shows, the state of an organization's EA program owes 
largely to the extent to which the program has benefited from sustained 
executive leadership. This is because virtually all of the barriers to 
effectively developing and using architectures, such as parochialism, 
cultural resistance, adequate resources, and top management 
understanding, can be addressed through such leadership. In this 
regard, Air Force officials attributed their progress toward 
establishing a fully mature architecture program to sustained executive 
leadership. Without fully mature programs, the departments introduce 
increased risk of developing and implementing IT solutions that are 
duplicative, do not interoperate, and thus do not optimize 
departmentwide performance. 

The Military Departments' Full Satisfaction of Our Framework's Core 
Elements Varies, but None Have Programs That Are Fully Mature: 

To reach a given stage of maturity under our architecture framework and 
associated evaluation methodology, a military department had to fully 
satisfy all of the core elements at that stage. Using this criterion, 
each of the military departments is at Stage 1, meaning that none could 
demonstrate through verifiable documentation that it has established 
all of the core foundational commitments and capabilities needed to 
effectively manage the development, maintenance, and implementation of 
an architecture. However, this does not mean that the departments are 
at identical states of maturity. Rather, the Air Force is considerably 
more advanced than the Navy and Army. (See appendices IV through VI for 
details on the extent to which each military department satisfied each 
of the core elements of our maturity framework.) 

Assigning the military departments' architecture programs to a maturity 
stage based on whether all elements of a stage are fully satisfied 
provides only one perspective on these programs. Another is the extent 
to which each program has also fully satisfied core elements across 
higher stages of maturity. When the percentage of core elements that 
have been fully satisfied across all stages is considered, a similar 
picture of the departments' relative variability is evident. 
Specifically, the percent of all core elements that are fully satisfied 
ranges from a high of 61 percent for the Air Force to a low of 3 
percent for the Army (the Navy fully satisfied 13 percent of the core 
elements). Table 3 summarizes the percentage of core elements that are 
fully satisfied in total, by maturity stage, for each military 
department. 

Table 3: Percent of Framework Elements Fully Satisfied, by Maturity 
Stage: 

Military departments: Air Force; 
Framework elements fully satisfied: All stages: 61%; 
Framework elements fully satisfied: Stage 2: 78%; 
Framework elements fully satisfied: Stage 3: 83%; 
Framework elements fully satisfied: Stage 4: 38%; 
Framework elements fully satisfied: Stage 5: 50%. 

Military departments: Navy; 
Framework elements fully satisfied: All stages: 13%; 
Framework elements fully satisfied: Stage 2: 22%; 
Framework elements fully satisfied: Stage 3: 17%; 
Framework elements fully satisfied: Stage 4: 0%; 
Framework elements fully satisfied: Stage 5: 13%. 

Military departments: Army; 
Framework elements fully satisfied: All stages: 3%; 
Framework elements fully satisfied: Stage 2: 11%; 
Framework elements fully satisfied: Stage 3: 0%; 
Framework elements fully satisfied: Stage 4: 0%; 
Framework elements fully satisfied: Stage 5: 0%. 

Source: GAO analysis of agency data. 

[End of table] 

Notwithstanding this perspective, it is important to note that the 
staging of core elements in our framework provide a hierarchical or 
systematic progression to establishing a mature and effective 
architecture program. That is, core elements associated with lower 
framework stages generally support the effective execution of higher 
maturity stage core elements. For instance, if a program has developed 
its full suite of "as-is" and "to-be" architecture products, including 
a sequencing plan (Stage 4 core elements), but the products are not 
under configuration management (Stage 3 core element), then the 
integrity and consistency of the products cannot be adequately assured. 
In this regard, even though the Navy has partially developed its EA 
products, the quality of these products is questionable because the 
Navy has not placed them under configuration management. 

Further, not satisfying even a single lower stage core element can have 
a significant impact on the effectiveness of an architecture program. 
For example, not using a defined framework or methodology (Stage 2 core 
element) or not performing configuration management (Stage 3 core 
element), can significantly limit the quality and utility of an 
architecture. DOD's experience between 2001 and 2005 in developing its 
BEA is a case in point. During this time, we made a series of 
recommendations grounded in, among other things, our architecture 
management framework to ensure that it was successful in doing 
so.[Footnote 27] In 2005,[Footnote 28] we reported that despite 
investing hundreds of millions of dollars and 4 years in developing 
multiple versions of wide-ranging architecture products, the department 
did not have a well-defined architecture, and what it did develop had 
limited utility. Among other things, we attributed the poor state of 
its architecture products to methodological, human capital, and 
configuration management weaknesses. 

Looking at related groupings of core elements that are fully satisfied 
also provides a useful perspective on the state of the military 
departments' architecture programs. As noted earlier, these groupings 
of core elements are architecture governance, content, use, and 
measurement. Overall, the military departments varied in the extent to 
which each of the groups were met. For example, while the Air Force 
fully satisfied 71 percent of the governance core elements, the Navy 
and Army only fully satisfied 14 and 7 percent, respectively. The 
extent to which each department satisfied the core elements in each 
grouping are discussed below. (See table 4 for a summary of the extent 
to which each department fully satisfied these groupings.) 

* Governance refers to core elements that provide the management 
structures and processes needed to guide and direct an architecture 
program. Neither the Navy nor the Army has established effective 
architecture governance, having satisfied only 14 and 7 percent of 
these core elements, respectively. For example, neither has a written 
or approved departmentwide policy for EA development and maintenance or 
for requiring that IT investments comply with the EA. This is important 
because approved policies demonstrate institutional commitment to 
having and using an architecture. As our framework states, an approved 
enterprisewide policy provides the kind of top management support and 
sponsorship needed to overcome the barriers to architecture development 
and use. In contrast, the Air Force has fully satisfied the majority of 
governance core elements. However, the remaining unsatisfied core 
elements are significant. For example, it has not fully established a 
committee or group with representation from across the enterprise to 
direct, oversee, and approve the architecture. This is significant 
because the architecture is a corporate asset that needs to be 
enterprisewide in scope and accepted by senior leadership if it is to 
be leveraged effectively for organizational change. 

Table 4: Percent of Architecture Framework Core Elements Satisfied, by 
Group: 

Military departments: Air Force; 
Framework groups: Governance: 71%; 
Framework groups: Content: 60%; 
Framework groups: Use: 50%; 
Framework groups: Measurement: 40%. 

Military departments: Navy; 
Framework groups: Governance: 14%; 
Framework groups: Content: 10%; 
Framework groups: Use: 0%; 
Framework groups: Measurement: 20%. 

Military departments: Army; 
Framework groups: Governance: 7%; 
Framework groups: Content: 0%; 
Framework groups: Use: 0%; 
Framework groups: Measurement: 0%. 

Source: GAO analysis of agency data. 

[End of table] 

* Content refers to core elements that provide for the scope, depth, 
integrity, understanding, and consistency of products and artifacts 
that make up the architecture. Only the Air Force has fully satisfied 
much in the way of architecture content, having fully met 60 percent of 
the core elements, with the Navy and Army meeting only 10 and 0 
percent, respectively. For example, while the Air Force has placed its 
EA products under configuration management and provided for ensuring 
that its EA products will describe the "as-is" environment, "to-be" 
environment, and sequencing plan, neither the Navy nor the Army has 
done the same. Moreover, none of the departments have fully addressed 
security as part of their respective "as-is" and "to-be" products 
developed to date. This is important because security is relevant and 
essential to every aspect of an organization's operations, and 
therefore, the nature and substance of institutionalized security 
requirements, controls, and standards should be embedded throughout the 
architecture. Further, none of the departments is using an independent 
verification and validation agent to help ensure the quality of these 
products. As we have previously reported, independent verification and 
validation is a proven means for obtaining unbiased insight into such 
essential architecture qualities as completeness, understandability, 
and consistency. 

* Use refers to core elements that provide for an architecture-centric 
approach to IT investment management (i.e., treating architecture as 
the authoritative frame of reference for guiding and constraining IT 
investments). Again, the Air Force has fully satisfied 50 percent of 
this grouping's core elements, while the Navy and the Army have fully 
satisfied none of the core elements. For example, the Air Force has 
made its EA an integral component of its IT investment process by 
requiring that investments demonstrate their architectural compliance. 
This is important because in order for the benefits of an EA to be 
realized, IT investments must comply with it. However, neither the Air 
Force nor the other two departments could demonstrate that their 
respective IT investments are actually in compliance with their 
respective architectures. This is relevant because the benefits from 
using an EA, such as improved information sharing, increased 
consolidation, enhanced productivity, and lower costs, cannot be fully 
realized unless individual investments are actually in compliance with, 
for example, architectural rules and standards. 

* Measurement refers to core elements that provide for determining and 
disclosing progress in developing, maintaining, and using the EA, 
including measurement against plans, process standards, and product 
quality. None of the departments satisfied many of these core elements. 
Specifically, the Air Force fully satisfied 40 percent and the Navy 
fully satisfied 20 percent of these core elements, while the Army did 
not satisfy any. For example, while the Air Force has plans that call 
for the development of metrics to measure EA progress, quality, 
compliance, and return on investment, and the Navy is measuring and 
reporting on progress against plans, none of the departments is 
measuring and reporting on IT investment compliance with its 
architecture or return on investment from its architecture program. 
Without measuring architecture development, maintenance, and use, an 
organization is not positioned to take timely corrective action to 
address deviations from plans, expectations, and outcomes, which in 
turn limits the chances of EA program success. 

The Military Departments' Partial Satisfaction of Framework's Core 
Elements Also Varies, and Provides a Basis from Which to Build: 

In instances where the military departments have not fully satisfied 
certain core elements in our framework, two have at least partially 
satisfied[Footnote 29] many of these elements. To illustrate, the Air 
Force would improve to Stage 3 if the criterion for being at a given 
stage was relaxed to only partially satisfying a core element. 
Moreover, the Navy would advance to Stage 2 under this less demanding 
criterion. In contrast, the Army would remain at Stage 1. 

Partial satisfaction of a core element is an indicator of some progress 
and provides a basis on which to improve. Nevertheless, it also 
indicates that more work is needed, for example, to establish 
architectural commitments and capabilities and to demonstrate and 
verify that both exist and are functioning as intended. Moreover, even 
though a core element can be partially satisfied, what remains to fully 
satisfy it can be significant and can require considerable time and 
resources. Thus, it is important to note that even though the 
departments have partially satisfied some core elements, fully 
satisfying some of them may remain a challenge. It is also important to 
note that fully, rather than partially, satisfying certain elements, 
such as those that fall within the architecture content group, are key 
to the success of an EA. Not fully satisfying these elements can have 
important implications for the quality of an architecture, and thus its 
usability and results. The extent to which each of the departments 
partially satisfied the core elements at each stage of our framework is 
discussed below and summarized in table 5. 

* The Air Force has at least partially satisfied 100 percent of the 
elements associated with Stages 2 and 3 and has a solid base on which 
to build an effective architecture management foundation and its 
associated plans and products. Nevertheless, important aspects of a 
successful architecture program are still missing. For example, while 
the Air Force is developing its EA using a framework (the Air Force EA 
Framework) and an automated tool (Metis ArchitectTM[Footnote 30]), it 
does not have a documented methodology governing how its architecture 
is being developed and maintained. This is important because a 
methodology provides a common set of procedures for developing EA 
products and helps to ensure consistency in how these products are 
developed and maintained across the organization. Also, while the Air 
Force does have metrics related to its EA, these metrics do not address 
measuring progress against plans. As our framework states, progress in 
developing EA products should be measured against EA plans so that 
deviations from expectations can be examined for cause and impact and 
appropriate actions can be taken to address them. 

Table 5: Percent of Framework Elements at Least Partially Satisfied, by 
Stage: 

Military departments: Air Force; 
Framework elements fully or partially satisfied: All stages: 90%; 
Framework elements fully or partially satisfied: Stage 2: 100%; 
Framework elements fully or partially satisfied: Stage 3: 100%; 
Framework elements fully or partially satisfied: Stage 4: 75%; 
Framework elements fully or partially satisfied: Stage 5: 88%. 

Military departments: Navy; 
Framework elements fully or partially satisfied: All stages: 65%; 
Framework elements fully or partially satisfied: Stage 2: 100%; 
Framework elements fully or partially satisfied: Stage 3: 83%; 
Framework elements fully or partially satisfied: Stage 4: 63%; 
Framework elements fully or partially satisfied: Stage 5: 13%. 

Military departments: Army; 
Framework elements fully or partially satisfied: All stages: 16%; 
Framework elements fully or partially satisfied: Stage 2: 44%; 
Framework elements fully or partially satisfied: Stage 3: 0%; 
Framework elements fully or partially satisfied: Stage 4: 0%; 
Framework elements fully or partially satisfied: Stage 5: 13%. 

Source: GAO analysis of agency data. 

[End of table] 

* The Navy has at least partially satisfied 93 percent of the elements 
associated with Stages 2 and 3 and has in place many aspects of the 
core elements that support these stages, which it can use to continue 
establishing an effective architecture management foundation and 
associated plans and products. However, important aspects of certain 
core elements are missing. For example, similar to the Air Force, the 
Navy is developing its EA using a framework (the DOD Architecture 
Framework) and automated tools (Telelogic System Architectï¿½[Footnote 
31] and Metis ArchitectTM). Also, similar to the Air Force, the Navy 
does not have a documented methodology governing how its architecture 
is being developed and maintained. In addition, the Navy has yet to 
establish a committee or group representing the enterprise that is 
responsible for directing, overseeing, or approving the architecture. 
Establishing such a governance entity is important because it 
demonstrates the organization's commitment to building and using an 
architecture and helps to obtain necessary buy-in from across the 
organization. Further, while the Navy has drafted an EA policy, the 
policy has not yet been approved. Approval is important because it 
demonstrates senior leadership commitment to the architecture and 
clearly assigns responsibility and accountability for development and 
maintenance of the architecture. 

* The Army has at least partially satisfied 27 percent of the elements 
associated with Stages 2 and 3, but has not made progress toward 
establishing the commitments and capabilities that comprise the core 
elements integral to an effective architecture management foundation 
and associated plans and products. In particular, while the Army has an 
EA program office, the office does not have an approved charter. A 
program charter is important because it defines key aspects about how 
the office will operate in order to achieve program goals and outcomes. 
Further, while the Army is using an automated tool (System Architect) 
to capture architecture products, it is not using a framework or 
methodology. This is significant because the framework provides a 
defined structure and nomenclature for representing architecture 
information across the organization and the methodology describes a 
common set of procedures to be used for developing products in a 
coherent way. Together, they help to ensure that activities associated 
with capturing the architecture are understood by those involved and 
are completed in a consistent, accountable, and repeatable manner. 

The Military Departments Varied in the Extent to Which Their 
Architecture Programs Have Recently Improved: 

We reported in August 2006 that each of the military departments was at 
the initial stage of our architecture maturity framework.[Footnote 32] 
More specifically, we reported that the Air Force, Navy, and Army had 
fully satisfied 45, 32, and 3 percent, and that they had partially 
satisfied 26, 39, and 42 percent, of the 31 core elements, 
respectively. Accordingly, we made recommendations for each department 
to fully implement the framework's core elements. 

Since then, the departments have addressed our recommendations to 
varying degrees. Specifically, the Air Force has made the most 
progress, increasing the percentage of fully satisfied core elements 
from 45 to 61 percent and increasing the percentage of partially 
satisfied core elements from 26 to 29 percent. The Navy has made mixed 
progress, decreasing the percentage of fully satisfied core elements 
from 32 to13 percent and increasing the percentage of partially 
satisfied core elements from 39 to 52 percent. The Army has not made 
progress, keeping the percentage of fully satisfied core elements at 3 
percent while decreasing the percentage of partially satisfied core 
elements from 42 to 13 percent. The specific progress made by each 
department is discussed below and summarized in table 6. 

* The Air Force's 16 percent increase in the core elements that are 
fully satisfied relate to five core elements. For example, we 
previously reported that the Air Force's architecture program plans did 
not call for developing metrics to measure EA progress, quality, 
compliance, and return on investment. Since then, the Air Force has 
expanded its plans to include such metrics. The addition of these 
metrics is important because they provide the basis for knowing whether 
program expectations are being met, which could prompt timely 
corrective action to address any significant deviations. Also, while 
the Air Force did not previously have its architecture products under 
configuration management, it has since done so. This progress is 
important because it helps to integrate and align the products and to 
track and manage changes to them in a way that ensures product 
integrity. Finally, the Air Force has also established the architecture 
as an integral component of its IT investment management process by 
explicitly requiring that its IT investments align with the EA. This 
was not the case in 2006 and represents a significant improvement 
because without requiring alignment, investments are more likely to 
deviate from the architecture in ways that increase duplication of 
effort and decrease interoperability. 

The Air Force's 3 percent increase in the core elements that are 
partially satisfied relate to three core elements. For example, we 
reported in 2006 that the Air Force's EA products did not address 
security. Since then, the Air Force has developed an Information 
Assurance Domain Architecture to augment its EA products. To the Air 
Force's credit, this document addresses important aspects of security 
relative to its technical reference models. However, it does not 
similarly address security in relation to the EA's business, systems, 
and information models. For example, the business models do not address 
the goals and strategies for addressing current security risks and 
future security threats. In addition, while the Air Force now has a 
sequencing plan, this plan is not complete because it does not include, 
and is not grounded in, an analysis of the gap in capabilities between 
the department's "as-is" and "to-be" architectural environments. Such a 
gap analysis is necessary to determine what capabilities are currently 
lacking and which capabilities will need to be acquired or developed in 
a sequence that is based on a variety of factors. 

According to Air Force officials, the positive progress that it has 
made in maturing its architecture program is due, in large part, to the 
focus, commitment, and leadership of senior department executives, 
including the Secretary of the Air Force. In this regard, they said 
that their experiences and lessons learned show that such leadership 
paved the way for establishing the department's institutional 
commitment to its EA. Such commitment, they said, is demonstrated by 
the Air Force's approved EA policies and funding, and its capabilities 
to meet commitments, such as the department's structures and processes 
for governing architecture development and implementation. 

Table 6: Net Change in Percent of Framework Elements Satisfied from 
2006 to 2008: 

Military departments: Air Force; 
Fully satisfied: +16%; 
Partially satisfied: +3%. 

Military departments: Navy; 
Fully satisfied: -19%; 
Partially satisfied: +13%. 

Military departments: Army; 
Fully satisfied: 0%; 
Partially satisfied: -29%. 

Source: GAO analysis of agency data. 

[End of table] 

* The Navy's 19 percent decrease in the core elements that are fully 
satisfied relate to six core elements, and according to Navy officials, 
are largely due to the Navy's recent efforts to expand the scope of its 
architecture program beyond that which existed in 2006. More 
specifically, in 2006, the Navy's architecture program was focused on 
what it refers to as FORCEnet, which is the Navy's IT infrastructure 
architecture. During the course of our review, the Navy reported that 
it has adopted DOD's federated architecture approach, thereby expanding 
its program to include all Navy organizations and all architecture 
reference models (e.g., business, data, performance). However, it has 
yet to reflect this expansion in program scope in key governance 
documents, such as its EA policies and plans, that relate to these six 
core elements. Without fully satisfying these governance core elements, 
the department will be challenged in its ability to develop and 
implement a Navy-wide federated architecture. 

The 13 percent increase in the core elements that are partially 
satisfied are largely associated with three content-related core 
elements. For example, we reported in 2006 that the Navy's FORCEnet 
products did not include descriptions of the "to-be" architecture in 
terms of, for example, business, information/data, applications/ 
service, and performance. Under the Navy's expanded scope, it has since 
developed "to-be" architecture products (DOD Architecture Framework 
views) that address these areas. However, these products are not yet 
sufficiently complete. To illustrate, the functional areas (e.g., human 
resources) in the business or operational views have not been 
decomposed into functional activities (e.g., recruiting) and processes 
(e.g., interviewing), and the information exchanges between functional 
areas in the operational views are not defined. Moreover, there are no 
data models to identify and describe relationships among the data 
elements within each functional area. 

According to Navy officials, the shift to a broader, federated approach 
to developing and implementing a Navy enterprise architecture was 
recently endorsed by secretariat-level leadership and senior department 
executives. 

* The Army continues to fully satisfy one core element (having a chief 
architect). However, it has also experienced a 29 percent decrease in 
those core elements that it had partially satisfied in 2006 (nine core 
elements), most of which relate to such governance topics as EA 
policies and plans. Specifically, the plans and policies that the Army 
had in 2006 were not approved. Because they were not approved, they did 
not fully satisfy the various associated core elements. Since then, the 
Army official principally responsible for the program told us that the 
department's senior executives, including the Army Vice-Chief of Staff, 
have endorsed a new architecture approach in which the Army will use 
OMB's reference models (e.g., business, performance, information/ 
data). To this end, the officials said that decisions are to be made in 
the near future about how to best structure the program to implement 
this approach, including what the program's scope will be and what 
resources will be needed to sustain it. This official added that once 
these decisions are made, as part of the Army's budget submission and 
approval process, the EA policies and plans will be updated to reflect 
these decisions. 

Conclusions: 

If managed effectively, enterprise architectures can be a useful change 
management and organizational transformation tool. The conditions for 
effectively managing enterprise architecture programs are contained in 
our five stage architecture maturity framework. None of the military 
departments has fully satisfied all of the conditions needed to achieve 
Stage 2 or above in the framework, which means that none have programs 
that we would currently consider effective and mature. However, the 
Navy has partially satisfied most, and the Air Force has partially 
satisfied all, of the core elements needed to be at Stage 3. In 
addition, among the three military departments, the Air Force has 
satisfied the most core elements across all framework stages. Moreover, 
the Air Force has demonstrated the most progress in the last 2 years in 
satisfying the framework's core elements. However, the military 
departments have not yet met the conditions for the effective 
governance, content, use and measurement of their respective 
architecture programs. The Air Force has a solid foundation on which to 
continue building, but the Navy and, even more so, the Army has much to 
accomplish before either will have effective and mature architecture 
programs. As a result, DOD, as a whole, is not as well positioned as it 
should be to realize the significant benefits that a well-managed 
federation of architecture programs can provide. 

As we have previously reported, the key to having a mature architecture 
program, and thereby realizing the benefits of an architecture-centric 
approach to IT investment decision making, is sustained executive 
leadership. This is because virtually all of the barriers to 
effectively developing and using architectures, such as parochialism, 
cultural resistance, adequate resources, and top management 
understanding, can be addressed through such leadership. For the 
military departments to advance their respective architecture programs, 
sustained executive leadership will be needed. In this regard, the Navy 
and the Army could benefit from the lessons learned and experiences to 
date of the Air Force's efforts to mature its architecture program. 

Recommendation for Executive Action: 

Because we have outstanding recommendations to the Secretary of Defense 
aimed at, among other things, having the Departments of the Air Force, 
Navy, and Army fully satisfy each of the core elements in our 
architecture framework, we are not making additional recommendations 
relative to our framework at this time. However, given the uneven 
status and progress of the respective military departments, we 
reiterate our outstanding recommendations and further recommend that 
the Secretary of Defense direct the Secretaries of the Navy and Army to 
ensure that their respective departments reach out to the Department of 
the Air Force to learn from and apply the lessons and experiences that 
have allowed the Air Force to make the progress it has in maturing its 
architecture program. 

Agency Comments: 

In written comments on a draft of this report, signed by the Deputy 
Under Secretary of Defense (Business Transformation) and reprinted in 
appendix II, the department agreed with our recommendation, and 
described efforts underway and planned to implement it. 

We are sending copies of this report to interested congressional 
committees; the Director, Office of Management and Budget; and the 
Secretary of Defense. Copies of this report will be made available to 
other interested parties on request. This report will also be available 
at no charge on our Web site at [hyperlink, http://www.gao.gov]. 

If you or your staffs have any questions on matters discussed in this 
report, please contact me at (202) 512-3439 or [email protected]. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix VII. 

Signed by: 

Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 

List of Committees: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Daniel K. Inouye: 
Chairman: 
The Honorable Ted Stevens: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate: 

The Honorable Ike Skelton: 
Chairman: 
The Honorable Duncan L. Hunter: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Honorable John P. Murtha: 
Chairman: 
The Honorable C.W. Bill Young: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives: 

[End of section] 

Appendix I: Objective, Scope, and Methodology: 

Our objective was to determine the current status of the military 
departments' enterprise architecture efforts. To do so, we used a data 
collection instrument based on our Enterprise Architecture Management 
Maturity Framework (EAMMF), and related guidance, such as the Office of 
Management and Budget Circular A-130 and guidance published by the 
federal Chief Information Officers (CIO) Council, as well as our past 
reports and guidance on the management and content of enterprise 
architectures. We also met with the chief architects of the military 
departments to discuss our scope and methodology, share the data 
collection instrument, and discuss the type and nature of supporting 
documentation needed to verify responses to instrument questions. 

On the basis of documentation provided to support the departments' 
respective responses to our data collection instrument, we analyzed the 
extent to which each department satisfied the 31 core elements in our 
EAMMF. To guide our analyses, we used our standard evaluation criteria 
for determining whether a given core element was fully satisfied, 
partially satisfied, or not satisfied (See tables 7, 8, 9, and 10 for 
the core elements of Stages 2, 3, 4, and 5, respectively.) To fully 
satisfy a core element, sufficient documentation had to be provided to 
permit us to verify that all aspects of the core element were met. To 
partially satisfy a core element, sufficient documentation had to be 
provided to permit us to verify that at least some aspects of the core 
element were met. Core elements that did not meet criteria for fully or 
partially satisfied were judged to be not satisfied. 

Table 7: Stage 2 Evaluation Criteria: 

Core element: Adequate resources exist; 
Evaluation criteria: Agency responded that "very adequate," "somewhat 
adequate," or "neither adequate nor inadequate" resources exist for 
funding, personnel, and tools. 

Core element: Committee or group representing the enterprise is 
responsible for directing, overseeing, and approving enterprise 
architecture (EA); 
Evaluation criteria: Agency (1) responded that a committee or group 
representing the enterprise is responsible for direction, oversight, 
and approval of the enterprise architecture; (2) provided a charter or 
other documentation supporting the group's responsibilities; and (3) 
provided sample meeting minutes or other documentation confirming that 
meetings have been held. 

Core element: Program office responsible for EA development and 
maintenance exists; 
Evaluation criteria: Agency (1) responded that a program office is 
responsible for EA development and maintenance and (2) provided 
documentation supporting their assertion. 

Core element: Chief architect exists; 
Evaluation criteria: Agency (1) responded that chief architect exists 
and (2) provided documentation or assertion that the chief architect is 
responsible and accountable for EA and serves as the EA program 
manager. 

Core element: EA being developed using a framework, methodology, and 
automated tool; 
Evaluation criteria: Agency (1) responded that the enterprise 
architecture is being developed using a framework, methodology, and 
automated tool; (2) provided documentation supporting the use of a 
framework and automated tool; and (3) provided a documented methodology 
that includes steps for developing, maintaining, and validating the 
enterprise architecture. 

Core element: EA plans call for describing "as-is" environment, "to-be" 
environment, and sequencing plan; 
Evaluation criteria: Agency (1) responded that EA plans call for 
describing the "as-is" and "to-be" environments and a sequencing plan 
and (2) provided plans that document this assertion; or agency (1) 
responded that the EA describes the "as- is" and "to-be" environments 
and a sequencing plan and (2) provided documentation to support this 
assertion. 

Core element: EA plans call for describing enterprise in terms of 
business, performance, information/data, application/service, and 
technology; 
Evaluation criteria: Agency (1) responded that EA plans call for 
describing the enterprise in terms of business, performance, 
information/data, application/service, and technology and (2) provided 
plans that document this assertion; or agency (1) responded that the EA 
describes the enterprise in terms of business, performance, 
information/data, application/service, and technology and (2) provided 
documentation to support this assertion. 

Core element: EA plans call for business, performance, information/ 
data, application/service, and technology to address security; 
Evaluation criteria: Agency (1) responded that EA plans call for 
business, performance, information/data, application/service, and 
technology descriptions to address security and (2) provided plans that 
document this assertion; or agency (1) responded that the business, 
performance, information/data, application/service, and technology 
descriptions address security and (2) provided documentation to support 
this assertion. 

Core element: EA plans call for developing metrics to measure EA 
progress, quality, compliance, and return on investment; 
Evaluation criteria: Agency (1) responded that EA plans call for 
developing metrics to measure EA progress, quality, compliance, and 
return on investment and (2) provided plans to support this assertion; 
or responded (1) that EA progress, quality, compliance, and/or return 
on investment is measured and reported and (2) provided support for 
this assertion. 

Source: GAO. 

[End of table] 

Table 8: Stage 3 Evaluation Criteria: 

Core element: Written and approved policy exists for EA development; 
Evaluation criteria: Agency (1) responded that a written and approved 
organization policy exists for EA development and (2) provided a policy 
that supported this assertion. 

Core element: EA products are under configuration management; 
Evaluation criteria: Agency (1) responded that EA products are under 
configuration management and (2) provided their formally documented 
configuration management approach. 

Core element: EA products describe or will describe "as-is" 
environment, "to-be" environment, and sequencing plan; 
Evaluation criteria: Agency (1) responded that EA plans call for 
describing the "as-is" and "to-be" environments and a sequencing plan, 
(2) provided plans that document this assertion, and (3) responded that 
it is "in the process of developing the EA" or that it "has developed 
an EA"; or agency (1) responded that the EA describes the "as-is" and 
"to-be" environments and a sequencing plan and (2) provided 
documentation to support this assertion. 

Core element: Both "as-is" and "to-be" environments are described or 
will be described in terms of business, performance, information/data, 
application/service, and technology; 
Evaluation criteria: Agency (1) responded that EA plans call for 
describing the enterprise in terms of business, performance, 
information/data, application/service, and technology; (2) provided 
plans that document this assertion; and (3) responded that it is "in 
the process of developing the EA" or that it "has developed an EA"; or 
agency (1) responded that the EA describes the enterprise in terms of 
business, performance, information/data, application/service, and 
technology and (2) provided documentation to support this assertion. 

Core element: These descriptions address or will address security; 
Evaluation criteria: Agency (1) responded that EA plans call for 
business, performance, information/data, application/service, and 
technology descriptions to address security; (2) provided plans that 
document this assertion; and (3) responded that it is "in the process 
of developing the EA" or that it "has developed an EA"; or agency (1) 
responded that the business, performance, information/data, 
application/service, and technology descriptions address security and 
(2) provided documentation to support this assertion. 

Core element: Progress against EA plans is measured and reported; 
Evaluation criteria: Agency (1) responded that it measures and reports 
progress against plans; (2) provided a description of how progress 
against plans is measured and reported; and (3) provided sample reports 
that include sample measures. 

Source: GAO. 

[End of table] 

Table 9: Stage 4 Evaluation Criteria: 

Core element: Written and approved policy exists for EA maintenance; 
Evaluation criteria: Agency (1) responded that a written and approved 
organization policy exists for EA maintenance and (2) provided a policy 
that supported this assertion. 

Core element: EA products and management processes undergo independent 
verification and validation; 
Evaluation criteria: Agency (1) responded that EA products and 
management processes undergo independent verification and validation; 
(2) provided proof that independent verification and validation 
activities were conducted by an independent third party and reported 
outside the span of control of the chief architect; and (3) provided 
sample independent verification and validation (IV&V) reports to the 
audit team. Independence was a critical element for satisfaction of 
this item. 

Core element: EA products describe "as-is" environment, "to-be" 
environment, and sequencing plan; 
Evaluation criteria: Agency (1) responded that the EA describes the "as-
is" and "to-be" environments and a sequencing plan; (2) provided 
documentation to support this assertion; and (3) responded that it has 
developed an EA. In addition, an agency could not receive full credit 
for satisfying this element unless it fully satisfied the element, 
"Both 'as is' and 'to-be' environments are described in terms of 
business, performance, information/data, application/service, and 
technology." 

Core element: Both "as-is" and "to-be" environments are described in 
terms of business, performance, information/data, application/service, 
and technology; 
Evaluation criteria: Agency (1) responded that the EA describes the 
enterprise in terms of business, performance, information/data, 
application/service, and technology; (2) provided documentation to 
support this assertion; and (3) responded that it has developed an EA. 

Core element: These descriptions address security; 
Evaluation criteria: Agency (1) responded that the business, 
performance, information/data, application/service, and technology 
descriptions address security; (2) provided documentation to support 
this assertion; and (3) responded that it has developed an EA. 

Core element: Organization CIO has approved EA; 
Evaluation criteria: Agency (1) responded that the CIO has approved the 
current version of the EA and (2) provided a signature page or other 
proof that the CIO has approved the current version of the EA. 

Core element: Committee or group representing the enterprise or the 
investment review board has approved current version of EA; 
Evaluation criteria: Agency (1) responded that a committee or group 
representing the enterprise or the investment review board has approved 
the current version of the EA and (2) provided meeting minutes or other 
proof that a committee or group representing the enterprise or the 
investment review board has approved the current version of the EA. 

Core element: Quality of EA products is measured and reported; 
Evaluation criteria: Agency (1) responded that it measures and reports 
product quality, (2) provided a description of how quality is measured 
and reported, and (3) provided sample reports that include sample 
measures. 

Source: GAO. 

[End of table] 

Table 10: Stage 5 Evaluation Criteria: 

Core element: Written and approved organization policy exists for 
Information Technology (IT) investment compliance with EA; 
Evaluation criteria: Agency (1) responded that a written and approved 
organization policy exists for IT investment compliance with EA and (2) 
provided a written policy to support this assertion. 

Core element: Process exists to formally manage EA change; 
Evaluation criteria: Agency (1) responded that a process exists to 
formally manage EA change and (2) provided evidence to support this 
assertion. 

Core element: EA is integral component of IT investment management 
process; 
Evaluation criteria: Agency (1) responded that EA is an integral 
component of IT investment management process; (2) provided 
documentation describing how the EA is used when making IT investment 
decisions; (3) provided evidence that a sequencing plan exists to guide 
IT investments; and (4) partially or fully satisfied at least one of 
the following Stage 3 elements: (a) EA products describe or will 
describe "as-is" environment, "to-be" environment, and sequencing plan; 
(b) both "as-is" and "to-be" environments are described or will be 
described in terms of business, performance, information/data, 
application/service, and technology; or (c) these descriptions address 
or will address security. 

Core element: EA products are periodically updated; 
Evaluation criteria: Agency (1) responded that EA products are 
periodically updated and (2) provided a description of the process used 
for updating EA products. 

Core element: IT investments comply with EA; 
Evaluation criteria: Agency (1) responded that IT investments comply 
with EA; (2) provided evidence that IT is not selected and approved 
under the organization's capital planning and investment control 
process unless it is compliant with the EA; and (3) partially or fully 
satisfied at least one of the following Stage 3 elements: (a) EA 
products describe or will describe "as-is" environment, "to-be" 
environment, and sequencing plan; (b) both "as-is" and "to-be" 
environments are described or will be described in terms of business, 
performance, information/data, application/service, and technology; or 
(c) these descriptions address or will address security. 

Core element: Organization head has approved current version of EA; 
Evaluation criteria: Agency (1) responded that the organization head 
has approved the current version of the EA; (2) provided a signature 
page or other proof that organization head or a deputy organization 
head has approved current version of EA or provided proof of formal 
delegation of this activity and subsequent approval; and (3) partially 
or fully satisfied at least one of the following Stage 3 elements: (a) 
EA products describe or will describe "as-is" environment, "to-be" 
environment, and sequencing plan; (b) both "as-is" and "to-be" 
environments are described or will be described in terms given in Stage 
2; or (c) these descriptions address or will address security. 

Core element: Return on EA investment is measured and reported; 
Evaluation criteria: Agency (1) responded that return on investment has 
been measured and reported; (2) provided a description of how return on 
investment is measured and reported; and (3) provided sample reports 
that included sample measures. 

Core element: Compliance with EA is measured and reported; 
Evaluation criteria: Agency (1) responded that compliance has been 
measured and reported, (2) provided a description of how compliance is 
measured and reported, and (3) provided sample reports that included 
sample measures. 

Source: GAO. 

[End of table] 

In applying our methodology, we first analyzed and determined the 
extent to which each department satisfied the core elements in our 
framework, and then met with their representatives to discuss core 
elements that were not satisfied and why. As part of this interaction, 
we sought, and in some cases were provided, additional supporting 
documentation. We then considered this documentation when determining 
the degree to which each department satisfied each core element. In 
applying our evaluation criteria, we analyzed the results across 
different core elements to determine patterns and issues. 

We conducted our work in the metropolitan area of Washington, D.C., 
from September 2007 through March 2008, in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audit to obtain sufficient, appropriate evidence 
to provide a reasonable basis for our findings and conclusions based on 
our audit objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Office Of The Under Secretary Of Defense: 
Acquisition Technology And Logistics: 
3000 Defense Pentagon: 
Washington, DC 20301-3000: 

April 16, 2008: 

Mr. Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. Hite: 

This is the Department of Defense (DoD) response to the GAO Draft 
Report, GAO-08-519, "DOD Business Systems Modernization: Military 
Departments Need to Strengthen Management of Enterprise Architecture 
Programs" dated March 10, 2008 (GAO Code 310652). Detail comments on 
the report recommendations are enclosed. 

This report reiterates the value, already recognized by the Department, 
of sharing lessons learned in architecture development and 
implementation across the Business Mission Area (BMA). Enterprise 
architecture development across the Department has gained significant 
momentum and continues to mature at both the enterprise and component 
levels, although, as noted in the GAO report, some components are 
further along in development than others. 

To further maturation and share benchmark "best practices," two 
important information sharing venues have been created. The Bi-Weekly 
BMA CIO Meeting and the Architecture Summits provide forums for sharing 
the lessons learned described in the single GAO recommendation. The 
Department continues to receive positive feedback on the value of these 
venues and views them as clear channels for cross-departmental 
information sharing. 

Additionally, the Secretary of Defense, acting through the Chief 
Management Officer, will direct the Departments of the Army and Navy to 
reach out to the Department of the Air Force to seek additional 
opportunities to transfer architecture best practices at the enterprise 
and component levels. As we continue to improve the Department's 
business transformation efforts, we welcome GAO's insight and 
partnership in reaching the common goal of maturing and strengthening 
our enterprise architecture programs. 

Signed by: 

Paul A. Brinkley: 
Deputy Under Secretary of Defense: 
(Business Transformation): 

Enclosure: As stated: 

GAO Draft Report Dated March 10, 2008: 
GAO-08-519 (GAO CODE 310652): 

"DOD Business Systems Modernization: Military Departments Need To 
Strengthen Management Of Enterprise Architecture Programs" 

Department Of Defense Comments To The GAO Recommendation: 

Recommendation: The GAO recommends that the Secretary of Defense direct 
the Secretaries of the Army and Navy to ensure that their respective 
Departments reach out to the Department of the Air Force to learn from 
and apply the lessons and experiences that have allowed the Air Force 
to make the progress it has in maturing its architecture program. (p. 
38/GAO Draft Report) 

DOD Response: Concur. 

The Department strongly values the sharing of information across the 
enterprise. Many formal and informal architecture-related discussions 
are currently taking place. Two examples of formal discussion include 
the Bi-Weekly Chief Information Officer (CTO) Meetings and a recent 
Architecture Summit conducted in March 2008. These venues have included 
CIO representation from the DoD services and Agencies to include the 
Department of the Army, Navy and Air Force. Information exchanges such 
as these and others have proven valuable in transferring knowledge 
across the enterprise. 

Additionally, the Secretary of Defense, acting through the Chief 
Management Officer, will direct the Departments of the Army and Navy to 
reach out to the Department of the Air Force to seek additional 
opportunities to transfer architecture best practices at the enterprise 
and component levels where appropriate given the unique differences of 
each organization. 

[End of section] 

Appendix III: Brief History of Architecture Frameworks and Management 
Guidance: 

During the mid-1980s, John Zachman, widely recognized as a leader in 
the field of enterprise architecture, identified the need to use a 
logical construction blueprint (i.e., an architecture) for defining and 
controlling the integration of systems and their components.[Footnote 
33] Accordingly, Zachman developed a structure, or framework, for 
defining and capturing an architecture, which provides for six 
perspectives, or "windows," from which to view the enterprise.[Footnote 
34] Zachman also proposed six abstractions, or models, associated with 
each of these perspectives.[Footnote 35] Zachman's framework provides a 
way to identify and describe an entity's existing and planned component 
parts and the parts' relationships before the entity begins the costly 
and time-consuming efforts associated with developing or transforming 
itself. 

Since Zachman introduced his framework, a number of frameworks have 
emerged within the federal government, beginning with the publication 
of the National Institute of Standards and Technology framework in 
1989. Since that time, other federal entities have issued frameworks, 
including the Department of Defense and the Department of the Treasury. 
In September 1999, the federal Chief Information Officers Council 
published the Federal Enterprise Architecture Framework, which was 
intended to provide federal agencies with a common construct for their 
architectures, thereby facilitating the coordination of common business 
processes, technology insertion, information flows, and system 
investments among federal agencies. The Federal Enterprise Architecture 
Framework described an approach, including models and definitions, for 
developing and documenting architecture descriptions for 
multiorganizational functional segments of the federal government. 
[Footnote 36] 

More recently, Office of Management and Budget (OMB) established the 
Federal Enterprise Architecture Program Management Office to develop a 
federal enterprise architecture according to a collection of five 
"reference models" (see table 11). These models are intended to 
facilitate governmentwide improvement through cross-agency analysis and 
the identification of duplicative investments, gaps, and opportunities 
for collaboration, interoperability, and integration within and across 
government agencies. 

Table 11: Federal Enterprise Architecture Reference Models: 

Reference model: Performance Reference Model; 
Description: Provides a common set of general performance outputs and 
measures for agencies to use to achieve business goals and objectives. 

Reference model: Business Reference Model; 
Description: Describes the business operations of the federal 
government independent of the agencies that perform them, including 
defining the services provided to state and local governments. 

Reference model: Service Component Reference Model; 
Description: Identifies and classifies IT service (i.e., application) 
components that support federal agencies and promotes the reuse of 
components across agencies. 

Reference model: Data and Information Reference Model; 
Description: Describes, at an aggregate level, the types of data and 
information that support program and business line operations, and the 
relationships among these types. 

Reference model: Technical Reference Model; Description: 
Describes how technology is supporting the delivery of service 
components, including relevant standards for implementing the 
technology. 

Source: GAO. 

[End of table] 

Although these post-Zachman frameworks differ in their nomenclatures 
and modeling approaches, each consistently provides for defining an 
enterprise's operations in both logical and technical terms, provides 
for defining these perspectives for the enterprise's current and target 
environments, and calls for a transition plan between the two. 

Legislation and federal guidance address enterprise architecture. 
Specifically, the Clinger-Cohen Act of 1996 directs the CIOs of major 
departments and agencies to develop, maintain, and facilitate the 
implementation of information technology architectures as a means of 
integrating agency goals and business processes with information 
technology.[Footnote 37] Also, OMB Circular A-130, which implements the 
Clinger-Cohen Act, requires that agencies document and submit their 
initial enterprise architectures and that agencies submit updates when 
significant changes to their enterprise architectures occur. The 
circular also directs OMB to use various reviews to evaluate the 
adequacy and efficiency of each agency's compliance with the circular. 

Since then, OMB has developed and implemented an enterprise 
architecture assessment tool. According to OMB, the tool helps to 
illustrate the current state of an agency's architecture and assists 
agencies in integrating architectures into their decision-making 
processes. The latest version of the assessment tool (2.0) was released 
in December 2005 and includes three capability areas: (1) completion, 
(2) use, and (3) results. Table 12 describes each of these areas. 

Table 12: OMB EA Assessment Framework Capability Areas: 

Capability area: Completion; 
Description: Addresses ensuring that architecture products describe the 
agency in terms of processes, services, data, technology, and 
performance and that the agency has developed a transition strategy. 

Capability area: Use; 
Description: Addresses the establishment of important management 
practices, processes, and policies, such as configuration management, 
communications, and integration of the architecture with capital 
planning processes. 

Capability area: 
Results; Description: Addresses the effectiveness and value of the 
architecture by encouraging performance measurements and using it to 
ensure agency policies align to OMB IT policy. 

Source: OMB. 

[End of table] 

The tool also includes criteria for scoring an agency's architecture 
program on a scale of 0 to 5.[Footnote 38] In early 2006, the major 
departments and agencies were required by OMB to provide a self- 
assessment of their architecture programs using the tool. OMB then used 
the self assessment to develop its own assessment. These assessment 
results are to be used in determining the agency's e-Government score 
within the President's Management Agenda. 

[End of section] 

Appendix IV: Table: Department of the Air Force Assessment: 

Stages and elements: Stage 1: Creating EA awareness: Agency is aware of 
EA; 
Extent to which element is satisfied: n/a. 

Stages and elements: Stage 2: Building the EA management foundation: 
Adequate resources exist; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: 
Committee or group representing the enterprise is responsible for 
directing, overseeing, and approving EA; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: 
Program office responsible for EA development and maintenance exists; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: 
Chief architect exists; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
being developed using a framework, methodology, and automated tool; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for describing "as-is" environment, "to-be" environment, and 
sequencing plan; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for describing enterprise in terms of business, performance, 
information/data, applications/service, and technology; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for business, performance, information/data, 
applications/service, and technology to address security; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for developing metrics to measure EA progress, quality, 
compliance, and return on investment; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 3: Developing architecture products: Written 
and approved policy exists for EA development; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 3: Developing architecture products: EA 
products are under configuration management; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 3: Developing architecture products: EA 
products describe or will describe "as-is" environment, "to-be" 
environment, and sequencing plan; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 3: Developing architecture products: Both 
"as-is" and "to-be" environments are described or will be described in 
terms given in Stage 2; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 3: Developing architecture products: These 
descriptions address or will address security; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 3: Developing architecture products: 
Progress against EA plans is measured and reported; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: Written 
and approved policy exists for EA maintenance; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 4: Completing architecture products: EA 
products and management processes undergo independent verification and 
validation; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: EA 
products describe "as-is" environment, "to-be" environment, and 
sequencing plan; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: Both 
"as-is" and "to-be" environments are described in terms given in Stage 
2; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: These 
descriptions address security; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: 
Organization CIO has approved EA; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 4: Completing architecture products: 
Committee or group representing the enterprise or the investment review 
board has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: Quality 
of EA products is measured and reported; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Written and approved organization policy exists for IT 
investment compliance with EA; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Process exists to formally manage EA change; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: EA is integral component of IT investment management process; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: EA products are periodically updated; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: IT investments comply with EA; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Organization head has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Return on EA investment is measured and reported; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Compliance with EA is measured and reported; 
Extent to which element is satisfied: Partial. 

Source: GAO analysis of agency data. 

[End of table] 

[End of section] 

Appendix V: Table: Department of the Navy Assessment: 

Stages and elements: Stage 1: Creating EA awareness: Agency is aware of 
EA; 
Extent to which element is satisfied: n/a. 

Stages and elements: Stage 2: Building the EA management foundation: 
Adequate resources exist; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: 
Committee or group representing the enterprise is responsible for 
directing, overseeing, and approving EA; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: 
Program office responsible for EA development and maintenance exists; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: 
Chief architect exists; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
being developed using a framework, methodology, and automated tool; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for describing "as-is" environment, "to-be" environment, and 
sequencing plan; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for describing enterprise in terms of business, performance, 
information/data, applications/service, and technology; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for business, performance, information/data, 
applications/service, and technology to address security; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for developing metrics to measure EA progress, quality, 
compliance, and return on investment; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 3: Developing architecture products: Written 
and approved policy exists for EA development; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 3: Developing architecture products: EA 
products are under configuration management; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: EA 
products describe or will describe "as-is" environment, "to-be" 
environment, and sequencing plan; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 3: Developing architecture products: Both 
"as-is" and "to-be" environments are described or will be described in 
terms given in Stage 2; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 3: Developing architecture products: These 
descriptions address or will address security; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 3: Developing architecture products: 
Progress against EA plans is measured and reported; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 4: Completing architecture products: Written 
and approved policy exists for EA maintenance; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: EA 
products and management processes undergo independent verification and 
validation; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: EA 
products describe "as-is" environment, "to-be" environment, and 
sequencing plan; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: Both 
"as-is" and "to-be" environments are described in terms given in Stage 
2; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: These 
descriptions address security; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 4: Completing architecture products: 
Organization CIO has approved current version EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: 
Committee or group representing the enterprise or the investment review 
board has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: Quality 
of EA products is measured and reported; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Written and approved organization policy exists for IT 
investment compliance with EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Process exists to formally manage EA change; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: EA is integral component of IT investment management process; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: EA products are periodically updated; 
Extent to which element is satisfied: Yes. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: IT investments comply with EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Organization head has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Return on EA investment is measured and reported; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Compliance with EA is measured and reported; 
Extent to which element is satisfied: No. 

Source: GAO analysis of agency data. 

[End of table] 

[End of section] 

Appendix VI: Table: Department of the Army Assessment: 

Stages and elements: Stage 1: Creating EA awareness: Agency is aware of 
EA; 
Extent to which element is satisfied: n/a. 

Stages and elements: Stage 2: Building the EA management foundation: 
Adequate resources exist; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: 
Committee or group representing the enterprise is responsible for 
directing, overseeing, and approving EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 2: Building the EA management foundation: 
Program office responsible for EA development and maintenance exists; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: 
Chief architect exists; 
Extent to which element is satisfied: Yes. 

Stages and elements: EA being developed using a framework, methodology, 
and automated tool; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for describing "as-is" environment, "to-be" environment, and 
sequencing plan; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for describing enterprise in terms of business, performance, 
information/data, applications/service, and technology; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for business, performance, information/data, 
applications/service, and technology to address security; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 2: Building the EA management foundation: EA 
plans call for developing metrics to measure EA progress, quality, 
compliance, and return on investment; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: Written 
and approved policy exists for EA development; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: EA 
products are under configuration management; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: EA 
products describe or will describe "as-is" environment, "to-be" 
environment, and sequencing plan; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: Both 
"as-is" and "to-be" environments are described or will be described in 
terms given in Stage 2; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: These 
descriptions address or will address security; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 3: Developing architecture products: 
Progress against EA plans is measured and reported; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: Written 
and approved policy exists for EA maintenance; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: EA 
products and management processes undergo independent verification and 
validation; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: EA 
products describe "as-is" environment, "to-be" environment, and 
sequencing plan; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: Both 
"as-is" and "to-be" environments are described in terms given in Stage 
2; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: These 
descriptions address security; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: 
Organization CIO has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: 
Committee or group representing the enterprise or the investment review 
board has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 4: Completing architecture products: Quality 
of EA products is measured and reported; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Written and approved organization policy exists for IT 
investment compliance with EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Process exists to formally manage EA change; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: EA is integral component of IT investment management process; 
Extent to which element is satisfied: Partial. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: EA products are periodically updated; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: IT investments comply with EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Organization head has approved current version of EA; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Return on EA investment is measured and reported; 
Extent to which element is satisfied: No. 

Stages and elements: Stage 5: Leveraging the architecture to manage 
change: Compliance with EA is measured and reported; 
Extent to which element is satisfied: No. 

Source: GAO analysis of agency data. 

[End of table] 

[End of section] 

Appendix VII: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Randolph C. Hite, (202) 512-3439 or [email protected]: 

Staff Acknowledgments: 

In addition to the contact named above, Tonia Johnson (Assistant 
Director), Timothy Eagle, Elena Epps, Michael Holland, Neela Lakhmani, 
Rebecca LaPaze, Anh Le, and Freda Paintsil made key contributions to 
this report. 

[End of section] 

Footnotes: 

[1] GAO, Information Technology: Architecture Needed to Guide 
Modernization of DOD's Financial Operations, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-01-525] (Washington, D.C.: May 
17, 2001). 

[2] GAO, DOD Business Systems Modernization: Progress Continues to Be 
Made in Establishing Corporate Management Controls, but Further Steps 
Are Needed, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-733] 
(Washington, D.C.: May 14, 2007); GAO, Business System Modernization: 
Strategy for Evolving DOD's Business Enterprise Architecture Offers a 
Conceptual Approach, but Execution Details Are Needed, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-451] (Washington, D.C.: Apr. 
16, 2007), GAO, Defense Business Transformation: A Comprehensive Plan, 
Integrated Efforts, and Sustained Leadership Are Needed to Assure 
Success, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-229T] 
(Washington, D.C.: Nov. 16, 2006); GAO, Business Systems Modernization: 
DOD Continues to Improve Institutional Approach, but Further Steps 
Needed, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-658] 
(Washington, D.C.: May 15, 2006); GAO, DOD Business Systems 
Modernization: Long-Standing Weaknesses in Enterprise Architecture 
Development Need to Be Addressed, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-05-702] (Washington, D.C.: July 22, 2005); GAO, DOD 
Business Systems Modernization: Limited Progress in Development of 
Business Enterprise Architecture and Oversight of Information 
Technology Investments, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-731R] (Washington, D.C.: May 17, 2004); and GAO, DOD 
Business Systems Modernization: Important Progress Made to Develop 
Business Enterprise Architecture, but Much Work Remains, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-1018] (Washington, D.C.: Sept. 
19, 2003). 

[3] Ronald W. Reagan National Defense Authorization Act for Fiscal Year 
2005, Pub. L. No. 108-375, ï¿½ 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 
2004) (codified in part at 10 U.S.C. ï¿½ 2222). 

[4] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-451]. 

[5] GAO, Information Technology: A Framework for Assessing and 
Improving Enterprise Architecture Management (Version 1.1), [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-584G] (Washington D.C.: Apr. 
2003). 

[6] GAO, Enterprise Architecture: Leadership Remains Key to 
Establishing and Leveraging Architectures for Organizational 
Transformation, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-
831] (Washington D.C.: Aug. 14, 2006). 

[7] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-658]. 

[8] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-310]. 

[9] These 8 high-risk areas include DOD's overall approach to (1) 
business transformation, (2) business systems modernization, (3) 
financial management, (4) the personnel security clearance program, (5) 
supply chain management, (6) support infrastructure management, (7) 
weapon systems acquisition, and (8) contract management. 

[10] The 7 governmentwide high-risk areas are: (1) disability programs, 
(2) ensuring the effective protection of technologies critical to U.S. 
national security interests, (3) interagency contracting, (4) 
information systems and critical infrastructure, (5) information- 
sharing for homeland security, (6) human capital, and (7) real 
property. 

[11] GAO, Homeland Security: Efforts Under Way to Develop Enterprise 
Architecture, but Much Work Remains, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-777] (Washington, D.C.: Aug. 6, 2004); GAO, 
Information Technology: Architecture Needed to Guide NASA's Financial 
Management Modernization, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-43] (Washington, D.C.: Nov. 21, 2003); GAO, 
Information Technology: DLA Should Strengthen Business Systems 
Modernization Architecture and Investment Activities, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-01-631] (Washington, D.C.: June 
29, 2001); [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-731R]; 
and [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-1018]. 

[12] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-584G]. 

[13] Chief Information Officers Council, A Practical Guide to Federal 
Enterprise Architecture, Version 1.0 (February 2001). 

[14] This set of products is consistent with OMB's federal enterprise 
architecture reference models. 

[15] The OMB EA Assessment Framework Version 2.2 tool helps illustrate 
the current state of an agency's architecture and assists agencies in 
integrating architectures into their decision-making processes. The 
latest version of the assessment tool (2.2) was released in October 
2007 and includes three capability areas: (1) completion, (2) use, and 
(3) results. The tool also includes criteria for scoring an agency's 
architecture program on a scale of 0 to 5. A score of 0 means 
undefined, 1 means initial, 2 means managed, 3 means used, 4 means 
results-oriented, and 5 means optimized. 

[16] The Warfighting Mission Area focuses on transforming how DOD 
achieves its mission objectives by addressing joint warfighting 
capabilities and providing life-cycle oversight to applicable DOD 
component and combatant command IT investments. 

[17] The Business Mission Area is responsible for ensuring that 
capabilities, resources, and materiel are reliably delivered to the 
warfighter. Specifically, the Business Mission Area addresses areas 
such as real property and human resources management. 

[18] The DOD Intelligence Mission Area is focused on establishing 
advanced capabilities to anticipate adversaries and includes IT 
investments within the military intelligence program and defense 
component programs of the National Intelligence Program. 

[19] The Enterprise Information Environment Mission Area enables the 
functions of the other mission areas (e.g., Warfighting Mission Area, 
Business Mission Area, and Intelligence Mission Area) and encompasses 
all communications; computing; and core enterprise service systems, 
equipment, or software that provides a common information capability or 
service for enterprise use. 

[20] According to DOD, GIG consists of a globally interconnected, end- 
to-end set of information capabilities, associated processes, and 
personnel for collecting, processing, storing, disseminating, and 
managing information on demand to warfighters, policymakers, and 
support personnel. As such, the GIG architecture spans all four mission 
areas. 

[21] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-525], 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-458], [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-571R], [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-877R], [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-1018], [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-04-731R], [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-05-381], and [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-05-702]. 

[22] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-1018]. 

[23] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-451]. 

[24] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-831]. 

[25] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-451]. 

[26] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-733]. 

[27] See, for example, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-01-525], [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-03-458], [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-731R], [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-05-702], and GAO, DOD Business Systems Modernization: 
Important Progress Made in Establishing Foundational Architecture 
Products and Investment Management Practices, but Much Work Remains, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-219] (Washington, 
D.C.: Nov. 23, 2005). 

[28] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-702]. 

[29] Partially satisfied means that a department has addressed some, 
but not all, aspects of the core element. 

[30] Metis ArchitectTM is an architecture tool from Troux Technologies, 
Inc. 

[31] Telelogic System Architectï¿½ is a set of architecture tools from 
Telelogic, Inc. 

[32] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-831]. 

[33] J. A. Zachman, "A Framework for Information Systems Architecture," 
IBM Systems Journal vol. 26, no. 3 (1987). 

[34] The windows include (1) the strategic planner, (2) the system 
user, (3) the system designer, (4) the system developer, (5) the 
subcontractor, and (6) the system itself. 

[35] The models cover (1) how the entity operates, (2) what the entity 
uses to operate, (3) where the entity operates, (4) who operates the 
entity, (5) when entity operations occur, and (6) why the entity 
operates. 

[36] Similar to Zachman's framework, the Federal Enterprise 
Architecture Framework's proposed models describe an entity's business, 
data necessary to conduct the business, applications to manage the 
data, and technology to support the applications. 

[37] 40 U.S.C. sections 11315. 

[38] A score of 0 means undefined, 1 means initial, 2 means managed, 3 
means utilized, 4 means results-oriented, and 5 means optimized. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***