Enterprise Architecture: Leadership Remains Key to Establishing
and Leveraging Architectures for Organizational Transformation
(14-AUG-06, GAO-06-831).
A well-defined enterprise architecture is an essential tool for
leveraging information technology (IT) to transform business and
mission operations. GAO's experience has shown that attempting to
modernize and evolve IT environments without an architecture to
guide and constrain investments results in operations and systems
that are duplicative, not well integrated, costly to maintain,
and ineffective in supporting mission goals. In light of the
importance of enterprise architectures, GAO developed a five
stage architecture management maturity framework that defines
what needs to be done to effectively manage an architecture
program. Under GAO's framework, a fully mature architecture
program is one that satisfies all elements of all stages of the
framework. As agreed, GAO's objective was to determine the status
of major federal department and agency enterprise architecture
efforts.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-831
ACCNO: A58602
TITLE: Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation
DATE: 08/14/2006
SUBJECT: Agency missions
Enterprise architecture
Federal enterprise architecture
framework
Information technology
Program management
Systems design
Systems management
Technology modernization programs
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-831
* Report to the Chairman, Committee on Government Reform, House of
Representatives
* August 2006
* ENTERPRISE ARCHITECTURE
* Leadership Remains Key to Establishing and Leveraging
Architectures for Organizational Transformation
* Contents
* Results in Brief
* Background
* Enterprise Architecture Description and Importance
* Brief History of Architecture Frameworks and Management
Guidance
* A Decade of GAO Work Has Focused on Improving Agency
Enterprise Architecture Efforts
* GAO's Enterprise Architecture Management Maturity Framework
(EAMMF)
* EAMMF Stages
* EAMMF Attributes
* EAMMF Groups
* Overall State of Enterprise Architecture Management Is a Work-
in-Progress, Although a Few Agencies Have Largely Satisfied Our
Framework
* The Degree to which Major Departments and Agencies Have
Fully Satisfied Our Framework's Core Elements Is Uneven and
Their Collective Efforts Can Be Viewed as a Work-in-
Progress
* Most Agencies Have at Least Partially Satisfied Most
Framework Elements
* Seven Departments or Agencies Need to Satisfy Five or Fewer
Core Elements to Be at Stage 5
* Departments and Agencies Report Numerous Challenges Facing
Them in Developing and Using Enterprise Architectures
* Many Departments and Agencies Reported That They Have
Already Realized Significant Architecture Benefits, While
Most Expect to Do So in the Future
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Reported Enterprise Architecture Costs Vary, with Contractors and
Personnel Accounting for Most Costs
* Architecture Development and Maintenance Costs Vary
* Contractor Support Accounts for the Majority of Architecture
Development Costs
* Architecture Development Activities Were Reported as Largest
Component of Contractor-Related Costs
* Departments and Agencies Reported Experiences with Their Architecture
Tools and Frameworks
* Departments and Agencies Reported Using a Variety of Enterprise
Architecture Tools with Varying Degrees of Satisfaction
* Departments and Agencies Reported Using a Variety of Enterprise
Architecture Frameworks with Varying Levels of Satisfaction
* Objective, Scope, and Methodology
* Detailed Assessments of Individual Departments and Agencies against
Our EA Management Maturity Framework
* Department of Agriculture
* Department of the Air Force
* Department of the Army
* Department of Commerce
* Department of Defense - Business Enterprise Architecture
* Department of Defense - Global Information Grid
* Department of Education
* Department of Energy
* Department of Health and Human Services
* Department of Homeland Security
* Department of Housing and Urban Development
* The Department of the Interior
* Department of Justice
* Department of Labor
* Department of the Navy
* Department of State
* Department of Transportation
* Department of the Treasury
* Department of Veterans Affairs
* Environmental Protection Agency
* General Services Administration
* National Aeronautics and Space Administration
* National Science Foundation
* Nuclear Regulatory Commission
* Office of Personnel Management
* Small Business Administration
* Social Security Administration
* U.S. Agency for International Development
* Comments from the Department of Commerce
* Comments from the Department of Defense
* GAO Comments
* Comments from the Department of Education
* Comments from the Department of Energy
* Comments from the Department of Homeland Security
* GAO Comments
* Comments from the Department of Housing and Urban Development
* Comments from the Department of the Interior
* Comments from the Department of Justice
* GAO Comments
* Comments from the Department of State
* GAO Comments
* Comments from the Department of the Treasury
* Comments from the Department Veterans Affairs
* Comments from the Environmental Protection Agency
* GAO Comments
* Comments from the General Services Administration
* Comments from the National Aeronautics and Space Administration
* Comments from the Social Security Administration
* GAO Comments
* Comments from the U.S. Agency for International Development
* GAO Contact and Staff Acknowledgements
United States Government Accountability Office
Report to the Chairman, Committee on Government Reform, House of
Representatives
August 2006
ENTERPRISE ARCHITECTURE
Leadership Remains Key to Establishing and Leveraging Architectures for
Organizational Transformation
a
GAO-06-831
ENTERPRISE ARCHITECTURE
Leadership Remains Key to Establishing and Leveraging Architectures for
Organizational Transformation
What GAO Found
The state of the enterprise architecture programs at the 27 major federal
departments and agencies is mixed, with several having very immature
programs, several having more mature programs, and most being somewhere in
between. Collectively, the majority of these architecture efforts can be
viewed as a work-in-progress with much remaining to be accomplished before
the federal government as a whole fully realizes their transformational
value. More specifically, seven architecture programs have advanced beyond
the initial stage of the GAO framework, meaning that they have fully
satisfied all core elements associated with the framework's second stage
(establishing the management foundation for developing, using, and
maintaining the architecture). Of these seven, three have also fully
satisfied all the core elements associated with the third stage
(developing the architecture). None have fully satisfied all of the core
elements associated with the fourth (completing the architecture) and
fifth (leveraging the architecture for organizational change) stages.
Nevertheless, most have fully satisfied a number of the core elements
across the stages higher than the stage in which they have met all core
elements, with all 27 collectively satisfying about 80, 78, 61, and 52
percent of the stage two through five core elements, respectively (see
figure). Further, most have partially satisfied additional elements across
all the stages, and seven need to fully satisfy five or fewer elements to
achieve the fifth stage.
The key to these departments and agencies building upon their current
status, and ultimately realizing the benefits that they cited
architectures providing, is sustained executive leadership, as virtually
all the challenges that they reported can be addressed by such leadership.
Examples of the challenges are organizational parochialism and cultural
resistance, adequate resources (human capital and funding), and top
management understanding; examples of benefits cited are better
information sharing, consolidation, improved productivity, and reduced
costs.
Percentage of Framework Elements Collectively Satisfied by All Departments
and Agencies in Each Stage
0 1020304050607080 Percentage
Source: GAO analysis of department/agency data.
Note: There are no framework elements in stage 1.
United States Government Accountability Office
Contents
Letter 1
Results in Brief 2
Background 4
Overall State of Enterprise Architecture Management Is a Work-in-Progress,
Although a Few Agencies Have Largely Satisfied Our Framework 18
Conclusions 37
Recommendations for Executive Action 37
Agency Comments and Our Evaluation 38
Appendixes
:Objective, Scope, and Methodology 56
: Detailed Assessments of Individual Departments and Agencies against Our
EA Management Maturity Framework 64
:Departments and Agencies Reported Experiences with Their Architecture
Tools and Frameworks 51
:Reported Enterprise Architecture Costs Vary, with Contractors and
Personnel Accounting for Most Costs 45
: Comments from the Department of Commerce 135
:Comments from the Department of Defense 136
:Comments from the Department of Education 139
:Comments from the Department of Energy 141
:Comments from the Department of Homeland Security 142
:Comments from the Department of the Interior 150
:Comments from the Department of Justice 151
:Comments from the Department of State 154
: Comments from the Department of the Treasury 161
:Comments from the Department of Housing and Urban Development 146
GAO Comments 138
GAO Comments 144
GAO Comments 153
GAO Comments 160
Appendix XV: Comments from the Department Veterans Affairs 163
Appendix XVI: Comments from the Environmental Protection Agency 164 166
GAO Comments
Appendix XVII: Comments from the General Services Administration 167
Appendix XVIII: Comments from the National Aeronautics and Space
Administration 168
Appendix XIX: Comments from the Social Security Administration 169 172
GAO Comments
Appendix XX: Comments from the U.S. Agency for International
Development 173
Appendix XXI: GAO Contact and Staff Acknowledgements 174
Tables
Table 1: Federal Enterprise Architecture Reference Models 7
Table 2: OMB Enterprise Architecture Assessment Framework
Capability Areas 11
Table 3: Summary of EAMMF Version 1.1: Core Elements
Categorized by Group 17
Table 4: Maturity Stage of Major Department and Agency
Enterprise Architecture Programs 19
Table 5: Percent of Framework Elements Satisfied by Department
and Agency Architecture Programs within Each Maturity
Stage 21
Table 6: Departments and Agencies That Need to Satisfy 5 or Fewer
Core Elements to Achieve Stage 5 33
Table 7: Degree to Which Departments and Agencies Are
Experiencing Enterprise Architecture Challenges 34
Table 8: Enterprise Architecture Benefits Reported As Being or To
Be Achieved to a Significant Extent 36
Table 9: Department and Agency Reported Satisfaction with
Tools 53
Table 10: Department and Agency Framework Satisfaction Levels 55
Table 11: List of Architecture Programs Included in this Report 56
Table 12: Stage 2 Evaluation Criteria 58
Table 13: Stage 3 Evaluation Criteria 59
Table 14: Stage 4 Evaluation Criteria 60
Table 15: Stage 5 Evaluation Criteria 61
Table 16: Department of Agriculture Satisfaction of EAMMF 64
Table 17: Department of the Air Force Satisfaction of EAMMF 67
Table 18: Department of the Army Satisfaction of EAMMF 70
Table 19: Department of Commerce Satisfaction of EAMMF 73 Table 20: DOD
Business Enterprise Architecture Satisfaction of
EAMMF 75 Table 21: DOD Global Information Grid Satisfaction of EAMMF 78
Table 22: Department of Education Satisfaction of EAMMF 81 Table 23:
Department of Energy Satisfaction of EAMMF 83 Table 24: Department of
Health and Human Services Satisfaction of
EAMMF 85 Table 25: Department of Homeland Security Satisfaction of
EAMMF 87 Table 26: Department of Housing and Urban Development
Satisfaction of EAMMF 89 Table 27: Department of the Interior Satisfaction
of EAMMF 91 Table 28: Department of Justice Satisfaction of EAMMF 93 Table
29: Department of Labor Satisfaction of EAMMF 95 Table 30: Department of
the Navy Satisfaction of EAMMF 97 Table 31: Joint Enterprise Architecture
Satisfaction of EAMMF 100 Table 32: Department of Transportation
Satisfaction of EAMMF 103 Table 33: Department of the Treasury
Satisfaction of EAMMF 105 Table 34: Department of Veterans Affairs
Satisfaction of
EAMMF 107 Table 35: Environmental Protection Agency Satisfaction of
EAMMF 110 Table 36: General Services Administration Satisfaction of
EAMMF 113 Table 37: National Aeronautics and Space Administration
Satisfaction of EAMMF 116 Table 38: National Science Foundation
Satisfaction of EAMMF 119 Table 39: Nuclear Regulatory Commission
Satisfaction of
EAMMF 122 Table 40: Office of Personnel Management Satisfaction of
EAMMF 125 Table 41: Small Business Administration Satisfaction of EAMMF
127 Table 42: Social Security Administration Satisfaction of EAMMF 130
Table 43: U. S. Agency for International Development Satisfaction of
EAMMF 132
Figure 1: Summary of EAMMF Version 1.1: Maturity Stages, Critical
Figures
Success Attributes, and Core Elements 16 Figure 2: Overall Satisfaction of
Core Elements Associated with Architecture Governance 25
Page iii GAO-06-831 Enterprise Architecture
Figure 3: Overall Satisfaction of Core Elements Associated with
Architecture Content 27
Figure 4: Overall Satisfaction of Core Elements Associated with
Architecture Use 28
Figure 5: Overall Satisfaction of Core Elements Associated with
Architecture Measurement 29
Figure 6: Department/Agency Maturity Stage Based on Fully
Versus Partially Satisfied Criterion 31
Figure 7: Reported Development Costs to Date for Departments
and Agencies 46
Figure 8: Reported Estimated Completion Costs for Departments
and Agencies 47
Figure 9: Reported Estimated Annual Maintenance Costs for
Departments and Agencies 48
Figure 10: Breakdown of Enterprise Architecture Development
Costs for all Departments and Agencies 49
Figure 11: Reported Enterprise Architecture Contractor Costs by
Category 50
Figure 12: Enterprise Architecture Tools Used by Departments and
Agencies 52
Figure 13: Frameworks Used by Departments and Agencies 54
Abbreviations
BEA Business Enterprise Architecture
CIO Chief Information Officer
DHS Department of Homeland Security
DOD Department of Defense
DODAF Department of Defense Architecture Framework
DOJ Department of Justice
EA Enterprise Architecture
EAMMF Enterprise Architecture Management Maturity Framework
EAMS Enterprise Architecture Management System
EPA Environmental Protection Agency
FAA Federal Aviation Administration
FBI Federal Bureau of Investigation
FEAF Federal Enterprise Architecture Framework
FEAPMO Federal Enterprise Architecture Program Management Office
GIG Global Information Grid
GSA General Services Administration
HHS Department of Health and Human Services
HUD Department of Housing and Urban Development
IT Information Technology
IV&V Independent Verification and Validation
NASA National Aeronautics and Space Administration
NIST National Institute of Standards and Technology
NRC Nuclear Regulatory Commission
NSF National Science Foundation
OMB Office of Management and Budget
OPM Office of Personnel Management
SBA Small Business Administration
SSA Social Security Administration
TOGAF The Open Group Architecture Framework
USAID United States Agency for International Development
USDA United States Department of Agriculture
VA Department of Veterans Affairs
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
August 14, 2006
The Honorable Tom Davis Chairman Committee on Government Reform House of
Representatives
Dear Mr. Chairman:
A well-defined enterprise architecture1 is an essential tool for
leveraging information technology (IT) in the transformation of business
and mission operations. Our experience with federal departments and
agencies has shown that attempting to modernize and evolve IT environments
without an enterprise architecture to guide and constrain investments
often results in operations and systems that are duplicative, not well
integrated, unnecessarily costly to maintain and interface, and
ineffective in supporting mission goals. Moreover, the development,
implementation, and maintenance of architectures are widely recognized as
hallmarks of successful public and private organizations, and their use is
required by the Clinger-Cohen Act and the Office of Management and Budget
(OMB). In light of the importance of these architectures, you requested
that we determine the current status of major federal department and
agency enterprise architecture efforts.
To accomplish our objective, we surveyed 27 major federal departments and
agencies using a questionnaire that was based on our maturity framework
for assessing and improving enterprise architecture management,2 and we
collected and reviewed documentation to verify agency responses. We then
analyzed the results to determine the extent to which each of the 27
satisfied our maturity framework,3 and the challenges and benefits that
each department and agency sees. We also collected information about, for
example, department and agency architecture costs and architecture
framework and tool use and satisfaction, which is
1An enterprise architecture is a blueprint for organizational change
defined in models that describe (in both business and technology terms)
how the entity operates today and how it intends to operate in the future;
it also includes a plan for transitioning to this future state.
2GAO, Information Technology: A Framework for Assessing and Improving
Enterprise Architecture Management (Version 1.1), GAO-03-584G (Washington,
D.C.: April 2003).
3Our analysis reflects the state of department and agency architecture
efforts as of March 2006.
summarized in appendixes I and II. Because our framework defines what
needs to be done to effectively manage an enterprise architecture program,
and not the details surrounding how it needs to be done, the scope of our
review did not include assessing the quality of enterprise architecture
products and activities and associated management structures and processes
that make up our framework. As such, scoring high on our maturity scale
should be viewed as an indicator of, and not a guarantee that, a
department or agency necessarily has a well-defined architecture and that
it is being effectively implemented. We conducted our work in accordance
with generally accepted government auditing standards. Details of our
objective, scope, and methodology are in appendix III.
Results in Brief
The state of the enterprise architecture programs at the 27 major federal
departments and agencies is varied, with several having very immature
programs, several having more mature programs, and most being somewhere in
between. Collectively, this means that the bulk of the federal
government's enterprise architecture efforts can be viewed as a work in
process with much to be accomplished before their transformation value is
fully realized. To effectively establish and leverage enterprise
architectures as instruments of organizational transformation, research by
us and others show that architecture programs should be founded upon both
an institutional commitment to the architecture and a measured and
verified organizational capability to properly develop and use it to
affect operational and technological change. Our five stage architecture
framework for managing and evaluating the status of architecture efforts
consists of 31 core elements related to architecture governance, content,
use, and measurement that reflect these basic attributes.4 Of the 27
departments and agencies, 7 have advanced beyond the initial stage of our
framework, meaning that they have fully satisfied all the core elements
associated with the framework's second stage (establishing the management
foundation for developing, using, and maintaining the architecture). Of
these seven, four have also fully satisfied all the core elements
associated with the third stage (developing the architecture). None have
fully satisfied all of the core elements associated with the fourth
(completing the architecture) and fifth (leveraging the architecture for
organizational change) stages. Nevertheless, most of the departments and
agencies have fully satisfied a number of the core elements across stages
4 GAO-03-584G.
higher than that at which they have met all core elements. When this is
considered, the profile shows that about 77 percent of the programs
reviewed have fully satisfied the architecture governance core elements,
68 percent have fully satisfied the architecture content core elements, 52
percent have fully satisfied the architecture use core elements, and 47
have fully satisfied the architecture measurement core elements. Moreover,
most of the departments and agencies have also partially satisfied
additional core elements across all the stages. Seventeen of the
departments and agencies have at least partially satisfied the core
elements associated with achieving the framework's third stage, with four
having partially satisfied the elements associated with achieving higher
stages.
As we have previously reported, the key to these departments and agencies
building upon their current status, and ultimately realizing the many
benefits that they cited architectures providing, will be sustained
executive leadership, as virtually all the barriers that the agencies
reported can be addressed through such leadership. Examples of these
barriers or challenges are overcoming organizational parochialism and
cultural resistance, having adequate resources (human capital and
funding), and fostering top management understanding. Examples of the
benefits include better information sharing, consolidation, improved
productivity, and reduced costs. To assist the departments and agencies in
addressing their architectural barriers, managing their architecture
programs, and realizing their architecture benefits, we are making
recommendations to heads of major departments and agencies for developing
and implementing plans aimed at satisfying all of the conditions in our
architecture management maturity framework.
We received written or oral comments on a draft of this report from 25 of
the departments and agencies in our review.5 Of the 25, 24 fully agreed
with our recommendation and one department partially agreed. Nineteen
departments and agencies agreed with our findings and six partially
agreed. Of the six that disagreed with certain aspects of our findings,
the disagreements largely centered around (1) the adequacy of the
documentation that they provided to demonstrate satisfaction of a specific
core element and (2) recognition of steps that they reported taking after
we concluded our review. For the most part, these isolated areas of
5The Department of Defense submitted a single letter that included
comments from the Departments of the Air Force, Army, and Navy.
Representatives of the Departments of Health and Human Services and
Transportation stated that they did not have comments.
Page 3 GAO-06-831 Enterprise Architecture
disagreement did not result in any changes to our findings for two primary
reasons. First, our findings across the departments and agencies were
based on consistently applied evaluation criteria governing the adequacy
of documentation, and were not adjusted to accommodate any one particular
department or agency. Second, our findings represent the state of each
architecture program as of March 2006, and thus to be consistent do not
reflect activities that may have occurred after this time. Beyond these
comments, several departments and agencies offered suggestions for
improving our framework, which we will consider in issuing the next
version of the framework, and several provided technical comments, which
we have incorporated, as appropriate, in this report.
Background
An enterprise architecture is a blueprint that describes the current and
desired state of an organization or functional area in both logical and
technical terms, as well as a plan for transitioning between the two
states. Enterprise architectures are a recognized tenet of organizational
transformation and IT management in public and private organizations.
Without an enterprise architecture, it is unlikely that an organization
will be able to transform business processes and modernize supporting
systems to minimize overlap and maximize interoperability. The concept of
enterprise architectures originated in the mid-1980s; various frameworks
for defining the content of these architectures have been published by
government agencies and OMB. Moreover, legislation and federal guidance
requires agencies to develop and use architectures. For more than a
decade, we have conducted work to improve agency architecture efforts. To
this end, we developed an enterprise architecture management maturity
framework that provides federal agencies with a common benchmarking tool
for assessing the management of their enterprise architecture efforts and
developing improvement plans.
Enterprise Architecture Description and Importance
An enterprise can be viewed as either a single organization or a
functional area that transcends more than one organization (e.g.,
financial management, homeland security). An architecture can be viewed as
the structure (or structural description) of any activity. Thus,
enterprise architectures are basically systematically derived and captured
descriptions-in useful models, diagrams, and narrative.
More specifically, an architecture describes the enterprise in logical
terms (such as interrelated business processes and business rules,
information needs and flows, and work locations and users) as well as in
technical terms (such as hardware, software, data, communications, and
security attributes and performance standards). It provides these
perspectives both for the enterprise's current or "as-is" environment and
for its target or "tobe" environment, as well as a transition plan for
moving from the "as-is" to the "to-be" environment.
The importance of enterprise architectures is a basic tenet of both
organizational transformation and IT management, and their effective use
is a recognized hallmark of successful public and private organizations.
For over a decade, we have promoted the use of architectures, recognizing
them as a crucial means to a challenging end: optimized agency operations
and performance. The alternative, as our work has shown, is the
perpetuation of the kinds of operational environments that burden most
agencies today, where a lack of integration among business operations and
the IT resources supporting them leads to systems that are duplicative,
poorly integrated, and unnecessarily costly to maintain and interface.6
Employed in concert with other important IT management controls (such as
portfolio-based capital planning and investment control practices),
architectures can greatly increase the chances that the organizations'
operational and IT environments will be configured so as to optimize
mission performance.
Brief History of Architecture Frameworks and Management Guidance
During the mid-1980s, John Zachman, widely recognized as
a leader in the field of enterprise architecture, identified the need to
use a logical construction
blueprint (i.e., an architecture) for defining and controlling the
integration of systems and their components.7 Accordingly, Zachman
6See, for example, GAO, Homeland Security: Efforts Under Way to Develop
Enterprise Architecture, but Much Work Remains, GAO-04-777 (Washington,
D.C.: Aug. 6, 2004); DOD Business Systems Modernization: Limited Progress
in Development of Business Enterprise Architecture and Oversight of
Information Technology Investments, GAO-04- 731R (Washington, D.C.: May
17, 2004); Information Technology: Architecture Needed to Guide NASA's
Financial Management Modernization, GAO-04-43 (Washington, D.C.: Nov. 21,
2003); DOD Business Systems Modernization: Important Progress Made to
Develop Business Enterprise Architecture, but Much Work Remains,
GAO-03-1018 (Washington, D.C.: Sept. 19, 2003); and Information
Technology: DLA Should Strengthen Business Systems Modernization
Architecture and Investment Activities, GAO-01-631 (Washington, D.C.: June
29, 2001).
7J. A. Zachman, "A Framework for Information Systems Architecture," IBM
Systems Journal vol. 26, no. 3 (1987).
developed a structure or framework for defining and capturing an
architecture, which provides for six perspectives or "windows" from which
to view the enterprise.8 Zachman also proposed six abstractions or models
associated with each of these perspectives.9 Zachman's framework provides
a way to identify and describe an entity's existing and planned component
parts and the parts' relationships before the entity begins the costly and
time-consuming efforts associated with developing or transforming itself.
Since Zachman introduced his framework, a number of frameworks have
emerged within the federal government, beginning with the publication of
the National Institute of Standards and Technology (NIST) framework in
1989. Since that time, other federal entities have issued frameworks,
including the Department of Defense (DOD) and the Department of the
Treasury. In September 1999, the federal Chief Information Officers (CIO)
Council published the Federal Enterprise Architecture Framework (FEAF),
which was intended to provide federal agencies with a common construct for
their architectures, thereby facilitating the coordination of common
business processes, technology insertion, information flows, and system
investments among federal agencies. The FEAF described an approach,
including models and definitions, for developing and documenting
architecture descriptions for multi-organizational functional segments of
the federal government.10
More recently, OMB established the Federal Enterprise Architecture Program
Management Office (FEAPMO) to develop a federal enterprise architecture
according to a collection of five reference models (see table 1). These
models are intended to facilitate governmentwide improvement through
cross-agency analysis and the identification of duplicative investments,
gaps, and opportunities for collaboration, interoperability, and
integration within and across government agencies.
8The windows include (1) the strategic planner, (2) the system user, (3)
the system designer,
(4)
the system developer, (5) the subcontractor, and (6) the
system itself. 9The models cover (1) how the entity operates,
(2) what the entity uses to operate,
(3)
where the entity operates, (4) who operates the entity, (5) when
entity operations occur, and (6) why the entity operates.
10Similar to the Zachman framework, FEAF's proposed models describe an
entity's business, data necessary to conduct the business, applications to
manage the data, and technology to support the applications.
Table 1: Federal Enterprise Architecture Reference Models
Reference model Description
Performance Provides a common set of general performance outputs and
Reference Model measures for agencies to use to achieve business goals and
objectives.
Business Reference Describes the business operations of the federal
government Model independent of the agencies that perform them, including
defining the services provided to state and local governments.
Service Component Identifies and classifies IT service (i.e., application)
components Reference Model that support federal agencies and promotes the
reuse of components across agencies.
Data and Information Describes, at an aggregate level, the types of data
and information Reference Model that support program and business line
operations, and the relationships among these types.
Technical Reference Describes how technology is supporting the delivery of
service Model components, including relevant standards for implementing
the technology.
Source: GAO.
OMB has identified multiple purposes for the Federal Enterprise
Architecture, such as the following:
o informing agency enterprise architectures and facilitating their
development by providing a common classification structure and
vocabulary;
o providing a governmentwide framework that can increase agency
awareness of IT capabilities that other agencies have or plan to
acquire, so that they can explore opportunities for reuse;
o helping OMB decision makers identify opportunities for collaboration
among agencies through the implementation of common, reusable, and
interoperable solutions; and
o providing the Congress with information that it can use as it
considers the authorization and appropriation of funding for federal
programs.
Although these post-Zachman frameworks differ in their nomenclatures and
modeling approaches, each consistently provides for defining an
enterprise's operations in both logical and technical terms, provides for
defining these perspectives for the enterprise's current and target
environments, and calls for a transition plan between the two.
Several laws and regulations address enterprise architecture. For example,
the Clinger-Cohen Act of 1996 directs the CIOs of major departments and
agencies to develop, maintain, and facilitate the implementation of
information technology architectures as a means of integrating agency
goals and business processes with information technology.11 Also, OMB
Circular A-130, which implements the Clinger-Cohen Act, requires that
agencies document and submit their initial enterprise architectures to OMB
and that agencies submit updates when significant changes to their
enterprise architectures occur. The circular also directs OMB to use
various reviews to evaluate the adequacy and efficiency of each agency's
compliance with the circular.
A Decade of GAO Work Has Focused on Improving Agency Enterprise
Architecture Efforts
We began reviewing federal agencies' use of enterprise architectures in
1994, initially focusing on those agencies that were pursuing major
systems modernization programs that were high risk. These included the
National Weather Service systems modernization,12 the Federal Aviation
Administration (FAA) air traffic control modernization,13 and the Internal
Revenue Service tax systems modernization.14 Generally, we reported that
these agencies' enterprise architectures were incomplete, and we made
recommendations that they develop and implement complete enterprise
architectures to guide their modernization efforts.
Since then, we have reviewed enterprise architecture management at other
federal agencies, including the Department of Education (Education),15 the
1140 U.S.C. sections 11101-11703.
12GAO, Weather Forecasting: Systems Architecture Needed for National
Weather Service Modernization, GAO/AIMD-94-28 (Washington, D.C.: Mar. 11,
1994).
13GAO, Air Traffic Control: Complete and Enforced Architecture Needed for
FAA Systems Modernization, GAO/AIMD-97-30 (Washington, D.C.: Feb. 3,
1997).
14GAO, Tax Systems Modernization: Blueprint Is a Good Start but Not Yet
Sufficiently Complete to Build or Acquire Systems, GAO/AIMD/GGD-98-54
(Washington, D.C.: Feb. 24, 1998).
15GAO, Student Financial Aid Information: Systems Architecture Needed to
Improve Programs' Efficiency, GAO/AIMD-97-122 (Washington, D.C.: July 29,
1997).
Customs Service,16 the Immigration and Naturalization Service,17 the
Centers for Medicare and Medicaid Services,18 FAA,19 and the Federal
Bureau of Investigation (FBI).20 We have also reviewed the use of
enterprise architectures for critical agency functional areas, such as the
integration and sharing of terrorist watch lists across key federal
departments21 and DOD financial management,22 logistics management,23
combat identification,24 and business systems modernization.25 These
reviews continued to identify the absence of complete and enforced
enterprise architectures, which in turn has led to agency business
operations, systems, and data that are duplicative, incompatible, and not
16GAO, Customs Service Modernization: Architecture Must Be Complete and
Enforced to Effectively Build and Maintain Systems, GAO/AIMD-98-70
(Washington, D.C.: May 5, 1998).
17GAO, Information Technology: INS Needs to Better Manage the Development
of Its Enterprise Architecture, GAO/AIMD-00-212 (Washington, D.C.: Aug.
2000).
18GAO, Medicare: Information Systems Modernization Needs Stronger
Management and Support, GAO-01-824 (Washington, D.C.: Sept. 20, 2001).
19GAO, Federal Aviation Administration: Stronger Architecture Program
Needed to Guide Systems Modernization Efforts, GAO-05-266 (Washington,
D.C.: April 2005).
20GAO, Information Technology: FBI is Taking Steps to Develop an
Enterprise Architecture, but Much Remains to Be Accomplished, GAO-05-363
(Washington, D.C.: Sept. 9, 2005).
21GAO, Information Technology: Terrorist Watch Lists Should Be
Consolidated to Promote Better Integration and Sharing, GAO-03-322
(Washington, D.C.: April 15, 2003).
22GAO, Information Technology: Architecture Needed to Guide Modernization
of DOD's Financial Operations, GAO-01-525 (Washington, D.C.: May 17,
2001).
23 GAO-01-631.
24GAO, Combat Identification Systems: Strengthened Management Efforts
Needed to Ensure Required Capabilities, GAO-01-632 (Washington, D.C.: June
25, 2001).
25GAO, DOD Business Systems Modernization: Improvements to Enterprise
Architecture Development and Implementation Efforts Needed, GAO-03-458
(Washington, D.C.: Feb. 28, 2003); Information Technology: Observations on
Department of Defense's Draft Enterprise Architecture, GAO-03-571R
(Washington, D.C.: Mar. 28, 2003); DOD Business Systems Modernization:
Longstanding Management and Oversight Weaknesses Continue to Put
Investments at Risk, GAO-03-553T (Washington, D.C.: Mar. 31, 2003);
Business Systems Modernization: Summary of GAO's Assessment of the
Department of Defense's Initial Business Enterprise Architecture,
GAO-03-877R (Washington, D.C.: July 7, 2003); DOD Business Systems
Modernization: Long-Standing Weaknesses in Enterprise Architecture
Development Need to Be Addressed, GAO-05-702 (Washington, D.C.: July 22,
2005).
integrated; these conditions have either prevented agencies from sharing
data or forced them to depend on expensive, custom-developed system
interfaces to do so. Accordingly, we made recommendations to improve the
respective architecture efforts. In some cases progress has been made,
such as at DOD and FBI. As a practical matter, however, considerable time
is needed to completely address the kind of substantive issues that we
have raised and to make progress in establishing more mature architecture
programs.
In 2002 and 2003, we also published reports on the status of enterprise
architectures governmentwide. The first report (February 2002)26 showed
that about 52 percent of federal agencies self-reported having at least
the management foundation that is needed to successfully develop,
implement, and maintain an enterprise architecture, and that about 48
percent of agencies had not yet advanced to that basic stage of maturity.
We attributed this state of architecture management to four management
challenges: (1) overcoming limited executive understanding, (2) inadequate
funding, (3) insufficient number of skilled staff, and (4) organizational
parochialism. Additionally, we recognized OMB's efforts to promote and
oversee agencies' enterprise architecture efforts. Nevertheless, we
determined that OMB's leadership and oversight could be improved by, for
example, using a more structured means of measuring agencies' progress and
by addressing the above management challenges.
The second report (November 2003)27 showed the percentage of agencies that
had established at least a foundation for enterprise architecture
management was virtually unchanged. We attributed this to long-standing
enterprise architecture challenges that had yet to be addressed. In
particular, more agencies reported lack of agency executive understanding
of enterprise architecture and the scarcity of skilled architecture staff
as significant challenges. OMB generally agreed with our findings and the
need for additional agency assessments. Further, it stated that fully
implementing our recommendations would require sustained management
attention, and that it had begun by working with the CIO Council to
establish the Chief Architect Forum and to increase the information OMB
reports on enterprise architecture to Congress.
26GAO, Information Technology: Enterprise Architecture Use across the
Federal Government Can Be Improved, GAO-02-6 (Washington, D.C.: Feb. 19,
2002).
27GAO, Information Technology: Leadership Remains Key to Agencies Making
Progress on Enterprise Architecture Efforts, GAO-04-40 (Washington, D.C.:
Nov. 17, 2003).
Page 10 GAO-06-831 Enterprise Architecture
Since then, OMB has developed and implemented an enterprise architecture
assessment tool. According to OMB, the tool helps better understand the
current state of an agency's architecture and assists agencies in
integrating architectures into their decision-making processes. The latest
version of the assessment tool (2.0) was released in December 2005 and
includes three capability areas: (1) completion, (2) use, and (3) results.
Table 2 describes each of these areas.
Table 2: OMB Enterprise Architecture Assessment Framework Capability Areas
Capability area Description
Completion Addresses ensuring that architecture products describe the
agency in terms of processes, services, data, technology, and performance
and that the agency has developed a transition strategy.
Use Addresses the establishment of important management practices,
processes, and policies, such as configuration management, communications,
and integration of the architecture with capital planning processes.
Results Addresses the effectiveness and value of the architecture by
encouraging performance measurements and using it to ensure agency
policies align to OMB IT policy.
Source: OMB.
The tool also includes criteria for scoring an agency's architecture
program on a scale of 0 to 5.28 In early 2006, the major departments and
agencies were required by OMB to self assess their architecture programs
using the tool. OMB then used the self assessment to develop its own
assessment. These assessment results are to be used in determining the
agency's e-Government score within the President's Management Agenda.
GAO's Enterprise Architecture Management Maturity Framework (EAMMF)
In 2002, we developed version 1.0 of our Enterprise Architecture
Management Maturity Framework (EAMMF) to provide federal agencies with a
common benchmarking tool for planning and measuring their efforts to
improve enterprise architecture management, as well as to provide OMB with
a means for doing the same governmentwide. We issued an update of
28A score of 0 means undefined, 1 means initial, 2 means managed, 3 means
utilized, 4 means results-oriented, and 5 means optimized.
Page 11 GAO-06-831 Enterprise Architecture
the framework (version 1.1) in 2003.29 This framework is an extension of A
Practical Guide to Federal Enterprise Architecture, Version 1.0, published
by the CIO Council.30 Version 1.1 of the framework arranges 31 core
elements (practices or conditions that are needed for effective enterprise
architecture management) into a matrix of five hierarchical maturity
stages and four critical success attributes that apply to each stage.
Within a given stage, each critical success attribute includes between one
and four core elements. Based on the implicit dependencies among the core
elements, the EAMMF associates each element with one of five maturity
stages (see fig. 1). The core elements can be further categorized by four
groups: architecture governance, content, use, and measurement.
EAMMF Stages
Stage 1: Creating EA awareness. At stage 1, either an organization does
not have plans to develop and use an architecture, or it has plans that do
not demonstrate an awareness of the value of having and using an
architecture. While stage 1 agencies may have initiated some enterprise
architecture activity, these agencies' efforts are ad hoc and
unstructured, lack institutional leadership and direction, and do not
provide the management foundation necessary for successful enterprise
architecture development as defined in stage 2.
Stage 2: Building the EA management foundation. An organization at stage 2
recognizes that the enterprise architecture is a corporate asset by
vesting accountability for it in an executive body that represents the
entire enterprise. At this stage, an organization assigns enterprise
architecture management roles and responsibilities and establishes plans
for developing enterprise architecture products and for measuring program
progress and product quality; it also commits the resources necessary for
developing an architecture-people, processes, and tools. Specifically, a
stage 2 organization has designated a chief architect and established and
staffed a program office responsible for enterprise architecture
development and maintenance. Further, it has established a committee or
group that has responsibility for enterprise architecture governance
(i.e., directing, overseeing, and approving architecture development and
maintenance). This committee or group membership has enterprisewide
representation. At stage 2, the organization either has plans for
developing or has started
29 GAO-03-584G.
30CIO Council, A Practical Guide to Federal Enterprise Architecture,
Version 1.0 (February 2001).
Page 12 GAO-06-831 Enterprise Architecture
developing at least some enterprise architecture products, and it has
developed an enterprisewide awareness of the value of enterprise
architecture and its intended use in managing its IT investments. The
organization has also selected a framework and a methodology that will be
the basis for developing the enterprise architecture products and has
selected a tool for automating these activities.
Stage 3: Developing the EA. An organization at stage 3 focuses on
developing architecture products according to the selected framework,
methodology, tool, and established management plans. Roles and
responsibilities assigned in the previous stage are in place, and
resources are being applied to develop actual enterprise architecture
products. At this stage, the scope of the architecture has been defined to
encompass the entire enterprise, whether organization-based or
function-based. Although the products may not be complete, they are
intended to describe the organization in terms of business, performance,
information/data, service/application, and technology (including security
explicitly in each) as provided for in the framework, methodology, tool,
and management plans.31 Further, the products are to describe the current
(as-is) and future (to-be) states and the plan for transitioning from the
current to the future state (the sequencing plan). As the products are
developed and evolve, they are subject to configuration management.
Further, through the established enterprise architecture management
foundation, the organization is tracking and measuring its progress
against plans, identifying and addressing variances, as appropriate, and
then reporting on its progress.
Stage 4: Completing the EA. An organization at stage 4 has completed its
enterprise architecture products, meaning that the products have been
approved by the enterprise architecture steering committee (established in
stage 2) or an investment review board, and by the CIO. The completed
products collectively describe the enterprise in terms of business,
performance, information/data, service/application, and technology for
both its current and future operating states, and the products include a
plan for transitioning from the current to the future state. Further, an
independent agent has assessed the quality (i.e., completeness and
accuracy) of the enterprise architecture products. Additionally, evolution
of the approved products is governed by a written enterprise architecture
maintenance policy approved by the head of the organization.
31This set of products is consistent with OMB's federal enterprise
architecture reference models.
Page 13 GAO-06-831 Enterprise Architecture
Stage 5: Leveraging the EA to manage change. An organization at stage 5
has secured senior leadership approval of the enterprise architecture
products and a written institutional policy stating that IT investments
must comply with the architecture, unless granted an explicit compliance
waiver. Further, decision makers are using the architecture to identify
and address ongoing and proposed IT investments that are conflicting,
overlapping, not strategically linked, or redundant. As a result, stage 5
entities avoid unwarranted overlap across investments and ensure maximum
systems interoperability, which in turn ensures the selection and funding
of IT investments with manageable risks and returns. Also, at stage 5, the
organization tracks and measures enterprise architecture benefits or
return on investment, and adjustments are continuously made to both the
enterprise architecture management process and the enterprise architecture
products.
EAMMF Attributes
Attribute 1: Demonstrates commitment. Because the
enterprise architecture is a corporate asset for systematically managing
institutional
change, the support and sponsorship of the head of the enterprise are
essential to the success of the architecture effort. An approved
enterprise
policy statement provides such support and sponsorship, promoting
institutional buy-in and encouraging resource commitment from
participating components. Equally important in demonstrating
commitment is vesting ownership of the architecture with an executive
body that collectively owns the enterprise.
Attribute 2: Provides capability to meet commitment. The success of the
enterprise architecture effort depends largely on the organization's
capacity to develop, maintain, and implement the enterprise architecture.
Consistent with any large IT project, these capabilities include providing
adequate resources (i.e., people, processes, and technology), defining
clear roles and responsibilities, and defining and implementing
organizational structures and process management controls that promote
accountability and effective project execution.
Attribute 3: Demonstrates satisfaction of commitment. Satisfaction of the
organization's commitment to develop, maintain, and implement an
enterprise architecture is demonstrated by the production of artifacts
(e.g., the plans and products). Such artifacts demonstrate follow
through-that is, actual enterprise architecture production. Satisfaction
of commitment is further demonstrated by senior leadership approval of
enterprise architecture documents and artifacts; such approval
communicates institutional endorsement and ownership of the architecture
and the change that it is intended to drive.
Attribute 4: Verifies satisfaction of commitment. This attribute focuses
on measuring and disclosing the extent to which efforts to develop,
maintain, and implement the enterprise architecture have fulfilled stated
goals or commitments of the enterprise architecture. Measuring such
performance allows for tracking progress that has been made toward stated
goals, allows appropriate actions to be taken when performance deviates
significantly from goals, and creates incentives to influence both
institutional and individual behaviors.
Figure 1: Summary of EAMMF Version 1.1: Maturity Stages, Critical Success
Attributes, and Core Elements
Stage 4: Completing EA Stage 5:
Stage 2: Building Stage 3: Developing products Leveraging
the EA management EA products the EA to
Stage 1: CreatingEA foundation manage
awareness change
Attribute 1: Adequate resources Written and approved Written and approved Written and
Demonstratescommitment exist. Committee or organization policy organization policy exists approved
group representing existsfor EA for EA maintenance. organization
the enterprise is development. policy
responsible for exists for
directing, IT
overseeing, and investment
approving EA. compliance
with EA.
Attribute 2: Program office EA productsare under EA productsand management Process
Providescapability to responsible for EA configuration processesundergo independent exists to
meet commitment development and management. verification and validation. formally
maintenance exists. manage EA
Chief architect change. EA
exists. EA isbeing is integral
developed using component of
aframework, IT
methodology, and investment
automated tool. management
process.
Attribute 3: EA plans call for EA products describe EA products describe or will EA
Demonstratessatisfaction describing both the or will describe describe both the "as-is" productsare
of commitment "as-is" and the both the "as-is" and and the "to-be" environments periodically
"to-be" the "to-be" of the enterprise, as well updated. IT
environmentsof the environments of the asasequencing plan for investments
enterprise, as well enterprise, as well transitioning from the comply with
asasequencing plan asasequencing plan "as-is" to the "to-be." Both EA.
for transitioning for transitioning the "as-is" and the "to-be" Organization
from the "as-is" to from the "as-is" to environmentsare described or head
the "to-be." EA the "to-be." Both will be described in terms hasapproved
plans call for the "as-is" and the of business, performance, current
describing both the "to-be" information/data, version of
"as-is" and the environmentsare application/service, and EA.
"to-be" described or will be technology. Business,
environmentsin terms described in terms performance,
of business, of business, information/data,
performance, performance, application/service, and
information/data, information/data, technology
application/ application/service, descriptionsaddresssecurity.
service, and and technology. Organization CIO hasapproved
technology. EA plans Business, current version of EA.
call for business, performance, Committee or group
performance, information/data, representing the enterprise
information/data, application/service, or the investment review
application/service, and technology board hasapproved current
and technology descriptionsaddress version of EA.
descriptions to or will
addresssecurity. addresssecurity.
Attribute 4: EA plans call for Progressagainst EA Quality of EA products Return on EA
Verifiessatisfaction of developing metrics plansis measured and ismeasured and reported. investment
commitment for measuring EA reported. is measured
progress, quality, and
compliance, and reported.
return on Compliance
investment. with EA
ismeasured
and
reported.
Maturation
Source: GAO.
Note: Each stage includes all elements of previous stages.
EAMMF Groups
The framework's 31 core elements can also be placed in one of
four groups of architecture related activities, processes, products,
events, and structures. The groups are architecture governance, content,
use, and measurement. These groups are generally consistent with the
capability area descriptions in the previously discussed OMB enterprise
architecture assessment tool. For example, OMB's completion capability
area addresses ensuring that architecture products describe the agency in
terms of processes, services, data, technology, and performance and that
the agency has developed a transition strategy. Similarly, our content
group includes developing and completing these same enterprise
architecture products. In addition, OMB's results capability area
addresses performance measurement as does our measurement group, and OMB's
use capability area addresses many of the same elements in our governance
and use groups.
Table 3 lists the core elements according to EAMMF group.
Table 3: Summary of EAMMF Version 1.1: Core Elements Categorized by Group
Group Core element
Governance Adequate resources exist (stage 2).
Committee or group representing the enterprise is responsible
for directing, overseeing, and approving EA
(stage 2).
Program office responsible for EA development and maintenance
exists (stage 2).
Chief architect exists (stage 2).
EA being developed using a framework, methodology, and
automated tool (stage 2).
EA plans call for describing "as-is" environment, "to-be"
environment, and sequencing plan (stage 2).
EA plans call for describing enterprise in terms of business,
performance, information/data,
application/service, and technology (stage 2).
EA plans call for business, performance, information/data,
application/service, and technology to address
security (stage 2).
Written and approved policy exists for EA development (stage
3).
Written and approved policy exists for EA maintenance (stage
4).
Organization CIO has approved EA (stage 4).
Committee or group representing the enterprise or the
investment review board has approved current version
of EA (stage 4).
Written and approved organization policy exists for IT
investment compliance with EA (stage 5).
Organization head has approved current version of EA (stage 5).
Content EA products are under configuration management (stage 3).
(Continued From Previous Page)
Group Core element
EA products describe or will describe "as-is" environment, "to-be"
environment and sequencing plan (stage 3).
Both "as-is" and "to-be" environments are described or will be described in
terms given in stage 2 (stage 3).
These descriptions address or will address security (stage 3).
EA products and management processes undergo independent verification and
validation (stage 4).
EA products describe "as-is" environment, "to-be" environment, and sequencing
plan (stage 4).
Both "as-is" and "to-be"environments are described in terms given in stage 2
(stage 4).
These descriptions address security (stage 4).
Process exists to formally manage EA change (stage 5).
EA products are periodically updated (stage 5).
Use EA is integral component of IT investment management process (stage 5).
IT investments comply with EA (stage 5).
Measurement EA plans call for developing metrics to measure EA progress,
quality, compliance, and return on investment (stage 2).
Progress against EA plans is measured and reported (stage 3).
Quality of EA products is measured and reported (stage 4).
Return on EA investment is measured and reported (stage 5).
Compliance with EA is measured and reported (stage 5).
Source: GAO.
Overall State of Enterprise Architecture Management Is a Work-in-Progress,
Although a Few Agencies Have Largely Satisfied Our Framework
Most of the 27 major departments and agencies have not fully satisfied all
the core elements associated with stage 2 of our maturity framework. At
the same time, however, most have satisfied a number of core elements at
stages 3, 4, and 5. Specifically, although only seven have fully satisfied
all the stage 2 elements, the 27 have on average fully satisfied 80, 78,
61, and 52 percent of the stage 2, 3, 4, and 5 elements, respectively. Of
the core elements that have been fully satisfied, 77 percent of those
related to architecture governance have been fully satisfied, while 68,
52, and 47 percent of those related to architecture content, use, and
measurement, respectively, have been fully satisfied. Most of the 27 have
also at least partially satisfied a number of additional core elements
across all the stages. For example, all but 7 have at least partially
satisfied all the elements required to achieve stage 3 or higher.
Collectively, this means efforts are underway to mature the management of
most agency enterprise architecture programs, but overall these efforts
are uneven and still a work-in-progress and they face numerous challenges
that departments and agencies identified. It also means that some
architecture programs provide examples from which less mature programs
could learn and improve.
Without mature enterprise architecture programs, some departments and
agencies will not realize the many benefits that they attributed to
architectures, and they are at risk of investing in IT assets that are
duplicative, not well-integrated, and do not optimally support mission
operations.
The Degree to which Major Departments and Agencies Have Fully Satisfied
Our Framework's Core Elements Is Uneven and Their Collective Efforts Can
Be Viewed as a Work-in-Progress
To qualify for a given stage of maturity under our architecture management
framework, a department or agency had to fully satisfy all of the core
elements at that stage. Using this criterion, three departments and
agencies are at stage 2, meaning that they demonstrated to us through
verifiable documentation that they have established the foundational
commitments and capabilities needed to manage the development of an
architecture. In addition, four are at stage 3, meaning that they
similarly demonstrated that their architecture development efforts reflect
employment of the basic control measures in our framework. Table 4
summarizes the maturity stage of each architecture program that we
assessed. Appendix IV provides the detailed results of our assessment of
each department and agency architecture program against our maturity
framework.
Table 4: Maturity Stage of Major Department and Agency Enterprise
Architecture Programs
Stage when program
required to fully satisfy all
elements in one stage to
Department/Agency advance to the next
Department of Housing and Urban Development (HUD) 3
Department of the Interior (Interior) 3
Department of Justice (DOJ) 3
Department of Labor (Labor) 3
Department of Agriculture (USDA) 2
Department of Homeland Security (DHS) 2
Office of Personnel Management (OPM) 2
Department of the Air Force (Air Force) 1
Department of the Army (Army) 1
Department of Commerce (Commerce) 1
Department of Defense - Business Enterprise 1
Architecture (BEA)
Department of Defense - Global Information Grid (GIG) 1
(Continued From Previous Page)
Stage when program required to fully satisfy all elements in one stage to
Department/Agency advance to the next
Department of Education (Education)
Department of Energy (Energy)
Department of Health and Human Services (HHS)
Department of the Navy (Navy)
Department of State (State)
Department of the Treasury (Treasury)
Department of Transportation (Transportation)
Department of Veterans Affairs (VA)
Environmental Protection Agency (EPA)
General Services Administration (GSA)
National Aeronautics and Space Administration (NASA)
National Science Foundation (NSF)
Nuclear Regulatory Commission (NRC)
Small Business Administration (SBA)
Social Security Administration (SSA)
U.S. Agency for International Development (USAID)
Source: GAO analysis of department and agency data.
While using this criterion provides an important perspective on the state
of department and agency architecture programs, it can mask the fact that
the programs have met a number of core elements across higher stages of
maturity. When the percentage of core elements that have been fully
satisfied at each stage is considered, the state of the architecture
efforts generally shows both a larger number of more robust architecture
programs as well as more variability across the departments and agencies.
Specifically, 16 departments and agencies have fully satisfied more than
70 percent of the core elements. Examples include Commerce, which has
satisfied 87 percent of the core elements, including 75 percent of the
stage 5 elements, even though it is at stage 1 because its enterprise
architecture approval board does not have enterprisewide representation (a
stage 2 core element). Similarly, SSA, which is also a stage 1 because the
agency's enterprise architecture methodology does not describe the steps
for developing, maintaining, and validating the agency's enterprise
architecture (a stage 2 core element), has at the same time satisfied 87
percent of all the elements, including 63 percent of the stage 5 elements.
In contrast, the Army, which is also in stage 1, has satisfied but 3
percent of all framework elements. Overall, 10 agency architecture
programs fully satisfied more than 75 percent of the core elements, 14
between 50 and 75 percent, and 4 fewer than 50 percent. These four
included the three military departments. Table 5 summarizes for each
department and agency the percentage of core elements fully satisfied in
total and by maturity stage.
Table 5: Percent of Framework Elements Satisfied by Department and Agency
Architecture Programs within Each Maturity Stage
Percent of Percent of Percent of Percent Percent
of of
framework stage 2 stage 3 stage 4 stage 5
elements elements elements elements elements
Departments/Agencies satisfied satisfied satisfied satisfied satisfied
and Maturity Stages
Stage 3
Department of the 97 100 100 88 100
Interior
Department of Housing 94 100 100 75 100
and Urban Development
Department of Labor 87 100 100 88 63
Department of Justice 77 100 100 63 50
Stage 2
Office of Personnel 94 100 83 88 100
Management
Department of 77 100 83 75 50
Homeland Security
Department of 61 100 67 50 25
Agriculture
Stage 1
Department of 87 89 100 88 75
Commerce
Social Security 87 89 100 100 63
Administration
Department of 84 89 100 75 75
Education
Department of Energy 77 89 83 88 50
National Aeronautics
and Space 71 67 100 63 63
Administration
Small Business 71 78 67 75 63
Administration
Department of the 71 78 83 63 63
Treasury
Department of Health 71 89 100 38 63
and Human Services
Environmental 74 89 83 88 38
Protection Agency
Department of Defense
- Global Information 71 89 67 75 50
Grid
Department of Defense
- Business Enterprise 68 78 67 63 63
Architecture
Department of 65 78 83 50 50
Veterans Affairs
Department of 65 78 83 50 50
Transportation
Department of State 58 67 67 63 38
General Services 55 67 50 50 50
Administration
Nuclear Regulatory 55 67 83 50 25
Commission
National Science 52 78 67 25 38
Foundation
Department of the Air 45 56 67 38 25
Force
(Continued From Previous Page)
Percent of Percent of Percent of Percent of Percent
of
framework stage 2 stage 3 stage 4 stage 5
elements elements elements elements elements
Departments/Agencies satisfied satisfied satisfied satisfied satisfied
and Maturity Stages
Agency for
International 39 67 50 13 25
Development
Department of the 32 44 50 25 13
Navy
Department of the 3 11 0 0 0
Army
Source: GAO analysis of department and agency data.
Notwithstanding the additional perspective that the percentage of core
elements fully satisfied across all stages provides, it is important to
note that the staged core elements in our framework represent a
hierarchical or systematic progression to establishing a well-managed
architecture program, meaning that core elements associated with lower
framework stages generally support the effective execution of higher
maturity stage core elements. For instance, if a program has developed its
full suite of "asis" and "to-be" architecture products, including a
sequencing plan (stage 4 core elements), but the products are not under
configuration management (stage 3 core element), then the integrity and
consistency of the products will be not be assured. Our analysis showed
that this was the case for a number of architecture programs. For example,
State has developed certain "as-is" and "to-be" products for the Joint
Enterprise Architecture, which is being developed in collaboration with
USAID, but an enterprise architecture configuration management plan has
not yet been finalized.
Further, not satisfying even a single core element can have a significant
impact on the effectiveness of an architecture program. For example, not
having adequate human capital with the requisite knowledge and skills
(stage 2 core element), not using a defined framework or methodology
(stage 2 core element), or not using an independent verification and
validation agent (stage 4 core element), could significantly limit the
quality and utility of an architecture. The DOD's experience between 2001
and 2005 in developing its BEA is a case in point. During this time, we
identified the need for the department to have an enterprise architecture
for its business operations, and we made a series of recommendations
grounded in, among other things, our architecture management framework to
ensure that it was successful in doing so.32 In 2005,33 we reported that
the department had not implemented most of our recommendations. We further
reported that despite developing multiple versions of a wide range of
architecture products, and having invested hundreds of millions of dollars
and 4 years in doing so, the department did not have a well-defined
architecture and that what it had developed had limited utility. Among
other things, we attributed the poor state of its architecture products to
ineffective program governance, communications, program planning, human
capital, and configuration management, most of which are stage 2 and 3
foundational core elements. To the department's credit, we recently
reported that it has since taken a number of actions to address these
fundamental weaknesses and our related recommendations and that it is now
producing architecture products that provide a basis upon which to build.
The significance of not satisfying a single core element is also readily
apparent for elements associated with the framework's content group. In
particular, the framework emphasizes the importance of planning for,
developing, and completing an architecture that includes the "as-is" and
the "to-be" environments as well as a plan for transitioning between the
two. It also recognizes that the "as-is" and "to-be" should address the
business, performance, information/data, application/service, technology,
and security aspects of the enterprise. To the extent these aspects are
not addressed in this way, the quality of the architecture and thus its
utility will suffer. In this regard, we found examples of departments and
agencies that were addressing some but not all of these aspects. For
example, HUD has yet to adequately incorporate security into its
architecture. This is significant because security is relevant to all the
other aspects of its architecture, such as information/data and
applications/services. As another example, NASA's architecture does not
include a plan for transitioning from the "as-is" to the "to-be"
environments. According to the administration's Chief Enterprise
Architect, a transition plan has not yet been developed because of
insufficient time and staff.
32See, for example, GAO-01-525 , GAO-03-458 , GAO-04-731R , GAO-05-702 ,
and GAO, DOD Business Systems Modernization: Important Progress Made in
Establishing Foundational Architecture Products and Investment Management
Practices, but Much Work Remains, GAO-06-219 (Washington, D.C.: Nov. 23,
2005).
33 GAO-05-702.
Page 23 GAO-06-831 Enterprise Architecture
Looking across all the departments and agencies at core elements that are
fully satisfied, not by stage of maturity, but by related groupings of
core elements, provides an additional perspective on the state of the
federal government's architecture efforts. As noted earlier, these
groupings of core elements are architecture governance, content, use, and
measurement. Overall, departments and agencies on average have fully
satisfied 77 percent of the governance-related elements. In particular, 93
and 96 percent of the agencies have established an architecture program
office and appointed a chief architect, respectively. In addition, 93
percent have plans that call for their respective architectures to
describe the "as-is" and the "to-be" environments, and for having a plan
for transitioning between the two (see fig. 2). In contrast, however, the
core element associated with having a committee or group with
representation from across the enterprise directing, overseeing, and
approving the architecture was fully satisfied by only 57 percent of the
agencies. This core element is important because the architecture is a
corporate asset that needs to be enterprisewide in scope and accepted by
senior leadership if it is to be leveraged for organizational change.
Figure 2: Overall Satisfaction of Core Elements Associated with
Architecture Governance
Framework elements
Adequate resources exist (stage 2)
Committee or group representing the enterprise is responsible for
directing, overseeing, and approving EA (stage 2)
Program office responsible for EA development and maintenance exists(stage
2)
Chief architect exists (stage 2)
EA is being developed using a framework, methodology, and automated tool
(stage 2)
EA plans call for describing both the "as-is" and the "to-be" environments
of the enterprise, as well as a sequencing plan for transitioning from the
"as-is" to the "to-be" (stage 2)
EA plans call for describing both the "as-is" and the "to-be" environments
in terms of business, performance, information/data, application/service,
and technology (stage 2)
EA plans call for business, performance, information/data,
application/service, and technology descriptions to addresssecurity (stage
2)
Written and approved organization policy exists for EA development (stage 3)
Written and approved organization policy exists for EA maintenance (stage 4)
Written and approved organization policy exists for IT investment
compliance with EA (stage 5)
Organization CIO has approved current version of EA (stage 4)
Committee or group representing the enterprise or the investment review
board has approved current version of EA (stage 4)
Organization head has approved current version of EA (stage 5)
0 20 40 60 80 100 Percentage
Not satisfied Partially satisfied Satisfied Source: GAO analysis of
department/agency data.
Note: Numbers might not add to 100 percent due to rounding.
In contrast to governance, the extent of full satisfaction of those core
elements that are associated with what an architecture should contain
varies widely (see fig. 3). For example, the three content elements that
address prospectively what the architecture will contain, either in
relation to plans or some provision for including needed content, were
fully satisfied about 90 percent of the time. However, the core elements
addressing whether the products now contain such content were fully
satisfied much less frequently (between 54 and 68 percent of the time,
depending on the core element), and the core elements associated with
ensuring the quality of included content, such as employing configuration
management and undergoing independent verification and validation, were
also fully satisfied much less frequently (54 and 21 percent of the time,
respectively). The state of these core elements raises important questions
about the quality and utility of the department and agency architectures.
Figure3: Overall Satisfaction of Core Elements Associated with
Architecture Content Framework elementsEA products are under configuration
management (stage 3)
EA products describe or will describe both the "as-is" and the "to-be"
environments of the enterprise, as well as a sequencing plan for
transitioning from the "as-is" to the "to-be" (stage 3)
Both the "as-is" and the "to-be" environments are described or will be
described in terms of business, performance, information/data,
application/service, and technology (stage 3)
Business, performance, information/data, application/service, and
technology descriptions address or will addresssecurity (stage 3)
EA products and management processes undergo independent verification and
validation (stage 4)
EA products describe both the "as-is" and the "to-be" environments of the
enterprise, as well as a sequencing plan for transitioning from the
"as-is" to the "to-be" (stage 4)
Both the "as-is" and the "to-be" environments are described in terms of
business, performance, information/data, application/service, and
technology (stage 4)
Business, performance, information/data, application/service, and
technology descriptions addresssecurity (stage 4)
Process exists to formally manage EA change (stage 5)
EA products are periodically updated (stage 5)
0 20 40 60 80 100 Percentage
Not satisfied
Partially satisfied
Satisfied
Source: GAO analysis of department/agency data.
Note: Numbers might not add to 100 percent due to rounding.
The degree of full satisfaction of those core elements associated with the
remaining two groups-use and measurement-is even lower (see figs. 4 and 5,
respectively). For example, the architecture use-related core elements
were fully satisfied between 39 and 64 percent of the time, while the
measurement-related elements were satisfied between 14 and 71 percent. Of
particular note is that only 39 percent of the departments and agencies
could demonstrate that IT investments comply with their enterprise
architectures, only 43 percent of the departments and agencies could
demonstrate that compliance with the enterprise architecture is measured
and reported, and only 14 percent were measuring and reporting on their
respective architecture program's return on investment. As our work and
related best practices show, the value in having an architecture is using
it to affect change and produce results. Such results, as reported by the
departments and agencies include improved information sharing, increased
consolidation, enhanced productivity, and lower costs, all of which
contribute to improved agency performance. To realize these benefits,
however, IT investments need to comply with the architecture and
measurement of architecture activities, including accrual of expected
benefits, needs to occur.
Figure 4: Overall Satisfaction of Core Elements Associated with
Architecture Use Framework elementsEA is integral component of IT
investment management process (stage 5)
IT investments comply with EA (stage 5)
0 20 40 60 80 100 Percentage
Satisfied
Source: GAO analysis of department/agency data.
Note: Numbers might not add to 100 percent due to rounding.
Not satisfied Partially satisfied
Figure 5: Overall Satisfaction of Core Elements Associated with
Architecture Measurement Framework elementsEA plans call for developing
metrics for measuring EA progress, quality, compliance, and return on
investment (stage 2)
Progress against EA plans is measured and reported (stage 3)
Quality of EA products is measured and reported (stage 4)
Return on EA investment is measured and reported (stage 5)
Compliance with EA is measured and reported (stage 5) 0 20 40 60 80 100
Percentage
Not satisfied Partially satisfied Satisfied Source: GAO analysis of
department/agency data.
Note: Numbers might not add to 100 percent due to rounding.
Most Agencies Have at Least Partially Satisfied Most Framework Elements
In those instances where departments and agencies have not fully satisfied
certain core elements in our framework, most have at least partially
satisfied34 these elements. To illustrate, 4 agencies would improve to at
least stage 4 if the criterion for being a given stage was relaxed to only
partially satisfying a core element. Moreover, 11 of the remaining
agencies would advance by two stages under such a less demanding
criterion, and only 6 would not improve their stage of maturity under
these circumstances. A case in point is Commerce, which could move from
stage 1 to stage 5 under these circumstances because it has fully
satisfied all but four core elements and these remaining four (one each at
stages 2 and 4 and two at stage 5) are partially satisfied. Another case
in point is the SSA, which has fully satisfied all but four core elements
(one at stage 2 and three at stage 5) and has partially satisfied three of
these remaining four. If the criterion used allowed advancement to the
next stage by only partially satisfying core elements, the administration
would be stage 4. (See fig. 6 for
34Partially satisfied means that a department or agency has addressed
some, but not all, aspects of the core element.
Page 29 GAO-06-831 Enterprise Architecture
a comparison of department and agency program maturity stages under the
two criteria.)
Figure 6: Department/Agency Maturity Stage Based on Fully Versus Partially
Satisfied Criterion
Maturity stage based on Maturity stage based on highest highest stage in
which all stage in which all elementselements are fully satisfied are
fully or partially satisfied
Program 1 2 3 4 5 1 2 3 4 5 Department of Agriculture
Department of the Air Force
Department of the Army
Department of Commerce
Department of Defense - Business Enterprise Architecture
Department of Defense - Global Information Grid
Department of Education
Department of Energy
Department of Health and Human Services
Department of Homeland Security
Department of Housing and Urban Development
Department of the Interior
Department of Justice
Department of Labor
Department of the Navy
Department of State
Department of Transportation
Department of the Treasury
Department of Veterans Affairs
Environmental Protection Agency
General Services Administration
National Aeronautics and Space Administration
National Science Foundation
Nuclear Regulatory Commission
Office of Personnel Management
Small Business Administration
Social Security Administration
U.S. Agency for International Development
Source: GAO analysis of department/agency data.
As mentioned earlier, departments and agencies can require considerable
time to completely address issues related to their respective enterprise
architecture programs. It is thus important to note that even though
certain core elements are partially satisfied, fully satisfying some of
them may not be accomplished quickly and easily. It is also important to
note the importance of fully, rather than partially, satisfying certain
elements, such as those that fall within the architecture content group.
In this regard, 18, 18, and 21 percent of the departments and agencies
partially satisfied the following stage 4 content-related core elements,
respectively: "EA products describe `as-is' environment, `to-be'
environment and sequencing plan"; "Both `as-is' and `to-be' environments
are described in terms of business, performance, information/data,
application/service, and technology"; and "These descriptions fully
address security." Not fully satisfying these elements can have important
implications for the quality of an architecture, and thus its usability
and results.
Seven Departments or Agencies Need to Satisfy Five or Fewer Core Elements
to Be at Stage 5
Seven departments or agencies would meet our criterion for stage 5 if each
was to fully satisfy one to five additional core elements (see table 6).
For example, Interior could achieve stage 5 by satisfying one additional
element: "EA products and management processes undergo independent
verification and validation." In this regard, Interior officials have
drafted a statement of work intended to ensure that independent
verification and validation of enterprise architecture products and
management processes is performed. The other six departments and agencies
are HUD and OPM, which could achieve stage 5 by satisfying two additional
elements; Commerce, Labor, and SSA, which could achieve the same by
satisfying four additional elements; and Education which could be at stage
5 by satisfying five additional elements. Of these seven, five have not
fully satisfied the independent verification and validation core element.
Notwithstanding the fact that five or fewer core elements need to be
satisfied by these agencies to be at stage 5, it is important to note that
in some cases the core elements not being satisfied are not only very
important, but also neither quickly nor easily satisfied. For example, one
of the two elements that HUD needs to satisfy is having its architecture
products address security. This is extremely important as security is an
integral aspect of the architecture's performance, business,
information/data, application/service, and technical models, and needs to
be reflected thoroughly and consistently across each of them.
Table 6: Departments and Agencies That Need to Satisfy 5 or Fewer Core Elements
to Achieve Stage 5
Number of
unsatisfied
Department/agency elements Unsatisfied element(s)
Department of the 1 o EA products and management processes undergo
independent verification and validation.
Interior
Department of 2 o EA products and management processes undergo
independent verification and validation.
Housing and Urban o Business, performance, information/data,
application/service, and technology
descriptions
Development address security.
Office of 2 o Progress against EA plans is measured and
Personnel reported.
Management o EA products and management processes undergo
independent verification and validation.
o Committee or group representing the
Department of 4 enterprise is responsible for directing,
overseeing, and
Commerce approving EA.
o Committee or group representing the
enterprise or the investment review board has
approved
current version of EA.
o Written and approved organization policy
exists for IT investment compliance with EA.
o IT investments comply with EA.
Department of 4 o EA products and management processes undergo
Labor independent verification and validation.
o IT investments comply with EA.
o Return on EA investment is measured and
reported.
o Compliance with EA is measured and reported.
Social Security 4 o EA is being developed using a framework,
methodology, and automated tool.
Administration o Organization head has approved current
version of EA.
o Return on EA investment is measured and
reported.
o Compliance with EA is measured and reported.
o Committee or group representing the
Department of 5 enterprise is responsible for directing,
overseeing, and
Education approving EA.
o EA products and management processes undergo
independent verification and validation.
o Committee or group representing the
enterprise or the investment review board has
approved
current version of EA.
o Organization head has approved current
version of EA.
o Return on EA investment is measured and
reported.
Source: GAO.
Departments and Agencies Report Numerous Challenges Facing Them in
Developing and Using Enterprise Architectures
The challenges facing departments and agencies in developing and using
enterprise architectures are formidable. The challenge that most
departments and agencies cited as being experienced to the greatest extent
is the one that having and using an architecture is intended to overcome-
organizational parochialism and cultural resistance to adopting an
enterprisewide mode of operation in which organizational parts are
suboptimized in order to optimize the performance and results of the
enterprise as a whole. Specifically, 93 percent of the departments and
agencies reported that they encountered this challenge to a significant
(very great or great) or moderate extent. Other challenges reported to
this
same extent were ensuring that the architecture program had adequate
funding (89 percent), obtaining staff skilled in the architecture
discipline (86 percent), and having the department or agency senior
leaders understand the importance and role of the enterprise architecture
(82 percent).
As we have previously reported, sustained top management leadership is the
key to overcoming each of these challenges. In this regard, our enterprise
architecture management maturity framework provides for such leadership
and addressing these and other challenges through a number of core
elements. These elements contain mechanisms aimed at, for example,
establishing responsibility and accountability for the architecture with
senior leaders and ensuring that the necessary institutional commitments
are made to the architecture program, such as through issuance of
architecture policy and provision of adequate resources (both funding and
people). See table 7 for a listing of the reported challenges and the
extent to which they are being experienced.
Table 7: Degree to Which Departments and Agencies Are Experiencing
Enterprise Architecture Challenges
Percentage of departments and agencies experiencing the Challenge
challenge to a great or very great extent
Overcoming parochialism/cultural resistance
Ensuring adequate funding
Fostering top management understanding
Obtaining skilled staff
Source: GAO analysis based on department/agency data.
Many Departments and Agencies Reported That They Have Already Realized
Significant Architecture Benefits, While Most Expect to Do So in the
Future
A large percentage of the departments and agencies reported that they have
already accrued numerous benefits from their respective architecture
programs (see table 8). For example, 70 percent said that have already
improved the alignment between their business operations and the IT that
supports these operations to a significant extent. Such alignment is
extremely important. According to our IT investment management maturity
framework,35 alignment between business needs and IT investments is a
critical process in building the foundation for an effective approach to
IT investment management. In addition, 64 percent responded that they have
also improved information/knowledge sharing to a significant or moderate
extent. Such sharing is also very important. In 2005, for example, we
added homeland security information sharing to our list of high-risk areas
because despite the importance of information to fighting terrorism and
maintaining the security of our nation, many aspects of homeland security
information sharing remain ineffective and fragmented.36 Other examples of
mission-effectiveness related benefits reported as already being achieved
to a significant or moderate extent by roughly one-half of the departments
and agencies included improved agency management and change management and
improved system and application interoperability.
Beyond these benefits, departments and agencies also reported already
accruing, to a significant or moderate extent, a number of efficiency and
productivity benefits. For example, 56 percent reported that they have
increased the use of enterprise software licenses, which can permit cost
savings through economies of scale purchases; 56 percent report that they
have been able to consolidate their IT infrastructure environments, which
can reduce the costs of operating and maintaining duplicative
capabilities; 41 percent reported that they have been able to reduce the
number of applications, which is a key to reducing expensive maintenance
costs; and 37 percent report productivity improvements, which can free
resources to focus on other high priority matters.
Notwithstanding the number and extent of benefits that department and
agency responses show have already been realized, these same responses
also show even more benefits that they have yet to realize (see table 8).
For example, 30 percent reported that they have thus far achieved, to
little or no extent, better business and IT alignment. They similarly
reported that they have largely untapped many other effectiveness and
efficiency benefits, with between 36 and 70 percent saying these benefits
have been achieved to little or no extent, depending on benefit. Moreover,
for all the cited benefits, a far greater percentage of the departments
and agencies (74
35GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity. GAO-04-394G . (Washington, D.C.:
Mar. 2004).
36GAO, High Risk Series: An Update. GAO-05-207 . (Washington, D.C.: Jan.
2005).
Page 35 GAO-06-831 Enterprise Architecture
to 93 percent) reported that they expect to realize each of the benefits
to a significant or moderate extent sometime in the future. What this
suggests is that the real value in the federal government from developing
and using enterprise architecture remains largely unrealized potential.
Our architecture maturity framework recognizes that a key to realizing
this potential is effectively managing department and agency enterprise
architecture programs. However, knowing whether benefits and results are
in fact being achieved requires having associated measures and metrics. In
this regard, very few (21 percent) of the departments and agencies fully
satisfied our stage 5 core element, "Return on EA investment is measured
and reported." Without satisfying this element, it is unlikely that the
degree to which expected benefits are accrued will be known.
Table8: Enterprise Architecture Benefits Reported As Being or To Be
Achieved to a Significant Extent
Percent
Percent reporting reporting that
that the benefit is the benefit
will
Benefit being achieved be achieved
Improved business and information
technology
alignment 70 93
Improved information/knowledge sharing 64 93
Improved agency and change management 54 89
Increased infrastructure consolidation 56 81
Increased use of enterprise licenses 56 89
Improved systems interoperability 48 93
Improved application interoperability 46 81
Fewer applications 41 89
Optimized business processes 41 89
Improved data integration 39 89
Enhanced productivity 37 81
Improved data reuse 33 93
Lower system-related costs 30 85
Reduced system complexity 30 74
Source: GAO based on department and agency data.
Note: Significant extent means a very great, great, or moderate extent.
Conclusions
If managed effectively, enterprise architectures can be a useful change
management and organizational transformation tool. The conditions for
effectively managing enterprise architecture programs are contained in our
architecture management maturity framework. While a few of the federal
government's 27 major departments and agencies have fully satisfied all
the conditions needed to be at stage 2 or above in our framework, many
have fully satisfied a large percentage of the core elements across most
of the stages, particularly those elements related to architecture
governance. Nevertheless, most departments and agencies are not yet where
they need to be relative to architecture content, use, and measurement and
thus the federal government is not as well positioned as it should be to
realize the significant benefits that a well-managed architecture program
can provide. Moving beyond this status will require most departments and
agencies to overcome some significant obstacles and challenges. The key to
doing so continues to be sustained organizational leadership. Without such
organizational leadership, the benefits of enterprise architecture will
not be fully realized.
Recommendations for Executive Action
To assist the 27 major departments and agencies in addressing enterprise
architecture challenges, managing their architecture programs, and
realizing architecture benefits, we recommend that the Administrators of
the Environmental Protection Agency, General Services Administration,
National Aeronautics and Space Administration, Small Business
Administration, and U.S. Agency for International Development; the
Attorney General; the Commissioners of the Nuclear Regulatory Commission
and Social Security Administration; the Directors of the National Science
Foundation and the Office of Personnel Management; and the Secretaries of
the Departments of Agriculture, Commerce, Defense, Education, Energy,
Health and Human Services, Homeland Security, Housing and Urban
Development, Interior, Labor, State, Transportation, Treasury, and
Veterans Affairs ensure that their respective enterprise architecture
programs develop and implement plans for fully satisfying each of the
conditions in our enterprise architecture management maturity framework.
Agency Comments and Our Evaluation
We received written or oral comments on a draft of this report from 25 of
the departments and agencies in our review. 37 Of the 25 departments and
agencies, all but one department fully agreed with our recommendation.
Nineteen departments and agencies agreed and six partially agreed with our
findings. Areas of disagreement for these six centered on (1) the adequacy
of the documentation that they provided to demonstrate satisfaction of
certain core elements and (2) recognition of steps that they reported
taking to satisfy certain core elements after we concluded our review. For
the most part, these isolated areas of disagreement did not result in any
changes to our findings for two primary reasons. First, our findings
across the departments and agencies were based on consistently applied
evaluation criteria governing the adequacy of documentation, and were not
adjusted to accommodate any one particular department or agency. Second,
our findings represent the state of each architecture program as of March
2006, and thus to be consistent do not reflect activities that may have
occurred after this time. Beyond these comments, several agencies offered
suggestions for improving our framework, which we will consider prior to
issuing the next version of the framework. The departments' and agencies'
respective comments and our responses, as warranted, are as follows:
o Agriculture's Associate CIO provided e-mail comments stating that the
department will incorporate our recommendation into its enterprise
architecture program plan.
o Commerce's CIO stated in written comments that the department
concurred with our findings and will consider actions to address our
recommendation. Commerce's written comments are reproduced in appendix
V.
o DOD's Director, Architecture and Interoperability, stated in written
comments that the department generally concurred with our
recommendation to the five DOD architecture programs included in our
review. However, the department stated that it did not concur with the
one aspect of the recommendation directed at the GIG architecture
concerning independent verification and validation (IV&V) because it
believes that its current internal verification and validation
activities are
37Representatives from the Departments of Health and Human Services and
Transportation stated that they did not have comments.
Page 38 GAO-06-831 Enterprise Architecture
sufficient. We do not agree for two reasons. First, these internal
processes are not independently performed. As we have previously reported,
IV&V is a recognized hallmark of well managed programs, including
architecture programs, and to be effective, it must be performed by an
entity that is independent of the processes and products that are being
reviewed. Second, the scope of the internal verification and validation
activities only extends to a subset of the architecture products and
management processes.
The department also stated that it did not concur with one aspect of our
finding directed at BEA addressing security. According to DOD, because GIG
addresses security and the GIG states that it extends to all defense
mission areas, including the business mission area, the BEA in effect
addresses security. We do not fully agree. While we acknowledge that GIG
addresses security and states that it is to extend to all DOD mission
areas, including the business mission area, it does not describe how this
will be accomplished for BEA. Moreover, nowhere in the BEA is security
addressed, either through statement or reference, relative to the
architecture's performance, business, information/data,
application/service, and technology products. DOD's written comments,
along with our responses, are reproduced in appendix VI.
o Education's Assistant Secretary for Management and Acting CIO stated
in written comments that the department plans to address our findings.
Education's written comments are reproduced in appendix VII.
o Energy's Acting Associate CIO for Information Technology Reform stated
in written comments that the department concurs with our report.
Energy's written comments are reproduced in appendix VIII.
o DHS's Director, Departmental GAO/OIG Liaison Office, stated in written
comments that the department has taken, and plans to take, steps to
address our recommendation. DHS's written comments, along with our
responses to its suggestions for improving our framework, are
reproduced in appendix IX. DHS also provided technical comments via
e-mail, which we have incorporated, as appropriate, in the report.
o HUD's CIO stated in written comments that the department generally
concurs with our findings and is developing a plan to address our
recommendation. The CIO also provided updated information about
activities that the department is taking to address security in its
architecture. HUD's written comments are reproduced in appendix X.
o Interior's Assistant Secretary, Policy, Management and Budget, stated
in written comments that the department agrees with our findings and
recommendation and that it has recently taken action to address them.
Interior's written comments are reproduced in appendix XI.
o DOJ's CIO stated in written comments that our findings accurately
reflect the state of the department's enterprise architecture program
and the areas that it needs to address. The CIO added that our report
will help guide the department's architecture program and provided
suggestions for improving our framework and its application. DOJ's
written comments, along with our responses to its suggestions, are
reproduced in appendix XII.
o Labor's Deputy CIO provided e-mail comments stating that the
department concurs with our findings. The Deputy CIO also provided
technical comments that we have incorporated, as appropriate, in the
report.
o State's Assistant Secretary for Resource Management and Chief
Financial Officer provided written comments that summarize actions
that the department will take to fully satisfy certain core elements
and that suggest some degree of disagreement with our findings
relative to three other core elements. First, the department stated
that its architecture configuration management plan has been approved
by both the State and USAID CIOs. However, it provided no evidence to
demonstrate that this was the case as of March 2006 when we concluded
our review, and thus we did not change our finding relative to
architecture products being under configuration management. Second,
the department stated that its enterprise architecture has been
approved by State and USAID executive offices. However, it did not
provide any documentation showing such approval. Moreover, it did not
identify which executive offices it was referring to so as to allow a
determination of whether they were collectively representative of the
enterprise. As a result, we did not change our finding relative to
whether a committee or group representing the enterprise or an
investment review board has approved the current version of the
architecture. Third, the department stated that it provided us with IT
investment score sheets during our review that demonstrate that
investment compliance with the architecture is measured and reported.
However, no such score sheets were provided to us. Therefore, we did
not change our finding. The department's written comments, along with
more detailed responses, are reproduced in appendix XIII.
o Treasury's Associate CIO for E-Government stated in written comments
that the department concurs with our findings and discussed steps
being taken to mature its enterprise architecture program. The
Associate CIO also stated that our findings confirm the department's
need to provide executive leadership in developing its architecture
program and to codify the program into department policy. Treasury's
written comments are reproduced in appendix XIV.
o VA's Deputy Secretary stated in written comments that the department
concurred with our recommendation and that it will provide a detailed
plan to implement our recommendation. VA's written comments are
reproduced in appendix XV.
o EPA's Acting Assistant Administrator and CIO stated in written
comments that the agency generally agreed with our findings and that
our assessment is a valuable benchmarking exercise that will help
improve agency performance. The agency also provided comments on our
findings relative to five core elements. For one of these core
elements, the comments directed us to information previously provided
about the agency's architecture committee that corrected our
understanding and resulted in us changing our finding about this core
element. With respect to the other four core elements concerning use
of an architecture methodology, measurement of progress against
program plans, integration of the architecture into investment
decision making, and management of architecture change, the comments
also directed us to information previously provided but this did not
result in any changes to our findings because evidence demonstrating
full satisfaction of each core element was not apparent. EPA's written
comments, along with more detailed responses to each, are reproduced
in appendix XVI.
o GSA's Administrator stated in written comments that the agency concurs
with our recommendation. The Administrator added that our findings
will be critical as the agency works towards further implementing our
framework's core elements. GSA's written comments are reproduced in
appendix XVII.
o NASA's Deputy Administrator stated in written comments that the agency
concurs with our recommendation. NASA's written comments are
reproduced in appendix XVIII. NASA's GAO Liaison also provided
technical comments via e-mail, which we have incorporated, as
appropriate, in the report.
o NSF's CIO provided e-mail comments stating that the agency will use
the information in our report, where applicable, for future planning
and investment in its architecture program. The CIO also provided
technical comments that we have incorporated, as appropriate, in the
report.
o NRC's GAO liaison provided e-mail comments stating that the agency
substantially agrees with our findings and describing activities it
has recently taken to address them.
o OPM's CIO provided e-mail comments stating that the agency agrees with
our findings and describing actions it is taking to address them.
o SBA's GAO liaison provided e-mail comments in which the agency
disagreed with our findings on two core elements. First, and
notwithstanding agency officials' statements that its architecture
program did not have adequate resources, the liaison did not agree
with our "partially satisfied" assessment for this core element
because, according to the liaison, the agency has limited
discretionary funds and competing, but unfunded, federal mandates to
comply with that limit discretionary funding for an agency of its
size. While we acknowledge SBA's challenges, we would note that they
are not unlike the resource constraints and competing priority
decisions that face most agencies, and that while the reasons why an
architecture program may not be adequately resourced may be justified,
the fact remains that any assessment of the architecture program's
maturity, and thus its likelihood of success, needs to recognize
whether adequate resources exist. Therefore, we did not change our
finding on this core element. Second, the liaison did not agree with
our finding that the agency did not have plans for developing metrics
for measuring architecture progress, quality, compliance, and return
on investment. However, our review of documentation provided by SBA
and cited by the liaison showed that while such plans address metric
development for architecture progress, quality, and compliance, they
do not address architecture return on investment. Therefore, we did
not change our finding that this core element was partially satisfied.
* SSA's Commissioner stated in written comments that the report is
both informative and useful, and that the agency agrees with our
recommendation and generally agrees with our findings.
Nevertheless, the agency disagreed with our findings on two core
elements. First, the agency stated that documentation provided to
us showed that it has a methodology for developing, maintaining,
and validating its
* architecture. We do not agree. In particular, our review of SSA
provided documentation showed that it did not adequately describe
the steps to be followed relative to development, maintenance, or
validation. Second, the agency stated that having the head of the
agency approve the current version of the architecture is
satisfied in SSA's case because the Clinger-Cohen Act of 1996
vests its CIO with enterprise architecture approval authority and
the CIO has approved the architecture. We do not agree. The core
element in our framework concerning enterprise architecture
approval by the agency head is derived from federal guidance and
best practices upon which our framework is based. This guidance
and related practices, and thus our framework, recognize that an
enterprise architecture is a corporate asset that is to be owned
and implemented by senior management across the enterprise, and
that a key characteristic of a mature architecture program is
having the architecture approved by the department or agency
head. Because the Clinger-Cohen Act does not address approval of
an enterprise architecture, our framework's core element for
agency head approval of an enterprise architecture is not
inconsistent with, and is not superseded by, that act. SSA's
written comments, along with more detailed responses, are
reproduced in appendix XIX.
o USAID's Acting Chief Financial Officer stated in written comments
stated that the agency will work with State to implement our
recommendation. USAID's written comments are reproduced in appendix
XX.
As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from
the report date. At that time, we will send copies of this report to the
Administrators of the Environmental Protection Agency, General Services
Administration, National Aeronautics and Space Administration, Small
Business Administration, and U.S. Agency for International Development;
the Attorney General; the Commissioners of the Nuclear Regulatory
Commission and Social Security Administration; the Directors of the
National Science Foundation and the Office of Personnel Management; and
the Secretaries of the Departments of Agriculture, Commerce, Defense,
Education, Energy, Health and Human Services, Homeland Security, Housing
and Urban Development, Interior, Labor, State, Transportation, Treasury,
and Veterans Affairs. We will also make copies available to others upon
request. In addition, the report will be available at no charge on the GAO
Web site at http://www.gao.gov.
If you have any questions concerning this information, please contact me
at (202) 512-3439 or by e-mail at [email protected]. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on the
last page of this report. Key contributors to this report are listed in
appendix
XXI.
Sincerely yours,
Randolph C. Hite
Director, Information Technology Architecture and Systems Issues
Appendix I
Reported Enterprise Architecture Costs Vary, with Contractors and Personnel
Accounting for Most Costs
Department- and agency-reported data show wide variability in their costs
to develop and maintain their enterprise architectures. Generally, the
costs could be allocated to several categories with the majority of costs
attributable to contractor support and agency personnel.
Architecture Development and Maintenance Costs Vary
As we have previously reported, the depth and detail of the architecture
to be developed and maintained is dictated by the scope and nature of the
enterprise and the extent of enterprise transformation and modernization
envisioned. Therefore, the architecture should be tailored to the
individual enterprise and that enterprise's intended use of the
architecture. Accordingly, the level of resources that a given department
or agency invests in its architecture is likely to vary.
Departments and agencies reported that they have collectively invested a
total of $836 million to date on enterprise architecture development.
Across the 27 departments and agencies, these development costs ranged
from a low of $2 million by the Department of the Navy to a high of $433
million by the Department of Defense (DOD) on its Business Enterprise
Architecture (BEA). Department and agency estimates of the costs to
complete their planned architecture development efforts collectively total
about $328 million. The department and agencies combined estimates of
annual architecture maintenance costs is about $146 million. These
development and maintenance estimates, however, do not include the
Departments of the Army and Justice because neither provided these cost
estimates. Figures 7 through 9 depict the variability of cost data
reported by the departments and agencies.
Figure 7: Reported Development Costs to Date for Departments and Agencies
Dollars in millions
440 430 420 410 400
70 60 50 40 30 20
ce y
CommerBEA GIGEducatione
y
SHHticeInteriorHUDSDHsJu
Labor vy
tateS
T y
A
ASGASNA
F
OPMNRCBAS
SSA
AIDSU
DASU
VA
Arm
c
gEnerursa
DOEP
SN
Air For
Na
e
Tr
Departments and agencies
Source: Cited departmentsand agencies.
Note: The Departments of the Army and Justice did not provide development
costs.
ce y
CommerBEA GIGEducatione
gy S
Ju
DHHUDInteriorsticeS
Labor vy
State T y
A
SA
SA SF
NRC OPMSBA SSA
SAID
SDA
VA
Arm
c
DO
surEP
HH
Air For
Na
EnerN
G
NA
U
a
U
e
Tr
Departments and agencies
Source: Cited departmentsand agencies.
Note: The Departments of the Air Force, Army, Energy, Justice, and Navy,
the DOD's BEA and Global Information Grid (GIG), and OPM did not provide
completion costs. NSF reported completion costs ($15,000), which are not
identified on this figure.
Figure 9: Reported Estimated Annual Maintenance Costs for Departments and
Agencies Dollars in millions25
20
15
10
5
0
ce y
CommerBEA GIGEducatione
y
SHHticeInteriorHUDSDHsJu
Labor vy
tateS
T y
A
ASGASNA
F
S
NRC OPM BA SSA AID
DASU
VA
Arm
c
gEnerursa
DOEP
SN
Air For
Na
SU
e
Tr
Departments and agencies
Source: Cited departmentsand agencies.
Note: The Departments of the Army, Justice, and Navy did not provide
maintenance costs.
Contractor Support Accounts for the Majority of Architecture Development
Costs
All of the departments and agencies reported developing their architecture
in-house using contractor support. All but two of the departments and
agencies allocated their respective architecture development costs to the
following cost categories:1 contractor support, agency personnel, tools,
methodologies, training, and other.2 These 26 agencies accounted for about
$741 million of the $836 million total development costs cited above. The
vast majority (84 percent) of the $741 million were allocated to
contractor services ($621 million), followed next by agency personnel (13
percent or $94 million). The remaining $26 million were allocated as
follows: $12 million (2 percent) to architecture tools; $9 million (1
percent) to "other"
1The Departments of the Army and Justice did not provide cost data. 2The
"other" cost category includes costs that cannot be allocated to the other
categories.
Page 48 GAO-06-831 Enterprise Architecture
costs; $4 million (1 percent) to architecture methodologies; and $2
million (less than 1 percent) to training. (See fig. 10.)
Figure 10: Breakdown of Enterprise Architecture Development Costs for all
Departments and Agencies
<1%
Training
1%
Methodology
2%
Tools
1%
Other
Agency personnel
Contractor costs
Source: GAO analysis of department and agency data.
Note: Numbers do not add to 100 percent due to rounding.
Architecture Development Activities Were Reported as
Largest Component of Contractor-Related Costs
The departments and agencies allocated the
reported $621 million in contractor-related costs to the following five
contractor cost categories: architecture development, independent
verification and validation,
methodology, support services, and other.3 Of these categories,
architecture development activities accounted for the majority of costs-
about $594 million (87 percent). The remaining $85 million was allocated
as follows: $51 million (7 percent) to support services, $13 million (2
percent) to "other" costs, $11 million (2 percent) to independent
verification and validation, and $10 million (1 percent) to methodologies.
(See fig. 11.)
Figure 11: Reported Enterprise Architecture Contractor Costs by Category
1%
Methodology
2%
Independent verification & validation
2%
Other
Support services
Architecture development
Source: GAO analysis of department and agency data.
Note: Numbers do not add to 100 percent due to rounding.
3The "other" cost category is intended to include costs that cannot be
allocated to the categories we specified.
Page 50 GAO-06-831 Enterprise Architecture
Appendix II
Departments and Agencies Reported Experiences with Their Architecture Tools and
Frameworks
Departments and agencies reported additional information related to the
implementation of their enterprise architectures. This information
includes architecture tools and frameworks.
Departments and Agencies Reported Using a Variety of Enterprise
Architecture Tools with Varying Degrees of Satisfaction
As stated in our enterprise architecture management maturity framework, an
automated architecture tool serves as the repository of architecture
artifacts, which are the work products that are produced and used to
capture and convey architectural information. An agency's choice of tool
should be based on a number of considerations, including agency needs and
the size and complexity of the architecture.1
The departments and agencies reported that they use various automated
tools to develop and maintain their enterprise architectures, with 12
reporting that they use more than one tool. In descending order of
frequency, the architecture tools identified were System Architect (18
instances), Microsoft Visio (17), Metis (12), Rational Rose (8), and
Enterprise Architecture Management System (EAMS) (4). In addition, 21
departments and agencies reported using one or more other architecture
tools.2 Figure 12 shows the number of departments and agencies using each
architecture tool, including the other tools.3
1 GAO-03-584G.
2The "other" tool category is intended to include various tools that were
not listed on our survey.
3Other tools reported by the departments and agencies include: Adaptive
Information Technology Portfolio Manager, Adaptive-USDA EA Repository,
Caliber-RM, Catalyze by SteelTrace, Defense Architecture Repository System
(DARS), DARS MS Office, Embarcadero-ER Studio, Erwin, FRA Portal, MS Word,
OMG Component Collaborative Architecture (Component X Tool), ProSight,
Serena Tracker and Version Manager.
Figure 12: Enterprise Architecture Tools Used by Departments and Agencies
Number of agencies
25
Other System Visio MetisRational EAMS Architect Rose
Tools
Source: GAO analysis of department and agency data.
The departments and agencies also reported various levels of satisfaction
with the different enterprise architecture tools. Specifically, about 75
percent of those using Microsoft Visio were either very or somewhat
satisfied with the tool, as compared to about 67 percent of those using
Metis, about 63 percent of those using Rational Rose, about 59 percent of
those using System Architect, and 25 percent of those using EAMS. This
means that the percentage of departments and agencies that were
dissatisfied, either somewhat or very, with their respective tools ranged
from a high of 75 percent of those using EAMS, to a low of about 6 percent
of those using System Architect. No departments or agencies that used
Metis, Rational Rose, or Microsoft Visio reported any dissatisfaction. See
table 9 for a summary of department and agency reported satisfaction with
their respective tools.
Table 9: Department and Agency Reported Satisfaction with Tools
Number of
departments and
agencies
Somewhat
Very satisfied or Neither or very Undecided
satisfied nor (too
Tool name (Vendor) Using somewhat dissatisfied dissatisfied early to
tool satisfied say)
EAMS 4 1 0 3 0
Metis (Troux
Technologies) 12 8 1 0 3
Rational Rose (IBM
Corporation) 8 5 2 0 1
System Architect
(Popkin
Software/Telelogic 18 10 4 1 2
AB)
Visio (Microsoft
Corporation) 17 12 3 0 1
Other 21 17 0 0 1
Source: GAO based on department and agency data.
Note: One agency did not indicate its satisfaction or dissatisfaction with
System Architect and Visio.
Departments and Agencies Reported Using a Variety of Enterprise
Architecture Frameworks with Varying Levels of Satisfaction
As we have previously stated, an enterprise architecture framework
provides a formal structure for representing the architecture's content
and serves as the basis for the specific architecture products and
artifacts that the department or agency develops and maintains. As such, a
framework helps ensure the consistent representation of information from
across the organization and supports orderly capture and maintenance of
architecture content.
The departments and agencies reported using various frameworks to develop
and maintain their enterprise architectures. The most frequently cited
frameworks were the Federal Enterprise Architecture Program Management
Office (FEAPMO) Reference Models (25 departments and agencies), the
Federal Enterprise Architecture Framework (FEAF)4 (19 departments and
agencies), and the Zachman Framework (17 departments and agencies), with
24 reporting using more than one framework. Other, less frequently
reported frameworks were the Department of Defense
4This framework was issued in September 1999 by the federal CIO Council.
Architecture Framework (DODAF), the National Institute of Standards and
Technology (NIST) framework, and The Open Group Architecture Framework
(TOGAF). See figure 13 for a summary of the number of departments and
agencies that reported using each framework.
Figure 13: Frameworks Used by Departments and Agencies
Number of agencies
25
20
15
10
5
0 FEAPMO FEAF Zachman DODAF Other NIST TOGAF
Framework
Source: GAO analysis of department and agency data.
Departments and agencies also reported varying levels of satisfaction with
their respective architecture. Specifically, about 72 percent of those
using the FEAF indicated that they were either very or somewhat satisfied,
and about 67 and 61 percent of those using the Zachman framework and the
FEAPMO reference models, respectively, reported that they were similarly
satisfied.5 As table 10 shows, few of the agencies that responded to our
survey reported being dissatisfied with any of the frameworks.6
5Some agencies and departments did not indicate their level of
satisfaction or dissatisfaction with the framework(s) they reported using.
6The number of responses regarding frameworks is larger than the number of
agencies surveyed because some agencies reported using more than one
framework.
Page 54 GAO-06-831 Enterprise Architecture
Table 10: Department and Agency Framework Satisfaction Levels
Number of
Departments
and Agencies
Very satisfied Neither Somewhat or
Using or somewhat satisfied nor very Undecided
(too
Framework framework satisfied dissatisfied dissatisfied early to
(source) say)
DODAF
(Department of 7 4 2 0 0
Defense)
FEAF (CIO 19 13 3 1 1
Council)
FEAPMO
Reference 25 14 4 3 2
Models (OMB)
NIST Framework 2 1 0 1 0
(NIST)
TOGAF (The Open 1 1 0 0 0
Group)
Zachman
Framework (The
Zachman
Institute for 17 10 4 1 0
Framework
Advancement)
Other 3 3 0 0 0
Source: GAO based on department and agency data.
Appendix III
Objective, Scope, and Methodology
Our objective was to determine the current status of federal department
and agency enterprise architecture efforts. To accomplish this objective,
we focused on 28 enterprise architecture programs relating to 27 major
departments and agencies. These 27 included the 24 departments and
agencies included in the Chief Financial Officers Act.1 In addition, we
included the three military services (the Departments of the Army, Air
Force, and Navy) at the request of Department of Defense (DOD) officials.
For the DOD, we also included both of its departmentwide enterprise
architecture programs-the Global Information Grid and the Business
Enterprise Architecture. The U.S. Agency for International Development
(USAID), which is developing a USAID enterprise architecture and working
with the Department of State (State) to develop a Joint Enterprise
Architecture, asked that we evaluate its efforts to develop the USAID
enterprise architecture. State officials asked that we evaluate their
agency's enterprise architecture effort based the Joint Enterprise
Architecture being developed with USAID. We honored both of these
requests.
Table 11 lists the 28 department and agency enterprise architecture
programs that formed the scope of our review.
Table 11: List of Architecture Programs Included in this Report
Agency
Department of Agriculture
Department of the Air Force
Department of the Army
Department of Commerce
Department of Defense - Business Enterprise Architecture
Department of Defense - Global Information Grid
Department of Education
Department of Energy
Department of Health and Human Services
Department of Homeland Security
Department of Housing and Urban Development
Department of the Interior
1This Act requires 24 departments and agencies to establish chief
financial officers. See 31
U.S.C. section 901.
(Continued From Previous Page)
Agency
Department of Justice
Department of Labor
Department of the Navy
Department of State
Department of Transportation
Department of the Treasury
Department of Veterans Affairs
Environmental Protection Agency
General Services Administration
National Aeronautics and Space Administration
National Science Foundation
Nuclear Regulatory Commission
Office of Personnel Management
Small Business Administration
Social Security Administration
U.S. Agency for International Development
Source: GAO.
To determine the status of each of these architecture programs, we
developed a data collection instrument based on our Enterprise
Architecture Management Maturity Framework (EAMMF),2 and related guidance,
such as OMB Circular A-1303 and guidance published by the federal Chief
Information Officers (CIO) Council,4 and our past reports and guidance on
the management and content of enterprise architectures.5 We pretested this
instrument at one department and one agency. Based on the results of the
pretest, we modified our instrument as appropriate to ensure that our
areas of inquiry were complete and clear.
2 GAO-03-584G.
3Office of Management and Budget, Management of Federal Information
Resources, Circular A-130 (Nov. 28, 2000).
4Chief Information Officers Council, Federal Enterprise Architecture
Framework, Version
1.1 (September 1999) and Chief Information Officers Council, A Practical
Guide to Federal Enterprise Architecture, Version 1.0 (February 2001).
5 GAO-02-6; GAO-04-40; and, for example, GAO-03-1018 , GAO-04-777 ,
GAO-05-702 , GAO-06-219.
Next, we identified the Chief Architect or comparable official at each of
the 27 departments and agencies, and met with them to discuss our scope
and methodology, share our data collection instrument, and discuss the
type and nature of supporting documentation needed to verify responses to
our instrument questions.6
On the basis of department and agency provided documentation to support
their respective responses to our data collection instrument, we analyzed
the extent to which each satisfied the 31 core elements in our
architecture maturity framework. To guide our analysis, we defined
detailed evaluation criteria for determining whether a given core element
was fully satisfied, partially satisfied, or not satisfied. The criteria
for the stage 2, 3, 4, and 5 core elements are contained in tables 12, 13,
14, and 15 respectively. To fully satisfy a core element, sufficient
documentation had to be provided to permit us to verify that all aspects
of the core element were met. To partially satisfy a core element,
sufficient documentation had to be provided to permit us to verify that at
least some aspects of the core element were met. Core elements that were
neither fully nor partially satisfied were judged to be not satisfied.
Table 12: Stage 2 Evaluation Criteria
Core element Evaluation criteria
Adequate resources exist. Agency responded that "very adequate,""somewhat
adequate," or "neither adequate nor inadequate"resources exist for
funding, personnel, and tools.
Committee or group representing the enterprise is Agency (1) responded
that a committee or group representing the enterprise is responsible for
directing, overseeing, and responsible for direction, oversight, and
approval of the enterprise architecture; (2) approving EA. provided a
charter or other documentation supporting the group's responsibilities;
and (3) provided sample meeting minutes or other documentation confirming
that
meetings have been held.
Program office responsible for EA development Agency (1) responded that a
program office is responsible for EA development and and maintenance
exists. maintenance and (2) provided documentation supporting their
assertion.
Chief architect exists. Agency (1) responded that chief architect exists
and (2) provided documentation or assertion that the chief architect is
responsible and accountable for EA and serves as the EA program manager.
6The Social Security Administration was the only agency to decline such an
initial meeting.
Page 58 GAO-06-831 Enterprise Architecture
(Continued From Previous Page)
Core element Evaluation criteria
EA being developed using a Agency (1) responded that the
framework, enterprise architecture is being
developed using a
methodology, and automated tool. framework, methodology, and automated
tool; (2) provided documentation
supporting
the use of a framework and automated
tool; and (3) provided a documented
methodology that includes steps for
developing, maintaining, and
validating the
enterprise architecture.
EA plans call for describing "as-is" Agency (1) responded that EA plans
environment, call for describing the "as-is" and
"to-be"
"to-be" environment, and sequencing environments and a sequencing plan
plan. and (2) provided plans that document
this
assertion; or agency (1) responded
that the EA describes the "as-is" and
"to-be"
environments and a sequencing plan
and (2) provided documentation to
support this
assertion.
EA plans call for describing enterprise in terms of business, performance,
information/data, application/service, and technology.
Agency (1) responded that EA plans call for describing the enterprise in
terms of business, performance, information/data, application/service, and
technology and (2) provided plans that document this assertion; or agency
(1) responded that the EA describes the enterprise in terms of business,
performance, information/data, application/service, and technology and (2)
provided documentation to support this assertion.
EA plans call for business, performance, information/data,
application/service, and technology to address security.
Agency (1) responded that EA plans call for business, performance,
information/data, application/service, and technology descriptions to
address security and (2) provided plans that document this assertion; or
agency (1) responded that the business, performance, information/data,
application/service, and technology descriptions address security and (2)
provided documentation to support this assertion.
EA plans call for developing metrics to measure Agency (1) responded that
EA plans call for developing metrics to measure EA EA progress, quality,
compliance, and return on progress, quality, compliance, and return on
investment and (2) provided plans to investment. support this assertion;
or responded (1) that EA progress, quality, compliance, and/or
return on investment is measured and reported and (2) provided support for
this assertion.
Source: GAO.
Table 13: Stage 3 Evaluation Criteria
Core element Evaluation criteria
Written and approved policy exists Agency (1) responded that a written
for EA and approved organization policy
exists for EA
development. development and (2) provided a policy
that supported this assertion.
Agency (1) responded that EA products
EA products are under configuration are under configuration management and
(2)
management. provided their formally documented
configuration management approach.
EA products describe or will describe "as-is"environment, "to-be"
environment, and sequencing plan.
Agency (1) responded that EA plans call for describing the "as-is" and
"to-be"environments and a sequencing plan, (2) provided plans that
document this assertion, and (3) responded that it is "in the process of
developing the EA" or that it "has developed an EA"; or agency (1)
responded that the EA describes the "as-is"and "to-be" environments and a
sequencing plan and (2) provided documentation to support this assertion.
(Continued From Previous Page)
Core element Evaluation criteria
Both "asis" and "to-be" Agency (1) responded that EA plans call
environments are described or will for describing the enterprise in terms
be described in terms of business, of business, performance,
performance, information/data, information/data, application/service,
application/service, and and technology; (2) provided plans that
technology. document this assertion; and (3)
responded that it is "in the process of
developing the EA" or that it "has
developed an EA"; or agency (1)
responded that the EA describes the
enterprise in terms of business,
performance, information/data,
application/service, and technology and
(2) provided documentation to support
this assertion.
These descriptions address or will address Agency (1) responded that EA
plans call for business, performance, information/data,
security. plans that document this assertion; and (3) responded that it is
"in the process of developing the EA"or that it "has developed an EA"; or
agency (1) responded that the business, performance, information/data,
application/service, and technology descriptions address security and (2)
provided documentation to support this assertion.
Progress against EA plans is measured and Agency (1) responded that it
measures and reports progress against plans; (2) reported. provided a
description of how progress against plans is measured and reported; and
(3) provided sample reports that include sample measures.
Source: GAO.
Table 14: Stage 4 Evaluation Criteria
Core element Evaluation criteria
Written and approved policy exists for EA Agency (1) responded that a
written and approved organization policy exists for EA maintenance.
maintenance and (2) provided a policy that supported this assertion.
EA products and management processes undergo independent verification and
validation.
Agency (1) responded that EA products and management processes undergo
independent verification and validation; (2) provided proof that
independent verification and validation activities were conducted by an
independent third party and reported outside the span of control of the
chief architect; and (3) provided sample independent verification and
validation reports to the audit team. Independence was a critical element
for satisfaction of this item.
EA products describe "as-is" environment, "to-be"environment, and
sequencing plan.
Agency (1) responded that the EA describes the "as-is" and "to-be"
environments and a sequencing plan; (2) provided documentation to support
this assertion; and (3) responded that it has developed an EA. In
addition, an agency could not receive full credit for satisfying this
element unless it fully satisfied the element, "Both `as-is' and `to-be'
environments are described in terms of business, performance,
information/data, application/service, and technology."
Both "as-is" and "to-be" environments are described in terms of business,
performance, information/data, application/service, and technology.
Agency (1) responded that the EA describes the enterprise in terms of
business, performance, information/data, application/service, and
technology; (2) provided documentation to support this assertion; and (3)
responded that it has developed an EA. Agencies that completed four or
five required descriptions in both the "as-is" and "to-be"environments
received a yes for this item. Agencies that addressed less that two of the
five required descriptions in both the "as-is" and "to-be" environments
received a no for this element.
(Continued From Previous Page)
Core element Evaluation criteria
These descriptions address security. Agency (1) responded that the
business, performance, information/data, application/service, and
technology descriptions address security; (2) provided documentation to
support this assertion; and (3) responded that it has developed an EA.
Organization CIO has approved EA. Agency (1) responded that that CIO has
approved the current version of the EA and
(2) provided a signature page or other proof that the CIO has approved
current version of EA.
Committee or group representing the enterprise Agency (1) responded that a
committee or group representing the enterprise or the or the investment
review board has approved investment review board has approved current
version of EA and (2) provided current version of EA. meeting minutes or
other proof that a committee or group representing the enterprise
or the investment review board has approved current version of EA.
Quality of EA products is measured and Agency (1) responded that it
measures and reports product quality, (2) provided a reported. description
of howquality is measured and reported, and (3) provided sample reports
that include sample measures.
Source: GAO.
Table 15: Stage 5 Evaluation Criteria
Core element Evaluation criteria
Written and approved organization Agency (1) responded that a written
policy exists and approved organization policy
exists for IT
for IT investment compliance with EA. investment compliance with EA and
(2) provided a written policy to
support this
assertion.
Process exists to formally manage EA Agency (1) responded that a process
change. exists to formally manage EA change
and (2)
provided evidence to support this
assertion.
EA is integral component of IT investment management process.
Agency (1) responded that EA is an integral component of IT investment
management process; (2) provided documentation describing how the EA is
used when making IT investment decisions; (3) provided evidence that a
sequencing plan exists to guide IT investments; and (4) partially or fully
satisfied at least one of the following stage 3 elements: (a) EA products
describe or will describe "as-is"environment, "to-be" environment, and
sequencing plan, (b) both "as-is" and "to-be"environments are described or
will be described in terms of business, performance, information/data,
application/service, and technology, or (c) these descriptions address or
will address security.
EA products are periodically updated. Agency (1) responded that EA
products are periodically updated and (2) provided a description of the
process used for updating EA products.
IT investments comply with EA.
Agency (1) responded that IT investments comply with EA; (2) provided
evidence that IT is not selected and approved under the organization's
capital planning and investment control process unless it is compliant
with the EA; and (3) partially or fully satisfied at least one of the
following stage 3 elements: (a) EA products describe or will describe
"as-is" environment, "to-be" environment, and sequencing plan, (b) both
"as-is" and "to-be" environments are described or will be described in
terms of business, performance, information/data, application/service, and
technology, or (c) these descriptions address or will address security.
(Continued From Previous Page)
Core element Evaluation criteria
Organization head has approved current version of EA.
Agency (1) responded that the organization head has approved the current
version of the EA; (2) provided a signature page or other proof that
organization head or a deputy organization head has approved current
version of EA or provided proof of formal delegation of this activity and
subsequent approval; and (3) partially or fully satisfied at least one of
the following stage 3 elements: (a) EA products describe or will describe
"as-is" environment, "to-be" environment, and sequencing plan, (b) both
"as-is" and "to-be" environments are described or will be described in
terms given in Stage 2, or (c) these descriptions address or will address
security.
Return on EA investment is measured and Agency (1) responded that it
measures and reports return on investment; (2) reported. provided a
description of how return on investment is measured and reported; and (3)
provided sample reports that included sample measures.
Compliance with EA is measured and reported. Agency (1) responded that it
measures and reports compliance, (2) provided a description of how
compliance is measured and reported, and (3) provided sample reports that
included sample measures.
Source: GAO.
Our evaluation included first analyzing the extent to which each
department and agency satisfied the core elements in our framework, and
then meeting with department and agency representatives to discuss core
elements that were not fully satisfied and why. As part of this
interaction, we sought, and in some cases were provided, additional
supporting documentation. We then considered this documentation in
arriving at our final determinations about the degree to which each
department and agency satisfied each core element in our framework. In
applying our evaluation criteria, we analyzed the results of our analysis
across different core elements to determine patterns and issues. Our
analysis made use of computer programs that were developed by an
experienced staff; these programs were independently verified.
Through our data collection instrument, we also solicited from each
department and agency information on enterprise architecture challenges
and benefits, including the extent to which they had been or were expected
to be experienced. In addition, we solicited information on architecture
costs, including costs to date and estimated costs to complete and
maintain each architecture. We also solicited other information, such as
use of and satisfaction with architecture tools and frameworks. We
analyzed these additional data to determine relevant patterns. We did not
independently verify these data. The results presented in this report
reflect the state of department and agency architecture programs as of
March 8, 2006.7
We conducted our work in the Washington, D.C., metropolitan area, from May
2005 to June 2006, in accordance with generally accepted government
auditing standards.
7The Department of Defense submitted updated information to Congress about
its Business Enterprise Architecture on March 15, 2006. This information
was also considered as part of our evaluation.
Page 63 GAO-06-831 Enterprise Architecture
Appendix IV
Detailed Assessments of Individual Departments and Agencies against Our EA
Management Maturity Framework
Department of Agriculture
Table 16 shows USDA's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 16: Department of Agriculture Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data,
application/service, and technology to
address security.
EA plans call for developing metrics to Yes
measure and
report EA progress, quality, compliance, and
return on
investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for EA
development.
EA products are under No USDA plans to develop specific steps
configuration management. for configuration
management. However, specific
configuration management
steps have not been
developed.
EA products describe or will Yes
describe "as-is" environment,
"to-be" environment, and
sequencing plan.
Both "asis" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and
technology.
These descriptions address or Yes
will address security.
(Continued From Previous Page)
GAO basis for partially
Stages and core elements Satisfied? satisfied or not satisfied
determination
Progress against EA plans is Partial USDA has limited reporting of
measured and reported. EA progress. However,
progress is not measured and
reported relative to an EA
program plan.
Stage 4: Completing EA products
Written and approved
organization policy exists for Yes
EA
maintenance.
EA products and management No According to USDA's EA division,
processes undergo independent USDA component agencies conduct
verification and validation. reviews of the EA products of other
component agencies. However, the
documentation provided did not
address verification and validation
of EA management processes and did
not provide sufficient assurance
that the EA product reviews were
independent.
EA products describe "as-is" No According to USDA's EA division, EA
environment, "to-be"environment, products describe the "as-is"
and sequencing plan. environment. However, a description
of the "to-be"environment and a
sequencing plan have not been
developed.
Both "asis" and "to-be" No According to USDA's EA division,
environments are described in terms the "as-is" environment is
of business, performance, described in terms of business and
information/data, service/application. However, the
application/service, and "as-is" environment is not
technology. described in terms of performance,
information/data, and technology
and the "tobe" descriptions have
not been developed.
These descriptions address Partial According to USDA's EA division,
security. it has begun developing a
security architecture. However,
the security architecture is
still under development and does
not address all the
requisite descriptions.
Organization CIO has approved Yes
current version of EA.
Committee or group representing Yes
the enterprise or the
investment review board has
approved current version of
EA.
Quality of EA products is Yes
measured and reported.
Stage 5: Leveraging the EA for
managing change
Written and approved Partial USDA has a policy that encourages
organization policy exists for IT investment compliance with the
IT investment compliance with EA. However, the policy does not
EA. explicitly require IT investment
compliance with the EA.
Process exists to formally manage EA change. Partial USDA has assigned
responsibility for formally managing EA change to a board and has begun to
develop a process to formally manage EA change. However, evidence of this
process was not provided.
EA is integral component of IT Partial USDA provided documentation
investment management indicating that EA is an
process. integral component of the IT
investment management
process. However, the EA does not
include a sequencing
plan to guide such investments.
EA products are periodically Yes
updated.
(Continued From Previous Page)
GAO basis for partially
Stages and core elements Satisfied? satisfied or not satisfied
determination
IT investments comply with EA. Partial According to USDA EA officials,
IT investments comply with
the EA. However, documentation
provided to GAO did not
clearly indicate that all IT
investments comply with the EA.
Organization head has approved Yes
current version of EA.
Return on EA investment is No According to USDA EA officials,
measured and reported. they are beginning to
measure and report return on EA
investment. However,
documentation indicated that these
reports address the
integration of applications as a
result of specific initiatives
that are independent of the EA.
Compliance with EA is measured Partial Compliance with USDA component
and reported. agency architectures,
which are segments of the
departmentwide EA, is measured
and reported. However, USDA did not
provide
documentation that compliance with
the department EA as a
whole is measured and reported.
Source: GAO analysis of agency provided data.
Department of the Air Force
Table 17 shows the Air Force's satisfaction of
framework elements in version 1.1 of GAO's EAMMF.
Table 17: Department of the Air Force Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing Partial The Air Force has three
the enterprise is committees representing the
responsible for directing, enterprise, each of which is
overseeing, and approving responsible for directing,
EA. overseeing, and approving one
of three segments of the EA.
However, the charter for one
of these committees is not
approved and no evidence was
provided to support that this
committee has conducted any
meetings.
A program office responsible
for EA development and
maintenance exists. However,
Program office responsible for the directive establishing
EA development and maintenance this office has not been
exists. Chief architect exists. Partial Yes approved.
EA being developed using a framework, Partial The Air Force uses a
methodology, and framework and automated tool
to
automated tool. develop their EA. However,
our analysis of the
Department of
Defense Architecture
Framework (DODAF), which was
cited
as the Air Force's EA
methodology, determined that
it is not
an EA methodology.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of business,
performance, information/data,
application/service, and
technology.
EA plans call for business, Yes
performance, information/data,
application/service, and technology
to address security.
EA plans call for developing Partial The Air Force plans to develop
metrics to measure EA metrics to measure EA
progress, quality, compliance, progress, compliance, and return
and return on investment. on investment. However,
plans to develop metrics to
measure EA quality were not
provided.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under configuration management. No According to Air Force
Architecture Policy and Guidance Office officials, EA products are not
under configuration management.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
EA products describe or will Yes
describe "as-is" environment,
"to-be" environment, and sequencing
plan.
Both "asis" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and
technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Partial The Air Force has limited
measured and reported. reporting of EA progress.
However,
progress is not measured and
reported relative to an EA
program plan.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management No According to the Air Force
processes undergo independent Architecture Policy and Guidance
verification and validation. Office, EA products and
management processes have not
undergone independent
verification and validation.
EA products describe "as-is" No According to the Air Force
environment, "to-be" Architecture Policy and Guidance
environment, and sequencing plan. Office, EA products describe the
"asis" environment and "to
be" environment but do not
describe a sequencing plan.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address No According to the Air Force
security. Architecture Policy and Guidance
Office, the "as-is" and "to-be"
descriptions do not address
security.
Organization CIO has approved Yes
current version of EA.
Committee or group representing No According to the Air Force
the enterprise or the investment Architecture Policy and Guidance
review board has approved current Office, a committee or group
version of EA. representing the enterprise has
not approved the current version
of the EA.
Quality of EA products is No Yes According to the Air Force
measured and reported. Stage 5: Architecture Policy and Guidance
Leveraging the EA for managing Office, quality of EA products is
change Written and approved not measured and reported.
organization policy exists for IT
investment compliance with EA.
Process exists to formally manage EA change. Partial According to the Air
Force Architecture Policy and Guidance Office, a process exists to
formally manage EA change. However, the process by which changes are made
to the EA is not documented.
EA is integral component of IT No According to the Air Force Architecture
investment management Policy and Guidance
process. Office, EA is not currently an integral
component of the IT
investment management process.
EA products are periodically Yes
updated.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
IT investments comply with EA. Partial According to the Air Force
Architecture Policy and
Guidance Office, IT investments
comply with the EA. However, no
documentation of IT investment
compliance with the Air Force
EA was provided.
Organization head has approved No According to the Air Force
current version of EA. Architecture Policy and
Guidance Office, the
organization head has not
approved the current version of
the EA.
Return on EA investment is No According to the Air Force
measured and reported. Architecture Policy and
Guidance Office, return on EA
investment is not measured and
reported.
Compliance with EA is measured Partial According to the Air Force
and reported. Architecture Policy and
Guidance Office, compliance
with the EA is measured and
reported. However, compliance
measurement and reporting is
with respect to only one
segment of the EA.
Source: GAO analysis of agency provided data.
Department of the Army
Table 18 shows Army's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 18: Department of the Army Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage
1.
Stage 2: Building the EA
management foundation
Adequate resources exist. Partial According to Army officials,
adequate tools exist. However,
according to these same
officials, personnel resources
are somewhat inadequate and
funding resources are very
inadequate.
Committee or group representing No According to Army officials,
the enterprise is responsible committees or groups address
for directing, overseeing, and architecture issues. However, no
approving EA. committee or group has specific
responsibility for EA. Further,
documentation did not include a
charter or other description of a
committee or group representing
the enterprise that is
responsible for directing,
overseeing, and approving the EA.
Program office responsible for Partial According to Army officials, a
EA development and maintenance Yes program office responsible for EA
exists. Chief architect exists. Partial development and maintenance
EA being developed using a exists. However, the program
framework, methodology, and office charter is not approved.
automated tool. EA is being developed using a
framework and automated tool.
However, according to Army
officials, the EA is not
developed using a methodology.
EA plans call for describing Partial According to Army officials, EA
"as-is" environment, plans call for describing the
"to-be"environment, and "asis" environment, "to-be"
sequencing plan. environment, and sequencing plan.
However, these EA plans are not
approved.
EA plans call for describing Partial According to Army officials, EA
enterprise in terms of business, plans call for describing the
performance, information/data, enterprise in terms of business,
application/service, and performance, information/data,
technology. application/service, and
technology. However, these EA
plans are not approved.
EA plans call for business, No According to Army officials, EA
performance, information/ data, plans call for business,
application/service, and performance, information/data,
technology to address security. application/service, and
technology to address security.
However, documentation of these
plans were not provided.
EA plans call for developing Partial According to Army officials, EA
metrics to measure EA progress, plans call for developing metrics
quality, compliance, and return to measure EA progress, quality,
on investment. compliance, and return on
investment. However,
documentation of plans for
developing metrics to measure
compliance and return on
investment were not provided.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Stage 3: Developing EA
products
Written and approved for EA Partial The Army policy for EA
organization policy exists development is not approved.
development.
EA products are under Partial Some but not all EA products are
configuration management. under configuration management.
EA products describe or will Partial According to Army officials, EA
describe "as-is"environment, products will describe the
"to-be" environment, and "asis" environment, "to-be"
sequencing plan. environment, and sequencing
plan. However, plans to develop
these EA descriptions are not
approved.
Both "asis" and "to-be" Partial According to Army officials, EA
environments are described or plans call for describing the
will be described in terms of enterprise in terms of business,
business, performance, performance, information/data,
information/data, application/service, and
application/service, and technology. However, these EA
technology. plans are not approved.
These descriptions address or No According to Army officials, EA
will address security. plans call for business,
performance, information/data,
application/service, and
technology to address security.
However, documentation of these
plans was not provided.
Progress against EA plans is Partial Progress against plans is
measured and reported. Stage 4: Partial measured and reported for one
Completing EA products Written portion of Army EA. However,
and approved organization policy progress against EA plans is not
exists for EA maintenance. measured and reported for other
EA segments that are under
development. The Army policy for
EA maintenance is not approved.
EA products and management No According to Army officials, EA
processes undergo independent products undergo review by
verification and validation. engineering boards and others.
However, they did not provide
evidence that these reviews are
independent, constitute
verification and validation, and
include EA management processes.
EA products describe "as-is" No According to Army officials, the EA
environment, "to-be"environment, currently consists of a partial
and sequencing plan. description of the
"as-is"environment. These officials
stated that a complete description
of the "as-is" environment as well
as descriptions of the
"to-be"environment and sequencing
plan will be developed in the
future.
Both "asis" and "to-be" No According to Army officials, the
environments are described in terms "as-is" environment is described in
of business, performance, terms of technology. These
information/data, officials stated that descriptions
application/service, and of the "as-is" environment in terms
technology. of business, performance, and
information/data as well as
descriptions of the
"to-be"architecture will be
developed in the future.
These descriptions address No According to Army officials,
security. business, performance,
information/data,
application/service, and technology
descriptions address security.
However, documentation of these
descriptions was not provided.
Organization CIO has approved No According to Army officials, the
current version of EA. organization chief information
officer has not approved the EA.
(Continued From Previous Page)
GAO basis for partially satisfied or not satisfied
Stages and core elements Satisfied? determination
Committee or group representing No According to Army officials, a
the enterprise or the investment committee or group representing
review board has approved the enterprise or the investment
current version of EA. review board has not approved the
current version of the EA.
Quality of EA products is Partial According to Army officials, the
measured and quality of EA products is
reported. measured and reported. Further,
documentation indicated that
a quality evaluation was
performed. However, documentation
of
the evaluation results was not
provided.
Stage 5: Leveraging the EA for
managing
change
Written and approved No Army officials did not provide a
organization policy exists for written and approved organization
IT investment compliance with policy that explicitly requires
EA. IT investment compliance with the
EA.
Process exists to formally manage No Army officials did not provide
EA change. evidence of a process to formally
manage EA change.
EA is integral component of IT No Army officials did not provide
investment documentation indicating that EA
management process. is an integral component of the IT
investment management
process.
EA products are periodically No According to Army officials, EA
updated. products are not periodically
updated.
IT investments comply with EA. No Army officials did not provide
documentation indicating that IT
investments comply with the EA.
Organization head has approved No According to Army officials, the
current version of organization head has not
EA. approved the current version of the
EA.
Return on EA investment is measured No Army officials did not provide
and reported. documentation indicating that
return on EA investment is measured
and reported.
Compliance with EA is measured and No Army officials did not provide
reported. documentation indicating that
compliance with EA is measured and
reported.
Source: GAO analysis of agency provided data.
Department of Commerce
Table 19 shows Commerce's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 19: Department of Commerce Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial Commerce provided evidence
enterprise is that the Enterprise
Architecture
responsible for directing, Advisory Group and the
overseeing, and approving Enterprise Architecture
Review Board
EA. are responsible for
directing, overseeing, and
approving EA.
However, the documentation
did not indicate that either
group
represents the enterprise.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a Yes
framework,
methodology, and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-
be" environment, and sequencing
plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance,
information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Yes
to measure EA
progress, quality, compliance, and
return on
investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for
EA development.
EA products are under configuration Yes
management.
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Both "asis" and "to-be" Yes
environments are described
or will be described in terms of
business,
performance, information/data,
application/service,
and technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Yes
measured and
reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management Yes
processes undergo
independent verification and
validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" Yes
environments are described
in terms of business, performance,
information/data,
application/service, and
technology.
These descriptions address Yes
security.
Organization CIO has approved Yes
current version of
EA.
Committee or group representing Partial Commerce provided evidence that
the enterprise or the EA Review Board has
the investment review board has approved the current version of
approved current the EA. However, the
version of EA. documentation did not indicate
that the board represents the
enterprise.
Quality of EA products is Yes
measured and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved Partial Commerce provided evidence that includes
organization policy general references to
exists for IT
investment compliance IT investment compliance EA. However,
with EA. the documentation,
including a draft investment
policy, did not require compliance
IT
with the EA.
Process exists to
formally manage EA Yes
change.
EA is integral component Yes
of IT investment
management process.
EA products are Yes
periodically updated.
IT investments comply with EA. Partial Commerce provided evidence that
some but not all IT
investments comply with the EA.
Organization head has approved Yes
current version of
EA.
Return on EA investment is Yes
measured and reported.
Compliance with EA is measured Yes
and reported.
Source: GAO analysis of agency provided data.
Department of Defense - Business Enterprise Architecture
Table 20 shows the BEA's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 20: DOD Business Enterprise Architecture Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology, and automated tool.
EA plans call for describing "as-is" Yes
environment,
"to-be" environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, Partial According to BEA
performance, officials, the BEA will
address security as
information/data, evidenced by the fact
application/service, and that the GIG extends to
all DOD missions
technology to address security. areas, including the
business mission area.
However, documented
plans for how and when
this will be accomplished
for the business
mission were not
provided.
EA plans call for developing metrics EA Partial EA plans call for
to measure progress, quality, for Yes developing metrics to
compliance, and return on investment. measure EA progress,
Stage 3: Developing architecture quality, and compliance.
products Written and approved However, according to the
organization policy exists EA chief architect, EA plans
development. do not call for
developing metrics to
measure return on
investment.
EA products are under Partial According to BEA officials, EA products
configuration management. are under configuration
management, consistent with the
configuration management plan
they provided. However, no
documentation was provided to show
that BEA is complying with this plan.
(Continued From Previous Page) (Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
Both "asis" and "to-be" Yes
environments are described
or will be described in terms of
business,
performance, information/data,
application/service,
and technology.
These descriptions address or Partial According to BEA officials, the
will address security. BEA will address security as
evidenced by the fact that the
GIG extends to all DOD missions
areas, including the business
mission area. However,
documented
plans for how and when this will
be accomplished for the business
mission were not provided.
Progress against EA plans is Yes
measured and
reported.
Stage 4: Completing architecture
products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management Yes
processes undergo
independent verification and
validation.
EA products describe "as-is" No EA products describe the "to-be"
environment, "to-be" environment and sequencing
environment, and sequencing plan. plan. However, EA products do not
yet describe the "as-is"
environment.
Both "asis" and "to-be" environments are No The "to-be" environment is
described in terms of business, described in the requisite
performance, information/data, terms. However, the "as-is"
application/service, and technology. environment is not yet
described.
According to the chief
These descriptions address security. No architect, these descriptions
do not address
security.
Organization CIO has approved current Yes
version of
EA.
Committee or group representing the Yes
enterprise or
the investment review board has approved
current
version of EA.
Quality of EA products is measured and Yes
reported.
Stage 5: Leveraging the EA for managing
change
Written and approved organization policy Yes
exists for
IT investment compliance with EA.
Process exists to formally Partial According to BEA officials, a
manage EA change. process exists to formally manage
EA change. However, BEA officials
provided a configuration
management plan, which begins to
address EA change
management, but did not provide
evidence that EA change
management processes are fully
documented.
EA is integral component of IT Yes
investment
management process.
GAO basis for partially satisfied
or not satisfied
Stages and core elements Satisfied? determination
EA products are periodically Yes
updated.
IT investments comply with EA. Partial According to the chief architect,
IT investments comply with the
EA to some or little extent.
However, BEA officials provided
evidence that some investments
have been assessed for their
compliance with the BEA.
Organization head has approved Yes
current version of
EA.
Return on EA investment is measured No According to the chief architect,
and reported. return on EA investment is not
measured and reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Department of Defense - Global Information Grid
Table 21 shows the GIG's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 21: DOD Global Information Grid Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is responsible for directing,
overseeing,
and approving EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-
be" environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data, application/service, and
technology
to address security.
EA plans call for developing Partial A GIG official provided plans
metrics to measure and that call for developing metrics
to
report EA progress, quality, measure and report EA progress,
compliance, and return quality, and compliance.
on investment. However, plans that call for
developing metrics to measure
and
report return on EA investment
were not provided.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for
EA development.
EA products are under Partial According to a GIG official, EA
configuration management. products are under configuration
management and they provided
documentation of configuration
management procedures. However,
the documentation did not
describe detailed steps to
ensure the integrity and
consistency of
EA products.
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
(Continued From Previous Page) (Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Both "asis" and "to-be" Yes
environments are described
or will be described in terms of
business,
performance, information/data,
application/service,
and technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Partial According to a GIG official,
measured and progress against EA plans is
reported. measured and reported. However,
the evidence provided
consisted of (1) EA plans but no
reports of progress relative to
those plans and (2) contractor
progress reports that did not
relate
to the EA plans.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management Partial According to a GIG official, EA
processes undergo independent products and management
verification and validation. processes undergo independent
verification and validation.
However, the evidence provided
consisted of reports on a subset
of EA products and processes.
EA products describe "as-is" Partial According to a GIG official,
environment, "to-be" EA products describe the
"as-is"
environment, and sequencing plan. environment and "to-be"
environment. However, only
one of six
planned sequencing plans has
been completed.
Both "asis" and "to-be" environments Yes
are described
in terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of
EA.
Committee or group representing the Yes
enterprise or
the investment review board has
approved current
version of EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for
IT investment compliance with EA.
Process exists to formally manage EA Yes
change.
EA is integral component of IT Partial According to a GIG official, EA is
investment an integral component of the IT
management process. investment management process.
However, five of six
sequencing plans to guide IT
investments have not been
completed.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial According to a GIG official, IT
investments comply with the EA.
However, documentation indicated
that IT investments comply
with the Information Assurance
portion of the GIG.
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Organization head has approved Yes
current version of
EA.
Return on EA investment is No According to a GIG official, return
measured and reported. on EA investment is not
measured and reported.
Compliance with EA is measured Partial According to a GIG official,
and reported. compliance with the EA is measured
and reported. However,
documentation indicated that
compliance
is measured and reported relative
to a portion of the EA.
Source: GAO analysis of agency provided data.
Department of Education
Table 22 shows Education's satisfaction of framework elements in
version 1.1 of GAO's EAMMF.
Table 22: Department of Education Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial According to the chief
enterprise is architect, the EA executive
steering
responsible for directing, committee is responsible for
overseeing, and approving directing and overseeing the
EA
EA. and the investment review
board is responsible for
approving
the EA. However, the
investment review board
charter does not
discuss approving the EA and
no documentation of
investment
review board approval was
provided.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a Yes
framework, methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance,
information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Yes
to measure EA
progress, quality, compliance, and
return on investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under configuration Yes
management.
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing plan.
Both "asis" and "to-be" environments Yes
are described or
will be described in terms of
business, performance,
information/data,
application/service, and technology.
These descriptions address or will Yes
address security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes Partial Some EA products have
undergo independent verification and Yes Yes undergone independent
validation. EA products describe Yes Yes verification and validation.
"as-is" environment, However, according to the
"to-be"environment, and sequencing chief architect, EA
plan. Both "asis" and "to-be" management processes have
environments are described in terms not undergone independent
of business, performance, verification and validation.
information/data,
application/service, and technology.
These descriptions address security.
Organization CIO has approved
current version of EA.
Committee or group representing the Partial According to the chief
enterprise or the investment review Yes Yes architect, the investment
board has approved current version Yes Yes review board has approved
of EA. Quality of EA products is Yes Yes the current version of the
measured and reported. Stage 5: EA. However, no
Leveraging the EA for managing documentation of investment
change Written and approved review board approval was
organization policy exists for IT provided.
investment compliance with EA.
Process exists to formally manage EA
change. EA is integral component of
IT investment management process. EA
products are periodically updated.
IT investments comply with EA.
Organization head has approved No According to the chief architect,
current version of EA. the organization head has not
explicitly approved the current
version of the EA.
Return on EA investment is Partial Return on EA investment is measured
measured and reported. and reported. However,
this activity is limited to one EA
segment.
Compliance with EA is measured Yes
and reported.
Source: GAO analysis of agency provided data.
Department of Energy
Table 23 shows Energy's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 23: Department of Energy Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-
be" environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data, application/service, and
technology
to address security.
EA plans call for developing Partial EA plans call for developing
metrics to measure and metrics to measure and report EA
report EA progress, quality, progress, quality, and
compliance, and return on compliance. However, they do not
call for
investment. developing metrics to measure and
report EA return on
investment.
Stage 3: Developing EA products
Written and approved organization No According to the chief
policy exists for EA architect, a written and
approved
development. organization policy for EA
development does not exist.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "as-is" and "to-be" environments Yes
are described or
will be described in terms of business,
performance,
information/data, application/service,
and technology.
These descriptions address or will Yes
address security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization No According to the chief
policy exists for architect, a written and
approved
EA maintenance. organization policy for EA
maintenance does not exist.
EA products and management processes Yes
undergo
independent verification and
validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described
in terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of
EA.
Committee or group representing the Yes
enterprise or
the investment review board has
approved current
version of EA.
Quality of EA products is measured and Yes
reported.
Stage 5: Leveraging the EA for managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage Partial According to the chief
EA change. architect, a process exists to
formally
manage EA change. However, the
process does not include
specific steps to be followed.
EA is integral component of IT Yes
investment
management process.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial According to the chief architect,
IT investments comply with the EA to a "great extent."However, evidence
provided to GAO shows that some IT investments do not meet Energy's EA
compliance criteria.
Organization head has approved No According to the chief architect,
current version of EA. the organization head has not
approved the current version of
the EA.
Return on EA investment is measured No According to the chief architect,
and reported. return on EA investment is not
measured and reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Department of Health and Human Services
Table 24 shows HHS's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 24: Department of Health and Human Services Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial The HHS CIO council is
enterprise is responsible for directing,
overseeing, and
responsible for directing, approving EA. However, a
overseeing, and approving charter for the council is
under
EA. development.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment,
"to-be" environment, and sequencing
plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance,
information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Yes
to measure and
report EA progress, quality,
compliance, and return on
investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments Yes
are described
or will be described in terms of
business, performance,
information/data,
application/service, and technology.
These descriptions address or will Yes
address security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes undergo No EA products and management
processes have not undergone independent verification and validation.
independent verification and validation.
EA products describe "as-is" Partial EA products describe the "as-is"
environment, "to-be"environment, environment and
and sequencing plan. "to-be"environment. However, a
sequencing plan has been
developed for some EA segments
but has not been completed.
Both "asis" and "to-be" Both "asis" and "to-be"
environments are described in Partial environments are described.
However, the
terms of business, performance, chief architect indicated that
information/data, HHS plans to develop the
application/service, and descriptions in more detail.
technology.
These descriptions address Yes
security.
Organization CIO has approved No According to the chief architect,
current version of EA. the EA has not been approved by
the chief information officer.
Committee or group representing the No According to the chief architect,
enterprise or the current EA has not been
the investment review board has approved by a committee or group
approved current representing the enterprise.
version of EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage Yes
EA change.
EA is integral component of IT Partial HHS provided evidence that EA is
investment an integral component of the IT
management process. investment management process.
However, a sequencing plan to
guide IT investments has not been
completed.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
Organization head has approved No According to the chief architect,
current version of EA. the organization head has not
approved the current version of
the EA.
Return on EA investment is No According to the chief architect,
measured and reported. HHS plans to measure and report
return on EA investment. However,
return on EA investment is not
currently measured and reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Department of Homeland Security
Table 25 shows DHS's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 25: Department of Homeland Security Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data, application/service, and
technology
to address security.
EA plans call for developing metrics to Yes
measure EA
progress, quality, compliance, and return on
investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for EA
development.
EA products are under configuration Partial According to the chief
management. architect, EA products are
under
configuration management.
However, the configuration
management guidance documents
are in draft and not
approved.
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
Both "as-is" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and technology.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
These descriptions address or will Yes
address security.
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management No According to the chief architect, EA
processes undergo products and management
independent verification and processes undergo independent verification
validation. and validation.
However, the contractor that reviewed EA
products and
processes was not independent and the
reviews did not include
verification and validation.
EA products describe "as-is" Partial According to the chief
environment, "to-be" architect, EA products
describe the "as-is"
environment, and sequencing plan. environment, "to-be"
environment, and sequencing
plan.
However, the sequencing plan
is in draft and not
approved.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved Yes
current version of EA.
Committee or group representing the Yes
enterprise or the
investment review board has approved
current version
of EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage EA Partial According to the chief
change. EA is integral component of Yes Yes architect, DHS has a process
IT investment management process. EA to formally manage EA
products are periodically updated. change. However, the policy
that describes EA change
management is being revised
and is not approved.
IT investments comply with EA. Partial DHS has an IT investment management
process that requires investment compliance with the EA. However, the IT
investment management process does not include a methodology with detailed
compliance criteria.
Organization head has approved No According to the chief architect,
current the agency head has not
version of EA. approved the current version of
the EA.
Return on EA investment is measured No According to the chief architect,
and reported. return on EA investment is not
measured and reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Department of Housing and Urban Development
Table 26 shows HUD's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 26: Department of Housing and Urban Development Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology, and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-
be" environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data, application/service, and
technology to address security.
EA plans call for developing metrics to Yes
measure EA
progress, quality, compliance, and return on
investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for
EA development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments are Yes
described
or will be described in terms of business,
performance, information/data,
application/service,
and technology.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
These descriptions address or will Yes
address security.
Progress against EA plans is Yes
measured and
reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management No According to the chief architect, EA
processes undergo products and management
independent verification and processes undergo verification and
validation. validation. However, the
verification and independent.
validation were not
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" Yes
environments are described
in terms of business,
performance, information/data,
application/service, and
technology.
These descriptions address Partial HUD provided is addressing
security. evidence that it security in the
requisite descriptions. However,
according to HUD officials they
recognize the need to further develop
these security descriptions.
Organization CIO has Yes
approved current version of
EA.
Committee or group
representing the enterprise Yes
or
the investment review board
has approved current
version of EA.
Quality of EA products is Yes
measured and reported.
Stage 5: Leveraging the EA
for managing
change
Written and approved
organization policy exists Yes
for
IT investment compliance
with EA.
Process exists to formally Yes
manage EA change.
EA is integral component of Yes
IT investment
management process.
EA products are periodically Yes
updated.
IT investments comply with Yes
EA.
Organization head has Yes
approved current version of
EA.
Return on EA investment is Yes
measured and reported.
Compliance with EA is Yes
measured and reported.
Source: GAO analysis of agency provided data.
The Department of the Interior
Table 27 shows DOI's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 27: Department of the Interior Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment,
"to-be" environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data, application/service, and
technology to address security.
EA plans call for developing metrics to Yes
measure
and report EA progress, quality, compliance,
and
return on investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for
EA development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments are Yes
described
or will be described in terms of business,
performance,
information/data, application/service, and
technology.
These descriptions address or will address Yes
security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Yes
measured and
reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management processes No According to the chief
undergo architect, EA products and
management
independent verification and processes undergo independent
validation. verification and validation.
However, the evidence provided
did not show that EA product
and process quality were
assessed.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described
in terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of
EA.
Committee or group representing the Yes
enterprise or
the investment review board has
approved current
version of EA.
Quality of EA products is measured and Yes
reported.
Stage 5: Leveraging the EA for managing
change
Written and approved organization Yes
policy exists for
IT investment compliance with EA.
Process exists to formally manage EA Yes
change.
EA is integral component of IT Yes
investment
management process.
EA products are periodically updated. Yes
IT investments comply with EA. Yes
Organization head has approved current Yes
version
of EA.
Return on EA investment is measured and Yes
reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Department of Justice
Table 28 shows DOJ's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 28: Department of Justice Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
Stage 1
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-
be" environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/data, application/service, and
technology
to address security.
EA plans call for developing metrics to Yes
measure EA
progress, quality, compliance, and return on
investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for
EA development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments are Yes
described
or will be described in terms of business,
performance, information/data,
application/service,
and technology.
These descriptions address or will address Yes
security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management processes No According to the chief
undergo architect, EA products and
management
independent verification and processes have not undergone
validation. independent verification and
validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described
in terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of
EA.
Committee or group representing Partial According to the chief architect,
the enterprise or a committee or group
the investment review board has representing the enterprise has
approved current approved the current version of
version of EA. the EA. However, no documentation
of committee approval was
provided.
Quality of EA products is Partial According to the chief
measured and reported. architect, DOJ uses the OMB EA
assessment tool to measure and
report product quality. However,
no report documenting a quality
assessment or results was
provided.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage Yes
EA change.
EA is integral component of IT Yes
investment
management process.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial DOJ has an IT investment management
process that requires investment compliance with the EA. However, the IT
investment management process does not include a methodology to determine
compliance.
Organization head has approved current version of No According to the
chief architect, the organization head has not EA. approved the current
version of the EA.
Return on EA investment is measured and reported. No DOJ did not provide
documentation describing how return on EA investment is measured and
reported and no sample reports were provided.
Compliance with EA is measured and reported. Partial DOJ has begun to
measure and report IT investment compliance with the EA. However, no
compliance reports were provided.
Source: GAO analysis of agency provided data.
Department of Labor
Table 29 shows Labor's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 29: Department of Labor Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing enterprise in Yes
terms of
business, performance, information/data,
application/service, and technology.
EA plans call for business, performance, Yes
information/
data, application/service, and technology to
address
security.
EA plans call for developing metrics to Yes
measure and
report EA progress, quality, compliance, and
return on
investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for EA
development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments are Yes
described or
will be described in terms of business,
performance,
information/data, application/service, and
technology.
These descriptions address or will address Yes
security.
Progress against EA plans is measured and Yes
reported.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes No According to the EA program
undergo manager, EA products and
independent verification and management processes undergo
validation. independent verification and
validation. However, the
verification and validation
were
performed by a contractor
that had also performed other
EA
program activities and
therefore were not
independent.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved Yes
current version of EA.
Committee or group representing the Yes
enterprise or the
investment review board has approved
current version of
EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage EA Yes
change.
EA is integral component of IT Yes
investment management
process.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial According to the EA program
Organization head has approved Yes No manager, IT investments
current version of EA. Return on EA comply with the EA. However,
investment is measured and reported. the evidence provided did not
verify that IT investments
are not approved unless they
comply with the EA. According
to the EA program manager,
return on EA investment is
not measured and reported.
Compliance with EA is measured and reported. Partial According to the EA
program manager, compliance with the EA is measured and reported. However,
evidence of EA compliance reporting was not provided.
Source: GAO analysis of agency provided data.
Department of the Navy
Table 30 shows Navy's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 30: Department of the Navy Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing Partial According to the Navy CIO office,
the enterprise is Navy has a committee that is
responsible for directing, responsible for directing,
overseeing, and approving overseeing, and approving EA.
EA. However, the Navy did not provide
meeting minutes or other
documentation to verify that
committee meetings have occurred.
Program office responsible for Yes
EA development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a Partial The Navy uses a framework and
framework, methodology, and automated tool to develop their
automated tool. EA. However, our analysis of the
Department of Defense Architecture
Framework (DODAF), which was cited
as the Navy's EA methodology,
determined that the DODAF is not
an EA methodology.
EA plans call for describing "as-is" Partial EA plans call for describing
environment, "tobe" environment, and the "as-is" and "to-be"
sequencing plan. environments. However, EA
plans do not include
describing a sequencing plan.
EA plans call for describing EA plans do not include a
enterprise in terms of Partial "to-be" environment
description in
business, performance, terms of technology.
information/data,
application/service, and technology.
EA plans call for business, Yes
performance,
information/data,
application/service, and technology
to address security.
EA plans call for developing Partial EA plans call for developing
metrics to measure EA metrics to measure EA progress,
progress, quality, compliance, quality, and compliance.
and return on However, no evidence was
provided
investment. indicating plans to measure EA
return on investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for
EA development.
EA products are under configuration management. Partial Navy provided
evidence of their EA repository tools which Navy officials said they use
to track EA product configuration management. However, the evidence did
not describe detailed steps that would ensure the integrity and
consistency of EA products.
(Continued From Previous Page)
GAO basis for partially satisfied or not satisfied
Stages and core elements Satisfied? determination
EA products describe or will Partial EA plans call for describing
describe "as-is"environment, "to-be" the "as-is" and "to-be"
environment, and sequencing plan. environments. However, EA
plans do not include
describing a sequencing plan.
Both "asis" and "to-be" environments EA plans do not include a
are described Partial "to-be" environment
description in
or will be described in terms of terms of technology.
business, performance,
information/data,
application/service, and technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for
EA maintenance.
EA products and management No Navy indicated EA products and
processes undergo independent management processes are subject
verification and validation. to quality, integration, and
verification reviews. However,
these reviews are not performed by
independent entities.
EA products describe "as-is" No EA products describe the "as-is"
environment, "to-be"environment, environment. However, according to
and sequencing plan. the Department of the Navy CIO
office, their EA products will
include descriptions of the
"to-be" environment and sequencing
plan in the future.
Both "asis" and "to-be" No The "as-is" environment is
environments are described in described in business,
terms of business, performance, information/data,
information/data, application/service, and
application/service, and technology, but not performance
technology. terms. However, according to the
Navy CIO office, their "to-be"
environment will be described in
the future.
These descriptions address No Yes According to the Navy CIO office,
security. Organization CIO has EA descriptions will explicitly
approved current version of EA. address security in the future.
Committee or group representing No According to the Navy CIO office, EA
the enterprise or approvals are rendered by
the investment review board has either the Assistant Secretary for
approved current Research and Development or
version of EA. a duly appointed representative, in
addition to the Navy CIO.
However, a committee or group
representing the enterprise does
not approve the EA.
Quality of EA products is Partial According to the Navy CIO office,
measured and reported. quality of EA products is measured
and reported. However, Navy
officials provided evidence that
quality of EA products is measured
and reported for the Marine Corps
portion of the architecture, but
did not provide documentation that
quality of Navy architecture
products are measured and reported.
Stage 5: Leveraging the EA for
managing change
Written and approved No Navy policy does not explicitly
organization policy exists for require IT investment compliance
IT investment compliance with with the EA.
EA.
Process exists to formally manage EA change. No Navy officials did not
provide evidence of a process to formally manage EA change.
(Continued From Previous Page)
GAO basis for partially
Stages and core elements Satisfied? satisfied or not satisfied
determination
EA is integral component of IT According to the Navy CIO
investment Partial office, the department has
begun to
management process. integrate EA into its IT
investment management process.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial According to the Navy CIO
office, IT investments comply
with the EA. However,
documentation of IT investment
compliance with the EA was not
provided.
Organization head has approved No According to the Navy CIO office,
current version of EA. the organization head has not
approved the EA.
Return on EA investment is No According to the Navy CIO office,
measured and reported. the department does not currently
measure and report return on EA
investment.
Compliance with EA is measured Partial Navy provided documentation
and reported. indicating that compliance with the
EA is measured. However, sample
measurement reports were not
provided.
Source: GAO analysis of agency provided data.
Note: This analysis primarily focuses on the FORCEnet architecture because
FORCEnet was identified by the Department of the Navy as the overall
Department of Navy enterprise architecture. Additional information from
the Marine Corps Integrated Architecture Picture (MCIAP) was incorporated
where appropriate.
Department of State
Table 31 shows State's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 31: Joint Enterprise Architecture Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial According to the chief
enterprise is architect, State and USAID
are
responsible for directing, developing a governance
overseeing, and approving EA. plan that will describe
management
processes for directing,
overseeing, and approving
their EA.
However, this plan is not
approved.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Partial According to the chief
methodology, and automated tool. EA Yes Yes architect, the EA is being
plans call for describing "as-is" Yes developed using a
environment, "to-be"environment, and framework, methodology, and
sequencing plan. EA plans call for automated tool. However,
describing enterprise in terms of the methodology does not
business, performance, describe specific steps for
information/data, maintenance.
application/service, and technology.
EA plans call for business,
performance, information/data,
application/service, and technology
to address security.
EA plans call for developing Partial State and USAID have plans to
metrics to measure EA measure and report EA
progress, quality, compliance, progress, quality, and
and return on investment. compliance. However, no plans
for
developing metrics to measure
EA return on investment were
provided.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under Partial A draft configuration
configuration management. EA Yes management plan for EA products
products describe or will has been developed. However,
describe "as-is" environment, this plan has not been
"to-be" environment, and approved.
sequencing plan.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Both "asis" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and
technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Partial State has limited reporting of
measured and reported. EA progress. However,
progress is not measured and
reported relative to an EA
program plan.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management No According to the chief
processes undergo independent architect, EA products and
verification and validation. management processes have not
undergone independent
verification and validation.
EA products describe "as-is" No EA products describe the "as-is"
environment, and "to-be" environments as
"to-be" environment, and sequencing well as high-level transition
plan. activities. However, a
sequencing
plan for transitioning between
the "as-is" and "to-be"
environments is currently under
development.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version
of EA.
Committee or group representing the No According to the chief architect,
enterprise or the a committee or group
investment review board has representing the enterprise has
approved current version not approved the current
of EA. version of the EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage No According to the chief architect,
EA change. a process to formally manage EA
change does not currently exist.
EA is integral component of IT Partial According to the chief architect,
investment management EA is an integral component
process. of the IT investment management
process. However, the
sequencing plan to guide IT
investments is currently under
development.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
Organization head has approved No According to the chief architect,
current version of EA. the organization head has not
approved the current version of
the EA.
Return on EA investment is measured and reported. No According to the
chief architect, return on EA investment is not measured and reported.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Compliance with EA is measured Partial According to the chief
and reported. architect, compliance with the
EA is measured and reported and
officials provided a description
of how compliance is to be
measured. However, no
documentation demonstrating
compliance reporting was
provided.
Source: GAO analysis of agency provided data.
Note: Department of State officials asked that we evaluate their agency's
enterprise architecture based on efforts to develop the Joint Enterprise
Architecture, which is being developed by State and the U.S. Agency for
International Development.
Department of Transportation
Table 32 shows Transportation's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 32: Department of Transportation Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Partial Program officials stated
methodology, and automated tool. EA Yes Yes that the EA is being
plans call for describing "as-is" Yes developed using a framework,
environment, "to-be"environment, and methodology, and automated
sequencing plan. EA plans call for tool. However, the
describing enterprise in terms of methodology and other
business, performance, documentation did not
information/data, include steps for EA
application/service, and technology. maintenance.
EA plans call for business,
performance, information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Partial EA plans call for developing
to measure EA progress, quality, Yes metrics to measure EA
compliance, and return on investment. progress and quality.
Stage 3: Developing EA products However, EA plans do not
Written and approved organization call for developing metrics
policy exists for EA development. to measure compliance and
return on investment.
EA products are under configuration Partial A configuration management
management. plan for EA products is being
defined. However, this plan
has not been approved.
EA products describe or will Yes
describe "as-is" environment,
"to-be" environment, and sequencing
plan.
Both "asis" and "to-be" environments Yes
are described or will
be described in terms of business,
performance,
information/data,
application/service, and technology.
These descriptions address or will Yes
address security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes No According to the chief
undergo architect, EA products and
independent verification and validation. management processes do not
undergo independent
verification and validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments are Yes
described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. No The business, performance,
information/data, application/service, and technology descriptions do not
address security.
Organization CIO has approved No According to the chief architect,
current version of EA. the organization CIO has not
approved the EA.
Committee or group representing the No According to the chief architect,
enterprise or the a committee or group
investment review board has representing the enterprise or the
approved current version of investment review board
EA. has not approved the current
version of the EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage Partial According to the chief
EA change. architect, a process exists to
formally
manage EA change. However, the
department provided
evidence that a structure is in
place to manage EA change,
but a description of this
process for formally managing EA
change was not provided.
EA is integral component of IT Yes
investment management
process.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
Organization head has approved No According to the chief
current version of EA. architect, the organization head
has not approved the current
version of the EA.
Return on EA investment is measured and reported. No According to the
chief architect, return on EA investment is not measured and reported.
Compliance with EA is measured and reported. No According to the chief
architect, compliance with the EA is measured and reported. However,
department officials did not provide evidence that compliance with the EA
is measured and reported.
Source: GAO analysis of agency provided data.
Department of the Treasury
Table 33 shows the Treasury's satisfaction of
framework elements in version 1.1 of GAO's EAMMF.
Table 33: Department of the Treasury Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial Two committees are responsible
enterprise is for directing and
responsible for directing, overseeing the EA. However,
overseeing, and approving EA. the committee charters do
not indicate that either
committee is responsible for
approving the EA or represent
the enterprise (i.e., include
executive-level representation
from each line of business).
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a Yes
framework, methodology, and
automated tool.
EA plans call for describing Yes
"as-is" environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of business,
performance, information/data,
application/service, and
technology.
EA plans call for business, Yes
performance, information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Partial According to the EA program
to measure and report manager, EA plans call for
EA progress, quality, compliance, developing metrics to
measure and report EA
progress,
and return on investment. quality, and compliance.
However, evidence provided
did
not include plans for EA
compliance or return on
investment metrics.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for
EA development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is" environment,
"to-be" environment, and sequencing
plan.
Both "asis" and "to-be" environments Yes
are described or will
be described in terms of business,
performance,
information/data, application/service,
and technology.
These descriptions address or will Yes
address security.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Progress against EA plans is Partial According to the EA program
measured and reported. manager, progress is
measured and reported against
plans, including an EA
program management plan.
However, the EA program
management plan is in draft.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes No According to the EA program
undergo manager, EA products and
independent verification and validation. management processes have not
undergone independent
verification and validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "as-is" and "to-be" environments Yes
are described in terms
of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of EA.
Committee or group representing the A committee or group
enterprise or the No representing the enterprise
has not
investment review board has approved approved the current version
current version of EA. of the EA.
Quality of EA products is Partial Yes According to the EA program
measured and reported. Stage 5: Yes Yes Yes manager, the quality of EA
Leveraging the EA for managing products is measured and
change Written and approved reported. Further, evidence
organization policy exists for showed that a quality
IT investment compliance with evaluation was performed.
EA. Process exists to formally However, evidence of the
manage EA change. EA is evaluation results was not
integral component of IT provided.
investment management process.
EA products are periodically
updated.
IT investments comply with EA. Partial According to the EA program
manager, IT investments comply
with the EA. However, the
documents provided did not
demonstrate that the IT
investment management process
requires IT investments to
comply with the EA.
Organization head has approved No According to the EA program
current version of EA. manager, the organization head
has delegated approval of the
EA to the CIO, who has
approved the current version.
However, the evidence provided
did not explicitly show such a
delegation.
Return on EA investment is measured and No According to the EA program
reported. manager, return on EA
investment is not measured and
reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Department of Veterans Affairs
Table 34 shows VA's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 34: Department of Veterans Affairs Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial A committee representing
enterprise is the enterprise is
responsible for
responsible for directing, overseeing, directing and overseeing
and approving the EA. However, the
committee
EA. charter does not indicate
that the committee is
responsible for
approving the EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing enterprise Yes
in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance,
information/data, application/service,
and technology
to address security.
EA plans call for developing metrics to Partial Metrics to measure and
measure and report EA progress, Yes report EA progress and
quality, compliance, and return on quality have been
investment. Written and approved developed. However, plans
organization policy exists for EA do not include developing
development. metrics to measure and
report EA compliance and
return on investment.
EA products are under configuration management. Partial According to the
chief architect, EA products are under configuration management, which is
accomplished through the version control features of the EA repository.
However, configuration management procedures did not define specific steps
that would ensure the integrity and consistency of EA products.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
Both "asis" and "to-be" Yes
environments are described or
will be described in terms
business, performance,
information/data,
application/service, and
technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved policy exists Yes
for EA maintenance.
EA products and management processes No According to the chief
undergo architect, EA version 4.0 has
independent verification and undergone independent
validation. verification and validation.
However,
no documentation to support this
statement was provided.
Further, plans for independent
verification and validation of
EA management processes have not
been implemented.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Partial According to VA officials,
their security architecture addresses all the requisite descriptions.
However, evidence provided to support this statement shows that the
security architecture does not address all of the requisite descriptions.
Organization CIO has approved No According to the chief architect,
current version of EA. the CIO delegated EA approval
authority to the chief architect.
However, no evidence to support
this delegation was provided.
Committee or group representing the No No evidence that a committee or
enterprise or group representing the
the investment review board has enterprise has approved the
approved current current version of the EA was
version of EA. provided.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally Partial According to the chief architect,
manage EA change. the EA repository is used to
manage EA change. VA provided
documentation describing
the repository and its contents.
However, evidence provided
did not demonstrate that the
repository is used to manage EA
change or include detailed steps
for managing EA change.
EA is integral component of IT Yes
investment
management process.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Organization head has approved No According to the chief
current version of EA. architect, the organization head
delegated EA approval to the
CIO, who delegated this
authority to the chief
architect. However, no evidence
to support these delegations was
provided.
Return on EA investment is No According to the chief
measured and reported. architect, return on EA
investment is measured and
reported. However, no evidence
that the return on EA investment
is measured and reported was
provided.
Compliance with EA is measured No According to the chief
and reported. architect, compliance with the
EA is measured and reported.
However, no evidence that
compliance with the EA is
measured and reported was
provided.
Source: GAO analysis of agency provided data.
Environmental Protection Agency
Table 35 shows EPA's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 35: Environmental Protection Agency Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage
1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing,
and approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Partial EA is being developed using
methodology, a framework, methodology,
and
and automated tool. automated tool. However, the
methodology does not include
steps for maintaining the
architecture and has not
been
approved.
EA plans call for describing "as-is" Yes
environment,
"to-be" environment, and sequencing
plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance,
information/data,
application/service, and technology
to
address security.
EA plans call for developing metrics Yes
to measure EA
progress, quality, compliance, and
return on
investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments Yes
are described or
will be described in terms of
business, performance,
information/data,
application/service, and technology.
(Continued From Previous Page)
GAO basis for partially satisfied
Stages and core elements Satisfied? or not satisfied determination
These descriptions address or Yes
will address security.
Progress against EA plans is Partial EPA has limited reporting of EA
measured and reported. progress. However, progress
is not measured and reported
relative to an EA program plan.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes No According to the chief
undergo architect, EA products and
independent verification and validation. management processes have not
undergone independent
verification and validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments are Yes
described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of EA.
Committee or group representing the Yes
enterprise or the
investment review board has approved
current version
of EA.
Quality of EA products is measured and Yes
reported.
Stage 5: Leveraging the EA for managing
change
Written and approved organization policy Yes
exists for IT
investment compliance with EA.
Process exists to formally Partial Yes According to the chief
manage EA change. EA is Yes architect, a process exists to
integral component of IT formally manage EA change.
investment management process. However, evidence that the
EA products are periodically process has been implemented
updated. was not provided and the
process is not approved.
IT investments comply with EA. Partial An IT investment management
process exists and the process
considers investment compliance
with the EA. However, evidence
that the process has been
implemented was not provided.
No According to the chief
Organization head has approved architect, the organization
current version of EA. head has not approved the
current version of the EA.
Return on EA investment is No According to the chief
measured and reported. architect, return on EA
investment is not measured and
reported.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Compliance with EA is measured Partial According to the chief
and reported. architect, compliance with EA is
measured and reported and
documentation describes how
compliance is to be measured.
However, reports documenting
measurement were not provided.
Source: GAO analysis of agency provided data.
General Services Table 36 shows GSA's satisfaction of framework elements
in version 1.1 of GAO's EAMMF.
Administration
Table 36: General Services Administration Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving
EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a Partial According to the chief
framework, methodology, technology officer (CTO), the EA
is being
and automated tool. developed using a framework,
methodology, and automated
tool. However, the methodology
does not include specific steps
to maintain the architecture.
EA plans call for describing Yes
"as-is" environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and
technology.
EA plans call for business, Partial Draft EA plans call for
performance, information/data, business, performance,
application/service, and information/data,
technology to address security. application/service, and
technology descriptions to
address security. However, these
plans are not approved.
EA plans call for developing Partial According to the CTO, EA plans
metrics to measure EA call for developing metrics.
progress, quality, compliance, However, these plans do not
and return on specifically address EA
progress,
investment. quality, compliance, and return
on investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for
EA development.
EA products are under Partial According to the CTO, EA
configuration management. products are under configuration
management. However, the
configuration management plan is
being defined and has not been
approved.
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Both "asis" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and
technology.
These descriptions address or will address security. Partial Draft EA
plans call for business, performance, information/data,
application/service, and technology descriptions to address security.
However, these plans are not approved.
Progress against EA plans is No According to the CTO, progress
measured and reported. against EA plans is not
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management No According to the CTO, EA products
processes undergo independent and management processes have not
verification and validation. undergone independent verification
and validation.
EA products describe "as-is" Partial EA products describe both the
environment, "to-be" "asis" environment and the "to
environment, and sequencing plan. be" environment. However, the
sequencing plan has not been
approved.
Both "asis" and "to-be" Both the "as-is" and
environments are described Partial "to-be"environments are
described in
in terms of business, terms of business,
performance, information/data, application/service, and
technology.
application/service, and However, according to the CTO,
technology. descriptions of performance
and information/data have not
been completed.
These descriptions address No According to the CTO, business,
security. performance,
information/data, application/service,
and technology
descriptions do not address security.
Organization CIO has approved Yes
current version of
EA.
Committee or group representing Yes
the enterprise or
the investment review board has
approved current
version of EA.
Quality of EA products is Yes
measured and reported.
Stage 5: Leveraging the EA for
managing change
Written and approved
organization policy exists for Yes
IT
investment compliance with EA.
Process exists to formally No According to the CTO, no process to
manage EA change. formally manage EA change exists.
EA is integral component of IT Partial According to the CTO, EA is an
investment integral component of the IT
management process. investment management process.
However, a sequencing plan
to guide IT investments has not
been approved.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
Organization head has approved Yes
current version of
EA.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Return on EA investment is No According to the CTO, return on
measured and reported. EA investment is not measured
and reported.
Compliance with EA is measured No According to the CTO, compliance
and reported. with the EA is not measured and
reported.
Source: GAO analysis of agency provided data.
National Aeronautics and Space Administration
Table 37 shows NASA's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 37: National Aeronautics and Space Administration Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Partial According to the chief technology
officer, NASA has adequate funding and tools. However, the chief
technology officer also stated that the EA program has somewhat inadequate
EA personnel resources.
Committee or group representing the Partial According to the chief
enterprise is technology officer, the
operations
responsible for directing, management council, CIO
overseeing, and approving council, and strategic
management
EA. council are responsible for
directing, overseeing, and
approving
the EA. However, the
charters for these groups
are awaiting
executive approval and the
draft charters provided did
not
specifically mention EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Partial The EA is being developed
methodology, and automated tool. EA Yes Yes using a framework and
plans call for describing "as-is" Yes Yes automated tool. However,
environment, "to-be"environment, and Yes Yes documentation did not
sequencing plan. EA plans call for indicate the EA is being
describing enterprise in terms of developed using a
business, performance, methodology that includes
information/data, specific steps required to
application/service, and technology. develop, maintain, and
EA plans call for business, validate the EA.
performance, information/data,
application/service, and technology
to address security. EA plans call
for developing metrics to measure EA
progress, quality, compliance, and
return on investment. Stage 3:
Developing EA products Written and
approved organization policy exists
for EA development. EA products are
under configuration management.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
EA products describe or will Yes
describe "as-is"
environment, "to-be" environment,
and sequencing
plan.
Both "asis" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and
technology.
These descriptions address or will Yes
address security.
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management Yes
processes undergo
independent verification and
validation.
EA products describe "as-is" No EA products describe the
environment, "to-be"environment, "as-is" environment.
and sequencing plan. However, EA products do not
fully describe the "to-be"
environment and do not
include a sequencing plan.
Both "asis" and "to-be" Partial According to NASA's chief
environments are described technology officer, the "as-is"
in terms of business, environment is described in terms
performance, information/data, of business,
application/service, and application/service, and
technology. technology. However, the "as-is"
environment is not described in
terms of performance and
information/data and EA products
do not describe the "to-be"
environment.
These descriptions address Partial The "as-is" description
security. addresses security. However, the
"to-be"
description does not address
security.
Organization CIO has approved Yes
current version of EA.
Committee or group representing Yes
the enterprise or
the investment review board has
approved current
version of EA.
Quality of EA products is Yes
measured and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage Yes
EA change.
EA is integral component of IT Partial NASA's IT investment management
investment process guidance
management process. recognizes EA as an integral
component. However, a
sequencing plan to guide IT
investments does not exist.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial NASA evidence showed that some
investments were certified as compliant with the EA. However, evidence of
the certification criteria used to assess IT investment compliance was not
provided.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Organization head has approved Yes
current version of
EA.
Return on EA investment is measured No According to the chief technology
and reported. officer, return on EA
investment is not measured and
reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
National Science Foundation
Table 38 shows NSF's satisfaction of framework elements
in version 1.1 of GAO's EAMMF.
Table 38: National Science Foundation Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing Partial A committee or group representing
the enterprise is the enterprise is
responsible for directing, responsible for directing and
overseeing, and approving overseeing the EA. However, no
EA. committee or group has
responsibility for approving the
EA.
Program office responsible for Yes
EA development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Partial According to the chief
methodology, architect, the EA is being
developed
and automated tool. using a framework,
methodology, and automated
tool.
However, the methodology
does not document steps for
EA
development.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance, information/
data, application/service, and
technology to address
security.
EA plans call for developing metrics Yes
to measure EA
progress, quality, compliance, and
return on investment.
Stage 3: Developing architecture
products
Written and approved policy exists Partial According to the chief
for EA development. architect, a policy exists
for EA
development. However, the
policy is not approved.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments Yes
are described or
will be described in terms of
business, performance,
information/data,
application/service, and technology.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
These descriptions address or will Yes
address security.
Progress against EA plans is Partial According to the chief architect,
measured and reported. progress against EA plans is
measured and reported. However,
the plan against which
progress is measured is not
approved and no documentation
of progress reporting was
provided.
Stage 4: Completing architecture
products
Written and approved Partial According to the chief architect,
organization policy exists for a policy exists for EA
EA maintenance. maintenance. However, the policy
is not approved.
EA products and management No According to the chief architect,
processes undergo independent EA products and management
verification and validation. processes have not undergone
independent verification and
validation.
EA products describe "as-is" Partial According to the chief architect,
environment, "to-be"environment, EA products describe the "asis"
and sequencing plan. environment, "to-be" environment,
and sequencing plan. However, the
"as-is"environment is not fully
described in terms of performance
and the "to-be"environment is not
fully described in terms of
performance and information/data.
Both "asis" and "to-be" Partial According to the chief architect,
environments are described in the "as-is" is not fully
terms of business, performance, described in terms of performance
information/data, and the "to-be"environment is not
application/service, and fully described in terms of
technology. performance and information/data.
These descriptions address Partial The business, information/data,
security. application/service, and
technology descriptions address
security. However, the
performance description does not
address security.
Organization CIO has approved Yes
current version of EA.
Committee or group representing the No According to the chief architect,
enterprise or the no committee or group
investment review board has representing the enterprise has
approved current version approved the current version
of EA. of the EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing change
Written and approved organization Partial The chief architect provided an
policy exists for IT organization policy for IT
investment compliance with EA. investment compliance with the
EA. However, the policy is not
approved and does not explicitly
require IT investment
compliance with the EA.
Process exists to formally manage Yes
EA change.
EA is integral component of IT Yes
investment management
process.
EA products are periodically Yes
updated.
IT investments comply with EA. Partial According to the chief architect,
IT investments comply with the EA and NSF provided evidence of procedures
intended to determine compliance. However, evidence supporting that IT
investments are required to be compliant with the EA before they are
approved was not provided.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Organization head has approved No NSF did not provide evidence
current version of EA. that the organization head has
approved the current version of
the EA.
Return on EA investment is No According to the chief
measured and reported. architect, return on EA
investment is measured and
reported. However, evidence to
support this statement was not
provided.
Compliance with EA is measured Partial According to the chief
and reported. architect, compliance with the
EA is measured and reported, and
NSF provided evidence describing
how compliance is measured and
reported. However, evidence of
compliance was not provided.
Source: GAO analysis of agency provided data.
Nuclear Regulatory Commission
Table 39 shows NRC's satisfaction of framework elements
in version 1.1 of GAO's EAMMF.
Table 39: Nuclear Regulatory Commission Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Partial According to the chief
enterprise is responsible for Yes architect, a committee or
directing, overseeing, and group representing the
approving EA. Program office enterprise is responsible for
responsible for EA development and directing, overseeing, and
maintenance exists. approving the EA. However,
the charter for this
committee does not clearly
state that it is responsible
for directing, overseeing,
and approving the EA and the
charter is not approved.
Chief architect exists. EA being Partial A chief architect has been
developed using a framework, Yes Yes designated. However, the
methodology, and automated tool. EA Yes Yes chief architect's roles and
plans call for describing "as-is" responsibilities do not
environment, "to-be"environment, include functioning as the EA
and sequencing plan. EA plans call program manager.
for describing enterprise in terms
of business, performance,
information/data,
application/service, and
technology. EA plans call for
business, performance,
information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Partial EA plans call for
to measure EA progress, quality, developing metrics to
compliance, and return on investment. measure EA progress,
Stage 3: Developing EA products quality, and compliance.
However, EA plans do not
call for developing metrics
to measure return on
investment.
Written and approved organization Partial A draft policy exists for
policy exists for EA development. EA Yes Yes EA development. However,
products are under configuration Yes this policy has not been
management. EA products describe or approved.
will describe "as-is" environment,
"to-be" environment, and sequencing
plan. Both "asis" and "to-be"
environments are described or will be
described in terms of business,
performance, information/data,
application/service, and technology.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
These descriptions address or Yes
will address security.
Progress against EA plans is Yes
measured and reported.
Stage 4: Completing EA products
Written and approved A draft policy exists for EA
organization policy exists for Partial maintenance. However, this
EA policy
maintenance. has not been approved.
EA products and management processes No EA products and management
undergo processes have not
independent verification and undergone independent
validation. verification and validation.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of EA.
Committee or group representing the No According to the chief
enterprise or the architect, a committee or group
investment review board has approved representing the enterprise or
current version of the investment review board
EA. has not approved the current
version of the EA.
Quality of EA products is Partial NRC provided information on how
measured and reported. quality of EA products is to
be measured and reported.
However, documentation
demonstrating that quality is
actually being measured and
reported was not provided.
Stage 5: Leveraging the EA for
managing change
Written and approved Partial A draft policy exists for IT
organization policy exists for Partial Yes investment compliance with EA.
IT investment compliance with Yes However, this policy has not
EA. Process exists to formally been approved. According to the
manage EA change. EA is chief architect, a process
integral component of IT exists to formally manage EA
investment management process. change. However, evidence that
EA products are periodically the process has been
updated. implemented was provided for
some EA products but not
others.
IT investments comply with EA. Partial According to the chief architect,
IT investments comply with the EA
and evidence demonstrates that EA
is considered during the IT
investment management process.
However, documentation provided
did not demonstrate IT investment
compliance.
Organization head has approved No According to the chief architect,
current version of EA. the organization head has not
approved the current version of
the EA.
Return on EA investment is No According to NRC officials, return
measured and reported. on EA investment is not measured
and reported.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Compliance with EA is measured Partial According to the chief
and reported. architect, compliance with EA is
measured and reported. However,
documentation demonstrating that
compliance is actually being
measured and reported was not
provided.
Source: GAO analysis based on agency provided data.
Office of Personnel Management
Table 40 shows OPM's satisfaction of framework
elements in version 1.1 of GAO's EAMMF.
Table 40: Office of Personnel Management Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing,
and approving EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology,
and automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance, information/
data, application/service, and
technology to address
security.
EA plans call for developing metrics Yes
to measure EA
progress, quality, compliance, and
return on investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing
plan.
Both "asis" and "to-be" environments Yes
are described or
will be described in terms of
business, performance,
information/data, application/service,
and technology.
These descriptions address or will Yes
address security.
Progress against EA plans is measured Partial OPM has limited reporting
and reported. of EA progress. However,
progress is not measured
and reported relative to an
EA program plan.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management processes No According to the chief
undergo architect, EA products and
independent verification and management processes have not
undergone independent
validation. verification and validation.
The chief architect stated
that the
cost of independent
verification and validation is
not justified.
EA products describe "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments Yes
are described in
terms of business, performance,
information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved current Yes
version of EA.
Committee or group representing the Yes
enterprise or the
investment review board has approved
current version of
EA.
Quality of EA products is measured and Yes
reported.
Stage 5: Leveraging the EA for managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage EA Yes
change.
EA is integral component of IT Yes
investment management
process.
EA products are periodically updated. Yes
IT investments comply with EA. Yes
Organization head has approved current Yes
version of EA.
Return on EA investment is measured and Yes
reported.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Small Business Administration
Table 41 shows SBA's satisfaction of framework elements in
version 1.1 of GAO's EAMMF.
Table 41: Small Business Administration Satisfaction of EAMMF
GAO basis for partial or not
satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Partial According to SBA officials,
the agency has EA
program activities that do
not have adequate
resources.
Committee or group representing the Yes
enterprise is responsibl
for directing, overseeing, and
approving EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Yes
methodology, and
automated tool.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of business,
performance, information/data,
application/service, and
technology.
EA plans call for business, Yes
performance, information/data,
application/service, and technology to
address security.
EA plans call for developing Partial EA plans call for developing
metrics to measure and report EA metrics to measure and
progress, quality, compliance, report EA progress, quality, and
and return on investment. compliance.
However, documentation did not
include plans for
developing metrics to measure and
report EA return
on investment.
Stage 3: Developing EA products
Written and approved organization policy Yes
exists for EA development.
EA products are under configuration No Configuration management
management. procedures provided to GAO do
not describe steps to ensure
the integrity and consistency
of EA products.
EA products describe or will describe Yes
"as-is" environment, "to-be"
environment, and sequencing plan.
Both "asis" and "to-be" environments are Yes
described or will be described in terms
of business, performance,
information/data, application/service,
and technology.
(Continued From Previous Page)
GAO basis for partial or not
Stages and core elements Satisfied? satisfied determination
These descriptions address or will Yes
address security.
Progress against EA plans is No SBA officials did not provide
measured and reported. evidence that the
agency is measuring and
reporting progress against
EA plans.
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA maintenance.
EA products and management processes Yes
undergo independent verification and
validation.
EA products describe "as-is" nt, Yes
environment, "to-be" environmeand
sequencing plan.
Both "asis" and "to-be" environments Yes
are described in terms of business,
performance, information/data,
application/service, and technology.
These descriptions address security. Yes
Organization CIO has approved Yes
current version of EA.
Committee or group representing the No According to SBA officials, a
enterprise or the investment review committee or group
board has approved current version representing the enterprise
of EA. approved the current version
of the EA. However,
documentation indicated
approval of the 2003 EA
program policies and
procedures, not the current
version of the EA.
Quality of EA products is measured No According to SBA officials, the
and reported. Stage 5: Leveraging quality of EA products is measured
the EA for managing change and reported. However, SBA did not
provide documentation that supports
this statement.
Written and approved organization Yes
policy exists for IT investment
compliance with EA.
Process exists to formally manage Yes
EA change.
EA is integral component of IT Yes
investment management process.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
Organization head has approved No According to SBA officials, the
current version of EA. organization head has approved the
current version of the EA. However,
documentation indicated approval of
the 2003 EA program policies and
procedures, not the current version
of the EA.
Return on EA investment is No According to SBA officials, return
measured and reported. on EA investment is not measured
and reported.
(Continued From Previous Page)
GAO basis for partial or not
Stages and core elements Satisfied? satisfied determination
Compliance with EA is measured No According to SBA officials,
and reported. compliance with EA is measured
and reported. However, these
officials did not provide
documentation that supports this
statement.
Source: GAO analysis of agency provided data.
Social Security Administration
Table 42 shows SSA's satisfaction of framework elements in
version 1.1 of GAO's EAMMF.
Table 42: Social Security Administration Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in
stage 1.
Stage 2: Building the EA management
foundation
Adequate resources exist. Yes
Committee or group representing the Yes
enterprise is
responsible for directing, overseeing, and
approving EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a framework, Partial According to the chief
methodology, architect, the EA is being
developed
and automated tool. using a framework,
methodology, and automated
tool.
However, the methodology
does not include steps for
developing, maintaining, and
validating the agency's EA.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance, information/data,
application/service, and technology
to address security.
EA plans call for developing metrics Yes
to measure EA
progress, quality, compliance, and
return on investment.
Stage 3: Developing EA products
Written and approved organization Yes
policy exists for EA
development.
EA products are under configuration Yes
management.
EA products describe or will describe Yes
"as-is"
environment, "to-be" environment, and
sequencing plan.
Both "asis" and "to-be" environments Yes
are described or
will be described in terms of
business, performance,
information/data,
application/service, and technology.
These descriptions address or will Yes
address security.
Progress against EA plans is measured Yes
and reported.
(Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
Stage 4: Completing EA products
Written and approved organization Yes
policy exists for EA
maintenance.
EA products and management Yes
processes undergo
independent verification and
validation.
EA products describe "as-is" Yes
environment,
"to-be" environment, and sequencing
plan.
Both "asis" and "to-be" Yes
environments are described in
terms of business, performance,
information/data,
application/service, and
technology.
These descriptions address Yes
security.
Organization CIO has approved Yes
current version of EA.
Committee or group representing the Yes
enterprise or the
investment review board has
approved current version
of EA.
Quality of EA products is measured Yes
and reported.
Stage 5: Leveraging the EA for
managing
change
Written and approved organization Yes
policy exists for IT
investment compliance with EA.
Process exists to formally manage Yes
EA change.
EA is integral component of IT Yes
investment management
process.
EA products are periodically Yes
updated.
IT investments comply with EA. Yes
Organization head has approved No SSA officials stated that CIO
current version of EA. approval of the current version of
the EA constitutes approval by the
organization head. However, they
did not provide documentation
delegating EA approval authority
from the organization head to the
CIO.
Return on EA investment is Partial According to the chief architect,
measured and reported. return on EA investment is measured
and reported and a description of
how return on investment is
measured and reported was provided.
However, sample measures and
reports were not provided.
Compliance with EA is measured Partial According to the chief architect,
and reported. compliance with the EA is measured
and reported and a description of
how compliance is measured and
reported was provided. However,
sample measures and reports were
not provided.
Source: GAO analysis of agency provided data.
U.S. Agency for International Development
Table 43 shows USAID's satisfaction of framework elements
in version 1.1 of GAO's EAMMF.
Table 43: U. S. Agency for International Development Satisfaction of EAMMF
GAO basis for partially satisfied or
not satisfied
Stages and core elements Satisfied? determination
Stage 1: Creating EA
awareness
Agency is aware of EA. n/a No core element exists in stage 1.
Stage 2: Building the EA management foundation
Adequate resources exist. No According to the chief architect,
USAID has "somewhat
inadequate" EA resources.
Committee or group representing the Yes
enterprise is
responsible for directing,
overseeing, and approving EA.
Program office responsible for EA Yes
development and
maintenance exists.
Chief architect exists. Yes
EA being developed using a Partial According to the chief
framework, methodology, architect, USAID is
developing their
and automated tool. architecture using a
framework, methodology, and
automated tool. However, the
methodology does not
describe steps required to
maintain and validate the
architecture.
EA plans call for describing "as-is" Yes
environment, "to-be"
environment, and sequencing plan.
EA plans call for describing Yes
enterprise in terms of
business, performance,
information/data,
application/service, and technology.
EA plans call for business, Yes
performance, information/data,
application/service, and technology
to address security.
EA plans call for developing Partial According to the chief architect,
metrics to measure EA progress, USAID has plans for developing
quality, compliance, and return metrics to measure EA progress,
on investment. quality, compliance, and return
on investment. USAID provided
documentation of its plans to
measure and report EA compliance.
However, documentation of plans
to develop the other metrics was
not provided.
Stage 3: Developing EA products
Written and approved No According to the chief architect,
organization policy exists for USAID does not have a written and
EA development. approved policy for EA
development.
EA products are under configuration management. No USAID provided evidence
that discusses the need for EA products to be under configuration
management. However, the evidence did not describe detailed steps that
would ensure the integrity and consistency of EA products.
(Continued From Previous Page) (Continued From Previous Page)
GAO basis for partially
satisfied or not satisfied
Stages and core elements Satisfied? determination
EA products describe or will Yes
describe "as-is" environment,
"to-be" environment, and sequencing
plan.
Both "asis" and "to-be" Yes
environments are described or
will be described in terms of
business, performance,
information/data,
application/service, and
technology.
These descriptions address or will Yes
address security.
Progress against EA plans is No According to the chief architect,
measured and reported. progress against EA plans is
not measured and reported.
Stage 4: Completing EA products
Written and approved organization No According to the chief architect,
policy exists for EA maintenance. USAID does not have a written and
approved policy for EA maintenance.
EA products and management Partial EA products and management
processes undergo processes have undergone
independent verification and independent verification and
validation. validation. However, the
current
EA version has not undergone
independent verification and
validation.
EA products describe "as-is" According to the chief
environment, "to-be" No architect, EA products do not
fully
environment, and sequencing describe the "as-is"
plan. environment, "to-be"
environment, and
sequencing plan.
Both "asis" and "to-be" Partial Both "as-is" and "to-be"
environments are described in Partial Yes environment descriptions are
terms of business, performance, being developed. However, the
information/data, descriptions currently
application/service, and address only one segment of
technology. These descriptions the architecture. The "as-is"
address security. Organization and "to-be"environment
CIO has approved current descriptions partially
version of EA. address security. However,
the descriptions currently
address only one segment of
the architecture and do not
address security in each of
the required terms.
Committee or group representing Partial According to the chief architect,
the enterprise or the committees representing the
investment review board has enterprise have approved the
approved current version of current version of the EA.
EA. However, documentation of these
approvals was not
provided.
Quality of EA products is measured No According to the chief architect,
and reported. USAID has not implemented
plans to measure and report quality
of EA products.
Stage 5: Leveraging the EA for
managing change
Written and approved organization No USAID has a program plan that
policy exists for IT investment encourages IT investment compliance
compliance with EA. with the EA. However, the plan does
not explicitly require IT investment
compliance with the EA.
Process exists to formally Partial According to the chief architect,
manage EA change. USAID has chartered a
configuration control board to
manage EA change. However, USAID
did not provide any documentation
that this board is functioning as
chartered.
EA is integral component of IT Partial USAID provided documentation
investment management process. indicating that EA is an integral
component of its IT investment
management process. However, USAID
does not have an enterprisewide
sequencing plan to guide IT
investments.
GAO basis for partially satisfied
or not satisfied
Stages and core elements Satisfied? determination
EA products are periodically Yes
updated.
IT investments comply with EA. Partial USAID provided documentation
indicating IT investments comply with the EA. However, investment
compliance is limited to the one segment of the EA that has been
developed.
Organization head has approved No According to the chief architect,
current version of EA. the organization head has not
approved the current version of
the EA.
Return on EA investment is measured No According to the chief architect,
and reported. USAID is developing metrics
for measuring and reporting return
on EA investment, but
these metrics have not been
completed.
Compliance with EA is measured and Yes
reported.
Source: GAO analysis of agency provided data.
Note: The U.S. Agency for International Development is working with the
Department of State to develop the Joint Enterprise Architecture. However,
USAID asked that we evaluate its agency's enterprise architecture based on
efforts to develop the USAID enterprise architecture.
Appendix V
Comments from the Department of Commerce
Appendix VI
Comments from the Department of Defense
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
See comment 1.
See comment 1.
See comment 2.
GAO Comments
1. We do not agree for two reasons. First, DOD's internal processes for
reviewing and validating the Global Information Grid (GIG), while
important and valuable to ensuring architecture quality, are not
independently performed. As we have previously reported, independent
verification and validation is a recognized hallmark of well-managed
programs, including architecture programs.1 To be effective, it should
be performed by an entity that is independent of the processes and
products that are being reviewed to help ensure that it is done in an
unbiased manner and that is based on objective evidence. Second, the
scope of these internal review and validation efforts only extends to
a subset of GIG products and management processes. According to our
framework, independent verification and validation should address both
the architecture products and the processes used to develop them.
2. While we acknowledge that GIG program plans provide for addressing
security, and our findings relative to the GIG reflect this, this is
not the case for DOD's Business Enterprise Architecture (BEA). More
specifically, how security will be addressed in the BEA performance,
business, information/data, application/service, and technology
products is not addressed in the BEA either by explicit statement or
reference. This finding relative to the BEA is consistent with our
recent report on DOD's Business System Modernization.2
1 GAO-03-584G
2GAO. Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, GAO-06-658 . (May 15,
2006).
Page 138 GAO-06-831 Enterprise Architecture
Appendix VII
Comments from the Department of Education
Appendix VII Comments from the Department of Education
Appendix VIII
Comments from the Department of Energy
Appendix IX
Comments from the Department of Homeland Security
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
See comment 1.
GAO Comments
1. We acknowledge this recommendation and offer three comments in
response. First, we have taken a number of steps over the last 5 years to
coordinate our framework with OMB. For example, in 2002, we based version
1.0 of our framework on the OMB-sponsored CIO Council Practical Guide to
Federal Enterprise Architecture,1 and we obtained concurrence on the
framework from the practical guide's principal authors. Further, we
provided a draft of this version to OMB for comment, and in our 2002
report in which we assessed federal departments and agencies against this
version, we recommended that OMB use the framework to guide and assess
agency architecture efforts.2 In addition, in developing the second
version of our framework in 2003,3 we solicited comments from OMB as well
as federal departments and agencies. We also reiterated our recommendation
to OMB to use the framework in our 2003 report in which we assessed
federal departments and agencies against the second version of the
framework.4
Second, we have discussed alignment of our framework and OMB's
architecture assessment tool with OMB officials. For example, after OMB
developed the first version of its architecture assessment tool in 2004,
we met with OMB officials to discuss our respective tools and periodic
agency assessments. We also discussed OMB's plans for issuing the next
version of its assessment tool and how this next version would align with
our framework. At that time, we advocated the development of comprehensive
federal standards governing all aspects of architecture development,
maintenance, and use. In our view, neither our framework nor OMB's
assessment tool provide such comprehensive standards, and in the case of
our framework, it is not intended to provide such standards. Nevertheless,
we plan to continue to evolve, refine, and improve our framework, and will
be issuing an updated version that incorporates lessons learned from the
results of this review. In doing so, we will continue to solicit comments
from federal departments and agencies, including OMB.
1CIO Council, A Practical Guide to Federal Enterprise Architecture,
Version 1.0 (February 2001).
2 GAO-02-6 3 GAO-03-584G 4 GAO-04-40
Page 144 GAO-06-831 Enterprise Architecture
Third, we believe that while our framework and OMB's assessment tool are
not identical, they nevertheless consist of a common cadre of best
practices and characteristics, as well as other relevant criteria that,
taken together, are complementary and provide greater direction to, and
visibility into, agency architecture programs than either does alone.
Appendix X
Comments from the Department of Housing and Urban Development
Appendix XI
Comments from the Department of the Interior
Appendix XII
Comments from the Department of Justice
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
See comment 1.
See comment 2.
See comment 3.
1. See DHS comment 1 in appendix IX. Also, while we do not have a basis
GAO Comments
for commenting on the content of the department's OMB evaluation
submission package because we did not receive it, we would note that the
information that we solicit to evaluate a department or agency against our
framework includes only information that should be readily available as
part of any well-managed architecture program.
1. We understand the principles of federated and segmented architectures,
but would emphasize that our framework is intentionally neutral with
respect to these and other architecture approaches (e.g.,
service-oriented). That is, the scope of the framework, by design,
does not extend to defining how various architecture approaches should
specifically be pursued, although we recognize that supplemental
guidance on this approach would be useful. Our framework was created
to organize fundamental (core) architecture management practices and
characteristics (elements) into a logical progression. As such, it was
intended to fill an architecture management void that existed in 2001
and thereby provide the context for more detailed standards and
guidance in a variety of areas. It was not intended to be the single
source of all relevant architecture guidance.
2. We agree, and believe that this report, by clearly identifying those
departments and agencies that have fully satisfied each core element,
serves as the only readily available reference tool of which we are
aware for gaining such best practice insights.
Appendix XIII
Comments from the Department of State
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
See comment 1.
See comment 2.
See comment 3.
GAO Comments
1. We acknowledge the comment that both CIOs approved the
configuration management plan. However, the department did not provide us
with any documentation to support this statement.
2. We acknowledge the comment that the architecture has been approved by
State and USAID executive offices. However, the department did not
provide any documentation describing to which executive offices the
department is referring to allow a determination of whether they were
collectively representative of the enterprise. Moreover, as we state
in the report, the chief architect told us that a body representative
of the enterprise has not approved the current version of the
architecture, and according to documentation provided, the Joint
Management Council is to be responsible for approving the
architecture.
3. We acknowledge that steps have been taken and are planned to treat the
enterprise architecture as an integral part of the investment
management process, as our report findings reflect. However, our point
with respect to this core element is whether the department's
investment portfolio compliance with the architecture is being
measured and reported to senior leadership. In this regard, State did
not provide the score sheets referred to in its comments, nor did it
provide any other evidence that such reporting is occurring.
Appendix XIV
Comments from the Department of the Treasury
Appendix XV
Comments from the Department Veterans Affairs
Appendix XVI
Comments from the Environmental Protection Agency
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
See comment 1.
See comment 2.
See comment 3.
See comment 4.
GAO Comments
1. We agree and have modified our report to recognize evidence contained
in the documents.
2. We do not agree. The 2002 documents do not contain steps for
architecture maintenance. Further, evidence was not provided
demonstrating that the recently prepared methodology documents were
approved prior to the completion of our evaluation.
3. We do not agree. While we do not question whether EPA's EA Transition
Strategy and Sequencing Plan illustrates how annual progress in
achieving the target architectural environment is measured and
reported, this is not the focus of this core element. Rather, this
core element addresses whether progress against the architecture
program management plan is tracked and reported. While we acknowledge
EPA's comment that it tracks and reports such progress against plans
on a monthly basis, neither a program plan nor reports of progress
against this plan were provided as documentary evidence to support
this statement.
4. We do not agree. First, while EPA's IT investment management process
provides for consideration of the enterprise architecture in
investment selection and control activities, no evidence was provided
demonstrating that the process has been implemented. Second, while EPA
provided a description of its architecture change management process,
no evidence was provided that this process has been approved and
implemented.
Appendix XVII
Comments from the General Services Administration
Appendix XVIII
Comments from the National Aeronautics and Space Administration
Appendix XIX
Comments from the Social Security Administration
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
See comment 1.
See comment 2.
GAO Comments
1. We do not agree. Neither the governance committee charter nor the
configuration management plan explicitly describe a methodology that
includes detailed steps to be followed for developing, maintaining, and
validating the architecture. Rather, these documents describe, for
example, the responsibilities of the architecture governance committee and
architecture configuration management procedures.
2. We do not agree. The core element in our framework concerning
enterprise architecture approval by the agency head is derived from
federal guidance and best practices upon which our framework is based.
This guidance and related practices, and thus our framework, recognize
that an enterprise architecture is a corporate asset that is to be owned
and implemented by senior management across the enterprise, and that a key
characteristic of a mature architecture program is having the architecture
approved by the department or agency head. Because the Clinger-Cohen Act
does not address approval of an enterprise architecture, our framework's
core element for agency head approval of an enterprise architecture is not
inconsistent with, and is not superseded by, that act.
Appendix XX
Comments from the U.S. Agency for International Development
Appendix XXI
GAO Contact and Staff Acknowledgements
GAO Contact
Randolph C. Hite, (202) 512-3439
Staff Acknowledgments
In addition to the person named above, Edward Ballard, Naba Barkakati,
Mark Bird, Jeremy Canfield, Jamey Collins, Ed Derocher, Neil Doherty, Mary
J. Dorsey, Marianna J. Dunn, Joshua Eisenberg, Michael Holland, Valerie
Hopkins, James Houtz, Ashfaq Huda, Cathy Hurley, Cynthia Jackson, Donna
Wagner Jones, Ruby Jones, Stu Kaufman, Sandra Kerr, George Kovachick,
Neela Lakhmani, Anh Le, Stephanie Lee, Jayne Litzinger, Teresa M. Neven,
Freda Paintsil, Altony Rice, Keith Rhodes, Teresa Smith, Mark Stefan, Dr.
Rona Stillman, Amos Tevelow, and Jennifer Vitalbo made key contributions
to this report.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in
Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***