Information Technology: Architecture Needed to Guide NASA's	 
Financial Management Modernization (21-NOV-03, GAO-04-43).	 
                                                                 
The National Aeronautics and Space Administration (NASA) is in	 
the process of modernizing its financial management operations	 
and supporting information technology systems. This		 
modernization, known as the Integrated Financial Management	 
Program (IFMP), is intended to provide NASA with an agencywide,  
integrated approach to performing critical business functions,	 
such as contract management--an area that GAO first designated as
high risk in 1990 and continues to do so today. GAO was requested
to review various aspects of IFMP, and this report is one in a	 
series on the program. The objective of this review was to	 
determine whether NASA has been acquiring and implementing IFMP  
in the context of an enterprise architecture.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-04-43						        
    ACCNO:   A08913						        
  TITLE:     Information Technology: Architecture Needed to Guide     
NASA's Financial Management Modernization			 
     DATE:   11/21/2003 
  SUBJECT:   Financial management				 
	     Financial management systems			 
	     Information resources management			 
	     Information systems				 
	     Performance measures				 
	     Program evaluation 				 
	     Strategic planning 				 
	     Program management 				 
	     Procurement practices				 
	     Procurement policy 				 
	     Internal controls					 
	     Enterprise architecture				 
	     NASA Integrated Financial Management		 
	     Program						 
                                                                 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-04-43

United States General Accounting Office

GAO

                       Report to Congressional Committees

November 2003

                                  INFORMATION
                                   TECHNOLOGY

     Architecture Needed to Guide NASA's Financial Management Modernization

                                       a

GAO-04-43

Highlights of GAO-04-43, a report to congressional committees

The National Aeronautics and Space Administration (NASA) is in the process
of modernizing its financial management operations and supporting
information technology systems. This modernization, known as the
Integrated Financial Management Program (IFMP), is intended to provide
NASA with an agencywide, integrated approach to performing critical
business functions, such as contract management-an area that GAO first
designated as high risk in 1990 and continues to do so today. GAO was
requested to review various aspects of IFMP, and this report is one in a
series on the program. The objective of this review was to determine
whether NASA has been acquiring and implementing IFMP in the context of an
enterprise architecture.

GAO is making recommendations to the NASA Administrator for establishing
an effective enterprise architecture management capability, ensuring the
completeness of future releases of NASA's enterprise architecture, and
minimizing its exposure to risk on IFMP caused by system component
acquisition and implementation efforts that have proceeded to date in the
absence of an enterprise architecture. NASA concurred with GAO's
recommendations and described completed, ongoing, and planned actions to
address them.

www.gao.gov/cgi-bin/getrpt?GAO-04-43.

To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or [email protected].

November 2003

INFORMATION TECHNOLOGY

Architecture Needed to Guide NASA's Financial Management Modernization

To date, NASA has acquired and implemented significant components of IFMP
without an enterprise architecture to guide and constrain the program. An
enterprise architecture is an organizational blueprint that defines-in
both business and technology terms-how an organization operates today and
how it intends to operate in the future; it also provides a plan for
transitioning to this future state. Using an enterprise architecture to
guide and constrain systems modernization programs is a federal
requirement and a recognized best practice of successful public and
private organizations. In addition, GAO's research has shown that
attempting major modernization programs such as IFMP without a
well-defined enterprise architecture risks, among other things, building
systems that are duplicative, are not interoperable, and do not
effectively and efficiently support mission operations and performance.

During the course of GAO's work, NASA recognized the need for an
enterprise architecture and has taken steps to develop one. For example,
it has established an architecture program office, designated a chief
architect, and selected an architecture framework to use. In addition,
after GAO completed its audit work, NASA released an initial version of an
enterprise architecture, which the chief technology officer stated was not
yet complete and would be improved upon in future versions. However, the
agency has yet to establish other key architecture management
capabilities, such as designating an accountable corporate entity to lead
the architecture effort, having an approved policy for developing and
maintaining the architecture, and implementing an independent verification
and validation function to provide needed assurance that architecture
products and architecture management processes are effective. Moreover,
the architecture products used to date to manage NASA's investment in IFMP
did not provide sufficient context (depth and scope of agencywide
operational and technical requirements) to effectively guide and constrain
the program.

The chief technology officer agreed that NASA needs an effective
enterprise architecture program and stated that efforts are under way to
establish one. GAO's experience in reviewing other agencies has shown that
not having an effective enterprise architecture program can be attributed
to, among other things, an absence of senior management understanding and
support, as well as cultural resistance.

NASA's current approach to acquiring and implementing IFMP outside the
context of an architecture unnecessarily increases the risk that the
program's system components will not effectively and efficiently support
agencywide operations. The result will be costly system rework. It is
critical for NASA to discontinue this approach and adopt the best practice
of managing its IFMP system investments within the context of a
well-defined enterprise architecture.

Contents

  Letter

Results in Brief
Background
IFMP Has Proceeded without an Enterprise Architecture, and

NASA's Ongoing Architecture Management Efforts Are Missing

Key Elements Conclusions Recommendations for Executive Action Agency
Comments and Our Evaluation

1 3 4

16 30 30 32

Appendixes

Appendix I:

Appendix II:

Appendix III:

Appendix IV:

Appendix V:

Objective, Scope and Methodology

Detailed Results of GAO's Analyses of Architecture Products Used to Date
by NASA to Acquire and Implement IFMP

Assessment of NASA's EA Management Efforts against GAO's Architecture
Management Maturity Framework

Comments from the National Aeronautics and Space Administration

GAO Contacts and Staff Acknowledgments

GAO Contact Acknowledgments 34

36

46

50

58 58 58

Tables	Table 1: Table 2:

Table 3:

Table 4: Table 5:

Overview of NASA's Organizational Components
Overview of NASA's Strategic Enterprises and the
Contributing Centers
Description and Status of NASA's IFMP System
Modules
Summary of IFMP Management Structure
Summary of the Maturity Stages and Core Elements of
GAO's Enterprise Architecture (EA) Management
Framework

                                       5

                                       6

                                     10 11

                                       28

Figure Figure 1:	Summary of Extent to Which NASA's Architecture Products
Satisfy Key Elements Governing Architectural Content 19

Contents

Abbreviations

ARC Ames Research Center
CIO chief information officer
CTO chief technology officer
CRUD create, read, update, and/or delete
DFRC Dryden Flight Research Center
EAI enterprise application integration
EA enterprise architecture
FEAF Federal Enterprise Architecture Framework
GRC Glenn Research Center
GSFC Goddard Space Flight Center
IFMP Integrated Financial Management Program
IT information technology
JPL Jet Propulsion Laboratory
JSC Johnson Space Center
KSC Kennedy Space Center
LaRC Langley Research Center
MSFC Marshall Space Flight Center
NISSU NASA's Information System Services Utility
NASA National Aeronautics and Space Administration
OIG Office of the Inspector General
OMB Office of Management and Budget
SSC Stennis Space Center
TRM technical reference model

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

A

United States General Accounting Office Washington, D.C. 20548

November 21, 2003

The Honorable John McCain

Chairman

The Honorable Ernest F. Hollings

Ranking Minority Member

Committee on Commerce, Science and Transportation United States Senate

The Honorable Sherwood L. Boehlert
Chairman
The Honorable Ralph M. Hall
Ranking Minority Member
Committee on Science
House of Representatives

To improve its ability to manage its contractors, which is an area that we
designated as high risk in 1990, and continue to do so today,1 the
National
Aeronautics and Space Administration (NASA) began its third attempt at
modernizing its financial management systems in April 2000. This
modernization effort, known as the Integrated Financial Management
Program (IFMP), is expected to produce an integrated, NASA-wide
business systems environment by acquiring and incrementally
implementing commercial hardware and software components. Our
research of successful public- and private-sector organizations shows that
attempting a modernization program, like IFMP, without having and using a
well-defined modernization blueprint, commonly called an enterprise
architecture,2results in operations and systems that are duplicative, are
not

1In 1990, we began a special effort to review and report on the federal
program areas that our work had identified as high risk because of
vulnerabilities to waste, fraud, abuse, and mismanagement. We first issued
our High-Risk Series in December 1992 and have since continued to include
NASA's contract management as an area of high risk. See U.S. General
Accounting Office, High-Risk Series: NASA Contract Management,
GAO/HR-93-11 (Washington, D.C.: December 1992) and High-Risk Series: NASA
Contract Management, GAO-03-119 (Washington, D.C.: January 2003).

2An enterprise architecture is a blueprint that defines, both in logical
terms (including integrated functions, applications, systems, users, work
locations, and information needs and flows) and in technical terms
(including hardware, software, data, communications, and security), how an
organization's information technology systems operate today, how they are
to operate in the future, and a road map for the transition.

well integrated, are unnecessarily costly to maintain and interface, and
do not effectively optimize mission performance.

In April 2003, we issued the first in a series of reports on the program,
in which we concluded that NASA's approach to acquiring and implementing
IFMP components had and would continue to introduce risk and increase the
chances that the agency would fall short of meeting its program goal.3
Because of the importance of IFMP to overall mission performance, you
asked us to continue our review. Specifically, you requested that we
determine whether (1) NASA has been acquiring and implementing IFMP in the
context of an enterprise architecture, (2) the core financial module as
implemented in June 2003 would satisfy NASA's key external reporting
requirements, and (3) NASA's life-cycle cost estimate, program schedule,
and funding reserves for IFMP were reasonable.

We are responding to the second two issues in separate reports,4 as well
as summarizing our findings on all three areas in an additional report.5
This report addresses the first issue-whether NASA had and was using an
enterprise architecture to acquire and implement IFMP. To accomplish this,
we compared the architecture documents that NASA provided us, and
represented as being used to manage IFMP, against published guidance
governing the content of a well-defined architecture.6 We also compared
NASA's architecture development, maintenance, and implementation practices
against our enterprise architecture management maturity

3U.S. General Accounting Office, Business Modernization: Improvements
Needed in Management of NASA's Integrated Financial Management Program,
GAO-03-507 (Washington, D.C.: Apr. 30, 2003).

4U.S. General Accounting Office, Business Modernization: NASA's Integrated
Financial Management Program Does Not Fully Address Agency's External
Reporting Issues, GAO04-151 (Washington, D.C.: Nov. 21, 2003) and Business
Modernization: Disciplined Processes Needed to Better Manage NASA's
Integrated Financial Management Program, GAO-04-118 (Washington, D.C.:
Nov. 21, 2003).

5U.S. General Accounting Office, Business Modernization: NASA Challenges
in Managing Its Integrated Financial Management Program, GAO-04-255
(Washington, D.C.: Nov. 21, 2003).

6See, for example, Office of Management and Budget, Federal Enterprise
Architecture Business Reference Model, Version 1.0 (2002); Chief
Information Officer Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001); and Office of Management and
Budget, Management of Federal Information Resources, Circular No. A-130
(Nov. 28, 2000).

framework.7 Details on our objective, scope, and methodology are in
appendix I.

Results in Brief 	To date, NASA has acquired and implemented significant
components of IFMP without an enterprise architecture, or modernization
blueprint, to guide and constrain the program.8 During the course of our
review of this program, the agency recognized the need for an enterprise
architecture and began efforts to develop one. For example, NASA
established some important architecture management structures and process
controls advocated by best practices and federal guidance, such as having
an enterprise architecture program office; designating a chief architect;
and using an architecture development methodology, framework, and
automated tool. In addition, after we completed our audit work, the agency
released an initial version of an enterprise architecture, which the chief
technology officer stated was not yet complete and would be improved upon
in future versions. However, NASA has yet to establish other key
architecture management capabilities that are essential to having a
mature, effective enterprise architecture program. Moreover, the
architecture products that the agency has used to date in managing its
$983 million IFMP investment9 did not provide sufficient context (depth
and scope of agencywide operational and technical requirements) to
effectively guide and constrain the program. NASA's chief technology
officer agreed that NASA needs an effective enterprise architecture
program and stated that efforts are under way to establish one.

Our experience in reviewing other agencies has shown that not having an
effective enterprise architecture program can be attributed to, among
other things, an absence of senior management understanding and support,
and cultural resistance to having and using one. Our experience also shows
that attempting major modernization programs such as IFMP without having
and using an enterprise architecture often results in system
implementations that are duplicative, are not well integrated, and require

7U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management, Version 1.1,
GAO-03-584G (Washington, D.C.: April 2003).

8NASA has acquired and implemented five of the nine planned major
components of IFMP and is in the process of implementing the sixth
component.

9GAO-04-118.

costly rework to interface. In the case of IFMP, this is occurring.
Specifically, NASA's Office of the Inspector General (OIG) recently
reported10 that the agency would need to resolve several accounting and
costing issues before the IFMP core system component provides full cost
accounting capabilities. This means that the agency will now have to
reconfigure the software for this component to reflect the issues'
resolution. According to the chief technology officer, determining the
need for additional rework of already implemented IFMP system components
will be based on a future assessment of these components' alignment to an
initial version of the agency's enterprise architecture that NASA first
provided to us on September 24, 2003, which was after we had completed our
audit work.

To assist NASA in its efforts to effectively and efficiently acquire and
implement IFMP, as well as its recently launched efforts to develop and
use a well-defined enterprise architecture, we are making recommendations
to the Administrator related to establishing an effective enterprise
architecture management capability, ensuring the completeness of planned
future releases of its enterprise architecture, and minimizing its
exposure to risk on IFMP caused by system component acquisition and
implementation efforts that have proceeded to date without an enterprise
architecture. NASA concurred with our recommendations, and described
actions recently completed, ongoing, or planned to implement them.

Background	NASA's mission encompasses human exploration and development of
space, the advancement and communication of scientific knowledge, and
research and development of aeronautics and space technologies. Its
activities span a broad range of complex and technical endeavors-from
investigating the composition, evaluation, and resources of Mars; to
working with the agency's international partners to complete and operate
the International Space Station; to providing satellite and aircraft
observations of Earth for scientific and weather forecasting purposes; to
developing new technologies designed to improve air safety. NASA's
workforce comprises over 19,000 civil service employees, primarily located
at its headquarters and 10 major field centers, and more than 40,000
contractors and grantees, who collectively perform a wide range of roles

10National Aeronautics and Space Administration Office of Inspector
General, Integrated Financial Management Program Core Financial Module
Conversion to Full Cost Accounting, IG-03-015 (Washington, D.C.: May 30,
2003).

and responsibilities. Table 1 describes the roles and responsibilities of
NASA's headquarters and field centers.

Table 1: Overview of NASA's Organizational Components

NASA headquarters and
field centers Location Roles/responsibilities

NASA Headquarters Washington, D.C.	NASA headquarters manages the space
flight centers, research centers, and other installations. It is
responsible for determining programs and projects; establishing management
policies, procedures, and performance criteria; evaluating progress; and
reviewing and analyzing all phases of the aerospace program.

                          Moffett Field,     ARC is a principal center for    
Ames Research Center       Calif.         computational fluid dynamics,    
                                                     rotorcraft and           
                                                powered-lift technology,      
          (ARC)                               artificial intelligence, and    
                                                   airborne sciences.         
      Dryden Flight     Edwards Air Force   DFRC is the premier installation  
         Research                          for aeronautical flight research.  
      Center (DFRC)        Base, Calif.    
      Glenn Research                                 GRC, the lead center for 
          Center         Cleveland, Ohio   aeropropulsion, is responsible for 
                                                       developing, verifying, 
          (GRC)                             and transferring aeropropulsion   
                                             technologies to U.S. industry.   
                                            GSFC is a major U.S. laboratory   
Goddard Space Flight   Greenbelt, Md.      for developing and operating    
                                                        unmanned              
      Center (GSFC)                              scientific spacecraft.       
      Jet Propulsion                        JPL is the lead U.S. center for   
        Laboratory       Pasadena, Calif.   robotic exploration of the solar  
                                                     system, with a           
                                               primary focus on planetary     
          (JPL)                              exploration (e.g., missions to   
                                                       Mars) and              
                                           environmental research (e.g.,      
                                           Shuttle Imaging Radar). The        
                                           California Institute of            
                                            Technology manages JPL for NASA.  
                                           JSC is the primary center for      
Johnson Space Center   Houston, Tex.    designing, developing, and testing 
                                           spacecraft and                     
                                              associated systems for human    
          (JSC)                              flight, selecting and training   
                                                    astronauts, and           
                                             planning and conducting human    
                                                 space flight missions.       
                        Cape Canaveral,     KSC is primarily responsible for  
Kennedy Space Center Fla.                 ground turnaround and support    
                                                      operations,             
                                           prelaunch checkout, and launching  
          (KSC)                               of the space shuttle and its    
                                                       payloads,              
                                               including NASA's International 
                                           Space Station. KSC is the nation's 
                                                                spaceport-the 
                                              liftoff site for all manned     
                                                  missions into space.        
     Langley Research                      LaRC is primarily responsible for  
          Center           Hampton, Va.    basic research in aeronautics and  
                                                         space                
                                           technology. It is the lead center  
          (LaRC)                             for managing NASA's technology   
                                                      development             
                                           programs for future high-speed     
                                           civil transport, hypersonic        
                                           vehicle concepts, and              
                                                   general aviation.          
      Marshall Space                        MSFC is the premier organization  
          Flight         Huntsville, Ala.         for developing space        
                                                   transportation and         
      Center (MSFC)                            propulsion systems and for     
                                           conducting microgravity research.  
                        Hancock County,      SSC is the primary center for    
Stennis Space Center Miss.               testing large rocket propulsion   
                                                    systems for the           
                                                space shuttle and future      
          (SSC)                                generation space vehicles.     

Source: NASA.

Transcending NASA's organizational components are six strategic mission
enterprises or business areas, each with a unique set of strategic goals,
objectives, and implementation strategies focused on the requirements of
the agency's customers. Each enterprise draws on the capabilities of

several centers so that each center contributes to multiple enterprises.
Table 2 summarizes NASA's strategic enterprises and the contributing
centers.

 Table 2: Overview of NASA's Strategic Enterprises and the Contributing Centers

Strategic enterprise Primary goal Contributing NASA centers

Aerospace technology          Pioneer and develop advanced ARC, DFRC, GRC, 
                          technologies that, in turn, improve        and LaRC 
                        the air transportation system, access 
                        to space, and science                 
                        missions. Includes helping others use 
                        NASA technology for                   
                         nonaerospace commercial purposes and 
                                        developing technology 
                          partnerships with those in industry 
                                and academia that are outside 
                          of traditional aerospace fields.    

Biological and physical research	Offer a unique laboratory in which to
study biological and physical ARC, GRC, JPL, JSC, KSC, and processes.
Experiments that take advantage of this environment MSFC extend from basic
biology to quantum mechanics and from fundamental research to research
with near-term applications in medicine and industry.

Earth science   Seek to understand and protect our planet ARC, DFRC, GSFC, 
                                         by advancing Earth-       JPL, LaRC, 
                 system science with a near-term emphasis on  MSFC, and SSC   
                 global climate                              
                 change through the use of Earth             
                 remote-sensing spacecraft,                  
                        airborne observations, space shuttle 
                                  missions, and ground-based 
                                measurements.                

Education	Inspire students to pursue the study of science and engineering,
ARC, DFRC, GRC, GSFC, JPL, with the ultimate goal of having them choose
careers in JSC, KSC, LaRC, MSFC, and aeronautics and space at NASA. SSC

Space flight	Provide critical enabling capabilities that make possible the
ARC, GRC, JPL, JSC, KSC, and science, research, and exploration
achievements of the rest of the MSFC agency.

Space science   Seek to answer fundamental questions about ARC, GSFC, JPL, 
                                        life in the universe:        KSC, and 
                 how it arose, what its mechanisms are, where      MSFC       
                                          in the solar system 
                 life may have originated or may exist today, 
                                        and whether there are 
                 similar planetary environments around other  
                 stars where the                              
                       signature of life can be found.        

Source: NASA.

To execute its mission responsibilities, NASA performs numerous management
functions, such as contract management, financial management, and human
capital management, relying heavily on information technology (IT) to
assist it in performing these functions. For fiscal year 2003, the agency
estimated that it would spend approximately $2.3 billion on IT systems and
services. Of this amount, NASA anticipated spending $32.5 million on IT
security and $11 million on enterprise architecture.

NASA Continues to Face Challenges in Managing Large Programs

In January 2003, we reported11 that NASA faced challenges that threaten
its ability to effectively run its largest programs. We also reported that
because these challenges are rooted in NASA's culture and long-standing
ways of doing business, the agency needed to make a major transformation.
In particular, we identified the following four performance and
accountability challenges facing the agency:

o 	Strengthening strategic human capital management. NASA is facing
shortages in its workforce, which could likely worsen as the workforce
continues to age and the pipeline of talent shrinks. This dilemma is more
pronounced among areas crucial to NASA's ability to perform its mission,
such as engineering, science, and IT. NASA is addressing this challenge
through strategic planning, through a new workforce planning and analysis
system, and by requesting additional personnel flexibilities, among other
initiatives.

o 	Controlling International Space Station costs. Development costs for
this project have soared to the point where NASA has had to cut back the
program substantially, including reducing construction, the number of crew
members, and scientific research. These cutbacks have raised concern among
NASA's international partners, who have a large stake in the scientific
research to be performed on the station. Although NASA is instituting
management and cost-estimating reforms, it still needs to reach agreement
with its partners on its planned cutbacks.

o 	Reducing costs of space launches. The administration submitted an
amendment to NASA's fiscal year 2003 budget request, which (1) extends the
life of the space shuttle and enhances its reliability, (2) funds the
development of a new vehicle for ferrying crew to and from the space
station, and (3) alters the time frame for a shuttle replacement.
Accomplishing these and other goals related to space launches will be
difficult and risky in light of the technology advances that NASA would
like to pursue and the high degree of communication and coordination
required among industry and government partners.

11U.S. General Accounting Office, Major Management Challenges and Program
Risks: National Aeronautics and Space Administration, GAO-03-114
(Washington, D.C.: January 2003).

o 	Improving contract management. NASA spends most of its funds on
acquisitions.12 Yet, for many years, the agency has been unable to oversee
contracts effectively, principally because it lacked accurate and reliable
information on contract spending and placed little emphasis on end
results, product performance, and cost control. NASA has addressed many
acquisition-related weaknesses and is beginning to tackle one of its most
formidable barriers to sound contract management-the lack of a modern,
integrated financial management system. Considerable work remains to be
done since NASA is only in the early stages of designing and implementing
this new system, and NASA reported that it is already facing challenges in
terms of cost, interoperability, and security.

We also reported that NASA's ability to collect, maintain, and report the
full cost of its projects and programs is weakened by diverse and often
incompatible and nonintegrated center-level accounting systems; uneven and
nonstandard cost-reporting capabilities; decentralized policies,
procedures, and practices that are unique to its field centers;
nonstandard data formats; and online financial information that is not
readily available to program managers. Thus, it is difficult to ensure
that contracts are being efficiently and effectively implemented and that
budgets are executed as planned. This lack of integration and
standardization also impedes the agency's ability to provide data required
for external reporting purposes.

Recognizing the need for change, NASA's Administrator articulated a new
vision for the agency-one that is science-driven, not destination-driven.
To better enable NASA to fulfill this vision, the agency is taking on a
major transformation aimed at eliminating stovepipes, becoming more
integrated and results-oriented, and reducing risks while working more
economically, efficiently, and effectively.

12NASA spends 90 percent or $12.7 billion of its annual budget for
aeronautical and spacerelated projects on the work performed by its
contractors.

NASA Has Initiated a Large, Complex Systems Modernization to Address
Financial Management Concerns

A key transformation effort is IFMP, which is NASA's third attempt in more
than 12 years to modernize its financial management processes and systems.
NASA spent about $180 million on its two prior failed efforts, and NASA's
data indicate that the agency will spend approximately $983 million
through 2010 for its current effort, IFMP, which it began in April 2000.13
IFMP is expected to produce an integrated, NASA-wide financial management
system by acquiring and incrementally implementing commercial software
packages and related hardware and software components. The main objective
of IFMP is to improve the financial, physical, and human capital
management processes throughout the agency. According to NASA, once fully
implemented, IFMP will reengineer NASA's business operations around
industry "best practices" and use enabling technology to provide necessary
management information to support implementation of the agency's strategic
plan. To meet this objective and support these crosscutting activities,
NASA has identified the following business drivers for the program:

o 	providing timely, consistent, and reliable information for management
decisions;

o  improving NASA's accountability and enabling full cost management;

o  achieving increased efficiencies and operating effectively;

o 	exchanging information with customers and stakeholders in a timely and
reliable way; and

o  attracting and retaining a world-class workforce.

The IFMP system is to consist of nine modules supporting a range of
functionality (see table 3).

13GAO-04-118.

         Table 3: Description and Status of NASA's IFMP System Modules

Module Description Reported status

Core financial	Support full cost management by providing agencywide
standards for Operational as of June 2003. accounting and budget execution
processes and financial reporting. Includes eight financial subprocesses:
budget execution, purchasing, cost management, accounts payable, accounts
receivable, fixed assets, standard general ledger, and federal reporting.

    Travel management   Streamline and unify the agency's   Operational as of 
                        employee travel system and improve         June 2003. 
                        traveler and vendor reimbursement.  
                        (According to NASA, this product    
                         has been integrated into the core  
                                financial module.)          
                        Provide budget, cost, and                             
Executive financial  performance information for all     Operational as of 
                        major NASA                          July 2002.
        management          programs and projects in a      
       information             standardized format.         
     system (Erasmus)                                       

Resume management	Enable applicants to search for matching NASA job
listings and generate Operational as of December resumes online and allow
NASA's human resources community to 2001. generate job listings while
increasing efficiency and effectiveness.

        Position        Enable position descriptions to Operational as of     
       description             be rapidly generated and early 2002.           
                                         classified and 
       management         associated documents to be    
                           automatically generated.     
                              Enable the formulation of                       
                       project, program, institutional,       Currently being 
Budget formulation                   enterprise, and       implemented; to
                                    agency-level budget be operational at all 
                          requirements. It will promote          locations in 
                                   full cost management 
                        and real-time decision making.     February 2004.     

Integrated asset Enable financial reporting, physical inventory,
maintenance, and liability Not yet initiated; no milestones management
reporting for the functional areas of aircraft, environmental, facilities,
and set.

logistics management.

Procurement 	Support the procurement, receiving, invoicing, and payment of
materials. Not yet initiated; no milestones Will provide detailed and
quantitative data to facilitate, economize, and set. expedite procurement
processes.

Human resources	Provide a human resources infrastructure that meets
recordkeeping and Not yet initiated; no milestones process requirements
while helping NASA managers fill positions with set. staff that possess
the appropriate skill sets and career goals.

Source: NASA.

As structured, NASA is the IFMP system integrator and thus is responsible
for acquiring and integrating the multiple commercial components and
ensuring that they collectively perform in a manner that meets the defined
requirements. Table 4 describes the key IFMP program management
positions/entities and their respective responsibilities.

Table 4: Summary of IFMP Management Structure

Management position/entity Responsibilities

Program Executive	Manages, on a corporate level, program rollout, budget,
performance, and schedule requirements; has decision authority over all
program content, implementation schedules, and budget allocations; and
provides leadership and accountability for top-level program requirements,
implementation success criteria, overall performance definition, and
strategic planning in the direction and operation of the Integrated
Financial Management Program (IFMP).

Program Director	Implements IFMP according to specific guidelines (e.g.,
the program plan); reports to the Program Executive, and is under the
oversight of the agency's Chief Financial Officer and the IFMP Steering
Council.

Chief Financial Officer	Ensures that the program meets externally mandated
requirements while satisfying internal customer needs in a cost-effective
manner.

Steering Council	Acts as a forum for reviewing and approving the
agencywide crosscutting facets of the program, including, for example,
program strategy and budgets and expanding the scope of projects; resolves
functional conflicts and ensures functional integration.

Project Manager Plans and manages the implementation of each functional
module approved by the program office; (each NASA center has a coordinates
process team activities and supports the selection of software products,
including updating Project Manager) requirements on the basis of the
selected software's capabilities and the developed gap assessments,

which identify differences between NASA's requirements and the software's
capabilities.

Integration Project Manager	Establishes a viable technical infrastructure
and ensures that the various functional module requirements are
coordinated, ensures that each IFMP module is appropriately
integrated/interfaced, minimizes redundant data, ensures that data
definitions are consistent across modules, establishes lifecycle
requirements, and performs configuration management.

Source: NASA.

Early Problems with IFMP Have Been Reported

In April 2003, we reported14 that NASA was not following key best
practices for acquiring and implementing IFMP. For example, the agency had
not established an analytical capability to understand and proactively
manage the dependencies among IFMP commercial components. Further, in
implementing the core financial module component, NASA had deferred
addressing the needs of key systems users and had not properly developed
detailed system requirements. We concluded that the agency was at risk of
making a substantial investment in a system that would fall far short of
its stated goal of providing meaningful and reliable information to
support effective program management and congressional oversight.

14U.S. General Accounting Office, Business Modernization: Improvements
Needed in Management of NASA's Integrated Financial Management Program,
GAO-03-507 (Washington, D.C.: Apr. 30, 2003).

To address its problems, we recommended that NASA (1) develop and
implement a short-term plan to identify and mitigate the risks currently
associated with relying on already deployed IFMP commercial components and
to expeditiously stabilize these components' operational capability and
performance; (2) as part of the short-term plan, develop and properly
document requirements, reengineer acquisition management processes, and
fully engage stakeholders-including program managers, cost estimators, and
the Congress-in the development of user requirements; and (3) develop a
longer-term strategy for acquiring additional IFMP components that
includes implementing a methodology for analyzing commercial system
component dependencies. NASA concurred with the need for a short-term plan
but disagreed with most of our findings and recommendations related to
user needs and requirements and testing. NASA also agreed with the
importance of having an approach for acquiring additional IFMP components,
but stated that it already has an effective strategy in place.

In May 2003, NASA's OIG reported15 that the core financial module
software, which had been deployed at six NASA centers, had the capability
to implement full cost accounting. However, before this implementation
could take place, NASA needed to resolve several complex accounting and
costing issues. These issues involved how to allocate service and general
and administrative costs, civil service costs, and unassigned costs. Once
these accounting and costing issues were resolved, the OIG reported that
NASA would have to configure the IFMP software to reflect the changes. The
OIG recommended that NASA revise the IFMP plans to include (1) time frames
and milestones for completing steps to implement full cost accounting,
including addressing and resolving the cost issues identified above; (2)
identification of the personnel and other resources necessary to perform
the steps within the established time frames; and (3) senior management
approval and support of these additional procedures. IFMP officials
concurred with the recommendations and plan to have all phases of full
cost accounting implemented by October 1, 2003. (NASA reported that full
implementation of the core financial module at all centers was completed
in June 2003.)

15National Aeronautics and Space Administration Office of Inspector
General, Integrated Financial Management Program Core Financial Module
Conversion to Full Cost Accounting, IG-03-015 (Washington, D.C.: May 30,
2003).

An Enterprise Architecture Is Critical to an Organization's Ability to
Effectively Modernize Its Business Operations and Systems

Effective use of enterprise architectures, or modernization blueprints, is
a trademark of successful public and private organizations. For a decade,
we have promoted the use of architectures to guide and constrain systems
modernization, recognizing them as a crucial means to a challenging goal:
agency operational structures that are optimally defined in both business
and technological environments. The Congress, the Office of Management and
Budget (OMB), and the federal Chief Information Officer (CIO) Council have
also recognized the importance of an architecture-centric approach to
modernization. The Clinger-Cohen Act of 1996 mandates that an agency's CIO
develop, maintain, and facilitate the implementation of architectures as a
means for managing the integration of business processes and supporting
systems. Further, OMB has issued guidance that, among other things,
requires system investments to be consistent with these architectures.

An enterprise architecture provides a clear and comprehensive picture of
an entity, whether it is an organization (e.g., federal department or
agency) or a functional or mission area that cuts across more than one
organization (e.g., financial management). This picture consists of
snapshots of both the enterprise's current or "As Is" operational and
technological environment and its target or "To Be" environment, as well
as a capital investment road map for transitioning from the current to the
target environment. These snapshots further consist of "views," which are
basically one or more architecture products that provide conceptual or
logical representations of the enterprise.

The suite of products and their content that form a given entity's
enterprise architecture are largely governed by the framework used to
develop the architecture. Since the 1980's, various frameworks have
emerged and been applied. For example, John Zachman developed a structure
or "framework" for defining and capturing an architecture.16 This
framework provides for six windows from which to view the enterprise,
which Zachman terms "perspectives" on how a given entity operates: those
of (1) the strategic planner, (2) the system user, (3) the system
designer, (4) the system developer, (5) the subcontractor, and (6) the
system itself. Zachman also proposed six abstractions or models associated
with each of these perspectives: these models cover (1) how the entity
operates, (2) what the

16J.A. Zachman, "A Framework for Information Systems Architecture," IBM
Systems Journal 26, no. 3 (1987).

entity uses to operate, (3) where the entity operates, (4) who operates
the entity, (5) when entity operations occur, and (6) why the entity
operates.

In September 1999, the federal CIO Council published the Federal
Enterprise Architecture Framework (FEAF), which is intended to provide
federal agencies with a common construct for their respective
architectures, thereby facilitating the coordination of common business
processes, technology insertion, information flows, and system investments
among federal agencies. FEAF describes an approach, including models and
definitions, for developing and documenting architecture descriptions for
multiorganizational functional segments of the federal government. Similar
to most frameworks, FEAF's proposed models describe an entity's business,
data necessary to conduct the business, applications to manage the data,
and technology to support the applications.

More recently, OMB established the Federal Enterprise Architecture Program
Management Office to develop a federated enterprise architecture according
to a collection of five "reference models:"

o The Business Reference Model is intended to describe the business
operations of the federal government independent of the agencies that
perform them, including defining the services provided to state and local
governments.

o The Performance Reference Model is to provide a common set of general
performance outputs and measures for agencies to use to achieve business
goals and objectives.

o The Data and Information Reference Model is to describe, at an aggregate
level, the types of data and information that support program and business
line operations, and the relationships among these types.

o The Service Component Reference Model is to identify and classify IT
service (i.e., application) components that support federal agencies and
promote the reuse of components across agencies.

o The Technical Reference Model is to describe how technology is
supporting the delivery of service components, including relevant
standards for implementing the technology.

These various enterprise architecture frameworks differ in their
nomenclatures and modeling approach. However, the frameworks consistently
provide for defining an enterprise's operations in both (1) logical terms,
such as interrelated business processes and business rules, information
needs and flows, and work locations and users, and (2) technical terms,
such as hardware, software, data, communications, and security attributes
and performance standards. The frameworks also provide for defining these
perspectives for both the enterprise's current or "As Is" environment and
its target or "To Be" environment, as well as a transition plan for moving
from the "As Is" to the "To Be" environment.

The importance of developing, implementing, and maintaining an enterprise
architecture is a basic tenet of both organizational transformation and IT
management. Managed properly, an enterprise architecture can clarify and
help optimize the interdependencies and relationships among an
organization's business operations and the underlying IT infrastructure
and applications that support these operations. Employed in concert with
other important management controls, such as portfolio-based capital
planning and investment control practices, architectures can greatly
increase the chances that organizations' operational and IT environments
will be configured to optimize mission performance. Our experience with
federal agencies has shown that investing in IT without defining these
investments in the context of an architecture often results in systems
that are duplicative, not well integrated, and unnecessarily costly to
maintain and interface.17

17See, for example, U.S. General Accounting Office, DOD Business Systems
Modernization: Improvements to Enterprise Architecture Development and
Implementation Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28,
2003); Information Technology: DLA Should Strengthen Business Systems
Modernization Architecture and Investment Activities, GAO-01-631
(Washington, D.C.: June 29, 2001); and Information Technology: INS Needs
to Better Manage the Development of Its Enterprise Architecture,
AIMD-00-212 (Washington, D.C.: Aug. 1, 2000).

IFMP Has Proceeded without an Enterprise Architecture, and NASA's Ongoing
Architecture Management Efforts Are Missing Key Elements

NASA has thus far acquired and deployed IFMP without a sufficiently
complete enterprise architecture to guide and constrain program investment
decisions. During the course of our review of IFMP, NASA took steps to
correct this situation by establishing key architecture management
capabilities and undertaking the development of an initial version of an
enterprise architecture that, according to the chief technology officer,
will provide some missing contextual information (operational and
technical). However, NASA has not established other key architecture
management capabilities, such as designating an accountable corporate
entity to lead the architecture effort, having an approved policy for
developing and maintaining the architecture, and implementing an
independent verification and validation function to provide needed
assurance that architecture products and architecture management processes
are effective. The chief technology officer agreed that NASA needs an
effective enterprise architecture program and stated that efforts are
under way to establish one. Based on our experience in reviewing other
agencies, not having an effective enterprise architecture program is
attributable to, among other things, limited senior management
understanding and commitment and cultural resistance to having and using
an architecture. The result is an inability to implement modernized
systems in a way that minimizes overlap and duplication and maximizes
integration and mission support.

The Architecture Products that NASA Has Used for IFMP Are Limited

As previously discussed, the various frameworks used to develop
architecture products consistently provide for describing a given
enterprise in both logical (e.g., business, performance, application,
information) and technical (e.g., hardware, software, data) terms, and for
doing so for the enterprise's current or "As Is" environment and its
target or "To Be" environment; these frameworks also provide for defining
a capital investment sequencing plan to transition from the "As Is" to the
"To Be" environment. However, the frameworks do not prescribe the degree
to which the component parts should be described to be considered correct,
complete, understandable, and usable-essential attributes of any
architecture. This is because the depth and detail of the descriptive
content depends on what the architecture is to be used for (i.e., its
intended purpose).

NASA's stated intention is to use an architecture as the basis for
agencywide business transformation and systems modernization, including
IFMP. This purpose necessitates that NASA's architecture products provide

considerable depth and detail, as well as logical and rational structuring
and internal linkages. More specifically, it means that these architecture
products should contain sufficient scope and detail so that, for example,
(1) duplicative business operations and systems are eliminated; (2)
business operations are standardized and integrated, and supporting
systems are interoperable; (3) use of enterprisewide services is
maximized; and (4) related shared solutions are aligned, like OMB's
e-government initiatives.18 Moreover, this scope and detail should be
accomplished in a way that (1) provides flexibility in adapting to changes
in the enterprise's internal and external environments; (2) facilitates
its usefulness and comprehension by varying perspectives, users, or
stakeholders; and (3) provides for properly sequencing investments to
recognize, for example, the investments' respective dependencies and
relative business value.

The architecture artifacts that NASA's chief technology officer provided
to us and represented as those used to date in acquiring and implementing
IFMP do not contain sufficient context (depth and scope of agencywide
operational and technical requirements) to effectively guide and constrain
agencywide business transformation and systems modernization efforts. More
specifically, these artifacts do not satisfy the most basic
characteristics of architecture content, such as clearly distinguishing
between artifacts that represent the "As Is" and the "To Be" environments.
The agency's chief technology officer agreed that these existing
architecture products do not clearly distinguish between the two
environments. Therefore, for purposes of our analyses, the chief
technology officer told us to treat the architecture products as
descriptive of the "To Be" environment, and to assume that any "As Is"
content in these products represented capability intended for reuse in the
future environment. This characterization is consistent with NASA
contractual documents associated with developing these architecture
products. On the basis of this characterization, we did not assess these
artifacts for their "As Is" content and accepted the chief technology
officer's acknowledgment

18OMB has identified 24 e-government initiatives that are expected to
support the goal of the President's management agenda and ultimately
provide improved government services to citizens, businesses, and other
levels of government. Examples of these initiatives include: (1) e-Grants,
which will provide a single site intended to streamline the federal grants
management process and allow customers of federal grants to find and apply
for grants; (2) e-Payroll, which will simplify and integrate payroll
systems across the federal government; and (3) e-Travel, which will
streamline the government's travel administration by creating a
governmentwide Web-based travel management service.

that this content was missing. Instead, we focused on the "To Be" content
and the transition plan. To assess the "To Be" architecture products, we
divided them into five architectural components similar to those in OMB's
architecture reference models: the business, information/data,
services/applications, technical, and performance components; we added
security as a sixth component because of its recognized importance and
relevance to the other five. We then compared architecture products NASA
used to date for IFMP to relevant criteria19 governing the content of key
architectural elements for the transition plan and these six components of
the "To Be" architecture. Based on this comparison, we determined whether
the architecture products generally satisfied, did not satisfy,20 or
partially satisfied21 each architectural element.

Overall, we found that NASA's "To Be" architecture products did not
satisfy 18 of 35 (51 percent) key elements and partially satisfied the
remaining 17 (49 percent), and its transition plan partially satisfied 1
(25 percent) of 4 elements and did not satisfy the remaining 3 (75
percent) (see fig. 1).

19See, for example, Office of Management and Budget, Federal Enterprise
Architecture Business Reference Model, Version 1.0 (2002); Chief
Information Officer Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001); Office of Management and
Budget, Management of Federal Information Resources, Circular No. A-130
(Nov. 28, 2000); M.A. Cook, Building Enterprise Information Architectures:
Reengineering Information Systems (Prentice Hall Inc.: 1996); and National
Institute of Standards and Technology, Information Management Directions:
The Integration Challenge, Special Publication 500-167 (September 1989).

20The architecture does not satisfy any aspects of this key architectural
element.

21The architecture partially satisfies some aspects of this key
architectural element but does not satisfy at least one significant
aspect.

Figure 1: Summary of Extent to Which NASA's Architecture Products Satisfy
Key Elements Governing Architectural Content

This means that the architecture products that NASA used to date in
acquiring and implementing IFMP have not provided an adequate context in
which to wisely invest in the program. In general, these products were
limited to descriptions of (1) technology characteristics, which is one of
many enterprise architecture elements, and (2) one of nine business
operations (finance and accounting). Our specific analysis of "To Be" and
transition plan products follows.

"To Be" Products: A "To Be" architecture is intended to capture the vision
of future business operations and supporting technology. It should
describe the desired capabilities, structures (e.g., entities, activities,
and roles), and relationships among these structures at a specified
point(s) in the future. The "To Be" architecture should show, for example,
future business processes, information needs, and supporting
infrastructure and be fiscally

and technologically achievable. According to relevant guidance,22 the "To
Be" architecture should contain, among other things, a description of (1)
the future business operations that will be performed to support the
organization's mission, including the entities or people that will perform
the functions, processes, and activities, and the locations where the
functions, processes, and activities will be performed; (2) the logical
database model that is to be used to guide the creation of the physical
databases where information will be stored; (3) the systems to be
developed or acquired to support the business operations; (4) the physical
infrastructure (e.g., hardware and software) that will be needed to
support the business systems; (5) the organizations that will be
accountable for implementing security and the tools to be used to secure
and protect systems and data; and (6) the metrics that will be used to
evaluate the effectiveness of mission operations and supporting system
performance in achieving mission goals and objectives. By including these
elements, the architecture would provide NASA with the necessary frame of
reference for engineering business processes and systems in a manner that
supports agencywide goals and objectives, such as ensuring that decision
makers routinely receive timely, accurate, and reliable information.

The "To Be" architecture products used to date in acquiring and
implementing IFMP provide minimal descriptive content. On the positive
side, they contain a description of one future business operation (i.e.,
finance and accounting). However, they do not describe other future
business operations (e.g., asset management and human resources). In
addition, they do not describe (1) finance and accounting in terms of the
entities or people who will perform the functions, processes, and
activities and the locations where the functions, processes, and
activities will be performed; (2) the logical database model to be used to
create the physical databases; (3) the actual systems to be developed or
acquired to support future business operations; (4) the physical
infrastructure (e.g., hardware and software) that will be needed to
support the business systems; (5) the organizations that will be
accountable for implementing security and the

22See, for example, Office of Management and Budget, Federal Enterprise
Architecture Business Reference Model, Version 1.0 (2002); Chief
Information Officer Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001); Office of Management and
Budget, Management of Federal Information Resources, Circular No. A-130
(Nov. 28, 2000); M.A. Cook, Building Enterprise Information Architectures:
Reengineering Information Systems (Prentice Hall Inc.: 1996); and National
Institute of Standards and Technology, Information Management Directions:
The Integration Challenge, Special Publication 500-167 (September 1989).

tools to be used to secure and protect systems and data; and (6) the
metrics that will be used to evaluate the effectiveness of mission
operations and supporting system performance in achieving mission goals
and objectives. Without this information, the organization will not have a
common vision and frame of reference for defining a transition plan to
guide and constrain the transformation of business operations and
associated capital investments and, thus, will be unable to effectively
leverage technology to orchestrate logical and systematic change and
optimize enterprisewide mission performance. Detailed results of our
analysis are provided in appendix II.

Transition Plan Products: According to relevant guidance and best
practices,23 the transition plan should provide a temporal road map for
moving from the "As Is" to the "To Be" environment. An important step in
developing a well-defined transition plan is a gap analysis-comparison of
the "As Is" and "To Be" architectures to identify differences. Other
important steps include analyzing technology opportunities and marketplace
trends, as well as assessing fiscal and budgetary realities and
institutional acquisition and development capabilities. With the use of
such analyses and assessments, options are explored and decisions are made
regarding which legacy systems to retain, modify, or retire and which new
systems to introduce on a tactical (temporary) basis or to pursue as
strategic solutions. Accordingly, transition plans identify legacy,
migration, and new systems and sequence them to show, for example, the
phasing out and termination of systems and capabilities and the timing of
the introduction of new systems and capabilities, and they do so in light
of resource constraints, such as budget, people, acquisition/development
process maturity, and associated time frames.

The transition plan artifacts that NASA relied on in acquiring and
implementing IFMP generally do not possess these attributes. Specifically,
they do not (1) provide a gap analysis identifying the needed changes to
current business processes and systems; (2) identify all of the systems
that will not become part of the "To Be" architecture, as well as the time
frames for phasing out these systems; (3) show a time-based strategy for
replacing legacy systems, including identifying intermediate (i.e.,
migration) systems

23See, for example, Office of Management and Budget, Federal Enterprise
Architecture Business Reference Model, Version 1.0 (2002); Chief
Information Officer Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001); and Office of Management and
Budget, Management of Federal Information Resources, Circular No. A-130
(Nov. 28, 2000).

that may be temporarily needed; and (4) define the resources (e.g.,
funding and staff) needed to transition to the target environment. The
result is that NASA has not had a meaningful and reliable basis for
managing the disposition of its systems or for sequencing the introduction
of modernized business operations and supporting systems. Detailed results
of our analysis appear in appendix II.

The chief technology officer agreed that the architecture products used to
date to acquire and implement IFMP do not provide sufficient scope and
content to constitute a well-defined enterprise architecture. Based on our
experience in reviewing other agencies, not having an effective enterprise
architecture program is attributable to, among other things, limited
senior management understanding and commitment and cultural resistance to
having and using an architecture.

Our experience with federal agencies has shown that attempting to define
and build major IT systems without first completing an enterprise
architecture often results in IT systems that are duplicative, are not
well integrated, are unnecessarily costly to maintain and interface, and
do not effectively optimize mission performance. In fact, NASA's OIG
recently reported24 that the agency would need to resolve several
accounting and costing issues before the IFMP core financial module, which
is to implement NASA's finance and accounting business process, would be
able to provide full cost-accounting capabilities as envisioned. To
accomplish this, the agency will have to reconfigure the IFMP software to
reflect these changes, resulting in system rework and additional
associated costs that could have been prevented.

Beyond this known rework, additional corrective action could be necessary
to address any misalignment between already implemented IFMP system
components and NASA's just-released initial version of an enterprise
architecture. Specifically, the chief technology officer provided us with
an initial version of a NASA enterprise architecture on September 24,
2003, which was after we completed our audit work. According to this
official, although this initial version of the architecture is incomplete,
it does provide some of the missing contextual information (operational
and technical) that we had identified during our review. The official also
stated

24National Aeronautics and Space Administration Office of Inspector
General, Integrated Financial Management Program Core Financial Module
Conversion to Full Cost Accounting, IG-03-015 (Washington, D.C.: May 30,
2003).

that future versions of the architecture are to be issued quarterly
through June 2004 and semiannually thereafter, and that plans are
currently being developed for assessing IFMP's alignment with the
architecture. The IFMP deputy program manager affirmed these plans for
assessing IFMP's alignment. In the likely event that any misalignment is
found, NASA will be faced with additional system rework demands.

NASA Does Not Have Key Capabilities in Place for Effectively Managing Its
Recently Launched Enterprise Architecture Effort

As NASA proceeds with its enterprise architecture effort, it is critical
that it employs rigorous and disciplined management practices in doing so.
Such practices form the basis of our architecture management maturity
framework,25 which specifies by stages the key architecture management
controls that are embodied in federal guidance and best practices,
provides an explicit benchmark for gauging the effectiveness of
architecture management, and provides a road map for making improvements.
Each of the five stages is described below.

1.	Creating enterprise architecture awareness. The organization does not
have plans to develop and use an architecture, or it has plans that do not
demonstrate an awareness of the value of having and using an architecture.
While stage 1 agencies may have initiated some architecture activity,
these agencies' efforts are ad hoc and unstructured, lack institutional
leadership and direction, and do not provide the management foundation
necessary for successful architecture development.

2.	Building the enterprise architecture management foundation. The
organization recognizes that the architecture is a corporate asset by
vesting accountability for it in an executive body that represents the
entire enterprise. At this stage, an organization assigns architecture
management roles and responsibilities and establishes plans for developing
enterprise architecture products and for measuring program progress and
product quality; it also commits the resources necessary for developing an
architecture-people, processes, and tools.

25U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management, Version 1.1,
GAO-03-584G (Washington, D.C.: April 2003).

3.	Developing the enterprise architecture. The organization focuses on
developing architecture products according to the selected framework,
methodology, tool, and established management plans. Roles and
responsibilities assigned in the previous stage are in place, and
resources are being applied to develop actual enterprise architecture
products. The scope of the architecture has been defined to encompass the
entire enterprise, whether organization-based or function-based.

4.	Completing the enterprise architecture. The organization has completed
its enterprise architecture products, meaning that the products have been
approved by the architecture steering committee or an investment review
board, and by the CIO. Further, an independent agent has assessed the
quality (i.e., completeness and accuracy) of the architecture products.
Additionally, evolution of the approved products is governed by a written
architecture maintenance policy approved by the head of the organization.

5.	Leveraging the enterprise architecture to manage change. The
organization has secured senior leadership approval of the enterprise
architecture products and a written institutional policy stating that IT
investments must comply with the architecture, unless granted an explicit
compliance waiver. Further, decision makers are using the architecture to
identify and address ongoing and proposed IT investments that are
conflicting, overlapping, not strategically linked, or redundant. Also,
the organization tracks and measures architecture benefits or return on
investment, and adjustments are continuously made to both the architecture
management process and the enterprise architecture products.

For stage 2, our framework specifies nine key practices or core elements
that are necessary to provide the management foundation for successfully
launching and sustaining an architecture effort. Examples of stage 2 core
elements are described below.

o 	Establish a committee or group representing the enterprise that is
responsible for directing, overseeing, or approving the enterprise
architecture. This committee should include executive-level
representatives from each line of business, and these representatives
should have the authority to commit resources and enforce decisions within
their respective organizational units. By establishing this enterprisewide
responsibility and accountability, the agency

demonstrates its commitment to building the management foundation and
obtaining buy-in from across the organization.

o 	Appoint a chief architect. The chief architect should be responsible
and accountable for the enterprise architecture, supported by the
architecture program office, and overseen by the architecture steering
committee. The chief architect, in collaboration with the CIO, the
architecture steering committee, and the organizational head, is
instrumental in obtaining organizational buy-in for the enterprise
architecture, including support from the business units, as well as in
securing resources to support architecture management functions, such as
risk management, configuration management, quality assurance, and security
management.

o 	Use a framework, methodology, and automated tool to develop the
enterprise architecture. These elements are important because they provide
the means for developing the architecture in a consistent and efficient
manner. The framework provides a formal structure for representing the
enterprise architecture, while the methodology is the common set of
procedures that the enterprise is to follow in developing the architecture
products. The automated tool serves as a repository where architectural
products are captured, stored, and maintained.

o 	Develop an architecture program management plan. This plan specifies
how and when the architecture is to be developed. It includes a detailed
work breakdown structure; resource estimates (e.g., funding, staffing, and
training); performance measures; and management controls for developing
and maintaining the architecture. The plan demonstrates the organization's
commitment to managing architecture development and maintenance as a
formal program.

o 	Allocate adequate resources. An organization needs to have the
resources (funding, people, tools, and technology) to establish and
effectively manage its architecture. This includes, among other things,
identifying and securing adequate funding to support architecture
activities, hiring and retaining the right people, and selecting and
acquiring the right tools and technology to support activities.

Our framework similarly identifies key architecture management practices
associated with later stages of architecture management maturity. For
example, at stage 3, the stage at which organizations focus on
architecture

development activities, organizations need to satisfy six core elements.
Examples of these core elements are discussed below.

o 	Issue a written and approved organization policy for development of the
enterprise architecture. The policy defines the scope of the architecture,
including the requirement for a description of the baseline and target
architecture, as well as an investment road map or sequencing plan
specifying the move between the two. This policy is an important means for
ensuring enterprisewide commitment to developing an enterprise
architecture and for clearly assigning responsibility for doing so.

o 	Ensure that enterprise architecture products are under configuration
management. This involves ensuring that changes to products are
identified, tracked, monitored, documented, reported, and audited.
Configuration management maintains the integrity and consistency of
products, which is key to enabling effective integration among related
products and for ensuring alignment between architecture artifacts.

At stage 4, during which organizations focus on architecture completion
activities, organizations need to satisfy eight core elements. Examples of
these core elements are described below.

o 	Ensure that enterprise architecture products and management processes
undergo independent verification and validation. This core element
involves having an independent third party-such an as internal audit
function or contractor that is not involved with any of the architecture
development activities-verify and validate that the products were
developed in accordance with architecture processes and product standards.
Doing so provides organizations with needed assurance of the quality of
the architecture.

o 	Ensure that business, performance, information/data,
application/service, and technology descriptions address security. An
organization should explicitly and consistently address security in its
business, performance, information/data, application/service, and
technology architecture products. Because security permeates every aspect
of an organization's operations, the nature and substance of
institutionalized security requirements, controls, and standards should be
captured in the enterprise architecture products.

At stage 5, during which the focus is on architecture maintenance and
implementation activities, organizations need to satisfy eight core
elements. Examples of these core elements are described below.

o 	Make the enterprise architecture an integral component of the IT
investment management process. Because the road map defines the IT systems
that an organization plans to invest in as it transitions from the "As Is"
to the "To Be" environment, the enterprise architecture is a critical
frame of reference for making IT investment decisions. Using the
architecture when making such decisions is important because organizations
should approve only those investments that move the organization toward
the "To Be" environment, as specified in the road map.

o 	Measure and report return on enterprise architecture investment. Like
any investment, the enterprise architecture should produce a return on
investment (i.e., a set of benefits), and this return should be measured
and reported in relation to costs. Measuring return on investment is
important to ensure that expected benefits from the architecture are
realized and to share this information with executive decision makers, who
can then take corrective action to address deviations from expectations.

Table 5 summarizes our framework's five stages and the associated core
elements for each stage.

 Table 5: Summary of the Maturity Stages and Core Elements of GAO's Enterprise
                     Architecture (EA) Management Framework

Stage Core elements

Stage 1:  o  Agency is aware of EA. Creating EA awareness

Stage 2:
Building the EA management
foundation

o  Adequate resources exist.

o  Committee or group representing the enterprise is responsible for
directing, overseeing, or approving EA.

o  Program office responsible for EA development and maintenance exists.

o  Chief architect exists.

o  EA is being developed using a framework, methodology, and automated
tool.

o  EA plans call for describing the "As Is" environment, the "To Be"
environment, and a sequencing plan.

o  EA plans call for describing the enterprise in terms of business,
information/data, application/service, and technology.

o  EA plans call for business, performance, information/data,
application/service, and technology descriptions to address security.

o  EA plans call for developing metrics for measuring EA progress,
quality, compliance, and return on investment.

Stage 3:  o  Written and approved organization policy exists for EA
development.
Developing EA products  o  EA products are under configuration management.
(includes all elements from  o  EA products describe or will describe the
enterprise's business, performance, information/data,
stage 2) application/service, and the technology that supports them.

o  EA products describe or will describe the "As Is" environment, the "To
Be" environment, and a sequencing plan.

o  Business, performance, information/data, application/service, and
technology descriptions address or will address security.

o  Progress against EA plans is measured and reported.

Stage 4:
Completing EA products
(includes all elements from
stage 3)

o  Written and approved organization policy exists for EA maintenance.

o  EA products and management processes undergo independent verification
and validation.

o  EA products describe the "As Is" environment, the "To Be" environment,
and a sequencing plan.

o  EA products describe the enterprise's business, performance,
information/data, application/service, and the technology that supports
them.

o  Business, performance, information/data, application/service, and
technology descriptions address security.

o  Organization chief information officer has approved current version of
EA.

o  Committee or group representing the enterprise or the investment review
board has approved current version of EA.

o  Quality of EA products is measured and reported.

Stage 5:  o  Written and approved policy exists for IT investment
compliance with EA.
Leveraging the EA for managing  o  Process exists to formally manage EA
change.
change  o  EA is integeral component of IT investment management process.
(includes all elements from  o  EA products are periodically updated.
stage 4)  o  IT investments comply with EA.

o  Organization head has approved current version of EA.

o  Return on EA investment is measured and reported.

o  Compliance with EA is measured and reported.

Source: GAO.

The state of NASA's implementation of key enterprise architecture
management practices, conditions, and structures currently places the
agency at stage 1 of our maturity framework. Specifically, it has
satisfied all but one of the core elements associated with building the
architecture management foundation-stage 2 of our framework-but only about
23 percent (5 of the 22) of the core elements associated with stages 3, 4,
and 5. According to our framework, effective architecture management is
generally not achieved until an enterprise has a completed and approved
architecture that is being effectively maintained and is being used to
leverage organizational change and support investment decision making;
having these characteristics is equivalent to having satisfied all stage 3
core elements and many stage 4 and 5 elements.

Regarding stage 2 core elements, NASA has, for example, recently
established a program office, assigned a chief architect, and selected a
framework (Zachman) and automated tools (e.g., the Rational Rose by
Rational Software Corporation/IBM Software Group). However, the agency has
not satisfied a stage 2 core element that is critical to effective
architecture management. Specifically, a committee or group representing
the enterprise has not yet been established to guide, direct, or approve
the architecture. Instead, the CIO is guiding the architecture development
effort. Having such a corporate entity is critical to overcoming cultural
resistance to using an enterprise architecture. Without such an entity to
lead and be accountable for the architectural effort, there is increased
risk that the architecture will not represent a corporate decision-making
tool and will not be viewed and endorsed as an agencywide asset.

Concerning stage 3, NASA has not satisfied three of six core elements. For
example, the agency does not have a written and approved policy for
architecture development, which is a stage 3 core element. Without such a
policy that, for example, identifies the major players in the development
process and provides for architecture guidance, direction, and approval,
NASA will be challenged in overcoming cultural resistance to using an
enterprise architecture and achieving agencywide architecture commitment
and support.

The agency also has yet to implement numerous stage 4 and 5 core elements.
For example, NASA has not (1) documented and approved a policy for
architecture maintenance, (2) implemented an independent verification and
validation function that covers architecture products and architecture
management processes, and (3) made the architecture an integral component
of its IT investment management process. (The

detailed results of our assessment of NASA's satisfaction of each of the
stages and associated core elements are provided in app. III.)

According to the chief technology officer, the agency recognizes the
importance of having rigorous and disciplined architecture management
controls and is in the process of establishing them. Our research of
successful organizations and experience in reviewing other agency
enterprise architecture efforts shows that not having these controls is,
among other things, a function of limited senior management understanding
of and commitment to an enterprise architecture and cultural resistance to
having and using one. Until such barriers are addressed, and effective
architecture management structures and processes are established, it is
unlikely that any agency will be able to produce and maintain a complete
and enforceable architecture or implement modernized systems in a way that
minimizes overlap and duplication and maximizes integration and mission
support.

Conclusions 	NASA's acquisition and implementation of six major IFMP
system components outside the context of an enterprise architecture was
not a prudent decision. Such a systems modernization approach
unnecessarily increases the risk that system components will not
effectively and efficiently support agencywide operations, which in turn
leads to costly system rework. It is critical for NASA to discontinue this
approach and adopt the best practice of managing its IFMP system
investments within the context of a well-defined enterprise architecture.
In order to do so, it is important for NASA to establish an effective
means for developing and implementing an architecture, which includes
gaining top management understanding and support to lead the way in
overcoming any cultural resistance. It is equally important that the
agency ensure that the architecture contains sufficient depth and scope,
quickly determine whether existing and planned IFMP component systems
align with initial and subsequent versions of the architecture, and limit
further investment in these systems until such determinations are made. To
do less risks introducing additional system rework to that already facing
the agency on already implemented system components.

Recommendations for 	To ensure that NASA has the necessary agencywide
context within which to make informed IFMP and other systems modernization
decisions, we

Executive Action recommend that the NASA Administrator demonstrate an
institutional

commitment to developing and using an enterprise architecture by
establishing a NASA enterprise architecture policy and designating a NASA
architecture board, or comparable body, that is made up of agency
executives who are responsible and accountable for developing and
maintaining the architecture.

In carrying out its responsibility, we recommend that the Administrator
direct the architecture board, in collaboration with the CIO, to ensure
that the architecture content requirements identified in this report are
satisfied by first determining the extent to which NASA's initial release
of an enterprise architecture satisfies these content requirements and
then developing and approving a plan for incorporating any content that is
missing.

We further recommend that the Administrator direct the IFMP Program
Executive Officer to appropriately limit acquisition and implementation
activities until the agency ensures that the program's plans are aligned
with the initial and subsequent versions of the enterprise architecture.
In addition, we recommend that the Administrator direct the architecture
board, in collaboration with the CIO, to immediately map already
implemented IFMP components to the agency's enterprise architecture and
report to the Program Executive Officer any instances of misalignment, the
associated risks, and proposed corrective actions. Moreover, we recommend
that the Administrator direct the Program Executive Officer to develop
corrective action plans, as appropriate, that include specific milestones,
cost estimates, and detailed actions to be taken to align the program with
the enterprise architecture.

To further assist NASA, we recommend that the Administrator direct the
board, in collaboration with the CIO, to ensure that the best practices
involved in stages 3 through 5 of our enterprise architecture management
maturity framework are implemented. More specifically, we recommend that
the board and the CIO (1) establish a written and approved policy for
architecture development, (2) place enterprise architecture products under
configuration management, and (3) ensure that progress against
architecture plans is measured and reported.

In completing the architecture, we recommend that the board and CIO (1)
establish a written and approved policy for architecture maintenance; (2)
ensure that enterprise architecture products and management processes
undergo independent verification and validation; (3) ensure that
architecture products describe the enterprise's business and the data,

application, and technology that supports it; (4) ensure that enterprise
architecture products describe the "As Is" environment, the "To Be"
environment, and a sequencing plan; (5) ensure that business, performance,
data, application, and technology descriptions address security; (6)
ensure that the CIO approves the enterprise architecture; (7) ensure that
the steering committee and/or the investment review board has approved the
current version of the enterprise architecture; and (8) measure and report
on the quality of enterprise architecture products.

In implementing the architecture, we recommend that the board and CIO (1)
establish a written and approved policy for IT investment compliance with
the enterprise architecture, (2) ensure that the enterprise architecture
is an integral component of IT investment management processes, (3) ensure
that IT investments comply with the enterprise architecture, (4) obtain
Administrator approval of each enterprise architecture version, (5)
measure and report enterprise architecture return on investment, and (6)
measure and report on enterprise architecture compliance.

Agency Comments and Our Evaluation

In written comments on a draft of this report signed by the Deputy
Administrator (reprinted in app. IV), NASA concurred with our
recommendations and described recently completed, ongoing, or planned
efforts to address them. For example, the agency stated that it has
developed a 3-year plan for refining the latest version of its
architecture, as well as a plan to guide the agency in using the
architecture to achieve NASA's strategic goals. In addition, the agency
stated that it has adopted our architecture management maturity framework
and is currently working to satisfy the framework's core elements,
including establishing architecture policies and a function for
independently verifying and validating architecture artifacts and
management practices. Additionally, it stated that it plans to continually
validate IFMP against its architecture on a quarterly basis.

NASA also stated that its CIO board, which is chaired by NASA's CIO and
composed of the CIOs from the agency's six major lines of business and its
ten field centers, serves as the NASA architecture board or steering
committee. While we support CIO representation on an architecture steering
committee, recognized best practices and our maturity framework both
advocate that architecture ownership and accountability be vested with an
enterprise's business owners. Thus, we state in our framework that the
architecture steering committee should be composed of executive-level
representatives from each line of business and that these representatives

should have the authority to commit resources and enforce decisions for
their respective organizational units. Without such an entity to lead and
be accountable for the architectural effort, there is increased risk that
the architecture will be viewed solely as an IT tool and not represent a
corporate, business-driven decision-making tool and will not be viewed and
endorsed as an agencywide asset. Accordingly, it is important for NASA to
ensure that its architecture board's membership includes business owner
representation.

As agreed with your offices, unless you announce its contents earlier, we
will not distribute this report further until 30 days from its date. At
that
time, we will send copies to interested congressional committees as well
as
to the NASA Administrator and the Director of OMB. We will make copies
available to others upon request. In addition, the report will be
available at
no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please
contact me at (202) 512-3439 or [email protected]. Key contributors to this
report are acknowledged in appendix V.

Randolph C. Hite
Director
Information Technology Architecture

and Systems Issues

Appendix I

                        Objective, Scope and Methodology

To determine whether the National Aeronautics and Space Administration
(NASA) had and was using an enterprise architecture to guide and constrain
its investment in its Integrated Financial Management Program (IFMP), we
requested all NASA enterprise architecture artifacts and related
documentation that had been used to date to guide and constrain IFMP and,
based on what we were provided by NASA's chief technology officer,1
compared them to relevant guidance.2

In doing so, we first segmented our analysis of artifacts and guidance
into the three primary component parts of any architecture: the "As Is"
architecture, the "To Be," and the transition plan. We then further
divided the "As Is" and "To Be" architectures into five architectural
components similar to the Office of Management and Budget's architecture
reference models: business, information/data, services/applications,
technical, and performance. We also added security as a sixth component
because of its recognized importance in the various architecture
frameworks and relevance to the other five architectural components.
Because NASA had not clearly distinguished between its "As Is" and "To Be"
environments, the chief technology officer told us to treat the
architecture products provided as the "To Be" environment and assume that
any "As Is" content would be intended for reuse in the future environment.
As a result, we did not analyze whether NASA's architecture products
satisfied relevant "As Is" guidance; instead, we accepted the chief
technology officer's acknowledgment that NASA did not have any "As Is"
artifacts.

To augment our documentation reviews and analyses of architecture products
used to date in acquiring and implementing IFMP, we also interviewed
various officials, including the chief information officer and chief
technology officer, to determine, among other things, the agency's plans
to develop an enterprise architecture. Specifically, we inquired as to
NASA's basis for selecting already acquired IFMP commercial products and

1We reviewed technology architectures and an enterprise architecture for
the IFMP core financial module.

2See, for example, Office of Management and Budget, Federal Enterprise
Architecture Business Reference Model, Version 1.0 (2002); Chief
Information Officers Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001); Office of Management and
Budget, Management of Federal Information Resources, Circular No. A-130
(Nov. 28, 2000); M.A. Cook, Building Enterprise Information Architectures:
Reengineering Information Systems (Prentice Hall Inc.: 1996); and National
Institute of Standards and Technology, Information Management Directions:
The Integration Challenge, Special Publication 500-167 (September 1989).

Appendix I
Objective, Scope and Methodology

its plans for selecting future IFMP modules, including whether the agency
had developed an enterprise architecture to guide and constrain its future
investment in IFMP.

We also requested information on ongoing efforts to develop the initial
version of NASA's enterprise architecture, such as detailed program plans,
updated policies and procedures, and the architecture itself, but this
information was not provided until September 24, 2003, which was after we
had completed our audit work. As a result, our review did not include an
assessment of the initial version of NASA's enterprise architecture, which
the chief technology officer stated addressed some, but not all, of the
limitations discussed in this report.

To determine whether NASA's initial and subsequent versions of its
enterprise architecture were supported by effective management structures
and processes, we used our Enterprise Architecture Management Maturity
Framework,3 which describes the five stages of management maturity, and
determined the extent to which NASA has adopted key elements of
architecture management best practices embodied in the framework. To make
this determination, we reviewed program documentation, such as program
policies and procedures, an IBM report4 on the agency's efforts to
implement management processes and controls over its architecture
development activities, and the architecture products used to date in
acquiring IFMP system components, and we compared them to the elements in
our framework. We did not independently validate the cost and budget
information provided by the chief technology officer.

We conducted our work at NASA headquarters in Washington, D.C. We
performed our work from June to mid-September 2003 in accordance with
generally accepted government auditing standards.

3U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management, Version 1.1,
GAO-03-584G (Washington, D.C.: April 2003).

4IBM, NASA Enterprise Architecture: Roadmap for Building and Sustaining
Business Value Through an Integrated EA, Sept. 6, 2002.

Appendix II

Detailed Results of GAO's Analyses of Architecture Products Used to Date
by NASA to Acquire and Implement IFMP

           Detailed Analysis of NASA's "To Be" Architecture Products

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

Business

A description of the overall architectural vision and strategic goals that
define what an organization wants to achieve.

X	The available architecture products contain a high-level description of
the agency's OneNASA vision, which focuses on how technology will be
managed to improve services (e.g., providing secure and highly
interoperable information systems in support of all NASA operations). It
lists mission statements for both the agency and the lines of business.
The architecture also lists business architecture drivers, which can be
considered business goals.

However, the available architecture products do not contain a description
of the strategic goals, objectives, missions, and implementing strategies
established to support NASA lines of business. In addition, the
architecture products do not explain what the OneNASA vision encompasses
since it appears to be technology-centric, as opposed to business-centric
(i.e., it addresses business, information/data, services/applications, and
technology).

     A business strategy, which   X  The available architecture products list 
              defines                                    business strategies, 
how the enterprise's strategic         such as implementing the Integrated 
               goals                                     Financial Management 
       and objectives will be            Program (IFMP). However, they do not 
             achieved.                                     describe how these 
                                         strategies will be implemented.      

Common (standard and X
agencywide) policies, procedures,
and business and operational rules
for consistent implementation of the
architecture.

A description of key business X The available architecture products
contain a description of the
processes and how they support the finance and accounting processes (i.e.,
the processes to be
agency's mission, including the supported by the IFMP core financial
module). This description
organizational units responsible for also identifies the subprocesses
within these processes and
performing the business processes includes detailed diagrams of process
flows.
and the locations where the
business processes will be However, these products do not identify the
organizational
performed. This description should units responsible for performing the
finance and accounting
provide for the consistent alignment business processes nor the locations
where they will be
of (1) applicable federal laws, performed. Moreover, the architecture
products do not contain
regulations, and guidance; a description of other business processes, such
as asset
(2) agency policies, procedures, and management, human resource
management, and budget.
guidance; (3) operational activities;
(4) organizational roles; and
(5) operational events and
information.

                                  Appendix II
                     Detailed Results of GAO's Analyses of
                   Architecture Products Used to Date by NASA
                         to Acquire and Implement IFMP

                         (Continued From Previous Page)

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

A description of the operational X
management processes to ensure
that the agency's business
transformation effort remains
compliant with the business rules for
fault, performance, security,
configuration, and account
management.

A listing of opportunities X The available architecture products recognize 
          to unify and                                        the need for an 
      simplify systems or       implementing strategy to streamline financial 
           processes                                           operations and 
       across the agency.       identify IFMP as that strategy. However, the  
                                products do not                               
                                describe specific opportunities for improving 
                                weaknesses in                                 
                                the "As Is" financial systems or processes or 
                                                             how IFMP will be 
                                         implemented to achieve this          
                                         unification/simplification.          

A description of the organizational approach (processes and organizational
structure) for communications and interactions among business lines and
program areas for (1) management reporting, (2) operational functions, and
(3) architecture development and use (i.e., how to develop the
architecture description, implement the architecture, and govern/manage
the development and implementation of the architecture).

X	The available architecture products contain a description of the
management reporting lines for the agency's chief information officer
(CIO) organization as it relates to managing the architecture products and
standards. However, they do not describe the roles and responsibilities of
other organizations. For example, these products do not have a model for
roles and an organization chart that shows the lines of communication and
reporting responsibilities for financial management operations.

Information/data

A description of data management X
policies, processes, procedures, and
tools (e.g., CRUD matrixa) for
analyzing, designing, building, and
maintaining databases in an
enterprise architected environment.

    A description of the business     X The available architecture products   
                 and                    contain a description of              
     operational rulesb for data        technical standards currently being   
                                        used. However, these                  
    standardization to ensure data        products do not state whether these 
                                                 standards are still relevant 
     consistency, integrity, and        or will need to be updated to reflect 
              accuracy,                 changes to the current                
    such as business and security       environment. In addition, the         
                rules                   architecture products do not          
that govern access, maintenance,        identify data standards upon which 
                                                  business rules can later be 
           and use of data.                          developed.c              
    A data dictionary, which is a   X   
     repository of standard data        
    definitions for applications.       

Appendix II
Detailed Results of GAO's Analyses of
Architecture Products Used to Date by NASA
to Acquire and Implement IFMP

                         (Continued From Previous Page)

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

A conceptual data model that X
describes the fundamental
things/objects (e.g., invoices,
financial statements, inventory) that
make up the business, but without
regard for how they will be physically
stored. It represents the
consolidated structure of business
objects to be used by business
applications.

A logical database model that X
provides (1) the data structures that
support information flows and (2) the
basis for developing the schemas for
designing, building, and maintaining
physical databases.d

A metadatae model that specifies the X
rules and standards for access to
information.f

A description of information flows X
and relationships between
organizational units, business
operations, and system elements.

                             Services/applications

A description of the end-user X
services to be provided by the
application systems.

A list of application systems X
(acquisition/development and
production portfolio)g and their
relative importance to achieving the
agency's vision based on business
value and technical performance.

A description of the policies, X
procedures, processes, and tools for
selecting, controlling, and evaluating
application systems to enable
effective IT investment management.

A description of the enterprise X      The available architecture products 
                                                  contain a description of an 
       application systems and       enterprise application system that       
                                     supports finance and                     
        components and their          accounting (IFMP core financial module) 
            interfaces.g                                 functions. They also 
                                     identify legacy systems that interface   
                                     with this application                    
                                                     system.                  
                                     However, this description is limited to  
                                     this one system.                         

                                  Appendix II
                     Detailed Results of GAO's Analyses of
                   Architecture Products Used to Date by NASA
                         to Acquire and Implement IFMP

                         (Continued From Previous Page)

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

A description of the common technical approach, policies, and procedures
for developing/acquiring application systems throughout their life cycle,
including requirements management, design, implementation,
testing,deployment, operations, and maintenance. The common technical
approach should also describe the process for integrating legacy systems
with the systems to be developed/acquired.

X	The available architecture products list several architectural
principles for system development and acquisition (e.g., modular design,
open system approach) and identify a strategy (i.e., a hub-spoke
configuration) for integrating legacy systems. They also identify a
minimum set of technical standards for hardware and software. In addition,
the architecture products contain policies and guidance for implementing
systems.

However, these products do not describe a common technical approach. In
addition, the products did not state whether the existing policies and
procedures are common, complete, and sufficient to effectively implement
the architecture. They do recognize the need to revise existing policies
and procedures.

Technical

A list of infrastructure systems and X
their relative importance to achieving
the agency's vision based on
business value and technical
performance.

A description of the policies, X
procedures, processes, and tools for
selecting, controlling, and evaluating
infrastructure systems to enable
effective IT investment management.

A description of the technical  X      The available architecture products 
                                                     recognize the need for a 
     reference model (TRMh) that     TRM and contain a generic description of 
                                     the TRM. These                           
      describes the enterprise             products also note that technology 
                                                   services needed to support 
     infrastructure services, i           the application portfolio should be 
              including                          defined and identify several 
specific details regarding the               of these services.            
functionality and capabilities    
                that                 
these services will provide to    However, according to NASA's chief       
               enable                technology officer, the                  
     development of application          TRM is incomplete and flawed. In     
              systems.                        addition, the list of           
                                        technology services identified is     
                                                   incomplete.                

A description in the TRM that X   The available architecture products note 
                                                         that these standards 
identifies and describes the     should be identified and documented. They 
                                                          also contain a list 
    technical standardsj to be               of specific standards.           
       implemented for each        
            enterprise             
             service.               However, the architecture products do not 
                                                          state whether these 
                                      standards are for the current or future 
                                                    environment. In addition, 
                                   the architecture products do not identify  
                                                 the technical                
                                     standards to be implemented for specific 
                                                         enterprise services, 
                                           such as query processing.          

                                  Appendix II
                     Detailed Results of GAO's Analyses of
                   Architecture Products Used to Date by NASA
                         to Acquire and Implement IFMP

                         (Continued From Previous Page)

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

A description of the physical IT X The available architecture products     
                                      contain a high-level                    
infrastructure needed to support       description (i.e., diagrams without 
                 the                             supporting narrative) of the 
      developed and/or acquired          IFMP core financial network, OneNASA 
               systems,                                     network backbone, 
     including the relationships      and NASA's Information System Services  
                among                 Utility (NISSUk)                        
       hardware, software, and                 network architecture.          
       communications devices.        
                                      However, these networks do not          
                                      encompass the entire                    
                                        enterprise, but rather a subset of    
                                                    activities.               

Common policies and procedures for X The available architecture products   
                                        contain a list of policies            
developing infrastructure systems    and procedures for implementing       
                                        systems. However, the                 
      throughout their life cycle,        products do not state whether these 
               including                              policies and procedures 
    requirements management, design,     are common, complete, and sufficient 
                                                     to effectively implement 
        implementation, testing,                  the architecture.           
              deployment,               
operations, and maintenance. These   
     policies and procedures should     
                  also                  
       address the integration of       
     applications, including legacy     
                systems.                

Security

A description of the policies, procedures, goals, strategies, and
requirements relevant to information assurance and security and how they
(the policies, procedures, goals, strategies, and requirements) align and
integrate with other elements of the architecture (e.g., security
services).

X	The available architecture products contain a high-level description of
security goals and strategies and identify some security requirements.
They also note that an "Information Assurance Trust Model" is needed and
will be developed.

The architecture products do not contain a description of security
policies and procedures. They also do not identify important security
requirements (e.g., availability and access control), nor do they link
identified security requirements to security services. Moreover, the
architecture products do not define the "Information Assurance Trust
Model" or address plans for its completion. Finally, regarding the
strategies, they do not identify and summarize the agency's most
significant security risks.

According to NASA's chief technology officer, a clear computer security
policy does not exist within the agency, and there is a lack of
understanding as to how such a policy could be integrated into the network
infrastructure.

    Definitions of security and  X        The available architecture products 
                                                 contain definitions for some 
information assurance related                security-related terms (e.g., 
              terms.                     authentication, confidentiality, and 
                                   intrusion detection); however, they do not 
                                   define other key                           
                                   terms listed (e.g., integrity, physical    
                                   security, and encryption                   
                                   services). In addition, the definitions    
                                   for these terms are                        
                                   inconsistent with the definitions shown in 
                                   current standards                          
                                               (e.g., firewalls).             

                                  Appendix II
                     Detailed Results of GAO's Analyses of
                   Architecture Products Used to Date by NASA
                         to Acquire and Implement IFMP

                         (Continued From Previous Page)

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

A listing of accountable organizations and their respective
responsibilities for implementing enterprise security services.
Organizational relationships are important to show in an operational view
because they illustrate fundamental roles (e.g., who conducts operational
activities) and management relationships (e.g., what is the command
structure or relationship to other key players), and how they influence
the operational nodes.

                                       X

A description of operational security X
rules that are derived from security
policies.

A description of enterprise security infrastructure services (e.g.,
identification and authentication) that will be needed to protect the
agency's assets, and the means for implementing such a service (e.g.,
firewalls and intrusion detection software). This description should also
address how these services will align and integrate with other elements of
the architecture (e.g., security policies and requirements).

X	The available architecture products contain high-level descriptions of
enterprise security services; however, in most instances, these products
describe the technology components that will be implemented to provide the
security service, and not the security service. For example, the
architecture products classify "Audit Logs" as a security service;
however, audit logs are generally the function/component within an
"Auditing Service."

The architecture products also do not link the security services to
security policies, procedures, goals, strategies, and requirements.

    A description of the security   X     The available architecture products 
                                                     contain a description of 
standards to be implemented for      various security standards, but it is 
                                                   unclear if these standards 
    each enterprise service. These    are relevant to the "As Is" or "To Be"  
                                      environment or both.                    
standards should be derived from   
                                      In addition, the architecture products  
     security requirements. This      do not contain a                        
description should also address      traceability matrix that links goals, 
                 how                                strategies, requirements, 
    these services will align and      and services to the security standards 
                                                        and security products 
integrate with other elements of       (e.g., SmartCard). They also do not 
                 the                                clearly state whether the 
     architecture (e.g., security              list of security standards for 
               policies                      enterprise services is complete. 
          and requirements).          

                                  Appendix II
                     Detailed Results of GAO's Analyses of
                   Architecture Products Used to Date by NASA
                         to Acquire and Implement IFMP

                         (Continued From Previous Page)

Key architectural element Element satisfied? Explanation of partially
satisfied Yes No Partially

      A description of the    X The available architecture products contain a 
           protection           high-level                                    
    mechanisms that will be       description of protection mechanisms (e.g., 
                                                         firewalls). However, 
implemented to secure the    they do not describe the level of protection  
            agency's            needed and the                                
assets, such as firewalls    types of services the protection mechanisms   
              and               will provide to                               
      intrusion detection       protect IFMP applications that access         
           software,            information/data,                             
including a description of   business services/applications, and the       
              the               various networks.                             
relationships among these    
     protection mechanisms.     In addition, the architecture products do not 
                                contain a                                     
                                traceability matrix that links goals,         
                                strategies, requirements,                     
                                and services to the security standards, so it 
                                                           is unclear whether 
                                  this is the definitive list of protection   
                                                 mechanisms.                  

Performance

A description of the processes for X
establishing, measuring, tracking,
evaluating, and predicting business
performance regarding business
functions, baseline data, and service
levels.

A description of customer-focused X
measurable business goals and
outcomes for business products and
services through the execution of
financial and financial-related
business activities.

A description of measurable X
technical goals and outcomes for
managing technology products and
services for the "To Be" architecture
that enable the achievement of
business goals and outcomes.

Source: GAO analysis of NASA data.

aA CRUD (create, read, update, and/or delete) matrix shows the specific
business functions and applications that create, read, update, and/or
delete specific data elements, which enables the organization to develop
applications.

bBusiness and operational rules define specific constraints for the data,
such as security needs (e.g., confidentiality and accessibility of data)
and actions that should or should not occur, such as updating or deleting
data.

cThe framework that NASA is using for architecture development does not
identify a work product that supports the creation of business rules.

dAlthough the framework that NASA is using identifies a logical database
model as a work product, the available architecture products do not
include such a model, and there was contradictory evidence in the
architecture products that stated that NASA considered this model to be
nonessential. As a result, it is unclear whether the agency plans to
produce a logical database model as part of its architecture description.

eMetadata is "data about data" that enables automation and consistent
management and use of information, such as rules and standards.

Appendix II
Detailed Results of GAO's Analyses of
Architecture Products Used to Date by NASA
to Acquire and Implement IFMP

fThe framework that NASA is using does not identify the metadata model as
a type of work product nor does the agency's action plan address the
development of a metadata model for later inclusion in the architecture
description.

gWhile the framework that NASA is using does identify the application
portfolio as a type of work product, the agency's action plan does not
address the development of this portfolio for later inclusion in the
architecture description.

hThe technical reference model (TRM) describes how technology is
supporting the delivery of service components, including relevant
standards for implementing the technology. The TRM is a generally accepted
representation of the generic components of an information system. It
allows designers, developers, and users to agree on definitions, have a
common understanding of the services to be provided, and identify and
resolve issues affecting such requirements as interoperability,
portability, reliability, scalability, and serviceability.

iExamples of enterprise services include application services, such as Web
services, and collaboration services, such as instant messaging and video
conferencing.

jTechnical standards are strict rules and protocols governing how a given
enterprise service is to be implemented.

kNISSU was initially established to support the deployment of the IFMP
core financial module. However, NASA is now considering using this
technical infrastructure to support the deployment of other enterprise
applications that have yet to be identified.

                                  Appendix II
                     Detailed Results of GAO's Analyses of
                   Architecture Products Used to Date by NASA
                         to Acquire and Implement IFMP

             Detailed Analysis of NASA's Transition Plan Artifacts

Element satisfied? Key architectural element Yes No Partially Explanation
of partially satisfied Transition plan

Analysis of the gaps between the X
baseline and target architecture for
business processes,
information/data, and
services/application systems to
define missing and needed
capabilities.

A high-level strategya for implementing the enterprise architecture,
including specific timephased milestones for acquiring and deploying
systems, performance metrics, and financial and nonfinancial resource
needs.

This strategy should include:

o  A listing of the legacy systems that will not be part of the "To Be"
environment and the schedule for terminating these systems.

o  A description of the training strategy/approach that will be
implemented to address the changes made to the business operations
(processes and systems) to promote operational efficiency and
effectiveness. This plan should also address any changes to existing
policies and procedures affecting day-to-day operations, as well as
resource needs (staffing and funding).

o  A list of the systems to be developed/acquired/modified to achieve
business needs and a description of the relationship between the system
and the business need(s).

o  A strategy for employing enterprise application integration (EAI)
plans, methods, and tools to, for example, provide for efficiently reusing
applications that already exist concurrent with adding new applications
and databases.

X	The transition plan identifies only the core financial legacy systems
that have been or will be retired. It does not identify all legacy systems
or provide a schedule for terminating these systems.

X	The transition plan provides a high-level description of a training
strategy and references a change management strategy for IFMP. It also
identifies a business driver for improving human capital management within
the organization.

However, the architecture does not (1) address training needs for
nonfinancial business operations, (2) contain training plans, (3) identify
changes to existing policies and procedures, or (4) estimate resource
needs. Moreover, this generic strategy was developed without the benefit
of a gap analysis.

X	The transition plan notes that there is a list of project development
initiatives, such as core financial and travel manager, but it does not
provide a complete list of systems to be developed or acquired to achieve
business needs (e.g., human resources and budget).

X	The transition plan contains a description of an EAI strategy for IFMP
applications. However, the transition plan does not state whether this
strategy would be applied to other agencywide application systems.

Appendix II
Detailed Results of GAO's Analyses of
Architecture Products Used to Date by NASA
to Acquire and Implement IFMP

                         (Continued From Previous Page)

Element satisfied? Key architectural element Yes No Partially Explanation
of partially satisfied

A technical migration plan (systems, infrastructure, and data) that shows:

(a)	the transition from legacy to replacement systems with explicit
"sunset" dates and intermediate systems that may be temporarily needed to
sustain existing functionality during the transition period;

(b)	an analysis of system interdependencies, including the level of effort
required to implement related systems in a sequenced portfolio of projects
that includes milestones, timelines, costs, and capabilities; and

(c)	a cost estimate for the initial phase(s) of the transition and
high-level cost projection for transition to the target architecture.

                                       X

                                       X

                                       X

A description of the architecture X
governance and control structure
and the integrated procedures,
processes, and criteria (e.g.,
investment management, security)
to be followed to ensure that the
agency's business transformation
effort remains compliant with the
architecture.

Source: GAO analysis of .NASA data

aAcquisition/business strategy is a plan or action for achieving a
specific goal or result through contracting for software products and
services.

Appendix III

Assessment of NASA's EA Management Efforts against GAO's Architecture
Management Maturity Framework

            Stage Core element           Satisfied?         Comments          
Stage 1: EA awareness Agency is aware    Yes     In December 2002, the CIO 
                  of EA.                                    issued a          
                                                       memorandum stating the 
                                                           agency's intent to 
                                                     develop and use an EA.   

Stage 2: Building Adequate resources Yes According to the chief technology 
          the         exist (funding,                          officer (CTO), 
     EA management   people, tools, and       the agency has adequate program 
                        technology).                            funding. NASA 
      foundation                                  estimates that it will cost 
                                                      $750,000 to develop its 
                                             EA for fiscal years 2001 through 
                                                           2003. Further, the 
                                                   agency reports that it has 
                                                    skilled staff (government 
                                            employees and contractors) for    
                                            its architecture                  
                                                program. In addition, NASA is 
                                                              using automated 
                                                tools and technology, such as 
                                                             Rational Rose by 
                                            Rational Software Corporation/IBM 
                                            Software                          
                                                         Group.               

Committee or group representing  No   NASA has not assigned responsibility 
                               the                             for directing, 
     enterprise is responsible for       overseeing, or approving the EA to a 
                        directing,                               committee or 
      overseeing, or approving the      group comprising representatives from 
                               EA.                                 across the 
                                                                      agency. 
    Program office responsible for Yes   In December 2002, NASA established a 
                                EA                                    program 
       development and maintenance          office that is responsible for EA 
                           exists.                            development and 
                                                                 maintenance. 
           Chief architect exists. Yes   In January 2003, NASA designated the 
                                                                   CTO as the 
                                                             chief architect. 

EA is being developed using a Yes The EA is being developed using the
Zachman framework, methodology, and automated framework. According to the
CTO, the agency is tool. also using a defined methodology to develop the

EA.a In addition, NASA is using automated tools, such as Rational Rose by
Rational Software Corporation/IBM Software Group, to build the EA.

EA plans call for describing both Yes According to the CTO, the plansb for 
                                 the                                   the EA 
             "As Is" and the "To Be"      provide for describing both the "As 
                     environments of                              Is" and the 
        the enterprise, as well as a               "To Be" environments and a 
                          sequencing                         sequencing plan. 
     plan for transitioning from the     
                          "As Is" to     
                        the "To Be."     

EA plans call for describing both the
"As Is" and the "To Be" environments in
terms of business, performance,
information/data, application/service, and
technology.

Yes	According to the CTO, the plansb for the EA provide for describing
both the "As Is" and the "To Be" environments in terms of business,
performance, information/data, application/service, and technology.

EA plans call for business, performance, information/data,
application/service, and technology descriptions to address security.

Yes	According to the CTO, the plansb for the EA provide for addressing
security for the "As Is" and "To Be" environments.

Stage 2: Building EA plans call for       Yes    According to the CTO, the 
          the        developing metrics for                 plansb for the EA 
     EA management   measuring EA progress,            provide for developing 
                            quality,                       metrics to measure 
      foundation     compliance, and return                progress, quality, 
                     on investment.                 compliance, and return on 
                                                         investment.          

                                  Appendix III
                       Assessment of NASA's EA Management
                       Efforts against GAO's Architecture
                         Management Maturity Framework

                         (Continued From Previous Page)

       Stage            Core element       Satisfied?        Comments         
                                                      According to the CTO,   
      Stage 3:        Written/approved         No     the agency is revising  
     Developing     organization policy               its                     
                       exists for EA                  existing policy to      
    EA products         development.                  require the development 
                                                      of an                   
(includes all                                      EA. As written, the     
      elements                                        policy requires the CIO 
                                                      to                      
from stage 2)                                      develop an information  
                                                          technology (IT)     
                                                      architecture, which is  
                                                       one aspect of an EA.   
                   EA products are under       No      According to the CTO,  
                       configuration                    EA products are not   
                                                          currently under     
                        management.                        configuration      
                                                            management.       
                  EA products describe or     Yes      According to the CTO,  
                       will describe                   the plansb for the EA  
                  both the "As Is" and the               products provide for 
                          "To Be"                      describing the "As Is" 
                                                                      and the 
                  environments of the                 "To Be" environments,   
                  enterprise, as well as              as well as a sequencing 
                  a sequencing plan for                        plan.          
                  transitioning from                  
                   the "As Is" to the "To             
                            Be."                      

    Both the "As Is" and the "To Be" Yes According to the CTO, the plansb for 
                                         the EA                               
       environments are described or      provide for describing both the "As 
                             will be                          Is" and "To Be" 
     described in terms of business,       environments in terms of business, 
                                                                 performance, 
      performance, information/data,              information/data,           
                                               application/service, and       
            application/service, and                                          
                         technology.                              technology.

Business, performance, information/data, Yes According to the CTO, the
plansb for the EA
application/service, and technology provide for the business, performance,
descriptions address or will address information/data,
application/service, and
security. technology descriptions addressing security for the

"As Is" and "To Be" environments.

Progress against EA plans is No        According to the CTO, the agency is 
                       measured                                     measuring 
                  and reported.      and reporting progress against EA plans; 
                                                                     however, 
                                       NASA was unable to provide evidence of 
                                                                        these 
                                                                     reports. 

Stage 4: Completing EA products (includes all elements from stage 3)
Written/approved organization policy exists for EA maintenance.

No	There is no written/approved policy for EA maintenance.

    EA products and management processes No According to the CTO, management
                                 processes are

undergo independent verification and validation.

independently verified and validated, but EA products do not undergo
independent verification and validation. According to the CTO, EA products
will be subject to independent verification and validation in the future.

EA products describe both the "As Is" and No According to the CTO, the
plansb for the EA the "To Be" environments of the provide for describing
both the "As Is" and the enterprise, as well as a sequencing plan "To Be"
environments of the enterprise, as well as for transitioning from the a
sequencing plan for transitioning from the "As Is" "As Is" to the "To Be."
to the "To Be." However, the initial version of

NASA's EA was not provided to us in time to determine if its products
address this core element. Therefore, our analysis is based on the
products that were used to date to guide and constrain IFMP.

                                  Appendix III
                       Assessment of NASA's EA Management
                       Efforts against GAO's Architecture
                         Management Maturity Framework

                         (Continued From Previous Page)

                     Stage Core element Satisfied? Comments

Stage 4: Completing EA products

Both the "As Is" and the No According to the CTO, the plansb for the EA "To Be"
                         environments are described in

provide for describing both the "As Is" and "To Be" environments in terms
of business, performance, information/data, application/service, and
technology. However, the initial version of NASA's EA was not provided to
us in time to determine if its products address this core element.
Therefore, our analysis is based on the products that were used to date to
guide and constrain IFMP.

(includes all elements from stage 3)

terms of business, performance, information/data, application/service, and
technology.

Business, performance, information/data, No According to the CTO, the
plansb for the EA application/service, and technology provide for the
business, performance, descriptions address security. information/data,
application/service, and

technology descriptions addressing security. However, the initial version
of NASA's EA was not provided to us in time to determine if its products
address this core element. Therefore, our analysis is based on the
products that were used to date to guide and constrain IFMP.

Organization CIO has approved No The CIO approved the initial version of   
                         current    the EA.                                   
                  version of EA.    However, the initial version of NASA's EA 
                                                                      was not 
                                       provided to us in time to determine if 
                                                                 its products 
                                    address this core element. Therefore, our 
                                                                     analysis 
                                      is based on the products that were used 
                                                                   to date to 
                                                    guide and constrain IFMP. 

Committee or group representing the enterprise or the investment review
board has approved current version of EA.

No	According to the CTO, the plansb for the EA do not provide for approval
by a committee or group representing the enterprise or the investment
review board.

 Quality of EA products is measured and No According to the CTO, the quality of
              EA products is reported. not measured and reported.

      Stage 5:        Written/approved     No    There is no written/approved 
     Leveraging     organization policy                   policy requiring IT 
     the EA for   exists for IT investment     investment compliance with the 
      managing    compliance with                             EA. The current 
       change               EA.               policy requires the CIO to      
                                              ensure that new IT              
(includes all                              investments are in alignment    
      elements                                with technology                 
from stage 4)                              architectures, which are one    
                                              aspect of an EA.                
                                                   According to the CTO, this 
                                                     policy is being revised. 

Process exists to formally manage EA Yes According to the CTO, there is a
process for change. formally managing EA change.

EA is integral component of IT investment No Since the EA is currently
being developed and has

management process.	not been used to date in acquiring and implementing
IFMP, it is not part of the investment management process.

EA products are periodically updated. Yes	According to NASA, it plans to
update the EA quarterly through June 2004, and semiannually thereafter.

                                  Appendix III
                       Assessment of NASA's EA Management
                       Efforts against GAO's Architecture
                         Management Maturity Framework

                         (Continued From Previous Page)

       Stage        Core element    Satisfied?            Comments            
      Stage 5:     IT investments       No          Since the EA is currently 
     Leveraging    comply with EA.                    being developed and has 
     the EA for                                   not been used to date in    
      managing                                         acquiring and          
       change                                    implementing IFMP, it is not 
                                                       part of the investment 
(includes all                                                              
      elements                                      management process.
                  Organization head                                           
from stage 4)  has approved          No     The organization head has not  
                  current                      approved the
                   version of EA.                current version of the EA.   
                    Return on EA                                              
                    investment is       No          Metrics and processes for 
                      measured                          measuring EA benefits
                    and reported.                have not been developed, and 
                                                        an initial version of 
                                               the EA has not been completed, 
                                               thus precluding                
                                                    return-on-investment      
                                                        measurement.          

Compliance with EA is measured No Metrics for measuring EA compliance have 
                              and                                         not 
                        reported.      been developed, and an initial version 
                                                                    of the EA 
                                     has not been completed, thus precluding  
                                      measuring and reporting on compliance.  

Source: GAO analysis of NASA data.

aAccording to NASA, its methodology is from the following: Melissa A.
Cook, Building Enterprise Information Architectures: Reengineering
Information Systems, Prentice Hall Inc. (1996).

bWe requested NASA's architecture program plan, but NASA did not provide
it.

Appendix IV

Comments from the National Aeronautics and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix IV
Comments from the National Aeronautics
and Space Administration

Appendix V

                     GAO Contacts and Staff Acknowledgments

GAO Contact Cynthia Jackson, (202) 512-5086

Acknowledgments	Staff making key contributions to this report were Sophia
Harrison, Anh Le, Randolph Tekeley, William Wadsworth, and Lillian Whren.

GAO's Mission	The General Accounting Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.

Order by Mail or Phone	The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548

To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

To Report Fraud, 	Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm

Waste, and Abuse in E-mail: [email protected]

Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470

Public Affairs	Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

                               Presorted Standard
                              Postage & Fees Paid
                                      GAO
                                Permit No. GI00

United States
General Accounting Office
Washington, D.C. 20548-0001

Official Business
Penalty for Private Use $300

Address Service Requested
*** End of document. ***