Information Technology: FAA Has Many Investment Management	 
Capabilities in Place, but More Oversight of Operational Systems 
Is Needed (20-AUG-04, GAO-04-822).				 
                                                                 
The Federal Aviation Administration's (FAA) mission is to promote
the safe, orderly, and expeditious flow of air traffic in the	 
United States airspace system, commonly referred to as the	 
National Airspace System (NAS). To maintain its ability to	 
effectively carry out this mission FAA embarked, in 1981,on a	 
multi-billion dollar effort to modernize its aging air traffic	 
control (ATC) system, the principle technology component of the  
NAS. Yet the NAS modernization has continued to be plagued by	 
cost increases, schedule delays, and performance shortfalls. To  
gain insight into how FAA is meeting its management challenges,  
congressional requesters asked GAO to evaluate FAA's processes	 
for making IT investment management decisions. The objectives of 
this review included (1) evaluating FAA's capabilities for	 
managing its IT investments and (2) determining what plans, if	 
any, the agency might have for improving these capabilities.	 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-04-822 					        
    ACCNO:   A11881						        
  TITLE:     Information Technology: FAA Has Many Investment	      
Management Capabilities in Place, but More Oversight of 	 
Operational Systems Is Needed					 
     DATE:   08/20/2004 
  SUBJECT:   Air traffic control systems			 
	     Information technology				 
	     Internal controls					 
	     Schedule slippages 				 
	     Cost overruns					 
	     Investment planning				 
	     FAA Air Traffic Control System			 
	     FAA National Airspace System Plan			 
	     GAO Information Technology Investment		 
	     Management Framework				 
                                                                 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-04-822

                 United States Government Accountability Office

                     GAO Report to Congressional Requesters

August 2004

                                  INFORMATION
                                   TECHNOLOGY

FAA Has Many Investment Management Capabilities in Place, but More Oversight of
                         Operational Systems Is Needed

                                       a

GAO-04-822

August 2004

INFORMATION TECHNOLOGY

FAA Has Many Investment Management Capabilities in Place, but More Oversight of
Operational Systems Is Needed

Judged against the criteria of GAO's framework for information technology
investment management (ITIM), which measures the maturity of an
organization's investment management processes, FAA has established about
80 percent of the basic selection and control practices that it needs to
manage its mission-critical investments (see table below). For example,
business lines actively monitor projects throughout their life cycles.
However, the agency's senior IT investment board does not regularly review
investments that are in the "in-service management," or operational, phase
of their life cycles, and this creates a weakness in FAA's ability to
oversee more than $1 billion of its IT investments. In addition, the
agency has not yet established the key practices that would allow it to
manage all of its investments as one portfolio-an integrated set of
competing options. Until FAA has established the practices that would
enable it to effectively manage its annual IT budget of about $2.5
billion, agency executives lack assurance that they are selecting and
managing the mix of investments that best meets the agency's needs and
priorities.

The agency has initiated efforts to improve its investment management
processes, but it has not yet developed and implemented a comprehensive
plan-supported by management-to guide all of its improvement efforts. Such
a plan is crucial in helping FAA to coordinate and prioritize its
improvement efforts and sustain its commitment to the efforts it already
has under way. Without such a plan-and controls for implementing it-FAA
will be unlikely to develop a mature investment management capability.

Summary of Results for Foundational Critical Processes and Key Practices

Key practices

executed for

Critical process Purpose NAS systems

Highlights of GAO-04-822, a report to congressional requesters

The Federal Aviation Administration's (FAA) mission is to promote the
safe, orderly, and expeditious flow of air traffic in the United States
airspace system, commonly referred to as the National Airspace System
(NAS). To maintain its ability to effectively carry out this mission FAA
embarked, in 1981,on a multibillion dollar effort to modernize its aging
air traffic control (ATC) system, the principle technology component of
the NAS. Yet the NAS modernization has continued to be plagued by cost
increases, schedule delays, and performance shortfalls. To gain insight
into how FAA is meeting its management challenges, congressional
requesters asked GAO to evaluate FAA's processes for making IT investment
management decisions. The objectives of this review included (1)
evaluating FAA's capabilities for managing its IT investments and (2)
determining what plans, if any, the agency might have for improving these
capabilities.

To strengthen FAA's investment management capability, GAO recommends that
FAA develop and implement a plan to address the weaknesses identified in
this report. In commenting on a draft of this report, the Department of
Transportation commented that the report was balanced and fair, showing
where FAA has many capabilities in place and identifying areas that need
improvement.

www.gao.gov/cgi-bin/getrpt?GAO-04-822.

To view the full product, including the scope and methodology, click on
the link above. For more information, contact David Powner, 202-512-9286,
[email protected] or Lester Diamond, 202-512-7957, [email protected].

To define and establish an appropriate IT investment Instituting an
management structure and the processes for selecting, investment board
controlling, and evaluating IT investments. 88% Meeting business To ensure
that IT projects and systems support the needs organization's business
needs and meets users' needs. 86% Selecting an To ensure that a
well-defined and disciplined process is investment used to select new IT
proposals and reselect ongoing

                                     investments.                     70% 
                   To review the progress of IT projects and         
        Providing  systems, using                                    
       investment  pre-defined criteria and checkpoints, in meeting  
                                         cost,                       
        oversight   schedule, risk, and benefit expectations and to  
                                         take                        
                   corrective action when these expectations are not 
                   being                                             
                                         met.                         57% 
                   To make available to decision makers information  
        Capturing                         to                         
       investment  evaluate the impacts and opportunities created by 
       information     proposed (or continuing) IT investments.      100% 

Overall 79%

Source: GAO.

Contents

  Letter

Results in Brief
Background
FAA Has Established an IT Management Structure to Manage Its

NAS Investments FAA Has Initiated Efforts to Improve Its Investment
Management Process DOT is Taking Steps to Integrate Oversight of FAA's IT

Investments Conclusions Recommendations for Executive Action Agency
Comments

1 2 3

19

40

41 42 43 44

Appendixes

Appendix I: Objectives, Scope, and Methodology 45

Appendix II:	Investment Management Process Used by Some Organizational
Units to Manage Non-NAS Investments 48

Tables	Table 1: Table 2:

Table 3:

Table 4: Table 5: Table 6: Table 7: Table 8: Table 9:

NAS and Non-NAS IT Investments
Stage 2 Critical Processes-Building the Investment
Foundation
Summary of Results for Stage 2 Critical Processes and Key
Practices for NAS Investments
Instituting the Investment Board
Meeting Business Needs
Selecting an Investment
Providing Investment Oversight
Capturing Investment Information
Stage 3 Critical Processes-Developing a Complete
Investment Portfolio

                                       5

18

19 21 25 28 31 34

37 38

Table 10: Status of Stage 3 Critical Processes

                                   FAA's Life Cycle Management Process        
Figures Figure 1: Figure 2:   Detailed Breakdown of FAA's Life Cycle     8
                                               Management                  
                                                 Process                   12 
                     Figure 3:    The Five ITIM Stages of Maturity with    
                                                Critical                   
                                                Processes                  16 

Contents

Abbreviations

ABA Financial Services unit
AHR Human Resource Management unit
AIO Information Services unit
AMS Acquisition Management System
APB acquisition program baseline
ARA Research and Acquisition unit
ARC Region and Center Operations unit
ATC air traffic control
ATO Air Traffic Organization
ATS Air Traffic Services unit
AVR Regulation and Certification unit
CIO Chief Information Officer
DOT Department of Transportation
ECG En Route Communications Gateway
F&E facilities and equipment (predeployment stage of system life

cycle) FAA Federal Aviation Administration FTI FAA Telecommunications
Infrastructure IAT Investment Analysis Team IT information technology ITEB
Information Technology Executive Board ITIM Information Technology
Investment Management framework ITIPS Information Technology Investment
Portfolio System JRC Joint Resources Council NAS National Airspace System
NSIP NAS Support Integration Process OMB Office of Management and Budget
OPS operations (postdeployment stage of system life cycle) PIR
postimplementation review VCSU VSCS Control Subsystem Upgrade VSCS Voice
Switching and Control System

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

A

United States Government Accountability Office Washington, D.C. 20548

August 20, 2004

The Honorable Tom Davis Chairman Committee on Government Reform U.S. House
of Representatives

The Honorable Adam H. Putnam

Chairman

Subcommittee on Technology, Information Policy, Intergovernmental
Relations and the Census Committee on Government Reform U.S. House of
Representatives

The Federal Aviation Administration's (FAA) mission is to promote the
safe, orderly, and expeditious flow of air traffic in the United States
airspace system, commonly referred to as the National Airspace System
(NAS). To maintain its ability to effectively carry out this mission FAA
embarked, in 1981, on a multibillion dollar effort to modernize its aging
air traffic control (ATC) system, the principle technology component of
the NAS. Over the past 2 decades, individual FAA modernization projects
have experienced cost overruns, schedule delays, and performance
shortfalls of large proportions. Because of the size, complexity, cost,
and problemplagued past of FAA's modernization program, we have designated
it a highrisk information technology investment since 1995.1

This report is one in a series of reports responding to your request to
evaluate FAA's efforts to address the information technology (IT)
management challenges it faces as it continues to modernize the ATC
system. It focuses on FAA's processes for making IT investment management
decisions and uses our Information Technology Investment Management (ITIM)
framework,2 which was released at a hearing of the subcommittee on March
3, 2004. The framework provides a method for

1See GAO, High-Risk Series: An Overview, GAO/HR-95-1 (Washington, D.C.:
February 1995); GAO, High-Risk Series: Information Management and
Technology, GAO/HR-97-9 (Washington, D.C.: February 1997); GAO, High-Risk
Series: An Update, GAO/HR-99-1 (Washington, D.C.: January 1999); GAO,
High-Risk Series: An Update, GAO-01-263 (Washington, D.C.: January 2001);
and GAO, High-Risk Series: An Update, GAO-03-119 (Washington, D.C.:
January 2003).

2GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington, D.C.:
March 2004).

evaluating and assessing how well an agency is selecting and managing its
IT resources. As agreed, our objectives were to (1) evaluate FAA's
capabilities for managing its IT investments, (2) determine what plans the
agency might have for improving these capabilities, and (3) describe the
Department of Transportation's (DOT) oversight of FAA's investments and
investment management process. To address these objectives we analyzed
documents and interviewed agency officials to (1) validate and update
FAA's self-assessments of the key practices in the framework, (2) evaluate
FAA's plans for improving its capabilities, and (3) describe the
department's oversight role. We performed our work from October 2003,
through July 2004, in accordance with generally accepted government
auditing standards. Appendix I contains further details on our objectives,
scope, and methodology.

Results in Brief	FAA has established most-about 80 percent-of the basic
practices needed to manage its mission critical investments, including
many of the foundational practices for selecting and controlling IT
investments. These key practices provide additional assurance that the
investments selected will meet organizational needs and will be completed
on time and within budget. The practices also will enable the agency to
manage its IT investments as a portfolio, or integrated set of competing
options.

Even with these many capabilities in place, weaknesses remain in several
areas. Specifically, FAA

o 	does not involve its senior IT investment board in regular reviews of
investments that have entered the in-service management phase, that is,
those systems that have completed development and become operational;

o 	does not have standard practices for managing its mission-support and
administrative investments;

o 	has not developed a process where the senior IT investment board
regularly reviews the full portfolio of investments; and

o 	has not implemented postimplementation reviews of its major investments
to validate that they are providing the expected benefits after they
become operational.

FAA has begun to act to resolve the weaknesses described above, but until
FAA establishes the practices it needs to effectively manage its IT
investments, executives cannot be assured that they are selecting and
managing the mix of investments that best meets the agency's needs and
priorities. Establishing the capabilities needed to effectively manage
investments requires the development and implementation of a plan,
supported by management, that defines and prioritizes improvements to the
investment process. While FAA has initiated a series of efforts to improve
its investment management processes, it does not have such a plan. Without
this plan-and controls for implementing it-it is unlikely that the agency
will effectively establish mature investment management capabilities.

The Department of Transportation has recently initiated several efforts
that can serve to provide better departmental oversight of FAA
investments. For example, DOT is finalizing capital investment guidance
for all of its operating administrations to follow in implementing their
investment management processes, and it has initiated a process for
reviewing the fiscal year 2006 budget justifications for major programs,
including those of FAA. The department has also identified about a dozen
programs it plans to monitor on a regular basis and has asked FAA to
report cost, schedule, and performance data on some of its programs
quarterly.

To further strengthen FAA's investment management capability, we are
recommending that the agency develop and implement a plan aimed at
addressing the weaknesses identified in this report.

In commenting on a draft of this report, the Department of
Transportation's Director of Audit Relations stated that GAO did a good
job of keeping the report balanced and fair, showing where FAA has many
capabilities in place and identifying areas that need improvement. The
agency also provided a technical comment, which we have incorporated into
the report.

                                   Background

FAA's Mission and As an agency of the Department of Transportation, FAA's
mission is to

Organizational Structure	promote the safe, orderly, and expeditious flow
of air traffic in the national airspace. To fulfill its mission requires
the extensive use of technology. The achievement of the agency's mission
is also dependent in large part on the

skills and expertise of its workforce. Its workforce of nearly 50,000
people provides aviation services that include air traffic control;
maintenance of air traffic control equipment; and certification of
aircraft, airline operations, and pilots.

FAA is organized into several staff support offices (examples include the
Office of Information Services and the Office of Human Resource
Management) and five lines of business, which include Airports, Regulation
and Certification, Commercial Space Transportation, the Office of Security
and Hazardous Materials, and the newly formed Air Traffic Organization
(ATO). The ATO was formed on February 8, 2004, to better provide safe,
secure, and cost-effective air traffic services now and into the future.
The Air Traffic Services and the Research and Acquisitions units, which
had been primarily responsible for managing air traffic services within
FAA, were combined into one performance-based organization to create ATO.
ATO is led by FAA's Chief Operating Officer and consists of 10 service
units.3

FAA's Use of IT	FAA relies extensively on information technology to carry
out its NAS operations. It constantly depends on the adequacy and
reliability of the nation's ATC system, which comprises a vast network of
radars; automated data processing, navigation, and communications
equipment; and ATC facilities.4 Through this system, FAA provides services
such as controlling takeoffs and landings and managing the flow of traffic
between airports. For example, the Integrated Terminal Weather System is
employed to allow maximum use of airport runways in all kinds of weather
through a variety of weather sensors. The Wide Area Augmentation System is
used to provide

3The 10 service units that make up the ATO include Safety, Communications,
Operations Planning, Finance, Acquisition & Business Services, En Route
and Oceanic Services, Terminal Services, Flight Services, System
Operations Services, and Technical Operations Services.

4FAA uses three types of facilities to control traffic: airport towers,
terminal radar approach control facilities, and en route centers. Airport
towers direct traffic on the ground, before landing, and after takeoff
within 5 nautical miles from the airport and about 3,000 feet above the
airport. Terminal radar approach control facilities sequence and separate
aircraft as they approach and leave airports, beginning about 5 nautical
miles and ending about 50 nautical miles from the airport and generally up
to 10,000 feet above the ground. Air route traffic control centers, called
en route centers, control planes in transit and during approaches to some
airports, generally controlling air space that extends above 18,000 feet
for commercial aircraft.

vertically guided landing to aircraft at thousands of airports and
airstrips where there is currently no vertically guided landing
capability.

FAA also relies on IT to carry out its mission-support and administrative
operations (non-NAS operations). For example, FAA uses IT to support
accident and incident investigations, security inspections, and personnel
and payroll functions.

With an IT budget of about $2.5 billion for fiscal year 2004, FAA accounts
for over 90 percent of the Department of Transportation's IT budget. The
amount of investments in both NAS and non-NAS IT is shown in the table 1
below.

                    Table 1: NAS and Non-NAS IT Investments

                                                       Total IT investment in 
           Type of                                        fiscal year 2004 in 
        investment           Funding of IT investments    billions of dollars 
               NAS      Facilities and Equipment (F&E) 
                   (development through 2 years of                      1.464 
                   operations)                         
                   Operations (OPS) (through the rest  
                   of the life                         
                                                cycle)                  0.834 
           Non-NAS                          Operations           0.350-0.500a 

Source: FAA.

aAccording to FAA, these numbers will be verified via baselining by the
end of Fiscal Year 2004 to reflect non-NAS IT assets and their costs.

    Prior Reviews Identified Weaknesses in the Agency's IT Investment Management
    Process

In 1995, we designated FAA's modernization of its air traffic control
system, the principle technology component of the NAS, as a high-risk area
because of the size and complexity of the program and FAA's many failures
in meeting projects' cost, schedule, and performance goals. In our latest
High-Risk Series, issued in January 2003,5 we addressed the critical need
for FAA to continue to improve its investment management practices-the
management processes the agency uses to select, control and evaluate the
benefits realized from its IT spending-because the agency would be
spending nearly $16 billion more through FY 2007, after having already
spent $35 billion since 1981. Other reports have also noted weaknesses in

5GAO-03-119.

FAA's IT investment management processes and have made a number of
recommendations to address this area.6 For instance, last year we reported
that while FAA had improved its processes, several issues remained
unresolved. We noted, for example, that the agency had not yet implemented
processes for evaluating projects after implementing them, in order to
identify lessons learned and improve the investment management process.

    FAA's Current Approach to Investment Management

Process for Managing NAS Investments

FAA's process for managing an IT investment varies depending on the type
of investment-NAS systems in development through the second year of
operation (F&E), NAS systems in operation after the second year (OPS), and
non-NAS systems each follow different processes. NAS investments are
managed through a standardized process, the FAA Acquisition Management
System (AMS), and non-NAS investments are managed through a number of
different processes.

In April 1996, FAA implemented its AMS in response to legislation that
directed the agency to develop a new acquisition management system.7
Because of FAA's contention that some of its modernization problems were
caused by federal acquisition regulations, the Congress enacted
legislation in November 1995 that exempted the agency from most federal
procurement laws and regulations and directed FAA to develop and implement
a new acquisition management system that would address the unique needs of
the agency. AMS was intended to reduce the time and cost for fielding new
products and services by introducing (1) a new investment management
system that spans the entire life cycle of an acquisition, (2) a new
procurement system that provides flexibility in selecting and managing
contractors, and (3) organizational and human capital reforms that support
the new investment and procurement systems.

AMS provides high-level acquisition policy and guidance for selecting and
controlling FAA's NAS investments through all phases of the acquisition
life cycle, which is organized into a series of phases and decision points
that

6GAO, Air Traffic Control: FAA's Modernization Investment Management
Approach Could Be Strengthened, GAO/RCED/AIMD-99-88 (Washington, D.C.:
Apr. 30, 1999); GAO, Air Traffic Control: FAA's Modernization
Efforts-Past, Present, and Future, GAO-04-227T (Washington, D.C.: Oct. 30,
2003).

749 U.S.C. 40110(d).

include (1) mission analysis, (2) investment analysis, (3) solution
implementation, and (4) in-service management. To select investments, FAA
has established two processes-mission analysis and investment
analysis-which together constitute a set of policies, procedures, and
guidance that enhance the agency's ability to screen projects that are
submitted for funding. Also, through these two processes FAA is to assess
and rank each project based on its relative costs, benefits, risks, and
contribution to FAA's mission, and a senior, corporate-level
decisionmaking group selects projects for funding. After a project has
been selected, FAA officials are required to formally establish the life
cycle cost, schedule, benefits, and performance baselines that are used to
monitor the project's status throughout the remaining phases of the
acquisition management life cycle. See figure 1 for a graphic depiction of
FAA's life cycle management process.

Figure 1: FAA's Life Cycle Management Process

Source: GAO based on FAA documents.

Note: During the front end of the life cycle, research and system analysis
activities are undertaken to discover applications of new technology for
FAA's present services, explore new opportunities for service delivery,
solve problems within current operations, and define requirements.

Several groups are involved in managing FAA's NAS investments; they
perform functions from analysis of mission needs and alternative
investments through system development, implementation, operation, and,
ultimately, disposal. The roles and responsibilities of each group are
described below:

Joint Resources Council (JRC)-This board makes corporate-level resource
and investment decisions and establishes investment programs. Members
include Associate Administrators representing FAA's lines of

business, the FAA Acquisition Executive,8 the Chief Financial Officer, the
Chief Information Officer (CIO), and the Assistant Administrators for
System Safety, for Policy, Planning and International Aviation, and for
Region and Center Operations. The board is supported by the JRC
Secretariat Team, a group that facilitates the board's processes by
maintaining the meeting calendar and guidance documents, developing
records of decisions, and providing advisory and liaison support to
programs.

Systems Engineering/Operational Analysis Team-This team performs
affordability assessments for newly proposed investments and prepares
recommendations for the reprogramming of funds from lower priority
programs. It also prepares annual budget submissions for approval by the
JRC. This team is composed of representatives from each line of business
and from other functional disciplines and is chaired by the Director,
System Architecture and Investment Analysis.

Investment Analysis Team (IAT)-This team is assembled for a relatively
short period for each specific investment being considered, to conduct the
detailed analysis of alternatives that will lead to selecting and
recommending a preferred acquisition solution. The team draws experts from
the integrated product teams,9 the organizational unit with the need, the
investment analysis staff,10 and other organizations.

Corporate Mission Analysis Organization-Performs agency-level mission
analysis and coordinates service area analysis, an activity that is
conducted during mission analysis to (1) identify capability shortfalls
for or in

8May be delegated to an associate administrator.

9These teams may operate as entities or be organized into subintegrated
product teams or product teams to develop, procure, and deliver products
and services for users or customers. They are responsible for the
acquisition of new or improved capability for services and products
throughout their life cycles and for developing cost and schedule
baselines for candidate solutions during investment analysis.

10The investment analysis staff assists and oversees the work of all the
investment analysis teams, is responsible for all investment analyses, and
is responsible for developing the tools, techniques, and databases to
ensure quality performance of investment analysis on behalf of the JRC.

conjunction with service organizations,11 (2) ensure alignment with agency
strategic goals, and (3) eliminate redundant activity, duplicate benefits,
service gaps, and service overlaps. It also develops and maintains
standards and tools for conducting service area analysis, and it assists
service organizations in establishing a service area analysis capability.

In addition to identifying the roles and responsibilities of the groups
involved in the management process, AMS provides guidance on the documents
and decisions that result from each of the life cycle phases. For example,
through the mission analysis phase, FAA identifies critical needs that the
agency must meet for improving the safety, capacity, efficiency, and
effectiveness of the NAS. Approval of a mission need statement by the JRC
signifies that the agency agrees that the need is critical enough to
proceed to the next phase-investment analysis. During the investment
analysis phase, the IAT is to analyze and recommend a solution that best
satisfies FAA's performance goals and customer service needs. This team is
then to rank each proposed project based on a number of factors, including
how well it meets mission needs compared to other projects and whether it
has a favorable cost-benefit ratio. As part of the JRC selection process,
the life cycle cost, schedule, benefits, and performance baselines are
established in a formal document called the acquisition program baseline
(APB), which is designed to be used by program offices to monitor a
project's status in achieving those baselines throughout the remaining
phases of the acquisition management life cycle.

The solution implementation phase begins when the JRC approves and funds a
project, establishes its acquisition program baseline, and authorizes the
service organizations to implement and manage the project over its life
cycle. After the project has been implemented and is in operation (FAA's
inservice management phase), the service organizations monitor and assess
operational performance. Also during this phase, the project is monitored
to determine whether the current capability satisfies the demand for
services or whether another solution offers the potential for improving

11A service organization is any organization within FAA that delivers a
service, whether it is a business unit, project office, program
directorate, or integrated product team, or whether it is engaged in air
traffic services, security, regulation, certification, operations,
commercial space transportation, or airport development. These
organizations are responsible and accountable for managing service
delivery throughout the life cycle. Investment decisions are made to
support service delivery. Specifically, after the investment decision has
been made, the service organization assumes responsibility for the
investment program, implements the selected solution, and manages the
product throughout the in-service management phase of its life cycle.

safety or effectiveness or for significantly lowering costs. If the
current capability is lacking, FAA initiates a process whereby the mission
need would be revalidated and the investment analysis process begun again,
possibly leading to a new investment decision. Figure 2 provides detail on
the phases of FAA's IT investment management process and decision points.
The highlighted decision points represent those for which the JRC must
make an approval decision before a project can move forward.

Senior executives have stated that with the reorganization of the ATO in
February 2004, discussions have been held about realigning the investment
management process to make the heads of the service units responsible and
accountable for managing programs' capital investments and operating costs
from inception to retirement. In the past, the business units have been
organized to manage either capital investments or operating costs, but not
both. These discussions have not yet led to specific changes in FAA's
investment management processes and responsibilities.

Figure 2: Detailed Breakdown of FAA's Life Cycle Management Process

Decision point for which the JRC must make an approval decision before a
project can move forward. Source: GAO based on FAA documents.

aCorporate requirements organization takes the lead in planning for the
concept and requirements definition phase of the life cycle management
process.

bIn a February 2004 memorandum, FAA's Chief Operating Officer assumed the
in-service decision authority and stated that he would delegate this
responsibility to the vice presidents of the service organizations, unless
the JRC retained the in-service decision authority. If the JRC retains
this authority, it determines the in-service decision authority at the
time of the final investment decision.

Process for Managing Non-NAS While the AMS was intended to apply to all
FAA investment programs, it

Investments	has not been implemented for non-NAS investments. Each of the
agency's business line and staff offices that manage non-NAS investments12
has implemented its own processes for managing these investments. Examples
of these various non-NAS investment processes include the following:

o 	Regarding an investment management board structure, the Financial
Services staff office has an informal board consisting of the Chief
Financial Officer, Deputy Chief Financial Officer, and heads of offices
within Financial Services. The Financial Services life cycle process guide
directs the board's operations. In the Regulation and Certification unit,
the senior management team makes investment management decisions with
input from the Chief Information management team. This unit is developing
an IT investment management processes guide, which is expected to be
completed by the end of the fiscal year.

o 	When selecting investments, the Human Resource Management unit uses its
established annual budget formulation process, while the Region and Center
Operations unit is moving toward a new process whereby in order to be
selected investments need to demonstrate, at a minimum, that they (1) are
compliant with FAA's architecture, (2) have a business sponsor, (3) have a
solid business case, and (4) can be funded.

o 	In controlling investments, Information Services has developed
processes to monitor contract expenditures, and unit managers regularly
perform financial management reviews of the programs under their purview,
but there is no structured process for oversight of projects' performance
against expectations. In the Human Resource Management unit, division
managers hold quarterly reviews to assess projects' progress in meeting
cost and schedule expectations and aligning with strategic goals.

12Non-NAS business units include Information Services (AIO), Region and
Center Operations (ARC), Regulation and Certification (AVR), Financial
Services (ABA), Research and Acquisition (ARA), Air Traffic Services
(ATS), and Human Resource Management (AHR).

Descriptions of the processes used by each of the units responsible for
managing non-NAS investments can be found in appendix II.

In January 2004, the FAA Administrator established the Information
Technology Executive Board (ITEB) to "strengthen FAA's ability to use IT
as an agencywide strategic asset" and "guide fundamental changes in the
governance of IT assets." Its charter calls for the ITEB to assume
responsibility for making investment decisions about non-NAS IT
investments. However, the ITEB has not yet implemented this aspect of its
charter. Therefore, at the current time there is no single board or
investment management process for non-NAS investments that would be
analogous to the JRC board and AMS process that are used for NAS
investments.

ITIM Maturity Framework	The ITIM framework is a maturity model composed of
five progressive stages of maturity that an agency can achieve in its
investment management capabilities.13 It was developed on the basis of our
research into the IT investment management practices of leading
private-and publicsector organizations. The framework identifies critical
processes for making successful IT investments, organized into the five
increasingly mature stages. These maturity stages are cumulative; that is,
in order to attain a higher stage of maturity, the agency must have
institutionalized all of the requirements for all of the lower stages, in
addition to those for the higher stage.

The ITIM can be used both to assess the maturity of an agency's investment
management processes and as a tool for organizational improvement. The
overriding purpose of the framework is to encourage investment processes
that increase business value and mission performance, reduce risk, and
increase accountability and transparency in the decision process. We have
used the framework in several of our evaluations,14 and a number of

13GAO-04-394G.

14GAO, Information Technology: DLA Needs to Strengthen Its Investment
Management Capability, GAO-02-314 (Washington, D.C.: Mar. 15, 2002); GAO,
United States Postal Service: Opportunities to Strengthen IT Investment
Management Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002); GAO,
Information Technology: Departmental Leadership Crucial to Success of
Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept. 12,
2003); and GAO, Bureau of Land Management: Plan Needed to Sustain Progress
in Establishing IT Investment Management Capabilities, GAO-03-1025
(Washington, D.C.: Sept. 12, 2003).

agencies have adopted it. These agencies have used ITIM for purposes
ranging from self-assessment to redesign of their IT investment management
processes.

ITIM's five maturity stages represent steps toward achieving stable and
mature processes for managing IT investments. Each stage builds on the
lower stages; the successful attainment of each stage leads to improvement
in the organization's ability to manage its investments. With the
exception of the first stage, each maturity stage is composed of "critical
processes" that must be implemented and institutionalized in order for the
organization to achieve that stage. These critical processes are further
broken down into key practices that describe the types of activities that
an organization should be performing to successfully implement each
critical process. An organization may be performing key practices from
more than one maturity stage at the same time. This is not unusual, but
efforts to improve investment management capabilities should focus on
becoming compliant with lower-stage practices before addressing
higher-stage practices.

Stage 2 of the ITIM framework encompasses building a sound investment
management process by establishing basic capabilities for selecting new IT
projects. It also involves developing the capability to control projects
so that they finish predictably within established cost and schedule
expectations and the capability to identify potential exposures to risk
and put in place strategies to mitigate that risk. The basic selection
processes established in Stage 2 lays the foundation for more mature
selection capabilities in Stage 3.

Stage 3 requires that an organization continually assess both proposed and
ongoing projects as parts of a complete investment portfolio-an integrated
and competing set of investment options. It focuses on establishing a
consistent, well-defined perspective on the IT investment portfolio and
maintaining mature, integrated selection (and reselection), control, and
evaluation processes, which are to be evaluated during postimplementation
reviews (PIR). This portfolio perspective allows decision makers to
consider the interaction among investments and the contributions to
organizational mission goals and strategies that could be made by
alternative portfolio selections, rather than relying exclusively on the
balance between the costs and benefits of individual investments.

Stages 4 and 5 require the use of evaluation techniques to continuously
improve both the investment portfolio and investment processes in order

to better achieve strategic outcomes. At Stage 4 maturity an organization
has the capacity to conduct IT succession activities and therefore can
plan and implement the deselection of obsolete, high-risk, or low-value IT
investments. An organization with Stage 5 maturity conducts proactive
monitoring for breakthrough information technologies that will enable it
to change and improve its business performance. Organizations implementing
Stages 2 and 3 have in place the selection, control, and evaluation
processes that are required by the Clinger-Cohen Act. Stages 4 and 5
define key attributes that are associated with the most capable
organizations.

Figure 3 shows the five maturity stages and the critical processes
associated with each.

Figure 3: The Five ITIM Stages of Maturity with Critical Processes

Source: GAO.

As defined by the model, each critical process consists of "key practices"
that must be executed to implement the critical process.

    FAA Has Established Many Key Practices for Managing NAS Investments but
    Lacks Oversight of Operational Systems

In order to have the capabilities to effectively manage IT investments, an
agency should, at a minimum, (1) build an investment foundation by putting
basic, project-level control and selection practices in place (Stage 2
capabilities) and (2) manage its projects as a portfolio of investments,
treating them as an integrated package of competing investment options and
pursuing those that best meet the strategic goals, objectives, and mission
of the agency; and it should also conduct PIRs to maintain mature,
integrated selection, control, and evaluation processes (Stage 3
capabilities). In addition, an agency would be well served by implementing
capabilities for improving its investment process through performance
evaluations of its portfolio and succession management of current
investments (Stage 4 capabilities). In order to develop the capabilities
to effectively manage its investments, FAA would, at minimum, need to
implement Stage 2 capabilities for both its NAS and non-NAS investments
and Stage 3 capabilities for its portfolio of investments.

FAA's investment management capabilities vary depending on whether an
investment is considered to be NAS or non-NAS. Specifically:

o 	For NAS investments, FAA has executed 30 of the 38 Stage 2 key
practices that are required to establish a foundation for investment
management maturity. For these investments, the agency has in place a
strong set of processes to support investment management, although the JRC
does not regularly review investments that have passed into the in-service
management phase (i.e., operational systems).

o 	For its non-NAS investments, the agency has not yet adequately
implemented a single management line of responsibility and the standard
processes needed to manage in a consistent manner. Although some
structured processes exist within individual business units, this lack of
consistency undermines the agency's maturity.

o 	In Stage 3, the lack of regular JRC oversight of operational systems
and the absence of a structured approach to managing non-NAS investments
prevent FAA from managing its investments as a portfolio that includes all
major NAS and non-NAS investments. In addition, the agency is not
conducting PIRs on its major investments.

o 	FAA has not executed any of the Stage 4 key practices for managing the
succession of its information systems, although the agency has begun to
address this weakness by defining procedures for retiring investments in
the AMS.

When FAA implements all of the key practices associated with building the
investment foundation and managing its investments as a portfolio, the
agency will have greater assurance that it has selected the mix of
investments that best supports its strategic goals and that it will be
able to manage the investments to successful completion.

    FAA Has Established Much of the Foundation Needed to Manage Its NAS
    Investments

At the ITIM Stage 2 level of maturity, an organization has attained
repeatable, successful IT project-level investment control processes and
basic selection processes. Through these processes, the organization can
identify expectation gaps early and take appropriate steps to address
them. According to ITIM, critical processes at Stage 2 include (1)
defining IT investment board15 operations, (2) identifying the business
needs for each IT investment, (3) developing a basic process for selecting
new IT proposals and reselecting ongoing investments, (4) developing
projectlevel investment control processes, and (5) collecting information
about existing investments. Table 2 describes the purpose of each of the
Stage 2 critical processes.

Table 2: Stage 2 Critical Processes-Building the Investment Foundation Critical
                                process Purpose

Instituting the investment board	To define and establish an appropriate IT
investment management structure and the processes for selecting,
controlling, and evaluating IT investments.

Meeting business needs	To ensure that IT projects and systems support the
organization's business needs and meets users' needs.

Selecting an investment	To ensure that a well-defined and disciplined
process is used to select new IT proposals and reselect ongoing
investments.

Providing investment oversight	To review the progress of IT projects and
systems, using pre-defined criteria and checkpoints, in meeting cost,
schedule, risk, and benefit expectations and to take corrective action
when these expectations are not being met.

Capturing investment information	To make available to decision makers
information to evaluate the impacts and opportunities created by proposed
(or continuing) IT investments.

Source: GAO.

15An IT investment board is a decision-making body, made up of senior
program, financial, and information managers, that is responsible for
making decisions about IT projects and systems based on comparisons and
trade-offs among competing projects, with an emphasis on meeting mission
goals.

To its credit, FAA has put in place about 80 percent of the key practices
associated with managing its NAS investments through the Stage 2 critical
processes. The agency has satisfied all of the key practices associated
with capturing investment information and most of those associated with
instituting the investment board, meeting business needs, selecting an
investment, and providing investment oversight. Most of the weaknesses in
these critical processes relate to NAS investments in the in-service
management phase. Table 3 summarizes the status of FAA's critical
processes for Stage 2, showing how many key practices FAA has executed in
managing its NAS investments.

Table 3: Summary of Results for Stage 2 Critical Processes and Key
Practices for NAS Investments

                                Critical process

Key practices executed

                                           Total required by critical process

Percentage of key practices executed

                     Instituting the investment                
                                          board       7                     8 
                         Meeting business needs       6                     7 
                        Selecting an investment       7                    10 
                           Providing investment                
                                      oversight       4                     7 
                           Capturing investment                
                                    information       6                     6 
                                          Total       30                   38 

                                  Source: GAO.

  FAA Has Established an IT Management Structure to ManageIts NAS Investments

The establishment of decision-making bodies or boards is a key component
of the IT investment management process. At the Stage 2 level of maturity,
organizations define one or more boards, provide resources to support
their operations, and appoint members who have expertise in both
operational and technical aspects of proposed investments. The boards
operate according to a written IT investment process guide that is
tailored to the organization's unique characteristics, thus ensuring that
consistent and effective management practices are implemented across the
organization. Once board members are selected, the organization ensures
that they are knowledgeable about policies and procedures for managing
investments. Organizations at the Stage 2 level of maturity also take
steps to ensure that executives and line managers support and carry out
the

decisions of the IT investment board. According to ITIM, an IT investment
management process guide should be a key authoritative document that the
organization uses to initiate and manage IT investment processes and
should provide a comprehensive foundation for the policies and procedures
that are developed for all of the other related processes. (The complete
list of key practices is provided in table 4.)

FAA has executed 7 of the 8 key practices for this critical process. For
example, in 1996, Congress directed FAA to develop a new acquisition
management system as part of a broad mandate for acquisition reform at the
agency.16 In response, FAA implemented AMS in April 1996. AMS establishes
policy and guidance for all aspects of the agency's acquisition life cycle
and documents the investment management process used for NAS investments.
The agency established the JRC as its corporate-level investment board for
the NAS investments. The JRC makes select and control decisions, including
corporate decisions on mission needs, acquisition investments, and
acquisition program baseline changes; it also reviews and recommends
approval of the agency's F&E budget submission.

The board is adequately resourced to support its operations. The JRC
Secretariat Team supports the board in such ways as developing and
updating guidance, scheduling meetings, and preparing and executing the
JRC readiness process. In addition, the Mission Analysis Steering Group17
is responsible for assisting the board in prioritizing mission needs,
while the Systems Engineering/Operational Analysis Team is to assist in
addressing budget issues among investments. The JRC consists of senior
officials from both business and IT areas, including the Chief Information
Officer and the associate administrators representing FAA lines of
business. These members are to exhibit the core competencies required by
FAA in selecting executives and in assessing executive training needs. In
addition, the agency offers a 3-day AMS overview course for all employees,
including JRC members. Although the board as an entity does not oversee
the development and maintenance of AMS, it is involved through FAA's
Acquisition System Advisory Group, which evaluates all proposed changes to
AMS. To ensure that the board's decisions are carried out, an acquisition
program baseline document is approved at the JRC final investment

1649 U.S.C. 40110(d).

17An advisory group, composed of representatives from each line of
business, that establishes guidelines for conducting mission analysis and
developing mission need statements as well as resolving agencywide mission
analysis issues.

decision point; this document identifies the capabilities, benefits,
costs, and schedule for the approved investment, which are monitored by
FAA through its variance reporting process.

Despite these strengths, FAA has not yet clearly defined the relationship
between the JRC and the newly formed ITEB. Although the ITEB was
established by the Administrator to function as the central authority
responsible for assuring that FAA IT investments are based on sound
business practices, FAA has not yet clearly delineated the specific roles
the ITEB is to play and the relationship it will have with the JRC. This
task has been assigned to the ITEB as a longer-range initiative.

Table 4 shows the rating for each key practice required to implement the
critical process for instituting the investment board at the Stage 2 level
of maturity. Each of the "Executed" ratings shown below represents
instances where, based on the evidence provided by FAA officials, we
concluded that the specific key practices were executed by the
organization.

 Table 4: Instituting the Investment Board Type of practice Key Practice Rating
                              Summary of evidence

Organizational commitments

1.	 An enterprisewide IT investment board composed of senior executives
from IT and business units is responsible for defining and implementing
the organization's IT investment governance process.

Executed	The JRC, FAA's corporate-level investment board for the NAS
investments, is responsible for defining and implementing the agency's IT
investment governance process. It consists of the agency's most senior
executives, including the CIO, Chief Financial Officer, FAA's Acquisition
Executive, and Associate Administrators from its lines of business.

2.    The organization Executed    FAA's AMS sets forth acquisition policy 
                                                   and processes for the JRC. 
      has a documented             Also, the board has established its own    
                                   guidance to implement                      
            IT investment                                AMS core JRC policy. 
        process directing          
          each investment          
      board's operations.          

(Continued From Previous Page)

            Type of practice Key Practice Rating Summary of evidence

Prerequisites 1. Adequate resources, including people, funding, and tools,
are provided for supporting the operations of each IT investment board.

Executed	Adequate resources are provided to support the board's
operations. The JRC Secretariat Team provides such operations support as
developing and updating guidance, scheduling meetings, and preparing and
distributing records of decisions. Two other groups support the JRC
decision-making process. The Mission Analysis Steering Group assists the
board in ranking mission needs, while the Systems Engineering/Operational
Analysis Team provides assistance by performing affordability assessments
for the JRC when it is considering alternatives during investment
analysis.

2.	The board members understand the organization's IT investment
management policies and procedures and the tools and techniques used in
the board's decision-making process.

Executed	JRC members are senior managers representing all agency lines of
business. They include the CIO, Chief Financial Officer, and Associate
Administrators representing FAA lines of business such as Air Traffic
Services. Core executive competencies based on FAA's Executive Success
Profile are used in selecting executives and in assessing executive
training needs. A 3-day AMS overview course is also available.

3.   Each board's span Not executed The JRC is FAA's corporate-level       
                                       investment board for making            
         of authority and               decisions related to NAS investments. 
                                                    In January 2004, the ITEB 
        responsibility is              was established to oversee the         
                                       governance of the agency's IT          
      defined to minimize                assets. However, the ITEB has yet to 
                                               take significant action on the 
                                             charge in its charter to clearly 
         overlaps or gaps               delineate the roles it is to play and 
                                                                          its 
      among the boards.                            relationship with the JRC. 

Activities 1. The enterprisewide investment board has oversight
responsibilities for the development and maintenance of the organization's
documented IT investment process.

Executed	Although the JRC does not directly oversee the development and
maintenance of the FAA's documented investment process, it is involved in
this process through FAA's Acquisition System Advisory Group, which is a
corporate crossfunctional body that evaluates all proposed changes to AMS.
Membership consists of representatives from each line of business as well
as the JRC Secretariat Team. Policy changes that are endorsed by the Group
are presented, via the FAA Acquisition Executive, who is on the JRC, to
the Administrator for approval.

2.     Each investment Executed    The JRC is operating in accordance with 
                                                   its assigned authority and 
        board operates in             responsibility as FAA's corporate-level 
                                                  investment board for making 
      accordance with its          decisions related to NAS investments. The  
                                   charter for the ITEB                       
                                   specifically indicates the ITEB's          
       assigned authority          responsibilities, including for            
                                              making decisions for non-NAS IT 
      and responsibility.                                       acquisitions. 

(Continued From Previous Page)

            Type of practice Key Practice Rating Summary of evidence

3.   The organization Executed      FAA has controls for ensuring that the 
                                                   JRC's investment decisions 
         has established              are carried out as approved. At the JRC 
                                                    final investment decision 
                                       point, an acquisition program baseline 
              management                            document is finalized and 
            controls for          approved, which represents the mutual       
                                  agreement between the                       
           ensuring that          JRC, the provider organization, and the     
                                  user organization                           
      investment boards'                  concerning the expected capability, 
                                                benefits, costs, and schedule 
           decisions are          for the investment program. It also         
                                  establishes performance                     
            carried out.                  metrics for assessing the program's 
                                                                     success. 

                                  Source: GAO.

    FAA Has a Process for Ensuring That its Investments Support Business Needs
    and Meet Users' Needs

Defining business needs for each IT project helps to ensure that projects
and systems support the organization's business needs and meet users'
needs. This critical process ensures that a link exists between the
organization's business objectives and its IT management strategy.
According to ITIM, effectively meeting business needs requires, among
other things, (1) documenting business needs with stated goals and
objectives, (2) identifying specific users and other beneficiaries of IT
projects and systems, (3) providing adequate resources to ensure that
projects and systems support the organization's business needs and meet
users' needs, and (4) periodically evaluating the alignment of IT projects
and systems with the organization's strategic goals and objectives. (The
complete list of key practices is provided in table 5.)

FAA has in place 6 of the 7 key practices for meeting business needs. The
agency's AMS and mission analysis guidance calls for business needs for
both proposed and ongoing IT projects and systems to be identified in the
mission need statement developed during the mission analysis phase. FAA
also has detailed procedures for developing this document that call for
identifying business needs. Resources for ensuring that IT projects and
systems support the organization's business needs and meet users' needs
include service organizations, the Corporate Mission Analysis
Organization, the Mission Analysis Steering Group, and detailed procedures
and associated templates for developing mission need statements. FAA's
specific business mission, with stated goals and objectives, is defined in
the Federal Aviation Administration Flight Plan for fiscal years 2004
through 2008.

Further, FAA defines and documents business needs for both proposed and
ongoing IT projects and identifies users and other beneficiaries during
its mission analysis activities. In addition, the AMS policy calls for
users to participate in project management throughout the FAA life cycle

management process. For the three projects we reviewed,18 we verified that
business needs and specific users and other beneficiaries were identified
and documented in mission needs statements as well as in other documents.
In addition, users are involved in project management throughout the life
cycle of the projects. For example, according to project officials, En
Route Communications Gateway (ECG) users participate in project meetings,
weekly integrated product team status meetings, and monthly En Route
domain national deployment teleconferences. FAA Telecommunications
Infrastructure's (FTI) end users are heavily involved in the "operational
test" period, which determines whether the equipment can be safely
implemented in NAS. VSCS Control Subsystem Upgrade19 users are involved in
the project's life cycle via a Web site through which they review and
comment on project documentation.

Despite these strengths, the JRC has no process for evaluating the
organizational alignment of NAS systems through most of their in-service
management phase (and non-NAS investments, which are described separately
in this report). While the JRC does evaluate the alignment of projects and
systems with organizational goals throughout the systems' development and
2 years into their operations as part of the annual budget formulation
process, it does not use any consistent process to review projects and
systems after that point in their life cycles. For NAS systems in the
in-service management phase, these activities are carried out within the
business unit that owns the system, but the JRC does not regularly oversee
these processes and may go for several years without reviewing a system's
alignment with organizational goals. In-service NAS systems only return to
the JRC if they are judged to require additional funds for correction.
Until FAA establishes a process for periodic evaluation of systems
throughout the in-service management phase and takes corrective actions
when misalignment occurs, the agency will not be able to ensure that these
projects, totaling about $1.3 billion per year, are still continuing to
maintain alignment with the FAA's strategic plans and its business goals
and objectives.

18We reviewed the FAA Telecommunications Infrastructure, En Route
Communications Gateway, and Voice Switching and Control System (VSCS)
Control Subsystem Upgrade (VCSU) projects. The projects are described in
appendix I.

19VCSU is a subcomponent of the VSCS project. We decided to review VCSU
because its investment management process was carried out using FAA's AMS,
whereas the VSCS project was funded before the AMS became part of the
FAA's investment management process.

Table 5 shows the rating for each key practice required to implement the
critical process for meeting business needs at the Stage 2 level of
maturity and summarizes the evidence that supports these ratings.

Table 5: Meeting Business Needs Type of practice Key practice Rating Summary of
                                    evidence

Organizational commitments

1. 	The organization has documented policies and procedures for ensuring
IT projects or systems that support the organization's ongoing and future
business needs.

Executed	AMS and mission analysis guidance contain documented policies and
procedures for identifying the IT projects or systems that support the
organization's ongoing and future business needs.

     Prerequisites 2. The organization has a Executed The Federal Aviation
                     Administration Flight Plan (Strategic

documented business mission with stated goals and objectives.

Plan) for fiscal years 2004 through 2008 defines the agency's mission
goals and objectives.

3.        Adequate resources, Executed FAA has adequate resources for      
                                          ensuring that its IT                
      including people, funding,             projects and systems support the 
                                                      organization's business 
         and tools, are provided          needs and meet users' needs. They   
                                          include service                     
            for ensuring that IT          organizations, the Mission Analysis 
                                                          Steering Group, and 
            projects and systems               the Corporate Mission Analysis 
                                                       Organization. FAA also 
                     support the          has detailed procedures and         
                                          associated templates for            
         organization's business          developing mission need statements. 
           needs and meet users'          
                          needs.          

                    The organization                     AMS policy calls for 
Activities 1.         defines         Executed     business needs for both 
                                                                     proposed 
                 and documents business           and ongoing IT projects and 
                                                   systems to be specified in 
                 needs for both proposed          the mission need statement. 
                                                    We verified that business 
                 and ongoing IT projects               needs were defined and 
                                                   documented in mission need 
                                                   statements for the three   
                      and systems.                   projects we reviewed.    

2.           The organization Executed  FAA policy and procedures call for 
                                                           specific users and 
       identifies specific users           other beneficiaries of IT projects 
                                                            and systems to be 
      and other beneficiaries of          identified. We verified that        
                                          specific users and other            
        IT projects and systems.            beneficiaries were identified for 
                                                        the three projects we 
                                                                    reviewed. 

3.       Users participate in Executed    FAA policies and procedures call 
                                                     for users to participate 
              project management          in project management throughout an 
                                          IT project's or                     
      throughout an IT project's             system's life cycle. We verified 
                                                   that users participated in 
         or system's life cycle.            project management throughout the 
                                                      life cycle of the three 
                                                        projects we reviewed. 

(Continued From Previous Page)

            Type of practice Key practice Rating Summary of evidence

4.     The investment board Not executed The JRC evaluates the alignment   
                                            of systems through                
        periodically evaluates                   development and 2 years into 
                           the                            operations with the 
           alignment of its IT              organization's strategic goals    
                                            and objectives-through            
          projects and systems                  the annual budget formulation 
                                                            process-and takes 
       with the organization's                        corrective actions when 
                                                misalignment occurs. However, 
           strategic goals and                there is no process for the JRC 
                                                     to periodically evaluate 
          objectives and takes              the alignment of investments      
                                            later in their life cycles.       
       corrective actions when              
          misalignment occurs.              

                                  Source: GAO.

    FAA Has a Disciplined Process for Selecting New IT Proposals but Lacks a
    Similar Process for Reselecting Ongoing Investments

Selecting new IT proposals and reselecting ongoing investments requires a
well-defined and disciplined process to provide the agency's investment
board, business units, and developers with a common understanding of the
process and the cost, benefit, schedule, and risk criteria that will be
used both to select new projects and to reselect ongoing projects for
continued funding. According to ITIM, this critical process requires,
among other things, (1) making funding decisions for new proposals
according to an established process; (2) providing adequate resources for
investment selection activities; (3) using a defined selection process to
select new investments and reselect ongoing investments; (4) establishing
criteria for analyzing, prioritizing, and selecting new IT investments and
for reselecting ongoing investments; and (5) creating a process for
ensuring that the criteria change as organizational objectives change.
(The complete list of key practices is provided in table 6.)

FAA has executed 7 of the 10 key practices associated with selecting an
investment. For example, the AMS establishes two processes-mission
analysis and investment analysis-that together constitute a set of
policies and procedures, as well as guidance that is designed to enhance
the agency's ability to select investments. In addition, FAA has policies
and procedures for its annual F&E budget formulation process to reselect
ongoing IT projects. Also, FAA's AMS sets forth policies and procedures
for reselecting ongoing IT investments by identifying their capability
shortfalls and addressing them as new investments.

The AMS also integrates funding with the process of selecting an
investment by requiring the Systems Engineering/Operational Analysis Team
to perform affordability assessments for new proposed investment programs;
it may recommend funding reallocations from lower priority programs when
an alternative solution cannot be funded within FAA

planning and budgeting baselines. This team also supports the JRC to
ensure that the executives' funding decisions are aligned with selection
decisions during the investment analysis activities.

Resources for proposal selection activities include the program director,
the Integrated Product Team, and the Investment Analysis Team, as well as
detailed procedures and a template that have been defined for developing
investment analysis reports. The investment analysis reports identify the
evaluation criteria used, the alternatives analyzed, and the ranking of
each alternative so that the JRC can select the best overall solution
identified in the mission need statement. The criteria that were
established during the initial investment analysis phase are used by the
Investment Analysis Team to rank each proposed project on the basis of how
well it meets the agency's mission needs compared with other projects.

FAA uses the processes defined in the AMS for selecting new IT
investments. In addition, it uses two processes to reselect ongoing IT
investments. Specifically, the FAA uses its annual budget formulation
process for projects in development or in the first 2 years of operations.
It also uses the AMS process when a system's capability shortfall is
identified, and it treats the correction of the shortfall as a new
investment. The managers of the three projects we reviewed confirmed that
their projects were selected using the AMS process. One project's
officials stated that this included market, alternatives, investment, and
affordability analyses. The program managers also stated that the annual
F&E budget formulation process is used to reselect their projects. These
project officials also noted that if a project is scheduled for a hardware
replacement, a reselection is done. The AMS process is followed to explore
new alternatives and make sure the replacement is in the best interest of
the government.

Despite these strengths, FAA has not developed similarly strong processes
for NAS investments more than 2 years into their operations-those NAS
systems that are in the in-service management phase. For example, while
FAA's F&E budget formulation process establishes criteria for analyzing,
prioritizing, and reselecting IT investments for systems in development or
up until 2 years into operations, neither of the two processes used to
reselect IT investments has established criteria for investments beyond 2
years into operations. In addition, while FAA uses its annual budget
formulation process to reselect projects that are part of the F&E budget,
the agency does not have an analogous reselection process as part of its
operations budget formulation. Until FAA establishes consistent criteria
for reselecting all of its IT investments, it will not be adequately
assured that it

is consistently and objectively continuing to fund ongoing projects that
still meet the needs and priorities of the agency in a cost-effective and
riskinsured manner.

Table 6 shows the rating for each key practice required to implement the
critical process for selecting an investment at the Stage 2 level of
maturity and summarizes the evidence that supports these ratings.

                        Table 6: Selecting an Investment

Type of practice    Key practice      Rating      Summary of evidence      
                    1. The organization Executed    FAA's AMS policy, mission 
    Organizational          has                      analysis, and investment 
     commitments    documented policies          analysis guidance has        
                                    and          documented policies and      
                         procedures for          procedures for selecting new 
                              selecting                 IT proposals.         
                     new IT proposals.           

2.       The organization has Executed FAA has documented policies and     
                                          procedures for its                  
         documented policies and                annual F&E budget formulation 
                                                    process, which is used to 
      procedures for reselecting          reselect ongoing IT projects. In    
                                          addition, FAA's AMS                 
         ongoing IT investments.          policy has documented policies and  
                                          procedures for                      
                                           reselecting ongoing IT investments 
                                                         throughout the FAA's 
                                                      acquisition life cycle. 

3.        The organization has Executed  FAA's AMS and investment analysis 
                                                                guidance have 
          documented policies and          documented policies and procedures 
                                                              for integrating 
       procedures for integrating                 funding with the process of 
                                                     selecting an investment. 
      funding with the process of          
         selecting an investment.          

                                                       Adequate resources are 
Prerequisites 1.  Adequate resources,  Executed   provided for identifying 
                                                                          and 
                      including people,             selecting IT projects and 
                          funding,                  systems. They include the 
                       and tools, are                       program director, 
                        provided for                Integrated Product Teams, 
                                                                      and the 
                       identifying and             Investment Analysis Team.  
                        selecting IT               FAA also has detailed      
                    projects and systems.           procedures and associated 
                                                     templates for developing 
                                                      investment analysis     
                                                            reports.          

2.  Criteria for analyzing, Executed      The Investment Analysis Team has 
                                                         established criteria 
             prioritizing, and          for analyzing, prioritizing, and      
                     selecting          selecting new IT                      
             new IT investment          investment opportunities. The         
                                        investment analysis                   
       opportunities have been          report, which is submitted to the     
                                        JRC, identifies the                   
                  established.          evaluation criteria, the alternatives 
                                                            analyzed, and the 
                                                ranking for each alternative. 

3. 	Criteria for analyzing, Not executed While FAA's F&E budget
formulation process has prioritizing, and established criteria for
analyzing, prioritizing, and reselectinga IT investment reselecting IT
investments that are part of that budget, opportunities have been neither
of the two processes used to reselect IT established. investment
opportunities has established criteria for

investments beyond 2 years into operations.

(Continued From Previous Page)

            Type of practice Key practice Rating Summary of evidence

4. 	A mechanism exists to Not executed While FAA ensures that the criteria
continue to reflect ensure that the criteria organizational objectives for
selecting new IT continue to reflect investments, there are no consistent
criteria used by the organizational objectives. JRC to reselect
investments more than 2 years into

                                  operations.

Activities 1. The organization uses  Executed FAA uses the policies and    
                          its                    procedures defined in the    
                   defined selection              AMS, mission analysis, and  
                        process,                     investment analysis      
                                                    guidance to select new IT 
                  including predefined               investments. We verified 
                                                                         that 
                 selection criteria, to                 the three projects we 
                         select                  reviewed were selected using 
                                                                          the 
                                                 mission analysis and         
                  new IT investments.            investment analysis          
                                                 activities                   
                                                  defined in the AMS, mission 
                                                  analysis and the investment 
                                                      analysis guidance.      

2. 	The organization uses the Not executed FAA has a process defined in
the AMS to reselect defined selection process, ongoing IT investments. It
also uses its annual budget including predefined formulation process to
reselect projects that are part of selection criteria, to the F&E budget.
According to the project managers of reselecta ongoing IT the three
projects we reviewed, this budget process is investments. used to reselect
their projects. However, FAA does not

consistently use these defined processes to reselect IT

investments more than 2 years into operations.

3.   Executives' funding Executed The Systems Engineering/Operational      
                                     Analysis Team,                           
      decisions are aligned              which is composed of representatives 
                       with                                from FAA's service 
       selection decisions.          organizations, supports the JRC in       
                                     making funding                           
                                     decisions that are aligned with          
                                     selection decisions as                   
                                            part of FAA's investment analysis 
                                                                  activities. 

Source: GAO.

aAccording to the GAO ITIM framework, reselecting is the periodic
reconsideration of an investment's continuing value to the organization
and the decision to continue funding. It is a recurring process that
continues for as long as a project is receiving funding.

    FAA Does Not Have a Process for Effectively Overseeing Investments in All
    Phases of Their Life Cycles

An organization should provide effective oversight for its IT projects
throughout all phases of their life cycles. Its investment board should
maintain adequate oversight and observe each project's performance and
progress toward predefined cost and schedule expectations as well as each
project's anticipated benefits and risk exposure. The investment board
should also employ early warning systems that enable it to take corrective
action at the first sign of cost, schedule, or performance slippages. This
board has ultimate responsibility for the activities within this critical
process. According to ITIM, effective project oversight requires, among
other things, (1) having written policies and procedures for management
oversight; (2) developing and maintaining an approved management plan for
each IT project; (3) making up-to-date cost and schedule data for each
project available to the oversight boards; (4) having regular reviews by
each investment board of each project's performance against stated

expectations; and (5) ensuring that corrective actions for each
underperforming project are documented, agreed to, implemented, and
tracked until the desired outcome is achieved. (The complete list of key
practices is provided in table 7.)

FAA has in place 4 of the 7 key practices associated with effective
project oversight. The agency has developed written policies and
procedures for management oversight of its investments. These include (1)
AMS; (2) the integrated program plan, which is the detailed planning
document for all aspects of a program's implementation, including program
control; and (3) the Integrated Baseline Establishment and Management
Process document for reporting variances from the performance expectations
approved by the JRC in the acquisition program baseline.

We verified that cost, schedule, benefit, and risk expectations were
documented in the acquisition program baseline and that the integrated
program plan contained details for project execution for En Route
Communications Gateway and FAA Telecommunications Infrastructure. For the
VSCS Control Subsystem Upgrade, performance expectations and details on
project execution were both captured in the integrated program plan.20 In
addition, the JRC Secretariat Team maintains a tracking system for action
items that are assigned during a project's acquisition reviews, including
the action to be taken, the responsible FAA organization, and whether the
underlying problem has been resolved.

FAA has not established processes that bring investments before the JRC
for oversight on a regular basis. There is a process for reporting
variances from the performance expectations that were approved by the JRC
in the investment's acquisition program baseline. However, although this
process is carried out as part of the F&E budget formulation for IT
investments in development or less than 2 years into operations, it is not
being carried out for investments that are part of the operations budget.
Investments that are meeting performance expectations may not return to
the JRC for several years. FAA also conducts acquisition reviews as a
means for program offices to report to agency executives on the status of
investments compared to program baselines. However, since program offices
may select which investments they wish to bring forward for review, many
investments may never come forward. Until FAA develops (1) procedures for
reporting on an investment throughout its entire acquisition life cycle

20According to FAA, no acquisition program baseline was prepared for VCSU.

and (2) mechanisms for ensuring that all investments are reviewed
regularly, the agency is placing itself at risk that underperforming
investments will not be reported to the JRC in order for it to take
appropriate actions.

Table 7 shows the rating for each key practice that is required to
implement the critical process for project oversight at the Stage 2 level
of maturity and summarizes the evidence that supports these ratings.

  Table 7: Providing Investment Oversight Type of practice Key practice Rating
                              Summary of evidence

                                                    FAA has developed written 
Organizational 1. The organization has Executed    policies and procedures 
                                                                          for 
     commitment      documented policies           management oversight of IT 
                             and                   projects and systems.      
                                                   These include AMS, the     
                        procedures for             integrated program plan,   
                                                   and                        
                     management oversight             the Integrated Baseline 
                              of                            Establishment and 
                                                                   Management 
                       IT projects and             Process document for       
                           systems.                reporting variances from   
                                                   the                        
                                                   performance expectations   
                                                   approved in the            
                                                   acquisition                
                                                    program baseline for an   
                                                      investment program.     

Prerequisites 1. Adequate resources, Executed FAA has adequate resources   
                                                 for providing IT project     
                     including people,              oversight. The agency has 
                         funding,                 staff for compiling monthly 
                      and tools, are               variance reports submitted 
                       provided for                 by the investment program 
                        IT project                 areas, preparing quarterly 
                        oversight.                  baseline variance reports 
                                                                          for 
                                                       the JRC, and preparing 
                                                         semi-annual baseline 
                                                                     variance 
                                                           report for the FAA 
                                                  Administrator. An automated 
                                                                       system 
                                                    is used to facilitate the 
                                                   maintenance of information 
                                                                          for 
                                                        these reports.        

2. 	IT projects and systems, including those in steady state (operations
and maintenance), maintain approved project management plans that include
expected cost and schedule milestones and measurable benefit and risk
expectations.

Executed	AMS policy calls for an acquisition program baseline (APB)
document and an integrated program plan to be available at the JRC final
investment decision point. The APB document serves as the AMS
cost/schedule/technical performance/benefits/risks control document. The
integrated program plan specifies how the APB baselines will be controlled
and details the management, contracting, and technical actions and
activities to be performed in executing the acquisition. We verified that
cost, schedule, benefit, risk, and performance expectations were
documented in the APBs for ECG and FTI. For VCSU, these expectations were
documented in an integrated program plan.

                    Data on actual                      FAA has established a 
Activities 1.      performance      Not executed     process for reporting 
                                                                 variances to 
                   (including cost,                 the JRC from the          
                       schedule,                    performance expectations  
                                                    that have                 
                                                     been approved by the JRC 
                   benefit, and risk                        in the APB for an 
                                                                  investment. 
                   performance) are                 This process is carried   
                       provided                     out for IT investments    
                                                    that are                  
                                                      part of the F&E budget, 
                 to the appropriate IT                    but it is not being 
                                                              carried out for 
                                                         investments that are 
                   investment board.                   managed as part of the 
                                                                   operations 
                                                             budget.          

(Continued From Previous Page)

            Type of practice Key practice Rating Summary of evidence

2. Using verified data, each Not executed FAA does not have a process that
provides an investment board regularly opportunity for the JRC to
regularly review investment reviews the performance of performance. It has
a process for conducting acquisition IT projects and systems reviews where
program offices provide status against stated information to agency
executives on the progress of expectations. investments against their
acquisition program baselines. However, the individual program offices
choose which investments they want to discuss at these reviews. Also,
although the process for reporting variances from the performance
expectations approved by the JRC in the acquisition program baseline is
carried out for IT investments that are part of the F&E budget, this
process is not being carried out for investment programs that are part of
the operations budget, and it only results in investments with variances
to be reviewed.

3. 	For each underperforming Not executed During acquisition reviews,
action items are identified for IT project or system, investment programs
discussed, an organization appropriate actions are assigned responsibility
to carry them out, and the items taken to correct or are tracked until the
appropriate action is taken, at which terminate the project or time they
are closed out. Similarly, variance reports are system in accordance with
prepared quarterly for the JRC identifying investments defined criteria
and the with a 10 percent or greater variance from the documented policies
and established acquisition program baseline. An investment procedures for
program is to remain on the quarterly variance report management
oversight. until successful corrective action is taken. However, FAA has
no mechanism that provides assurance that every program has an acquisition
review regularly, since it is left to the individual program offices to
decide which programs they want discussed at the reviews. Also, although
variance reports are prepared for IT investments that are part of the F&E
budget, reports are not prepared for investments that are part of the
operations budget.

4.        The investment board Executed The JRC's Secretariat Team         
                                           maintains a tracking               
             regularly tracks the          system for action items assigned   
                                           during a project's                 
                implementation of            acquisition reviews. This system 
                                                     identifies the action to 
      corrective actions for each          be taken, what FAA organization is 
                                           to perform it, and                 
          underperforming project               whether it is open or closed. 
            until the actions are          
                       completed.          

                                  Source: GAO.

    FAA Has a Structured Process for Capturing Investment Information and Using
    It to Support Investment Management

To make good IT investment decisions, an organization must be able to
acquire pertinent information about each investment and store that
information in a retrievable format. During this critical process an
organization identifies its IT assets and creates a comprehensive
repository of investment information. This repository provides information
to investment decision makers to help them evaluate the impacts and
opportunities that would be created by proposed or continuing investments.
It can provide insights and trends about major IT cost and management
drivers. The repository can take many forms and does not have to be
centrally located, but the collection method should identify each IT
investment and its associated components. This critical process may be
satisfied by the information contained in the organization's current
enterprise architecture, augmented by additional information-such as
financial information and information on risk and benefits-that the
investment board may require to ensure that informed decisions are being
made. According to ITIM, effectively managing this repository requires,
among other things, (1) developing written policies and procedures for
identifying and collecting the information, (2) assigning responsibility
for ensuring that the information being collected meets the needs of the
investment management process, (3) identifying IT projects and systems and
collecting relevant information to support decisions about them, and (4)
making the information easily accessible to decision makers and others.
(The complete list of key practices is provided in table 8.)

FAA's AMS guidance identifies specific information that is needed in the
investment management process, including information for its investment
analysis phase. FAA maintains a number of repositories of relevant
information, including its Simplified Program Information Reporting &
Evaluation database, which reports variances in cost, schedule,
performance, or benefits from an investment's approved acquisition program
baseline. The information that is collected is made available to the JRC
in several documents, including program plans and the acquisition program
baseline document. The JRC Secretariat Team ensures that the investment
board has all the relevant information it needs for its decisionmaking
process.

Table 8 shows the rating for each key practice required to implement the
critical process for capturing investment information at the Stage 2 level
of maturity and summarizes the evidence that supports these ratings.

Table 8: Capturing Investment Information

Type of practice      Key practice       Rating     Summary of evidence    
                                                        FAA has developed     
                     1. The organization   Executed  policies and procedures  
    Organizational           has                               for            
                     documented policies            identifying and           
     commitments             and                    collecting information to 
                                                    support the               
                            procedures for              investment management 
                           identifying and          process. For example, AMS 
                    collecting information            guidance indicates what 
                                  about IT          information is needed for 
                                                                          the 
                      projects and systems          investment analysis phase 
                                to support             of FAA's investment    
                        the investment                 management process.    
                          management                
                           process.                 

2. 	An official is assigned Executed AMS guidance specifies which
officials are responsible responsibility for ensuring that for approving
the completion of reports containing the information collected during
information prepared for the investment management project and systems
identification process. The JRC Secretariat Team ensures through meets the
needs of the its readiness process that all of the necessary investment
management information is available to the JRC for its decision process.
making.

Prerequisite 1. Adequate resources,  Executed FAA has adequate resources   
                   including                     for meeting this key         
                   people, funding, and              practice. Several teams, 
                        tools, are               including a mission analysis 
                       provided for              team and an investment       
                      identifying IT             analysis team, collect the   
                   projects and systems                   relevant investment 
                           and                      information needed by the 
                                                                       JRC to 
                   collecting relevant           make its decisions on which  
                        investment               investments to approve.      
                    information about                The JRC Secretariat Team 
                          them.                      ensures that the JRC has 
                                                 all the relevant information 
                                                 for its decision making.     

Activities 1.  The organization's IT  Executed FAA maintains data relevant 
                        projects                       to the investment      
                     and systems are                    management process in 
                     identified, and               several sources, including 
                                                                          its 
                 specific information is          financial management        
                 collected to                     system, its Simplified      
                                                  Program                     
                 support decisions about              Information Reporting & 
                          them.                      Evaluation tool, and its 
                                                                      Capital 
                                                  Investment Plan. Also, AMS  
                                                      guidance identifies     
                                                  information to be collected 
                                                      for the investment      
                                                  management                  
                                                  process-including for the   
                                                  investment                  
                                                  analysis phase-to aid the   
                                                  JRC in its final investment 
                                                          decisions.          

2. The information that has been Executed Information collected for the
JRC decision-making collected is easily accessible and process is compiled
in documents such as detailed understandable to decision program plans,
acquisition program baselines, and makers and others. investment analysis
reports. The JRC Secretariat Team ensures that the JRC has all the
relevant information for its decision making through its readiness
process.

3.         The information Executed  The JRC Secretariat Team, through its 
                repository is                                   JRC readiness 
           used by investment          process, collects information for the  
                     decision          JRC to use in its                      
         makers and others to          decision-making process. The JRC also  
                      support          receives an                            
       investment management.                 investment analysis report that 
                                                          contains all of the 
                                           information that has been gathered 
                                                            during investment 
                                                         analysis activities. 

                                  Source: GAO.

    FAA Does Not Have Structured Processes to Manage Its Non-NAS Investments

FAA does not have a single set of processes for making consistent basic
selection and control decisions for its non-NAS investments (Stage 2
capabilities). As previously discussed in the background section of this
report, several business units within FAA make decisions about non-NAS
investments. We reviewed the investment management processes of seven of
these units-Information Services, Region and Center Operations, Regulation
and Certification, Financial Services, Research and Acquisition, Air
Traffic Services, and Human Resource Management. Appendix II describes the
investment management processes we found in these units. The extent to
which these processes comply with the ITIM framework for Stage 2 varies
considerably by business unit, and FAA currently does not specify non-NAS
investment management processes in a coordinated manner. Since the ITIM
framework calls for a consistent investment management process, we
assessed FAA's non-NAS investment management capability at an aggregate
level. That is, we assessed FAA's capability to manage its non-NAS
investments, not the capability of each individual business unit. Even
though individual business units may have some of these processes in
place, FAA as a whole has not yet defined

o
an investment management structure that allows the agency to consistently
manage its non-NAS investments,

o
a uniform process for ensuring that non-NAS investments are linked to
business needs and meet users' needs,

o
a process for selecting new IT proposals and reselecting ongoing
investments,

o
a single process for reviewing the progress of investments and taking
corrective action when performance expectations are not being met, or

o
a comprehensive inventory of project and system information to support
investment decisions.

According to FAA officials, the agency has not defined a coherent
investment management structure and a set of processes for non-NAS
investments in the past because many of these investments have not had the
agencywide impact of the NAS investments. However, because there is now
recognition that a disciplined approach to managing non-NAS investments
could help control FAA's IT assets and costs in general, efforts are
currently under way to address this weakness. As previously discussed, an
IT Executive Board (ITEB) has been chartered with responsibility for,

among other things, making decisions about non-NAS IT investments, but it
has not yet taken action on developing a standard process. Until FAA fully
establishes the consistent practices it needs to make basic project
selection and control decisions, executives will be hampered in their
ability to effectively manage non-NAS investments and ultimately to find
the opportunities to achieve the cost savings they are seeking.

    FAA Lacks Key Capabilities Needed to Manage All IT Investments as a
    Portfolio and Does Not Conduct Postimplementation Reviews

During Stage 3, the investment board enhances the IT investment management
process by developing a complete investment portfolio and carrying out
PIRs. An IT investment portfolio is an integrated, agencywide collection
of investments that are assessed and managed collectively on the basis of
common criteria. Managing investments within the context of such a
portfolio is a conscious, continuous, and proactive approach to expending
limited resources on an organization's competing initiatives in light of
the relative benefits expected from these investments. Taking an
agencywide perspective enables an organization to consider its investments
comprehensively, so that collectively the investments optimally address
the organization's missions, strategic goals, and objectives. Managing IT
investments with a portfolio approach also allows an organization to
determine priorities and make decisions about which projects to fund, and
continue to fund, based on analyses of the relative organizational value
and risks of all projects, including projects that are proposed, under
development, and in operation. For an organization to reap the full
benefits of the portfolio process, it should collect all of its
investments into an enterprise-level portfolio that is overseen by its
senior investment board. Although investments may initially be selected
into subordinate portfolios-based on, for example, lines of business or
life cycle stages-and managed by subordinate investment boards, they
should ultimately be aggregated into this enterprise-level portfolio.

The purpose of a PIR is to evaluate an investment after its development
has been completed (i.e., after its transition from the implementation
phase to the in-service management phase) in order to validate actual
investment results. This review is conducted to (1) examine differences
between estimated and actual investment costs and benefits and their
possible ramifications for unplanned funding needs in the future and (2)
extract "lessons learned" about the investment selection and control
processes that can be used as the basis for management improvements.
Similarly, PIRs should be conducted for investment projects that were
terminated before completion, to help to readily identify potential
management and process improvements.

According to ITIM, critical processes performed by Stage 3 organizations
include (1) defining the portfolio criteria, (2) creating the portfolio,
(3) evaluating the portfolio, and (4) conducting PIRs. Table 9 shows the
purpose of each critical process in Stage 3.

 Table 9: Stage 3 Critical Processes-Developing a Complete Investment Portfolio
                            Critical process Purpose

Defining the portfolio criteria	To ensure that the organization develops
and maintains IT portfolio selection criteria that support its mission,
organizational strategies, and business priorities.

Creating the portfolio	To ensure that IT investments are analyzed
according to the organization's portfolio selection criteria and that an
optimal IT investment portfolio with manageable risks and returns is
selected and funded.

Evaluating the portfolio	To review the performance of the organization's
investment portfolio(s) at agreed-upon intervals and to adjust the
allocation of resources among investments as necessary.

Conducting postimplementation reviews	To compare the results of recently
implemented investments with the expectations that were set for them and
to develop a set of lessons learned from these reviews.

Source: GAO.

FAA has executed only 1 of the 27 key practices associated with Stage 3
critical processes: it has a process for distributing portfolio criteria
to project management personnel and other stakeholders. The remaining 26
key practices were not executed-primarily because FAA does not involve the
JRC in the regular oversight of non-NAS investments or in NAS investments
during their in-service management phase, weaknesses that we noted in our
assessment of Stage 2 requirements. Since Stage 3 requires an
enterprisewide perspective, the lack of oversight of these classes of
investments precludes the successful completion of most Stage 3 critical
processes. In addition, Stage 3 requires an enterprisewide perspective
that FAA has not adopted, which would enable the JRC to oversee all major
IT investments, regardless of life cycle phase or business unit. Although
it can be appropriate for FAA to manage its NAS, in-service NAS, and
non-NAS investments as separate subordinate portfolios-depending on the
successful execution of all Stage 2 key practices-its enterprise-level
portfolio should contain all major IT investments regardless of life cycle
stage or business line. In building this enterprise-level portfolio, the
JRC can choose whether to include specific investments based on
predetermined criteria, as described by the ITIM framework. Until FAA
fully implements the critical processes associated with managing its
investments as a complete portfolio, it will not have the data or

enterprisewide perspective it needs to make informed decisions about all
of its major IT investments.

In addition, FAA has not executed the six key practices for conducting
PIRs. In June 2004, in response to a recommendation contained in our 1999
report21 that FAA initiate PIRs for projects or programs within 3 to 12
months of deployment or termination, the NAS Configuration Management and
Evaluation Staff developed a proposed approach to PIRs, but this approach
was not implemented. In November 2003, the life cycle management policy
team proposed a change to the AMS that would require conducting these
reviews, but there has been no action on the proposal. Although the JRC
has recently reaffirmed its commitment to implement PIRs, there is no
policy and no established process to carry them out. If PIRs are not
conducted on a routine basis, then FAA will not be able to effectively
evaluate the results of its IT investments; this will affect the agency's
ability to determine whether to continue, modify, or terminate an IT
investment in order to meet its stated mission objectives.

Table 10 summarizes the status of FAA's critical processes for Stage 3,
showing how many associated key practices it has executed.

        Table 10: Status of Stage 3 Critical Processes Critical process

Key practices executed

                                           Total required by critical process

Percentage of key practices executed

                   Defining the portfolio criteria      1                   7 
                            Creating the portfolio      0                   7 
                          Evaluating the portfolio      0                   7 
                                        Conducting              
                        postimplementation reviews      0                   6 
                                            Totals      1                  27 

Source: GAO.

21GAO/RCED/AIMD-99-88.

    FAA Has Not Established a Process for Managing the Succession of Its
    Information Systems

Once an agency has attained Stage 3 maturity, it evaluates its IT
investment processes and portfolios to identify opportunities for
improvement (Stage 4 capabilities). This entails (1) improving the
portfolio's performance and (2) managing systems and technology
succession. We did not assess FAA's capability for improving the
portfolio's performance, because it did not claim to be executing any of
the relevant key practices in its self-assessment.

According to ITIM, regarding system and technology succession management
includes (1) defining policies and procedures for managing the IT
succession process, (2) assigning responsibility for the IT succession
process, (3) developing criteria for identifying IT investments that may
meet succession status, and (4) periodically analyzing IT investments to
determine whether they are ready for succession. This critical process
enables an organization to recognize low-value or high-cost IT investments
and augments the routine replacement of systems at the end of their useful
lives. It also promotes the development of a forward-looking,
solution	oriented view of IT investments that anticipates future resource
requirements and allows the organization to plan appropriately. This
process differs from the reselection activity in Stages 2 and 3 in that it
focuses on anticipating and planning for the retirement of legacy systems
and on meeting remaining requirements with other, perhaps new, systems. In
addition, succession management takes place at the end of a system's life
cycle.

FAA has not executed any of the nine key practices required to implement
this critical process. Although the agency has defined procedures in AMS
for retiring investments, it still needs to describe how to regularly
review systems that are in operations in order to identify candidates for
retirement. According to FAA, decisions on succession are made by the
service organizations. However, no individual or group has been assigned
responsibility for managing the succession process from an enterprise
perspective, which would allow the FAA to better anticipate and plan for
future resource requirements. Without an institutionalized process for
succession management, the FAA may not be able to identify those IT
investments that are eligible for succession in enough time to minimize
the effect of the transition on their successors. In addition, by
establishing an effective succession management process, the agency can
identify systems for retirement, freeing resources for other, superior,
investments.

  FAA Has Initiated Efforts to Improve Its Investment Management Process

We have previously reported that to effectively implement IT investment
management processes, organizations need to be guided by a plan that (1)
is based on an assessment of strengths and weaknesses; (2) specifies
measurable goals, objectives, and milestones; (3) specifies needed
resources; (4) assigns clear responsibility and accountability for
accomplishing tasks; and (5) is approved by senior management.

FAA has begun to take steps to resolve some of the weaknesses identified
in this report. For example, at a June 10, 2004, meeting, the JRC decided
to incorporate budget justification documents (Exhibit 300s), which are
currently prepared for the Office of Management and Budget (OMB) as part
of the President's Budget formulation process, into the AMS process for
managing NAS investments. The Exhibit 300 will become the board's
decision-making document, and essential information from existing
AMS	required documents-the investment management report, the acquisition
strategy paper, the integrated program plan, and the requirements
documents-will be incorporated into the Exhibit 300. The JRC also recently
decided to implement PIRs in order to track metrics during program
implementation. Finally, at that same meeting, the board decided to
collectively determine, at the meeting where the F&E budget is approved,
which F&E and OPS programs should be brought forward for review the
following year. This decision serves to bring certain investments in the
in-service management phase under the JRC's direct purview, although it
does not specify that consistent criteria be established, as the ITIM
framework requires.

FAA has also begun to initiate steps to bring more clarity to the ITEB's
responsibilities, although the specifics have yet to be defined. In its
charter, the ITEB is charged with making investment decisions about
non-NAS IT investments. This action would begin to bring all of the
non-NAS investments under a single authority. The charter suggests that
the ITEB choose among three options: (1) to send major non-NAS investment
decisions to the JRC, (2) to make the decision itself, given an acceptable
review process similar to the JRC processes, or (3) have the CIO, Chief
Financial Officer, and owning assistant/associate administrator make the
decision jointly. This description of the ITEB's roles and
responsibilities further alludes to the senior board's evolving
responsibility toward major non-NAS IT investments, although it falls
short of laying out specific criteria for selecting which investments
should be sent forward to the JRC. The ITEB has been given responsibility
for four short-term initiatives as well, including establishing an
agencywide cost control program for non-

NAS expenditures and ensuring that all OMB Exhibit 300s receive a passing
grade for the 2006 budget year. The ITEB has been charged with the
long	term initiative of clearly delineating the roles it plays and its
relationship with the more senior board. The successful completion of this
initiative is likely to satisfy the single key practice that FAA has not
yet executed in the Instituting the Investment Board critical process of
the ITIM.

The Chief Operating Officer's recent reorganization of the ATO is intended
to make the heads of the service units responsible for IT projects from
their inception through the in-service management phase. This new
organization is designed to support his expressed intentions to increase
accountability for systems in operation in order to manage costs more
effectively. According to the Chief Operating Officer, FAA recognizes that
good processes are needed for both NAS and non-NAS to improve the way the
agency manages its investments.

While FAA has initiated these improvement efforts, it has not linked them
together in a plan with the characteristics listed above that would help
coordinate and guide the efforts. Until FAA develops a plan that would
allow for the systematic prioritization, sequencing, and evaluation of
improvement efforts, the agency risks not being able to effectively
establish mature investment management processes.

  DOT is Taking Steps to Integrate Oversight of FAA's IT Investments

DOT has recently initiated several efforts that can serve to provide
better departmental oversight of FAA investments. This fiscal year DOT and
FAA reached an agreement by which DOT reviews FAA's Exhibit 300s as part
of the department's annual budget process, in which all departmental
components participate. Under this agreement, DOT conducts a review of all
FAA Exhibit 300s starting in June of each budget year and culminating in
the review of all Exhibit 300s by the Department Investment Review Board
in late August, prior to the submission of the budget to OMB in September.
As part of this agreement, DOT has outlined a process and schedule for
reviewing the fiscal year 2006 budget justifications for major FAA
programs and is monitoring FAA's progress in meeting this schedule. In
addition, the department has identified about a dozen programs that it
plans to monitor regularly and has begun reviewing these programs through
its senior investment management decision-making board, on which the FAA
Administrator is a voting member. DOT has also requested that FAA set
reasonable expectations for cost, schedule, and performance for its major
projects and that it then report quarterly on variances to those
expectations. FAA submitted its first quarterly report as of June 2004.

These regular reports are intended to help DOT maintain oversight of FAA's
processes and ensure that they are appropriate and consistent with OMB's
requirements. Furthermore, the department is currently planning to issue
an investment management guide that specifies minimum expectations that
its operating administrations (including FAA) are to follow in managing
their investments. According to DOT officials, FAA has been complying with
the department's requests for information to facilitate its oversight
process.

Department officials are attributing their increased oversight-and
cooperation from FAA-to the fact that the department has recently
reinstituted its own investment management processes. In addition, DOT
officials said that FAA now understands the role the department can play
in helping it to obtain the funding it needs for its programs.

Conclusions
FAA has established most of the project selection and control capabilities
needed to manage its NAS investments. This should help provide the
executive-level decision-making and oversight capabilities required to
establish accountability and guide major IT investments through most of
their life cycles. However, weaknesses remain. For example, although
business units are involved in the regular review of investments
throughout their life cycles, the JRC may not review the performance of
operations systems for several years unless they require significant
additional funds. Also, FAA has yet to define and implement the practices
it needs to select and control its non-NAS investments. Ultimately,
because the JRC does not regularly review NAS systems during the
in-service management phase and does not regularly review the non-NAS
systems in general, significant portions of FAA's approximately $2.5
billion investment in IT go without top-level executive oversight and are
not viewed as part of an enterprisewide portfolio. FAA has taken some
initial steps to implement PIRs, but it has not yet established a process
to carry them out.

The agency has begun to take some steps to develop improvements to address
some of these weaknesses, such as establishing an Information Technology
Executive Board with relevant responsibilities. In addition, the JRC has
begun integrating some budgeting and oversight processes, and the Chief
Operating Officer has begun to articulate a vision that includes
additional accountability for investments in operations. But FAA has not
developed a comprehensive plan to guide all improvement efforts. Such a
plan would help coordinate and prioritize improvement efforts and help
sustain commitment to the efforts under way. The increasing collaboration

between FAA and DOT further contributes to the likelihood that the
management of FAA's investments will improve as FAA's Exhibit 300s have
the benefit of department-level review and the departmental investment
review board conducts periodic reviews of selected projects.

  Recommendations for Executive Action

To strengthen FAA's investment management capability and address the
weaknesses discussed in this report, we recommend that the Secretary of
the Department of Transportation direct the FAA Administrator to develop
and implement a plan for improving FAA's IT investment management
processes. The plan should address the weaknesses described in this
report, beginning with those we identified in our Stage 2 analysis and
continuing with those we identified in our Stage 3. The plan should also
draw together ongoing efforts as well as instituting new initiatives where
called for. The plan should, at a minimum, provide for accomplishing the
following:

In Stage 2

o  Define procedures for aligning the JRC and the newly established ITEB.

o
Establish a process for the JRC to periodically reevaluate the alignment
of projects in the in-service management phase with strategic goals and
objectives.

o
Establish a process for the JRC to regularly review the performance of IT
systems throughout their life cycles and take corrective actions when
expected performance is not being met.

o
Define and implement an IT investment management structure, including an
investment management board and a disciplined process for managing all
non-NAS investments.

In Stage 3

o
Define and implement processes for managing major investments as part of
an enterprise-level portfolio, including NAS F&E investments, NAS
investments in the in-service management phase, and non-NAS investments.

o
Define and implement processes for carrying out PIRs on investments as
they enter the in-service management stage.

In developing the plan, the FAA Administrator should ensure that it (1)
specifies measurable goals, objectives, and milestones; (2) specifies
needed resources; (3) assigns clear responsibility and accountability for
accomplishing tasks; and (4) is approved by senior management. In
implementing the plan, the FAA Administrator should ensure that the needed
resources are provided to carry out the plan and that progress is measured
and reported periodically to the Secretary of Transportation.

Agency Comments
In commenting on a draft of this report, DOT's Director of Audit Relations
stated via e-mail that DOT appreciated the opportunity to review and offer
comment on our report and that GAO had done a good job keeping the report
balanced and fair, showing where FAA has many capabilities in place and
identifying areas that need improvement. The Director also provided a
technical comment, which we have incorporated into the report.

As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from
the date of this letter. At that time, we will send copies to other
interested congressional committees, the Director of the Office of
Management and Budget, the Secretary of Transportation, FAA's
Administrator and CIO, and other interested parties. We also will make
copies available to others upon request. In addition, the report will be
available at no charge on the GAO Web site at www.gao.gov.

Should you or your offices have questions on matters discussed in this
report, please contact me at (202) 512-9286 or Lester P. Diamond,
Assistant Director, at (202) 512-7957. We can also be reached by e-mail at
[email protected], or [email protected], respectively. Key contributors to
this report were William G. Barrick, Niti Bery, Joanne Fiorino, Michael
Giannone, Sabine R. Paul, and Nik Rapelje.

David A. Powner Director, IT Management Issues

Appendix I

                       Objectives, Scope, and Methodology

The objectives of our review were to (1) evaluate FAA's capabilities for
managing its IT investments, (2) determine what plans the agency might
have for improving these capabilities, and (3) describe how DOT oversees
FAA's investments and investment process. Because FAA told us that it
managed its NAS and non-NAS investments differently, we performed separate
assessments for the practices to evaluate FAA's capabilities for managing
IT investments.

To address the first objective, for the NAS investments we reviewed the
results of the agency's self-assessment of Stages 2, 3, and 4 practices
using GAO's ITIM framework1 and validated and updated the results of the
self	assessment through document reviews and interviews with officials. We
reviewed written policies, procedures, and guidance and other
documentation providing evidence of executed practices, including FAA's
Acquisition Management System guidance, mission analysis and investment
analysis guidance, and memorandums. We also reviewed JRC guidance and
records of decision, acquisition review guidance and meeting minutes, and
variance reporting procedures and reports. We did not assess FAA's
progress in establishing the capabilities found in one of the two Stage 4
critical processes, entitled Improving the Portfolio's Performance, or in
any of the Stage 5 critical processes, because FAA acknowledged that it
had not executed any of the key practices in these critical processes. For
the non-NAS investments, we reviewed the results of FAA's self	assessments
of Stage 2 practices using GAO's ITIM framework and conducted interviews
to clarify and update the results. We did not perform a detailed
assessment of these practices because they most likely will be superseded
by a new process (when it is defined) for managing non-NAS investments,
and non-NAS investments are of lower cost and impact to FAA.

As part of our analysis, we selected three IT projects as case studies to
verify that the critical processes and key practices were being applied.
We selected projects that (1) supported different FAA functional areas,
(2) were in different life cycle phases, and (3) required different levels
of funding. The three projects are described below:

o
FAA Telecommunications Infrastructure (FTI)-FTI is a performance	based
telecommunications services contract for voice, video, and data
point-to-point support for telecommunications for the National Airspace

1GAO-04-394G.

Appendix I
Objectives, Scope, and Methodology

System and its support system. It contributes to both the separation of
aircraft (the mission-support network) and other FAA uses (the operational
network, e.g., e-mail and phone). FTI will replace the current telecom
system. FTI will eliminate the need for other subnetworks, of which there
are currently eight or nine, and therefore eliminate the management
overhead associated with operating so many networks. The integration of
multiple networks and subnetworks will provide a single source and single
vehicle for telecom. FTI is in the Technical Operations unit and has
estimated life cycle costs of $2 billion. The contract for FTI was awarded
in June 2002.

o
En Route Communications Gateway (ECG)-ECG is a mission critical gateway,
or interface, for data from radar sites to Air Route Traffic Control
Centers. ECG will serve as a single domain communications gateway and will
provide the path for exchanging flight plan data from outside sources and
transfer data among systems. ECG provides a commercial-off-the-shelf
nondevelopmental item digital gateway using a modern, open and extensible
platform consisting of modular scalable hardware components. ECG will
incorporate interface capability to support legacy and future systems and
will provide the capability to transition to modern network communications
and access more surveillance sources. The flexibility provided by the ECG
system architecture will facilitate the evolution of the En Route domain
modernization. ECG will replace the Peripheral Adapter Module Replacement
Item system and provide a modern domain gateway that will support the
current and future En Route infrastructure. ECG is in the En Route &
Oceanic Service group and has estimated life cycle costs of $442.5 million
through September 2015.

o
Voice Switching and Control System (VSCS)-In our review of the VSCS
program, we focused our review on one of VSCS's subcomponents, the VSCS
Control Subsystem Upgrade (VCSU). The VCSU program, part of the Technical
Operations Communications service group, is designed to maintain overall
supportability of VSCS2 by replacing the hardware for the existing control
subsystem, associated VSCS operational and application software, required
software licenses, and supporting software and hardware documentation.
Deliverables for the VCSU

2VSCS is FAA's highly distributed, computer-controlled communications and
control system for U.S. air traffic management that allows air traffic
controllers to establish all air-to	ground and ground-to-ground
communications with pilots and other air traffic controllers.

Appendix I
Objectives, Scope, and Methodology

program include all hardware, spare parts, software, software licenses,
system baseline documentation, training, and other technical documentation
necessary to support the product at 21 locations. According to FAA, the
VCSU program has a funding baseline of over $59 million and is in the
operations and maintenance phase.

For these projects, we reviewed project management documentation, such as
mission needs statements, acquisition program baselines, and integrated
program plans. We also interviewed the project managers for these
projects.

We compared the evidence collected from our document reviews and
interviews to the key practices in ITIM. We rated the key practices as
"executed" on the basis of whether the agency demonstrated (by providing
evidence of performance) that it had met the criteria of the key practice.
A key practice was rated as "not executed" when we found insufficient
evidence of a practice during the review or when we determined that there
were significant weaknesses in FAA's execution of the key practice.

To address our second objective, we obtained and evaluated documents
showing what management actions had been taken and what initiatives had
been planned by the agency. This documentation included JRC records of
decisions, the agency's capital investment guidance, and the recently
formed ITEB charter and meeting minutes. We also interviewed the Chief
Information Officer, other members of the JRC, and the Chief Operating
Officer to determine what efforts FAA had undertaken to improve IT
investment management processes.

To address our third objective, we reviewed documentation on DOT's process
for reviewing FAA's budget proposals and capital planning and investment
control reviews. We also conducted interviews with both FAA and DOT
officials, including DOT's CIO and Director for Capital Planning and
Investment Control to determine DOT's oversight role in FAA's investments
and investment management processes.

We conducted our work at FAA Headquarters in Washington, D.C., from
October 2003 through July 2004, in accordance with generally accepted
government auditing standards.

Appendix II

Investment Management Process Used by Some Organizational Units to Manage
Non-NAS Investments

                            Financial Services (ABA)

Instituting the investment board	ABA has an investment board that conducts
periodic and monthly program reviews for all IT programs to determine
whether a program will be approved as an IT investment. A life cycle
process guide is now in place to direct the activities of the investment
board along with providing oversight of IT projects within ABA.

Meeting business needs	The business needs of a project within ABA, along
with the dates for achieving them, need to be aligned with the strategic
goals established in the FAA Flight Plan. Projects or systems that are no
longer aligned with the Flight Plan will be decommissioned.

A project management plan identifies, among other things, the system's
users, customers, and types of services to be provided.

Selecting an investment	Selecting and reselecting an IT investment within
ABA involves both the executive management team and ABA's CIO team. The
executive management team reviews the business needs of the investment and
compares them against the ABA's IT budget, while ABA's CIO team is
involved with the selecting and reselecting processes by analyzing the
technical costs associated with the IT investment and comparing those
technical costs against the ABA's IT budget.

Providing investment oversight	ABA uses its life cycle process guide to
help manage its $25 million IT budget, which consists of 22 or 23
financial systems, 5 or 6 of them considered major programs under OMB's
definition of a major IT investment. A requirement of the life cycle
process guide is for every critical system in ABA to have a detailed
project management plan that addresses performance measures such as cost,
schedule, benefits, and risks.

The day-to-day progress of IT projects is tracked against critical
milestones that have been already established through weekly summary
reviews with IT staff. For major IT projects, biweekly meetings are
conducted that address any concerns with meeting the performance measures.

Capturing the investment information	ABA captures its IT asset information
using its Information Technology Investment Portfolio System (ITIPS),
which is available to all ABA management and system support personnel. The
information in ITIPS is used to manage projects that are in production as
well as ensuring that the life cycle activities are in alignment with
FAA's mission statements.

                        Research and Acquisitions (ARA)

Instituting the investment board	ARA uses its Operations Resource
Management Team guide to select, control, and evaluate ARA IT investments.
The team composed of representatives from ARA service units. ARA
investments are controlled and tracked through quarterly reviews. These
reviews look at the cost, schedule, and overall performance of the
investment.

Meeting business needs	The business needs for ARA investments need to be
mapped back to the Flight Plan. A monthly status review report is prepared
in order to ensure that the business needs are tracking back to the Flight
Plan.

Selecting an investment	ARA does not have any well-defined selection
criteria since each program uses its own configuration management plan.
ARA Ops build process guides the establishment of new projects.

 Appendix II Investment Management Process Used by Some Organizational Units to
                           Manage Non-NAS Investments

                         (Continued From Previous Page)

Providing investment oversight	A project plan does exist, along with
established expenditures, which the program managers submit to the ARA CIO
on a monthly basis. These monthly status reports occur between the CIO and
the program managers to decide if an investment's resources, such as
funding, need to be reallocated. Once the CIO and program manager decide
that it is necessary for an investment's resources to be reallocated, the
CIO will discuss the need further with the Deputy Associate Administrator
for ARA, who ultimately will determine whether a program will receive
additional resources, such as funding.

With respect to the level of interaction that ARA has had with the JRC in
the past, only one program from ARA, NextGen, has gone before the JRC.
According to the ARA CIO, in order for a program to go to the JRC, there
must be justification made to the council that the program is fully
operational and is considered to be a benefit and a priority to FAA. The
ARA Deputy Associate Administrator will determine if a program should go
before the JRC for approval and funding.

Capturing investment information	The configuration control board uses a
database to capture asset inventory data about the systems that are owned
by the ARA CIO. According to the ARA CIO, in order for IT assets to be
effectively managed in ARA, there needs to be vision from AIO about what
programs to invest in over the next 5 years.

                           Air Traffic Services (ATS)

Instituting the investment board	The Information Resource Management
Executive Board is responsible for selecting, controlling, and evaluating
ATS IT investments.

Meeting business needs	Not all services within ATS have defined their
business needs. Even though ATS has the NAS Support Integration Process
(NSIP) data repository available for capturing IT asset information,
including business needs, and for defining system users, there is no
consistency in terms of the records being complete because there are
systems within ATS that have not registered with NSIP.

Selecting an investment	The ATS CIO manages the selection process, which
begins with the NSIP registration criteria.

Providing investment oversight	Each business unit within ATS has its own
project management plan and procedures. The day-to-day tracking of
projects as well as the monitoring of whether corrective actions are being
executed is also the responsibility of the individual business units. Even
though the individual business units are tasked with this level of
responsibility, the ATS CIO does play an oversight role by setting the
criteria and policies for the investments to be made for the projects.

Capturing investment information	ATS uses the NSIP meta data repository to
collect any changes to the IT projects and systems by providing a full
declaration of the project or system. This includes providing information
to help ATS avoid unwanted costs due to systems having redundant
functionality and determining whether a system's or a project's functions
match the stated mission goals for ATS. NSIP also handles the technical
rollover for ATS systems or projects.

                           Information Services (AIO)

Instituting the investment board	AIO's investment management process can
be characterized as iterative and well managed, but undocumented. The AIO
Business Plan and IT Strategy are used to ensure that when funds are
appropriated and allocated that they map back to the Flight Plan.
Investments are controlled or tracked by the Deputy CIO on a monthly basis
to get an indication as to where the program is in the process against the
expenditures that have been already established. Weekly meetings are held
with the unit's CIO to discuss any issues regarding AIO's investment
management process.

 Appendix II Investment Management Process Used by Some Organizational Units to
                           Manage Non-NAS Investments

                         (Continued From Previous Page)

Meeting business needs	AIO does not have any written policies or
procedures for identifying business needs for its IT projects. Only one of
its major projects, NAS Adaptation Service and Environment, has documented
its requirements, which includes specific users.

Selecting an investment	AIO uses an undocumented process for reviewing new
IT proposals to reach an agreement on selection.

Providing investment oversight	There are no AIO-wide policies or
procedures for managing projects or investment oversight. The Information
Technology Executive Board (ITEB)a has been formed to provide a governing
structure for non-NAS programs. One of the targets for ITEB is to look at
cost control and cross-cutting IT initiatives by involving the heads of
the lines of business. The ITEB is also going to be involved with
improving the scores on the Exhibit 300 business cases for OMB.

Capturing the investment information	AIO uses ITIPS to track its asset
inventory and IT investments. The Deputy CIO of AIO is responsible for
ensuring that the inventory located in ITIPS meets the needs of AIO's
investment management process. According to AIO, the information within
ITIPS is updated at least twice a year.

                        Human Resource Management (AHR)

Instituting the investment board	AHR does not have an investment board.
Instead, AHR's senior managementb is responsible for selecting,
controlling, and evaluating all IT investments by using established agency
acquisition policies and procedures to conduct investment management
decisions.

Meeting business needs	Business needs and specific users for each project
are identified within the project plan and are aligned with the AHR
Strategic Plan, the FAA Flight Plan, and the AIO Plan. AHR is also
aligning its business needs to the ITEB plans. Business needs are
re-evaluated on a quarterly basis to ensure that a project is aligned with
FAA's strategic goals and objectives.

Selecting an investment	AHR senior management uses its prioritization
process to evaluate and select investments for funding. The office and
center directors determine their requirements and then a budget request is
submitted for proposal funding. AHR receives an allowance amount from the
budget office. The first priority is to handle personnel payments. The
remaining balance is then redistributed to the business divisions. The
"building blocks" process starts at this point. This is when base funding
is reviewed to decide if a current investment needs continued funding by
asking questions about the importance of continuing the funding of a
particular project by looking at the project activity and what the impact
will be if this project is no longer funded. Each division will submit a
list of prioritized projects with costs to the directorate. This list may
exceed the budget level. The directorate will reprioritize the original
list.

Providing investment oversight	AHR has a Human Resource Management
Automation Plan that contains procedures for approving IT projects, and
describes the policies and procedures that AHR uses for project
management. Despite having project management policies and procedures, not
all projects within AHR have a formal project plan. The size and scope of
the project are two factors that help determine whether a project has a
formal project plan. AHR Division Managers ensure that projects are on
time by performing quarterly reviews that assess a project's cost and
schedule. AHR uses a color scheme (red, green, and yellow) to indicate the
schedule status of major milestones.

Capturing the investment information	AHR uses the ITIPS as its inventory
for making investment management decisions. AHR projects are listed in
ITIPS, along with business cases.

 Appendix II Investment Management Process Used by Some Organizational Units to
                           Manage Non-NAS Investments

(Continued From Previous Page)

                      Regions and Center Operations (ARC)

Instituting the investment board	The IT Configuration Management Board is
ARC's investment review board. The board's charter has recently been
redone to provide more traceability back to the Flight Plan. The board
functions include evaluating potential IT investment options for ARC,
making recommendations on IT investment, establishing ARC-wide IT
standards, and developing and maintaining investment policies and
procedures. The board is led by the unit's CIO and includes four IT
managers from the regional offices and aeronautical center and two members
from the ARC Management Team. The ARC Management Team makes the
finalselection decisions. The IT investment management decisions are then
incorporated into the ARC Business Plan. The ARC unit is also involved
with cross-organizational investment decisions for FAA through its
membership on the FAA CIO Council.

Meeting business needs	Business needs are identified through entries made
in ITIPS, along with documentation from Exhibit 300s and Exhibit 53s.

Selecting an investment	ARC does not have its selection criteria
documented. To evaluate and select IT investments, the ARC IT
Configuration Management Board considers such things as benefits to ARC
across the regions, expected return on investment, technical feasibility,
and risk. The ARC business plan and the Flight Plan are the documents that
address these priorities.

Providing investment oversight	ARC does not have policies or procedures
for project management. Instead, ARC uses a weekly teleconference to
address expectations and progress of ARC-wide IT initiatives at the IT
manager level across ARC. According to the ARC CIO, a second
teleconference has been added to discuss portfolio management-schedule,
budget, training, and deployment along with whether the project will be
integrated with other lines of business.

Capturing the investment information	ARC uses ITIPS as its standardized
repository for collecting asset information that will be useful for ARC's
IT investment management decisions by providing information about what
types of systems and functions are available and how they are supporting a
specific business issue.

                       Regulation and Certification (AVR)

Instituting the investment board	Similar to an IT investment board, AVR
has a two-tiered management structure that is composed of the AVR
management team and the CIO management team. The AVR management team
includes the Associate Administrator and the Service Directors who make
the final decisions based upon recommendations and input from the CIO
management team and its business partners from each of the service units.
According to AVR, its IT investment process guide is still under
development and will be completed at the end of Fiscal Year 2004.

Meeting business needs	Each line of business within AVR identifies and
documents its business needs including project requirements and specific
users. Once the business needs have been identified, the IT Management and
Resources section prioritizes them for funding.

Selecting an investment	Programs in AVR are reviewed quarterly. For major
projects, meetings are designed to look at project milestones to see if
they are being met. These meetings are carried out biweekly and presented
to the AVR management team.

Providing investment oversight	The AVR CIO management team is responsible
for monitoring projects and reporting to the AVR Management team. Biweekly
meetings are held for major projects within AVR.

Capturing the investment information	AVR's system inventory is a part of
its enterprise architecture. The system inventory is being used primarily
in developing the Exhibit 300s. The performance of IT projects in AVR is
monitored daily, based upon each project's individual plan, using project
management tools such as MS Project. According to AVR, not all projects
have a project plan in place, but AVR is trying to make it a requirement.

Source: GAO, based on information from FAA.

Appendix II Investment Management Process Used by Some Organizational
Units to Manage Non-NAS Investments

aITEB is a board that can provide a governing structure so that
information technology is used as an agency wide strategic asset.

bComposed of Assistant Administrator; two Deputy Assistant Administrators;
three Office Directors; the Director, Center for Management and
Development; and the AHR Business Officer.

GAO's Mission	The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost

is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to

www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone	The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone:	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

  To Report Fraud, Contact:Waste, and Abuse in Web site:
  www.gao.gov/fraudnet/fraudnet.htm

E-mail: [email protected] Federal Programs Automated answering system:
(800) 424-5454 or (202) 512-7470

Congressional	Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125

Relations Washington, D.C. 20548

Public Affairs	Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

                               Presorted Standard
                              Postage & Fees Paid
                                      GAO
                                Permit No. GI00

United States Government Accountability Office Washington, D.C. 20548-0001

Official Business Penalty for Private Use $300

Address Service Requested
*** End of document. ***