Air Traffic Control: FAA Needs to Ensure Better Coordination When
Approving Air Traffic Control Systems (17-NOV-04, GAO-05-11).	 
                                                                 
The Federal Aviation Administration's (FAA) process for ensuring 
that air traffic control (ATC) systems will operate safely in the
national airspace system is an integral part of the agency's	 
multibillion-dollar ATC modernization and safety effort. GAO was 
asked to review (1) FAA's process for approving ATC systems for  
safe use in the national airspace system; (2) challenges FAA has 
faced approving ATC systems and how these challenges affected the
cost, schedule, and performance estimates of the systems; and (3)
actions FAA has taken to improve its process for approving ATC	 
systems.							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-05-11						        
    ACCNO:   A13693						        
  TITLE:     Air Traffic Control: FAA Needs to Ensure Better	      
Coordination When Approving Air Traffic Control Systems 	 
     DATE:   11/17/2004 
  SUBJECT:   Air traffic control systems			 
	     Air traffic controllers				 
	     Aircraft						 
	     Aviation						 
	     Internal controls					 
	     Safety standards					 
	     Strategic planning 				 
	     Systems design					 
	     Transportation safety				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-05-11

Report to the Chairman, Subcommittee on Aviation, Committee on
Transportation and Infrastructure, House of Representatives

November 2004

AIR TRAFFIC CONTROL

FAA Needs to Ensure Better Coordination When Approving Air Traffic Control
Systems

Contents

Tables

Figures

November 17, 2004Letter

The Honorable John L. Mica Chairman, Subcommittee on Aviation Committee on
Transportation and Infrastructure House of Representatives

Dear Mr. Chairman:

The Federal Aviation Administration's (FAA) process for ensuring that air
traffic control systems will operate safely in the national airspace
system is an integral part of FAA's multibillion-dollar air traffic
control modernization and safety effort. New air traffic control systems
cannot be used in the national airspace system until FAA has determined
that the systems will operate safely. Over the years, FAA has approved
about 45,000 pieces of air traffic control equipment for safe use in the
national airspace system. Some in the aviation industry and government
contend that FAA's approval process for air traffic control systems is too
lengthy and, therefore, contributes to cost growth, schedule delays, and
performance problems that have plagued many of the systems that FAA has
been trying to develop for years. In addition, some in the aviation
industry have raised concerns about whether FAA's approval process has
kept pace with changes in technology. For example, more of today's new air
traffic control systems are integrated-that is, involving both ground
systems1 and equipment used exclusively in aircraft (aircraft equipment)
that must work together-than in the past.

In response to your request, we examined

o FAA's process for approving air traffic control systems for safe use in
the national airspace system;

o challenges FAA faces in approving air traffic control systems and how
these challenges have affected the cost, schedule, and performance of the
systems; and

o actions FAA has taken to improve its process for approving air traffic
control systems.

In this report, we use the word "approval" to describe the process of
ensuring the safety of an air traffic control system when it has both a
ground system and aircraft equipment. We also use the word "approval" to
describe the process of ensuring the safety of ground systems exclusively.
We use the word "certification" to describe the process of ensuring the
safety of aircraft equipment for safe use in the national airspace system.

To identify FAA's process for approving air traffic control systems for
safe use in the national airspace system, we reviewed FAA documents that
describe the agency's process for approving such systems and equipment and
RTCA's 1999 and 2001 reports that also address this process.2 To determine
the challenges FAA has faced in approving air traffic control systems and
how these challenges affected the cost, schedule, and performance of the
systems, we (1) conducted case illustrations on 5 of FAA's 25 air traffic
control systems currently receiving funding that were approved or in the
process of being approved for safe use in the national airspace system and
(2) reviewed reports prepared by GAO and the Department of
Transportation's Inspector General. The 5 air traffic control systems are

o Airport Surface Detection Equipment - Model X (ASDE-X),

o Controller-Pilot Data Link Communications (CPDLC),

o Local Area Augmentation System (LAAS),

o Standard Terminal Automation Replacement System (STARS), and

o Wide Area Augmentation System (WAAS).

We selected these 5 systems because collectively they accounted for about
46 percent of FAA's air traffic control modernization costs in fiscal year
2002 and 3 of the 5 systems are integrated-that is, they require the
approval of the ground systems as well as certification of aircraft
equipment before they can be used in the national airspace system. In
addition, we interviewed, among others, officials from FAA program
offices; RTCA; aviation industry groups; manufacturers of aircraft
equipment; ground system developers, including Honeywell, Raytheon, and
Sensis Corporation; industry experts; Wide Area Augmentation System
Integrity Performance Panel3 and Local Area Augmentation System Integrity
Panel members;4 and unions representing air traffic controllers and
maintenance technicians. We also reviewed reports on air traffic control
systems prepared by GAO, the Department of Transportation's Inspector
General, RTCA, and the Commission on the Future of the U.S. Aerospace
Industry (Aerospace Commission). To identify what actions FAA has taken to
improve its processes for approving air traffic control systems, we
interviewed representatives from FAA, RTCA, the Aerospace Commission, and
aviation industry groups. See appendix I for additional information on our
objectives, scope, and methodology. We conducted our review from October
2003 through September 2004 in accordance with generally accepted
government auditing standards.

Results in Brief

FAA has separate processes for approving ground systems and certifying
aircraft equipment for safe use in the national airspace system. FAA's
process for approving ground systems, such as radar systems, is done in
accordance with policies and procedures in FAA's Acquisition Management
System. The process to approve ground systems, which are usually
developed, owned, and operated by FAA, involves FAA's Air Traffic
Organization determining whether a vendor is in compliance with contract
requirements and/or FAA operational requirements, followed by a rigorous
test-and-evaluation process to ensure that the new system will operate
safely in the national airspace system. In contrast, federal aviation law
requires that aircraft equipment, which is usually developed by private
companies, be certified in accordance with Federal Aviation Regulations,
with FAA serving as the regulator. Unlike the approval of ground systems,
which FAA accomplishes with the help of a contractor, FAA is not typically
involved in the development of the equipment. An applicant, such as a
manufacturer of aircraft equipment, generally brings fully developed
aircraft equipment to FAA for certification. If an air traffic control
system has both a ground system and aircraft equipment, as was the case
for 3 of the 5 systems we reviewed, then the system must go through both
processes before it is approved for safe use in the national airspace
system.

FAA has faced challenges in approving air traffic control systems for safe
use in the national airspace system. This report focuses on three specific
challenges we identified through our past work and our case illustrations
of 5 air traffic control systems. Most of these challenges have made it
more difficult for FAA to meet the systems' cost, schedule, or performance
estimates. These challenges are as follows:

o Involving appropriate stakeholders, such as users and technical experts,
throughout the ground system approval process. For example, during the
design and development phase of the Standard Terminal Automation
Replacement System, which is designed to replace air traffic controller
workstations with new color displays, FAA did not involve users such as
air traffic controllers and maintenance technicians in human factor
evaluations, which examine how humans interact with machines, because the
aggressive development schedule limited the amount of time available to
involve them. Consequently, FAA and the contractor later had to
restructure the contract to address the controllers' and technicians'
concerns, such as the inconsistency of visual warning alarms and color
codes, which contributed to the system being delayed by 3 years and a cost
increase of $500 million.

o Ensuring that the FAA offices that have responsibility for approving
ground systems and certifying aircraft equipment effectively coordinate
their efforts for integrated systems. For example, although the Wide Area
Augmentation System was being developed by an integrated product team that
included representatives from various FAA offices, the team did not
function effectively in resolving issues related to meeting an important
functional requirement to alert the pilot in a timely manner when the
system should not be used because of a possible error. According to FAA
officials, the reason coordination was not effective was because the two
offices had competing priorities that were not associated with development
of the Wide Area Augmentation System. This ineffective coordination,
combined with other factors,  contributed to a 6-year delay in
commissioning the Wide Area Augmentation System and a $1.5 billion
increase in its development costs.

o Accurately estimating the amount of time needed to meet complex
requirements at the beginning of the design and development phase. For
example, FAA accelerated the schedule for the Standard Terminal Automation
Replacement System in 1995. This acceleration in schedule left only
limited time for human factor evaluations and, according to FAA officials,
added $500 million to the Standard Terminal Automation Replacement
System's cost and 3 years to the schedule because the agency had to revise
its strategy for acquiring and approving it.

FAA has taken actions to address two of the three challenges we
identified. However, FAA has not taken action to fully involve all
stakeholders, such as air traffic controllers, maintenance technicians,
technical experts, and industry representatives, throughout the approval
process. To ensure that the two offices effectively coordinate their
ground system approval and aircraft equipment certification processes, FAA
officials believe that the agency's new Safety Management System, which is
designed to formalize and standardize the agency's safety process, will
improve overall coordination among FAA stakeholders once the system is
implemented. FAA stated that coordination would improve because, as part
of the new Safety Management System, the agency plans to realign its
organizational structure to create a formal link between the Air Traffic
Organization, which currently approves ground systems, and the Office of
Regulation and Certification. FAA expects full implementation of this
system to take 3 to 5 years. We are reserving judgment on whether this
change will fully address the challenge because of the early state of this
effort and because FAA's problems with internal coordination when
approving air traffic control systems are long-standing. In addition,
because FAA has historically faced internal and external coordination
challenges in approving air traffic control systems for safe use in the
national airspace system, we believe that as FAA moves forward with
implementing the agency's new Safety Management System, it should, in the
interim, develop plans that describe

how both internal and external coordination will occur on a
system-specific basis. In addition, plans to include external stakeholders
are particularly important since the Safety Management System is not
intended to address this challenge.

We are recommending that FAA develop early in the approval process air
traffic control system-specific plans that specify how and when the
approving and certifying offices within FAA and other stakeholders,
including controllers, maintenance technicians, technical experts, and
industry representatives, will meet to ensure coordination.

Background

Several offices within FAA's Air Traffic Organization and Office of
Regulation and Certification have responsibility for approving ground
systems and certifying aircraft equipment, as shown in figure 1.

Figure 1: Current FAA Offices with Responsibility for Approving Air
Traffic Control Systems

Note: The Office of Regulation and Certification's Air Traffic Safety
Oversight Service oversees and collaborates with the Air Traffic
Organization's Safety Services on the safety of air traffic control
systems.

Before the creation of the Air Traffic Organization in November 2003,
FAA's Research and Acquisitions (acquisitions office) and Air Traffic
Services were the primary offices responsible for approving ground systems
for safe use in the national airspace system. The 5 systems that we
reviewed began the approval process under that structure. Currently, these
offices, although renamed, form the core of the Air Traffic Organization.
The responsibilities of Air Traffic Services are now distributed among
several offices, including System Operations Services and Terminal
Services. The responsibilities of Research and Acquisitions are
distributed among several offices, including Technical Operations Services
and En Route and Oceanic Services. In addition, the Air Traffic
Organization includes Safety Services, which is its focal point for
safety, quality assurance, and quality control and is the primary
interface with FAA's Office of Regulation and Certification.

FAA's Office of Regulation and Certification has responsibility for
certifying and regulating aircraft and its equipment. The following 3
offices within the Office of Regulation and Certification are involved in
the certification of aircraft equipment:

o Aircraft Certification Service (aircraft certification office) is
responsible for administering safety standards for aircraft and aircraft
equipment that are manufactured in the United States.

o Flight Standards Service is responsible for granting operational
approval to air carriers that plan to use equipment on their aircraft.

o Air Traffic Safety Oversight Service is responsible for monitoring the
safety of air traffic operations through the establishment, approval, and
acceptance of safety standards and the monitoring of safety performance
and trends. It will also improve coordination between the Office of
Regulation and Certification and the Air Traffic Organization.

In addition to the internal FAA stakeholders, the approval of air traffic
control (ATC) systems can also involve a number of other external
stakeholders. FAA generally makes the decision about which other
stakeholders will be involved in approving ATC systems for safe use in the
national airspace system. For example, stakeholders involved in approving
ATC systems may include

o technical experts;

o ground system developers;

o manufacturers of aircraft equipment;

o aviation industry groups;

o general aviation; and

o users, such as controllers and maintenance technicians.

FAA also regularly  requests RTCA, a private, not-for-profit corporation,
to develop consensus-based performance standards for the aircraft
equipment component of ATC systems. RTCA functions as a federal advisory
committee that provides recommendations used by FAA as the basis for
policy, program, and regulatory decisions and by the private sector as the
basis for development, investment, and other business decisions.

In this report, we focus on the approval of the 5 ATC systems described in
table 1 and further discussed in appendixes II through VI.

FAA Has Separate Processes for Approving Ground Systems and Certifying
Aircraft Equipment

FAA has separate processes for approving ground systems and certifying
aircraft equipment for safe use in the national airspace system. FAA's
process for approving ground systems, such as radar systems, is done in
accordance with policies and procedures in FAA's Acquisition Management
System.5 This process involves a determination by FAA's Air Traffic
Organization regarding whether a vendor is in compliance with contract
requirements and/or FAA operational requirements, followed by a rigorous
test-and-evaluation process to ensure that the new system will operate
safely in the national airspace system. In contrast, the process for
certifying aircraft equipment, which is usually developed by private
companies, is done in accordance with Federal Aviation Regulations, with
FAA serving as the regulator. If an ATC system has both a ground system
and aircraft equipment, as was the case for 3 of the 5 systems we
reviewed, then the system must go through both processes before it is
approved for safe use in the national airspace system.

Ground System Approval Process

The approval of a ground system focuses on safety and is done in
accordance with FAA contract documents and policies and procedures that
are part of the agency's Acquisition Management System. Most ground
systems that provide air traffic services and air navigation services are
developed, owned, and operated by FAA. Prior to November 2003, FAA's
Research and Acquisitions and Air Traffic Service offices were responsible
for the approval of ground systems. Currently, FAA's Air Traffic
Organization has primary responsibility for the approval of ground
systems. FAA's ground system approval process includes the following six
phases-concept of operations, requirements setting, design and
development, test and evaluation, operational readiness, commissioning-and
involves various stakeholders, which are also noted below.

o Concept of operations: The ground system approval process begins with
the concept of operations phase. If the system being developed has both a
ground system and aircraft equipment, FAA's Office of Regulation and
Certification, Air Traffic Services Office, and Acquisitions Office may

work together to develop the concept of operations.6 During this phase,
FAA generally identifies and defines a service or capability to meet a
particular need in the national airspace system and may involve other
stakeholders, such as air traffic controllers.7 FAA also defines the roles
and responsibilities of key participants, such as controllers and
maintenance technicians, and the key elements of the required capability.
The concept of operations phase is not a static process. As FAA obtains
more information about the system it develops, the concept is revised to
reflect the new information even though the next phase of the process may
have already begun. Potential stakeholders in this phase include FAA's
Office of Regulation and Certification, FAA's Air Traffic Organization,
aircraft manufacturers, aviation industry associations, airlines, air
traffic controllers, maintenance technicians, manufacturers of aircraft
equipment, ground system developers, and representatives of general
aviation.

o Requirements setting: During the requirements-setting phase, FAA
establishes a minimum set of requirements, including safety objectives,
and specifies how well the new system must perform its intended functions.
For example, it was during this phase that FAA established WAAS' and LAAS'
integrity requirement-which is that the system cannot fail to warn pilots
of misleading information that could potentially create hazardous
situations more than once in 10 million approaches.  After analyzing the
initial requirements and comparing the cost, benefits, schedule, and risk
of various solutions, FAA sets final requirements and presents them to the
Joint Resources Council as part of the investment plan.8 After the council
has approved the requirements for the new system, FAA will issue a request
for proposals, evaluate the offers received, and select a contractor to
design a system based on the requirements set by FAA. Potential
stakeholders in this phase include FAA's Office of Regulation and
Certification, FAA's Air Traffic Organization, aircraft manufacturers,
aviation industry associations, airlines, air traffic controllers,
maintenance technicians, manufacturers of aircraft equipment, ground
system developers, and representatives of general aviation.

o Design and development: The design and development of ground systems is
generally completed by a contractor and monitored by FAA. During this
phase, the contractor conducts preliminary and critical design reviews,
which include plans for how it will conduct the testing phase. FAA must
approve these plans before the contractor can proceed to the next phase.
Potential stakeholders in this phase include FAA, ground system
developers, air traffic controllers, and maintenance technicians.

o Test and evaluation: After FAA has approved the design and development
of the system, it is ready to be tested and evaluated. The testing and
evaluation of ground systems typically includes three major tests:
development tests, operational tests, and an independent operational test
and evaluation. Development testing is performed by the contractor to
verify compliance with contractual requirements and is overseen by FAA.
Operational testing is performed by FAA and is designed to demonstrate
that a new system is operationally effective and suitable for use in the
national airspace system. An independent operational test and evaluation
is a full system-level evaluation conducted by FAA in an operational
environment to confirm the operational readiness of a system to be part of
the national airspace system. Potential stakeholders in this phase include
FAA, ground system developers, air traffic controllers, and maintenance
technicians.

o Operational readiness: During the operational readiness phase, FAA
personnel are trained to operate and maintain the new system, usually in
conjunction with its predecessor system. Following operational readiness
approval, the system is ready to be commissioned. Potential stakeholders
in this phase include FAA, ground system developers, air traffic
controllers, and maintenance technicians.

o Commissioning: The commissioning phase ensures that the new ground
system as installed meets the intended mission and operational
requirements and is fully supported by the national airspace system
infrastructure. Potential stakeholders in this phase include FAA, ground
system developers, air traffic controllers, and maintenance technicians.

Aircraft Equipment Certification Process

In contrast to the ground system approval process, certification of
aircraft equipment is done in accordance with procedures outlined in  the
Federal Aviation Regulations, Title 14, Code of Federal Regulations, Part
21. Under Title 49, Section 44704, of the U.S. Code, FAA has the authority
to issue type certificates, supplemental type certificates, and production
certificates, among others, for aircraft and equipment that will be used
in the national airspace system.9 Unlike the approval of ground systems,
which FAA accomplishes with the help of a contractor, FAA is the regulator
of aircraft equipment and is not typically involved in the development of
the equipment. An applicant, such as a manufacturer of aircraft equipment,
generally brings fully developed aircraft equipment to FAA for
certification. The aircraft equipment certification process includes the
following five phases-concept of operations, requirements setting, design
and production approval, installation approval, and operational
approval-and involves several stakeholders, which are also noted below:

o Concept of operations: Like the ground system approval process, the
aircraft equipment certification process generally begins with the concept
of operations phase, when the aircraft equipment is part of an ATC system.
If the aircraft equipment certification process is not associated with the
approval of a new ground system, then the certification process may begin
with an idea for better equipment. During this phase, FAA, sometimes with
the help of industry, identifies and defines a service or capability to
meet a particular need in the national airspace system.10 Potential
stakeholders in this phase include FAA's Office of Regulation and
Certification, FAA's Air Traffic Organization, aircraft manufacturers,
aviation industry associations, airlines, air traffic controllers,
maintenance technicians, manufacturers of aircraft equipment, ground
system developers, and representatives of general aviation.

o Requirements setting:  Once FAA has identified the need for a new system
with aircraft equipment, FAA determines the requirements for

the aircraft equipment.11 In some cases, the requirements for aircraft
equipment may already exist in the Federal Aviation Regulations. In other
cases, FAA may ask RTCA to develop the requirements, including safety
requirements, which are referred to as minimum operating performance
standards. RTCA typically takes 1 to 5 years to develop the standards
because of the need to reach consensus between FAA and the industry and
the increasing complexity of systems being developed today. According to a
RTCA official, the time required to develop recommended standards is a
function of many variables, including urgency of the situation and the
commitment and availability of government and industry volunteers to
collaboratively develop the standards. For example, in the case of WAAS,
RTCA began setting performance standards in 1994, completed the original
version of the standards in January 1996, and completed the most recent
version of WAAS performance standards in November 2001. Potential
stakeholders in this phase include FAA's Office of Regulation and
Certification, FAA's Air Traffic Organization, aircraft manufacturers,
aviation industry associations, airlines, air traffic controllers,
maintenance technicians, manufacturers of aircraft equipment, ground
system developers, and representatives of general aviation.

o Design and production approval:  The requirements/performance standards,
most often developed by RTCA, typically form the basis for a technical
standard order, which FAA uses to grant design and production approval for
most new aircraft equipment developed in support of national airspace
system modernization efforts. Technical standard orders are FAA's
requirements for materials, parts, processes, and appliances used on civil
aircraft.12 Most aircraft manufacturers want technical standard orders
because they make installation approval simpler and less costly and allow
for operation in any type of aircraft. Technical standard orders are
issued for items ranging from safety belts to navigation equipment. If the
applicant successfully completes the design and production approval phase,
FAA provides the applicant with a technical standard order authorization
letter, which states that the applicant has met a specific technical
standard order and the product is now ready for the installation approval
phase. Potential stakeholders in this phase include FAA's Aircraft
Certification Service, manufacturers of aircraft equipment, and aircraft
manufacturers.

o Installation approval: After receiving a technical standard order
authorization for new aircraft equipment, the initial applicant must
receive installation approval from FAA before the aircraft equipment may
be used in the national airspace system. To receive installation approval,
the applicant submits a certification plan and test plan to one of FAA's
aircraft certification offices for review and approval. In addition, the
applicant conducts ground and flight tests under FAA's supervision to
ensure that the new equipment operates properly upon installation. Once
the tests are completed to FAA's satisfaction, FAA issues a supplemental
type certificate, which is evidence of FAA's approval to modify an
aircraft from its original design. Potential stakeholders in this phase
include FAA's Aircraft Certification Service, manufacturers of aircraft
equipment, and aircraft manufacturers.

o Operational approval: Finally, for the aircraft equipment to become
certified for use in the national airspace system by air carrier
operators, operational approval is also needed from FAA. To obtain
operational approval, the applicant must successfully demonstrate, among
other things, that the pilots are properly trained to use the aircraft
equipment and that maintenance personnel are properly trained to maintain
the equipment. Potential stakeholders in this phase include FAA's Flight
Standards Service, airlines, and representatives of general aviation.

FAA Faced Challenges in Approving Several ATC Systems

FAA faced challenges in approving systems for safe use in the national
airspace system that contributed to cost growth, delays, and performance
shortfalls in deploying these systems. We identified three specific

challenges through the review of 5 ATC systems and our past work.13 These
challenges are the need to

o involve appropriate stakeholders, such as users and technical experts,
throughout the approval process;

o ensure that the FAA offices that have responsibility for approving
ground systems and certifying aircraft equipment effectively coordinate
their efforts for integrated systems; and

o accurately estimate the amount of time needed to meet complex technical
requirements at the beginning of the design and development phase.

Although most of the challenges we found relate to the ground system
approval process, RTCA and the Aerospace Commission have identified
challenges with FAA's aircraft equipment certification process. For
example, RTCA found that there was a need for better internal FAA
communication and coordination, including the establishment of an
organizational focal point to provide coordinated responses to all matters
related to ground systems and aircraft equipment. In addition, the
Aerospace Commission found that FAA's regulatory process needs to be
streamlined to enable the timely development of regulations needed to
address new technologies.

FAA Did Not Always Adequately Involve Appropriate Stakeholders, Such as
Users and Technical Experts, Throughout Its Approval Process

FAA failed to adequately involve appropriate stakeholders, such as air
traffic controllers and maintenance technicians, for 3 of the 5 systems we
reviewed. For example, FAA did not adequately involve controllers and
maintenance technicians throughout the approval process of STARS, which
will replace controller workstations with new color displays, processors,
and computer software. Although controllers and technicians were involved
in developing requirements for STARS in 1994 prior to the 1996 contract
award to Raytheon, the original approved acquisition plan provided for
only limited human factors evaluation by controllers and technicians
during STARS' design and development because the aggressive development
schedule limited the amount of time available to involve them.14
Consequently, FAA and Raytheon had to restructure the contract to address
controllers' concerns that were identified later, such as the
inconsistency of visual warning alarms and color codes with the new
system. According to FAA officials, not involving controllers and
maintenance technicians in the design phase caused the agency to revise
its strategy for acquiring and approving STARS, which contributed to
STARS' overall cost growth of $500 million and added 3 years to the
schedule.

FAA also did not always sufficiently involve technical experts early in
its approval process for 2 additional systems that we reviewed. For
example, FAA did not obtain technical expertise on how to resolve the
integrity requirement of WAAS, a navigation system for aviation that
augments the Global Positioning System (GPS), until late in the design and
development phase.15 FAA acknowledges that the agency's in-house technical
expertise was not sufficient to address the technical challenges of WAAS.
Initially, FAA and the contractor believed they could meet the WAAS
integrity requirement to alert the pilot in a timely manner when the
system should not be used. However, although WAAS was being developed by
an integrated product team that included representatives from several FAA
offices, the team did not function effectively in resolving issues related
to meeting an important functional requirement to alert the pilot in a
timely manner when the system should not be used because of a possible
error. According to FAA officials, the reason coordination did not occur
was that the two offices had competing priorities that were not associated
with WAAS' development.  Consequently, in 2000, FAA convened the WAAS
Integrity Performance Panel to help it meet the integrity requirement. The
WAAS Integrity Panel worked for about 2-1/2 years before it came up with a
solution to the integrity requirement. In addition, in August 2000, the
agency established an Independent Review Board, which is independent of
the panel and included experts in satellite navigation and safety
certification, to oversee the panel and evaluate the soundness of its
efforts. According to a member of the WAAS Integrity Panel, if FAA had
involved these technical groups immediately after the contract was awarded
to Raytheon in 1996, these groups could have started devising a solution
in 1996, rather than in 2000. This lack of technical expertise contributed
to a 6-year delay in WAAS' commissioning and a $1.5 billion increase in
its development costs from the 1994 baseline.16

FAA also did not fully engage technical experts early in the approval
process of LAAS, a precision approach and landing system that will augment
GPS. According to FAA officials, meeting the LAAS integrity requirement to
alert the pilot in a timely manner when the system should not be used is
perhaps the most difficult part of approving this system for safe use in
the national airspace system. According to the Department of
Transportation's Inspector General, although FAA had a LAAS Integrity
Panel in place since 1996 to assist with its research and development
activities, the panel was not formally tasked with resolving LAAS'
integrity issues. According to one satellite navigation expert and the
Department of Transportation's Inspector General, focusing the LAAS
Integrity Panel on resolving the integrity requirement early in the
approval process may have enabled FAA to develop a quicker solution.17 In
2003, FAA focused the LAAS Integrity Panel on developing a solution to
meet the integrity requirement. However, FAA and another satellite expert
maintain that the technical complexity of this problem is the main reason
that LAAS is not commissioned. According to FAA officials, the need to
validate integrity requirements and further software development has
resulted in FAA placing LAAS in its research and development program and
suspending funding for fiscal year 2005.

In contrast, FAA faced fewer schedule and cost problems in approving
ASDE-X for use in the national airspace system. This was, in part, because
FAA included stakeholders early and throughout the approval process and
because program managers had strong technical expertise. The ASDE-X
program office brought in stakeholders, including maintenance technicians
and air traffic controllers, during the concept of operations phase and
continued to involve them during requirements setting, design and
development, and test and evaluation. FAA also brought ASDE-X stakeholders
together at technical meetings to provide input on ASDE-X design and
development, which allowed the ASDE-X program office to design a system
that met requirements and incorporated stakeholders' needs. By obtaining
the input of controllers and technicians at the beginning of the approval
process, FAA was able to ensure that ASDE-X requirements were set at
appropriate levels and not overspecified or underspecified. Some
stakeholders commented that the program managers' strong technical
expertise was one reason that ASDE-X's requirements were set
appropriately. As a result, this system was initially commissioned only 5
months behind schedule and its cost increased moderately from $424 million
to $510 million.

FAA Did Not Always Effectively  Coordinate Its Certification and Approval
Processes

FAA did not always effectively coordinate its certification and approval
processes for CPDLC, WAAS, and LAAS. Coordination between FAA's offices
responsible for approval of ground systems and certification of aircraft
equipment is becoming increasingly important given that more and more ATC
systems have both ground systems and aircraft equipment. However, we found
that coordination was not effective on CPDLC Build 1A, which allows pilots
and controllers to transmit digital data messages directly between FAA
ground automation systems and suitably equipped aircraft.18 In the
interest of meeting the original cost and schedule estimates, FAA awarded
the contract before it had a full understanding of system requirements.
Requirements that specify how the ground system and aircraft equipment
would operate together were not yet completed prior to award of the Build
1A contract. Consequently, changes needed to be made after the contract
was awarded. New hardware requirements, software requirements, and other
system requirement changes were added, which increased CPDLC's costs by
$41 million, almost 61 percent of the total cost increases associated with
CPDLC.

The lack of effective coordination among FAA offices responsible for
approving WAAS also contributed to delays and increased costs in
commissioning WAAS. Although WAAS was being developed by an integrated
product team that included representatives from various FAA offices, the
team did not function effectively in resolving issues related to meeting
an important functional requirement to alert the pilot in a timely manner
when the system should not be used because of a possible error. According
to FAA officials, the reason coordination was not effective was because
the two offices had competing priorities that were not associated with
development of WAAS. Consequently, it was not until September 1999, when
the aircraft certification office became fully involved, that FAA
recognized that its solution to meet WAAS' integrity requirement was not
sufficient and that it did not have the technical expertise needed to
develop a solution. This lack of coordination contributed to a 6-year
delay in WAAS' commissioning and a $1.5 billion increase in its
development costs.

LAAS is another example of how FAA did not effectively coordinate its
efforts. For example, FAA's Office of Regulation and Certification
completed the design and production approval of LAAS aircraft equipment
without effectively coordinating with the offices responsible for
acquisition to determine the consequences of certifying aircraft equipment
before approval of the associated ground system. According to an FAA
official, once the Office of Regulation and Certification has given design
and production approval to the LAAS aircraft equipment, it is not possible
to make a change to the requirements for the aircraft equipment so that
they are better integrated with the associated LAAS ground system.
Consequently, LAAS ground system developers may have to make more costly
and time-consuming changes to the ground system than would have been
necessary if the Office of Regulation and Certification and acquisitions
offices had coordinated their efforts.

FAA Did Not Always Prepare Accurate Estimates of the Amount of Time Needed
to Meet Complex Technical Requirements

We have reported in the past that when FAA attempts to combine different
phases of system development in an effort to more quickly implement the
systems to meet milestones, it repeatedly experiences major performance
shortfalls and rework, which leads to schedule delays and cost
increases.19 We found that WAAS, STARS, and LAAS all experienced delays
and cost increases in part because FAA did not prepare accurate estimates
of the amount of time needed to meet complex technical requirements,
leading to an accelerated schedule that sometimes failed to include
activities such as human factors evaluations and technical expert
consultations. For example, in 1994, in response to the concerns of
government and aviation groups, FAA accelerated implementation of WAAS
milestones from 2000 to 1997. FAA planned to develop, test, and deploy
WAAS within 28 months, an unrealistic goal given that software development
alone was expected to take 24 to 28 months. It was not until July 2003,
over 6 years later, that FAA was able to commission WAAS for initial
operating capability. The accelerated schedule contributed to the 6-year
delay in the commissioning of the system because the schedule itself was
unrealistic and additional design work needed to be completed. During that
time, the cost to develop the system increased about $1.5 billion, and the
system has yet to meet its original performance goal of providing pilots
with the ability to navigate down to 200 feet during their approach to the
runway.

FAA also accelerated the schedule for STARS in 1995. FAA's approach to
commissioning STARS was oriented to rapid deployment to meet critical
needs for new equipment. To meet these needs, FAA compressed its original
development and testing schedule from 32 to 25 months. Consequently, this
acceleration in schedule left only limited time for human factors
evaluations and, according to FAA officials, contributed to STARS' overall
cost growth of $500 million and added 3 years to the first deployment
because the agency had to revise its strategy for acquiring and approving
STARS.

Although FAA had not developed a solution for meeting the integrity
requirement, FAA also accelerated the LAAS schedule in 1999 by setting
system milestones before completely designing the system. FAA originally
planned to deploy LAAS in 2002 but has since moved it to fiscal year 2009
because the system's software development is not complete and a solution
for meeting LAAS' integrity requirements has yet to be developed.

RTCA and the Aerospace Commission Found Challenges with FAA's Process for
Approving Ground Systems and Certifying Aircraft Equipment

RTCA and the Aerospace Commission also identified challenges with FAA's
process for approving ground systems and certifying aircraft equipment. In
1998, at the request of the FAA Administrator, RTCA reviewed FAA's
certification/approval process to determine if it could be made more
responsive to the changing state of aviation, including its more
integrated technologies. RTCA found that FAA's ground system approval
process and aircraft equipment certification process took too long and
cost too much, and RTCA made several recommendations to improve the
processes. For example, in 2001, RTCA recommended that FAA implement a
coordinated approval process that, among other things, would ensure that
all stakeholders, including those outside FAA's program offices,
participate in all phases of the approval process. Specifically, similar
to our finding that the FAA offices that had responsibility for approving
ground systems and certifying aircraft equipment did not always
effectively coordinate their efforts, RTCA found that there was a need for
better internal FAA communication and coordination, including the
establishment of an organizational focal point to provide coordinated
responses to all matters related to ground systems and aircraft equipment.
RTCA also found that there was a need for an earlier and better exchange
of information between FAA and those involved in the approval and
certification processes from outside FAA, such as manufacturers of
aircraft equipment.20

In 2000, Congress asked the Commission on the Future of the U.S. Aerospace
Industry to study the health of the aerospace industry and identify
actions that the United States needs to take to ensure the industry's
health. As part of this study, the Aerospace Commission reviewed FAA's
certification process for aircraft equipment and made recommendations. The
Aerospace Commission found that FAA's certification of new aircraft
technologies has become uncertain in terms of time and cost and
recommended that FAA's regulatory process be streamlined to enable the
timely development of regulations needed to address new technologies.
According to the Aerospace Commission, instead of focusing on rules and
regulations that dictate the design and approval of equipment, FAA should
focus on certifying that manufacturing organizations have safety built
into their processes for designing, testing, and ensuring the performance
of an overall system. The commission believed that such an approach would
allow FAA personnel to better keep up with technological progress by
becoming less design-specific and more safety-focused.

FAA Has Taken Action to Improve Its Process for Approving ATC Systems

FAA has taken action to address two of the three management challenges
that we identified. However, FAA has not taken action to ensure that all
stakeholders, such as air traffic controllers, maintenance technicians,
technical experts, and industry representatives, are involved throughout
the ground system approval process. FAA has also taken some action to
address recommendations made by RTCA and the Aerospace Commission.
Examples of some of the actions FAA has taken that address the management
challenges that we found as well as RTCA and Aerospace Commission
recommendations are discussed below:

o Coordinating FAA's acquisitions offices and Office of Regulation and
Certification efforts for approving systems with ground and aircraft
components: FAA officials believe that the agency's new Safety Management
System, which is designed to formalize the agency's safety process, will
also improve coordination among FAA internal stakeholders once it is
implemented. FAA stated that coordination would improve because as part of
the new Safety Management System the agency plans to realign its
organizational structure to create a formal link between the Air Traffic
Organization and the Office of Regulation and Certification. Within the
Office of Regulation and Certification,  there is the newly created Air
Traffic Safety Oversight Service, which oversees the safety operations of
the Air Traffic Organization and collaborates with the Air Traffic
Organization's Safety Services. In addition, according to FAA officials,
both ground systems and aircraft equipment will be more consistently
assessed for their effect on safety as safety terminology is standardized.
FAA expects full implementation to take 3 to 5 years. We are reserving
judgment on whether this change will fully address the challenge because
of the early state of this effort and because FAA's problems with internal
coordination when approving ATC systems are long-standing. In addition,
because FAA has historically faced internal and external coordination
challenges in approving ATC systems for safe use in the national airspace,
we believe that as FAA moves forward with the agency's new Safety
Management System, it should, in the interim, develop plans that describe
how both internal and external coordination will occur on a
system-specific basis. In addition, plans to include external stakeholders
are particularly important since the Safety Management System is not
intended to address this challenge.

o Estimating the amount of time needed to meet complex technical
requirements: During the development of WAAS and STARS, FAA adopted an
incremental approach to developing and testing these systems to get them
back on track, which is referred to as the "build a little, test a little"
or spiral development approach.  For example, to get WAAS back on track,
FAA decided to take a more incremental approach to implementing the new
navigation system-focusing more on the successful completion of research
and development before starting system approval. In particular, FAA
allowed time for collecting and evaluating data on key system performance
requirements like the WAAS integrity requirement before moving forward.
FAA officials acknowledged that the manner in which FAA decided to
implement WAAS development before implementing this incremental approach
was a high-risk approach and was a primary issue underlying the system's
problems. Some aviation stakeholders believe this approach is advantageous
because, although it can increase costs initially, money can be saved in
the long run because the approach may help to avoid mistakes that are very
costly to fix once a system has been developed. This approach also helps
to ensure that the necessary building blocks of a system are tested along
the way through the early and ongoing involvement of key stakeholders,
those who will use and maintain the system. These stakeholders are key to
identifying critical omissions and issues that could prevent a system from
operating as intended.21

As previously discussed, RTCA and the Aerospace Commission reviewed FAA's
approval process and made a number of recommendations to improve it. FAA
has taken some action to address these recommendations. For example:

o In response to RTCA's recommendation to implement a process in which the
regulators and applicants come to an early and clear agreement on their
respective roles, responsibilities, expectations, schedules, and standards
to be used in certification projects, FAA issued The FAA and Industry
Guide to Avionics Approval in 2001, which is intended to help FAA reduce
the time and cost for the certification of aircraft equipment. This guide
describes how to plan, manage, and document an effective, efficient
aircraft equipment certification process and how to develop a working
relationship between FAA and the applicant. In addition, as part of the
1999 FAA and Industry Guide to Product Certification, FAA encourages the
manufacturers of aircraft equipment to develop a Partnership for Safety
Plan that defines roles and responsibilities, describes how the
certification process will be conducted, and identifies the milestones for
completing the certification. A WAAS aircraft equipment manufacturer said
that the certification of the WAAS aircraft equipment it developed went
smoothly, primarily because of this up-front agreement with FAA. Although
FAA's actions address the aircraft equipment certification process, it
does not have a similar process for its ground system approval process.

o In response to RTCA's recommendation to establish an organizational
focal point to provide one-stop service to users, industry, and other
governments in all matters related to advanced ground electronics and
aircraft equipment, FAA has completed a Web site that provides a broad
range of information on the certification process for aircraft equipment.
However, there is still no focal point to which industry can address
questions about the approval process and be assured of getting a fully
coordinated FAA answer.

o In response to the Aerospace Commission's recommendation to streamline
its aircraft equipment certification process to ensure timely development
of regulations needed to address new technologies and to focus on
certifying that manufacturing organizations have built safety into their
processes for designing, testing, and ensuring the performance of an
overall system, FAA proposed creating an Organizational Designation
Authorization program in January 2004. The program would expand the
approval functions of FAA organizational designees,22 standardize these
functions to increase efficiency, and expand eligibility for
organizational designees.

Conclusions

FAA did not always include stakeholders throughout the process for
approving ATC systems for safe use in the national airspace system.
Including stakeholders is particularly important because the new ATC
systems are more integrated today than in the past and thus require more
coordination among all the stakeholders, particularly FAA's  Office of
Regulation and Certification  and the recently created Air Traffic
Organization, but also between FAA and other stakeholders, such as
technical experts, controllers, and maintenance technicians. When
decisions regarding integrated ATC systems are made in isolation, they may
contribute to the ineffective use of resources and time. We found that 3
of the 5 ATC systems we reviewed experienced cost growth and schedule
delays, in part, because FAA did not always involve all necessary
stakeholders, such as controllers and technical experts, throughout the
approval process. In 2001, RTCA recommended that FAA implement a
coordinated approval process that, among other things, would ensure that
all stakeholders, including those outside FAA's program offices,
participate in all phases of the approval process. We agree with RTCA's
recommendation, which FAA has not fully implemented, and believe that
fully implementing it would help address some of the challenges we found
with FAA's approval and certification processes.

In addition, although FAA's new Safety Management System and the planned
alignment between FAA's Air Traffic Organization and Office of Regulation
and Certification have the potential to improve FAA's internal
coordination, FAA has just begun implementing these initiatives with full
implementation 3 to 5 years away. FAA also has historically faced internal
coordination challenges in approving ATC systems for safe use in the
national airspace system as we found for each of the 3 integrated systems
that we reviewed. We believe that the implementation of the Safety
Management System, coupled with the new formal link between FAA's Air
Traffic Organization and Office of Regulation and Certification, will give
FAA the opportunity to improve its internal coordination among its offices
that are responsible for ground system approval and aircraft equipment
certification. However, the system will not be implemented until 3 to 5
years. Therefore, because of FAA's history of internal and external
coordination challenges, such as the lack of effective coordination
between FAA offices responsible for approving WAAS, which contributed to
WAAS' cost increase of about $1.5 billion and schedule delays of 6 years,
we believe that specific plans for improving coordination both internally
and externally on a system-specific basis are needed now.

Recommendation for Executive Action

To ensure that key stakeholders, such as air traffic controllers,
maintenance technicians, and technical experts, outside FAA's acquisitions
offices and Office of Regulation and Certification, are involved early and
throughout FAA's ground system approval process and to ensure better
internal coordination between FAA's offices responsible for approving
ground systems and certifying aircraft equipment, we recommend that the
Secretary of Transportation direct the Administrator of FAA to develop ATC
system-specific plans early in the approval process that specify how and
when the approving and certifying offices within FAA and other
stakeholders, including controllers, maintenance technicians, technical
experts, and industry representatives, will meet to ensure coordination.

Agency Comments

We provided a draft of this report to the Secretary of Transportation for
review and comment. FAA generally agreed with our findings and
recommendation and provided technical corrections, which we incorporated
as appropriate. FAA also commented that it has started to take actions to
improve its coordination efforts for integrated ATC systems.

We are sending copies of this report to interested congressional
committees, the Secretary of Transportation, and the FAA Administrator. We
will also make copies available to others on request. In addition, the
report will be available at no charge on the GAO Web site at
http://www.gao.gov. Should you or your staff have questions on matters
discussed in this report, please contact me on (202) 512-2834 or at
[email protected]. GAO contacts and key contributors to this report are
listed in appendix VII.

Sincerely yours,

Katherine Siggerud Director, Physical Infrastructure Issues

Objectives, Scope, and MethodologyAppendix I

To complete our first objective, to describe FAA's process for approving
air traffic control (ATC) systems for safe use in the national airspace
system, we obtained and analyzed documents from the Federal Aviation
Administration (FAA) and RTCA's1 1999 report that discussed FAA's process
for certifying aircraft equipment and approving ground systems. We also
interviewed FAA officials, contractors, industry experts, and unions
representing air traffic controllers and maintenance technicians that are
involved in approving ATC systems.

To complete our second objective, to describe the challenges FAA has faced
approving ATC systems and how those challenges affected the cost,
schedule, and performance estimates of the systems, we conducted case
illustrations on 5 of FAA's 25 air traffic control systems that are
currently receiving funding:

o Airport Surface Detection Equipment - Model X (ASDE-X),

o Controller-Pilot Data Link Communications (CPDLC),

o Local Area Augmentation System (LAAS),

o Standard Terminal Automation Replacement System (STARS), and

o Wide Area Augmentation System (WAAS).

We selected these 5 systems because collectively they accounted for about
46 percent of FAA's ATC modernization costs in fiscal year 2002 and 3 of
the 5 systems are integrated-that is, they require the approval of the
ground systems as well as aircraft equipment. To select the 5 case
illustration systems, we used FAA's capital investment project data file.
We met with knowledgeable FAA officials to discuss issues related to the
accuracy and completeness of the data file, which was deemed adequate for
the purpose of our work. We also met with knowledgeable FAA officials to
determine the number of ATC systems from the data file that needed to be
approved before entry into the national airspace system. For each of the
case illustrations, we reviewed FAA documents, including acquisition
program baseline reports, Joint Resource Council decisions, and briefing
documents. We also reviewed GAO and Department of Transportation's
Inspector General reports and testimonies. In addition, we interviewed
officials from FAA program offices; RTCA; the General Aviation
Manufacturers Association; the Air Transport Association; the Aircraft
Owners and Pilots Association; NavCanada; Transport Canada; the MITRE
Corporation; Boeing; Garmin; Rockwell Collins; contractors, including
Honeywell, Raytheon, and the Sensis Corporation; industry experts; the
WAAS Integrity Performance Panel; the LAAS Integrity Panel members; and
unions representing air traffic controllers and maintenance technicians.

To compete our third objective, to describe actions FAA has taken to
improve its processes for approving ATC systems, we interviewed
representatives from FAA; RTCA; the Commission on the Future of the U.S.
Aerospace Industry; aviation industry groups, including the General
Aviation Manufacturers Association, the Air Transport Association, and the
Aircraft Owners and Pilots Association; manufacturers of aircraft
equipment, including Garmin and Rockwell Collins; Boeing; and contractors,
including Honeywell, Raytheon, and the Sensis Corporation; industry
experts; and unions representing air traffic controllers and maintenance
technicians.

We conducted our review in Washington, D.C., from October 2003 through
September 2004 in accordance with generally accepted government auditing
standards.

Airport Surface Detection Equipment - Model X Case IllustrationAppendix II

Background

ASDE-X is an airport surface surveillance system that air traffic
controllers use to track aircraft and vehicle surface movements. (See fig.
2.) ASDE-X uses a combination of surface movement primary radar and
multilateration1 sensors to display aircraft position and vehicle position
on an ATC tower display. According to FAA, the integration of these
sensors provides accurate, up-to-date, and reliable data for improving
airport safety in all weather conditions. ASDE-X was developed to prevent
accidents resulting from runway incursions,2 which have increased since
1993. The number of reported runway incursions rose from 186 in 1993 to
383 in 2001. According to FAA, because air traffic in the United States is
expected to double by 2010, runway incursions may pose a significant
safety threat to U.S. aviation.

FAA expects that ASDE-X will increase the level of safety at airports and
provide air traffic controllers with detailed information about aircraft
locations and movement at night and in bad weather due to the (1)
association of flight plan information with aircraft position on
controller displays; (2) continuous surveillance coverage of the airport
from arrival through departure; (3) elimination of blind spots and
coverage gaps; and (4) availability of surveillance data with an accuracy
and update rate suitable for, among other things, awareness in all weather
conditions.

Figure 2: Airport Surface Detection Equipment - Model X

Status

In October 2003, FAA commissioned ASDE-X at Mitchell International Airport
in Milwaukee, Wisconsin, for use in the national airspace system. ASDE-X
came in close to its original schedule and cost baselines. The ASDE-X
system was approximately 5 months over its original schedule baseline, but
maintained its original performance baselines. In June 2002, FAA approved
$80.9 million in additional funding to add ASDE-X at 7 additional sites.
(See table 2.) FAA is currently scheduled to deploy ASDE-X at 25 U.S.
airports over the next 4 years and to update existing surface detection
systems (i.e., ASDE-3) at 9 other facilities. FAA plans to introduce an
upgraded ASDE-X system at T.F. Green Airport in Providence, Rhode Island,
with deployment tentatively slated for the 4th quarter of 2004. FAA is
also investigating whether to add ASDE-X at 25 airports that use ASDE-3
and Airport Movement Area Safety Systems.

Table 2: Cost and Schedule Estimate Changes to ASDE-X

Dollars in millions                                         
Baseline/Cost               Estimated  Initial operating    Full operating 
estimate year       development costs  capability           capability     
September 2001                $424.3a  May 2003             2007           
(baseline)                                                  
June 2002 (upgrade)             80.9b  September 2004       2005           
October 2003                                                               
                                  510.2c  October 2003         2007d
(in-service                                                 
decision)                                                   

Source: GAO presentation of FAA data.

aIncludes 25 operational ASDE-X sites, 4 support systems, and 1 ASDE-3
upgrade.

bIncludes 7 ASDE-3 site upgrades.

cThe October 2003 cost estimate includes a $5 million congressional
addition for Dulles Airport.

dAlthough the last approved baseline included the 2007 date for last
deployment, internal and external reprogramming for other high-priority
activities and budget decrements in fiscal years 2004 and 2005 will slip
the last deployment to fiscal year 2009. The ASDE-X program office is
preparing a baseline management notice to adjust the baseline.

FAA Faced Fewer Challenges in Approving ASDE-X

Of the five systems we reviewed, FAA faced fewer schedule and cost
challenges in approving ASDE-X for safe use in the national airspace
system. This is partly because FAA included stakeholders early and
throughout the approval process and because of the strong technical
expertise of its managers. The ASDE-X program office brought in
stakeholders, including maintenance technicians and air traffic
controllers, beginning with the concept of operations phase and continued
their stakeholder involvement through the requirements-setting,
design-and-development, and test-and-evaluation phases and then continued
involvement throughout the deployment phase. For example, FAA obtained the
input of controllers and technicians at the beginning of the approval
process, which helped to ensure that ASDE-X requirements were set at
appropriate levels and not overspecified or underspecified. Stakeholders
pointed toward the strong technical expertise of the program's managers as
a reason for the appropriate specification of ASDE-X's requirements. In
addition, FAA brought ASDE-X stakeholders together at technical meetings
to provide input on ASDE-X design and development, which allowed the
ASDE-X program office to design a system that met requirements and
incorporated stakeholders' needs.

Controller-Pilot Data Link Communications Case Illustration Appendix III

Background

CPDLC will allow pilots and controllers to transmit digital data messages
directly between FAA ground automation computers and suitably equipped
aircraft. (See fig. 3.) CPDLC is a new way for controllers and pilots to
communicate that is analogous to e-mail. The pilot can read the message
displayed on a screen in the cockpit and respond to the message with the
push of a key. In the future, this will alleviate frequency congestion
problems and increase controller efficiency. One of the most important
aspects of this technology is its intended reduction of operational errors
from misunderstood instructions and readback errors. The initial phase
(Build 1) consisted of four services: initial contact, altimeter1 setting,
transfer of communication, and predefined instructions via menu text. The
CPDLC program will ultimately develop additional capabilities in an
incremental manner through further development stages. Originally, Build 1
was to be followed by Build 1A, which was designed to increase the CPDLC
message set and include assignment of speeds, headings, and altitudes as
well as a route clearance function.

Figure 3: Controller-Pilot Data Link Communications

Status

CPDLC was commissioned for initial daily use by controllers at Miami on
October 7, 2002. This completed the stage called Build 1, which included
four services. American Airlines is the CPDLC launch airline with about 25
aircraft operating in the Miami Center airspace. Further deployment of
CPDLC has been deferred until about 2009 after the Joint Resources Council
did not approve the program in April 2003. The council made this decision
because it believed that the benefits of CPDLC did not outweigh the costs.
A number of factors contributed to this decision. First, FAA had concerns
about how quickly aircraft would install the new airborne equipment.
Second, the approved program baseline was no longer valid as Build 1A
investment costs had increased from $114.5 million to $181.7 million,
while the number of locations decreased from 20 to 8 as shown in table 4.
Third, CPDLC would add $83 million to the operations account.

Table 4: Cost and Schedule Estimate Changes to CPDLC

Dollars in                                 
millions                                   
                                 Initial       Initial              Locations 
Baseline/Cost      Estimated  operational   operational                    
estimate year    development  capability -  capability -      (after Build 
                         costsa  Build 1       Build 1A                  1A - 
                                                                  completion) 
1999 (Build 1A)       $114.5  June 2002     June 2005                   20 
April 2003b            181.7  October 2002  Undetermined                 8 

Source: GAO presentation of FAA data.

aCPDLC Build 1 costs were $52.2 million.

bFAA did not approve this cost estimate.

For fiscal year 2005, program officials requested $3 million for CPDLC.
According to FAA, this amount would be suitable for shutdown of CPDLC at
Miami, closeout of Build 1, and alternatives analysis for a follow-on
program. The contractor, ARINC, had been providing messaging service for
Miami at no cost. However, the contract for this free service expired on
June 30, 2004.

Challenges in Approving CPDLC

Lack of full coordination between FAA's aircraft certification and
acquisition offices, in which there would have been a full understanding
of all requirements, compromised the schedule and cost of CPDLC. FAA's
acquisitions office, in the interest of meeting the original cost and
schedule estimates, awarded the contract before FAA had a full
understanding of system requirements, including those of FAA's aircraft
certification office. Requirements that specified in detail how the air
and ground equipment would operate together were not yet completed prior
to award of the Build 1A contract. The addition of CPDLC hardware and
software requirements increased costs by $26 million, 39 percent of
CPDLC's Build 1A development cost growth. In addition, other system
requirement changes after contract award increased CPDLC's baseline
development cost estimate by another $15 million. In total, these
requirement additions increased costs by $41 million, almost 61 percent of
the total cost increases associated with CPDLC Build 1A. (See tables 5, 6,
and 7 for timelines of CPDLC's ground system approval and aircraft
equipment certification.)

Table 5: CPDLC Ground System Approval Timeline (Build 1)

Phase                                     Date                             
Concept of operations (initial)           October 1991                     
Requirements setting                      
Final requirements document               October 1998; revised April 2003 
Contract award                            January 1999                     
Design and development                    
Critical design review                    September 2000                   
Test and evaluation                       
Development test                          February 2002                    
Operational test                          December 2001                    
Independent operational test and          Early assessment - March 2003    
evaluation                                
Initial operating capability              October 2002                     
Operational readiness                     October 2002                     
Commissioning (Build 1)                   October 2002                     

Source: GAO presentation of FAA data.

Table 6: CPDLC Ground System Approval Timeline (Build 1A)

Phase                                          Date                        
Concept of operations (initial)                October 1991                
Requirements setting                           
Final requirements document                    November 2002               
Investment analysisa                           July 2003                   

Source: GAO presentation of FAA data.

aProgram has been deferred since completion of the investment analysis.

Table 7: CPDLC Aircraft Equipment Certification Timeline

Phase                                             Date                     
Concept of operations (initial)                   October 1991             
Requirements setting                              
Certification plan (American Airlines)            August 2000              
Design and production approval                    May 2001                 
Installation approval                             May 2001                 
Operational approval                              September 2002           

Source: GAO presentation of FAA data.

Local Area Augmentation System Case IllustrationAppendix IV

Background

LAAS is a precision approach and landing system that will augment the
Global Positioning System (GPS)  1 to broadcast highly accurate
information to aircraft on the final phases of a flight. LAAS is being
developed specifically to provide augmentation to GPS satellites to
support Category I, II, and III precision approach and landing capability2
to aircraft operating within a 20- to 30-mile radius of an airport. LAAS
approaches are to be designed to avoid obstacles, restricted airspace,
noise-sensitive areas, or congested airspace. In addition, a single LAAS
ground station is to be capable of providing precision approach capability
to multiple runways. LAAS has both ground and air components. LAAS ground
components include four or more GPS reference receivers, which monitor and
track GPS signals; very high frequency transmitters for broadcasting the
LAAS signal to aircraft; and ground station equipment, which generates
precision approach data and is housed at or near an airport. (See fig. 4.)
LAAS users will have to purchase aircraft equipment to take advantage of
the system's benefits.

Figure 4: LAAS Infrastructure

Status

FAA's fiscal year 2005 budget request eliminated funding for LAAS, which
is being moved from the acquisition program into a research and
development effort. LAAS was slated for a 2006 rollout, but the target has
now been deferred until at least 2009. FAA officials said they will
reconsider national deployment when more research results are completed.

Before FAA decided to suspend funding for LAAS in fiscal year 2005, the
LAAS program office was negotiating with Honeywell to develop a plan for
determining how to meet the integrity requirements for the LAAS Category I
system. According to FAA officials, the LAAS program office will use the
$18 million remaining in fiscal year 2004 to continue the LAAS Integrity
Panel for developing the LAAS Category I system, to validate LAAS Category
II/III requirements, and to solve radio frequency interference issues. The
$18 million will last through 2005, and FAA's goal is to meet LAAS
integrity requirement by September 2005. Because of the budget cuts in
fiscal year 2005, the LAAS program office will not be developing a
Category II/III prototype.

As shown in table 8, the LAAS Category I system was initially expected to
be operational in 2002. However, FAA was unable to meet the milestone,
primarily due to development and integrity requirement issues. According
to FAA officials, the research needed to validate the integrity
requirement of LAAS Category I is scheduled to be completed by September
2005. If funds are fully restored in fiscal year 2005, FAA officials said
that a LAAS Category I system can be developed and deployed by fiscal year
2009.

Table 8: Cost and Schedule Estimate Changes to LAAS

Dollars in millions                                      
Baseline/Cost               Estimated  Initial operating  Full operating   
estimate year       development costs  capability         capability       
January 1998                   $530.1  2002               To be determined 
(baseline)                                                
September 1999                  696.1  2001               To be determined 

Source: GAO presentation of FAA data.

FAA Faced Challenges in Approving LAAS

FAA faced a number of challenges in approving LAAS for safe use in the
national airspace system, including (1) its inability to meet LAAS'
integrity requirement, (2) not always communicating with the contractor
about what was required to satisfy LAAS ground system requirements, and
(3) accelerating the LAAS schedule by setting milestones before designing
the system.

According to Honeywell officials, meeting the integrity requirement has
been perhaps the most difficult part of approving LAAS for safe use in the
national airspace system. Under FAA's integrity requirement for LAAS, the
system must alert the pilot with timely warnings when it should not be
used. However, FAA has not been able to develop a solution to meet this
requirement because it has not been able to prove that the system is safe
during solar storms. According to FAA officials, one of the reasons that
FAA has not been able to develop a solution to meet this requirement is
that a solar storm's effect on the ionosphere has not been modeled. The
modeling is scheduled for completion in September 2004, and it will be
used to design a monitor for ionosphere anomalies that could be developed
and deployed by fiscal year 2009.

FAA also did not always communicate with the contractor about what was
required to satisfy LAAS ground system requirements. Initially, FAA was in
a partnership with industry, including Honeywell and others, to develop a
LAAS Category I precision approach and landing system, which has a
200-foot ceiling height and one-half mile visibility. FAA partnered with
industry to develop LAAS because FAA would have to pay industry only if
industry achieved preset milestones, such as an analysis of the LAAS
system integrity requirement. However, the partnership was not able to
develop a system that FAA believed would operate safely in the national
airspace system. Consequently, FAA decided to acquire LAAS on its own. In
April 2003, FAA awarded a contract to Honeywell to develop a LAAS Category
I precision approach and landing system. At the time the contract was
awarded, FAA believed that 80 percent of the LAAS was developed and met
its ground system requirements based on a review of documents. However, 5
months later, after further review, FAA discovered that only about 20
percent of development was complete. Nevertheless, Honeywell believes it
met 80 percent of the LAAS requirements. Both parties attribute the
disagreement to lack of communication about what was needed to satisfy the
LAAS ground system requirements. In fiscal year 2005, FAA decided to
suspend funding and placed LAAS into its research and development program
due to a lack of software development and the inability of the system to
meet the integrity requirement. According to FAA officials, the research
needed to validate the integrity requirement of LAAS Category I is
scheduled to be completed by September 2005. If funds are fully  restored
in fiscal year 2005, FAA believes that a LAAS Category I system can be
developed and deployed by fiscal year 2009.

FAA also experienced challenges in approving LAAS because it accelerated
the schedule in 1998 to meet system milestones before completely designing
the system and developing a solution for meeting the LAAS integrity
requirement. FAA originally planned to deploy LAAS in 2002 but had to
subsequently delay deployment to 2006 because of additional development
work, evolving requirements, and unresolved issues regarding how the
system would be approved. Lack of a solution for verifying that its
integrity requirement had been met and incomplete software development
were significant approval issues facing the LAAS program.

Table 9 shows the major phases and time frames for approving the LAAS
ground system.

Table 9: LAAS Ground System Approval Timeline

Phase                                                    Date              
Concept of operations (initial)                          1992              
Requirements setting                                     
RTCA performance standards                               September 1998    
Creation of LAAS Integrity Panel                         1996              
Establishment of LAAS government industry partnership    1999              
Rebaseline #1                                            September 1999    
Integrity requirement concerns identified                December 2001     
Requirements document final                              June 2002         
LAAS cost estimate change (Category I only)              April 2002        
Contract award                                           April 2003        
Design and development                                   
Software development issues identified                   September 2003    
Critical design review                                   Not complete      
Test and evaluation                                      
Development test                                         Not complete      
Operational test and evaluation                          Not complete      
Independent operational test and evaluation              Not complete      
Operational readiness                                    Not complete      
Commissioning/Initial operating capability               Not complete      

Source: GAO presentation of FAA and RTCA data.

Certification of LAAS Aircraft Equipment Has Been Affected by Delays in
Ground System Approval

LAAS aircraft equipment received design and production approval in August
2004. It still awaits installation approval. (See table 10.) Because LAAS'
aircraft and ground components are linked, certification of LAAS aircraft
equipment has been affected by delays occurring during ground system
approval. For example, according to aviation industry officials,
requirement additions on LAAS' ground system led to requirement additions
on LAAS' aircraft equipment. According to aviation industry officials, the
addition of requirements to the ground system increased the cost and time
to develop aircraft equipment, which changed the calculation for industry
about whether developing LAAS aircraft equipment was a worthwhile
investment and discourages future investment in aircraft equipment that
will modernize the national airspace system.

FAA's Aircraft Certification Office Needs to Coordinate Better with
Acquisitions Offices

FAA's aircraft certification office completed the design and production
approval of LAAS aircraft equipment without coordinating with the offices
responsible for acquisition to determine the consequences of certifying
aircraft equipment before approval of the associated ground system.
According to an FAA official, once the aircraft certification office has
given design and production approval to the LAAS aircraft equipment, it is
not possible to make a change to the requirements for the aircraft
equipment so that they are better integrated with the associated LAAS
ground system. Consequently, LAAS ground system developers may have to
make more costly and time-consuming changes to the ground system than
would have been necessary if the aircraft certification and acquisitions
offices had coordinated their efforts.

Table 10: LAAS Aircraft Equipment Certification Timeline

Phase                                                Date                  
Concept of operations (initial)                      1992                  
Requirements setting                                 
LAAS minimum operating performance standards         1995 to 2001          
LAAS technical standard order development            March 2003            
Design and production approval                       August 2004           
Installation approval                                Not complete          
Operational approval                                 Not requireda         

Source: GAO presentation of FAA data.

aFAA first approved the use of GPS for aviation navigation in 1993, so new
aircraft equipment that uses GPS did not require a new operational
approval.

Standard Terminal Automation Replacement System Case IllustrationAppendix
V

Background

STARS is a joint Department of Transportation, FAA, and Department of
Defense (DOD) program established under 31 U.S.C. 1535, the Economy Act,
as amended, to replace aging FAA and DOD legacy terminal automation
systems with state-of-the-art terminal ATC systems. The joint program is
intended to avoid duplication of development and logistic costs while
providing easier transition of controllers between the civil and military
sectors. Civil and military air traffic controllers across the nation are
using STARS to direct aircraft near major airports. FAA's goal for STARS
is to provide an open, expandable terminal automation platform that can
accommodate future air traffic growth and allow for the introduction of
new hardware- and software-based tools to promote safety, maximize
operational efficiency, and improve controllers' productivity. FAA
believes that STARS will facilitate efforts to optimally configure the
terminal airspace around the country, exchange digital information between
pilots and controllers, and introduce new position and surveillance
capabilities for pilots. (See fig. 5.)

Figure 5: Standard Terminal Automation Replacement System

Status

In June 2003, FAA first commissioned STARS for use at the Philadelphia
International Airport in Pennsylvania. Currently, STARS is fully
operational at 25 FAA terminal radar control facilities and 17 DOD
facilities. Under the Air Traffic Organization's new business model of
breaking large and complex programs into smaller phases to control cost
and schedule, STARS is a candidate for further deployment to about 120 FAA
terminal radar control facilities. As shown in table 11, in April 2004,
FAA changed STARS' cost and schedule estimates for the third time and now
estimates that it will cost $1.46 billion to deploy STARS at the 50 most
important terminal radar control facilities that provide air traffic
control services to 20 of the nation's top 35 airports.  The original
baseline in February 1996 was $940 million for 172 systems. The April 2004
estimate is an increase of about $500 million for 122 fewer systems (i.e.,
over 70 percent less) than originally planned.

Table 11: Cost and Schedule Estimate Changes to STARS

Dollars in                                                     
billions                                                       
                     Estimated  Projected date for  Projected       Number of 
Baseline/Cost   development  first deployment of date for last FAA systems 
estimate year        costsa  STARS               deployment of   receiving 
                                                    STARS               STARS 
February 1996 b       $0.94  1998                2005                  172 
October 1999           1.40  2002                2008                  188 
March 2002             1.33  2002                2005                   73 
April 2004c            1.46  2003                2008                   50 

Source: GAO presentation of FAA data.

aThis estimate includes development costs only and does not include
technology refresh and terminal automation enhancement.

bThe February 1996 baseline included limited human factors evaluations and
a basic commercial off-the-shelf configuration.

cThe April 2004 baseline occurred after STARS' commissioning in June 2003
in Philadelphia, Pennsylvania.

FAA Faced Challenges in Approving STARS

FAA faced challenges in approving STARS. Although controllers and
technicians were involved in developing requirements for STARS prior to
the 1996 contract award to Raytheon, the original approved acquisition
plan provided only limited human factors evaluation from controllers and
technicians during STARS' design and development phase. The acquisition
approach was to employ a commercial off-the-shelf system with limited
modifications, and the competition was limited to companies with already
operational ATC systems. In 1997, FAA controllers, who were accustomed to
using the older equipment, began to voice concerns about computer-human
interface issues that could hamper their ability to monitor air traffic.
For example, the controllers noted that many features of the old equipment
could be operated with knobs, allowing controllers to focus on the screen.
By contrast, the STARS commercial system was menu-driven and required the
controllers to make several keystrokes and use a trackball, diverting
their attention from the screen. The maintenance technicians also
identified differences between STARS and its backup system that made
monitoring the system less efficient. For example, the visual warning
alarms and color codes identifying problems were not consistent between
the two systems. In 1997, FAA, the National Air Traffic Controllers
Association, the Professional Airways System Specialists, and Raytheon
formed a team to deal with these computer-human interface issues. The team
identified 98 air traffic and 52 airway facilities computer-human
interface enhancements to address these issues.

FAA and Raytheon restructured the contract to address the technicians' and
controllers' concerns. According to FAA, not involving controllers and
maintenance technicians caused FAA to revise its strategy for approving
STARS, which FAA estimates added $500 million and 3 years to the schedule.
The original STARS cost estimate of $940 million included limited human
factors evaluations and the use of a basic commercial off-the-shelf
configuration. This acquisition strategy was replaced by an incremental
development strategy that incorporated up front the majority of human
factors considerations and additional functionality that were not included
in the original cost estimate. This new acquisition strategy added years
to the development schedule and significantly increased the system's
requirements specifications. These additional requirements resulted in
both cost and schedule growth. FAA's own guidance showed that limiting
human factors evaluations will result in higher costs and schedule delays.
Initially, it is more expensive (in terms of time and funding) to deal
with human factors considerations than to ignore them. However, an initial
human factors investment pays high dividends, in terms of costs and
schedule, in later stages of acquisition when changes are more costly and
difficult to make.

FAA also experienced challenges in approving STARS, partly, because of
aggressive scheduling. FAA's approach to approving STARS was oriented to
rapid deployment to meet critical needs. To meet these needs, FAA
compressed its original development and testing schedule from 32 months to
25 months. This acceleration in schedule left only limited time for human
factors evaluations and not enough time for involvement of controllers and
maintenance technicians.

Table 12 shows the major phases and time frames associated with the STARS
approval process.

Table 12: STARS Ground System Approval Timeline

Phase                                                       Date           
Concept of operations (initial)                             1993           
Requirements setting                                        
Requirements setting occurred                               1994           
Contract award                                              September 1996 
Design and development                                      
System design review                                        December 1996  
Human factors issues identified                             1997           
STARS baseline change                                       October 1999   
Test and evaluation                                         
Development test (Philadelphia, Full Stars-2 Plus)          January 2002   
Operational test and evaluation (Philadelphia)              August 2002    
Independent operational test and evaluation (Philadelphia)  January 2003   
Operational readiness/Commissioning (Philadelphia)          June 2003      

Source: GAO representation of FAA data.

Wide Area Augmentation System Case IllustrationAppendix VI

Background

WAAS is a GPS-based navigation and landing system. According to FAA, WAAS
is to improve safety by providing precision guidance to aircraft in all
phases of flight at thousands of airports and landing strips, including
runways, where there is no ground-based landing capability. To use WAAS
for navigation, an aircraft must be equipped with a certified WAAS
receiver that is able to process the information carried by GPS and WAAS
geostationary satellite signals. Pilots are able to use this information
to determine their aircrafts' time and speed, and latitude, longitude, and
altitude positions. WAAS currently consists of a network of 25 ground
reference stations, 2 leased geostationary satellites, 2 master stations,
and 4 uplink (ground earth) stations. The ground reference stations are
strategically positioned across the United States to collect GPS satellite
data. (See fig. 6.) WAAS is designed to improve the accuracy, integrity,
and availability of information coming from GPS satellites and to correct
signal errors caused by solar storms, among other things.

Figure 6: WAAS Architecture

FAA expects that WAAS will improve the national airspace system by (1)
increasing runway capability; (2) reducing separation standards that allow
increased capacity in a given airspace without increased risk; (3)
providing more direct en route flight paths; (4) providing new precision
approach services; (5) reducing the amount of and simplifying equipment on
board aircraft; (6) saving the government money due to the elimination of
maintenance costs associated with older, more expensive ground-based
navigation aids; and (7) providing vertical guidance in all phases of
flight to improve safety.

Status

In July 2003, FAA commissioned WAAS to provide initial operating
capability for 95 percent of the United States. In July 2003, the first of
the LPV1 approaches were provided whereby pilots could safely descend to a
250-foot decision height.2 As of August 2004, there were about 20 LPV
landing procedures published for WAAS. With over 4,000 runways needing
them, much work still needs to be done to fully utilize the WAAS
capability. FAA expects to have WAAS available in the rest of the country,
with the exceptions of a few parts of Alaska, by the end of 2008 when it
completes the addition of 13 ground reference stations and 2 leased
geostationary satellites. WAAS is not scheduled to achieve full (Category
I) operating capability, the final phase of WAAS when pilots will be able
to use it to navigate as low as 200 feet above the runway, until the
2013-2019 time frame.3

As shown in table 13, FAA changed WAAS' cost and schedule estimates for
the third time in May 2004. According to FAA, the reasons for the May 2004
rebaselining were that the system was not able to achieve full Category 1
capability and because of FAA internal and congressional budget cuts.
Under the May 2004 baseline, FAA estimates that WAAS development costs
will be about $2.0 billion, which is $1.5 billion higher than the 1994
estimated development costs. Also, FAA has not yet met some of its
original performance goals, such as providing pilots with the ability to
navigate as low as 200 feet above the runway. According to FAA, WAAS
cannot easily achieve Category I as a single frequency system because the
error sources caused by solar storms are difficult to correct without the
use of a second civil aviation frequency in space, which is the
responsibility of the Department of Defense. FAA, realizing the difficulty
and risk associated with developing a single frequency Category I system,
decided to wait and leverage the benefits of the White House policy to
include the second civil frequency on the GPS satellite network. According
to FAA, budget cuts and the decision to wait until the second civil
frequency is placed on the GPS constellation have caused it to extend the
timeline for reaching WAAS' full Category I operating capability to
between 2013 and 2019.

Table 13: Cost and Schedule Baseline Changes to WAAS

Dollars in                                                  
millions                                                    
Baseline year          Estimated  Initial operating         Full operating 
                  development costs  capability                capability     
1994                        $509  June 1997                 December 2000  
January 1998               1,007  August 1999               December 2001  
September 1999            1,683a  September 2000            December 2006  
May 2004                  2,036b  July 2003                 2013-2019      

Source: GAO presentation of FAA data.

aThe September 1999 estimate for WAAS development does not include $1.3
billion in satellite service acquisition through 2020. In earlier
estimates, satellite service acquisition costs were included in the cost
of operating WAAS, not developing WAAS.

bThe May 2004 estimate for WAAS development does not include $1.3 billion
in satellite service acquisition through 2028. In earlier estimates,
satellite service acquisition costs were included in the cost of operating
WAAS, not developing WAAS.

FAA Faced Challenges in Approving WAAS

FAA faced challenges in approving WAAS ground and satellite components for
use in the national airspace system, partly because of FAA's accelerated
scheduling, lack of effective coordination between its aircraft
certification office and acquisitions office, and technical challenges
which resulted in a delay meeting the integrity requirement. FAA's
challenges in approving WAAS began in 1994 when FAA accelerated the
implementation of milestones, including moving up the commissioning of
WAAS by 3 years. FAA originally planned to commission WAAS in 2000;
however, at the urging of government and aviation industry groups in the
1990s, it decided to change WAAS' commissioning date to 1997. FAA tried to
develop, test, and deploy WAAS within 28 months, despite the fact that
software development alone was expected to take 24 to 28 months. FAA also
set system milestones before completing the research and development
required to prove the system's capability.  Although FAA attempted to
accelerate the implementation of WAAS, it wasn't until July 2003, 6 years
later, that it was able to commission WAAS with initial operating
capability.

Lack of full involvement between FAA's aircraft certification members and
the rest of the integrated product team contributed to delays in approving
WAAS. For example, although an integrated product team, which included
representatives from aircraft certification and acquisition offices, was
developing WAAS, it was not until September 1999, when the aircraft
certification office became fully involved, that FAA recognized (1) the
difficulty of meeting  the integrity requirement-that WAAS must alert the
pilot in a timely manner when the system should not be used-and (2) it did
not have the technical expertise needed. According to FAA officials, the
reason coordination did not occur was because the two offices had
competing priorities, such as the day-to-day aircraft equipment
certification activities not associated with the development of a new ATC
system. This situation may have developed because FAA's aircraft
certification organization is more accustomed to being involved after a
project is developed, rather than actively participating throughout
project development.

The need to meet WAAS' integrity requirement also hampered FAA's ability
to approve WAAS for safe use in the national airspace system. In December
1999, FAA found that WAAS did not meet the agency's integrity requirement
for precision approaches, and FAA recognized that it did not have the
technical expertise required to resolve the issue. Therefore, in 2000, FAA
established a team of satellite navigation experts, which was referred to
as the WAAS Integrity Performance Panel and included representatives from
the MITRE Corporation, Stanford University, Ohio University, and the Jet
Propulsion Laboratory. Developing a solution to prove that the WAAS design
met the integrity requirement added about 2 years and 4 months to the
approval process and contributed to WAAS' cost growth. All of these
challenges contributed to a 6-year delay in WAAS' commissioning and a $1.5
billion increase in its estimated total development costs through 2028,
exclusive of operating and maintaining geostationary satellites, which
were not part of WAAS' original 1994 baseline. Table 14 shows the major
phases and time frames associated with approving WAAS' ground system.

Table 14: WAAS Ground System Approval Timeline

Phase                                             Date                     
Concept of operations                             June 1992                
Requirements setting                              
Operational requirements document                 June 1994                
Original contract award                           August 1995              
Current contract award                            May 1996                 
Design and development                            
Critical design review                            December 1997            
Test and evaluation                               
Development test (failed)                         December 1999            
WAAS Integrity Performance Panel formed           January 2000             
Development test (passed)                         September 2002           
Operational test and evaluation                   March 2003               
Operational readiness/Commissioning               July 2003                

Source: GAO presentation of FAA and RTCA data.

FAA Did Not Experience Major Challenges in Certifying the Aircraft
Equipment of WAAS

In contrast to the challenges that it encountered during the approval of
the WAAS ground system, FAA did not encounter major challenges with the
certification of WAAS aircraft equipment, primarily because FAA had an
up-front approval agreement with one of the first applicants, United
Parcel Service Aviation Technology, through the creation and approval of a
safety plan and a project-specific certification plan. Table 15 shows the
major phases and time frames associated with certifying the aircraft
equipment of WAAS. Currently, WAAS GPS receivers have been certified and
are available for use.

Table 15: WAAS Aircraft Equipment Certification Timeline

Phase                                           Date                       
Concept of operations                           June 1992                  
Requirements setting                            
RTCA WAAS minimum operational performance       1994 to November 2001      
standards (four major revisions)                
WAAS technical standard orders (four major      May 1998 to September 2002 
revisions)                                      
Design and production approval                  
Data submitted for supplemental type                                       
certificate and technical standard order        June 2, 2003
authorization                                   
Technical standard order authorization (United  June 13, 2003              
Parcel Service Aviation Technology)             
Installation approval - Type                                               
certificate/Supplemental type certificate       June 27, 2003
(United Parcel Service Aviation Technology)     
Operational approval                            Not requireda              

Source: GAO presentation of FAA data.

aFAA first approved the use of GPS for aviation navigation in 1993;
therefore, new aircraft equipment that use GPS did not require a new
operational approval.

GAO Contacts and Staff AcknowledgmentsAppendix VII

GAO Contacts

Katherine Siggerud, (202) 512-2834 or [email protected] Tammy Conquest,
(202) 512-5234 or [email protected]

Staff Acknowledgments

In addition to the individuals named above, other key contributors to this
report were Geraldine Beard, Gerald Dillingham, Seth Dykes, David Hooper,
Kevin Jackson, Gregg Justice III, Donna Leiss, and Kieran McCarthy.

(540057)
*** End of document. ***