National Airspace System: Persistent Problems in FAA's New Navigation
System Highlight Need for Periodic Reevaluation (Letter Report,
06/12/2000, GAO/RCED/AIMD-00-130).
Pursuant to a congressional request, GAO provided information on the
Federal Aviation Administration's (FAA) new navigation system, focusing
on whether: (1) the Department of Defense's (DOD) current Global
Positioning System (GPS) or its planned improvements for GPS can meet
FAA's navigation requirements; (2) the benefits of FAA's chosen approach
to an augmented system outweigh the cost of this system; and (3) other
technologies are available to meet FAA's requirements and users' needs
for a new navigation system.
GAO noted that: (1) according to studies GAO reviewed and the experts
GAO contacted, the current GPS system does not meet all of FAA's civil
aviation navigation requirements for accuracy, integrity, and
availability; (2) even though DOD has made the APS signal provided for
civilian use more accurate and plans to make more improvements, GPS
still will not fully meet FAA's requirements for navigation and landing;
(3) this is because GPS does not provide the assurance that its signal
will be available virtually all the time; (4) FAA's analysis concluded
that the quantified benefits of its approach would outweigh the cost;
(5) since completing this analysis, FAA has experienced delays and cost
increases primarily because of difficulties in meeting its integrity
requirement; (6) as a result, it is unclear whether quantified benefits
will still outweigh cost; (7) at the present time, no other navigation
technologies, including variations of ground-based and less robust
satellite-based systems, are available to meet FAA's requirements and
users' needs for precise landing guidance at more airports; (8) the Wide
Area Augmentation System is designed to provide such guidance to improve
safety and offer access to more airports; (9) however, this system is
experiencing difficulties that cast doubt on whether it will perform as
designed at a reasonable cost and be delivered on a reasonable schedule;
and (10) moreover, experts in alternative navigation technologies, some
of which compete with the Wide Area Augmentation System, told GAO that
users may have overstated their need for precise landing guidance and
that other navigation technologies could satisfy most of these needs.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED/AIMD-00-130
TITLE: National Airspace System: Persistent Problems in FAA's New
Navigation System Highlight Need for Periodic
Reevaluation
DATE: 06/12/2000
SUBJECT: Air traffic control systems
Navigation aids
Cost effectiveness analysis
Requirements definition
Procurement planning
Transportation safety
Systems conversions
IDENTIFIER: FAA Wide Area Augmentation System
NAVSTAR Global Positioning System
FAA Local Area Augmentation System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO/RCED/AIMD-00-130
Appendix I: Objectives, Scope, and Methodology
28
Appendix II: Experts Contacted
31
Appendix III: FAA Identified Four Alternatives for a Benefit-Cost
Analysis
33
Appendix IV: Comments From the Department of Defense
37
Appendix V: GAO Contacts and Staff Acknowledgments
38
Table 1: Development Costs and Schedules for WAAS and LAAS,
1994 Through September 1999 9
Table 2: Summary of FAA's Economic Analysis of Four Alternatives 35
Figure 1: The Operating System for WAAS 8
ATA Air Transport Association of America
DOD Department of Defense
FAA Federal Aviation Administration
GPS Global Positioning System
LAAS Local Area Augmentation System
OMB Office of Management and Budget
WAAS Wide Area Augmentation System
Resources, Community, and
Economic Development Division
B-283632
June 12, 2000
The Honorable Richard C. Shelby
Chairman, Subcommittee on Transportation
Committee on Appropriations
United States Senate
Dear Mr. Chairman:
Currently, the Federal Aviation Administration (FAA) relies principally on a
ground-based navigation system that uses various types of equipment to
assist pilots in navigating their assigned routes and to provide them with
guidance for landing their aircraft safely in different types of weather.
However, this ground-based navigation system is aging and limited in its
geographic coverage. FAA is planning a transition from its ground-based
navigation system to a satellite-based system using radio signals generated
by the Global Positioning System (GPS) to provide greater geographic
coverage, among other things. The Department of Defense developed GPS to
support military missions and functions. However, the system is now a
dual-use system, and other users--pilots, truckers, and boaters--rely on
signals from the GPS satellites to calculate their time, speed, and position
anywhere on or above the earth's surface. As part of its efforts to maintain
GPS and make it more useful for civilians, in May 2000, Defense ceased its
practice of intentionally degrading the accuracy of the GPS signal available
for civil use.1 Furthermore, Defense plans to begin gradually replacing the
existing satellites with new ones that will also improve system performance.
Although GPS already provides some critical information to pilots, FAA
believes that even with the greater accuracy and other improvements expected
from Defense's newer satellites, this system will not satisfy all civil
aviation requirements for ensuring safe aircraft operations. For example,
FAA requires that its navigation system be unavailable no more than 5
minutes per year for some types of navigation and landing. In contrast, GPS
could be unavailable for periods of time equaling up to 4 days per year. To
satisfy its requirements, FAA decided in the 1980s to augment GPS with other
navigational aids--the Wide Area Augmentation System (WAAS) and the Local
Area Augmentation System. These systems consist primarily of a network of
ground stations that receive, process, and validate data from GPS before
transmitting these data to pilots.
In deciding to augment GPS, FAA has conducted several benefit-cost
analyses--most recently in 1999. In its latest analysis, FAA estimated that
its future investment in this augmented navigation system could exceed $8
billion from 2000 through 2020.2 WAAS is the largest component of this
augmented system--37 percent of the total. Over the years, the cost of
developing WAAS has increased by over $500 million primarily because of
unanticipated development and additional program support costs. In addition,
WAAS has been delayed for over 3 years and has experienced performance
problems.
In light of the expected cost of this new navigation system and continuing
concerns about WAAS' ability to achieve cost, schedule, and performance
goals, you asked us to provide information on whether (1) the Department of
Defense's current GPS or its planned improvements for GPS can meet FAA's
navigation requirements, (2) the benefits of FAA's chosen approach to an
augmented system currently outweigh the cost of this system, and (3) other
technologies are available to meet FAA's requirements and users' needs for a
new navigation system. In conducting this review, we examined studies and
spoke with experts in aviation navigation and related technologies to obtain
their views on the capability of FAA's new navigation system and
alternatives to that system. Appendix I discusses our detailed scope and
methodology. Appendix II lists the experts we spoke with, some of whom
represent various technologies.
According to the studies we reviewed and experts we contacted, the current
Global Positioning System does not meet all of FAA's civil aviation
navigation requirements for accuracy, integrity, and availability. FAA
defines accuracy as the degree to which a navigation system calculates an
aircraft's true position. Integrity is the ability of a navigation system to
provide timely warnings when its signal is providing misleading information
that could potentially create hazards for pilots and thus should not be
used. Availability is the probability that a navigation system meets the
accuracy and integrity requirements. Even though the Department of Defense
has made the current GPS signal provided for civilian use more accurate and
plans to further improve GPS by, for example, providing an additional,
higher-powered signal to combat interference, GPS still will not fully meet
FAA's requirements for navigation and landing. This is because GPS does not
provide the assurance that its signal will be available virtually all the
time.
FAA's 1999 analysis concluded that the quantified benefits of its approach
would outweigh the cost. Since completing this analysis, FAA has experienced
delays and cost increases primarily because of difficulties in meeting its
integrity requirement. As a result, it is unclear whether quantified
benefits will still outweigh cost. For example, to meet FAA's integrity
requirement--which requires the Wide Area Augmentation System to virtually
never fail to warn pilots of potentially hazardously misleading
information--we estimate that the agency may need 3 or more years, at an
additional cost of between $200 million to $240 million, to demonstrate that
this requirement can be met. The system, with the ability to provide
integrity, was to have been operational by September 2000. We are making
recommendations to better ensure that FAA delivers its new navigation
system, on time and within budget, and that it meets performance
requirements. Transportation and Defense officials as well as the officials
from the Satellite Navigation User Group--representing commercial, general
avaition, and Defense users--acknowledged the problems encountered in
developing the new navigation system. Transportation officials concurred
with GAO's conclusions and recommendations.
At the present time, no other navigation technologies--including variations
of ground-based and less robust satellite-based systems--are available to
meet FAA's requirements and users' needs for precise landing guidance at
more airports. The Wide Area Augmentation System is designed to provide such
guidance to improve safety and offer access to more airports. However, as
noted, this system is experiencing difficulties that cast doubt on whether
it will perform as designed at a reasonable cost and be delivered on a
reasonable schedule. Moreover, experts in alternative navigation
technologies, some of which compete with the Wide Area Augmentation System,
told us that users may have overstated their need for precise landing
guidance and that other navigation technologies could satisfy most of these
needs.
Currently, civil aviation relies principally on a ground-based navigation
system that uses various types of equipment to provide navigation and
landing services to pilots in different types of weather. This equipment
meets FAA's performance requirements for accuracy, availability, and
integrity; however, it is aging and has limitations in its geographic
coverage. For example, under today's ground-based navigation system, pilots
fly structured routes (referred to as highways-in-the-sky) that may not be
the most direct and fuel-efficient. The wider coverage provided by WAAS,
coupled with other improvements, would not restrict pilots to these
structured routes and would therefore result in more direct and
fuel-efficient flights.
To a lesser degree, civil aviation relies on GPS for its navigation needs.
The Department of Defense (DOD) maintains a constellation of 24 orbiting GPS
satellites for both military and civilian use.3 These satellites are
positioned so that at any given time the signals from a minimum of four
satellites will be available to users.
FAA is developing a new navigation system to augment GPS in order to provide
broader geographic coverage, among other things. The largest component of
this system is WAAS. When fully developed, WAAS could comprise a network of
up to 76 ground stations and three to four geostationary communications
satellites. (see fig. 1.) The WAAS network is being designed to provide the
same level of service as today's ground-based equipment and is expected to
support navigation through all phases of flight as well as nonprecision and
category I precision landing approaches for a wider geographic area.4 In a
nonprecision approach, the pilot relies on instruments on board the aircraft
to guide it safely from a height ranging from between 700 and 400 feet above
touchdown. In contrast, in a category I precision approach, the pilot relies
on instruments to provide an aircraft with safe vertical guidance to a
height of not less than 200 feet above touchdown. Currently, pilots can make
precision approaches at only about 625 airports. With WAAS, FAA estimates
that it may be possible to expand this capability to approximately 3,300
airports, thus providing benefits to more users. To obtain the full benefit
from WAAS, it would be necessary for these airports to incur costs for items
they would need for providing greater access at 200 feet, such as airport
lighting systems, which could cost between $1 million to $2 million per
airport system. Additionally, many pilots are not now qualified to fly these
precision approaches and will therefore need training to obtain the full
benefits of WAAS.
Source: FAA.
As part of its augmentation of GPS, FAA, in partnership with industry,5 is
also developing the Local Area Augmentation System (LAAS) to support, among
other things, even more stringent precision approach guidance than expected
from WAAS. For example, in these approaches, LAAS is expected to provide
pilots with safe vertical instrument guidance to heights ranging from less
than 200 feet to down to the runway surface. While LAAS is independent of
WAAS, it is also expected to complement WAAS and provide precision
approaches at airports where WAAS does not provide sufficient geographic
coverage. LAAS will require the development, testing, evaluation, and
fielding of a new generation of ground stations--up to 160. In January 1998,
FAA approved development costs and schedules for LAAS. At that time, FAA
estimated that LAAS would cost $530 million and would be operational by
2003.
Table 1 shows the history of development costs and schedules for WAAS and
LAAS.
Dollars in millions
As of
1994 Jan. 1998 Jan. 1999 Sept. 1999
Types of costs
Total WAAS development
costs $508 $1,007a $1,007 $2,484b
Total LAAS development
costs c $530 $530 $720d
Schedule Information
WAAS' initial capability June July 1999 Sept. 2000 Sept. 2000
1997
As of
1994 Jan. 1998 Jan. 1999 Sept. 1999
Schedule Information
WAAS' full capability Dec. Dec. 2001 To be determined Dec. 2006
2000
LAAS' first site
implementation c 2003 2003 2003
LAAS' last site
implementation c 2006 2006 2010
Note: Since 1996, FAA has been including life-cycle costs, which include
costs for developing, operating, and maintaining projects. The current
life-cycle cost estimate for WAAS is $3,187.6 million.
a The Jan. 1998 program development costs for WAAS include the prime
contractor costs, development of standards and procedures, technical
engineering and program support, and the first year of costs for satellites.
b The Sept. 1999 estimate for WAAS development costs includes $1.3 billion
in satellite service acquisitions through 2020. In earlier estimates,
satellite service acquisition costs were included in the cost of operating
WAAS.
c Costs and schedules were not developed until 1998.
d The Sept. 1999 LAAS cost increase is due in part to FAA now planning to
acquire up to 160 systems. In earlier cost estimates, FAA only planned to
acquire 143 systems.
Source: FAA.
FAA is retaining about 30 percent of its ground-based navigation
infrastructure to address concerns about the vulnerability of the GPS
signal, which WAAS relies on, and to support those users who choose not to
purchase the equipment that must be used with WAAS. This infrastructure,
along with WAAS and LAAS, make up the components of FAA's new navigation
system. Both WAAS and LAAS would require airlines and general aviation users
to purchase on-board equipment for receiving signals from this new
technology. These purchases are expected to occur over time, as the new
navigation system is developed.
WAAS and LAAS are being developed under a single FAA integrated product
team, which includes representatives from FAA's aircraft certification and
acquisition organizations. FAA established cross-functional teams to help
ensure that systems are developed and implemented in an efficient and
effective manner. These teams are to be empowered to make decisions
affecting systems and services, from their inception to their eventual
disposal or termination. The effective operation of the integrated teams is
key to FAA's goal of producing timely, cost-effective acquisitions.
Requirements
Studies we reviewed, experts we contacted, and DOD officials we interviewed,
agreed that to meet FAA's civil aviation requirements, FAA must augment even
the improved GPS. For example, according to a 1999 Johns Hopkins University
study,6 augmentation was needed because GPS failed to meet the critical
requirement of system availability, which is the probability that at any
given time a navigation system will meet the accuracy and integrity
requirements for a specific phase of flight.7 FAA defines accuracy as the
degree to which an aircraft's position, as calculated by its navigation
system, conforms to its true position. Integrity is the ability of a
navigation system to provide timely warnings when its signal is providing
misleading information that could potentially create hazards for pilots and
thus should not be used for navigation. This study considered the impact of
DOD's decision to stop intentionally degrading the accuracy of the signal
for civilian use and concluded that augmentation was still necessary. In May
2000, DOD actually stopped degrading its signal for civilian use. As a
result, the predicted accuracy for civilian use will increase from about 100
meters (about 300 feet) to 20 meters (about 60 feet). However, FAA requires
an accuracy of about 7.6 meters (about 23 feet) for landing aircraft.
Moreover, Defense plans to begin gradually replacing the existing satellites
with new ones that will have an additional higher-powered signal to help
combat interference and improve overall system performance. However, even
with these improvements, GPS still falls short in meeting the requirement
that it be available virtually all of the time. For example, the Johns
Hopkins study found that GPS could be unavailable for periods of time
equaling up to 4 days per year for en route navigation.8, 9 In contrast, FAA
requires that its navigation system be unavailable no more than 5 minutes
per year for en route navigation and nonprecision approaches. Furthermore,
GPS alone does not provide guidance for precision approaches.
In addition, an expert we spoke with noted that FAA's efforts to augment GPS
are in line with those of other nations to augment a navigation system based
on global satellites.10 Nations with navigation systems based on global
satellites will likely augment the signal provided by the global system in
order to maintain surveillance over civil navigation services. This would
enable them to independently monitor--from the ground--the usability of the
GPS signals and provide GPS status for their air traffic controllers and
aircraft.
Questions About Whether the Quantified Benefits of FAA's New Navigation
System Outweigh Its Cost
According to FAA's 1999 analysis, its new navigation system yielded
quantified benefits that exceeded costs over the period 2000 through 2020,
but development problems--principally related to proving the WAAS integrity
requirements--will raise costs, making it unclear whether quantified
benefits still exceed the cost of the new system. Also, in the short term,
because of the delays resulting from these problems, users will not receive
the benefits they expected from the system later this year, such as greater
capability to land at more airports in bad weather. Additional costs are
occurring because FAA has not appropriately overseen its contractor11 and
has underestimated software costs. Finally, it has been suggested that the
system's cost may be higher if performance problems indicate that FAA needs
to maintain separate navigation equipment at the airport.
Most Likely Reduce These Benefits
In 1999, in response to a congressional request, FAA reevaluated its plans
for a new navigation system.12 As part of this reevaluation, FAA conducted a
benefit-cost analysis of four alternatives. According to FAA's analysis,
continuing with the WAAS/LAAS system, coupled with using about 30 percent of
the agency's current ground-based infrastructure as a backup to WAAS, would
yield the greatest net quantified benefits (dollar value of benefits minus
costs),13 regardless of whether passengers' time savings are included.14, 15
According to FAA, for 2000 through 2020, there is an 80-percent chance that
the net benefits of its preferred alternative are about $2.5 billion if
passengers' time savings are counted and $72 million if these savings are
not counted. Expressed another way, the benefit-cost ratios for these two
scenarios are 2.4 and 1.1, respectively.16 The benefit-cost analysis also
identified at least 11 benefits that were not quantified. Included among
these were safety benefits, such as improved surface surveillance, as well
as operational benefits, such as enabling more landings at airports whose
operations are today limited by the lack of ground-based navigational
aids.17 According to FAA, its decision to pursue this approach had the
support of major users.18 (See app. III for a discussion of FAA's analysis.)
While we did not conduct a detailed benefit-cost analysis, we believe that
under certain conditions the quantified net benefits of FAA's chosen
alternative, without passenger value of time, could be negative, and the
benefit-cost ratio could be less than 1. This could occur because of added
costs resulting from delays or other problems. Furthermore, these delays
could have implications for FAA and users.
Was Not Addressed in a Timely Manner by FAA Management and the Integrated
Product Team
In December 1999, FAA found that the WAAS design could not be relied upon to
satisfy the agency's requirement for system integrity for precision
approaches, which stipulates that WAAS cannot fail to warn pilots of
misleading information that could potentially create hazardous situations
more than once in 10 million approaches. Consequently, FAA has determined
that it will not make its scheduled date of September 2000 to begin
providing an initial capability for precision guidance (category I
approaches) through WAAS.19 The delay could have implications for FAA,
system users, and equipment manufacturers. For example, FAA may need to buy
new ground-based navigation equipment or maintain existing equipment longer
than expected--maintaining existing equipment costs about $170 million
annually, according to FAA. Likewise, system users and equipment
manufacturers could question the wisdom of making further investments in
WAAS technology. Because of these implications, FAA, with users' support,
has decided to provide only a limited precision guidance capability with
WAAS by 2002.20 FAA has yet to determine when WAAS will achieve its initial
capability. However, FAA and major user groups contend that they still will
receive benefits from a limited WAAS. Using information provided by FAA and
its experts, we estimate that resolving the integrity problem could
potentially delay making WAAS' initial capability available by 3 years or
more and add approximately $200 million to $240 million to the cost of
developing WAAS.
To satisfy the integrity requirement, FAA and its contractor plan to make
changes to improve the calculations for better identifying misleading
information that could potentially create hazards for pilots; these changes
will cause software changes and may result in hardware upgrades. A team
consisting of FAA officials, its contractor, and consultants proposed these
changes and plans to be actively involved in ensuring that they will result
in proving the system's integrity performance. By the end of 2000, the team
expects to determine what, if any, changes may be needed to the WAAS design
to achieve precision approaches down to 200 feet above touchdown.
The difficulties in proving the integrity requirement have occurred largely
because FAA management and the integrated product team underestimated the
complexity of resolving the integrity issue and, as a result, failed to
recognize the seriousness of the problem. Moreover, FAA did not closely
monitor the contractor's effort to demonstrate integrity, and members of the
team did not have a clear understanding of their roles. Consequently, team
members did not effectively communicate with each other and the contractor.
Lack of monitoring and poor communications have been recurring problems in
FAA's air traffic control modernization program. For example, in 1996, we
reported that inadequate oversight of contractors' performance was a major
contributor to FAA's recurring cost, schedule, and performance problems with
other projects in the modernization program.21
According to a WAAS study group FAA convened in October 1997, it would be
difficult to prove WAAS' stringent requirements. Therefore, FAA would need
both a sound mathematical approach and evidence acquired through operational
experience.22 This group's conclusions were validated in December 1997, when
according to FAA, it discovered at a key project milestone that the
contractor did not have an adequate plan for proving the WAAS integrity
requirement. Over the next 21 months, FAA, its experts, and the prime
contractor attempted to resolve the integrity issue; however, not until
September 1999, when the aircraft certification members of the integrated
product team became actively involved, did FAA fully understand the
difficulty in trying to prove this requirement. We found that FAA's progress
in resolving this issue was hampered because (1) the contractor took about a
year to submit the limited results of its assessment of the integrity issue
and (2) the integrated product team was slow to respond to advice from other
FAA experts that the contractor's integrity assessment was inadequate.
FAA officials agreed that they should have monitored this situation more
closely but provided three reasons for not addressing the integrity problems
more promptly. These reasons largely concerned the actions of FAA's senior
management and the integrated product team for WAAS:
� Competing priorities between FAA's acquisition and aircraft certification
organizations, which are part of the integrated product team, negated the
effectiveness of this team's approach for meeting the agency's WAAS goals.
This situation may have developed because FAA's aircraft certification
organization is more accustomed to being involved after a project is
developed, rather than actively participating throughout its development. As
we reported in 1996, FAA's product teams have not always forged true
partnerships across organizational "stovepipes."23
� A shortage of in-house technical expertise and the team's attention to
other important issues, such as systems engineering, prevented the team from
monitoring the situation more closely.
� FAA did not have a sufficiently defined process for identifying and
conveying to the contractor the results that would be acceptable for proving
WAAS' integrity.
Moreover, some team members did not feel empowered to take actions to
address performance issues because they did not believe that they had senior
management's support in dealing with contractor-related problems. In
addition to these internal problems, an FAA official told us that the
contractor lacked sufficient expertise to prove the integrity requirement.
In any case, after being alerted to the difficulty in proving this
requirement 2 years ago, FAA is only now beginning to pay it serious
attention.
Underlying these problems is a basic issue of how FAA decided to proceed
with WAAS' development. Essentially, FAA took an acknowledged high-risk
approach: It agreed on a design for the system and established milestones
for system deployment before completing the research and development needed
to demonstrate the system's capability. If FAA continues along this path, it
could incur significant costs for other system components, such as
satellites, to make the design fully operational--without knowing whether
the system will meet its performance requirements. A FAA senior manager
acknowledged that, in hindsight, the agency should have placed more emphasis
on how it would prove WAAS' integrity performance prior to agreeing to a
design for the system. As we have reported for other FAA modernization
projects, when the agency attempts to combine different phases of system
development in an effort to more quickly implement systems, it repeatedly
experiences major performance shortfalls, which lead to delays and
additional costs.24
Recognizing these problems, FAA is in the process of implementing a new
approach to developing WAAS. Under this approach, before making additional
investments, FAA plans to allow time for collecting and evaluating data on
(1) system performance, (2) the extent to which users have purchased
equipment, and (3) the availability of emerging new technologies for the new
navigation system. In essence, FAA plans to reevaluate WAAS at critical
points--"checkpoints"--in its development.
We believe that reevaluating WAAS at checkpoints should provide critical
information for deciding about the need for future investments. Moreover,
such checkpoints would allow FAA to better assess whether it should shift
resources to other parts of the navigation system, such as LAAS, to meet
users' needs. While FAA recognizes the need for checkpoints, it has not
developed a detailed plan explaining when these checkpoints would occur,
what they would accomplish, and who would be responsible for overseeing
them. We also believe that it would be prudent for FAA to have progress on
WAAS independently validated at the established checkpoints before the
Congress approves additional funding.
Given past problems and the amount of and potential complexity associated
with the remaining systems engineering development effort--including
developing algorithms to resolve the integrity issue--we believe that the
WAAS software effort will continue to experience delays. For example, the
schedule for the initial WAAS capability has already slipped 14 months--from
July 1999 to September 2000--largely because of problems in developing the
WAAS design, which led to more software development. At the end of its
initial phase, WAAS will have approximately 350,000 lines of code and
370,000 additional lines or more of code will be required before the system
becomes fully operational.25 According to FAA, this additional code will be
needed to, among other things, provide for the security of the system and to
expand its operating capability.
We have reported on numerous occasions that in acquiring software, FAA has
not adequately detailed its design requirements and overseen contractors'
development of the software.26 Software development--the most critical
component of key FAA modernization programs--has been the Achilles' heel of
FAA's efforts to deliver programs on time and within budget.
Even if all navigation system and development issues were resolved, the
quantified cost of the new navigation system is likely to outweigh the
quantified benefits because FAA underestimated the cost of maintaining
software by between 120 percent to over 200 percent, according to our
analysis.27 That is, we found that FAA's estimate of $85 million for WAAS
software maintenance, which includes the staffing cost for modifying or
adding software throughout the life of the project, was understated by
between $101 million to $181 million. FAA's estimate differed from our
estimate in part because the agency used more optimistic assumptions in two
instances. First, FAA assumed that 7 percent of the software would be
changed annually throughout the life of the project. In our analysis,
however, we assumed that 10 percent of the software would be changed
annually. We made this assumption because (1) FAA's own data suggested that
an annual factor of between 5 percent to 10 percent could be used and (2)
the model we used suggested that for software projects such as WAAS, annual
software change could approach 10 to 20 percent. Second, FAA assumed that
the government would maintain the software and used an hourly labor rate of
$75 per hour--the rate the agency uses when it maintains software in-house.
FAA's own data suggested that having the government maintain the WAAS
software would be optimistic, given its considerable complexity and
safety-of-life purpose and the government's unfamiliarity with the
software's development. For our analysis, we used the prime contractor's
software labor rate of $127 per hour because the contractor is very familiar
with the software's development and well qualified to maintain it.
Be Used in Conjunction With WAAS
Costs for WAAS may also be higher than FAA anticipates if, as some
navigation experts believe, separate equipment is needed at airports. This
equipment would be used to monitor the information WAAS provides to aircraft
as they approach an airport for a landing. Currently, FAA has no plans to
have such equipment in place. But these two experts note that FAA has always
ensured safety by requiring such equipment and that the WAAS design should
conform to this requirement.
To illustrate the need for such equipment, these experts noted that local
conditions around an airport, such as interference from man-made and natural
occurrences, may affect the GPS signal. The WAAS reference station, which
would send corrections to an aircraft, might be located hundreds of miles
from the airport and might not pick up these local conditions. Local
navigation equipment would factor in local conditions and their effect on an
aircraft's true position. Without such information, these experts maintain
that a pilot will not have the critical information needed to land safely.
Even though this occurrence has not been observed in tests to date, experts
contend that the lack of observance is not proof that it will not occur.
In response, FAA stated the agency's experts have reasoned that the WAAS
design and the location of the reference stations are sufficient to account
for and correct any inaccuracies in the GPS signal. FAA disagrees that
separate, independent monitoring equipment is necessary because, given the
WAAS design, an airport-based monitor will not provide the required
integrity. This might occur because an airport monitor could receive its
information from a set of GPS satellites that are different from those that
are providing information to the aircraft. Nevertheless, FAA asked its
experts to model extreme conditions, and these experts have yet to detect
local inaccuracies. FAA also notes that it will not provide guidance for the
category I precision approaches until it has had an opportunity to evaluate
operational data and feedback from pilots. As it gains experience, FAA
acknowledges it may need to make changes to WAAS, which could increase the
cost of the WAAS investment.
but Experts in Competing Technologies Contend That Some Needs Are Overstated
At the present time, alternative technologies, including other ground-based
and less robust satellite-based systems, fall short in meeting FAA's
requirements and users' needs.28 WAAS is designed to provide precise
position and landing guidance (category I precision approaches) to meet
FAA's requirements and users' needs for improved safety and greater access
to more airports. The alternative technologies cannot meet FAA's requirement
for category I precision approaches. However, according to several experts
supporting alternative technologies, they believed users overstated the
level of precision needed to safely land at more airports and therefore a
less robust WAAS or other technologies could satisfy users' needs. However,
FAA contends that WAAS serves other purposes in addition to serving users'
needs. FAA told us that other modernization efforts depend on the precise
position information expected from WAAS and that if the agency were to rely
on technologies other than WAAS, it would have to consider redesigning these
other projects.
Experts on technologies that compete with WAAS acknowledge that, unlike
WAAS, these technologies cannot provide vertical guidance to a height of not
less than 200 feet above touchdown (category I precision approaches) at more
airports. Users--primarily general aviation pilots--told FAA that they need
this level of precision. It is important to note that FAA has yet to
demonstrate if and when WAAS will provide users with guidance for category I
precision approaches.
While recognizing that other technologies cannot provide category I
precision approach guidance, these experts question whether such guidance is
needed. They note that providing users with access to more airports depends
upon factors other than the precise position information WAAS offers. For
example, to experience the full benefits of WAAS, airports would need to
invest in infrastructure, such as lighting systems, and pilots would need
training to fly precision approaches into these airports. Given this
potential investment and other considerations, these experts, who support
technologies that compete with WAAS, pointed out three different available
combinations of ground-based and satellite-based technologies that could
meet the vast majority of users' needs for precise landing information at a
lower cost than WAAS: (1) Long Range Navigation (LORAN C), a ground-based
navigation system that provides position and timing information to both
aviation and nonaviation users;29 (2) a scaled-back WAAS, which would not
provide category I precision approaches; and (3) LORAN C with an enhancement
known as Eurofix. For example, according to a 1998 study conducted for the
Department of Transportation, the cost to develop and operate LORAN C is
estimated at over $600 million for 2000 through 2015.30 Similarly, the cost
to develop and operate WAAS during this period is estimated at $2.3 billion.
Most of the difference in cost is attributable to the fact that WAAS would
rely on newly launched geostationary satellites, which LORAN C does not
need.
These experts maintain that the competing technologies, combined with
on-board equipment that provides an increased vertical guidance capability,
can guide users down to a height of about 250 feet and therefore achieve
"near" precision approaches.31 They therefore contend that a less costly
technology than the WAAS now contemplated could still meet the vast majority
of the aviation community's needs for precise landing
information--especially for general aviation users, who are expected to
benefit the most from WAAS. At runways where a 200-foot minimum category I
approach is critical, these experts stated that these requirements could be
satisfied by LAAS.
According to FAA, it analyzed the technical capability of these three
alternatives as part of its 1999 reevaluation of its plan for a new
navigation system. It rejected the LORAN C and scaled-back WAAS alternatives
because they did not provide the precision navigation guidance expected from
WAAS. The agency rejected the Eurofix alternative because it did not meet
the integrity requirements for precision approaches. FAA also believes that
the alternative technologies and on-board equipment would bring pilots down
to about 350 feet, not the 250 feet the experts contend could be achieved.
Moreover, implementing these technologies would shift a large cost from the
government to general aviation system users--to purchase additional
equipment estimated at about $1,000 to $4,000 per aircraft--and yet these
users would still be limited in their ability to land at more airports in
foul weather. A group representing major users reaffirmed its support for
WAAS, with the expectation that it will provide category I precision
guidance. In addition, according to an interest group representing general
aviation aircraft owners and pilots, these individuals support FAA's
approach because while proponents of the three alternative technologies make
claims about their capabilities, these capabilities have yet to be
validated. Furthermore, owners and pilots are not in favor of changing the
design of the new navigation system after FAA and aviation manufacturers
have agreed to specifications for equipment design and invested hundreds of
millions of dollars in this new design.
Modernization Projects That Depend on Precise Position Information
As currently designed, several of FAA's airspace modernization projects
depend on WAAS for precise position information and, according to FAA, could
not readily be developed without WAAS. While GPS provides position
information, WAAS is designed to ensure the integrity of that information.
If an alternative technology is used, these programs may need to be
redesigned. Chief among these is the Automatic Dependent Surveillance
Broadcast, one of a group of technologies designed to allow FAA to implement
a new system of traffic management. Under this system, known as free flight,
pilots and air traffic controllers would receive more precise information
about the location of aircraft in order to improve system safety while
allowing airspace and airport resources to be used more efficiently.
Specifically, the Automatic Dependent Surveillance Broadcast communicates
information about an aircraft's position from on-board equipment that
receives signals from global satellites and sends this information directly
to ground receivers and to nearby aircraft. With such information, more
aircraft can fly with increased efficiency and safety.
Other projects that depend on precise position information include enhanced
systems to warn pilots and air traffic controllers of proximity to the
ground and related obstacles. Depending on the application, the
effectiveness of these projects might be somewhat reduced in the absence of
WAAS. LAAS could provide monitoring if WAAS was not deployed; however, it
would only provide for such monitoring at a specific location/airport and
would not provide the coverage expected from WAAS.
FAA's recently announced delays in the initial deployment of WAAS, because
of difficulties in proving integrity, are likely to delay the benefits and
raise the cost of the new navigation system. These delays have not changed
FAA's original plans for WAAS, which were based on users' needs. However, if
it is determined that WAAS cannot perform as intended, FAA will need to
revisit its investment in the new navigation system.
With regard to WAAS integrity, FAA's management and its integrated product
team should have addressed these concerns in a more timely manner. FAA
recognizes that its management of this new system, particularly WAAS, has
fallen short, but its plans for resolving current problems, including the
integrity requirement, and preventing future delays are ambiguous. FAA talks
about establishing checkpoints to ensure progress and the appropriate use of
funds, but it has not developed a plan to assure the Congress that these
checkpoints are in place and are likely to function appropriately.
Moreover, program success will only come about if senior FAA management
embraces and fully supports the integrated product team concept and
establishes an on-going process for the team to reach consensus on how the
contractor must demonstrate that a project meets the agency's performance
requirements and to convey this information to the contractor. Otherwise,
more projects may experience the same problems WAAS encountered.
Finally, our past reviews of FAA's efforts to develop systems show that the
agency does not always inform the Congress in a timely fashion of problems
it is encountering before requesting additional funds. Potential users also
need information on system performance so that they can make informed
decisions on the merits of purchasing equipment to use with the new
navigation system. If further delays occur, these users may not continue to
support FAA in the development of this new system, as they now do.
To enhance FAA's ability to develop its new navigation system within budget
and on time while meeting performance requirements, we recommend that the
Secretary of Transportation direct the Administrator of FAA to take the
following actions:
� Develop a comprehensive plan that would provide the framework for the
agency's future investments in its new navigation system. This plan should
establish future checkpoints at which FAA would determine whether (1) the
contractor's approach for meeting performance requirements conforms with the
agency's guidelines, (2) users' needs have changed, and (3) other
technologies have matured and could meet users' needs and the agency's
requirements.
� Have an external organization evaluate the agency's progress at these
checkpoints and include the results of this evaluation in the agency's
request for future funding of the navigation system.
GAO provided the departments of Transportation and Defense and the Satellite
Navigation User Group with a draft of this report for their review and
comment.
GAO met with officials from the Department of Transportation, including the
Program Manager, Radionavigation and Positioning, Office of the Assistant
Secretary for Transportation Policy and FAA's Product Team Lead for the
Global Positioning System. These officials acknowledged the problems
addressed in the draft report concerning the Wide Area Augmentation System
and agreed with the conclusions and recommendations. However, they expressed
two major concerns. First, these officials stated that our characterization
of the increases in costs for the Wide Area Augmentation System as "cost
growth" was misleading because the increases are attributable to cost growth
as well as other factors, including a change in how the department
calculated project costs. The officials provided revised estimates to
reflect total lifecycle costs covering a 20-year period. To address this
concern, we revised the draft report to clarify that the cost increases were
due to growth as well as other factors, such as additional program support
costs. The focus of our review was system development; therefore, we also
clarified our draft report to indicate that the cost estimates represented
development costs only.
Second, these officials expressed the view that our discussion of the
benefits from the Wide Area Augmentation System implied that users would
only benefit if they were able to do the more stringent precision approaches
and that airports would need to spend a minimum of $1 million in
infrastructure upgrades to get precision approach capabilities from the Wide
Area Augmentation System. These officials maintained that users would
receive major benefits from any level of precision guidance into airports
and that airports would not necessarily need to upgrade their facilities to
provide for these approaches. We agree with the officials that users would
receive benefits even from the limited precision approach capability provide
by this system and that airports would not necessarily need to upgrade their
investment to provide for these approaches. We revised the draft report
accordingly. These officials also provided us with technical clarifications,
which we incorporated into the draft report as appropriate.
The Department of Defense generally concurred with the draft report. In
addition, it offered two comments. First, it clarified the department's
strategy for maintaining the Global Positioning System, and we revised the
draft report accordingly. Second, it commented that the continued delays in
the Wide Area Augmentation System and questions about worldwide
interoperability of this augmentation system could create additional cost
and schedule delays and adversely impact the department's operations if not
addressed. (See app. IV for the Department of Defense's comments.)
GAO met with representatives from the Satellite Navigation User Group, who
generally agreed with the draft report. These representatives reiterated
their support for FAA's efforts to field a new navigation system. In light
of this, they were concerned that the draft report did not recognize that
users would receive benefits even from the limited precision approach
capability provided by the Wide Area Augmentation System and that airports
would not necessarily need to upgrade their investment to provide for these
approaches. We agreed and revised the draft report accordingly. These
officials also provided us with technical clarifications, which we
incorporated into the draft report as appropriate.
We are sending copies of this report to interested Members of Congress; the
Honorable Rodney E. Slater, Secretary of Transportation; the Honorable
William S. Cohen, Secretary of Defense; and the Honorable Jane F. Garvey,
Administrator, Federal Aviation Administration. We will also make copies
available upon request.
If you have any questions about this report, please contact me at (202)
512-2834. Key contributors are listed in appendix V.
Sincerely yours,
Gerald L. Dillingham, Ph.D.
Associate Director,
Transportation Issues
Objectives, Scope, and Methodology
In light of the expected cost of the Federal Aviation Administration's (FAA)
new navigation system and continuing concerns about the ability of the
largest component of that system--the Wide Area Augmentation System
(WAAS)--to achieve cost, schedule, and performance goals, the Chairman,
Subcommittee on Transportation, Senate Committee on Appropriations, asked us
to provide information on whether (1) the Department of Defense's (DOD)
current Global Positioning System (GPS) or its planned improvements for GPS
can meet FAA's navigation requirements, (2) the benefits of FAA's chosen
approach to an augmented system currently outweigh the cost of this system,
and (3) other technologies are available to meet FAA's requirements and
users' needs for a new navigation system.
To address the first objective, we consulted with DOD officials and GPS
experts. In addition, we reviewed a 1999 study prepared by Johns Hopkins
University's Applied Physics Laboratory, sponsored in part by FAA,32 that
assessed, among other things, whether GPS can meet navigation requirements
for civil aviation. In addition, we reviewed a 1998 study by the Mitre
Corporation, which supports FAA in developing the national airspace system,
that analyzed GPS' current and future ability to meet navigation
requirements.
To respond to the second objective, we discussed with FAA and the Mitre
Corporation, the process they followed to develop a range of alternatives.
We reviewed guidance from the Office of Management and Budget (OMB) to
identify analytical issues agencies should address when doing benefit-cost
analyses. We also reviewed FAA's analysis of the costs and benefits of
various technologies to determine whether the agency followed OMB guidance
and used empirical data to support its findings. In our review, we also
confirmed the reasonableness of the largest costs estimates--for equipment
on board the aircraft and additional satellites--with equipment
manufacturers, satellite providers, and aviation industry trade associations
and special groups, including the General Aviation Manufacturers
Association, Aircraft Owners and Pilots Association, the Air Transport
Association of America, and the Regional Airline Association. The latter
four associations and other users are also part of the Satellite Navigation
User Group, which was created to achieve a consensus in the user community
and in FAA for making the transition to the new navigation system.
We also interviewed representatives from the Air Line Pilots Association
International and the National Air Traffic Controllers Association to obtain
their views on the reasonableness of cost and benefit assumptions made in
FAA's analysis. Furthermore, we relied on our past work related to WAAS to
compare the process FAA used for, and the results of, its most recent
benefit-cost analysis with a similar analysis performed in early 1998. While
we did not extensively review the model FAA used to calculate net benefits
and benefit-cost ratios, the methodology of this model, including
spreadsheet analysis and Monte Carlo Simulation, is commonly used in making
such calculations. Finally, we interviewed representatives from Raytheon,
the prime contractor for WAAS, to obtain data on the nature and extent of
problems in demonstrating that WAAS can meet its integrity requirement and
in developing software for navigation. To determine whether FAA's estimates
of the costs of software development and maintenance were reasonable, we
performed an independent assessment of the contractor's cost estimates using
a model known as "CostXpert," which is used in the federal systems
acquisition community to, among other things, determine the costs of
developing projects using historical costs of completed projects with
software development factors similar to WAAS.
Finally, for the third objective, we elicited the views of experts in
aviation navigation and related technologies to discuss the capability of
other technologies to meet FAA's requirements and users' needs. To
accomplish this effort, we provided these experts with detailed data on the
alternative technologies developed by the Mitre Corporation that served as
the basis for FAA's decision to continue with the new navigation system. We
asked these experts a series of questions that focused on whether FAA
adequately considered a full range of alternatives and whether they agreed
with FAA when the agency rejected certain technologies from its new
navigation system. Some of these experts supported WAAS, while others
supported technologies that compete with WAAS. Therefore, where we
identified conflicting views between the FAA and these experts, we discussed
the points of contention openly with both sides to fully understand their
positions, and we present the views for and against using alternative
technologies. (See app. II for background on the experts we contacted during
this review.) Finally, we interviewed FAA officials to discuss whether
interdependencies exist between WAAS, LAAS, and other modernization
projects, and the impact of these other efforts if WAAS is not deployed as
scheduled or is terminated.
We conducted our work from July 1999 through May 2000 in accordance with
generally accepted government auditing standards.
Experts Contacted
Mr. John M. Beukers
Founder, Beukers Laboratories, Inc. (today known as Beukers Technologies,
Inc.), a company specializing in the implementation and use of
radionavigation systems since 1963. Mr. Beukers is a director of the
International Loran Association and the International Navigation Association
and also serves as a consultant to the U.S. government, the European Union,
and others on radionavigation policy.
Dr. John Diesel
Chief Scientist, Litton Aero Products. Litton Aero is a manufacturer and
supplier of inertial navigation and global positioning systems to the
aviation industry. Dr. Diesel has analyzed several navigation
systems--including GPS, Inertial Reference System, Loran, VHF
Omni-directional Range/Distance Measuring Equipment, and Instrument Landing
Systems.
Mr. Paul R. Drouilhet
Consultant to the Director of the Massachusetts Institute of Technology's
Lincoln Laboratories, where he oversaw programs in surveillance and control
technology and air traffic control. Mr. Drouilhet is also the Chairman of
FAA's Subcommittee on Air Traffic Services and is an airplane pilot.
Dr. Per Enge
Professor, Stanford University, Department of Aeronautics and Astronautics;
Co-Director of the GPS Research Laboratory. Dr. Enge is also a developer of
the prototype system and signal for WAAS and a researcher for the prototype
of LAAS.
Dr. G. Benjamin Hocker
Principal Research Fellow, Honeywell Technology Center. Currently, Mr.
Hocker focuses on the evaluation of technologies that include, among other
things, inertial navigation systems.
Mr. George H. Quinn
Until 1994, Mr. Quinn was FAA's National Project Manager for the
establishment of the LORAN C navigation system for aviation. Currently, Mr.
Quinn is a self-employed engineering consultant, developing ways to use GPS
in vessel tracking systems.
Mr. William Roland
Private consultant. Former Loran C Branch Chief and Commanding Officer of
the U.S. Coast Guard's New Jersey Electronics Engineering Center. Mr. Roland
recently retired from his position as President of Megapulse, the company
responsible for the development and construction of a large part of the
world's Loran transmitters.
Dr. G. Linn Roth
President, International Loran Association. Dr. Roth is also the president
of Locus, Inc., a developer and manufacturer of spread-spectrum radio
modules for integration into industrial, utility, GPS, and high-performance
digital Loran receivers for navigation and timing applications.
Mr. Robert Siegfried
Former commercial airline pilot with approximately 24,000 hours of
experience. Mr. Siegfried's general aviation experience totals approximately
12,000 hours--including about 1,200 hours operating helicopters.
Mr. Victor Strachan
Director, Strategic Development, Litton Aero Products. Mr. Strachan served
as a navigator in the Royal Air Force for 18 years, including 5 years in
flight testing. Mr. Strachan is a director of the International Navigation
Association and a member of the Civil Aviation Council of the Aerospace
Industries Association. Litton Aero is a manufacturer and supplier of
inertial navigation and global positioning systems to the aviation industry.
FAA Identified Four Alternatives for a Benefit-Cost Analysis
In 1999, in response to congressional concerns in late 1998 about its
proposed investment in WAAS and LAAS and the vulnerabilities associated with
satellite navigation, FAA reevaluated whether the investment was sound. As a
first step in this reevaluation, FAA identified four alternatives that
combined a range of technologies, based on different levels of investment.33
Three of the four alternatives contained one or more variations of WAAS and
LAAS as well as variations of ground-based navigation aids that could serve
as the principal backup system. The four alternatives were the following:
� Alternative I: No WAAS or LAAS. FAA retains its existing ground-based
navigation infrastructure and enhances it to accommodate demand.
� Alternative II: Simplified WAAS With LAAS. FAA develops a simplified WAAS,
without incorporating precision approach guidance, develops LAAS, and
retains about 50 percent of the existing ground-based infrastructure used
for navigation and about 30 percent of the ground-based infrastructure used
for precision approaches.
� Alternative III: Less Robust WAAS With LAAS. FAA develops WAAS with
precision approach capability, including LAAS, for more stringent approach
requirements, and retains about 50 percent of its ground-based
infrastructure.
� Alternative IV: Full WAAS With LAAS. FAA develops WAAS with precision
approach capability, including LAAS, for more stringent approach
requirements, and retains about 30 percent of its existing ground-based
infrastructure.
In developing the alternatives, FAA made three major assumptions. First, it
assumed that the new navigation system would not be the only system that
pilots would use to navigate and land--a "sole-means" system--as originally
intended. By moving away from this assumption, FAA realized that it would
need to retain a backup system. Second, for all the alternatives, FAA
assumed that all new commercial jets would be produced with avionics on
board the aircraft to help pilots navigate (inertial navigation systems),
which would be used in conjunction with other ground-based navigation aids,
and that affordable inertial systems would not be available until about
2008. Finally, FAA assumed that the Long Range Navigation (LORAN C)
technology would continue to be provided and used by some general aviation
pilots until 2008.
After identifying the four alternatives, FAA analyzed the costs and benefits
of each. According to this analysis, continuing with the WAAS/LAAS system,
coupled with using about 30 percent of its current ground-based
infrastructure as a backup to GPS, would yield the greatest net quantified
benefits (dollar value of benefits minus costs),34 regardless of whether
passengers' time savings are included.35, 36 According to FAA, over the
period 2000 through 2020, there is an 80-percent chance that the net
benefits of its preferred alternative is about $2.5 billion if passengers'
time savings are counted and $72 million if these savings are not counted.
The benefit-cost analysis also identified at least 11 benefits that were not
quantified. Included among these were safety benefits, such as improved
surface surveillance, as well as operational benefits, such as enabling more
landings at airports where today's operations are limited by lack of
ground-based navigational aids. According to FAA, its decision to pursue
this approach had the support of all major user groups.37
Table 2 summarizes FAA's analysis for the four basic alternatives the agency
considered in terms of each alternative's benefit-cost ratio and net
benefits, expressed as present values.38 FAA considered these alternatives
with and without passengers' value of time. As the table shows, the fourth
alternative--which is the one FAA decided to pursue--provided a benefit-cost
ratio of 2.4, when passengers' value of time are included, and a ratio of
1.1, when these values are not included.
Dollars in millions
With passenger value of Without passenger value of
time time
Alternatives Benefit-cost Net Benefit-cost Net
ratio benefits ratio benefits
I. No WAAS; No LAAS 1.5 $280 0.5 ($287)
II. Simplified WAAS,
With LAAS 1.0 $94 0.5 ($1,007)
III. Less Robust
WAAS, with LAAS 2.1 $1,857 0.9 ($25)
IV. Robust WAAS with
LAAS While Retaining
30 Percent of Ground 2.4 $2,469 1.1 $72
Infrastructure
Note: Benefit-cost ratios and net benefits are calculated in present value
terms, which account for future benefits and costs, here expressed in 1999
dollars.
Source: FAA.
We found that FAA combined empirical data, where available, with
professional judgment to develop its estimates of costs and benefits. For
example, FAA's estimates for key cost elements, such as geostationary
satellites, on-board aircraft equipment, and ground-based navigation aids,
were based on vendor quotes and manufacturer pricing surveys. We
independently validated FAA's cost for developing WAAS software and found
that it was within an allowable range. Moreover, we found that FAA did not
assume that some benefits, such as direct routing--which yields savings from
shorter flight times--could only be achieved with WAAS and did not fully
attribute these benefits to WAAS as it had in earlier studies.
Furthermore, in accordance with OMB guidance and acceptable practice, FAA
identified a number of benefits that could not be quantified because of the
lack of verifiable data but that are nonetheless important for
decisionmakers to consider when choosing an investment option. For example,
OMB guidance notes that a comprehensive evaluation of the different types of
benefits and costs, quantified or not, can be helpful in identifying the
full range of program effects. We have also reported that estimates of
benefits are typically uncertain because of imprecision in both underlying
data and modeling assumptions, and it is appropriate to consider
nonquantified benefits when choosing investment options. Industry
representatives share FAA's views on the nonquantified benefits the agency
has identified, and some believe that these and other nonaviation benefits
could be substantial if they could be quantified.
Despite these strengths, we found that one key benefit--FAA's estimate of
the $4.4 billion in savings achieved by allowing planes to decrease the time
needed for an approach to landing--was based largely on professional
judgment, without empirical data. This benefit accounted for nearly 42
percent of all quantified benefits. Specifically, FAA assumed that 40
percent of all flights during nonpeak periods could decrease approach time
into airports, even though it recognized that the percentage could range
from 20 percent to 50 percent during nonpeak periods. Because of the lack of
empirical data, we asked FAA to perform a more conservative analysis,
assuming that only 20 percent of the flights would benefit. We found that
this resulted in a benefit-cost ratio of 1.9 if passenger value of time was
counted and a ratio of .8 if not counted.
Comments From the Department of Defense
GAO Contacts and Staff Acknowledgments
Gerald L. Dillingham, (202) 512-2834
Belva M. Martin, (202) 512-2834
In addition to those named above, Charles W. Bausell, Jennifer W. Clayborne,
Richard B. Hung, Peter G. Maristch, John T. Noto, Madhav S. Panwar, Karen A.
Richey, and Carol Herrnstadt Shulman made key contributions to this report.
(348184)
Table 1: Development Costs and Schedules for WAAS and LAAS,
1994 Through September 1999 9
Table 2: Summary of FAA's Economic Analysis of Four Alternatives 35
Figure 1: The Operating System for WAAS 8
1. In the past, the Department degraded the accuracy of the GPS signal using
a process known as "selective availability."
2. Throughout this report, we refer to the augmented satellite navigation
system and its components--including the Local Area Augmentation System and
existing ground-based navigation aids--as FAA's new navigation system. This
system also provides guidance to help pilots land at airports. Also,
throughout this report, unless otherwise specified, costs are presented in
"then-year-dollars," which are current dollars, inflated using Office of
Management and Budget guidance.
3. Replacement satellites will be launched as needed.
4. Navigation guidance is provided to pilots through all phases of
flight--at high altitudes and in areas close to airports.
5. Beginning is fiscal year 1999, FAA established a partnership with
interested commercial entities for the purpose of developing LAAS. FAA
expects this partnership to culminate in the development of a certified
category I precision approach using LAAS by the end of fiscal year 2002.
6. This study, entitled GPS Risk Assessment Study, Final Report (Jan. 1999),
was conducted by the Johns Hopkins University Applied Physics Laboratory and
was co-sponsored by FAA, the Air Transport Association, and the Aircraft
Owners and Pilots Association. According to the Air Transport Association,
the primary purpose of this study was to determine the capability of the
augmented GPS signal using WAAS and LAAS.
7. Continuity and service volume are also considered major requirements, and
they are derived from the accuracy, integrity, and availability
requirements. Continuity is the probability that a navigation signal will
meet accuracy and integrity requirements continuously for a specified
period. Service volume is the area of coverage for which a navigation signal
will meet availability requirements.
8. En route navigation occurs when planes are in transit over the
continental United States.
9. According to DOD, GPS' availability is contingent upon the alignment of
satellites with the earth's rotation.
10. Europe and Japan are currently developing augmentation systems.
11. The Raytheon Company has been the prime contractor since May 1996.
12. See Satellite Navigation Investment Analysis Report, Federal Aviation
Administration (Sept. 25, 1999).
13. As an alternative to the benefit-cost ratio, for which the present value
of benefits is divided by the present value of costs, analysts sometimes
calculate the present value of net benefits. This value is equal to the
present value of benefits minus the present value of costs.
14. DOT values passenger time from $22 to $33 per hour, depending on the
nature of air travel. In 1998, our review of the economic literature found
that no consensus exists on the validity of valuing small increments of
time, in part because passengers might not perceive and value time savings
of as little as 30 seconds.
15. It should be noted that the other three options had benefit-cost ratios
of less than 1.
16. A benefit-cost ratio is a measure of the relationship between the
present value of a project's benefits and costs. Since benefits are divided
by costs, any ratio above 1.0 indicates that the project is cost-beneficial;
any ratio below 1.0 indicates that the project is not cost-beneficial.
17. Data were not available to quantify 5 of the 11 benefits. For the other
six, FAA determined that the benefits would derive from WAAS, combined with
other technologies, and therefore did not attempt to separate and quantify
the WAAS benefits.
18. The Satellite Navigation User Group--which includes representatives from
commercial, general aviation and Department of Defense users--was created to
achieve a consensus throughout the user community and within FAA for making
the transition to the new navigation system. In Mar. 2000, this group
reaffirmed its support for WAAS.
19. WAAS' initial capability was defined as vertical guidance of 200 feet
above touchdown with a one-half to three-fourths mile visibility and a 19.2
meter vertical protection limit in which an aircraft can maneuver and still
land safely. This capability was to be available 95 percent of the time to
about 50 percent of the continental United States.
20. By 2002, FAA plans to provide vertical guidance of 350 feet above
touchdown with one mile visibility and a 50 meter vertical protection limit
in which an aircraft can maneuver and still land safely. This capability is
to be available 95 percent of the time to about 75 percent of the
continental United States.
21. See Aviation Acquisition: A Comprehensive Strategy is Needed for
Cultural Change at FAA (GAO/RCED-96-159 , Aug. 22, 1996).
22. Final Report of the WAAS Study Group (Oct. 16, 1997).
23. See Aviation Acquisition: A Comprehensive Strategy is Needed for
Cultural Change at FAA (GAO/RCED-96-159 , Aug. 22, 1996).
24. See Air Traffic Control: Observations on FAA's Air Traffic Control
Modernization Program (GAO/T-RCED/AIMD-99-137 , Mar. 25, 1999).
25. Software size is usually measured in lines of code. A line of code is a
set of instructions for the computer to perform a certain task and is one
basis for estimating costs for items such as the coding, analysis, design,
and test efforts required for producing the line of code.
26. See Air Traffic Control: Immature Software Acquisition Processes
Increase FAA System Acquisition Risks (GAO/AIMD-97-47 , Mar. 21, 1997).
27. This assumes no value to any passenger time savings.
28. In considering FAA requirements and users' needs, it is important to
factor in the cost of providing precision landing guidance. FAA notes that
it could provide this guidance through technologies such as instrument
landing systems at all runway ends in the United States, but this would be
cost prohibitive.
29. LORAN C was originally designed as a primary navigation system for the
maritime community. This technology is ground-based and can provide an
independent navigation service that does not rely on GPS.
30. An Assessment of the Proposed Phase Out of the LORAN C Navigation
System, Booz-Allen & Hamilton, Inc., July 17, 1998.
31. Users can receive vertical guidance from different types of equipment.
For example, some commercial aircraft are currently equipped with flight
management systems that are coupled with altimeters to provide vertical
guidance. Using this equipment, pilots can make "near" precision approaches.
32. The Johns Hopkins University study was performed under contract for the
Air Transport Association of America (ATA) and jointly sponsored by ATA,
FAA, and the Aircraft Owners and Pilots Association.
33. In total, FAA considered 12 alternatives for its new navigation system.
34. As an alternative to the benefit-cost ratio, for which the present value
of benefits is divided by the present value of costs, analysts sometimes
calculate the present value of net benefits. This value is equal to the
present value of benefits minus the present value of costs.
35. The Department of Transportation values passenger time from $22 to $33
per hour, depending on the nature of air travel. In 1998, we reviewed the
economic literature and found that no consensus exists as to the validity of
using small increments of time, in part because passengers might not
perceive and value time savings of as little as 30 seconds.
36. It should be noted that the other three options had benefit-cost ratios
of less than 1.
37. The Satellite Navigation User Group was created to achieve a consensus
throughout the user community and with FAA for making the transition to the
new navigation system.
38. Since benefits are divided by costs, any benefit-cost ratio above 1.0
indicates that the project has benefits exceeding costs, and any ratio below
1.0 indicates that the project has benefits less than costs.
*** End of document. ***