Aviation Safety: Safer Skies Initiative Has Taken Initial Steps to Reduce
Accident Rates by 2007 (Chapter Report, 06/30/2000, GAO/RCED-00-111).
Pursuant to a congressional request, GAO reviewed the Federal Aviation
Administration's (FAA) Safer Skies Initiative, focusing on: (1) to what
extent addressing the safety problems to be addressed by the initiative
will help reduce the fatal accident rate; (2) what progress the
initiative has made in identifying and implementing interventions to
address each of these safety problems; (3) what progress has been made
in assessing the effectiveness of those interventions; and (4) how FAA
is coordinating the Safer Skies initiative with other safety activities
conducted throughout the agency, in partnership with the aviation
industry, and by other federal agencies.
GAO noted that: (1) the Safer Skies initiative addresses the safety
problems that have contributed to fatal accidents in the past, and in
conjunction with other safety problems, it can be expected to reduce the
fatal accident rate and thus enhance the safety of the nation's air
passengers; (2) in commercial aviation, the initiative addresses safety
problems that accounted for over three-quarters of the fatal accidents
in those operations in 1988-1997; (3) in general aviation, the Safer
Skies initiative plans to address safety problems that appear to be the
most common causes of fatal accidents; (4) the initiative has adopted a
less aggressive goal in general aviation of reducing the number of fatal
accidents to 350 in 2007, which represents about a 20-percent reduction;
(5) the initiative addressed four safety problems in cabin safety; (6)
to date, safety improvement efforts by FAA and the initiative have
focused on reducing the causes of past accidents and incidents, which
may not be entirely predictive of future ones; (7) as of April 1, 2000,
Safer Skies teams had started work on 13 of the 16 safety problems and
had begun implementing interventions for 5 of these--2 in commercial
aviation and 3 in cabin safety; (8) since most of the interventions
developed under the Safer Skies initiative are in early implementation
stages, little progress has been made in evaluating their effectiveness;
(9) of the five Safer Skies teams that have begun implementing
interventions, only one has developed a performance measure to evaluate
whether the interventions it has selected are helping to reduce the
safety problems that cause fatal accidents and are worth what they cost;
(10) FAA has coordinated extensively with aviation experts from
industry, other federal government agencies, and its own staff, but
GAO's review identified three coordination problems that could undermine
the implementation and evaluation of Safer Skies' interventions; (11)
although FAA officials have repeatedly committed to funding
interventions agreed upon by all parties, skepticism still exists among
some participants as to whether this commitment can or will be honored;
(12) furthermore, if funding is limited, it remains unclear what process
will be used to reprioritize available resources to ensure funding for
interventions that emerge later but have greater potential for reducing
the fatal accident rate; and (13) the Safer Skies initiative, FAA, and
the Department of Transportation (DOT) have not agreed on how they will
measure progress in achieving the accident reduction goal for commercial
aviation.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-00-111
TITLE: Aviation Safety: Safer Skies Initiative Has Taken Initial
Steps to Reduce Accident Rates by 2007
DATE: 06/30/2000
SUBJECT: Safety standards
Commercial aviation
Transportation safety
Interagency relations
Air transportation operations
Aircraft accidents
Performance measures
Accident prevention
IDENTIFIER: FAA Safer Skies Initiative
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO/RCED-00-111
8
Principal Findings 13
20
FAA Expects Continued Growth in Aviation 20
Fatal Accident Rates Have Decreased for U.S. Aviation 22
The Safer Skies Initiative Continued Ongoing Efforts to Use
Data Analysis to Address Safety Problems 29
Objectives, Scope, and Methodology 32
Safety
34
The Safer Skies Initiative Addresses Major Safety Problems in
Commercial Aviation 34
Accident Data and Other Resources Were Used to Identify Safety
Problems That Caused Many Fatal Accidents in General Aviation 44
Improving Cabin Safety Is Important but Will Have
Little Impact on Lowering the Fatal Accident Rate 48
Additional Work on Future Hazards Could
Help Anticipate and Prevent Fatal Accidents 49
Conclusions 50
Recommendations 51
Agency Comments 52
and Implementing Interventions
53
The Safer Skies Methodology Is Based on Previous Efforts to
Identify Safety Problems 54
The Safer Skies Initiative Has Made the Most
Progress With Problems Studied Previously 57
Early Experience Indicates That Future
Problems Will Require More Analysis 61
Effective Implementation Is Critical Next Step in Making
Progress Toward the Goals Set for Reducing Fatal Accidents 61
Conclusions 68
Recommendations 68
Agency Comments 69
Performance Measures to Evaluate the Effectiveness of Most Interventions
71
Federal Law Requires the Development of Performance
Measures as Part of the Budget Process 71
Determining the Effectiveness of Safer Skies' Initiatives Will
Require the Development of Additional Performance Measures 73
Most Safer Skies' Interventions Are Being Implemented
Without Determining How to Evaluate Their Effectiveness 74
Conclusions 84
Recommendations 85
Agency Comments 85
Improvement for the Safer Skies Initiative to Succeed
86
The Safer Skies Initiative Involves an Unprecedented Level
of Coordination Between Industry and Government 86
The Funding, Prioritization, and Evaluation of Safer Skies
Interventions Could Be Better Coordinated 87
Conclusions 94
Recommendations 95
Agency Comments 95
Appendix I: GAO Contacts and Staff Acknowledgments
98
99
Table 1: Number of Fatal Accidents and Deaths by Type of
Aviation Operation, 1988-97 10
Table 2: Fatal Accidents and Deaths by Type of Aviation
Operations, 1988-97 22
Table 3: Projected Numbers of Fatal Accidents by Type of
Operation in 2007 26
Table 4: The 16 Safety Problems Addressed by Safer Skies 31
Table 5: Numbers of Fatal Accidents and Fatalities in Commercial
Aviation 1988-97 41
Table 6: Number of Past and Projected Fatal Accidents by Type
of Aviation Operation 44
Table 7: Status of Safer Skies' Activities for Commercial and
General Aviation 58
Table 8: Baselines and Goals for Reducing the Occurrence of
Uncontained Engine Failure in Commercial Aviation 76
Figure 1: Commercial Aviation and General Aviation Flight
Hours, 1982-99 21
Figure 2: Fatal Accident Rates for Commercial and General
Aviation, 1982-99 25
Figure 3: Safety Problems That Caused Fatal Accidents in
U.S. Commercial Aviation, 1988-97 37
Figure 4: Fatalities in U.S. Commercial Aviation by Safety
Problem, 1988-97 43
CFIT Controlled Flight Into Terrain
DOT Department of Transportation
FAA Federal Aviation Administration
GAO General Accounting Office
NTSB National Transportation Safety Board
PICS Partners in Cabin Safety
Resources, Community, and
Economic Development Division
B-283391
June 28, 2000
The Honorable John J. Duncan, Jr.
Chairman
The Honorable William O. Lipinski
Ranking Democratic Member
Subcommittee on Aviation
Committee on Transportation
and Infrastructure
House of Representatives
As requested, we are reporting on the actions taken by the Federal Aviation
Administration (FAA), through the Safer Skies initiative, towards the goal
of reducing the nation's fatal aviation accident rates by 2007. Our report
contains recommendations designed to ensure that the implementation of
interventions approved by FAA and the Safer Skies steering committees is
tracked and that the interventions are evaluated for their effectiveness in
meeting the goal.
As arranged with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 7 days after
the date of this letter. At that time, we will send copies to o the
appropriate congressional committees; the Honorable Rodney E. Slater,
Secretary of Transportation; the Honorable Jane F. Garvey, Administrator,
FAA; and the Honorable Jacob Lew, Director, Office of Management and Budget.
We will also make copies available to others upon request.
If you have any questions about this report please contact me or Robert
White at (202) 512-2834. Other key contributors to this report are listed in
appendix I.
Sincerely yours,
Gerald L. Dillingham, Ph.D.
Associate Director, Transportation
Issues
Executive Summary
The continued growth forecast for U.S. aviation in the coming decade will
likely bring a rise in fatal accidents if the current accident rate is not
reduced.1 Commercial aviation, used by most Americans when they fly,
experienced an average of 6 fatal accidents a year in the United States in
1994-96; general aviation experienced an average of 380 a year.2 If the
projected growth in flight hours occurs and the fatal accident rate is not
reduced, GAO estimates in this report that the number of fatal commercial
aviation accidents could rise to 9 per year and the number of fatal general
aviation accidents to 484 by 2007. The Federal Aviation Administration
(FAA), the Congress, and the aviation industry have acknowledged this
potential danger and have recommended ways to address it. In 1997, two major
commissions on aviation safety recommended reducing the nation's aviation
accident rate by 80 percent by 2007. To meet this challenging goal, both the
White House Commission on Aviation Safety and Security and the
congressionally mandated National Civil Aviation Review Commission
recommended that FAA and the aviation industry work together to identify and
address the causes of fatal accidents. To unify government and industry
efforts to reduce the accident rate by addressing the greatest threats to
aviation safety, FAA announced the Safer Skies initiative in April 1998 with
the broad initial goal of reducing the number of fatal accidents per million
flight hours by 80 percent by 2007.
Aviation experts from FAA, the aviation industry, and other government
agencies formed three steering committees to oversee the initiative's work
in three broad areas: commercial aviation, general aviation, and cabin
safety. The steering committees will analyze data to identify the most
serious threats to safety, to find the root causes of accidents, and then to
determine the best actions to break the chain of events that lead to
accidents. Resources will be directed first to implementing those best
actions, referred to as interventions. The steering committees have
identified 16 specific safety problems--6 related to commercial aviation, 6
to general aviation, and 4 to cabin safety. These problems will be addressed
by teams of aviation experts who can recommend one or many interventions for
the safety problems they are addressing. Some of the safety problems, such
as weather, will be addressed by both commercial aviation and general
aviation teams because their causes and interventions may differ for these
types of operations.
In light of the critical importance of the Safer Skies initiative in
improving aviation safety, the Chairman and Ranking Democratic Member of the
Subcommittee on Aviation, House Committee on Transportation and
Infrastructure, asked GAO to review the implementation of this initiative.
Specifically, they asked GAO to determine (1) to what extent addressing the
safety problems selected by the Safer Skies initiative will help reduce the
fatal accident rate; (2) what progress the initiative has made in
identifying and implementing interventions to address each of these safety
problems; (3) what progress has been made in assessing the effectiveness of
those interventions; and (4) how FAA is coordinating the Safer Skies
initiative with other safety activities conducted throughout the agency, in
partnership with the aviation industry, and by other federal agencies.
Since 1982, air travel in the United States has increased dramatically, and
flight safety has improved. The number of hours flown by commercial aircraft
more than doubled from 8 million hours in 1982 to nearly 18 million hours in
1999. FAA estimates that commercial aviation aircraft will fly more than 24
million hours in 2007, an increase of 37 percent from 1999. Growth in
general aviation has been less consistent, but FAA estimates that general
aviation flight hours will increase to about 36 million hours in 2007, a
growth of nearly 19 percent over 1999. Although the accident rates for both
types of operations are low, both the number and the frequency of aviation
deaths will likely increase if these rates are not reduced as the growth in
air travel continues. In the 10-year period from 1988 through 1997, the
United States had 4,471 fatal aviation accidents that resulted in a total of
9,802 deaths. Commercial aviation accounted for only 2 percent of the fatal
accidents, while general aviation accounted for 98 percent. (See table 1.)
Fatal accidents Deaths
Type of operation Number Percentage Number Percentage
Commercial aviation 85 2 1,756 18
General aviation 4,386 98 8,046 82
Total 4,471 100 9,802 100
Source: GAO's analysis of data from the National Transportation Safety
Board.
The Safer Skies initiative addresses the safety problems that have
contributed to fatal accidents in the past, and in conjunction with other
safety programs, it can be expected to reduce the fatal accident rate and
thus enhance the safety of the nation's air passengers. In commercial
aviation, the initiative addresses safety problems that accounted for over
three-quarters of the fatal accidents in those operations in 1988-97. Other
FAA initiatives are addressing additional safety problems, which should
complement Safer Skies' efforts to meet the goal of an 80-percent reduction
in the fatal accident rate for this segment of the aviation industry. In
general aviation, the Safer Skies initiative plans to address safety
problems that appear to be the most common causes of fatal accidents. The
initiative has adopted a less aggressive goal in general aviation of
reducing the number of fatal accidents to 350 in 2007, which represents
about a 20-percent reduction. Finally, the initiative addressed four safety
problems in cabin safety. Improving cabin safety will have little impact on
lowering the fatal accident rate because cabin safety accounted for only two
U.S. commercial aviation fatalities in 1988-97. No quantitative goal was set
for safety improvements in cabin safety. To date, safety improvement efforts
by FAA and the initiative have focused on reducing the causes of past
accidents and incidents, which may not be entirely predictive of future
ones. Studying growth and technological changes in the aviation industry can
help anticipate and prevent the safety problems and accidents that are
likely to arise from such changes. An international work group has been
formed to address future hazards, and a number of FAA staff participate in
this work group. Coordinating these efforts with Safer Skies' work should
enhance the initiative's efforts to reduce the fatal accident rate.
As of April 1, 2000, Safer Skies teams had started work on 13 of the 16
safety problems and had begun implementing interventions for 5 of these-- 2
in commercial aviation and 3 in cabin safety. Teams have made the most
progress in selecting interventions to address safety problems when they
have been able to build on previous studies for which widely supported
recommendations already existed. Since its inception in April 1998, the
Safer Skies initiative has evolved as new safety problems have been
addressed. For example, the process used to analyze safety problems and
select interventions has been modified as Safer Skies teams have begun to
address safety problems that have received less extensive study. Because
many of these safety problems are long-standing ones that have not been
fully resolved by prior efforts, progress will depend on effectively
implementing the chosen interventions. The initiative has developed a
process for tracking the implementation of interventions to improve safety
in commercial aviation. However, the implementation of Safer Skies'
interventions is not assured because the tracking system for commercial
aviation is not sufficiently detailed to assess progress in implementing
interventions, the system for general aviation is still in development, and
no system exists for tracking recommended interventions in cabin safety.
Without such systems, the Safer Skies initiative cannot ensure that all of
the interventions approved to increase aviation safety will be put into
action.
Since most of the interventions developed under the Safer Skies initiative
are in early implementation stages, little progress has been made in
evaluating their effectiveness. Of the five Safer Skies teams that have
begun implementing interventions, only one has developed a performance
measure to evaluate whether the interventions it has selected are helping to
reduce the safety problems that cause fatal accidents and are worth what
they cost. Such evaluation requires that performance measures be developed
to serve as the yardsticks for measuring the progress made toward the
program goals--a process required for federal programs by the Government
Performance and Results Act of 1993. The initiative has established safety
goals based on reducing the fatal accident rate for commercial aviation by
80 percent and reducing the number of fatal general aviation accidents to
350 by 2007. It plans to measure annual progress toward meeting those goals.
Fatal accidents occur too rarely, especially in commercial aviation, to
serve as measures of the effectiveness of specific interventions. Other
indicators exist or can be developed to measure the unique effect of
individual interventions.
FAA has coordinated extensively with aviation experts from industry, other
federal government agencies, and its own staff, but GAO's review identified
three coordination problems that could undermine the implementation and
evaluation of Safer Skies' interventions. First, although FAA officials have
repeatedly committed to funding interventions agreed upon by all parties
working on the initiative, skepticism still exists among some participants
as to whether this commitment can or will be honored. This is particularly
true in general aviation. Furthermore, if funding is limited, it remains
unclear what process will be used to reprioritize available resources to
ensure funding for interventions that emerge later but have greater
potential for reducing the fatal accident rate. Finally, the Safer Skies
initiative, FAA, and the Department of Transportation (DOT) have not agreed
on how they will measure progress in achieving the accident reduction goal
for commercial aviation.
We presented a draft of this report to DOT and FAA for comment. DOT and FAA
officials characterized the report as fair and reasonable and provided
technical clarifications, which were made as appropriate. The officials
concurred with most of our recommendations. However, DOT and FAA officials
disagreed with our recommendation that some basic criteria and processes
should be established for reprioritizing interventions if funding is
limited. Because we believe that such criteria and processes would be useful
in assessing the potential impact of safety interventions, we did not modify
or delete our recommendation. The officials also disagreed with two of our
recommendations calling for an analyses of safety problems that have not
been studied previously and of problems that may arise in the future. The
officials provided information showing that FAA is taking these actions, so
we withdrew these recommendations.
The six safety problems that Safer Skies addresses in commercial aviation
accounted for about 79 percent of the fatal accidents in commercial aviation
in 1988-97. Three of these safety problems accounted for 58 of the 85
accidents during this period: pilots' losing control of their aircraft,
pilots' flying otherwise controllable aircraft into the ground or water, and
accidents during approach and landing.3 Commercial aviation teams will
develop interventions for these safety problems in large aircraft, while
accidents that involved smaller commuter aircraft were referred to the
general aviation steering committee for review. To further reduce the fatal
accident rate for commercial aviation, the commercial aviation teams will
address three other safety problems4 that resulted in fewer fatal accidents
but have the potential to cause many more in the future. FAA also has
ongoing initiatives to address the causes of several additional safety
problems that caused four fatal accidents, including sabotage, fuel tank
explosions, and structural problems.
In general aviation, the Safer Skies initiative also addresses major safety
problems, but the goal chosen does not encourage aggressive steps to
decrease general aviation accidents. Although the data available on general
aviation accidents are less detailed than those available on commercial
aviation accidents, the problems to be addressed in general aviation include
ones identified in past safety reports and in the National Transportation
Safety Board's (NTSB) accident reports as major causes of fatal accidents.
These include, for example, weather, loss of control, and runway incursions.
In establishing a goal for general aviation, the initiative did not adopt
the 80-precent goal proposed by the two aviation safety commissions. The
initiative chose a goal of 350 fatal general aviation accidents in 2007.
This represents a 20-percent reduction in the number of fatal accidents
projected for that year given expected growth. The Safer Skies initiative
also set an interim goal of 379 fatal accidents for each of the next 3
years. This interim goal represents a 7-percent increase over the number of
fatal accidents in 1999 and does not challenge the general aviation
community to continue the kinds of safety improvements that helped reduce
such accidents to 354 last year.
Initially, the Safer Skies initiative focused on addressing the safety
problems that caused past fatal accidents and serious incidents. Aviation's
significant growth and changes in the industry's operations are likely to
lead to types of accidents that differ from those in the past. To anticipate
and prevent accidents that could result from such changes, the Joint Safety
Strategy Initiative in Europe5 has formed a work group to develop a method
for examining future hazards. Representatives from FAA associated with the
Safer Skies initiative participate in this work group to help coordinate the
initiative's work on past accidents and incidents with the international
work on future hazards. This work on future hazards is preliminary and is
currently focused on developing a methodology for identifying and addressing
these potential safety problems. As this work group's efforts progress,
coordinating these two efforts should help avoid duplication of effort and
foster awareness of and solutions to these potential problems
internationally.
In Selecting and Implementing Interventions
As of April 1, 2000, Safer Skies teams had identified a number of
interventions, and efforts were being implemented to address 5 of the 16
safety problems; study is under way on an additional 8; and 3 have not yet
been addressed. The teams dealing with well-studied safety problems have
made the most progress in selecting and implementing interventions. These
include uncontained engine failure, controlled flight into terrain, and
weather in commercial aviation and controlled flight into terrain and
weather in general aviation. Progress occurred because these teams were able
to take advantage of existing studies and safety recommendations. For
example, the team reviewing uncontained engine failure has completed its
work; the more extensive engine inspections it recommended are under way. In
addition, air carriers had taken action on some high-priority
recommendations before FAA issued a final rule or the Safer Skies team
issued its final report. Specifically, air carriers began installing the
enhanced navigational equipment in their aircraft to prevent accidents from
controlled flight into terrain before FAA issued its final rule in March
2000 or the commercial aviation team working on controlled flight into
terrain issued its final report in June 2000. Educational information has
also been made available to improve cabin safety by publicizing the safety
benefits of greater passenger use of seat belts and child restraint systems
and issues associated with the stowage of carry-on baggage.
GAO found that the process Safer Skies teams have been using to choose
interventions and implement them is reasonable and has allowed FAA and
industry groups to reach consensus on how to address safety problems. The
first Safer Skies teams that used this process were able to compare their
results with those of prior FAA and industry studies. The process for
analyzing data and selecting interventions has been modified by Safer Skies
teams to accommodate differences in available data on other safety problems.
For example, the runway incursion team analyzed incidents because so few
fatal accidents result from this safety problem. Such evolution in the
process will be critical when addressing safety problems that have not been
studied previously.
The interventions selected by Safer Skies teams can reduce the fatal
accident rate only if they are implemented effectively. GAO's past work has
shown that FAA does not consistently follow through on implementing key
safety recommendations.6 The initiative has developed a system for tracking
the implementation of interventions to improve safety in commercial
aviation. However, the implementation of Safer Skies' interventions is not
assured because the tracking system for commercial aviation is not
sufficiently detailed to assess progress in implementing interventions.
Furthermore, although the general aviation steering committee is approaching
final approval on interventions to address two safety problems, it is still
developing a tracking system, and no system was developed to track
interventions implemented in cabin safety.
Evaluate the Effectiveness of Most Interventions Under Way
The Safer Skies initiative has developed a way to evaluate the effectiveness
of one intervention it has under way to reduce the fatal accident rate.
Performance measures are needed both to fulfill the mandate of the
congressional commission that such performance measures be established and
to meet the requirements of federal law, which requires federal departments
requesting funding to evaluate the effectiveness of their programs. Since
fatal aviation accidents are infrequent, the effectiveness of Safer Skies'
interventions must be measured using information about other kinds of
events, such as incidents that typically precede accidents. The Safer Skies
steering committees recognized early on that alternative measures would be
needed to measure the unique effect of individual interventions. Thus far,
however, only the uncontained engine failure team has developed a
quantifiable performance measure. In contrast, the two general aviation
teams that have completed their work included no specific, quantifiable
measures for evaluating the effectiveness of their interventions. For
example, one team recommended better marking of towers and power wires to
prevent fatal accidents that result when low-flying aircraft strike these
objects. The team's performance measure was a decrease in the number of
accidents involving wires and towers. The effectiveness of this intervention
cannot be measured without specific, quantified baseline information on how
many of these accidents occurred in the past, what growth is expected in
general aviation, and what reduction the team hopes to achieve with this
intervention. No performance measures were developed to evaluate the
educational interventions implemented for the four cabin safety problems.
Impact of Safer Skies' Interventions
Although Safer Skies steering committees and teams included many government
and industry aviation experts, three aspects of coordination could be
improved to enhance the chances of implementing and evaluating the
initiative's safety interventions. First, the steering committees for
commercial aviation and general aviation have both sought the commitment of
all participants to implementing and funding interventions before giving
final approval to move forward. However, FAA's commitment has come at
different points in the approval process for interventions recommended by
these steering committees. FAA's commitment to the general aviation
interventions is still uncertain even after some participants believed that
the steering committee had granted its final approval. As a consequence,
general aviation participants were more skeptical about whether FAA would
implement or fund their safety interventions. In October 1999, FAA formed an
executive council to help coordinate the implementation of the agency's
safety agenda, but this council has not yet documented its process for
approving and funding interventions.
Second, if funding is limited, it remains unclear what process will be used
to reprioritize available resources to ensure funding for interventions that
emerge later but have greater potential for reducing the fatal accident
rate. A Safer Skies team has just begun work on loss of control--which
caused the greatest number of fatal accidents in commercial aviation in
1988-97. Interventions to address loss of control are thus likely to be
critical for reducing the fatal accident rate. If funding is limited, some
resources may have to be shifted from existing programs and safety
initiatives. The Safer Skies initiative and FAA's executive council have not
yet established any process for reprioritizing interventions if funding is
limited.
Finally, coordination among Safer Skies steering committees, FAA, and DOT
needs to improve to ensure the effective evaluation of Safer Skies
interventions. DOT is responsible for setting safety goals for all modes of
transportation under its authority, including aviation. Each of its
agencies, including FAA, provides more detailed information on how it will
achieve those safety goals. Currently, DOT and FAA measure progress toward
the goal of an 80-percent reduction in the fatal accident rate for
commercial aviation in different ways. Specifically, DOT's performance plan
measures progress using a fatal accident rate based on flight hours, while
FAA's strategic plan and the Safer Skies initiative use an accident rate
based on aircraft departures as the measure of activity. Since the ultimate
mission of all three groups is to reduce the fatal accident rate, using the
same activity measure to calculate that rate would make sense. Because most
commercial aviation accidents occur during takeoff and landing, GAO believes
that using departures better measures passengers' exposure to risk.
To improve FAA's safety agenda for decreasing fatal aviation accidents, GAO
makes a number of recommendations in this report to the Secretary of
Transportation to direct the FAA Administrator in her work with the Safer
Skies steering committees. These recommendations focus on establishing a
more challenging goal for reducing fatal accidents in general aviation and
improving the implementation and evaluation of the Safer Skies initiative.
GAO provided copies of a draft of this report to the Department of
Transportation and FAA for their review and comment. GAO met with FAA
officials, including the Deputy Associate Administrator for Regulation and
Certification and the Director of Aircraft Certification. The FAA officials
concurred with the majority of our recommendations and characterized the
Safer Skies report as generally "fair and reasonable." They informed GAO of
actions taken by the agency since GAO completed its audit work in March.
This information has been incorporated as appropriate. FAA concurred with
the need to set more challenging interim and long-term goals for general
aviation and plans to do so in the future. FAA officials agreed that
improvements were needed in how the Safer Skies initiative tracks the
implementation of interventions, although they disagreed with the level of
detail suggested by GAO. They also agreed with GAO's recommendations to
improve the baseline data and performance measures used to evaluate the
impact of these interventions.
The Safer Skies initiative has taken steps to expand and improve its
tracking of interventions, but the system still does not clearly identify
and specify time frames for major commitments and deliverables for each
approved intervention. Without a reliable tracking system, FAA and Safer
Skies will not be in a position to ensure that recommended interventions are
implemented to improve aviation safety.
Background
The Federal Aviation Administration (FAA) has forecast continued growth for
commercial and general aviation over the next decade. 7 Growth over the past
few decades brought innovations to improve flight safety that contributed to
a dramatic lowering of the accident rate by the mid-1970s. Further
reductions in the accident rate have, however, remained elusive. Unless the
current accident rate can be reduced, the number of fatal accidents is
likely to increase as aviation operations continue to grow. During the
1990s, FAA, the aviation industry, and the Congress all acknowledged and
studied this potential danger. They set ambitious targets for reducing the
accident rate, made over a thousand recommendations for improving aviation
safety, and implemented a number of safety initiatives. In spite of these
efforts, the accident rate, which is already low, has remained fairly
steady. The FAA Administrator, White House and congressional task forces,
and aviation industry groups have concluded that FAA and the aviation
industry must coordinate their efforts to prioritize safety recommendations
and focus resources on those with the most potential to decrease the
accident rate. In 1998, the FAA Administrator announced the Safer Skies
initiative, a joint government-industry effort to identify and address the
greatest threats to aviation safety in order to reduce the fatal accident
rate by 80 percent by the year 2007.
Over the past several decades, aviation has grown substantially in the
United States, and FAA expects this growth to continue into the next
century. Commercial aviation has grown consistently since 1982, while growth
in general aviation has been less consistent. One key measure of aviation
activity shows that the number of flight hours for commercial aircraft more
than doubled from 8 million hours in 1982 to nearly 18 million hours in
1999. In contrast, general aviation activity dropped fairly steadily from
the early 1980s until 1995. While general aviation has grown since 1995, it
has not yet returned to 1990 levels. The number of general aviation flight
hours decreased by nearly 9 percent from 32.6 million hours in 1982 to 29.9
million hours in 1999. (See fig. 1.)
Source: GAO's analysis of data from the National Transportation Safety
Board.
FAA has forecast continued growth for commercial aviation as well as for
general aviation into the next century. The number of planes will increase,
and these aircraft will fly more miles, spend more hours in the air, and
carry more people. For example, FAA estimates that commercial aviation
flight hours will grow to 24 million hours in 2007--an increase of 37
percent from 1999. In commercial aviation, FAA projects that the use of
large air carriers will grow at an annual rate of 4 percent, while the use
of commuter air carriers will grow at 3 percent per year. Although growth
has been more erratic in general aviation than in commercial aviation, FAA
projects an annual growth rate of 2.2 percent for general aviation into the
next century. FAA estimates that general aviation flight hours will increase
to about 36 million hours in 2007, a growth of nearly 19 percent over 1999.
Even with the growth in aviation, fatal accidents remain relatively rare,
especially in commercial aviation. Fatal accident rates for U.S. aviation
are low and have decreased over the past decades for both commercial and
general aviation. The fatal accident rate can be calculated as the number of
accidents with one or more fatalities divided by a measure of aviation
activity, such as the number of aircraft miles flown, aircraft hours flown,
or departures.
Commercial Aviation Accidents Can Be Catastrophic
In the 10-year period preceding the initiative, 4,471 fatal aviation
accidents occurred in the United States, resulting in a total of 9,802
deaths. Table 2 shows the distribution of accidents and deaths for
commercial aviation, which includes large and commuter air carriers, and
general aviation, which includes on-demand air taxis. General aviation
accounted for the largest number of fatal accidents and deaths in 1988-97.
The initiative addresses both commercial and general aviation, but increased
attention is focused on further improving the safety of commercial aviation
because large and commuter air carriers are the primary forms of air
transportation for most Americans. While fatal commercial aviation accidents
are rare, large airplane accidents can cause more deaths in an instant than
most events, other than wars or natural disasters. They consequently raise
concerns with both the public and the media, and commercial aviation is held
to a higher standard of safety than other forms of transportation. With
commercial aviation expected to grow steadily into the next century,
aviation accidents will occur with a frequency that will be unacceptable to
the public unless steps are taken to decrease the fatal accident rate. While
such accidents remain rare, FAA recognizes that the public demands a high
standard of safety and expects continued improvement.
Fatal accidents Deaths
Type of operation Number Percentage Number Percentage
Commercial aviation 85 2 1,756 18
General aviation 4,386 98 8,046 82
Total 4,471 100 9,802 100
Source: GAO's analysis of data from the National Transportation Safety
Board.
FAA tracks the number of passenger fatalities for various types of aviation
operations and calculates accident rates. Basically, the rates are
calculated by dividing the number of accidents with one or more fatalities
by one of the various measures of aviation activity. For example, the fatal
accident rate for commercial aviation for 1988-97 is 0.058 per 100,000
flight hours, which was calculated by dividing the number of fatal accidents
(85)8 by the number of flight hours (151 million). This translates into
about one fatal accident for every 2 million hours flown. The three activity
measures generally used to calculate fatal accident rates are the number of
individual flights (referred to as departures), aircraft miles flown, and
aircraft hours flown. Each activity measure reflects different exposures to
the risks associated with flying. For example, most commercial aviation
accidents occur during takeoff or landing, rather than during the cruise
phase, which constitutes the largest part of the total mileage and hours
flown. For this reason, we believe that departures are usually the best
measure of exposure to risk.
For large and commuter air carriers, all three fatal accident rates are
tracked. But for general aviation, the only measure of exposure is the
number of flight hours estimated from survey data. Thus, fatal accident
rates for commercial aviation (large and commuter air carriers) are
generally expressed in terms of the number of fatal accidents per 100,000
departures, while fatal accident rates for general aviation are expressed as
the number of fatal accidents per 100,000 flight hours as estimated by FAA's
annual survey.9 General aviation flight hours are not sufficiently reliable
for use in calculating a fatal accident rate for general aviation because
they are estimated from a voluntary survey, according to FAA.
Over the past few decades, the annual rate of fatal aviation accidents has
decreased significantly for both commercial and general aviation. While the
accident rates are low, they have shown little improvement recently. For
large commercial air carriers, the U.S. accident rate was 26 fatal accidents
per million departures in 1959. Following the advent of large jet aircraft
in the 1960s, the rate fell to one or fewer fatal accidents per million
departures and has remained fairly steady for three decades. The fatal
accident rate for commuter aircraft has also fallen over the last several
decades. The accident rate for commuter air carriers fell from about 2 fatal
accidents per million departures in 1982 to 3 per 10 million departures in
1996. While there were no fatal commuter accidents in 1998, the five fatal
accidents in 1999 resulted in a fatal accident rate of nine per million
departures. This increase in the fatal accident rate reflects a 1997
narrowing in the definition of commuter air carrier to include only small
aircraft with nine or fewer seats.10 Similarly, the accident rate for
general aviation aircraft has dropped since 1960. The fatal accident rate of
six per 100,000 flight hours in 1960 fell to less than two by the early
1980s. The fatal accident rate for general aviation continued to decrease
fairly steadily through the 1980s, increased slightly in the early 1990s,
and has dropped steadily since 1995. In 1999, the fatal accident rate for
general aviation was 1.2 fatal accidents per 100,000 flight hours. (See fig.
2.)
Source: GAO's analysis of NTSB data.
The reductions in the fatal accident rates resulted from a combination of
technological advances that improved safety. In commercial aviation, these
advances included the replacement of large, piston-engine aircraft with jet
aircraft with far more reliable engines, the development of navigational
equipment to warn pilots of impending crashes, better ground navigation
aids, improved aircraft instrumentation, and increased air traffic radar
coverage. Some of these improvements have also benefited smaller commuter
and general aviation aircraft. As commuter air carriers switched from small
aircraft to sophisticated turboprop aircraft, the accident rate among the
larger commuter aircraft became comparable to that of large air carriers.
the Fatal Accident Rate Must Be Reduced
If the current fatal accident rate holds steady and aviation activity grows
as FAA has projected, the increased air traffic will result in greater
numbers of crashes and fatalities. We estimate that the average of six fatal
commercial aviation accidents per year in 1994-96 will likely rise to nine
per year by 2007. Similarly, the fatal accidents for general aviation will
probably mount from an average of 380 in 1996-98 to 484 in 2007.11 Table 3
shows our projections of the number of fatal accidents in 2007 calculated
from FAA's growth estimates and the current fatal accident rate for each
type of aviation operation.
Number of fatal accidents
Type of operation Annual averagea Projected for 2007
Commercial aviation 6 9
General aviation 380 484
aThe annual average is for the baseline years chosen by FAA and Safer Skies
steering committees in establishing their goals: 1994-96 for commercial
aviation and 1996-98 for general aviation.
Source: GAO's analysis of data from FAA and NTSB.
The prospect of more accidents and deaths is unacceptable to the public,
FAA, and the aviation industry. Avoiding that outcome means reducing the
fatal accident rate significantly. The final report of the National Civil
Aviation Review Commission concluded in 1997 that the "anticipated growth in
aviation between now and the first quarter of the next century will almost
certainly lead to an occurrence of aviation accidents with a frequency that
will be wholly unacceptable to the public." The Commission called for a
joint industry-government effort to reduce the accident rate substantially.
Fatal Accident Rate
During the 1990s, FAA and aviation industry groups had separate and joint
efforts under way to use available data to identify and address the major
causes of accidents. A series of fatal crashes and concern that the number
of accidents and fatalities will increase as air traffic increases prompted
these efforts to reduce the accident rate. Many of the reports that resulted
from these efforts set specific goals and included recommendations for
decreasing aviation accidents. Although FAA and the aviation industry acted
on some of these recommendations, the fatal accident rate has remained
fairly stable but low.
The effectiveness of previous efforts to reduce the fatal accident rate is
believed to have been undercut by their limited scope and a lack of
coordination between government and industry groups. Many of the studies
issued during the 1990s were under the leadership of either FAA or a
particular segment of the aviation industry. For example, FAA, on its own,
studied controlled flight into terrain (CFIT)12 accidents and runway
incursions. Separately, the Flight Safety Foundation brought together
participants from many segments of the aviation industry to study CFIT and
approach and landing but initially had only limited FAA involvement. The
Aerospace Industries Association initiated an extensive study on the causes
of safety-related problems in aircraft engines, including uncontained engine
failure.13 (For a list of key aviation studies and our related reports, see
app. II.) According to FAA and industry officials we interviewed, efforts to
address specific safety issues were generally unsuccessful when one group
failed to coordinate its work with that of other groups that had important
roles in aviation safety.
Many of these reports issued during the 1990s set specific goals for
reducing the overall fatal accident rate or for addressing specific aviation
safety problems that result most often in fatalities. They also included
numerous specific recommendations to FAA and the aviation industry to help
meet these goals. Among the key reports were the following:
� In 1993, the Flight Safety Foundation led an international task force on
CFIT, the leading cause of fatal commercial aviation accidents worldwide.
The task force provided specific recommendations and training aids aimed at
reducing CFIT accidents. The task force set a goal of reducing these
accidents 50 percent worldwide by 1998 and other goals targeting
improvements in the regions of the world with the highest CFIT rates.14
� In January 1995, over 1,000 government, industry, and union officials
attended an FAA-sponsored safety conference. The officials agreed that they
shared responsibility for pursuing a goal of zero accidents. Their report
identified 173 high-priority safety initiatives in the areas of crew
training, air traffic control and weather, safety data collection and use,
applications of emerging technologies, aircraft maintenance procedures and
inspections, and development of flight operating procedures.15
� Following the May 1996 ValuJet crash, an FAA task force recommended in
September 1996 that FAA target agency resources to safety risks, improve the
certification and oversight of new air carriers, and address concerns about
inspector guidance and resources.16
� In February 1997, the White House Commission on Aviation Safety and
Security recommended that the government and the aviation industry establish
a national goal to reduce the aviation fatal accident rate by a factor of
five (meaning 80 percent) within 10 years. To achieve that goal, the
Commission made specific recommendations for reengineering FAA's regulatory
and certification programs.17 The Commission did not explicitly state
whether the national goal should apply to all types of aviation operations.
� In December 1997, the National Civil Aviation Review Commission
recommended that the government and the aviation industry work together to
achieve the White House Commission's goal of an 80-percent reduction in the
accident rate over the next 10 years and recommended specific safety
improvements for achieving that goal.18 While the Commission did not
explicitly state whether the 80-percent goal should apply to all types of
aviation operations, the Commission specifically discussed the accident
rates for large jets, commuter air carriers, general aviation operations,
and air taxis.
Both the White House Commission on Aviation Safety and Security and the
National Civil Aviation Review Commission called for FAA and the aviation
industry to work together on aviation safety issues.
Analysis to Address Safety Problems
On April 14, 1998, the Vice President, the Secretary of Transportation, and
the FAA Administrator announced the Safer Skies initiative, a new aviation
safety program committed to reducing the fatal accident rate by 80 percent
by 2007. Under the initiative, experts from FAA, the aviation industry, and
other government agencies with responsibility for aviation are to jointly
analyze U.S. and global data to identify the most serious threats to
aviation and to find the root causes of accidents. They will then determine
the best actions to break the chain of events that lead to accidents and
direct resources first to those actions. These actions are also referred to
as interventions.
to Join the Safer Skies Initiative
When FAA announced the Safer Skies initiative, the agency invited
participants from a number of ongoing industry and government safety groups
to join in creating a unified safety agenda. In establishing the agenda for
the initiative, the commercial and general aviation steering committees
joined with and expanded the preexisting efforts. To develop the unified
agenda, key government and industry aviation officials are to conduct data
analyses to identify the causes of fatal accidents and determine what
interventions are needed to prevent them.
Several of the preexisting safety groups were already using data-driven
approaches to study aviation safety issues. Representatives of air carriers,
aircraft and engine manufacturers, and related associations had established
a commercial aviation group in January 1997 to analyze fatal commercial
aviation accidents and to recommend ways to prevent them. Before joining the
initiative, this group had outlined a process for obtaining accident data
from U.S. and international sources and for reaching consensus on the safety
problems to be addressed. Another industry group analyzing data on
uncontained engine failure had developed a process for analyzing safety
data, using case studies to identify root causes, and evaluating the
feasibility of proposed interventions. A third group that represented a
cross-section of various general aviation constituencies, such as pilots and
small aircraft manufacturers, was addressing the causes of fatal general
aviation accidents.19 A joint government-industry group sponsored by FAA was
continuing work on issues pertaining to the safety of passengers and crew
members in the aircraft cabin that had been started separately by FAA,
industry associations, and unions representing flight attendants.20 FAA
invited members from all four of these groups to participate in the
initiative.
Problems for the Safer Skies Initiative to Address
Safer Skies formed steering committees of safety experts from government and
industry to lead the work in each of its three agenda areas: commercial
aviation, general aviation, and cabin safety. Each steering committee has
co-chairs and participants from both FAA and industry. The commercial and
general aviation steering committees used available data to select the
safety problems to be addressed in their respective agenda areas. In
contrast, the cabin safety steering committee continued the work on safety
problems that had already been under way as a joint FAA-industry effort that
preceded Safer Skies. The three Safer Skies steering committees ultimately
chose to address 16 safety problems: 6 in commercial aviation, 6 in general
aviation, and 4 in cabin safety. The commercial aviation and general
aviation steering committees selected several of the same safety problems,
including weather and loss of control over the aircraft. Because safety
problems can affect large and small aircraft differently, the commercial and
general aviation steering committees planned to have separate teams study
each safety problem with one exception. A joint team will study runway
incursions because commercial and general aviation aircraft often share the
same runways and accidents have occurred involving both types of aircraft.
Table 4 lists and briefly explains each safety problem.
Problem identified Definition of problem
Commercial aviation
Controlled flight into Flying an otherwise controllable aircraft
terrain into terrain.
Accidents resulting from situations in which
Loss of control the pilot should have maintained or regained
aircraft control but did not.
Small cracks in high-speed rotating parts
Uncontained engine failure that, if left undetected, can propagate until
failure.
An occurrence at a towered or nontowered
airport, involving an aircraft, vehicle, or
Runway incursion pedestrian within the runway safety area,
that creates a real or potential collision
hazard with an aircraft taking off, intending
to take off, or landing or intending to land.
Approach and landing Situations in which a pilot may have been
able to land an aircraft safely, but did not.
Meteorological conditions (icing, turbulence,
Weather etc.) that adversely affect aircraft
performance.
General aviation
Controlled flight into Flying an otherwise controllable aircraft
terrain into terrain.
Accidents resulting from situations in which
Loss of control the pilot should have maintained or regained
aircraft control but did not.
An occurrence at a towered or nontowered
airport, involving an aircraft, vehicle, or
Runway incursion pedestrian within the runway safety area,
that creates a real or potential collision
hazard with an aircraft taking off, intending
to take off, or landing or intending to land.
Often a fundamental element in accident
Aeronautical decision-making causal chains, where a pilot did not make the
best safety decision about a flying or
nonflying situation.
Meteorological conditions (icing, turbulence,
Weather etc.) that adversely affect aircraft
performance.
Survivability Safety research and initiatives that would
reduce fatalities.
Cabin safety
Passenger interference Passengers who hinder crew members in
performing their duties.
Injuries sustained by passengers who are not
Passenger seat belt use wearing their seat belts when encountering
unexpected air turbulence.
Carry-on baggage Articles brought into the airplane cabin by
passengers.
Safety issues associated with the commercial
Child safety restraints aviation transportation of children under the
age of 2 years old.
Source: FAA.
At the request of the Chairman and Ranking Democratic Member of the
Subcommittee on Aviation, House Committee on Transportation and
Infrastructure, we reviewed the design and implementation of the Safer Skies
initiative. Specifically, they asked us to determine (1) to what extent
addressing the safety problems selected by the Safer Skies initiative will
help reduce the fatal accident rate; (2) what progress the initiative has
made in identifying and implementing interventions to address each of these
safety problems; (3) what progress the Safer Skies initiative has made in
assessing the effectiveness of those interventions; and (4) how FAA is
coordinating the Safer Skies initiative with other safety activities
conducted throughout the agency, in partnership with the aviation industry,
and by other federal agencies.
Because Safer Skies is a 10-year project that hopes to reach its goals in
2007, we analyzed domestic flight operations and accident data for the
decade that preceded the 1998 announcement of Safer Skies and for the decade
to come. We examined data on fatal accidents and their causes for all types
of aviation operations in the United States from 1988 through 1997. We also
examined projected data for aviation operations and accidents through 2007.
To determine whether addressing the safety problems chosen by Safer Skies
will help reduce the fatal accident rate, we interviewed FAA officials
responsible for overseeing Safer Skies, officials at the Department of
Defense and the National Aeronautics and Space Administration involved in
aviation safety, and the chairs and many members of the steering committees
for commercial aviation, general aviation, and cabin safety. We reviewed
documents related to each of these steering committees as well as data used
by these groups in choosing the problems on which Safer Skies would focus.
We also discussed the problems Safer Skies selected as priorities with staff
at the National Transportation Safety Board, the Flight Safety Foundation,
and other aviation safety groups.
To determine what progress has been made in identifying, developing, and
implementing intervention strategies for the Safer Skies initiative, we
interviewed the FAA and industry chairs of the teams formed to address each
problem under study. We obtained and reviewed team reports completed for
each safety problem to understand the analysis process, modifications made
to it by successive work groups, and actions planned to improve aviation
safety in each problem area.
To determine what progress has been made to date in assessing the
effectiveness of its actions to improve aviation safety, we reviewed
implementation plans to determine whether schedules were being met and
whether ways had been chosen to measure the success of such actions. We also
reviewed available team reports and relevant data to determine whether
sufficient data were available to measure Safer Skies' progress in improving
aviation safety.
To determine how FAA coordinated the Safer Skies initiative with safety
activities conducted throughout FAA and in partnership with the aviation
industry, we reviewed information on related industry and government safety
activities. Specifically, we sought information on activities under the
auspices of FAA, the Department of Defense, the National Aeronautics and
Space Administration, the National Transportation Safety Board, selected
engine and aircraft manufacturers, several major air carriers, the Air
Transport Association, and the Aircraft Owners and Pilots Association.
During our interviews with members of the Safer Skies steering committees
and teams, we discussed efforts to coordinate their work with other
government and industry safety activities. We also reviewed the reports from
each Safer Skies team for safety problems where coordination would be
appropriate. We discussed the budgetary implications of the Safer Skies
initiatives and the criteria for prioritizing resources with FAA officials
and steering committee members.
We conducted our work from August 1999 through June 2000 in accordance with
generally accepted government auditing standards.
The Safer Skies Initiative Should Help Improve Aviation Safety
Addressing the 16 safety problems chosen by the Safer Skies initiative
should help reduce the nation's fatal accident rate. In commercial aviation,
eliminating the six safety problems to be addressed by the initiative would
approach the 80-percent goal. Other FAA initiatives are addressing
additional safety problems in commercial aviation, which should complement
efforts under the Safer Skies initiative. In general aviation, the
initiative will address six problems that appear to be among the most common
causes of fatal accidents for this type of operation, according to available
accident data. While the initiative has adopted the 80-percent goal in
commercial aviation, which transports most passengers who fly in the United
States, the initiative adopted a less aggressive goal for general aviation,
which accounted for the vast majority of the fatal aviation accidents. The
goal in general aviation is to reduce the number of fatal accidents to 350
in 2007, which represents about a 20-percent reduction. Finally, the
initiative addressed four problems in cabin safety. Improving cabin safety
will have little impact on lowering the fatal accident rate because cabin
safety accounted for only two U.S. fatalities in commercial aviation in
1988-97. No quantitative goal was set for safety improvements in cabin
safety. To date, safety improvement efforts by FAA and the Safer Skies
initiative have focused on past accidents and incidents, which may not be
entirely predictive of future ones. Studying growth and technological
changes in the aviation industry can help anticipate and prevent the safety
problems and accidents that are likely to arise from such changes.
Preliminary international efforts have been initiated to address future
hazards, and coordinating these efforts with Safer Skies work could enhance
the initiative's efforts to reduce the fatal accident rate.
Commercial Aviation
The Safer Skies initiative plans to address six safety problems that
accounted for 79 percent of the fatal accidents in commercial aviation in
1988-97. If past accident causes continue, completely eliminating these six
safety problems might approach the 80-percent goal. FAA also has safety
initiatives under way to address several of the safety problems in
commercial aviation not addressed by the initiative. These include sabotage,
fuel tank explosions, and structural problems. In combination with the Safer
Skies initiative, FAA's safety initiatives have potential for reducing the
fatal accident rate in commercial aviation. For commercial aviation, the
Safer Skies initiative has established a goal of reducing the fatal accident
rate by 80 percent in 2007 in accordance with the goal envisioned by the
White House and congressional commissions on aviation safety.
The Safer Skies initiative will address six safety problems that accounted
for 79 percent of the fatal commercial aviation accidents in 1988-97. Three
of these safety problems were major ones both worldwide and in the United
States: pilots' losing control of their aircraft, pilots' flying otherwise
controllable aircraft into the ground or water (CFIT), and accidents during
approach and landing. These three safety problems accounted for 58 of the 85
fatal accidents in U.S. commercial aviation during this period. The
commercial aviation teams are examining 34 of these accidents, which
involved larger aircraft. The commercial aviation steering committee
referred the remaining 24 fatal accidents to the general aviation steering
committee for review because they involved small commuter aircraft with nine
or fewer seats that operated scheduled commercial service. This was done
because (1) the aircraft involved are more similar to general aviation
aircraft than to larger commercial aircraft, (2) the types of operating
environments and safety problems that caused the accidents more closely
resemble those of general aviation than those of commercial aviation, and
(3) the interventions to address safety problems in general aviation are
more likely to correct these safety problems than interventions designed for
large commercial aircraft. We reviewed the National Transportation Safety
Board's (NTSB) reports for the 24 small commuter accidents and found that
most of the accidents happened in Alaska when pilots flew into mountains
after deteriorating weather reduced visibility. On the basis of our review,
we concur with the commercial aviation steering committee's assessment that
these accidents more closely resemble general aviation accidents and would
likely benefit from the interventions that emerge to address these safety
problems in general aviation aircraft.
The potential for improving safety in these smaller commuter aircraft exists
with a number of the interventions proposed by the general aviation teams
working on weather and CFIT. It is unclear whether the initiative or FAA has
mechanisms in place to ensure that small commuter operators will benefit
from the interventions developed. For example, many of the interventions
involve providing additional training to pilots on weather conditions and
assessing the risk factors associated with each flight. Because the
initiative plans to deliver much of this training jointly with the Aircraft
Owners and Pilots Association, it is essential that notification about this
training also be provided to small commuter operators and pilots who could
benefit from this training but may not be members. Although members of
several other organizations participate in the general aviation steering
committee and study teams,21 neither the initiative nor FAA has made
specific provisions to ensure that such interventions are also directed at
small commuter aircraft operators and pilots, as well as at general aviation
pilots. Because small commuter accidents accounted for 28 percent of the 85
fatal accidents in commercial aviation in 1988-97, reducing the fatal
accident rate by 80 percent depends on addressing these safety problems in
small commuter aircraft, as well as in large commercial aircraft.
To further reduce the fatal accident rate for commercial aviation, the
initiative will address three additional safety problems that have resulted
in fewer fatal accidents in the United States from 1988 through 1997. The
steering committee chose runway incursions, uncontained engine failure, and
weather, each of which resulted in from two to four fatal accidents. These
three safety problems accounted for an additional 9 accidents, or 11 percent
of the 85 fatal accidents. The committee selected these problems because
they caused past fatal accidents or serious incidents that could have cost
many lives. These areas were also included because each occurred with
greater frequency in the United States than worldwide and because FAA or the
aviation industry had already begun work on these safety problems. (See fig.
3.)
Source: FAA.
Our analysis of aviation data and review of safety reports confirmed that
the initiative is addressing three major safety problems that caused fatal
accidents in commercial aviation, as well three other safety problems that
have the potential to cause accidents with large numbers of fatalities.
Reducing or eliminating safety problems resulting from CFIT, loss of
control, approach and landing, runway incursions, weather, and uncontained
engine failure should help lower the fatal accident rate. Safer Skies
participants, FAA officials, and industry aviation experts whom we
interviewed also believe that the initiative is addressing the most
important aviation safety problems. Most of these aviation experts indicated
strong support for addressing such major safety concerns as CFIT, approach
and landing, and loss of control. Furthermore, because of the increasingly
global nature of commercial aviation, addressing these safety problems means
that many of the interventions recommended by the initiative might have
applicability worldwide, as well as in the United States. Many aviation
experts we interviewed also supported the inclusion of safety problems with
fewer fatalities but with a high potential for fatalities, such as runway
incursions and uncontained engine failure. They agreed that reducing or
eliminating these safety problems should help reduce the fatal accident
rate.
Accident Rate
In addition to successfully addressing the major safety problems discussed
above, addressing additional safety problems could further reduce the fatal
accident rate in commercial aviation. FAA has a number of aviation safety
initiatives under way that potentially can contribute to improvements in the
safety of smaller commuter aircraft sometimes used in commercial aviation.
For example, FAA's Capstone Project focuses on improving general aviation
safety in Alaska by providing additional navigational aids but also has
potential application for addressing the safety problems of small commuter
aircraft used elsewhere. FAA also has ongoing initiatives to address the
causes of 4 of the 18 commercial aviation accidents not being addressed by
Safer Skies teams. These include programs overseen by the agency's Office of
Civil Aviation Security to reduce the threat of sabotage, hijacking, and the
transportation of hazardous cargo. Other FAA initiatives are under way to
address the structural problems of aging aircraft and fuel tank explosions.
FAA has, for example, published a notice proposing requirements for design
reviews and mandatory maintenance actions for fuel tank systems on large
transport aircraft.
Of the remaining fatal accidents in commercial aviation that the initiative
is not addressing, 12 were on-ground fatalities, and 2 resulted from other
causes.22 The on-ground accidents each involved the death of a single worker
or unauthorized individual at the airport. Most of these accidents occurred
near the boarding gate or ramp. For example, several employees were fatally
injured when struck by an aircraft's propeller or nose gear during the
course of their work. Of the on-ground fatalities, two resulted from
individuals gaining unauthorized access to airport areas that should have
been secured, nine involved various airline or airport employees who
sustained injuries in the workplace, and one involved a passenger who fell
out of an aircraft catering door and onto the ground. Because on-ground
accidents accounted for 14 percent of the 85 fatal accidents in commercial
aviation in 1988-97, reducing the fatal accident rate by 80 percent by 2007
will be difficult if these safety problems are not addressed.
FAA has initiatives to address some of the safety problems that caused
on-ground fatalities, but it is unclear how systematically these problems
are being addressed. Specifically, FAA's Office of Civil Aviation Security
oversees airline and airport programs to limit access to secure areas to
authorized individuals. The status of FAA's efforts to address workplace
safety issues that resulted in on-ground fatalities is less clear. FAA is
responsible for regulating the safety and health aspects of the work
environment of aircraft crew members when the aircraft is in operation.
However, FAA has not promulgated specific regulations that address all
employee safety and health issues associated with working conditions on
aircraft. FAA held a public meeting in December 1999 to gather information
on issues associated with working conditions on and around aircraft and to
determine whether additional regulations should be proposed. However, FAA
does not currently have a group addressing workplace safety issues and could
not identify any regulations, guidance, or other initiatives that have been
developed to address the types of workplace safety problems that caused most
of the on-ground fatalities.
Considering More Than Reducing the Fatal Accident Rate
Looking at the number of fatalities associated with various safety problems,
as well as their contribution to the fatal accident rate, provides
additional perspective on Safer Skies' commercial aviation agenda.
Reductions in the fatal accident rate are closely linked to reductions in
the number of fatal accidents. Following this logic, the greatest reductions
in the fatal accident rate can be achieved by eliminating the safety
problems that caused the greatest number of accidents with one or more
fatalities. However, strict adherence to the goal of reducing the fatal
accident rate could result in focusing attention and resources on the causes
of accidents that resulted in single fatalities, rather than on those causes
that result in multiple fatalities, as well as multiple accidents. In
choosing which safety problems to address, the commercial aviation steering
committee selected safety problems that will help reduce fatalities, as well
as the fatal accident rate.
The fatal accident rate in commercial aviation can most quickly be reduced
by addressing the three safety problems that form the core of the Safer
Skies agenda in commercial aviation: CFIT, loss of control, and approach and
landing. These three safety problems accounted for 34 fatal accidents
involving larger aircraft that commercial aviation teams are handling and 24
additional small commuter accidents that general aviation teams are
handling. (See table 5.) If the initiative is successful in developing and
implementing interventions to eliminate these three safety problems for both
large aircraft and small commuter aircraft, it would make progress toward
preventing the kinds of safety problems that caused 68 percent of the fatal
accidents in 1988-97. If the initiative could successfully eliminate all six
safety problems on its agenda for commercial aviation, it would approach the
goal of an 80-percent reduction in the fatal accident rate. However, other
safety problems actually resulted in more fatal accidents and thus could
reduce the fatal accident rate more quickly if eliminated.
Fatal accidents Fatalities
Number Percentagea Number Percentagea
Commercial aviation accidents
to be addressed by Safer
Skies commercial aviation
teams
Loss of control 19 22.4 530 30.2
Approach and landing 9 10.6 128 7.3
CFIT 6 7.1 339 19.3
Runway incursion 4 4.7 25 1.4
Weather 3 3.5 16 0.9
Uncontained engine failure 2 2.4 113 6.4
(Subtotal) 43 50.6 1,151 65.5
Commercial aviation accidents
involving small commuter
aircraft to be addressed by
Safer Skies general aviation
teams
CFIT 16 18.8 38 2.2
Loss of control 7 8.2 37 2.1
Approach and landing 1 1.2 5 0.3
(Subtotal) 24 28.2 80 4.6
Commercial aviation accidents
to be addressed by other FAA
safety initiatives
Structural problems 2 2.4 10 0.6
Fuel tank explosion 1 1.2 230 13.1
Sabotage 1 1.2 270 15.4
(subtotal) 4 4.8 510 29.1
Gate/ramp 12 14.1 12 0.7
Other 2 2.4 3 0.2
Total 85 100 1,756 100
aTotals do not add to 100 because of rounding.
Source: GAO's analysis of data from FAA and NTSB.
The initiative could approach the 80-percent goal more quickly by
eliminating on-ground accidents, which caused more fatal accidents in
commercial aviation than all other safety problems except loss of control
and CFIT. On-ground accidents caused 12 fatal accidents in commercial
aviation--14 percent of the total. While the safety problems that caused
on-ground accidents merit addressing, the safety problems that Safer Skies'
commercial aviation team has chosen to address resulted in multiple fatal
accidents and many more fatalities. For this reason, the initiative will
probably have more impact on improving the safety of air transportation for
the majority of the nation's passengers than addressing other safety
problems, such as on-ground accidents, whose elimination could reduce the
fatal accident rate more but would save fewer lives.
While focusing on reducing the fatal accident rate by addressing the safety
problems that caused the most commercial aviation accidents, the approach
taken by the initiative also resulted in choices that recognized where the
greatest number of fatalities have occurred or could occur. The three major
problems addressed by the initiative's commercial aviation teams (CFIT, loss
of control, and approach and landing) accounted for 57 percent of the 1,756
fatalities in 1988-97. This rises to 66 percent when all six safety problems
on the commercial aviation agenda are considered. The additional small
commuter accidents that are to be addressed by general aviation teams
account for another 5 percent of the total fatalities. The only other safety
problems that resulted in hundreds of fatalities were sabotage and fuel tank
explosions. The initiative did not focus on these problems for two reasons.
First, only one fatal accident resulted from each of these safety problems
in 1988-97. Second, FAA already has initiatives under way to address both
sabotage and fuel tank explosions. The initiative did include two safety
problems on its commercial aviation agenda that each accounted for only
about 1 percent of the fatalities in U. S. commercial aviation during this
period. However, the initiative recognized the potential of runway
incursions, which accounted for 25 U.S. fatalities, to result in hundreds of
fatalities. While weather resulted in few commercial aviation accidents and
16 fatalities, the commercial aviation steering committee members felt that
the problems of turbulence and icing merited attention. In contrast, the 12
on-ground accidents each resulted in a single fatality that together
accounted for fewer than 1 percent of the nation's commercial aviation
fatalities. Eliminating the safety problems that caused on-ground fatalities
could reduce the fatal accident rate more quickly than eliminating either
CFIT or approach and landing accidents that involved large commercial
aircraft. The commercial aviation steering committee has selected safety
problems that will help reduce fatalities, as well as the fatal accident
rate. (See fig. 4.)
Source: FAA.
Commercial Aviation
The Safer Skies initiative and FAA have adopted the goal of reducing the
fatal accident rate for commercial aviation by 80 percent by 2007.
Specifically, the goal is to reduce the fatal accident rate for commercial
aviation from a 1994-96 baseline of 0.037 fatal accidents per 100,000 flight
hours to 0.007 fatal accidents per 100,000 flight hours in 2007. The meaning
of this goal can be more readily understood by considering the current
number of fatal commercial aviation accidents and the number of accidents
projected for 2007 if further safety improvements are not undertaken. In
1994-96, the United States averaged six fatal commercial aviation accidents
each year. Given the projected growth of commercial aviation, we estimate
that this number could increase to nine in 2007 if safety is not improved.
If the initiative achieves the goal of an 80-percent reduction in the fatal
accident rate for commercial aviation, we estimate that the number of fatal
accidents expected in 2007 would drop to two. (See table 6.)
Number of fatal accidents
Type of Annual Projected Safer Skies 80-percent
operation averagea for 2007b goal for 2007c reductiond
Commercial
aviation 6 9 2 2
General aviation380 484 350 97
Total 386 493 352 99
aThe annual average is for the baseline years chosen by FAA and Safer Skies
steering committees in establishing their goals: 1994-96 for commercial
aviation and 1996-98 for general aviation.
b To project the number of fatal accidents likely in 2007, we used FAA's
most recent projected growth rates for flight hours in commercial aviation
and general aviation, which were released in March 2000. The updated growth
rate for general aviation was larger than the earlier growth rate available
to the general aviation steering committee, which projected 437 fatal
accidents in general aviation for 2007.
cGiven FAA's most recent projected growth rates, the Safer Skies goal for
commercial aviation of an 80-percent reduction in the fatal accident rate
would likely result in two fatal accidents in 2007. The general aviation
steering committee set a goal of reducing the number of fatal general
aviation accidents to 350 in 2007.
dFor commercial aviation, we computed the number of fatal accidents that
would result from decreasing the fatal accident rate in 2007 by 80 percent.
For general aviation, we computed the number of fatal accidents that would
result from decreasing by 80 percent the number of such accidents projected
for 2007.
Source: GAO's analysis of data from FAA and NTSB.
Problems That Caused Many Fatal Accidents in General Aviation
The general aviation steering committee used available accident data, safety
reports, and professional expertise in aviation to identify safety problems
that caused many of the fatal accidents in general aviation. The six safety
problems chosen were controlled flight into terrain, loss of control,
aeronautical decision-making, runway incursions, weather, and survivability.
Steering committee members told us that they selected these safety problems
after reviewing the available data on general aviation accidents and past
industry and government-sponsored safety reports on general aviation. They
said that the NTSB accident reports were challenging to analyze because many
lacked the detail needed to determine the root causes of accidents. They
noted, for example, that most general aviation aircraft are not equipped
with such key equipment as flight data recorders that would help identify
the safety problems that caused the accidents. To meet these additional
challenges, FAA developed a training course tailored to the needs of those
responsible for analyzing general aviation accidents. Both FAA and industry
members attended this course before starting the analysis phase. The general
aviation analysis reports on CFIT and weather also included recommendations
to address problems with the quality of the data on general aviation
accidents. In response to these recommendations, the general aviation
steering committee chartered a team in April 2000 to develop strategies to
(1) provide increased detail about factors that have contributed to or
caused general aviation accidents and incidents and (2) improve the quality
and timeliness of estimates of general aviation activity. Members of the
steering committee told us that, in cases where the safety problems that
caused the fatal accidents were unclear, they used their experience as
either pilots or experts in general aviation to determine the possible
causal factors involved in the accidents.
Members of the steering committee also examined past industry and government
reports on the causes of general aviation accidents. One key report was the
Nall Report, a report on general aviation accident trends and factors
published annually by the Aircraft Owners and Pilots Association's Air
Safety Foundation. According to the 1998 Nall Report, the major causes of
fatal general aviation accidents were weather, loss of control or other
errors during flights in which the pilot was maneuvering the plane, and
accidents on approach to the airport. Another key report was FAA's study of
the causes of general aviation CFIT accidents.23 FAA's study concluded that
CFIT accidents accounted for 17 percent of the general aviation fatalities
and 32 percent of general aviation accidents in weather conditions requiring
pilots to have instrument ratings to fly.
Steering committee members also told us that several reports indicated
growing problems with runway incursions involving general aviation aircraft.
For example, a study by DOT's Office of Inspector General showed that
general aviation pilots caused the majority of runway incursions
attributable to pilot error in 1990-96.24 Members of the steering committee
told us that they also decided to address survivability in an effort to
decrease the number of fatalities among those who survive the impact of a
crash but ultimately die from their injuries.
Although the data available on general aviation accidents are less detailed
than those available on commercial aviation accidents, the general aviation
problems the initiative plans to address represent reasonable choices. Most
of the safety problems chosen have been identified in past safety reports
and NTSB accident reports as major causes of fatal accidents in general
aviation. These include weather, loss of control, CFIT, and runway
incursions. Aeronautical decision-making has also been cited repeatedly as a
factor in such safety problems as weather, when pilots exercise judgment
about whether to depart or turn back when faced with potential danger. In
addition, aeronautical decision-making includes those decisions made
relating to aircraft maintenance. Most of the Safer Skies participants, FAA
officials, and aviation experts we interviewed concurred that the six
general aviation safety problems to be addressed by the initiative are
reasonable ones that will help to reduce the fatal accident rate.
Fatal Accidents in General Aviation to 350 in 2007
Although both the White House and congressional commissions on aviation
safety called for an 80-percent reduction in the nation's fatal accident
rate, FAA and the Safer Skies initiative applied this goal only to
commercial aviation and adopted a less aggressive accident reduction goal
for general aviation. The goal is to reduce the number of fatal general
aviation accidents to 350 in 2007. This represents a 20-percent reduction in
the number of fatal accidents that would likely result from projected growth
in general aviation. Because general aviation accounted for 98 percent of
U.S. fatal accidents in 1988-97, the goal of an 80-percent reduction in the
nation's fatal accident rate set forth by the two major aviation commissions
is unreachable if these fatal accidents are not greatly reduced. The
congressionally mandated commission on aviation safety discussed the fatal
accident rates for all kinds of aviation operations, including general
aviation. Because this commission did not explicitly apply the 80-percent
goal to general aviation, it remains unclear whether it intended the goal to
apply to general aviation as well as commercial aviation. The goal
adopted--350 fatal accidents--contrasts sharply with the 97 fatal accidents
that would likely result if the 80-percent goal were achieved. (See table
6.)
The steering committee did not adopt the 80-percent goal for general
aviation because of strong objections from the general aviation community.
Representatives of the general aviation community argued that, given the
varied experience levels of its pilots, reducing fatal accidents by 80
percent would be impossible without grounding the fleet. One general
aviation representative said that there was a prevailing concern in the
general aviation community that any agreement on a solid goal would lead to
more regulation and less growth. In addition, these representatives objected
to establishing a goal that involved a fatal accident rate. The fatal
accident rate for general aviation is calculated by dividing the number of
fatal accidents by the number of flight hours. Data on general aviation
flight hours are estimated using an annual survey of general aviation
operators conducted by FAA. Response to the survey is voluntary. Because the
flight hours are estimated on the basis of this survey, representatives of
the general aviation community questioned the reliability of these data and
expressed concern about using flight hours to calculate past and future
fatal accident rates. As a result, the Safer Skies steering committee for
general aviation agreed not to use the survey data on flight hours to
calculate a fatal accident rate until the data are more reliable. Instead,
the accident reduction goal for general aviation was expressed in terms of
the number of fatal accidents, rather than the fatal accident rate.
To set its goal of reducing fatal accidents to 350, the general aviation
steering committee reviewed available data on fatal accidents. The steering
committee found the number had declined fairly steadily since 1990 in
response to past initiatives to improve safety. The data used by the
steering committee showed that, in 1996-98, an average of 379 fatal general
aviation accidents occurred each year.25 The steering committee used this
average and the 1.6-percent annual growth expected in general aviation to
project that 437 accidents would occur in 2007 if additional safety
initiatives were not undertaken. They agreed that a reduction to 350 fatal
accidents would be achievable. This represents a 20-percent reduction in the
number of fatal accidents that they estimated would occur without additional
safety initiatives (437). According to projections by the steering committee
and the general aviation community, a reduction of this magnitude would
prevent 363 accidents from 2000 through 2007.
The goal of reducing the number of fatal accidents to 350 in 2007 is
probably achievable, but this goal is not likely to push the general
aviation community toward more safety improvement as aggressively as it
could. We believe that this goal is achievable for two reasons. First,
although the level of general aviation activity has increased, the number of
fatal accidents decreased to 354 in 1999, a decrease of 17 percent since
1994. Both FAA and industry officials attributed this decrease in part to
ongoing safety initiatives. The goal of 379 accidents established for each
of the next 3 years represents a 7-percent growth in the current number of
fatal accidents. Second, the goal of 350 accidents set for 2007 is only 4
fewer fatal accidents than occurred last year. Hence, the long-term goal is
achievable if the general aviation community is able to hold its number of
fatal accidents steady as its air traffic grows by an expected 2.2 percent
per year in the coming decade.
We recognize that an 80-percent reduction in fatal accidents is probably not
achievable in general aviation at this time because of the diversity in
pilots' experience levels, aircraft types, and operating environments.
However, we believe that a more aggressive goal would encourage greater
efforts by general aviation operators, manufacturers, associations, and FAA
to make safety improvements in general aviation operations that could save
lives.
Little Impact on Lowering the Fatal Accident Rate
Improving cabin safety is unlikely to have much impact on reducing the
overall fatal accident rate. In contrast to the safety problems addressed by
the commercial and general aviation steering committees, the safety problems
addressed by the cabin safety steering committee have not resulted in
numerous fatalities, and few data are available on any injuries that result
from these problems. The Safer Skies initiative identified only two
fatalities in U.S. commercial aviation in 1988-97 related to cabin safety
problems.26 While passengers and crew have been injured in the cabin
environment, few data exist on these incidents because air carriers are not
required to report such incidents unless they involve a serious injury or
fatality. The study of cabin safety problems thus relies more on information
shared by flight attendants and air carriers than on analysis of the limited
data available. Because cabin safety resulted in few fatalities and affords
few data for analysis, it is arguable whether it was appropriate to include
cabin safety issues in an initiative directed at reducing the fatal accident
rate through a data-driven analysis of safety problems.
Although not appropriate for Safer Skies' focus on the safety problems that
caused fatal aviation accidents, cabin safety issues are an appropriate
topic for FAA to address jointly with the aviation industry. NTSB and flight
crews have raised concerns about the potential for injuries and fatalities
in the cabin. FAA and industry were jointly studying cabin safety problems
before the initiative was announced. The safety problems under study
included those involving child restraint systems, passenger seatbelt use,
passenger interference with crew, and carry-on baggage. Concerns about these
safety problems are not new. For example, NTSB has long advocated FAA's
requiring the use of child restraints for passengers under the age of 2.
NTSB was concerned enough about the use of child restraints to launch a
campaign aimed at making parents aware of the benefit of putting children in
approved child restraint systems and to declare 1999 as the "year of child
transportation safety." Similarly, representatives of air carrier crews have
expressed concern that the incidents of passengers interfering with crew
members are increasing.
Help Anticipate and Prevent Fatal Accidents
In December 1997, the congressionally mandated commission on aviation safety
recommended that FAA and the aviation industry jointly develop a strategic
plan to improve aviation safety and that the process "begin with analysis of
both previous and potential failures to meet safety expectations." These
failures include accidents, incidents, insight from flight operational data,
and aviation system changes. The analysis of the causes of past accidents
provides insights into safety problems that exist within the current
aviation system, while the analysis of aviation system changes can help
anticipate future hazards that may arise from such changes as growth and
technological advances (e.g., vertical takeoff and landing by aircraft). The
approaches to the analysis of past safety problems and future hazards are
distinct. A data-driven approach is particularly useful for analyzing the
safety problems that caused past fatal accidents. Data on nonfatal accidents
and incidents can also be used to identify and address safety problems that
did not result in fatalities but could have. The data-driven approach is
based on the assumption that identifying a problem is possible where
historical data are available. While this approach can be used to address
the safety problems in the current operating environment, other types of
analyses may be more useful for anticipating and preventing the safety
problems that could result in new types of fatal accidents. For example, the
anticipated growth in air traffic will lead to more congestion around
airports, increasing the possibility of runway incursions and midair
collisions near airports. Anticipating how changes in the aviation industry
may increase existing safety problems or bring about new ones can better
position both FAA and the aviation industry to prevent accidents.
While FAA, Safer Skies, and industry groups have made progress in the
analysis of the causes of past accidents and incidents, efforts to analyze
and anticipate future hazards are more preliminary. The Joint Safety
Strategy Initiative in Europe27 has formed a work group to develop a method
for examining future hazards. A number of FAA staff participate in this work
group, which should facilitate the cooperative exchange of ideas and
information on this topic. As of April 2000, the Safer Skies initiative had
not established a process for analyzing future hazards. A systematic
analysis of the changes occurring in the aviation industry could enhance
Safer Skies' ongoing efforts to reduce the fatal accident rate. Several of
the aviation experts interviewed suggested that the initiative could benefit
from going beyond the analysis of data on past accidents to consider safety
problems that may arise from rapid changes in the aviation operating
environment. Participants on Safer Skies' commercial aviation steering
committee also indicated that while data-driven approaches are helpful, it
is also important to consider future hazards. FAA's Director of the Aircraft
Certification Service28 said that the initiative's first priority was to
understand and eliminate the safety problems that caused past accidents but
that the commercial aviation steering committee also plans to address future
hazards and recently added this topic to its agenda for consideration.
Because work on future hazards could help anticipate and prevent fatal
accidents, this topic is important for the Safer Skies steering committees
to address, especially as it applies to commercial aviation. Coordinating
this effort with the work initiated by European and FAA staff on future
hazards should help avoid duplication of effort and foster awareness of and
solutions to these potential problems internationally.
The premise of both the White House and congressional commissions on
aviation safety was that data on past and possible future causes of
accidents could be used to focus resources on substantially reducing the
fatal accident rate. While the Safer Skies initiative has made significant
strides, it has not yet carried out this mandate as fully as it could. The
six safety problems that the initiative is addressing accounted for almost
80 percent of the fatal accidents in commercial aviation in 1988-97. Our
review showed that the initiative and FAA have work under way to address
these and other safety problems in commercial aviation. However, the
initiative has not challenged all sectors of the aviation community to push
aggressively for safety improvements. Although the initiative has adopted
the challenging goal of reducing the fatal accident rate for commercial
aviation by 80 percent by 2007, general aviation is not being asked to set a
similarly challenging goal. While an 80-percent reduction in fatal accidents
is probably not achievable in general aviation at this time, the goal
adopted by the initiative does not push the general aviation community
toward implementing the kinds of interventions that could substantially
lower the fatal accident rate. A more rigorous goal would encourage greater
efforts by general aviation operators, manufacturers, associations, and FAA
to make needed safety improvements. In addition, many of the interventions
developed to improve general aviation safety could also benefit small
commuter operators and pilots, but this benefit will not be realized without
a systematic way of ensuring that training and other interventions are also
directed at small commercial aviation operations. Finally, the Safer Skies
initiative and most aviation safety studies to date have focused on the
causes of past accidents. While analyses of accident data are useful for
determining the causes of past accidents, reducing fatal accidents during a
period of rapid growth in aviation will probably require the analysis of the
changing aviation environment to anticipate future safety problems.
Preliminary international efforts have been initiated to consider future
hazards, and integrating these efforts with Safer Skies' work would enhance
the initiative's efforts to reduce the fatal accident rate.
To further reduce the nation's fatal accident rate and save lives in the
type of aviation operation that causes the most fatal accidents and
fatalities, we recommend that the Secretary of Transportation direct the FAA
Administrator to work with the general aviation community to
� set a more challenging goal for reducing the number of fatal general
aviation accidents by 2007,
� set interim goals to assess progress toward this new goal, and
� ensure that training and other interventions that emerge from general
aviation teams are communicated to small commuter operators and pilots who
may benefit from them.
DOT and FAA officials concurred with our recommendations aimed at setting a
more challenging interim goal and long-term goals for general aviation and
said that they planned to do so in the future. However, the officials noted
that existing general aviation accident data are too inaccurate to be used
as the basis for setting an accident reduction goal. The general aviation
steering committee has established a work group to recommend ways to improve
the quality of general aviation data. The officials stated that FAA and the
general aviation community would review the accident reduction goal when the
quality of the data improves.
DOT and FAA officials disagreed with our recommendation aimed at ensuring
that training and other interventions emerging from general aviation teams
are communicated to small commuter operators because they believe that
mechanisms already exist to do this. The officials explained that a number
of associations representing smaller commuter aircraft participate on the
general aviation steering committee and on its analysis and implementation
teams. These organizations provide conduits for transmitting interventions
developed by the general aviation teams to small commuter operators. We
agree that these organizations may facilitate the transfer of safety
interventions developed by the general aviation teams to small commuter air
carriers. However, it will be difficult to achieve the mandated 80-percent
reduction in commercial aviation fatalities without systematic improvements
in the safety record of small commuter air carriers, which accounted for 28
percent of fatal commercial aviation accidents. We believe that Safer Skies
would benefit from a systematic plan for ensuring that interventions
developed by general aviation teams are communicated to and implemented by
small commuter operators. For this reason, we did not modify or delete our
recommendation.
DOT and FAA officials disagreed with our recommendation calling for an
analysis of future safety problems arising from the rapid growth and changes
in aviation. The officials noted that efforts involving FAA, Safer Skies,
and the European aviation industry are already under way to address future
hazards in aviation. On the basis of the information presented by DOT and
FAA officials, we withdrew this recommendation.
The Safer Skies Initiative Has Made Progress in Selecting and Implementing
Interventions
Joint FAA and industry teams have started work on 13 of the 16 problems
being addressed by the initiative. A two-part process has been developed for
use by these teams to first analyze accident and incident data and then to
use that analysis to identify, select, and implement safety interventions to
help prevent accidents in the future. That process is reasonable and has
allowed FAA and industry groups to reach consensus on how to address safety
problems identified under the initiative. This process was not used to
address cabin safety problems because the cabin safety steering committee
had already begun its work before the process was developed. The Safer Skies
teams have made progress primarily in those areas that had been studied
extensively in the past for which widely supported recommendations already
existed. The interventions recommended for five problems are now being
implemented: uncontained engine failure and CFIT in commercial aviation and
passenger seatbelt use, child restraint systems, and carry-on baggage in
cabin safety. The process being used will require more extensive analysis in
the future as teams begin to address safety problems that have not been
studied previously. Finally, the success of the interventions that the Safer
Skies teams have chosen to address these long-standing safety problems
depends in part on effective implementation. Our past work has shown that
FAA does not consistently follow through on implementing key safety
recommendations. Furthermore, FAA and the aviation industry began
implementing some of the Safer Skies safety interventions before having a
process in place to track their progress. The initiative has developed a
process for tracking the implementation of interventions to improve safety
in commercial aviation. However, the implementation of Safer Skies'
interventions is not assured because the tracking system for commercial
aviation is not sufficiently detailed to assess progress in implementing
interventions. Furthermore, the cabin safety steering committee implemented
its interventions without having a tracking process in place, and the
general aviation steering committee is working toward the final approval of
interventions to address two safety problems without having a tracking
process. Without a complete tracking process, FAA and the industry cannot
ensure that the initiative will improve aviation safety in each of these
areas.
Safety Problems
For the Safer Skies initiative, FAA and the aviation industry jointly
developed a two-part process to analyze accident data and then to choose
from among the possible interventions. This process grew out of a previous
FAA effort that used a data-driven approach to identify threats to aviation
safety and develop interventions to address those threats. During the first
part of this process, an analysis team reviews accident data to determine
what went wrong, why it went wrong, and what interventions might be the most
effective in preventing similar accidents in the future. The second part of
the process involves another team that assesses the feasibility of each
potential intervention, prioritizes the interventions on the basis of their
effectiveness and feasibility, and submits plans for implementing projects
to the steering committee for approval. However, as we discuss later in this
chapter, the steering committee addressing cabin safety problems did not use
this process.
Interventions
The initiative uses a two-part process to analyze data and identify
interventions to address safety problems in commercial aviation and general
aviation. This process is modeled on an analysis of the most significant
threats to aviation safety conducted in 1997 by staff from FAA's Aircraft
Certification Service. The two-part process was developed for use by the
teams addressing safety problems in commercial aviation but has also been
used by the general aviation teams with some modifications. Under the
process, the steering committee forms an analysis team for each aviation
safety problem. The team, which includes members from FAA and the aviation
industry, reviews accident data, determines accident causes, and identifies
possible interventions to prevent future accidents. For selected accidents,
the team develops a detailed sequence of events that includes the actions by
pilots and air traffic controllers as well as any system or equipment
failure. The team determines what went wrong and why and then considers
various interventions that could have prevented the accident. In its final
report, the analysis team ranks all of the identified interventions by their
effectiveness in preventing similar accidents and presents them to the
steering committee for further action.
Once the analysis team completes its work on a safety problem, the steering
committee forms a second team to assess the feasibility of implementing the
interventions suggested by the analysis team. The implementation team
assesses feasibility in six areas: the cost of the intervention; the time
needed to implement it; whether it requires regulatory changes; technical
feasibility; the practicality of the project within the operating
environment or the nationwide aviation system; and political feasibility.
The implementation team prioritizes the interventions by both effectiveness
and feasibility and then presents the resulting prioritized list to the
steering committee. Once the steering committee initially approves an
intervention, the implementation team develops a detailed project plan for
implementation that is sent to the steering committee for final approval.
Once detailed plans are approved, the interventions are then implemented by
the responsible organizations.
The general aviation teams have made some modifications to the analysis
process initially developed for use by the commercial aviation teams.
Although the first commercial aviation analysis team considered feasibility
as well as effectiveness, the two-part process ultimately approved for
commercial aviation teams considers only effectiveness at the analysis
stage. Any consideration of such matters as cost and the need for developing
new regulations is left to the implementation team. In contrast, the general
aviation analysis teams consider both effectiveness and feasibility. Our
review of the general aviation analysis team's reports for CFIT and weather
confirmed that such feasibility criteria as cost and the need for new
regulations have been considered far earlier in the assessment of general
aviation interventions than in the process now used by commercial aviation
teams. While other feasibility factors are also considered, cost, the
avoidance of interventions that would require new regulations, and
acceptability to the general aviation community have weighed heavily in the
choice of interventions to address general aviation safety problems. In
emphasizing cost and acceptability to the aviation community, the general
aviation teams have selected training and other interventions that will be
more affordable to general aviation pilots.
While the initiative is using a systematic, defined approach to consider
ways to address safety problems in commercial and general aviation, a
different approach was used to address cabin safety problems. Several months
before the announcement of the Safer Skies initiative, FAA established the
Partners in Cabin Safety (PICS) team to provide information to the public
about four cabin safety problems: passenger interference with flight crews,
the safety benefits of greater use of seat belts by passengers, the safety
benefits of child safety restraints, and potential safety issues arising
from the stowage of carry-on baggage. According to PICS team members, FAA
identified these problems before assigning them as tasks to the team in
January 1998. Team members discussed such additional issues as in-flight
medical emergencies and cabin air quality but settled on the four that were
eventually included. Unlike the commercial and general aviation teams, the
PICS team limited the possible interventions to ones that did not require
that FAA create new regulations, a process that was viewed by some
participants as too slow and unlikely to result in consensus among various
industry and government participants. Consequently, the PICS team focused on
interventions that involved educating passengers.
The four cabin safety problems addressed differ in several important ways
from those safety problems addressed by the Safer Skies teams in commercial
and general aviation. First, the cabin safety problems resulted in only two
fatalities in U.S. commercial aviation from 1988 through 1997, both
involving passengers not using their seat belts when the aircraft
encountered turbulence. In contrast, during the same period of time, there
were more than 9,800 fatalities in all commercial and general aviation
accidents. Second, air carriers are not required to maintain or submit data
on cabin safety incidents unless they involve fatalities or serious
injuries. Since only limited historical data on cabin safety accidents and
injuries were available for analysis, the PICS team did not conduct a causal
analysis as has been done by the analysis and implementation teams for both
commercial and general aviation.
The PICS team disbanded in January 1999 after it completed the development
of passenger education materials. As part of its passenger education
efforts, the team distributed brochures on child restraint systems from a
previous campaign by FAA. In addition, the Luggage and Leather Goods
Manufacturers of America, along with FAA, developed a brochure addressing
carry-on baggage concerns, which the team members were asked to distribute
to airlines, luggage stores, and airports. It was also put on FAA's World
Wide Web site for further distribution by interested parties. Steering
committee members and FAA officials also told us that the team worked with
air carriers to develop additional cabin announcements for the stowage of
carry-on baggage and the importance of seat belt usage. Finally, the PICS
team developed a passenger safety checklist for publication on FAA's Web
site, which addressed passenger interference with flight crews, seat belt
usage, child restraint systems, and carry-on baggage. This checklist,
however, is not currently available on FAA's Web site. According to an
official at FAA's Flight Standards Service, the passenger safety checklist
project is on hold until the agency appoints a new national resource
specialist for cabin safety who will review the document before it is made
available to the public.
Progress With Problems Studied Previously
Since the FAA Administrator announced the Safer Skies initiative in April
1998, work has started on 9 of the 12 safety problems to be addressed in the
commercial and general aviation safety areas. Teams have made the most
progress in selecting interventions for safety problems when they could
build on previous studies for which widely supported recommendations exist.
The commercial aviation steering committee plans to have work started on all
of the identified problems before the end of fiscal year 2000, but the
general aviation steering committee has not yet determined when work on
three of its six problems will begin. Table 7 shows the status of the work
on each of the 12 safety problems to be addressed in commercial and general
aviation as of April 1, 2000.
Analysis team Implementation team
Final
Team report Team Final report
formed formed issued
issued
Commercial aviation
CFIT X X Xa X
Loss of control X
Uncontained engine
failureb X X X X
Runway incursionsc X
Approach and landing X X Xa X
Weatherd
Turbulence X
Icing
General aviation
CFIT X X X e
Loss of control
Runway incursions X
Aeronautical
decision-making
Survivability
Weather X X X e
aThe commercial aviation steering committee combined the implementation
teams for CFIT and approach and landing accidents because many of the
interventions chosen by the analysis teams for these two safety problems
overlapped.
bThe joint FAA/industry team working on uncontained engine failure developed
the prototype process used by the Safer Skies analysis and implementation
teams.
cThe analysis activities for runway incursions include both commercial and
general aviation accidents reviewed by a joint team.
dThe commercial aviation steering committee will form separate teams to
address two weather issues--turbulence and icing.
e The general aviation CFIT and weather reports were presented to us as
final reports. However, in responding to our draft report, FAA told us that
these reports had not received final approval.
Source: GAO `s analysis of Safer Skies' data.
The steering committees charged with developing interventions for each of
the safety problems first formed analysis teams to work on problems for
which major studies had already been done or were under way. The ongoing and
completed studies conducted by FAA and the industry provided information the
analysis teams could use to identify the causes of accidents and potential
interventions. For example, the Flight Safety Foundation had completed an
extensive study on CFIT, examining over 250 accidents and incidents
worldwide. The foundation had also developed training materials for pilots
and made other recommendations to prevent CFIT accidents. In another
instance, the team analyzing weather-related accidents involving general
aviation aircraft identified 11 safety studies that had preceded its
efforts, all of which recommended interventions similar to the ones the team
ultimately identified. FAA participants on the Safer Skies commercial
aviation steering committee told us that beginning with previously studied
safety problems helped team members make progress in developing the team's
two-part process for analyzing data and identifying interventions and become
comfortable with the analysis and selection process before moving onto more
complex issues that may involve original research and analysis. However,
this approach meant that work on another area that is important for reducing
the fatal accident rate in commercial aviation did not start work until
September 1999--17 months after the initiative's announcement. FAA
identified loss of control as the single largest cause of fatal commercial
aviation accidents involving U.S. operators.
To date, the implementation of interventions has concentrated mostly in
areas for which analysis and implementation were well under way or complete
when the initiative began. The first of the interventions to be implemented
addressed uncontained engine failure: FAA issued a series of airworthiness
directives requiring enhanced inspections of high-speed rotating parts in
certain jet engines.29 The directives require industry maintenance personnel
to perform additional, more detailed inspections to check for cracks and
other signs of irregularities whenever an engine is disassembled for
overhaul or maintenance. According to staff at FAA's Engine and Propeller
Directorate, these directives affect more than 90 percent of the jet engines
that U.S. airlines currently use. FAA and the industry are also taking steps
to implement an intervention endorsed by the commercial aviation team
examining CFIT. The team has recommended that enhanced navigational
equipment be installed on new and existing aircraft to warn pilots of
impending crashes. Air carriers began installing the enhanced navigational
equipment to prevent CFIT accidents in their aircraft before FAA issued the
final rule requiring that the equipment be installed in the commercial
fleet. Specifically, air carriers began installing the enhanced navigational
equipment to prevent CFIT accidents in their aircraft before FAA issued its
final rule in March 2000 and before the commercial aviation team working on
CFIT issued its final report in June 2000. This equipment is now being
included on some new aircraft, and airlines had equipped about 4,000
aircraft already in service with the new technology by December 31, 1999.
The timetable for analysis and implementation teams addressing the problems
included under the initiative has changed since the initiative was announced
in April 1998. According to the chairs of the Safer Skies steering
committees, some of these schedule changes occurred because the analysis
process took longer than anticipated. In other cases, changes to the
analysis approach required rescheduling Safer Skies' efforts. For example,
FAA officials explained that the final report date for the commercial
aviation CFIT implementation team was rescheduled after the steering
committee decided that combining the CFIT and approach and landing teams for
the implementation analysis made sense because of overlap in the
interventions they had identified. Several high-priority interventions to
address CFIT accidents in commercial aviation were, however, forwarded to
the steering committee for final approval and implementation without waiting
for the implementation team's final report. An FAA co-chair of the general
aviation steering committee told us that they changed the start dates for
several of the general aviation teams because general aviation accidents are
more numerous than commercial aviation accidents and analyzing them proved
more time-consuming than anticipated. This FAA official also said that some
general aviation groups participating in the initiative do not have enough
people or resources to serve on multiple teams simultaneously. While we
believe that these decisions were justified, they also effectively mean that
the interventions to resolve some key safety problems will not be identified
or implemented until later than originally anticipated.
Problems Will Require More Analysis
Additional analysis will be needed to identify interventions to address
current and future safety problems for which few or no previous studies
exist. Safer Skies teams relied initially on a limited number of case study
analyses to identify the causes of accidents and incidents, as well as the
interventions that could prevent them in the future. The teams compared the
results of these case studies with the causes and interventions identified
by previous studies to determine whether they are consistent. For example,
the team working on CFIT in commercial aviation completed detailed event
sequences for 10 accidents and found that the causes identified and the
interventions it recommended were similar to those of prior studies. Safer
Skies teams working on approach and landing accidents in commercial aviation
and weather-related accidents in general aviation also compared the results
of their analyses with those of prior studies.
Along with other changes as the Safer Skies initiative has evolved, this
approach has been modified as teams addressed additional safety problems.
For example, the runway incursion analysis team expanded its case studies to
include incidents because there were so few fatal accidents involving runway
incursions. Similarly, the analysis team now working on loss of control in
commercial aviation has selected a larger number of case studies because
this safety problem has not been the subject of extensive prior analysis.
Toward the Goals Set for Reducing Fatal Accidents
The Safer Skies initiative has identified the major safety problems to be
addressed, has made progress in identifying their root causes, and has
developed interventions to address some of them. Reducing fatal accidents
depends in part on the effective implementation of these interventions. As
discussed in chapter one, however, many of these safety problems are
long-standing ones that have persisted in spite of previous studies and
recommendations. In addition, FAA has not consistently followed through on
implementing safety recommendations in the past. The Safer Skies initiative
does not yet have in place a process to track the implementation of these
interventions that is sufficiently detailed and covers interventions chosen
to improve safety in commercial aviation, general aviation, and cabin
safety.
Implementation
Reducing the fatal accident rate in commercial aviation and the number of
general aviation accidents will depend in part on effective implementation
of the interventions chosen by the Safer Skies teams. Many of the safety
problems that the initiative addresses are long-standing ones that have been
studied extensively in the past. Actually resolving these problems has
proven difficult in the past and remains very challenging. Similar
interventions have been recommended, but the desired reductions in fatal
accident rates have not been achieved. For example, extensive prior studies
of CFIT and approach and landing accidents in commercial aviation
recommended many of the same interventions that are now being implemented by
the Safer Skies commercial aviation steering committee. Furthermore,
reaching the 80-percent goal in commercial aviation will depend heavily on
the successful implementation of interventions to address the safety
problems that caused the most fatal accidents: loss of control, CFIT, and
approach and landing. To reach the goal in commercial aviation,
interventions must be effectively implemented for both small commuter
aircraft and large commercial air carriers. Even after safety interventions
have been identified, implementing them has proven challenging.
As DOT's Inspector General and we have reported previously, FAA does not
consistently follow through on implementing safety recommendations. Our
review showed that FAA usually agreed with the recommendations on aviation
safety made by GAO, NTSB, and DOT's Inspector General. FAA had implemented
64 percent of the 256 recommendations that we reviewed; however, FAA had not
completed actions to implement the remaining 36 percent of the
recommendations.30 We found that FAA sometimes did not establish time frames
for implementing the recommendations or did not meet established times for
implementing them. Similarly, DOT's Inspector General found that of the 23
near-term actions FAA planned for addressing runway incursions in its 1998
Action Plan, 15 had not been completed on time.31 We found that even safety
recommendations that received specialized attention, intensive follow-up,
and heightened awareness among industry, the Congress, and the public have
not been fully implemented. For example, NTSB considered runway incursions
so serious that it repeatedly placed this safety problem on its lists of
critical safety recommendations in the early 1990s. Although FAA concurred
with NTSB's recommendations, our review found that several of the corrective
actions needed had not been implemented, including actions to improve (1)
visibility at airports; (2) runway lighting, signage, and surface markings;
and (3) radar and related equipment to alert air traffic controllers to
impending runway incursions. FAA developed several plans in the 1990s to
decrease runway incursions. In spite of these programs, the actual number of
runway incursions has increased. DOT's Inspector General noted in 1999 that
the number of runway incursions had increased from 292 in 1997 to 325 in
1998, in part because FAA had not set aside the funds needed to support the
initiatives and projects in the runway incursion action plan.32 As a result,
FAA has made limited progress in implementing its plan, and milestones have
been missed and extended. DOT's latest performance report for fiscal year
1999 shows continuing problems in this area. The actual number of runway
incursions (321) was 19 percent higher than the goal of 270 established in
DOT's performance plan.33
Industry participants in the Safer Skies initiative have voiced concern that
some interventions may not be implemented promptly or at all. Some of the
same Safer Skies participants questioned whether enough resources would be
available to complete the implementation of the selected interventions.
Without assurance of adequate resources, it is likely that the choice of
interventions by Safer Skies teams will be constrained by cost
considerations and the implementation of recommended interventions will be
incomplete. Effective implementation will also depend on having a process
for tracking the implementation of interventions to be carried out by all
Safer Skies participants, including FAA; other government agencies;
manufacturers; airlines; and other industry participants.
for Tracking the Implementation of Interventions
FAA and the aviation industry began implementing some of the Safer Skies
safety interventions before developing a systematic way of tracking the
progress being made. This occurred in part because the steering committees
incorporated some safety initiatives already under way and endorsed the
resulting interventions before they developed a systematic tracking process.
In addition, Safer Skies teams have recommended that a few high-priority
safety initiatives be started before final implementation reports are
issued. While moving forward on important safety initiatives makes sense,
ensuring their successful implementation depends on effective tracking.
Interventions have been implemented in both commercial aviation and cabin
safety with no tracking process in place. The general aviation steering
committee is moving toward approval of interventions for CFIT and weather
but has not yet developed a tracking process. Several of the Safer Skies
participants we interviewed voiced some concerns about whether all the
interventions being identified would eventually be implemented, given FAA's
past problems in implementing recommended safety improvements.
In its December 1997 report, the congressionally mandated commission on
aviation safety recommended that FAA's and the industry's strategic plan
include milestones for accomplishing specific tasks. The commission noted
that the plan should be detailed enough that milestones for accomplishing
specific tasks can be readily recognized by agency management and the
industry, as well as the public. In addition, the commission directed FAA to
report periodically on where initiatives stand, why any delays are
occurring, and whether and why changes are being made to the plan. These
recommendations are in accordance with sound internal controls for program
management.
The Safer Skies initiative, which was announced in April 1998, implemented a
number of interventions without first developing a process for tracking
their progress. In some cases, these were interventions that were developed
by teams whose work was incorporated into the Safer Skies effort. In
commercial aviation, for example, FAA, relying on the work of the
uncontained engine failure team, published airworthiness directives
beginning in April 1999 to require more extensive inspections of aircraft
engines. The commercial aviation team working on CFIT also implemented
several interventions in or before September 1999. These included
interventions to verify the operational status of radar equipment to provide
minimum safe altitude warnings to pilots and to develop a template for
standard operating procedures to be used by airlines in training their
pilots in techniques to avoid CFIT accidents.
In September 1999, the commercial aviation steering committee recognized the
need for the systematic tracking of interventions and directed a work group
to develop a proposal. At the commercial aviation steering committee's
meeting in January 2000, the work group presented its proposal for a Joint
Implementation Measurement Team. The team designed the tracking process to
provide a high-level report on whether each intervention is being
implemented as planned. Specifically, this team's responsibilities will
include tracking whether the implementation of approved interventions
complies with the implementation plans and their milestones; helping to
predict the potential effectiveness of the proposed interventions; and
identifying ways of measuring whether the intervention is achieving the
desired risk reduction. The team will also provide a brief explanation of
what is causing noncompliance with the plan and whether a solution has been
found to resolve the problem. As conceived, the tracking report is to be a
high-level progress report that does not intrude on the internal planning of
the organizations responsible for carrying out the interventions. The
tracking report thus does not provide detailed information on interim and
long-term milestones or identify individuals responsible for implementing
the plan and preparing progress reports for the tracking committee. Without
more detailed information than is currently provided in the proposed
tracking report, it may be difficult for the steering committee to assess
progress in implementing interventions. For example, the tracking team's
January report notes that FAA has completed a plan for implementing two
programs critical to gaining access to safety data34 and that other industry
and government groups have plans in development. However, the tracking
report provides no information about the milestones established by FAA's
plan for establishing these key programs, both of which have experienced
delays in the past. After we identified concerns about the tracking system,
the commercial aviation steering committee agreed that improvements are
needed, and it is working on revisions. A draft version provided for our
review in June 2000 still lacked key information about major commitments,
deliverables, and milestones.
Tracking implementation is even more critical for the more complex
initiatives whose success depends on coordinated efforts by both FAA and the
aviation industry. For example, successful implementation of the
highest-priority intervention to prevent CFIT accidents in commercial
aircraft--the installation of enhanced aircraft navigational equipment to
warn pilots of impending crashes--requires coordination among many parties:
� FAA must certify that the equipment works, issue technical standards for
manufacturers, and issue a final regulation to require that the equipment be
installed on new and existing aircraft.
� Aircraft manufacturers need to make the equipment standard on new aircraft
and retrofit it in older aircraft.
� Air carriers need to incorporate the appropriate procedures for
maintaining and using this equipment into maintenance and flight manuals and
to train pilots in its use.
� FAA needs to update its guidance to its inspectors so that they can ensure
that air carriers properly carry out their responsibilities for training,
maintenance, and use of the equipment.
Without a tracking system that provides more detailed information on the
implementation of complex interventions, the commercial aviation steering
committee will not have the information needed to ensure that they are fully
implemented in accordance with planned milestones.
The implementation of interventions to improve cabin safety has also not
been adequately tracked. The cabin safety steering committee, which
completed the development of passenger education materials before it
disbanded in January 1999, carried out most of its interventions with no
Safer Skies tracking process in place. However, we found that educational
materials related to passenger interference with crew had not been
distributed or made available on FAA's Web page as of April 2000.
Furthermore, according to a member of the cabin safety steering committee,
the distribution of other cabin safety brochures was, in some instances,
never completed. The absence of a systematic process for tracking Safer
Skies interventions may have contributed to inaccuracies in reporting on the
status of cabin safety interventions. Specifically, the DOT FY 2001
Performance Plan and FY 1999 Performance Report states that all initiatives
relating to cabin safety were completed as planned. However, planned actions
to include material on passenger interference with crew had not been
completed as of April 2000.
Finally, although the general aviation steering committee is reviewing draft
implementation team reports that recommend interventions to address CFIT and
weather, it has no process in place to track the implementation of
interventions once they are approved. According to the FAA co-chairs of the
general aviation steering committee, this group has committed to track the
interventions selected but has not yet developed a process for doing that
and plans to discuss this issue at a future meeting. Without coordinated,
detailed implementation plans that assign responsibilities, FAA and the
Safer Skies steering committees will not be able to ensure that all parties
complete their portion of the plan and that implementation occurs on time.
In addition, as part of the Safer Skies process, FAA and the general
aviation community identified efforts that could be accomplished in the
short term or were already under way to address the safety areas to be
addressed by the initiative. FAA and the industry implemented a number of
these short-term initiatives, such as the development and distribution of
various safety videos and training aids. However, Safer Skies did not track
the implementation of these interventions or evaluate their effectiveness.
The progress made by the initiative to date has resulted in the
implementation of interventions for five safety problems--two in commercial
aviation and three in cabin safety. However, a coordinated, centralized
method of tracking will be necessary to ensure full implementation of these
and future interventions. In the past, FAA has developed plans to make
safety improvements but has not consistently implemented them successfully.
An effective tracking system would provide for identifying the individuals
or entities responsible for implementation, setting milestones, establishing
resource estimates, and preparing progress reports. Without a systematic
tracking mechanism, there is no assurance that any of the selected
interventions will be fully implemented. While the commercial aviation
steering committee has developed a system to track the implementation of the
interventions it approves, this system is not sufficiently detailed to
ensure their implementation. The general aviation steering committee, which
is nearing final approval on interventions to address safety problems
related to weather and controlled flight into terrain, is only now
developing a tracking system modeled after the one used by the commercial
aviation steering committee. Finally, nothing comparable has been developed
to track interventions recommended by the cabin safety teams.
To ensure that interventions are implemented and that effective and feasible
interventions are identified in the future for issues that the initiative
has yet to address, we recommend that the Secretary of Transportation direct
the FAA Administrator to advise the Safer Skies steering committees to take
the following actions:
� Develop a systematic way of tracking the implementation of interventions
approved by all Safer Skies steering committees. This tracking system should
include the identification of responsibility for implementation, the
establishment of short- and long-term milestones and resource estimates, and
the preparation of progress reports. The progress reports should provide
information on the detailed steps to be taken by all government and industry
participants to ensure the successful implementation of each intervention.
Progress reports should highlight and explain any delays in meeting the
milestones. This system should be shared with the relevant Safer Skies
steering committees and FAA's focal point for the initiative as well as with
the team that recommended the intervention.
DOT and FAA officials concurred with our recommendation on the need to track
the implementation of interventions to achieve results, but they disagreed
with the level of detail we advised. The officials stated that the
commercial aviation steering committee's draft revised tracking system
provides better information for tracking the major commitments and
deliverables. The expectation is that more detailed implementation plans
will be maintained within each implementing organization. The officials do
not believe that it is realistic for steering committees to review the
details of every organization's action plan. They also noted that the
general aviation steering committee is developing a tracking system similar
to that used to track commercial aviation interventions. We agree that the
Safer Skies initiative has taken steps to improve its tracking system for
commercial aviation and to work toward the development of a similar system
for general aviation. However, the revised tracking system provided for our
review in June 2000 did not clearly identify and include time frames for
major commitments and deliverables for each of the interventions approved by
the commercial aviation steering committee. We agree that individual FAA and
industry organizations responsible for implementing Safer Skies
interventions would logically have far more detailed systems for tracking
implementation than the steering committees. However, without a reliable
tracking system in place that contains basic information on major
deliverables, responsibilities, and time frames, FAA and Safer Skies will
not be in a position to ensure that recommended interventions are
implemented to improve aviation safety.
DOT and FAA officials disagreed with our recommendation that FAA and the
Safer Skies steering committees should analyze a sample of safety problems
that were not studied previously. The officials presented information that
showed some Safer Skies' work groups were using or would be using a sample
of previously unexamined safety problems in their work. For this reason, we
withdrew the recommendation.
The Safer Skies Initiative Has Not Yet Developed Performance Measures to
Evaluate the Effectiveness of Most Interventions
Of the five Safer Skies teams that have begun implementing interventions,
only one has developed a performance measure to evaluate whether the
interventions it has selected are helping to reduce the safety problems that
cause fatal accidents and are worth what they cost. Such evaluations depend
on performance measures that serve as the yardsticks for measuring progress
toward program goals. The initiative's ultimate goal is saving lives by
reducing fatal accidents. Federal law requires that federal departments
evaluate the effectiveness of the program activities for which they request
funding. FAA will evaluate progress toward its broad goals for aviation
safety using performance measures based on reducing the fatal accident rate
for commercial aviation and the number of fatal accidents in general
aviation. However, additional performance measures will be needed for
evaluating the effectiveness of the interventions selected by the teams
working on each of the safety problems. Most teams are still analyzing data
on safety problems and selecting safety interventions and thus have not yet
determined how to evaluate the effectiveness of interventions selected.
Although teams working on 5 of the 16 safety problems have recommended
interventions that are being implemented, only one of these teams developed
an adequate performance measure before its interventions were implemented.
of the Budget Process
To ensure that programs achieve their objectives and that funds are expended
wisely, federal law requires that each department develop performance
measures as part of its budget request. Performance measures are the
yardsticks used to evaluate the effectiveness of the activities undertaken
as part of federal programs. The initiative plans to develop performance
measures to evaluate the effectiveness of the interventions it recommends to
save lives by addressing the safety problems that cause fatal accidents.
However, developing good performance measures can be difficult. While it is
useful to establish a baseline of information about past fatal accidents,
they occur too rarely to serve as performance measures to evaluate the
effectiveness of interventions. Years may elapse between specific types of
fatal accidents, such as uncontained engine failure, making it difficult to
see trends or evaluate the effectiveness of interventions. Instead, the
initiative must develop performance measures based on events that occur more
frequently and that can be linked closely to interventions.
A congressional mandate exists for the measurement and evaluation of all
federal programs. Performance measurement is a central premise of the
Government Performance and Results Act of 1993 (Results Act). This act
requires annual performance plans to cover each program activity set out in
a federal agency's budget. Among other requirements, performance plans are
to (1) establish performance indicators to be used in measuring or assessing
the outcomes of each program activity, (2) determine how to compare actual
results with the performance goals, and (3) describe the means to verify and
validate information used to report on performance. In accordance with this
law, DOT develops annual plans that include performance measures for
specific programs and activities. Agencies under DOT, such as FAA, develop
more detailed plans and performance measures for each program activity.
Because of its impact on FAA's programmatic and budgeting activities, the
Safer Skies initiative falls under the Results Act's requirement to evaluate
program performance. Moreover, it was developed in response to the National
Civil Aviation Review Commission's report, which specifically directed FAA
and the aviation industry to establish performance measures and milestones
to assess the initiative's progress in meeting safety goals, to review
priorities periodically, and to monitor progress. The Safer Skies initiative
incorporates the idea of establishing performance measures to evaluate
progress toward safety goals. As a result, the Safer Skies teams that
recommend interventions are tasked with developing the performance measures
for those interventions approved by the steering committees.
For a performance measure to be useful, a baseline must be established
against which to measure the effect of the intervention. Good evaluation
criteria include (1) definitions of baseline information on the extent of
the safety problem over a particular period prior to the implementation of
the intervention and (2) timeframes for evaluating changes using the
performance measure. Goals and time frames must also be established to
determine what the program is expected to achieve and by when. For the
initiative, appropriate baseline information includes both the total number
of fatal accidents and the number of fatal accidents caused by each safety
problem within each type of aviation operation (i.e., commercial aviation
and general aviation).
Good performance measures have several key features: the event to be
measured (e.g., a runway incursion) or desired outcome (a reduction in the
number of runway incursions) is measurable; data on the event are or could
be collected; and the event occurs with sufficient frequency between
evaluations for progress to be measurable. The performance measures under
development to evaluate Safer Skies' initiatives can be assessed against
these criteria.
Require the Development of Additional Performance Measures
Determining the effectiveness of Safer Skies interventions will require the
development of performance measures other than the overall goals set for
commercial and general aviation. Fatal aviation accidents occur so
infrequently that their usefulness is limited as a measure of the success of
Safer Skies' interventions. This is especially true for commercial aviation,
which had a total of 85 fatal accidents in the United States from 1988
through 1997. The fact that a particular type of accident has not occurred
for several years does not mean that the underlying safety problem has been
successfully addressed. Furthermore, for several reasons it may be difficult
or impossible to match a specific implementation plan to a numerical
reduction in fatal accidents overall or attributable to a specific safety
problem. For example, in general aviation the lack of detail in accident
reporting makes it difficult to determine specific accident causes; the lack
of pilot profiles makes it difficult to evaluate the effectiveness of pilot
training strategies; and it is hard to predict how many aircraft owners will
install new safety equipment in the future. Thus, to determine to what
extent an intervention is reducing fatal accidents attributed to a specific
safety problem, teams will need to develop additional performance measures.
The commercial aviation steering committee recognized early on this need to
develop interim measures to evaluate the unique effect of individual
interventions.
Even if a team identifies suitable performance measures for a specific
safety problem, it may be difficult to determine whether a particular
intervention, cluster of interventions, or other outside factors influenced
changes in the performance being measured. This is especially true for
situations in which teams choose numerous interventions to address a safety
problem. While the uncontained engine failure team developed a single
primary intervention, the team working on CFIT in commercial aviation has
already initiated several interventions and is contemplating dozens more.
Similarly, the general aviation team working on weather recommended 17
interventions. Without some way to independently evaluate the effectiveness
of individual interventions or clusters of interventions, the initiative
will have little way of knowing whether particular interventions save lives
and are thus worth the time or money being expended on them.
In developing performance measures, one option involves using the precursors
to accidents as proxies for the likelihood of fatal accidents. Precursors
are events that, although they typically precede a particular type of fatal
accident, often occur without culminating in a crash. For example, approach
and landing accidents are almost always preceded by unstable approaches to
the airport, but many unstable approaches may culminate in a hard or late
landing that does not result in injuries or a crash. Performance measures
based on precursors have been developed to evaluate initiatives for one of
the safety problems the initiative is addressing, uncontained engine
failure. The success of this approach depends on identifying appropriate
accident precursors that can serve as proxies for the specific safety
problem the team is addressing. Precursors are most useful when they follow
the criteria for good performance measures: they are measurable, relevant
data on them are available, and they occur with sufficient frequency.
Without Determining How to Evaluate Their Effectiveness
Of the 16 Safer Skies teams, 8 have recommended safety interventions for
implementation; interventions from 5 of these teams have been or are being
implemented; but only one has developed a performance measure that can show
whether the intervention is effective at saving lives. Most Safer Skies
teams are still analyzing data on safety problems and selecting
interventions and have not yet determined how to evaluate the effectiveness
of interventions selected. Of the five teams whose recommendations are being
implemented, three have developed some performance measures. Only the
uncontained engine failure team has developed two quantifiable performance
measures that are based on accident precursors. In contrast, the general
aviation teams working on CFIT and weather developed some general
performance measures for reducing accidents resulting from these safety
problems but did not quantify these measures. No performance measures were
developed to evaluate the educational interventions implemented to address
the four cabin safety problems. Finally, the team working on CFIT accidents
in commercial aviation has implemented one intervention in advance of the
team's final report. While this team has not yet developed a performance
measure for this intervention, it is considering using an accident
precursor. Performance measures based on accident precursors have potential
for use in evaluating the effectiveness of additional interventions being
considered to address CFIT and other safety problems. FAA does not presently
collect data on some accident precursors that could be used to evaluate the
effectiveness of Safer Skies interventions and faces significant barriers to
collecting such data.
Two Accident Precursors as Performance Measures
The Safer Skies team working on uncontained engine failure chose two
accident precursors as performance measures for evaluating the effectiveness
of the intervention it recommended: more extensive engine inspections.
Because uncontained engine failure caused just two fatal accidents in the
United States in 1988-97, fatal accidents are too infrequent to serve as a
performance measure. But well-established trend data show that the safety
problem occurs much more frequently, resulting not in fatal accidents but in
incidents with severe or serious consequences on an average of about 1.5
times a year.35 The team chose the rate of these incidents as the primary
performance measure for its recommended intervention. The team also chose
another accident precursor as a second performance measure: the number of
cracks detected in engine disks when engines are overhauled.36 Data analysis
identified cracked disks as the primary cause of uncontained engine failure.
According to staff at FAA's Engine and Propeller Directorate, each crack
detected during inspections probably avoids an uncontained engine failure
that could have had severe or serious consequences.
Both accident precursors chosen--the rate of uncontained engine failure with
severe or serious consequences and the detection of cracks in engine
disks--have some of the attributes of a good performance measure. Both can
be counted, and reporting mechanisms are in place for collecting the key
data needed for both measures. Hence, it will be possible to evaluate
whether the more extensive engine inspections lead to the detection of more
cracks and fewer instances of uncontained engine failure with severe or
serious consequences. However, good performance measures track events that
occur often enough between evaluations to show whether progress is being
made. Uncontained engine failure with severe or serious consequences occurs
from one to three times a year, according to data from 1992-98, while cracks
in engine disks are likely to be discovered about once in 25,000
inspections, according to staff at FAA's Engine and Propeller Directorate.
Hence, 2 to 5 years may elapse before the effectiveness of the more
extensive engine inspections can be judged. Nonetheless, tracking both
measures should provide sufficient data for reasonable interim and final
performance measures, and the enhanced inspections provide an opportunity to
avert potentially catastrophic accidents.
The uncontained engine failure team established much of the information
needed to use its performance measures to evaluate the effectiveness of
enhanced engine inspections. During our review, we worked with FAA staff on
the team to develop additional information to provide a more complete
context for how that intervention relates to the overall Safer Skies effort
and to the fatal accident rate in commercial aviation. We then developed a
template for this information that can serve as a model for other Safer
Skies implementation teams. (See table 8.) The template displays the data
critical for understanding the extent of the safety problem and the baseline
for measuring progress in addressing it, including the frequency of the
problem's occurrence in 1988-97 and projections of its occurrence with and
without the recommended intervention by 2007, the target year for Safer
Skies to achieve an 80-percent reduction in the overall fatal accident rate.
The template reflects the team's goal of reducing the rate and projected
number of uncontained engine failures with severe or serious consequences by
50 percent by 2007.
Baseline information on uncontained engine failure
Cause of fatal Baseline Annual Projected for 80-percent
accident (1988-97 average 2007 without reduction
intervention d
All causes 85 6 9 2
Uncontained
engine failure 2 c c c
Baseline information on intervention and performance measures
Performance Annual Projected for Projected for
measure Baseline average 2007 without 2007 with
intervention e intervention
Rate of
detection of 1 per 25,000
cracks a a a inspections
(estimated)
Rate of
uncontained
engine failure 0.9 per 10 0.9 per 10 0.9 per 10 0.45 per 10
events with million million million million
severe or departuresb departures departures departures
serious
consequences
Number of
uncontained
engine failure
events with 11b 1.5 2 1
severe and
serious
consequences
aPast data on the number of disk inspections are not available. Thus, data
on the rate of detection of cracks under the prior requirements for engine
inspections are not available.
bEstablished using available baseline data from 1992-98.
cBecause uncontained engine failure occurs so infrequently, both the annual
average of 0.2 and any projections are unreliable.
dOur projection uses FAA's projected annual growth rates of 4.0 percent for
large air carriers and 3.0 percent for commuter air carriers.
eFAA's projection is based on Boeing's data showing growth in annual
aircraft departures from 16 million in 1998 to 22 million in 2007.
f The annual average is for the baseline years chosen by FAA and the Safer
Skies steering committees in establishing their goal for commercial
aviation: 1994-96.
Source: GAO's analysis of data from Safer Skies, FAA, and NTSB.
Performance Measures
The general aviation implementation teams for CFIT and weather have
completed their draft reports but did not develop quantified, specific
performance measures to evaluate the effectiveness of the interventions they
recommended. The general aviation CFIT team recommended 5 interventions
subdivided into 22 distinct subinterventions. None of the 22
subinterventions included specific, quantified performance measures. For
example, the CFIT team recommended developing criteria for standardizing the
marking of wires, towers, and support structures to help decrease the number
of CFIT accidents that occur when pilots of low-flying aircraft, such as
helicopters and small planes, fly into these obstacles. As one measure of
effectiveness, the team chose a decrease in the number of CFIT accidents
involving wires or towers. However, the team did not provide any baseline
information about the number of past CFIT accidents that involved wires or
towers or the types of aircraft involved. To the extent that such baseline
information is available, it provides a yardstick against which to measure
progress in reducing these accidents. Furthermore, the team did not provide
any specific interim or long-term accident reduction goals for the number of
accidents or the percentage of the fleet affected. Without such information,
it will be impossible to determine whether or by how much CFIT accidents
involving wires or towers have decreased. Other performance measures for
general aviation CFIT initiatives share this lack of quantification and
specificity. Without baseline information on the occurrence of the problem
prior to the implementation of the intervention and specific quantified
goals, it will be impossible to evaluate the effectiveness of the
interventions implemented.
The general aviation team working on weather experienced similar problems in
setting performance measures for its interventions. The team's final report
recommended 17 interventions subdivided into 49 distinct subinterventions.
Of the 49 subinterventions, only 1 included a quantified, specific
performance measure. The rest had either no performance measures or
performance measures that were not quantified or specific. Some of the
interventions for which no performance measures were established involve
research that is still ongoing to develop the technology suggested in the
intervention. For example, NASA has the lead in developing equipment to
sense turbulence and warn flight crews so that they can avoid or reduce the
dangers associated with turbulence. Because research on this technology is
preliminary, the performance measures are described broadly as reducing
fatalities and injuries. It is likely too early to establish performance
measures for these interventions. The performance measures included for many
other subinterventions were too broad to allow actual evaluation of their
effectiveness. The performance measure for most of these was a "decrease in
the number of weather-related accidents." These performance measures are
neither quantified nor linked in any specific way to the interventions,
which makes it impossible to determine what portion of the reduction, if
any, is attributable to individual interventions or clusters of
interventions. Of the performance measures developed, several measure
progress in implementing training interventions, rather than the
effectiveness of the training in reducing safety threats. For example, one
intervention involves training Flight Service Station specialists and
supervisors on in-depth weather analysis and interpretation to improve the
weather briefings given to general aviation pilots. The associated
performance measure involves training all of these FAA staff by 2002, rather
than measuring the effectiveness of that training. In other cases, the team
did not include a performance measure when one could have been developed.
For example, one intervention involves conducting a refresher clinic for
flight instructors to update them about current weather information and
provide appropriate training materials for them to use with general aviation
pilots. No performance goal was specified for this intervention. To measure
how well this intervention has been implemented, it is possible to determine
the number of flight instructors, to establish a goal for how many attend
this training each year, and to have them provide information on how many
pilots they subsequently train using the information. To determine whether
the intervention is effective, the pilots who receive the training could
later be surveyed to determine whether they had used the weather information
provided or their safety records could be compared with the records of
pilots who did not have the training. The link between accident reduction
and such training is more tenuous than the link between crack detection and
the prevention of uncontained engine failure, but it is possible to gain at
least some information about the effectiveness of the training. Without such
feedback, it is difficult to determine whether the training is effective and
should be continued.
Without more specific baseline information on these performance measures
prior to the implementation of the interventions and interim and long-term
goals for progress, the initiative will not be able to evaluate the impact
of these interventions. In responding to our draft report, FAA noted that
the implementation teams for CFIT and weather relied on the expertise of
team members, following analysis of the root causes of accidents, to
determine the probable effectiveness of the interventions. Safer Skies
analysis and implementation reports described problems with the quantity,
quality, and type of data currently available about general aviation. These
problems include shortcomings in the data for the types and numbers of
operations and in the level of detail of the actual accident investigations.
FAA concluded that the problems with general aviation data make it difficult
to measure the effectiveness of individual intervention strategies by the
traditional approach of how they affect accident rates. While we acknowledge
the need to improve general aviation data, we also believe that such data
can provide some indication of the relative frequency and importance of the
causes of fatal accidents. Such information is also important for making
decisions about which interventions to fund and expedite, considering their
potential effectiveness and the number of fatal accidents that their use
might prevent. While it may not be possible to develop quantitative
performance measures for all interventions proposed by the implementation
teams, good performance measures depend on having measurable events, a way
to collect data on those events, and an event that occurs with sufficient
frequency between evaluations for progress to be measurable. The performance
measures for both general aviation weather and CFIT could be improved where
possible by identifying and quantifying baseline information, ensuring that
a means exists for collecting data on the performance measure, and setting
interim and long-term goals against which to measure progress in
implementing the intervention.
Cabin Safety Interventions
The Safer Skies cabin safety steering committee completed work on four
safety problems and implemented most interventions without developing any
strategy for evaluating the interventions. Although the steering committee
completed its work in January 1999, it did not develop performance measures
for the interventions it selected. While the initiative's broad goal is
reducing the fatal accident rate, the broad goal for cabin safety is
educating the flying public about four areas: passenger interference with
flight crews, passenger use of seat belts, child restraint systems, and
carry-on baggage. The steering committee distributed brochures about
carry-on baggage and the importance of child restraint systems and worked
with air carriers to develop additional cabin announcements to remind
passengers to use their seat belts. The team did not, however, set up any
evaluation to show whether the public's knowledge about these issues
improved as a result of these interventions and whether that improved
knowledge results in fewer fatalities.
While useful performance measures could be defined in each of the four cabin
safety areas, the steering committee did not develop a strategy for
evaluating the impact of its educational initiatives. For example, the
steering committee did not plan or track the distribution of the flyers it
issued about carry-on baggage or child restraint systems, and it developed
no performance measures for evaluating the effectiveness of these
initiatives to educate the public. Furthermore, FAA does not have a
mechanism for consistently collecting data about any of these areas.
Airlines are required to report information related to cabin safety only if
something happens in the cabin that results in serious injuries or death. As
a consequence, the agency does not have baseline data for measuring
improvements that may result from its initiatives. Thus, the Safer Skies
initiative has no way of measuring the effectiveness of its educational
efforts in the cabin safety area.
Performance Measures in Other Safer Skies Areas
Precursors to accidents have the potential for use as performance measures
for evaluating interventions to address at least three other Safer Skies
safety problems: CFIT, runway incursions, and approach and landing.
Precursors are needed because fatal aviation accidents caused by all three
safety problems occur rarely. The precursors for each safety problem have at
least some of the attributes of good performance measures.
Navigational Alerts Could Serve as a Performance Measure for One CFIT
Intervention
The Safer Skies team working on CFIT accidents in commercial aviation is
considering using an accident precursor to evaluate the effectiveness of one
of its interventions: the installation of enhanced navigational equipment on
aircraft that sounds alerts to warn pilots of impending crashes. The
equipment tracks data on the frequency of the alerts and the situations in
which they occur. Although these data are not currently collected by FAA,
they could be used to develop a performance measure based on the alerts
sounded as precursors to CFIT accidents. The performance measure of alerts
sounded could indicate the number of dangerous situations avoided.
Alerts sounded by this navigational equipment have several features of a
good performance measure. First, the alerts can be measured. Second, the
equipment itself tracks such warnings. Finally, the alerts are sounded with
sufficient frequency to be useful as a performance measure. According to the
manufacturer, enhanced navigational equipment was installed in over 4,000
aircraft from March 1996 through December 1999. In 14 instances, the alerts
enabled pilots to recover from impending crashes.
Runway Incursion Incidents Could Serve as a Performance Measure
Runway incursion incidents that do not result in accidents provide another
useful performance measure and are being used as such by FAA. From 1988
through 1997, 2,345 runway incursions resulted in five fatal accidents and
59 fatalities in the United States.37 However, runway incursions have the
potential to cause much greater numbers of fatalities; the collision of two
large aircraft on the ground in the Canary Islands in 1977 resulted from a
runway incursion and took more than 580 lives. Because runway incursion
incidents are increasing in the United States and have the potential to lead
to fatal accidents, FAA's Performance Plan for FY 2000 has used these
incidents to establish a performance measure for a series of safety
recommendations designed to reduce accidents caused by runway incursions.
The Safer Skies team addressing runway incursions has not yet identified
interventions, but FAA's ongoing work offers some useful performance
measures for measuring progress in addressing this safety problem.
Runway incursion incidents have all three features of a good performance
measure. First, the incidents can be counted. Second, the data can be
collected because FAA already has a mechanism for reporting runway
incursions.38 Moreover, FAA has collected data on them for years, and
therefore has historical data that can be used to establish baselines
against which the effectiveness of interventions intended to reduce runway
incursions can be measured. For example, one intervention now in use by FAA
involves deploying action teams to airports that have experienced high
numbers of runway incursion incidents to determine the causes and develop
action plans to resolve them. Data on runway incursion incidents can be used
to determine whether the use of action teams reduces such incidents at the
airports in question. Finally, runway incursion incidents occur with
sufficient frequency to make it possible to measure progress between
evaluations. Several hundred runway incursion incidents have been reported
each year this decade.
Unstable Approaches Could Serve as a Performance Measure for Approach and
Landing
The Safer Skies team working on approach and landing accidents in commercial
aviation is considering using an accident precursor to evaluate the
effectiveness of training and other related interventions. The team
determined that unstable aircraft approaches to airports were clearly
precursors to many approach and landing accidents.39 Several problems can
contribute to unstable approaches, including excess speed on approach,
aircraft flaps not in position, and an approach that is too steep or too
shallow. Data on each of these key aspects are recorded on an aircraft's
flight data recorder. Thus, the team has an opportunity to develop a
performance measure based on reducing the number of unstable approaches.
Unstable approaches have some features of good performance measures. First,
they are measurable. Second, data on them can be obtained from flight data
recorders. However, there are barriers to obtaining these data that must be
overcome before unstable approaches can be used as a performance measure for
approach and landing interventions. Finally, unstable approaches occur
frequently enough to measure progress resulting from interventions.
Accident Precursors as Performance Measures
Barriers exist to using some accident precursors as performance measures.
For example, the use of unstable approaches as a performance measure depends
on access to information from aircraft flight data recorders. While some
airlines use data from flight recorders to analyze the causes of safety
problems on routine flights, there are barriers to sharing this information
with FAA or with other airlines. Logistical barriers include (1) the limited
information tracked by older flight data recorders still in use and (2)
differences in the ways that air carriers have programmed flight data
recorders to track key information. Because of these differences, the kinds
of data items needed to track unstable approaches are not being captured
with enough consistency for this measure to be a good indication of
performance throughout commercial aviation.
Other potential barriers also prevent the use of unstable approaches as a
performance measure. Among these barriers are the ongoing debate about how
data from flight recorders are to be shared, who should have access to these
data, and whether legal enforcement cases can be initiated on the basis of
these data. Numerous major aviation safety reports in this decade have
advocated a program that would gather and analyze information from flight
data recorders about routine flights. FAA has for years promised to
establish such a program.40 However, the inability of FAA, the aviation
industry, and other federal agencies to reach consensus on key aspects of
this program has delayed its finalization. While shared data can move safety
forward, concerns about potential litigation, criminal indictments, and the
violation of an air carrier employee's privacy have served as barriers to
the establishment of the program. Such concerns have also delayed the
finalization of other programs to enhance the sharing of aviation safety
data. For example, safety reports have for years recommended the
establishment of Aviation Safety Action Programs to encourage voluntary
self-reporting of safety violations by pilots; FAA issued an advisory
circular providing guidance for these programs on March 17, 2000.41
Most Safer Skies teams have not finished analyzing the causes of the safety
problems they are working on and have not yet selected interventions to
prevent the problems. Thus, these teams have not developed methods to
evaluate the effectiveness of their interventions. But when interventions
have been selected, most have been implemented without first determining how
to evaluate their effectiveness. Neither FAA nor the aviation industry will
have the information that will be critical in determining whether the
interventions have made progress in resolving the safety problems until
appropriate performance measures are developed. Evaluating the impact of
safety interventions depends on having good baseline data on the extent of
the problem prior to the implementation of the intervention, explicit short-
and long-term goals against which to measure progress, and performance
measures that are clearly linked to the safety problem being addressed. In
addition, as Safer Skies teams select interventions to address the safety
problems that caused fatal aviation accidents, it would be useful to
identify clearly any existing barriers to the development of performance
measures. These barriers include differences in aircraft equipment and the
absence of needed data. Once such problems are clearly identified, FAA and
the aviation industry can work jointly to resolve them.
To improve the ability to determine the effectiveness of Safer Skies
interventions, we recommend that the Secretary of Transportation direct the
FAA Administrator to work with the Safer Skies steering committees to direct
the teams to identify the extent of fatal accidents resulting from the
safety problems they are working on. If possible, data should be developed
to establish a consistent baseline against which to measure the progress
that results from the Safer Skies initiative. If an analysis team has
already completed its report, the implementation team working on the same
safety problem should develop these baseline data. More specifically, to
better measure progress toward overall safety goals, we recommend that the
FAA Administrator work with the Safer Skies steering committees to revise
the implementation guidance to (1) develop an overall performance measure or
measures to determine progress toward eliminating the safety problem the
team is addressing; (2) consider using accident precursors as performance
measures for the safety problem in question; and (3) identify any barriers
that may impede the implementation of performance measures.
DOT and FAA officials agree in principle with the need for baseline data on
the extent of each safety problem and performance measures to determine
progress toward overall safety goals. They concur with the potential of
accident precursors as possible performance measures and with the importance
of identifying any barriers that may impede the implementation of
performance measures.
Coordination Has Been Extensive but Needs
Improvement for the Safer Skies Initiative to Succeed
FAA coordinated extensively with numerous representatives from the aviation
industry, other federal agencies involved in aviation safety, and its own
staff on the identification of safety problems and the selection of
interventions. However, efforts to prioritize, fund, and evaluate Safer
Skies initiatives could be better coordinated with industry and within FAA
and the Department of Transportation (DOT). Joint government-industry
efforts to improve safety are not new, but participants noted that the
initiative was more inclusive than prior joint efforts. This inclusive
approach should help FAA gain consensus on which interventions will best
address aviation safety problems. However, our review identified three
coordination problems that could undermine the implementation and evaluation
of Safer Skies interventions. First, although FAA officials have repeatedly
committed to funding interventions agreed upon by all parties working on the
initiative, skepticism still exists among some participants as to whether
this commitment can or will be honored. This is particularly true in general
aviation. It also remains unclear what process will be used, if funding is
limited, to reprioritize available resources to ensure funding for
interventions that emerge later but have greater potential for reducing the
fatal accident rate. Finally, Safer Skies steering committees, FAA, and DOT
have not coordinated how they will measure progress in achieving the
accident reduction goal for commercial aviation.
Coordination Between Industry and Government
FAA included aviation experts from a wide range of government and industry
organizations on the Safer Skies steering committees and the teams working
on the 16 safety problems. Many participants represent groups that are
directly responsible for the nation's aviation safety, such as the air
carriers and the manufacturers of aircraft and engines. Other participants
come from trade associations that represent various aviation groups or from
federal agencies that share responsibility for aviation safety, including
the Department of Defense and the National Aeronautics and Space
Administration. In addition, while giving priority initially to reducing the
U.S. accident rate, the initiative recognized the increasingly global nature
of aviation. In an effort to address both domestic and worldwide aviation
safety problems, the commercial aviation steering committee included
representatives from two international aviation authorities, the Joint
Aviation Authorities and the International Civil Aviation Organization.
Joint efforts between industry and government officials to study aviation
safety problems are not new. In prior years, government and industry
convened various joint teams to review aviation safety issues and make
recommendations; however, according to Safer Skies participants, those
earlier teams did not always include representatives from major
organizations who were responsible for aviation safety. As a result, FAA was
not always successful in obtaining consensus on the safety interventions
that those teams recommended. Safer Skies participants noted that the level
of participation and cooperation for this initiative is unprecedented among
the major groups responsible for aviation safety and should enhance FAA's
chances of implementing the safety interventions made by the various teams.
Moreover, the initiative coordinated ongoing aviation safety work that was
being conducted independently by FAA, industry, and other federal agencies.
For example, aircraft manufacturers had initiated an exhaustive study on
ways to prevent uncontained engine failure. FAA eventually joined the
aircraft manufacturers in this study, and it subsequently became part of the
Safer Skies agenda. In addition, the industry and FAA had been conducting
independent studies on runway incursions and CFIT. Under the initiative,
representatives from the aircraft manufacturers, airline industry, and
government are members of the teams studying 16 safety problems, and
together they will decide on the strategies to address them.
Interventions Could Be Better Coordinated
While coordination between government and industry organizations
participating in the initiative has been extensive, we identified three
areas in which coordination could be improved. First, although FAA has
committed to funding interventions approved by the Safer Skies steering
committees, uncertainty remains about the agency's ability to fund these
safety interventions. The steering committees for commercial aviation and
general aviation have both sought commitment to the implementation and
funding of interventions before giving final approval to move forward.
However, FAA's commitment has come at different points in the approval
process for interventions recommended by these steering committees, and
FAA's commitment to the general aviation interventions was still uncertain
even after some industry and FAA officials believed the steering committee
had given its final approval. As a consequence, general aviation
participants were more skeptical about whether FAA would implement or fund
their safety interventions. Second, it remains unclear what process will be
used to reprioritize available resources if funding is limited. Finally,
Safer Skies steering committees, FAA, and DOT have not coordinated how they
will measure Safer Skies' progress in achieving the goal of reducing the
fatal accident rate in commercial aviation by 80 percent by 2007.
Skepticism persists about whether FAA can or will be able to honor its
commitments to fund the interventions approved by the Safer Skies steering
committees to reduce the fatal accident rate. This is especially true in the
general aviation community. This skepticism results partly because the
process for approving and funding Safer Skies interventions has worked
differently for general aviation than it has for commercial aviation thus
far. This has contributed to differing perceptions about the likelihood of
the funding and implementation of interventions. These perceptions have
resulted in part from the different processes used by the two steering
committees to seek approval and funding from participating organizations,
from the way interventions have moved forward within these two Safer Skies
committees, and from FAA's handling of the interventions recommended by
them.
The Process for Final Approval of Interventions Has Worked Differently in
the Two Steering Committees
The final approval of recommended safety interventions has worked
differently in the commercial aviation and general aviation steering
committees. The commercial aviation steering committee has documented its
process for approving interventions, which involves members' gaining the
approval of their respective organizations for both implementation and
funding. This approval comes in two stages. First, steering committee
members brief their respective organizations on the general concept of each
intervention under consideration and seek preliminary approval of each
intervention. Changes and modifications may be suggested by the
organizations. For organizations that will be involved in the implementation
of an intervention, the preliminary approval also involves a tentative
commitment to fund the cost of implementing any interventions for which they
are responsible. Once members grant preliminary approval, the steering
committee asks the team to draw up detailed implementation plans for each
intervention. These implementation plans are then submitted to the steering
committee for the next level of approval. Members subsequently seek final
approval of these plans from the organizations they represent, including
firm resource and funding commitments if appropriate. When participating
organizations concur with the detailed implementation plans, the steering
committee grants final approval. To date, most of the commercial aviation
teams have forwarded a few interventions at a time for final approval by the
steering committee, rather than complete lists of interventions to address
multiple aspects of complex safety problems, such as CFIT. Thus, when the
commercial aviation steering committee has given its final approval for an
intervention, members interviewed told us they assumed that the intervention
had a high priority and that implementation would take place because the
organizations responsible for implementation had already committed both the
staff and funding needed.
In contrast, the general aviation steering committee had not documented its
process for approving interventions at the time of our review, although it
recently developed draft procedures, according to FAA's response to our
draft report.42 Furthermore, commitment to provide resources for them is
still pending, although some members of both FAA and industry who serve on
the steering committee understood that final approval had been given to the
interventions chosen to address CFIT and weather. Once these two
implementation teams submitted their draft reports to the general aviation
steering committee, the steering committee asked members to have their
organizations review and comment on each intervention. This process resulted
in preliminary approval or disapproval of the concept of each intervention,
in some cases after the intervention was modified. Organizations responsible
for the implementation of interventions also were expected to give a
tentative commitment to fund the cost of their implementation. The steering
committee then asked the teams to develop detailed implementation plans for
each intervention and to submit those for its final approval. These two
teams recommended and developed plans for a total of 17 interventions, many
of which involve subinterventions and will require substantial resources
either in the form of staff or funding from FAA. Because of the number and
potential cost of interventions contained in the two general aviation
reports, FAA requested that the general aviation steering committee
prioritize the interventions. The general aviation steering committee
prioritized the interventions in the letter that transmitted the final CFIT
and weather reports to the FAA Administrator in March 2000. Unlike the
commercial aviation teams, which have presented one intervention at a time
to the steering committee, the general aviation teams have presented their
complete series of interventions for each safety problem.
As the general aviation CFIT and weather reports moved toward final
approval, however, confusion arose. Some industry and FAA participants
believed that these reports had received final approval. This perception is
supported by a March 22, 2000, letter from the industry and FAA co-chairs of
the general aviation steering committee transmitting to the FAA
Administrator the final CFIT and weather implementation reports with their
detailed implementation plans. The letter and accompanying reports
identified high-priority interventions for immediate implementation. These
participants were concerned because FAA was still undecided which
interventions would actually be implemented and funded. In contrast, FAA's
informal written comments in response to our draft report state that final
approval has not been given to either implementation report and depends on
the completion of detailed implementation plans by the FAA offices
responsible for carrying out the implementation. According to the Director
of Aircraft Certification, confusion arose because some members of the
steering committee had "misperceptions" about what levels of approval had
been agreed to.
FAA's Internal Review and Funding Process for Safer Skies Interventions Has
Led to Some Uncertainty About Whether Some Interventions Will Be Funded
FAA's internal review and funding process for Safer Skies interventions has
led to uncertainty about whether some interventions will be funded, in part
because interventions forwarded by the commercial aviation and general
aviation steering committees have been handled somewhat differently thus
far. Like the other organizations participating in the initiative, FAA must
commit its own resources to the interventions that it is responsible for
implementing. In October 1999, FAA formed an executive council to help
coordinate the implementation of the agency's safety agenda, including how
to provide funding and staff resources for Safer Skies interventions. The
executive council includes the heads of each of FAA's major program offices,
its general counsel, and a regional administrator. The executive council has
not yet documented its process for approving and funding interventions,
however, and it remains unclear at what point FAA is committing resources to
implement Safer Skies interventions. This uncertainty has led to different
perceptions on the part of some FAA and industry participants about the
likelihood that interventions will be implemented and funded.
FAA staff working on the initiative described differences in the way the
executive council has handled interventions proposed by the two steering
committees. These differences have resulted in a clear indication of funding
for commercial aviation interventions before that steering committee's final
approval is given, while the general aviation steering committee's final
approval was given on a series of weather and CFIT interventions that have
yet to be approved and funded by FAA. When proposed Safer Skies
interventions are under serious consideration by the steering committees,
they are also presented to the executive council for discussion of their
possible impact on workload and budget, according to FAA staff who serve as
co-chairs of the two steering committees. The executive council provides
feedback to the steering committees before interventions are approved. FAA
staff serving on Safer Skies committees presented conflicting views,
however, of when FAA commits to funding interventions. Several of the FAA
staff interviewed said that FAA's commitment of staffing and funding to
commercial aviation interventions occurs before that steering committee
gives its final approval to interventions.
However, the Director of FAA's Aircraft Certification Service, who serves as
co-chair of the commercial aviation steering committee, described the
executive council's role as having more room for interpretation of the
intervention and a subsequent determination of whether funding is available.
She said that, once the intervention is approved, the executive council
again discusses it, determines whether to accept it as stated or to modify
it, assigns it to an FAA office for implementation, and determines how it
fits in with the office's existing priorities. The program office then
reviews the intervention, can suggest modifications that will achieve the
same goal, and determines whether the intervention can be accomplished with
existing resources or requires a request for additional funding. She said
that the executive council could also request that the steering committee
modify or prioritize interventions. For example, she said that FAA agreed to
implement the commercial aviation CFIT team's recommendation to develop
precisionlike airport approaches,43 concluded that the agency's resources
would not permit the completion of approaches for all airports in the time
frame envisioned by the intervention, and is now working with the steering
committee to identify which airports present the greatest risks and should
be completed first. Similarly, she said that the council asked that the
general aviation steering committee approve a different way to accomplish
one intervention without hiring additional staff and prioritize its list of
CFIT and weather interventions according to which ones will have the most
impact on improving safety and reducing fatalities. Because the executive
council's role is new and its procedures remain undocumented, confusion
persists about when FAA commits its resources to implementing the safety
interventions approved by the steering committees. For example, although
FAA's executive safety council had agreed in principle to the highest
priority interventions to address general aviation safety problems caused by
weather and CFIT, FAA's response to our draft report indicated that final
approval and funding depend on the completion of detailed implementation
plans. As a consequence, several Safer Skies participants from FAA and
industry, especially those working on general aviation issues, expressed
some concern about whether the recommended interventions would be funded or
implemented.
These concerns stem partly from FAA's past record for implementing safety
recommendations. FAA's budget does not specifically identify and commit
resources to implementing Safer Skies interventions. For example, FAA has no
funds set aside in its budgets for fiscal years 2000 or 2001 for general
aviation interventions. However, FAA's Deputy Associate Administrator for
Regulation and Certification said that the agency's approach to budgeting is
to retain flexibility by not identifying specific budget amounts for such
efforts as the Safer Skies initiative. While we do not advocate including
specific Safer Skies line items in FAA's budget, the uncertainty about
funding and implementation also exists because FAA has either not fully
funded or not implemented some safety recommendations in the past. Several
industry participants in the initiative specifically mentioned concerns
about FAA's lack of follow through on safety recommendations to decrease the
number of runway incursions. Although FAA has received many recommendations
for reducing runway incursions, continuing problems in this area have been
partially attributable to insufficient funding of the safety plans FAA
developed, according to DOT's Inspector General.44 Additionally, after
initially planning to fund the agency's new inspection system,45 FAA has
still not provided funding to hire analysts to review inspection data on the
nation's 10 major airlines for possible safety concerns. While FAA has
implemented many safety recommendations over the years, concerns still
persist about the agency's ability to fund new safety initiatives. Greater
assurance about the implementation of Safer Skies interventions could be
provided in two ways. First, as mentioned in chapter 3, stronger mechanisms
for tracking the implementation of interventions from all three steering
committees need to be established. Second, clarifying FAA's process for
committing resources for implementing interventions would provide greater
assurance of their implementation. Both of these steps would improve
coordination between FAA and other Safer Skies participants.
Thus far, the interventions approved by steering committees have not
required a major commitment of time and resources by either FAA or industry
groups. But future interventions may require substantial resources not
included in FAA's current budget, and choices may have to be made about
which interventions to fund. Furthermore, FAA addresses and funds many
issues beyond those on the Safer Skies agenda, including security issues and
improvements to the air traffic control and airport infrastructure. FAA's
executive council provides a forum for agency managers to discuss and
prioritize program and resource needs. However, without clear priorities and
a unified aviation safety agenda that also takes such issues into account,
FAA will continue to address aviation piecemeal, rather than as an
integrated system. While the Safer Skies initiative represents a major step
in the direction of coordinating the nation's aviation safety agenda, a more
far-reaching effort has not yet been undertaken to coordinate the nation's
complete aviation agenda.
Interventions to Ensure the Implementation of Those With the Greatest
Potential to Reduce the Fatal Accident Rate
The initiative has not developed a process for prioritizing interventions to
ensure the implementation of those with the greatest potential to reduce the
fatal accident rate if funding is limited. The initiative has involved
prioritization at several points thus far. First, the teams addressing
safety problems in commercial aviation and general aviation have prioritized
the interventions they considered. For example, the general aviation weather
team considered numerous possible safety interventions and eventually
developed a list of 17 that it presented in order of priority. The steering
committees have also prioritized interventions. For example, the commercial
aviation steering committee has moved quickly on several interventions that
the CFIT implementation team considered as having a high priority and
potential for effectiveness. At the request of the executive council, the
general aviation team created a unified list to prioritize its CFIT and
weather interventions. Given the constraints of FAA's budget, such
prioritization is critical to ensuring that funds are expended on the
interventions that will be most effective in reducing the fatal accident
rate.
The ability to reprioritize resources for Safer Skies interventions and
other aviation work may also become critical. The Safer Skies team has just
begun work on loss of control--the safety problem that caused the greatest
number of fatal accidents in commercial aviation in 1988-97. Interventions
to address loss of control are thus likely to be critical for reducing the
fatal accident rate. If funding is limited, this may mean reprioritizing
funding from existing programs and Safer Skies interventions that have
already been approved to those with more potential to reduce the fatal
accident rate and save lives. The initiative's success will depend in part
on its ability to identify those interventions with the most potential
impact and to prioritize their implementation and funding. Safer Skies
steering committees and FAA's executive council have not yet established any
process for reprioritizing interventions if funding is limited.
Measuring Progress in Reducing Commercial Aviation's Fatal Accident Rate
A lack of coordination among Safer Skies steering committees, FAA, and DOT
has resulted in their having different ways of measuring whether the goal of
reducing the fatal accident rate for commercial aviation by 80 percent is
achievable by 2007. DOT is responsible for setting safety goals for all
modes of transportation under its authority, including aviation. Generally,
FAA and other agencies under DOT have established specific goals and use
measurements that evaluate their progress in meeting those goals that are in
line with those set by DOT. But currently, DOT and FAA measure progress
toward the goal of an 80-percent reduction in the fatal accident rate for
commercial aviation in different ways. DOT's Performance Plan for fiscal
year 2001 establishes goals for reducing the fatal accident rate in
commercial aviation that rely on the Safer Skies initiatives. To determine
the progress made in reducing the rate, DOT's plan uses aircraft flight
hours as the activity measure. In contrast, the commercial aviation steering
committee and FAA use aircraft departures as the measure of aviation
activity. Because DOT, FAA, and Safer Skies all share a common goal of
reducing the fatal accident rate, consistency would be desirable in the
aviation activity measure they use to calculate the progress being made
toward that goal. Since most commercial aviation accidents occur during
takeoff and landing, we believe that using departures would better measure
the effectiveness of the Safer Skies interventions for commercial aviation.
Additional steps need to be taken to ensure that those safety interventions
most critical to reducing the nation's fatal accident rate are given top
priority and funding. If FAA's process for prioritizing and funding Safer
Skies interventions is not clarified, there is no assurance that the agency
will be able to implement these interventions. If funding is limited, a
process may well be needed for reprioritizing available staffing and funding
to ensure that the interventions with the greatest potential for reducing
the nation's fatal accident rate and saving lives are implemented first.
Even if Safer Skies steering committees and FAA agree on the priorities for
the nation's safety agenda, these priorities will continue to compete for
resources with other aviation needs until FAA develops a unified aviation
agenda. Finally, FAA, the Safer Skies commercial aviation steering
committee, and DOT are not using the same aviation activity measure to
calculate the progress of Safer Skies interventions in reducing the fatal
accident rate for commercial aviation. Consequently, they may reach
different conclusions about the effectiveness of the Safer Skies
interventions in achieving the goal of reducing the fatal commercial
aviation accident rate by 80 percent by 2007.
To ensure the implementation of the Safer Skies safety interventions, we
recommend that the Secretary of Transportation direct the FAA Administrator
to clarify the executive council's process for committing to the funding and
implementation of interventions and coordinate with the Safer Skies steering
committees about the meaning and timing of this commitment. To ensure that
the interventions with the greatest potential for reducing the fatal
accident rate and improving aviation safety receive needed resources, we
recommend that the Secretary of Transportation direct the FAA Administrator
to ensure that the executive council has a process in place for
reprioritizing interventions if funding is limited.
To ensure that the extent of progress toward reducing the fatal accident
rate for commercial aviation is measured consistently, we recommend that the
Secretary of Transportation ensure that DOT, FAA, and the Safer Skies
commercial aviation steering committee all use departures as the activity
measure for calculating the rate.
DOT and FAA officials concurred with our recommendations to clarify the
executive council's process for committing to the funding and implementation
of interventions and to use departures as the activity measure for
calculating the fatal accident rate in commercial aviation. They disagreed
with our recommendation that FAA's executive council should develop a
process for reprioritizing interventions if funding is limited. The
officials said that such reprioritization falls under the agency's normal
processes for reprogramming funding. However, the role of the executive
council is to help coordinate the implementation of the agency's safety
agenda--including how to provide funding and staff resources for Safer Skies
interventions. We believe that it would be useful for the executive council
to establish some basic criteria and processes for evaluating and comparing
the potential impact of existing and emerging safety interventions. For this
reason, we did not modify or withdraw our recommendation.
GAO Contacts and Staff Acknowledgments
Gerald L. Dillingham, (202) 512-2834
Robert White, (202) 512-2834
In addition to those named above, Leslie Albin, Bonnie A. Beckett, Fran
Featherston, David K. Hooper, Christopher M. Jones, and Phillis L. Riley
made key contributions to this report.
Key Aviation Studies
AOPA Air Safety Foundation. 1998 Nall Report: Accident Trends and Factors
for 1997.
Frederick, MD.: 1998.
Boeing Commercial Airplane Group. Statistical Summary of Commercial Jet
Airplane Accidents: Worldwide Operations, 1959-1991. Seattle: June, 1992.
Federal Aviation Administration. Zero Accidents…A Shared
Responsibility. Feb. 9, 1995.
Federal Aviation Administration and The Aerospace Industries Association.
Technical Report on Propulsion System and Auxiliary Power Unit (APU) Related
Safety Hazards.
June 25, 1999.
Federal Aviation Administration. FAA 90 Day Safety Review. Sept. 16, 1996.
Federal Aviation Administration, Office of Aviation Research. General
Aviation Accidents, 1983-1994: Identification of Factors Related to
Controlled-Flight-Into-Terrain (CFIT) Accidents. July 1997.
Flight Safety Foundation. Controlled Flight Into Terrain Education and
Training Aid, Vols. 1 and 2.
Flight Safety Foundation. Killers in Aviation: FSF Task Force Presents Facts
About Approach-and-Landing and Controlled-Flight-Into-Terrain Accidents.
Nov.-Dec. 1998, Jan.-Feb. 1999.
The MITRE Corporation. Reports by Airline Pilots on Airport Surface
Operations: Part 1. Identified Problems and Proposed Solutions for Surface
Navigation and Communications. McLean, VA.: May 1994.
The MITRE Corporation. Reports by Airline Pilots on Airport Surface
Operations: Part 2: Identified Problems and Proposed Solutions for Surface
Operational Procedures and Factors Affecting Pilot Performance. McLean, VA.:
Mar. 1996.
The MITRE Corporation. Reports by Airport Traffic Control Tower Controllers
on Airport Surface Operations: The Causes and Prevention of Runway
Incursions. McLean, VA.: Sept. 1998.
National Civil Aviation Review Commission. Avoiding Aviation Gridlock and
Reducing the Accident Rate: A Consensus for Change. Dec. 11, 1997.
U.S. Department of Transportation, Office of Inspector General. Federal
Aviation Administration's Runway Incursion Program (AV-1998-015).
Washington, D.C.: Dec. 8, 1997.
U.S. Department of Transportation, Office of Inspector General. Runway
Incursion Program, Federal Aviation Administration (AV-1998-075).
Washington, D.C.: Feb. 9, 1998.
U.S. Department of Transportation, Office of Inspector General. Follow-Up
Review of FAA's Runway Safety Program (AV-1999-114). Washington, D.C.: July
21, 1999.
U.S. Department of Transportation, Research and Special Programs
Administration. Investigation of Controlled Flight Into Terrain, Vols. 1 and
2. Mar. 1996.
U.S. Department of Transportation, Research and Special Programs
Administration. Investigation of Controlled Flight Into Terrain,
Descriptions of Flight Paths for Selected Controlled Flight into Terrain
(CFIT) Aircraft Accidents, 1985-1997. Mar. 1999.
White House Commission on Aviation Safety and Security. Final Report of the
White House Commission on Aviation Safety and Security. Feb. 12, 1997.
Aviation Safety: FAA's New Inspection System Offers Promise, but Problems
Need to Be Addressed (GAO/RCED-99-183, June 28, 1999).
Aviation Safety: FAA Has Not Fully Implemented Weather-Related
Recommendations (GAO/RCED-98-130, June 2, 1998).
Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA in
Identifying and Responding to Risks (GAO/RCED-98-6, Feb. 27, 1998).
Aviation Safety: FAA Oversight of Repair Stations Needs Improvements
(GAO/RCED-98-21, Oct. 24, 1997).
Aviation Safety: Efforts to Implement Flight Operational Quality Assurance
Programs (GAO/RCED-98-10, Dec. 2, 1997).
Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing
Safety Recommendations (GAO/RCED-96-193, Sept. 23, 1996).
(348180)
Table 1: Number of Fatal Accidents and Deaths by Type of
Aviation Operation, 1988-97 10
Table 2: Fatal Accidents and Deaths by Type of Aviation
Operations, 1988-97 22
Table 3: Projected Numbers of Fatal Accidents by Type of
Operation in 2007 26
Table 4: The 16 Safety Problems Addressed by Safer Skies 31
Table 5: Numbers of Fatal Accidents and Fatalities in Commercial
Aviation 1988-97 41
Table 6: Number of Past and Projected Fatal Accidents by Type
of Aviation Operation 44
Table 7: Status of Safer Skies' Activities for Commercial and
General Aviation 58
Table 8: Baselines and Goals for Reducing the Occurrence of
Uncontained Engine Failure in Commercial Aviation 76
Figure 1: Commercial Aviation and General Aviation Flight
Hours, 1982-99 21
Figure 2: Fatal Accident Rates for Commercial and General
Aviation, 1982-99 25
Figure 3: Safety Problems That Caused Fatal Accidents in
U.S. Commercial Aviation, 1988-97 37
Figure 4: Fatalities in U.S. Commercial Aviation by Safety
Problem, 1988-97 43
1. The fatal accident rate is calculated by dividing the number of fatal
accidents by a measure of aviation activity such as the number of aircraft
hours flown.
2. Commercial aviation includes both large air carrier operations and
smaller commuter operations. General aviation includes a wide variety of
aircraft, ranging from corporate jets to small piston-engine aircraft as
well as helicopters, gliders, and aircraft used in operations such as
firefighting and agricultural spraying. In establishing accident reduction
goals, FAA and Safer Skies steering committees used 1994-96 as the baseline
years for commercial aviation and 1996-98 as the baseline years for general
aviation.
3. Loss of control, which refers to accidents in which the pilot should have
maintained or regained control of the aircraft but did not, is the largest
cause of fatal large air carrier accidents in the United States. Controlled
flight into terrain, which refers to flying an otherwise controllable
aircraft into the ground or water, is the leading cause of fatal large air
carrier accidents worldwide. Approach and landing refers to situations in
which a crash occurs during the approach to the airport or attempt to land
when the pilot might have been able to land safely but did not.
4. These problems, which resulted in a total of nine fatal accidents,
include weather, uncontained engine failure, and runway incursions.
Uncontained engine failure occurs when a heavy engine part rotating at high
speed cracks and breaks out of the engine housing. In two U.S. accidents,
engine parts have breached the body of the aircraft resulting in fatalities.
Runway incursions are occurrences at a towered or nontowered airport,
involving an aircraft, vehicle, or pedestrian within the runway safety area,
that creates a real or potential collision hazard with an aircraft taking
off, intending to take off, or landing or intentding to land.
5. The Joint Safety Strategy Initiative includes members from European
aviation manufacturers, associations, and regulators.
6. Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing
Safety Recommendations (GAO/RCED-96-193 , Sept. 23, 1996).
7. Commercial aviation includes both large air carriers and commuter air
carriers. Specifically, commercial aviation includes all air carriers
offering scheduled and nonscheduled service by major air carriers flying
under 14 C.F.R. part 121 and all air carriers operating scheduled service
under 14 C.F.R. part 135. General aviation aircraft include all
U.S.-registered civil aircraft not operated under 14 C.F.R. part 121 or part
135. General aviation includes a wide variety of aircraft, ranging from
corporate jets to small piston-engine aircraft as well as helicopters,
gliders, and aircraft used in operations such as logging, firefighting, and
agricultural spraying. General aviation also includes on-demand air carriers
that operate nonscheduled service under 14 C.F.R. part 135.
8. We used the 85 fatal accidents in commercial aviation for the purpose of
illustration. When calculating fatal accident rates in aviation, the
National Transportation Safety Board (NTSB) excludes accidents that resulted
from sabotage and hijacking. In 1988-97, one fatal U.S. accident resulted
from sabotage and would thus be excluded. The number of accidents that NTSB
would use to calculate the fatal accident rate for the period is 84
accidents.
9. FAA uses an annual survey to estimate flight hours because it does not
require general aviation operators to report such key measures as the number
of hours flown or the number of takeoffs. The General Aviation and Air Taxi
Activity Survey provides FAA with information on the operations of these
aircraft.
10. Since March 20, 1997, aircraft with 10 or more seats formerly operating
scheduled service under 14 C.F.R. part 135 have been required to follow the
more stringent safety requirements that apply to larger aircraft under 14
C.F.R. part 121.
11. FAA and Safer Skies steering committees have chosen different baseline
years for commercial aviation (1994-96) and general aviation (1996-98). We
have used those years in table 3 projecting the number of fatal aviation
accidents and in subsequent tables in this report comparing the accident
reduction goals chosen by FAA and Safer Skies steering committees with the
80-percent goal set forth by the White House and congressional commissions
on aviation safety.
12. CFIT is flying an otherwise controllable aircraft into the ground or
water.
13. Uncontained engine failure occurs when a heavy engine part rotating at
high speed cracks and breaks out of the engine housing. In two U.S.
accidents, engine parts breached the body of the aircraft. One accident on
takeoff resulted in the death of 2 passengers, while the other accident
crippled key aircraft systems in flight, resulting ultimately in a crash
that killed 111 passengers and crew.
14. Flight Safety Foundation, Controlled Flight Into Terrain: Education and
Training Aid (Disseminated under the sponsorship of the U.S. Department of
Transportation.).
15. Zero Accidents … A Shared Responsibility, U.S. Dept. of
Transportation, FAA, Feb. 9, 1995.
16. FAA 90 Day Safety Review, U.S. Dept. of Transportation, FAA, Sept. 16,
1996.
17. Final Report to President Clinton, White House Commission on Aviation
Safety and Security, Feb. 12, 1997.
18. Avoiding Aviation Gridlock and Reducing the Accident Rate: A Consensus
for Change, National Civil Aviation Review Commission, Dec. 11, 1997.
19. Prior to joining the Safer Skies initiative, the Joint Safety Coalition
was known as the General Aviation Coalition.
20. This group was called Partners in Cabin Safety.
21. Other participating groups include, for example, National Air
Transportation Association, the Helicopter Association International, and
the National Business Aviation Association.
22. In one of these two accidents, a private general aviation aircraft
collided in flight with a commuter aircraft, which landed safely. The other
accident involved the on-board fatality of a pilot who entered an
unpressurized area of a cargo aircraft. The co-pilot landed the aircraft
without further incident.
23. General Aviation Accidents, 1983-1994: Identification of Factors Related
to Controlled-Flight-Into Terrain (CFIT) Accidents, U.S. DOT, FAA
(DOT/FAA/ARR-100-97-2, July 1997).
24. Runway Incursion Program, DOT, FAA (AV-1998-075, Feb. 9, 1998).
25. The general aviation steering committee used preliminary NTSB data to
compute the average number of fatal accidents for 1996-98, which resulted in
an annual average of 379. In table 6 and throughout our report, we used the
official NTSB accident statistics for 1999, which were released after the
steering committee's projections. This resulted in a slightly higher annual
average of 380 fatal accidents in general aviation.
26. Both accidents resulted in the death of a passenger who was not secured
by a seat belt when the aircraft encountered turbulence. The commercial
aviation steering committee lists weather as the safety problem that caused
these two accidents.
27. The Joint Safety Strategy Initiative includes members from European
aviation manufacturers, associations, and regulators.
28. The Director of the Aircraft Certification Service co-chairs the
commercial aviation steering committee.
29. FAA issues airworthiness directives to address unsafe mechanical
conditions that surface after an aircraft has been certified and in use. The
directives contain FAA's requirements for airlines to correct unsafe
aircraft conditions that have occurred or are likely to occur in aircraft of
the same design.
30. Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing
Safety Recommendations (GAO/RCED-96-193 , Sept. 23, 1996).
31. Follow-up Review of FAA's Runway Safety Program, DOT, FAA, (AV-1999-114,
July 21, 1999).
32. Federal Aviation Administration: Aviation Safety, DOT Inspector General
Report No. AV-1999-069 (Statement of Alexis M. Stefani, Deputy Assistant
Inspector General for Aviation, U.S. Department of Transportation before the
Subcommittee on Transportation and Related Agencies, Committee on
Appropriations, U.S. House of Representatives, Mar. 10, 1999).
33. DOT FY 2001 Performance Plan: FY 1999 Performance Report.
34. These two programs are the Flight Operational Quality Assurance program
and the Aviation Safety Action Program. Both programs facilitate the
collection, protection, and analysis of safety data voluntarily submitted by
airlines or pilots.
35. Severe consequences (level 4 events) include fatal or serious injury,
loss of the aircraft hull, and forced landing of the aircraft. Serious
consequences (level 3 events) include substantial damage to the aircraft or
an unrelated system, uncontrolled fire, rapid cabin depressurization,
temporary or permanent inability to climb or fly the aircraft 1,000 feet
above terrain, and temporary or permanent impairment of the aircraft's
controllability.
36. Disks are heavy, high-speed rotating parts inside an engine with
attached fan blades that produce thrust. Undetected manufacturing flaws or
contaminants can undermine a disk's structural integrity, allowing a crack
to occur. If a crack causes a disk to fail and break apart, fast-moving
fragments of the disk can disable or damage the airplane and may have
catastrophic results.
37. Five runway incursions involving commercial aircraft occurred in the
United States during this period. In classifying fatal accidents, the
commercial aviation steering committee classified one of these accidents as
an approach and landing accident for purposes of analysis because one plane
landed on top of another plane.
38. Nonetheless, runway incursions are now underreported because FAA does
not keep statistics on runway incursions at airports without towers because
there are no air traffic controllers present to report the incidents.
39. Unstable approaches can lead to loss of control, landing short of the
runway, and overrunning the runway, among other problems. An approach can
become unstable for any of the following reasons: late air traffic control
clearance to descend, late notification of the landing runway, late
selection of the landing configuration, rapidly changing weather, poor prior
planning by the crew, or a pilot's misjudgment of the circumstances.
40. The program that FAA has promised to implement, the Flight Operational
Quality Assurance program, uses flight data to detect technical flaws,
unsafe practices, or conditions outside of desired operating procedures
early enough to allow timely intervention to avert accidents or incidents.
41. An Aviation Safety Action Program is a partnership program between FAA
and the aviation industry to encourage voluntary self-reporting by pilots of
safety violations. The program provides some protection from enforcement
actions for inadvertent violations in return for valuable data that can be
used to analyze safety problems.
42. We did not have an opportunity to review the general aviation steering
committee's draft procedures.
43. Precisionlike approaches improve aviation safety by enabling all flight
crews and aircraft to fly a stabilized vertical path to the runway end for
all instrument approach procedures.
44. Federal Aviation Administration: Aviation Safety, DOT Inspector General
Report No. AV-1999-069 (Statement of Alexis M. Stefani, Deputy Assistant
Inspector General for Aviation, U.S. Department of Transportation before the
Subcommittee on Transportation and Related Agencies, Committee on
Appropriations, U.S. House of Representatives, Mar. 10, 1999).
45. We presented our findings and recommendations about the new inspection
system, the Air Transportation Oversight System, in Aviation Safety: FAA's
New Inspection System Offers Promise, but Problems Need to Be Addressed
(GAO/RCED-99-183 , June 28, 1999).
*** End of document. ***