Aviation Safety: Efforts to Implement Flight Operational Quality
Assurance Programs (Letter Report, 12/02/97, GAO/RCED-98-10).
Pursuant to a congressional request, GAO provided information on: (1)
how the Federal Aviation Administration's (FAA) and U.S. airlines'
Flight Operational Quality Assurance (FOQA) programs will enhance
aviation safety; (2) the costs and benefits of such programs; and (3)
the factors that could impede their full implementation and actions that
could be taken to overcome any impediments.
GAO noted that: (1) the early experience of domestic airlines with
established FOQA programs, as well as the testimony of foreign airlines
with extensive experience in this area, attests to the potential of such
programs to enhance aviation safety by identifying possible safety
problems that could lead to accidents; (2) airlines have used FOQA
programs to identify potential problems that were previously unknown or
only suspected; (3) where potential problems where already known,
airlines have used these programs to confirm and quantify the extent of
the problems; (4) on the basis of analyses of flight data, airlines have
taken actions to correct problems and enhance aviation safety; (5) costs
associated with implementing a FOQA program depend on a large number of
factors, including the technology used to capture flight data, the
number and types of aircraft to be equipped with this technology, and
personnel costs; (6) although the program is primarily viewed as a
safety program, U.S. and foreign airlines have reported financial
benefits as well; (7) with additional data on aircraft systems and
engine conditions, airlines are better able to achieve optimum fuel
consumption and avoid unneeded engine maintenance; (8) enhanced safety
should result in lower costs over time as a result of accidents avoided
and lower insurance premiums; (9) FAA's estimates suggest a net savings
from 50 aircraft of $892,000 per year; (10) the primary factor impeding
the implementation of FOQA programs among the major domestic carriers is
the resolution of data protection issues; (11) airline managers and
pilots raise three significant data protection concerns: (a) use of data
for enforcement and disciplinary purposes; (b) disclosure to the media
and the public under the provisions of the Freedom of Information Act;
and (c) disclosure through the civil litigation discovery process; (12)
FAA has taken a number of actions that may resolve these issues,
although it is not clear whether the aviation community will be
satisfied with FAA's actions; (13) FAA has begun work on a rulemaking
procedure to establish what protections from enforcement actions, if
any, will apply to information submitted to FAA under a FOQA program;
(14) Congress enacted legislation, and FAA has begun work on a
rulemaking procedure, that would prohibit the Administrator from
disclosing voluntarily submitted safety information under certain
circumstances; and (15) airlines seek to protect voluntarily collected
safety information from disclosure in civil litigation on a case-by-case
basis.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-98-10
TITLE: Aviation Safety: Efforts to Implement Flight Operational
Quality Assurance Programs
DATE: 12/02/97
SUBJECT: Transportation safety
Aircraft accidents
Airline industry
Pilots
Airline regulation
Aircraft maintenance
Information disclosure
Cost effectiveness analysis
Litigation
Air transportation operations
IDENTIFIER: FAA Flight Operations Quality Assurance Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
December 1997
AVIATION SAFETY - EFFORTS TO
IMPLEMENT FLIGHT OPERATIONAL
QUALITY ASSURANCE PROGRAMS
GAO/RCED-98-10
Flight Operational Quality Assurance Programs
(341514)
Abbreviations
=============================================================== ABBREV
APMS - Aviation Performance Measuring System
AQP - Advanced Qualification Program
ASAP - Aviation Safety Action Programs
ASRP - Aviation Safety Reporting Program
ASRS - Aviation Safety Reporting System
BA - British Airways
DEMOPROJ - FOQA demonstration project
DFDAU - digital flight data acquisition unit
FAA - Federal Aviation Administration
FAR - Federal Aviation Regulation
FOIA - Freedom of Information Act
FOQA - Flight Operational Quality Assurance
FSF - Flight Safety Foundation
GAIN - Global Analysis and Information Network
KLM - Royal Dutch Airlines
NASA - National Aeronautics and Space Administration
PCMCIA - Personal Computer Memory Card International Association
QAR - quick access recorder
UTRS - Universal Technical Resource Services, Inc.
Letter
=============================================================== LETTER
B-275990
December 2, 1997
The Honorable Wendell H. Ford
The Honorable Ron Wyden
United States Senate
The analysis of aircraft data recorded during flight has played a
crucial role in determining the causes of crashes. Recently,
however, some U.S. airlines have begun to analyze flight data from
uneventful airline flights to identify potential problems and correct
them before they lead to accidents. In your letter of December 2,
1996, you asked us to examine efforts by the Federal Aviation
Administration (FAA) and U.S. airlines to implement Flight
Operational Quality Assurance (FOQA) programs. The objective of a
FOQA program is to use flight data to detect technical flaws, unsafe
practices, or conditions outside of desired operating procedures
early enough to allow timely intervention to avert accidents or
incidents. These programs are voluntary efforts by airlines that
involve equipping aircraft with specialized devices to continuously
record up to hundreds of different flight data parameters from
aircraft systems and sensors, analyzing the data, identifying trends,
and taking action to correct potential problems. The analysis of
flight data allows airlines to reconstruct entire flights on the
basis of the values over time of flight data parameters, such as
heading, altitude, throttle settings, ground speed, and many others.
Currently, about 33 foreign airlines and 4 U.S. airlines--United, US
Airways, Continental, and Alaska--have implemented FOQA or FOQA-type
programs.
You requested that we determine (1) how FOQA programs will enhance
aviation safety, (2) the costs and benefits of such programs, and (3)
the factors that could impede their full implementation and actions
that could be taken to overcome any impediments.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
The early experience of domestic airlines with established Flight
Operational Quality Assurance programs, as well as the testimony of
foreign airlines with extensive experience in this area, attests to
the potential of such programs to enhance aviation safety by
identifying possible safety problems that could lead to accidents.
Airlines have used Flight Operational Quality Assurance programs to
identify potential problems that were previously unknown or only
suspected. Where potential problems were already known, airlines
have used these programs to confirm and quantify the extent of the
problems. And most important, on the basis of analyses of flight
data, airlines have taken actions to correct problems and enhance
aviation safety.
The costs associated with implementing a Flight Operational Quality
Assurance program depend upon a large number of factors, including
the technology used to capture flight data, the number and types of
aircraft to be equipped with this technology, and personnel costs.
Although the program is primarily viewed as a safety program, U.S.
and foreign airlines have reported financial benefits as well. With
additional data on aircraft systems and engine conditions, airlines
are better able to achieve optimum fuel consumption and avoid
unneeded engine maintenance. Although more difficult to quantify,
enhanced safety should result in lower costs over time as a result of
accidents avoided and lower insurance premiums. FAA's preliminary
estimates place the annual cost of a program with 50 aircraft at
approximately $760,000. Savings from reduced expenditures for fuel,
engine maintenance, and accident costs for a 50-aircraft program are
estimated at $1.65 million per year. FAA's estimates suggest a net
savings from 50 aircraft of $892,000 per year.
The primary factor impeding the implementation of Flight Operational
Quality Assurance programs among the major domestic carriers is the
resolution of data protection issues. Airline managers and pilots
raise three significant data protection concerns: (1) use of the
data for enforcement/disciplinary purposes; (2) disclosure to the
media and the public under the provisions of the Freedom of
Information Act; and (3) disclosure through the civil litigation
discovery process. FAA has taken a number of actions that may
resolve these issues, although it is not clear whether the aviation
community will be satisfied with FAA's actions. First, FAA has begun
work on a rulemaking procedure to establish what protections from
enforcement actions, if any, will apply to information submitted to
FAA under a Flight Operational Quality Assurance program. Second, on
October 9, 1996, the Congress enacted legislation and FAA has begun
work on a rulemaking procedure that would prohibit the Administrator
from disclosing voluntarily submitted safety information under
certain circumstances. These actions may ameliorate concerns about
the Freedom of Information Act. And third, airlines currently seek
to protect voluntarily collected safety information from disclosure
in civil litigation on a case-by-case basis.
BACKGROUND
------------------------------------------------------------ Letter :2
Modern commercial aircraft contain sophisticated electronic systems
that gather, process, and manage digital data on many aspects of
flight. These data originate from various systems and sensors
throughout the aircraft. The data range from pilot operations to the
outputs of sensors and systems. Some of these data are continuously
recorded by the aircraft's digital flight data recorder to help
investigators understand what happened if the aircraft is involved in
an accident or a serious incident.\1 Designed to survive crashes,
flight data recorders typically retain the data recorded during the
last 25 hours of flight.
Rather than analyzing flight data only after an incident or accident,
some airlines routinely analyze the flight data from regular flights.
Their aim is to identify problems that occur in routine operations
and to correct these problems before they become accidents or
incidents. In its 1992 study for FAA,\2 the Flight Safety Foundation
coined the term "Flight Operational Quality Assurance" to describe
this function. The Foundation defined FOQA as "a program for
obtaining and analyzing data recorded in flight to improve
flight crew performance, air carrier training programs and
operating procedures, air traffic control procedures, airport
maintenance and design, and aircraft operations and design."
FOQA has its origin in the use of flight data recorders\3 as mandated
by the Civil Aeronautics Administration in 1958. Although the first
flight data recorders captured only six parameters,\4 they were a
valuable tool for reconstructing what had occurred preceding a crash.
In addition to recording data to assist in crash investigations, some
airlines began to monitor data recorded on routine flights.
Initially, the monitoring systems captured airworthiness data, but
over time they have expanded to include operational data. FOQA
programs were first established in Europe and Asia, and only within
the past few years have some U.S. airlines begun adopting such a
system on a trial basis. At present, about 33 foreign airlines and 4
U.S. airlines--United, US Airways, Continental, and Alaska--have
implemented FOQA or FOQA-type programs. (See app. I for more
detailed background information on FOQA and U.S. airlines'
experience with these programs; see app. II for a list of airlines
worldwide that have implemented FOQA programs.)
As part of FAA's strategy to achieve significant reductions in
aviation accident rates despite the rapid increase in air travel
anticipated over the next decade, in 1995 the agency initiated a FOQA
demonstration project to promote the voluntary implementation of FOQA
programs by U.S. airlines. The objective of such a program is to
use flight data to detect technical flaws, unsafe practices, or
conditions outside of desired operating procedures early enough to
allow timely intervention to avert accidents or incidents. For
example, identifying repeated instances of unstabilized approaches to
a particular airport could help to define a new approach pattern less
likely to lead to an accident under adverse conditions, or to
improved pilot training. Such a system has potentially broad
application to flight crews' performance and training, aircraft
operating procedures, air traffic control procedures, aircraft
maintenance, and airport design and maintenance. Major airlines in
Europe and Asia, as well as the U.S. airlines that have FOQA
programs, are uniform in their support of the program.
How FOQA Works. FOQA involves (1) capturing and analyzing flight
data to determine if the pilot, the aircraft's systems, or the
aircraft itself deviated from typical operating norms; (2)
identifying trends; and (3) taking action to correct potential
problems. Airlines with FOQA programs typically use a device called
a quick access recorder to capture flight data onto a removable
optical disk that facilitates the data's frequent removal from the
aircraft.\5 Periodically, the optical disks are removed from the
aircraft, and the flight data are analyzed by the ground analysis
system at a centralized location. The data are analyzed by a
computer system that evaluates about 40 to 80 predefined events for
deviations from the airline's specified tolerance thresholds. For
example, an event might be the descent rate during approach.
Deviations of more than certain predetermined values, called
"exceedances," are flagged and evaluated by a monitoring team. After
investigating these exceedances to determine their validity and
analyzing them to understand possible causes, the monitoring team
will propose and evaluate corrective actions. Periodically, airlines
aggregate exceedances over time to determine and monitor trends.
(For a more complete discussion of FOQA operations, see app. I.)
The FOQA Demonstration Project. In July 1995, FAA initiated a
3-year, $5.5 million demonstration project to facilitate the start-up
of voluntary airline FOQA programs and to assess the costs, benefits,
and safety enhancement associated with such programs. FAA provided
hardware and software to each of the three airlines--United, US
Airways, and Continental--that have implemented FOQA programs
according to the demonstration project's requirements. FAA purchased
quick access recorders to equip 15 Boeing 737 aircraft at each of the
three airlines. FAA also purchased a ground analysis system--the
computer hardware and software for analyzing and visualizing FOQA
data--for US Airways and Continental. Because United already had
purchased a ground analysis system that analyzes these data for other
types of aircraft, FAA purchased for the airline the additional
software needed to analyze FOQA data from 737s. For their part,
these airlines funded the cost of obtaining supplemental type
certification\6 of the airborne equipment, the costs of installation
and maintenance, and the cost of personnel to run and monitor the
program. Alaska Airlines is the fourth U.S. airline to have begun a
FOQA program, but it has only recently met the demonstration
project's requirement for an agreement on FOQA by the airline's pilot
union. Consequently, the project has not yet provided any equipment
to the airline. Alaska Airlines, however, received quick access
recorders and a ground analysis system from the FAA Structural Loads
Program and uses this equipment to operate its FOQA program. (See
app. III for more information on the Structural Loads Program.)
Other airlines that are participating in the demonstration project
and are considering the implementation of a FOQA program are America
West, Delta, Northwest, Trans World, Southwest, Continental Express,
and United Parcel Service.\7 (See app. I for a detailed description
of the FAA demonstration project.) As a research and development
effort of the FOQA initiative, FAA is developing the Aviation
Performance Measuring System, an advanced system for conducting
automated analysis and research on FOQA data. (See app. III for a
description of this system and FAA's other related technical
programs.)
Rather than requiring airlines to implement FOQA, FAA has chosen to
promote the initiative through a cooperative demonstration project in
partnership with the industry. According to the demonstration
project's program manager, it would be premature for FAA to mandate
FOQA at this time because U.S. aviation is still in the early stages
of developing FOQA and is primarily in a learning mode. The program
manager contends that a mandated program would stifle innovation,
encounter substantial resistance from airlines and pilots, and most
likely result in minimal compliance. Thus, at present, FAA is
working with the industry to raise interest in the concept,
facilitate the design and implementation of voluntary FOQA programs,
provide financial and technical assistance, and foster innovation.
--------------------
\1 The National Transportation Safety Board, the official source of
information on airline accidents, defines accidents as events in
which individuals are killed or suffer serious injury, or the
aircraft is substantially damaged. Incidents are defined as
occurrences other than accidents associated with the operation of an
aircraft that affect or could affect the safety of operations. 49
C.F.R. 830.2.
\2 Flight Safety Foundation, Air Carrier Voluntary Flight Operational
Quality Assurance Program (1992).
\3 The flight data recorder is commonly referred to as the "black
box."
\4 The six required parameters were time, airspeed, heading,
altitude, vertical acceleration, and time of radio transmission.
\5 These data typically include the parameters required to be
collected on the aircraft's flight data recorder plus many more
parameters. See app. I for more information on quick access
recorders and flight data recorders.
\6 An FAA type certificate is issued when an aircraft, aircraft
engine, propeller, or appliance is properly designed and
manufactured, performs properly, and meets the regulations and
minimum standards prescribed by the Administrator. An FAA
supplemental type certificate is required when there is a change to
an aircraft, aircraft engine, propeller, or appliance. 49 U.S.C.
44704.
\7 Although not a participant in the demonstration project, American
Airlines is considering the implementation of an internal FOQA-type
program.
FOQA IDENTIFIES POTENTIAL
SAFETY PROBLEMS
------------------------------------------------------------ Letter :3
The primary characteristic that distinguishes FOQA from other safety
reporting programs, such as the Aviation Safety Reporting Program or
Aviation Safety Action Programs,\8 is that FOQA provides objective,
quantitative data on what occurs during flights rather than what is
subjectively reported by individuals. Instead of needing to rely on
perceived problems or risks, FOQA yields precise information on many
aspects of flight operations, and this information can be used to
help objectively evaluate a wide range of safety-related issues.
U.S. and foreign airlines have reported on previously unknown or
suspected problems for which FOQA has provided objective information
that resulted in corrective actions. One airline found through its
FOQA program that more exceedances occurred during visual flying than
during instrument flying. This finding prompted the airline's
flight-training managers to rethink the relative emphasis given
visual and instrument flying in the airline's training programs.
Another airline's FOQA analysis confirmed that the incidence of
descent rate exceedances during approaches was significantly higher
at a particular runway at a U.S. airport than at other runways.
After investigating the problem, the airline concluded that the air
traffic control approach had been set too high, requiring pilots to
descend more steeply than usual during their final approach. When
the airline shared its findings with FAA management, the approach was
modified to correct this potential problem.
For landings, some airports' air traffic control procedures require
pilots to approach high and fast and then descend steeply. These
approaches can result from a number of factors, including noise
abatement rules, traffic volume, terrain, or weather conditions.
Although airline managers know about the situations from pilots'
reports, FOQA gives them the quantitative information to demonstrate
the extent of this problem at the various airports. With these data
in hand, managers can be more effective in addressing the problem and
taking action to mitigate or eliminate risks.
FOQA can also help airlines determine the frequency of certain
occurrences rather than having to rely on human judgment,
particularly for the level of maintenance required. Two examples of
these types of occurrences are hard landings and exceedances in
engine temperatures. Prior to FOQA, airlines generally relied on
pilots' judgment of the necessity for corrective action if a hard
landing occurred or an engine overheated. FOQA, however, can provide
better information on the amount of force the aircraft experienced
during a hard landing. Similarly, FOQA gives more data on the
engines' temperatures and the duration of overheating in some
aircraft than were previously available without FOQA. With these
data, managers can make more informed decisions about whether the
aircraft needs to be inspected to check for structural damage or
whether an engine needs to be overhauled.
U.S. and foreign airlines have reported that they have used FOQA
analysis to identify a variety of potential safety problems and take
corrective action to resolve or mitigate them. These have included
steep takeoffs, which can damage the aircraft's tail; approaches that
are outside the prescribed procedures for a "stabilized" approach;
descent rates or bank angles that are considered excessive; high taxi
speeds; hard landings; wind shear occurrences; ground proximity
warnings; and engine malfunctions. Corrective action can include
notifying pilots of a change in standard operating procedures or
restating and emphasizing them, correcting an equipment problem, or
providing additional training. The continued monitoring of trends
will tell the airline if the corrective action has been effective or
whether additional measures are needed.
A number of airlines plan to complement the use of FOQA data with
information from safety reporting systems, such as Aviation Safety
Action Programs or internal pilot reporting systems. FOQA data,
originating from aircraft sensors and systems, tell "what" happened
to the aircraft. Internal safety reporting systems, based on reports
of pilots, flight crews, and other persons, are more likely to tell
"why" something happened. Together, information from FOQA and
internal reporting systems can provide valuable insight into current
and emerging problems.
--------------------
\8 See app. IV for a description of these programs.
FOQA'S POTENTIAL COSTS AND
BENEFITS
------------------------------------------------------------ Letter :4
Based on preliminary estimates from an ongoing cost-benefit study by
Universal Technical Resource Services, Inc. (UTRS), an FAA
contractor, table 1 summarizes the estimated annual costs for
airlines to equip 15, 50, and 100 aircraft with quick access
recorders, purchase a ground analysis system, and pay FOQA-related
salaries.\9
Table 1
Estimated Total Annual Costs, by Fleet
Size
15 aircraft 50 aircraft 100 aircraft
---------------------------- ------------ ------------ ------------
Equipment costs $98,500 $259,000 $492,000
Personnel costs 385,000 500,000 775,000
======================================================================
Total annual costs $483,500 $759,000 $1,267,000
----------------------------------------------------------------------
Note: Equipment costs are based on the invoice price paid to vendors
in the FOQA demonstration project. To annualize the figures, the
equipment purchase costs have been spread over a 5-year period.
Personnel costs are based primarily on estimates of FOQA management,
analysis, monitoring, and engineering costs from an airline
participating in the demonstration project.
Source: UTRS.
The cost-benefit study estimates that airlines will reduce their
expenditures for fuel and maintenance as well as reduce the number of
accidents and incidents over time, avoiding their associated costs.
Because FOQA programs analyze additional data on aircraft systems and
engine conditions, airlines are better able to achieve optimum fuel
consumption and avoid unneeded engine maintenance. Although more
difficult to quantify and directly relate to a FOQA program, enhanced
safety should result in lower costs over time as a result of
accidents avoided and lower insurance premiums. Table 2 summarizes
the estimated annual savings for fleet sizes of 15, 50, and 100
aircraft. Fuel savings and engine savings figures are based on
estimates of a 0.5-percent reduction in fuel consumption and a
1-percent reduction in engine maintenance costs. The safety savings
figure is based on a hypothetical 1-percent reduction in the annual
costs incurred from accidents. FAA's contractor based its safety
savings calculation on a current loss rate of 2 aircraft per million
departures at a cost of $150 million for each loss.
Table 2
Estimated Total Annual Savings, by Fleet
Size
15 aircraft 50 aircraft 100 aircraft
---------------------------- ------------ ------------ ------------
Fuel savings $145,800 $486,000 $972,000
Engine savings 300,000 1,000,000 2,000,000
Safety Savings 49,500 165,000 330,000
======================================================================
Total annual savings $495,300 $1,651,000 $3,302,000
----------------------------------------------------------------------
Note: Fuel and engine savings were estimated on the basis of
discussions with an airline participating in the FOQA demonstration
project. Safety savings were estimated on the basis of information
from a European airline with a long-term FOQA program. Savings
estimates were also based on an assumption of 3,000 flight hours per
aircraft per year.
Source: UTRS.
According to these annual cost and savings estimates, FOQA would
result in net annual savings of $11,800 for 15 aircraft, $892,000 for
50 aircraft, and $2,035,000 for 100 aircraft. See table 3.
Table 3
Estimated Net Annual Savings, by Fleet
Size
15 aircraft 50 aircraft 100 aircraft
---------------------------- ------------ ------------ ------------
Total annual costs $483,500 $759,000 $1,267,000
Total annual savings 495,300 1,651,000 3,302,000
======================================================================
Net annual savings $11,800 $892,000 $2,035,000
----------------------------------------------------------------------
Source: UTRS.
--------------------
\9 Because FAA's cost-benefit study is in progress, we were not able
to verify FAA's estimates of FOQA costs and savings. The cost and
savings figures are preliminary and may change as more data are
gathered.
FACTORS IMPEDING IMPLEMENTATION
AND ACTIONS TO OVERCOME
IMPEDIMENTS
------------------------------------------------------------ Letter :5
Although airline officials, pilot organizations, and FAA officials
recognize the potential for improving safety and operations through
FOQA programs, airline officials and representatives of the pilot
organizations were unanimous in their view that data protection
issues need to be resolved. Both airline officials and pilots'
representatives stated that the lack of protections for FOQA data has
been a major contributor to pilot unions' reluctance to sign FOQA
agreements with airlines and airlines' reluctance to implement FOQA
programs.
According to the Flight Safety Foundation's 1992 report, the greatest
impediment to the implementation of FOQA in the United States is
associated with the "protection of data from use for other than
safety and operational improvement purposes." Basically, airline
managers and pilots have three concerns: (1) that the information
may be used in enforcement/discipline actions, (2) that such data in
the possession of the federal government may be obtained by the
public and the media through the provisions of the Freedom of
Information Act (FOIA), and (3) that the information may be obtained
in civil litigation through the discovery process. Similar concerns
have been expressed in connection with other programs under which
information is submitted voluntarily to FAA.
ENFORCEMENT
---------------------------------------------------------- Letter :5.1
Representatives from each of the major airlines as well as the unions
that represent pilots from the major airlines--the Air Line Pilots
Association, the Allied Pilots Association, the Independent
Association of Continental Pilots, and the Southwest Airlines Pilot
Association--told us that the airlines and pilots fear the
possibility that FOQA data might be used against them in FAA
enforcement proceedings. In addition to these concerns, pilots'
representatives were concerned that airline managers could use FOQA
data to punish or discipline pilots.
FAA Enforcement. Many U.S. airlines and their pilots appear
frustrated with FAA's delay in issuing a regulation implementing the
nonenforcement policy articulated in a February 1995 policy letter
from the Administrator to the Air Line Pilots Association and the Air
Transport Association. FAA's letter said that no enforcement action
will be taken on the basis of the information gained through FOQA.
Specifically the letter stated:
"The FAA commits that it will not use information collected by a
carrier in an FOQA program to undertake any certificate or other
enforcement action against an air carrier participating in such
a program or one of its individual employees. Notwithstanding,
the FAA reserves its rights to use, for any other purpose,
information obtained from sources other than FOQA, including
flight-recorder parameters specifically required by the Federal
Aviation Regulations. The limitation on the use of information
applies only to information collected specifically in an FOQA
program."
In an April 1997 letter to the Air Transport Association's FOQA
Steering Committee, the Director of FAA's Flight Standards Service
said that the 1995 policy letter will remain in effect until the
regulation on enforcement is issued. The letter stated that a
proposed rulemaking setting forth FAA's enforcement protection policy
should be ready by the end of 1997.
According to airline officials and a pilot union's representative,
FAA's delay in promulgating an enforcement regulation has hampered
efforts to reach agreement with some pilot unions and threatens the
continuance of agreements already reached. One of the issues facing
FAA is how broad the enforcement protection should be. FAA attorneys
have concluded that it is beyond the scope of FAA's authority and in
violation of its statutory duties to issue a regulation that
precludes the agency from taking action if FOQA data reveal that an
airplane was not in a condition for safe flight or that a pilot
lacked qualifications. Pilots' representatives, however, have cited
the precedent of FAA's cockpit voice recorder regulation that
prohibits the agency from using the record in enforcement actions
without exceptions.\10
FAA officials told us that the agency is trying to find the proper
balance between carrying out its enforcement responsibilities and
providing incentives for implementing safety programs and sharing
information with FAA. In similar programs, such as the Aviation
Safety Reporting Program, Air Carrier Voluntary Disclosure Reporting
Procedures, and Aviation Safety Action Programs,\11 under which
safety information is voluntarily submitted, the agency has a policy
of addressing alleged violations through administrative actions or
forgoing and/or waiving the imposition of any legal enforcement if
certain qualifying criteria are met. These programs are intended to
encourage prompt reporting of violations, sharing of important safety
information, and pilot training to enhance future compliance. While
the qualifying criteria differ for each program, these programs
exclude actions that are deliberate or demonstrate or raise questions
of qualifications. Generally, the parameters of the programs,
including the qualifying criteria, are spelled out in the governing
advisory circular. It is FAA's belief that by offering incentives,
such as forgoing legal enforcement actions under certain conditions,
more problems may be reported and ultimately corrected than could be
discovered through other means, such as inspections.
Airline Enforcement. Airline managers are working with their
respective pilot unions to enter into data-use agreements that
include individual protection provisions. According to the Flight
Safety Foundation study, data-use agreements with pilot associations
have existed since flight data recorders were first required in the
late 1950s. Having such an agreement is a precursor to becoming a
full partner in the FOQA demonstration project. Generally, these
agreements provide, among other things, the company's assurance not
to use the recorded flight data for punitive or disciplinary action
against a crew member, or as evidence in any proceeding. Also, to
ensure the protection of the company's employees, the data-use
agreements generally provide for the de-identification of the
information as soon as possible, usually within 7 days. This
practice ensures the confidentiality and anonymity of the flight crew
members participating in the program.
--------------------
\10 The cockpit voice recorder regulation provides that: "The
Administrator does not use the cockpit voice recorder record in any
civil penalty or certificate action." 14 C.F.R. 91.609(g). FAA's
regulations also provide enforcement protection with some
qualifications to information collected under the Aviation Safety
Reporting Program. Specifically, the regulation provides that "The
Administrator of the FAA will not use reports submitted to the
National Aeronautics and Space Administration under the Aviation
Safety Reporting Program (or information derived therefrom) in any
enforcement action except information concerning accidents or
criminal offenses which are wholly excluded from the Program." 14
C.F.R. 91.25.
\11 See app. IV for a description of these programs.
FREEDOM OF INFORMATION ACT
REQUESTS
---------------------------------------------------------- Letter :5.2
Both airlines and pilots are concerned that FOQA data could become
public and available to the media through the federal FOIA, if such
data are provided directly to FAA.\12 The federal FOIA sets forth a
policy of broad disclosure of government documents to ensure "an
informed citizenry, vital to the functioning of a democratic
society." NLRB v. Robbins Tire & Rubber Co., 437 U.S. 214, 242
(1978). The Congress understood, however, that "legitimate
governmental and private interest could be harmed by release of
certain types of information." FBI v. Abramson, 456 U.S. 615, 621
(1982). Accordingly, the act provides for nine categorical
exemptions.
In the past, safety information voluntarily submitted to FAA, for
example under Air Carrier Voluntary Disclosure Reporting Procedures,
has been protected under exemption 4 of FOIA. Exemption 4 protects
trade secrets and commercial or financial information obtained from a
person that is privileged or confidential. Airline officials and
pilots' representatives expressed concern that FOQA data may not be
protectable under this exemption.
Recently, the Congress enacted the Federal Aviation Reauthorization
Act of 1996, which contains a provision that protects voluntarily
submitted information under certain circumstances. Specifically,
under the provision, notwithstanding any other provision of law, the
Administrator is barred from disclosing voluntarily provided safety-
or security-related information if the Administrator finds that
"(1) the disclosure of the information would inhibit the
voluntary provision of that type of information and that the
receipt of that type of information aids in fulfilling the
Administrator's safety and security responsibilities; and (2)
withholding such information from disclosure would be consistent
with the Administrator's safety and security responsibilities."
49 U.S.C. 40123.\13
The provision also requires the Administrator to issue regulations to
implement the section.
The House report accompanying this legislation noted with approval
the data-sharing programs such as FOQA and the Committee's intent to
encourage and promote these sorts of innovative safety programs. The
report provides that information submitted under these programs would
arguably be protected from release under exemption 4 of FOIA;
however, the report notes that such a decision to withhold the
information would be discretionary with the agency. The report
states that to provide assurance that such information is not
publicly released, the legislation would prohibit FAA from disclosing
voluntarily submitted safety information. According to the report,
this protection should "alleviate the aviation community's concerns
and allow the data-sharing safety programs to move forward."
Moreover, the report noted that the provision would not reduce the
information available to the public, since the public does not
receive the data. Rather, the report states that public safety will
be enhanced by the increase in FAA's understanding of ongoing trends
in operations and technologies.\14
FAA is currently working on a rulemaking procedure that will prohibit
the release of voluntarily submitted safety data through FOIA.\15 It
is expected that the rulemaking will provide the procedures that the
agency will use in making the required determinations. It is also
expected that FOQA data will be proposed as qualifying for the
protection. According to an FAA attorney, the determinations for the
FOQA program may be included in the notice of proposed rulemaking on
the FOQA nonenforcement policy. The anticipated FOIA rulemaking and
the subsequent findings to include the FOQA program within the
protection should help mitigate or resolve the industry's fears about
the possible disclosure of FOQA data through FOIA requests if FOQA
data are provided directly to FAA.
--------------------
\12 Currently, airlines provide no FOQA data to FAA. Rather, FAA
reviews aggregated trend information on the airlines' premises.
\13 A similar provision was included in the National Transportation
Safety Board Amendments of 1996 to protect information that is
voluntarily submitted to the Board.
\14 H.R. Rep. No. 104-714 pt. 1 at pp. 40-41 (1996).
\15 In the Final Report of the White House Commission on Aviation
Safety and Security, dated Feb. 12, 1997, a recommendation was made
that FAA should work with the aviation community to develop and
protect the integrity of standard safety databases that can be shared
in accident prevention programs. The report stated that FAA needed
to expeditiously complete rulemaking to implement the voluntary
disclosure protection provision and that the agency should assess the
adequacy of the new legislative authority and implementing regulation
1 year after the regulations take effect. The report further stated
that any necessary regulatory or legislative modifications identified
at that time should be promptly addressed.
DISCOVERY PROCESS IN CIVIL
LITIGATION
---------------------------------------------------------- Letter :5.3
Some airline officials have told us that although they want to
improve aviation safety by implementing a FOQA program, the voluntary
collection of data may potentially expose airlines to greater
liability in civil litigation. FOQA data may indicate conditions
outside of desired operating procedures. Airline officials and pilot
representatives told us that they are concerned that through broad
discovery rules, FOQA data could be inappropriately used or disclosed
to the public. The general purpose of discovery is to remove
surprise from trial preparation so that parties may obtain the
evidence necessary to evaluate and resolve their dispute.\16 Since
FOQA data are retained at the airlines and are not currently provided
directly to FAA, the focus has been on the airlines' ability to
protect the information.
Under federal rules, parties in litigation in federal court are
authorized to obtain discovery of any matter, not privileged, which
is relevant to the subject matter involved in the pending action,
whether it relates to the claim or defense of the party seeking
discovery or to the claim or defense of any other party. Generally,
privileges are narrowly construed and in some cases are qualified.
However, even in the absence of a privilege, a district court has
broad discretion under the federal rules to issue an order to protect
a person from annoyance, embarrassment, oppression, or undue burden
or expense if there is a good cause for issuance of the order.
Courts generally invoke a balancing test to decide when a protective
order is appropriate and how it is to be applied.\17
In two recent cases, the airlines have tried to convince federal
courts that voluntarily collected safety data similar to FOQA data
should be protected from discovery or, at the very least, covered
under a protective order.\18 In both cases, the courts sought to
achieve a balance between the airlines' desire to protect the
information and the plaintiffs' right to a fair trial. In the first
case, the court rejected a claim that the information should be
protected under the self-critical evaluation privilege but limited
the possible uses of the documents it ordered to be produced.\19 This
determination was effected through a protective order. In the other
case, the court also rejected the claim of self-critical evaluation
privilege but at the same time recognized a new qualified privilege
for information collected under a partnership program with FAA, the
American Airlines Safety Action Program.\20
Although airlines are generally pleased with the court's decision to
grant a qualified privilege to Aviation Safety Action Program
materials, it is not clear whether other courts will recognize this
new privilege or extend it to other safety and security information
that has been voluntarily collected. Nor is there a guarantee that
FOQA data or other similar information, if found not to be
privileged, would be covered under a protective order. However, we
found no instances to date in which FOQA data have been subject to a
discovery request. This situation may result from the fact that
airlines are just beginning to institute FOQA programs. However,
some of the pilot union officials we spoke with noted that discovery
is a concern because of the potentially large amounts of data that
will be collected. While some in the aviation community believe that
one way to ensure protection would be through legislation, there does
not appear to be a consensus to seek legislation at this time.\21
Concern has been expressed that the failure of a legislative effort
may adversely affect how courts treat voluntarily collected safety
information.
In the event that FAA does receive FOQA data directly, according to
FAA attorneys, it has provisions in place for dealing with requests
from private litigants for documents in the agency's possession. FAA
attorneys noted that a request for records from a private litigant,
when the agency is not a party to the action, will generally be
treated as a FOIA request (see 49 C.F.R. 9.13). If the agency is a
party to the litigation, FAA will seek to protect the information, if
appropriate, under a claim of government privilege and, if that
fails, to release the information under a protective order.
--------------------
\16 6 Moore's Federal Practice, section 26.02 (Matthew-Bender 3d
ed.).
\17 6 Moore's Federal Practice, ch. 26 (Matthew-Bender 3d ed.).
\18 Court Order of Oct. 26, 1995, In re: Air Crash at Charlotte,
North Carolina, on July 2, 1994, MDL Docket No. 1041 (D.S.C. 1995)
(the court rejected the claim of self-critical evaluation privilege);
but see, Court Order of Nov. 14, 1995, In re: Air Crash at
Charlotte, North Carolina, on July 2 1994, MDL Docket No. 1041
(D.S.C. 1995) (the court issued a protective order); and In re: Air
Crash Near Cali, Columbia, on Dec. 20, 1995, 959 F. Supp. 1529
(S.D. Fla. 1997) (the court rejected the claim of self-critical
evaluation privilege but recognized a new qualified privilege for the
American Airlines Safety Action Program). For a more detailed
discussion of these court cases, see app. V.
\19 The self-critical evaluation privilege, when recognized, protects
documents that reflect an internal self-analysis.
\20 Thus, the court provided that the plaintiff could come forward
with a persuasive showing of need and hardship. In such case, the
court would review the voluntarily collected information in camera
and evaluate whether the plaintiff's interests overcome the powerful
interest that weighs in favor of preserving the confidentiality of
the information. No such showing was made in this case.
\21 Limited legislative protection has been provided for cockpit
voice recorders, 49 U.S.C. 1154.
AGENCY COMMENTS
------------------------------------------------------------ Letter :6
We provided copies of a draft of this report to the Department of
Transportation and FAA for their review and comment. We met with
officials, including FAA's Deputy Associate Administrator for
Regulation and Certification and the demonstration project's program
manager. They agreed with the report and provided several technical
corrections, which were incorporated into the report.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :7
To obtain the information in this report, we reviewed FAA's FOQA
demonstration project's requirements, policies, and plans to assist
airlines in implementing FOQA programs. We discussed specific
details of the project with FAA's Deputy Associate Administrator for
Regulation and Certification as well as the demonstration project's
program manager and contractor. We conducted interviews with FAA and
National Aeronautics and Space Administration officials responsible
for developing the Aviation Performance Measuring System. We
discussed FOQA issues with the National Transportation Safety Board.
We interviewed representatives of each of the 10 largest passenger
airlines: Alaska, America West, American, Continental, Delta,
Northwest, Southwest, Trans World, United, and US Airways;
representatives of each of the four unions--the Air Line Pilots
Association, the Allied Pilots Association, the Independent
Association of Continental Pilots, and Southwest Airlines Pilot
Association--representing the pilots of these airlines; and United
Parcel Service. We also conducted interviews with the Air Transport
Association, the Flight Safety Foundation, and the vendors providing
hardware and software for the demonstration project. Last, we
interviewed and collected information from foreign airlines and
Britain's Civil Aviation Authority on their respective FOQA efforts.
We performed our work primarily at FAA headquarters in Washington,
D.C., and conducted our evaluation from January through October 1997
in accordance with generally accepted government auditing standards.
---------------------------------------------------------- Letter :7.1
As requested, we plan no further distribution of this report until 30
days after the date of this letter unless you publicly announce the
report's contents earlier. At that time, we will send copies to the
appropriate congressional committees and the Department of
Transportation and FAA. We will also make copies available to others
upon request. Please call me at (202) 512-2834 if you have any
questions about this report. Major contributors to this report are
listed in appendix VI.
John H. Anderson, Jr.
Director, Transportation Issues
THE FOQA CONCEPT AND ITS
IMPLEMENTATION IN THE UNITED
STATES
=========================================================== Appendix I
FOQA'S BACKGROUND
------------------------------------------------------- Appendix I:0.1
Flight Operational Quality Assurance (FOQA) had its origin in the use
of flight data recorders as mandated by the Civil Aeronautics
Administration in 1958. Although the first flight data recorders
captured only six parameters--time, airspeed, heading, altitude,
vertical acceleration, and time of radio transmission--they were a
valuable tool for reconstructing what had occurred before and during
accidents. By the 1960s, airlines had begun to monitor data on
routine flights. Initially, the monitoring systems captured
airworthiness data, but over time they have expanded to include
operational data. In the late 1960s, Trans World Airlines began a
program to monitor a limited number of parameters related to
approaches and landings as flight data recorders received periodic
maintenance.
At least eight foreign airlines have had FOQA-type programs in
operation for over 25 years. A program using data from flight data
recorders was begun by British Airways (BA) in 1962 to validate
airworthiness criteria. Although limited by today's standards, BA's
program contained the seeds of a modern, safety-oriented FOQA
program. Currently, BA analyzes the flight data from all of the
aircraft in its fleet through its Special Events Search and Master
Analysis program. Over the years, the number of foreign airlines
that have implemented a FOQA-type program has steadily risen. Japan
Airlines' FOQA program of over 15 years includes a printer in the
cockpit so that pilots can monitor their own performance. All Nippon
Airways began a program to analyze flight data in 1974. Other
foreign airlines with established FOQA programs include Scandinavian
Airlines System, Royal Dutch Airlines (KLM), and Lufthansa. Many of
these airlines are convinced that FOQA is a critical component in
their safety efforts and that the program has paid valuable safety
dividends over the years. Currently, about 33 foreign airlines and 4
U.S. airlines--United, US Airways, Continental, and Alaska--have
implemented FOQA programs (see app. II for the complete list).
Recognizing the value of operational flight data and the critical
nature of flight crews' performance in accidents, the Flight Safety
Foundation (FSF) proposed and was selected by the Federal Aviation
Administration (FAA) in 1991 to study FOQA. In its 1992 report on
FOQA, FSF said that
"The proposal was based on FSF's conviction, formed by the
positive experiences of its international member airlines using
FOQA, that the appropriate use of FOQA data by airlines, pilot
associations and aircraft and equipment manufacturers would
result in a significant improvement of flight safety by
identifying operational irregularities that can foreshadow
accidents and incidents."
The FSF study concluded that FOQA must proceed in the United States
and that the implementation of FOQA by U.S. airlines would have a
more positive impact on Part 121 operational safety than any other
human factors program included in FAA's research and development
plans. FSF recommended that FAA promote voluntary FOQA programs by
instituting a demonstration program in partnership with industry. In
1992, FAA's Flight Standards Service proposed funding for a
demonstration program. On February 9, 1995, FAA announced its plans
for an FAA-industry demonstration project, and the Administrator sent
a policy letter to the Air Transport Association and the Air Line
Pilots Association stating that FAA would not use FOQA data for
enforcement purposes, provided that the airlines met certain
requirements.
HOW FOQA WORKS
------------------------------------------------------- Appendix I:0.2
At a minimum, FOQA involves the analysis of flight data on a routine
basis to reveal situations requiring corrective actions before
problems occur. To institute such a program, airlines need methods
to capture flight data, transform the data into the appropriate
format for analysis, and generate reports and visualizations to
assist personnel in analyzing the data. Although different methods
are available, the following describes how a representative FOQA
program operates; the descriptions are based on the experience of the
four U.S. airlines that have implemented FOQA.
Management. A typical program is managed and operated by a FOQA
manager, one or more analysts, and a FOQA monitoring team (sometimes
referred to as the exceedance guidance team) made up of airline
pilots who work on FOQA on a part-time basis. Generally, the
majority of the monitoring team's pilots are also representatives of
the pilot union. These individuals manage the FOQA program in strict
adherence to the agreements made with the pilot union, most notably
on ensuring the confidentiality of pilots' identities. This group is
responsible for defining and refining exceedances and parameters,
reviewing and analyzing data, and determining and monitoring
corrective actions.
Data capture. The first step is the capture of data over the
duration of the flight. Flight data comprise snapshots of values or
measurements from various aircraft systems. Each data item
represents information from a discrete source, such as an instrument
or sensor. Generally, these data items are referred to as
"parameters." Examples of parameters are "altitude" or "landing gear
position." Recording rates vary, depending on the parameter, ranging
from many times per second to about once per minute.
Although flight data recorders continuously record, at a minimum,
FAA-mandated parameters during every flight, they typically are not
designed to provide frequent access to their data but rather to
survive the extreme conditions during and after crashes to preserve
flight data for accident investigations. These devices are housed in
crash-resistant, sealed containers designed to withstand high "g"
forces, submersion in water, and fire. Obtaining frequent access to
flight data recorders for FOQA purposes, however, would produce
increased wear on internal mechanisms and result in shortened
mechanical life and increased expense for a very specialized
device.\22 Also, flight data recorders may not capture a sufficient
number of parameters to be useful for FOQA purposes. Currently, FAA
requires from 16 to 29 parameters to be recorded on flight data
recorders in transport aircraft;\23 a FOQA program, however, would
likely capture many more parameters. Typically, the 200-500
parameters available on modern digital aircraft allow a more
comprehensive set of conditions to be monitored. Finally, flight
data recorders hold about 25 hours of flight data, a relatively short
time period. Instead, some U.S. airlines use a device called a
quick access recorder (QAR) to record FOQA data to a removable
optical disk or Personal Computer Memory Card International
Association (PCMCIA) card.\24 QARs record flight data that are output
from the aircraft's digital flight data acquisition unit (DFDAU), the
same device that feeds parameters to the flight data recorder. On
average, QARs hold from 100 to 200 hours of flight data.
Data transfer. As aircraft receive periodic servicing, the medium
(optical disk or PCMCIA card) containing flight data is removed from
the QAR and sent to a central location for analysis. A new disk or
card is inserted into the QAR for the next round of flights.
Airlines retrieve the data on schedules ranging from 3 to 20 days.
An alternative to physical recording media is the use of datalink
systems to transmit information directly to the ground-based system,
eliminating the need to retrieve data from the aircraft. Two
participating airlines are investigating the use of automatic
wireless data transfer upon landing at specially equipped airports.
Data would be transmitted on a radio frequency link from the aircraft
to a receiving station after the aircraft lands. In turn, a local
area network would transfer the data to the ground analysis station.
Data encryption and other methods would be used to ensure the
security of the transmitted FOQA data.
Data processing and analysis. Each airline has a ground analysis
system where airborne collected data are processed and analyzed. The
ground analysis system transforms the raw digital flight records into
usable form for processing, analyzes the flight information, and
generates information on any detected exceedances that represent
deviations from normal operating practices or exceptional conditions.
The flight data analysis component of the ground analysis system
categorizes operational events to be flagged by defining a set of
parameters that indicate normal operating envelopes. The associated
thresholds for these parameters vary by the type of aircraft and
associated operating limits, accepted practices for safe operations,
the phase of flight, and the duration of any irregularity. For
example, the threshold of selected parameters may be defined for
various altitudes, e.g., 1,000, 500, 250, and 100 feet, during
landing mode events. Typically, 40 to 80 events are defined and
analyzed for a particular aircraft. For example, events might be the
ground speed during taxi or the descent rate during approach. The
analysis software will track the descent over time to calculate a
rate in terms of feet per minute. Depending on the aircraft's
altitude, a descent rate in excess of specified thresholds will
trigger an exceedance. Various categorization schemes are used to
classify the seriousness of the exceedance. U.S. airlines use two
or three categories to describe the seriousness of exceedances,
ranging from minor deviations to major deviations. Exceedances are
typically specified on the basis of a strategy for identifying those
that have the greatest potential for safety and performance
considerations. Once the initial exceedance categories and
associated parameters have been defined and utilized, they are
subject to an ongoing evaluation and refinement process.
The ground analysis software also validates the quality and integrity
of the collected data and filters out any marginal or transitory
irregularities. Ground analysis systems also include protective
mechanisms, such as the de-identification of pilot and specific
flight information and user access privileges based on assigned
passwords. As the data are processed, the flight number and day of
the month are removed and saved into a separate controlled file.
This step "de-identifies" the FOQA data.
The FOQA monitoring team investigates each exceedance to determine
what occurred and the severity of the exceedance. An analyst will
review the parameter values surrounding the event and other
information to determine if the exceedance was valid or if the
exceedance was based on bad data, a faulty sensor, or some other
invalidating factor. For example, one flight had excessive rudder
input on landing that correctly registered as an exceedance. On
closer examination, it was determined that because the aircraft was
making a cross-wind landing, the use of large rudder input was
justified. In this case, the exceedance was deemed invalid and was
removed from the exceedance database.
Depending on the particular circumstances of the exceedance, the
pilot association's representative may contact the flight crew to
gather more information. After reviewing the situation to determine
the exceedance's cause, the FOQA monitoring team and pilot
association's representative will determine any necessary corrective
action. Corrective action can range from additional flight crew
training, to revisions of the operating procedures, to redesigns of
equipment.
Trend analysis. On a periodic basis, airlines aggregate and analyze
exceedances over time--for example, the number of unstabilized
approaches at a particular airport per month, over the last 12
months. This type of analysis provides valuable information to the
airline, especially in terms of whether the airline's performance is
improving, holding steady, or deteriorating. This look at aggregate
exceedances over time provides airline managers with a new
perspective on potential problems that would not be visible
otherwise. On the basis of the trend analysis, airline managers can
take corrective action to reduce or eliminate these exceedances by
focusing on the root causes and making or recommending changes.
Data retention. Detailed FOQA data, including exceedances, are
destroyed in 30 days or less by three of the four U.S. airlines with
FOQA programs. Trend data, however, are kept indefinitely.
Aircraft equipping decisions. The U.S. airlines with active FOQA
programs have each equipped a portion of their available fleets with
QARs. They began their programs by equipping their more modern,
technically advanced aircraft with QARs--late-generation aircraft
already contain the sensors and advanced digital systems that acquire
and control many more flight data parameters than earlier-generation
aircraft. Generally, these airlines do not plan to equip any of
their older, analog-based aircraft, such as Lockheed L-1011,
McDonnell Douglas DC-9 and DC-10, and Boeing 727, 737-100, and
737-200, with QARs to record flight data because these aircraft would
be expensive to retrofit and because the airlines plan to retire many
of them in the near future.
Several U.S. airlines plan to equip all new aircraft with QARs or
other technology to capture FOQA data. Some new aircraft, for
example, are delivered with QARs as standard equipment. Airlines
cited several advantages in having new aircraft delivered with
factory-equipped QARs. One advantage is that aircraft are not taken
out of service to be retrofitted with equipment. Another advantage
is that the additional cost of a QAR can be spread over the finance
period of the new aircraft.
Depending on the specific goals of a FOQA program, an airline may
wish to equip some or all of its fleet to collect flight data. If a
program's goal is to identify broad trends in flight operations and
safety, the airline may choose to equip only a portion of its fleet.
If a program's goal, however, is to more closely monitor the flight
operations and performance of individual aircraft, the airline may
want to equip more or all of its fleet. For an airline that begins
by equipping only a portion of its fleet, more aircraft will likely
be added to the program so that these data can be monitored as its
FOQA program matures and efficiency and maintenance functions are
added to the program. Some U.S. airlines, for example, are planning
to use FOQA data to cut aircraft maintenance costs by more closely
monitoring engine conditions and fuel consumption.
--------------------
\22 The newer solid-state flight data recorders, however, have no
moving parts and would not experience wear problems. Transferring
data from these devices takes several minutes to perform.
\23 Under a recently issued rule, FAA requires the recording of 16 to
29 parameters by the flight data recorder on all existing transport
aircraft, depending on the aircraft model, its internal systems, and
its date of manufacture. Aircraft manufactured after the rule,
however, will be required to record 88 parameters within 5 years. 62
Fed. Reg. 38362 (July 17, 1997).
\24 Other airborne data collection systems in use around the world
include QARs using tape cartridges and solid-state devices.
THE FOQA DEMONSTRATION
PROJECT
------------------------------------------------------- Appendix I:0.3
On July 11, 1995, FAA awarded a 2-year contract\25 to execute a FOQA
demonstration project, referred to as DEMOPROJ by FAA, to Universal
Technical Resource Services, Inc. (UTRS), of Cherry Hill, New
Jersey. The contract stated that
"The goal of DEMOPROJ is to facilitate the start-up of the FOQA
initiative and to comprehensively assess the cost-benefits and
safety enhancement effectiveness of an implemented FOQA program
in which airlines voluntarily employ in-flight recorded data to
routinely monitor their flight operations."
UTRS facilitated the establishment of collaborative partnerships
between FAA, UTRS, and interested airlines. Airlines may participate
in DEMOPROJ at one of three levels within the project, ranging from
attending meetings and expressing interest to a full partnership with
FAA. Level 3 participation refers to the airlines that have not yet
established an official FOQA program but attend meetings to learn
about FOQA. At Level 2, the airlines already have their own
equipment or will acquire equipment using airline funding, but they
allow UTRS to monitor and document their program. Level 1 describes
a full partnership in which equipment and software are provided
through DEMOPROJ. Currently, 11 airlines are participating in
DEMOPROJ. The airlines participating at Level 1 are United, US
Airways, and Continental. All other participating airlines in
DEMOPROJ are at Level 3: Alaska, America West, Delta, Northwest,
Trans World, Southwest, Continental Express, and United Parcel
Service.
The airline participants were selected on the basis of a number of
characteristics, including financial stability, management
commitment, resource commitment, fleet characteristics, fleet size,
aircraft availability, and an approved implementation and operation
plan. Additionally, airlines are required to sign nondisclosure and
cooperation agreements that define the treatment of confidential and
proprietary information, enumerate data access control and security
provisions, and specify the responsibilities and contributions of
each party. Participating airlines also had to secure agreements
with their pilot associations for the collection and analysis of
flight data. These airlines made the commitment to record and
process FOQA data on all scheduled flights that are equipped with
FAA-supplied equipment, participate in periodic project reviews, and
allow UTRS to interview airline personnel during the project to
document procedures, problems, issues, and solutions.
UTRS assisted airlines in determining the equipment best suited to
their needs, acquiring the equipment, and delivering it for
installation by the airlines. Hardware and software were selected
from commercially available, off-the-shelf sources. As part of this
effort, the contractor developed an Equipment Overview to facilitate
the airlines' analysis and selection of available equipment.
UTRS also monitors and documents the airlines' FOQA demonstration
programs' policies, procedures, usage, and effectiveness. The
contractor is collecting and analyzing information on how each
airline is implementing FOQA, including data processing and analysis;
the retention of detail and trend data; the selection of flight data
parameters; and the adjustment of threshold values, system
effectiveness, technical problems, and resource information for
establishing and maintaining a FOQA program. These findings are
integrated and disseminated among participants throughout the study.
UTRS is also collecting information about the projects' costs and
anticipated benefits. The contractor is determining how each airline
transforms FOQA data into information and how this information is
used in the airline's decision-making. UTRS holds periodic meetings
for all partners to promote the sharing of information and lessons
learned.
UTRS, with airlines' and pilot associations' involvement, is
developing a FOQA advisory circular to provide information and
guidance to airlines on how to design, implement, and maintain a FOQA
program. This document is scheduled to be issued approximately 90
days after FAA issues its proposed rulemaking on enforcement policy
in connection with FOQA. UTRS is also developing a cost-benefit
analysis that will provide estimates of (1) the costs that an airline
would incur when starting and maintaining a FOQA program and (2)
potential savings. The cost-benefit study is scheduled to be
completed in January 1998.
UTRS will issue a technical report and a set of FOQA guidelines in
June 1998. The technical report will be an overall description of
the technical effort to implement FOQA, summarizing the airlines'
experiences with commercially available equipment and systems. The
FOQA guidelines will synthesize the airlines' experiences in
implementing FOQA with a view toward helping new airlines learn from
the airlines that have implemented a FOQA program. The guidelines
will include information on (1) designing a FOQA program; (2) the
start-up and initial operation of a system; (3) the use of FOQA for
trend analysis, knowledge building, and decision-making; and (4)
critical success factors for implementing a FOQA program.
In fiscal years 1995 through 1997, according to the FAA FOQA program
manager, FAA allocated $5.5 million for DEMOPROJ. The manager stated
that, as of September 26, 1997, DEMOPROJ had expended $2.1 million,
including $1.1 million for the purchase of hardware and software for
the three Level 1 airline participants. FAA plans to pursue
follow-on development focused on the acquisition and use of FOQA
information by FAA for safety monitoring purposes.
--------------------
\25 The term of the contract was later revised to 3 years.
U.S. AIRLINES WITH ACTIVE
FOQA PROGRAMS
------------------------------------------------------- Appendix I:0.4
Currently, four U.S. airlines have active FOQA programs: United
Airlines, US Airways, Continental Airlines, and Alaska Airlines.
These airlines have equipped a number of their aircraft with QARs,
from 7 aircraft at Alaska Airlines to 52 aircraft at United Airlines.
The number of parameters continuously recorded on the QARs range from
about 38 to over 300, depending on the airline and the type of
aircraft.
United Airlines. United Airlines has the largest and longest-running
FOQA program of any U.S. airline, begun in 1995. As of August 1997,
United had 52 aircraft equipped with QARs and had collected FOQA data
on over 25,000 flights. The aircraft currently equipped include
Boeing 737-500s and 777s and Airbus 319s and 320s. United plans to
equip over 120 aircraft by 1999, including all new aircraft currently
on order. DEMOPROJ has funded the purchase of QARs to equip 15
Boeing 737-500s and additional data analysis packages and computer
equipment to run on systems that United had already established. The
remainder of the hardware and software was purchased by United, which
has been tracking and correcting exceedance events for more than a
year. United has identified and taken corrective action to reduce
the incidence of a number of safety- and maintenance-related
exceedances.
US Airways. US Airways has 23 QAR-equipped aircraft. Its program,
begun in September 1996, has collected FOQA data on over 18,000
flights to date. Aircraft equipped include Boeing 737-400s and 767s.
US Airways, however, characterizes its program as being in the data
collection and trouble-shooting phase and just beginning the data
analysis and trending phase. DEMOPROJ has funded the purchase of
QARs to equip 15 Boeing 737-400s and a ground analysis system. Six
additional 737-400s have been equipped with QARs paid for by a
separate FAA program, the Structural Loads Program (see app. III).
In addition to these aircraft, US Airways is in the process of
purchasing QARs and equipping 12 Boeing 767s. Data from all QARs are
being accessed by both programs. DEMOPROJ has also funded a trial
program of a wireless ground datalink system with five specially
equipped Boeing 757s.
Continental Airlines. Continental has equipped 15 Boeing 737-500s
with QARs. In addition, Continental plans to equip with QARs all new
aircraft on order. These include Boeing 737-500s, -600s, -700s, and
-800s as well as a number of 757s. Begun in December 1996,
Continental's program has analyzed the flight data from over 11,000
flights to date. According to the program manager, this program is
in the data collection phase and will soon be making the transition
to the data analysis and trending phase.
Alaska Airlines. Alaska Airlines has equipped six McDonnell Douglas
MD-80s and one Boeing 737-400 with QARs. In addition, Alaska has
equipped a flight simulator with equipment to record hundreds of
flight parameters. Begun in July 1996, the program has analyzed the
flight data from over 5,000 flights to date. Still in the early
stages of the program, Alaska plans to "go slow" and refine its
program. Alaska's FOQA manager said that the airline may eventually
equip every aircraft in its fleet.
Unlike United, US Airways, and Continental, which are Level 1
participants in DEMOPROJ, Alaska is a not yet a full partner in
DEMOPROJ because it has only recently secured the required agreement
with its pilot union on FOQA. The airline, however, has received six
QARs and a ground analysis system from FAA's Structural Loads Program
(see app. III). Alaska uses the equipment and analysis system for
both the Structural Loads Program and FOQA.
U.S. AND FOREIGN AIRLINES WITH
FOQA OR FOQA-TYPE PROGRAMS
========================================================== Appendix II
Adria Airways
Aeroflot
Air Afrique
Air France
Air Inter
Air Liberte
Alaska Airlines
All Nippon Airways
Asiana Airlines
Balkan Airlines
Britannia
British Airways
British Midland
Cathay Pacific Airways
China Airlines
China Southern Airlines
China South West Airlines
Continental Airlines
Emirates
Ethiopian Airlines
EVA Airways
Garuda Indonesia
GB Airways
Gulf Air
Japan Airlines
Royal Dutch Airlines (KLM)
Kuwait Airways
Lufthansa
Qantas Airways
Saudia Arabian Airlines
Scandanavian Airline System (SAS)
Singapore Airlines
TAP Air Portugal
Thai Airways International
United Airlines
US Airways
Wideroe\26
Source: GAO and The Flight Data Company, Ltd.
--------------------
\26 The list may not be comprehensive.
FAA'S RELATED TECHNICAL PROGRAMS
========================================================= Appendix III
Aviation Performance Measuring System. In 1993, FAA contracted with
the National Aeronautics and Space Administration (NASA) to establish
and demonstrate the feasibility of developing an Aviation Performance
Measuring System (APMS). The objective of the APMS effort is to
develop tools and methodologies to allow large quantities of flight
data to be processed in a highly automated manner to address
questions relating to operational performance and safety. APMS is
concerned with converting flight data into useful safety information
in support of the national air transport system, airlines, and air
crews. Although concerned with all aspects of flight operations,
APMS primarily will develop an objective method for continuously
evaluating air crews' technical performance in support of FOQA and
the Advanced Qualification Program (discussed below).
Current FOQA programs focus on exceedances; APMS, however, will
expand FOQA's scope by utilizing all flight data. The tools will
facilitate multiple functions, including the acquisition of flight
data, their storage in a database management system, the study of
statistical characteristics and trends, the development of "data
mining" techniques, and better methods of visualizing flight data.
APMS will also investigate flight animation capabilities to assist
flight crews in replaying and understanding exceedances. Finally,
APMS will facilitate the sharing of data among databases, products,
and interested parties. According to NASA officials, one of the most
important components to be developed by APMS is a risk assessment
tool to measure how much risk is associated with certain activities,
for example, the riskiness of flights to/from certain airports.
After APMS began in 1993, the project documented the status of the
technologies, systems, and software used by foreign airlines with
FOQA programs. According to the NASA project manager, the project
has conducted user needs studies at Alaska Airlines, United Airlines,
and US Airways and has commitments to conduct user needs studies at
America West, Trans World Airlines, Comair, and United Parcel
Service. The APMS team is also building prototype systems at several
airlines. Alaska Airlines is now in its third prototype APMS system.
The project was scheduled to begin building the initial prototype
system at United Airlines on November 1, 1997. Eventually the
developed technology will be transferred to industry so that a
relatively low-cost system will be commercially available. APMS
management hopes to initiate the transfer of this technology to
commercial vendors in 12 to 18 months.
To date, NASA has received $2.9 million in funding from FAA for the
development of APMS. NASA contributed $300,000 to the project in
fiscal year 1997. The extent of future NASA and FAA funding for
further development and implementation of APMS has not yet been
determined.
Structural Loads Program. As part of FAA's Aging Aircraft Research
and Development Program, the Structural Loads Program is a
cooperative FAA and NASA effort to collect information about the
external loads to which airframe components are subjected during
flight. The collected data will be used to develop and maintain an
extensive database of transport aircraft usage to continuously
validate and update flight and landing load airworthiness
certification standards on the basis of actual measured usage. To
date, the Structural Loads Program has equipped with QARs six MD-80s
at Alaska Airlines and six Boeing 737-400s at US Airways. Data
collected from these QARs are also being made available for FOQA
analysis.
Advanced Qualification Program. FAA's Advanced Qualification Program
(AQP) is an alternate method of qualifying, training, certifying, and
ensuring the competency of flight crew members and other operations
personnel subject to the training and evaluation requirements of
Federal Aviation Regulation (FAR) parts 121 and 135. AQP's intent is
to achieve the highest possible standards of individual and crew
performance without undue increases in training costs. FOQA and APMS
will be used to continuously evaluate air crews' technical skills and
airlines' procedures and training in support of AQP. For example,
FOQA data could be used to identify problems occurring during
recurrent flight simulator training and to highlight training areas
for increased emphasis.
Global Analysis and Information Network. The Global Analysis and
Information Network (GAIN) is a concept being actively explored by
the aviation community to facilitate the analysis, sharing, and
dissemination of aviation safety information with a goal of achieving
zero accidents. GAIN would have many information sources--FOQA
information would be one of the most important. Proposed by FAA in
May 1996, GAIN will function as a "significantly improved operational
early warning capability that is sensitive enough to detect and alert
the aviation community to existing and emerging problems."
Information will be shared among airlines and manufacturers and at
the different functional levels within organizations. Although GAIN
is still in the conceptual phase, the aviation community and FAA are
working to address the needs and concerns of prospective members as
well as explore potential designs for a prototype system.
FAA'S VOLUNTARY SAFETY REPORTING:
SELECTED PROGRAMS
========================================================== Appendix IV
FAA has implemented a number of voluntary programs involving the
self-reporting of safety-related information to enhance aviation
safety. Although these programs involve the reporting of information
by people instead of by automated systems, they are similar to FOQA
in that they involve voluntary efforts to identify and correct
potential safety problems. We have highlighted three such programs.
Aviation Safety Reporting Program. Established by FAA in 1975 and
administered by NASA, the Aviation Safety Reporting Program (ASRP)
promotes the voluntary reporting of problems into the Aviation Safety
Reporting System (ASRS). Under 14 C.F.R. 91.25, FAA will not use
reports submitted under the program in any enforcement action (except
accidents or criminal offenses). Under FAA's policy, although a
finding of violation may be made, no sanction will be imposed if (1)
the violation was inadvertent and not deliberate, (2) the violation
did not involve a criminal offense or an accident or an action that
discloses a lack of qualification or competency, (3) the person
filing the report has not been found in any prior FAA enforcement
action to have committed a violation of federal aviation regulations
or law within a period of 5 years prior to the occurrence, and (4)
the report was filed within 10 days after the violation. AC 00-46D
(Feb. 26, 1997), "Aviation Safety Reporting Program."\27
Air Carrier Voluntary Disclosure Reporting Procedures. Initiated by
FAA in 1990 for air carriers,\28 the Voluntary Disclosure Procedures
encourage airlines to promptly disclose to FAA any instances of
noncompliance with the requirements for maintenance, flight
operations, and security. FAA initiated a policy of forgoing civil
penalty actions if five conditions are met: (1) the certificate
holder immediately notifies FAA of the apparent violation after
detecting it and before the agency learns of it; (2) the apparent
violation is inadvertent; (3) the apparent violation does not
indicate a lack, or reasonable question, of the basic qualification
of the certificate holder; (4) immediate action must have been taken,
or begun, upon discovery to terminate the conduct that resulted in
the apparent violation; and (5) the certificate holder must develop
and implement a comprehensive solution satisfactory to the FAA. AC
120-56 (Jan. 23, 1992), "Air Carrier Voluntary Disclosure Reporting
Procedures."
Aviation Safety Action Programs. FAA has established several
demonstration Aviation Safety Action Programs (ASAP), including the
USAir Altitude Awareness Program, the Alaska Airlines Altitude
Awareness Program, and the American Airlines Safety Action
Program.\29 These programs established incentives to encourage the
employees of the air carriers that are participating in the programs
to disclose information and identify possible violations of the
Federal Aviation Regulations without fear of punitive legal
enforcement sanctions. FAA has recently expanded the use of ASAP
through the implementation of a 2-year demonstration program. Under
this program, apparent violations will normally be addressed with
administrative action if the apparent violations do not involve (1)
deliberate misconduct, (2) a substantial disregard for safety and
security, (3) criminal conduct, or (4) conduct that demonstrates or
raises a lack of qualifications. For apparent violations not
excluded under an ASAP, neither administrative action nor punitive
legal enforcement actions will be taken against an individual unless
there is sufficient evidence of the violation other than the
individual's safety-related report. AC 120-66 (Jan. 8, 1997),
"Aviation Safety Action Programs (ASAP)."
--------------------
\27 Provisions concerning air traffic controllers involved in
incidents reported under ASRS are addressed in FAA Order 7210.3.G,
Facility Operations and Administration.
\28 According to an assistant chief counsel at FAA, the procedures
have since been expanded to include production approval holders and
repair stations.
\29 The Altitude Awareness programs at USAir and Alaska Airlines were
joint programs with the Air Line Pilots Association and FAA to
eliminate altitude deviations. USAir's program, in operation from
October 1990 through February 1992, and Alaska Airlines' program, in
operation from August 1994 through February 1995, encouraged flight
crews to report altitude problems so that corrective action could be
taken. The American Airlines Safety Action Program is a joint
program with the Allied Pilots Association (American's pilot union)
and FAA. Begun in June 1994, the program encourages pilots to report
all types of potential safety problems.
DISCOVERY-RELATED COURT ACTIONS
=========================================================== Appendix V
Airline officials and pilot unions' representatives are concerned
about the use of discovery in civil litigation to reveal voluntarily
collected safety information. In two recent cases, the courts have
sought to find a balance between the airlines' desire to protect such
information and the plaintiffs' right to a fair trial. In one case,
the documents were required to be produced, but under a protective
order. In the other case, the court recognized a new qualified
privilege.
In 1995, the United States District Court, District of South
Carolina, Columbia Division, rejected USAir, Inc.'s\30 argument that
certain safety data were protected under the self-critical evaluation
privilege. Court Order of Oct. 26, 1995, In re Air Crash at
Charlotte, North Carolina, on July 2, 1994, MDL Docket No. 1041
(D.S.C. 1995). This privilege, when recognized, protects documents
that reflect self-analysis.
The district court noted that the self-critical evaluation privilege
is a privilege of recent origin and one that is narrowly applied even
in those jurisdictions where it is recognized. The court described
the privilege by citing to Dowling v. American Hawaii Cruises, Inc.,
971 F.2d 423, 425-426 (9th Cir. 1992), which explained:
"[O]ther courts have generally required that the party asserting
the privilege demonstrate that the material to be protected
satisfies at least three criteria: first, the information must
result from a critical self-analysis undertaken by the party
seeking protection; second, the public must have a strong
interest in preserving the free flow of the type of information
sought; finally, the information must be of the type whose flow
would be curtailed if discovery were allowed. . . .To these
requirements should be added the general proviso that no
document will be accorded a privilege unless it was prepared
with the expectation that it would be kept confidential, and has
in fact been kept confidential."
The court found that the safety documents did not meet the criteria
for the privilege. According to the court, the most significant
stumbling block for the airline was meeting the third criterion--that
the flow of the information would be curtailed if discovery was
allowed. Specifically, the court found that the airline industry is
highly competitive and tightly regulated and that airlines have a
keen interest in advancing and promoting safety as well as services.
Thus, the court reasoned that the airlines were likely to conduct
internal audits. The court reasoned that while the disclosure of
such audits to competitors would deter their use in the future,
disclosure for limited use in litigation is unlikely to have such an
impact.
Subsequently, the court limited the possible uses of the documents it
ordered to be produced. Specifically, the court ordered:
". . . plaintiff and their counsel shall be prohibited from
disclosing, disseminating or communicating in any manner to any
person or entity not involved in this litigation any portion of
the information contained in those documents. . . .
Plaintiff and their counsel shall be further precluded from
utilizing these documents or the information contained in them
for any purpose other than for this multidistrict litigation.
"In furtherance of this order, plaintiffs' counsel shall insure
that each person who is to be given access to the referenced
documents, including plaintiff and their attorneys, shall first
sign a document acknowledging that they are aware of and will
comply with this order. Plaintiffs' counsel shall maintain a
list of those persons which shall be provided to USAir's
attorney upon request, subject to protection upon application to
this court for good cause shown." Court Order of Nov. 14, 1995,
In Re: Air Crash at Charlotte, North Carolina, on July 2, 1994,
MDL Docket No. 1041 (D.S.C. 1995).
In October 1996, the Supreme Court let stand the district court order
rejecting the airline's assertion of a self-critical evaluation
privilege. 65 U.S.L.W. 3221 (Oct. 8, 1996).
Recently, in another case involving documents prepared by American
Airlines' employees collected under the American Airlines Safety
Action Program, the United States District Court, Southern District
of Florida, on a motion for reconsideration, also rejected the
airline's self-critical analysis privilege claim. However, in this
case the court recognized a new qualified privilege to protect these
documents. In re Air Crash Near Cali, Colombia, on Dec. 20, 1995,
959 F. Supp. 1529 (S.D. Fla. 1997).
With respect to the self-critical analysis privilege, the court
stated that "the touchstone of a self-critical analysis is that it is
an 'in house' review undertaken primarily, if not exclusively, for
the purpose of internal quality control." In this case, the court
rejected the application of the privilege, finding the following:
"Even assuming that the materials prepared by American's pilots
in conjunction with the ASAP program may be of a type whose
creation might be curtailed if discovery is allowed, these
materials were prepared for dissemination to representatives of
entities unaffiliated with American (a federal regulatory agency
and a union)."
The court, however, recognized a new, qualified common law privilege
for the ASAP materials. In recognizing a new privilege, the court
considered the principles for evaluating claims of federal common law
privileges recently articulated in the Supreme Court case, Jaffee v.
Redmond, __ U.S. __, 116 S. Ct. 1923 (1996): (1) the "private
interest" involved--in other words whether the dissemination of the
information would chill the frank and complete disclosure of fact;
(2) the "public interests" furthered by the proposed privilege; (3)
the "likely evidentiary benefit that would result from the denial of
the privilege;" and (4) the extent to which the privilege has been
recognized by state courts and legislatures.\31
The court found that American had met its burden of proving that a
qualified "ASAP privilege" is appropriate. Specifically, the court
stated as follows:
"The ASAP materials in dispute . . . were prepared
voluntarily, in confidence and for use in a discrete, limited
context in cooperation with the FAA and the pilot's union.
There is a genuine risk of meaningful and irreparable chill from
the compelled disclosure of ASAP materials in connection with
the pending litigation."
The court specified that the privilege should be qualified.
Accordingly, the plaintiffs could overcome the privilege with a
persuasive showing of need and hardship. The plaintiffs did not make
such a showing in the case.
--------------------
\30 USAir changed its name to US Airways on Feb. 27, 1997.
\31 The court recognized a psychotherapist-patient privilege in this
case.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix VI
RESOURCES, COMMUNITY, AND ECONOMIC
DEVELOPMENT DIVISION
David E. Bryant, Jr.
Thomas F. Noone
Robert E. White
OFFICE OF THE GENERAL COUNSEL
Mindi Weisenbloom
*** End of document. ***