Human Factors: Status of Efforts to Integrate Research on Human Factors
Into FAA's Activities (Letter Report, 07/27/96, GAO/RCED-96-151).
Pursuant to a congressional request, GAO examined the Federal Aviation
Administration's (FAA) organizational structure for incorporating human
factors into aviation-related research.
GAO found that: (1) FAA has incorporated a human factors includes a
human factors policy order, a Chief Scientific and Technical Advisor for
human factors, and guidance for considering human factors in the
acquisition process; (2) the order assigns responsibility for ensuring
that human factors are considered in FAA research activities, but does
not establish minimal standards for meeting this requirement; (3) recent
legislative and organizational changes may affect the application of
human factors research in FAA acquisitions and operations; (4) the FAA
Acquisition Management System considers human factors at an earlier
stage in the acquisition process, but there is no mention of the extent
to which such factors should be considered; (5) the FAA Human Factors
Division (HFD) consults with other members of the aviation community and
participates in industry task forces and conferences to identify issues
associated with human factors in aviation; (6) HFD solicits ideas for
research from FAA acquisition and operating units and is responsible for
internal and external coordination of FAA research; (7) HFD allocates
most FAA funding for core research, and enters into interagency
agreements with the National Space and Aeronautics Administration and
the Department of Defense to coordinate the agencies' human factors
research; and (8) the possibility of duplicating human factors research
exits because FAA units are not required to coordinate their research
activities.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-96-151
TITLE: Human Factors: Status of Efforts to Integrate Research on
Human Factors Into FAA's Activities
DATE: 07/27/96
SUBJECT: Human factors engineering
Transportation safety
Behavioral sciences research
Interagency relations
Aircraft industry
Human resources training
Safety standards
Air transportation operations
Aircraft accidents
IDENTIFIER: FAA Acquisition Management System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chair, Subcommittee on Technology, Committee on
Science, House of Representatives
June 1996
HUMAN FACTORS - STATUS OF EFFORTS
TO INTEGRATE RESEARCH ON HUMAN
FACTORS INTO FAA'S ACTIVITIES
GAO/RCED-96-151
Human Factors
(341458)
Abbreviations
=============================================================== ABBREV
AF SMART - Airway Facilities Systems Model for Assessment,
Recruitment, and Training program
AQP - Advanced Qualification Program
ATC - Air Traffic Control
ATCS/PTS - Air Traffic Control Specialist Pre-Training Screen
CAMI - Civil Aeromedical Institute
CRM - Crew Resource Management
CTI - Collegiate Training Initiative
DOT - Department of Transportation
FAA - Federal Aviation Administration
HFCC - Human Factors Coordinating Committee
NAS - National Airspace System
NASA - National Aeronautics and Space Administration
NDT - Nondestructive Test
RTP - Readiness to Perform
Letter
=============================================================== LETTER
B-270246
June 27, 1996
The Honorable Constance A. Morella
Chair, Subcommittee on Technology
Committee on Science
House of Representatives
Dear Madam Chair:
Human error has contributed to about 80 percent of the fatal aviation
crashes, according to Federal Aviation Administration (FAA)
officials. The study of human factors, which identifies and tries to
reduce the chances for human error through improvements in design and
training, has emerged as one of the most promising means of
increasing aviation safety. Consisting of both research and its
applications, the human factors discipline is used to (1) identify
systemic errors in the operation of machines or implementation of
procedures and (2) design equipment or procedures to eliminate or
mitigate the effects of such errors.\1
Recognizing the importance of human factors in aviation, the Congress
enacted the Aviation Safety Research Act of 1988 (P.L. 100-591),
which mandated that FAA augment its research on human factors and
coordinate its work with that of the National Aeronautics and Space
Administration (NASA) and the Department of Defense. On August 10,
1995, you asked us to review FAA's efforts to ensure the
consideration of human factors in the agency's activities. As agreed
with your office, this report describes FAA's (1) organizational
structure for incorporating the consideration of human factors in the
agency's acquisition of new systems and operation of other systems
and (2) aviation-related research on human factors, including the
agency's processes for identifying research issues, and methods for
allocating and coordinating resources for internal and external
research on human factors.
--------------------
\1 FAA defines its work on human factors as a multidisciplinary
effort to generate and compile information about human capabilities
and limitations and to apply this information to equipment, systems,
facilities, procedures, jobs, environments, training, staffing, and
personnel management for safe, comfortable, effective human
performance.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
FAA's structure for incorporating the consideration of human factors
in the agency's programs includes a human factors policy order, a
Chief Scientific and Technical Advisor for Human Factors (Chief
Scientist), and guidance for acquisitions. The order, issued in
1993, assigns responsibility for ensuring that human factors are
considered in the agency but does not establish minimal standards for
meeting this requirement. The Chief Scientist chairs the Human
Factors Coordinating Committee and manages the Human Factors Division
within FAA's Office of Aviation Research. Recent legislative and
organizational changes may affect the application of human factors in
acquisitions and in operations, such as safety.
The Human Factors Division consults with other members of the
aviation community and participates in industry task forces and
conferences in order to identify issues associated with human factors
in aviation. In addition, it solicits ideas for research from FAA's
acquisition and operating units. The Human Factors Division is
primarily responsible for the internal and external coordination of
FAA's research on human factors. Internally, this division allocates
most of the agency's funding for core research on human factors.
Externally, the division has entered into interagency agreements with
NASA and Defense to coordinate the agencies' research on human
factors. However, FAA's other units are not required to coordinate
their research on human factors with the Human Factors Division when
the research is performed internally, by the units themselves, or
externally, through interagency agreements or by contractors. As a
result, the possibility of duplication exists, and the opportunity to
leverage resources for research could be lost.
BACKGROUND
------------------------------------------------------------ Letter :2
The study of human factors examines how humans interact with machines
and other people (pilots, air traffic controllers, or design and
acquisition personnel) and determines whether procedures and
regulations take into account human abilities and limitations.
Identifying chances for human error can reduce the need for later
replacing or modifying equipment and procedures. Human factors
affect the operation of all of FAA's functions, including research,
the acquisition of equipment, and safety. FAA's work on human
factors focuses on such issues as whether equipment is designed to
enhance operators' performance and minimize errors and whether the
procedures used by air traffic controllers promote safe operations.
For example, much of the information conveyed to pilots by air
traffic controllers has been standardized to minimize the possibility
of misunderstanding. (See app. I for a more complete definition of
human factors and examples of how human factors have affected safety
in specific situations.)
The Aviation Safety Research Act of 1988 directed FAA to augment its
research on human factors and coordinate its work with that of NASA
and Defense because the Congress believed that FAA did not have
sufficient expertise in all areas of human factors. A report by the
Office of Technology Assessment,\2 cited in the House report on the
act\3 as the basis for the legislation, recommended that FAA allocate
resources for developing its regulatory support staffs' expertise in
human factors and establish a focal point for human factors within
the agency. In addition, the Congress has indicated through the
budget process that research on human factors should be a priority in
FAA's overall research program. Figure 1 compares the congressional
appropriations for FAA's research on human factors with FAA's funding
requests.
Figure 1: Appropriations for
FAA's Research on Human Factors
Compared With FAA's Funding
Requests, Fiscal Years 1991-96
(See figure in printed
edition.)
Source: GAO's analysis of FAA's budget documents.
--------------------
\2 Safe Skies for Tomorrow: Aviation Safety in a Competitive
Environment (Washington, D.C.: July 1988).
\3 House Report 100-894.
FAA'S ORGANIZATION FOR HUMAN
FACTORS
------------------------------------------------------------ Letter :3
Key aspects of FAA's human factors organization are the 1993 policy,
the position of Chief Scientist, and the guidance on considering
human factors in the acquisition process. The 1993 policy prescribes
the roles and responsibilities of FAA's assistant and associate
administrators and program directors, as well as of the Human Factors
Coordinating Committee (HFCC), including its chair, the Chief
Scientist. The Chief Scientist also manages the Human Factors
Division,\4 which is housed in FAA's Office of Aviation Research.
(Fig. 2 illustrates the location of the Human Factors Division
within FAA's organizational structure.) On April 1, 1996, FAA changed
its acquisition process and method of incorporating the consideration
of human factors into that process. The creation of an Office of
System Safety in 1995 may further affect the organizational structure
for human factors.
Figure 2: Location of Human
Factors Division Within FAA's
Organizational Structure
(See figure in printed
edition.)
Source: FAA.
--------------------
\4 Known officially as the Office of the Chief Scientific and
Technical Advisor for Human Factors, this office is commonly referred
to as the Human Factors Division.
HUMAN FACTORS POLICY
---------------------------------------------------------- Letter :3.1
In October 1993, FAA issued an order for incorporating and
coordinating the consideration of human factors throughout the
agency.\5 Under the order, assistant and associate administrators and
program directors are responsible for, among other things,
establishing formal procedures to ensure the systematic consideration
of human factors within their organizations. However, FAA's order
does not prescribe the (1) methods for considering human factors, (2)
minimal standards for incorporating human factors, or (3)
requirements for seeking guidance on human factors from specialists\6
that the administrators and directors are to follow. FAA officials
in the three units where we held discussions--research and
acquisitions, regulation and certification, and air traffic
services--indicated that they have not fully established formal
procedures for incorporating the consideration of human factors in
their activities.
FAA created the Human Factors Coordinating Committee in 1989 to
facilitate the agency's work on human factors and enhance the use of
information on human factors. However, according to the Chief
Scientist, the committee is not a decision-making body, even though
its members are designated by the agency's assistant and associate
administrators and program directors. Instead, the Chief Scientist
said, the committee is primarily a forum for exchanging information.
As the committee's chair, the Chief Scientist carries out most of the
committee's responsibilities.
--------------------
\5 FAA Order 9550.8, Human Factors Policy, Oct. 27, 1993.
\6 Human factors specialists are certified by accrediting
organizations such as the Board of Certification in Professional
Ergonomics or have graduated from programs accredited by
organizations such as the Human Factors and Ergonomics Society.
CHIEF SCIENTIST
---------------------------------------------------------- Letter :3.2
In addition to chairing the Human Factors Coordinating Committee, the
Chief Scientist heads the Human Factors Division. This division is
housed within the headquarters Office of Aviation Research, under the
Associate Administrator for Research and Acquisitions. Among other
things, the Human Factors Division develops policies on human factors
that promote the productivity and safety of the national airspace
system.\7 The division is staffed by seven professional human factors
specialists--six full-time and one part-time.
According to its mission statement, the Human Factors Division seeks
to provide scientific and technical support for FAA's research on
human factors in civil aviation and its applications in the agency's
programs for acquisitions, regulation and certification, and air
traffic services. However, we found that the Human Factors
Division's ability to provide this support depends on the extent to
which the associate administrators and program directors use the
division. FAA does not require the division--or any other unit with
scientific and technical expertise in human factors--to review the
quality of the work on human factors performed by other FAA units or
contractors. FAA does not require its administrators to seek
guidance from human factors specialists, such as those in the Human
Factors Division. Although the scope of our audit did not include a
detailed examination of the application of human factors in
acquisitions, we have previously found inadequate technical oversight
in FAA's management of acquisitions. For example, in a previous
review of FAA's modernization program, we found that not following
the technical principles of the human factors discipline in designing
equipment delayed some projects.\8 Instead of relying on the
discipline's objective criteria for measuring the performance of
alternative designs, FAA consulted users' preferences, only to find
that its efforts were misdirected because different groups of users
had different preferences.
--------------------
\7 These functions are prescribed in the Human Factors Division:
Mission and Functions statement (rev. Sept. 1995).
\8 Advanced Automation System: Implications of Problems and Recent
Changes (GAO/T-RCED-94-188, Apr. 13, 1994).
INCORPORATING THE
CONSIDERATION OF HUMAN
FACTORS IN ACQUISITIONS AND
OPERATIONS
---------------------------------------------------------- Letter :3.3
Recent legislative and organizational changes may affect how the
formal consideration of human factors is incorporated in the
acquisition process and may strengthen the application of human
factors in operations, such as safety.
ACQUISITIONS AND HUMAN
FACTORS
-------------------------------------------------------- Letter :3.3.1
Several offices under the Associate Administrator for Research and
Acquisitions are responsible for developing and acquiring new
systems, such as air traffic control equipment. According to staff
in the Human Factors Division, applying considerations of human
factors increases a product's or a process's performance and
efficiency while decreasing developmental, operational, and
maintenance costs over the lifetime of the product or process.
To develop and deploy equipment more efficiently, in 1995, FAA
adopted a new management approach that relies on integrated product
teams,\9 whose members include end-users, contractors, and all other
parties responsible for developing or procuring new equipment or
processes.\10 As a first step in ensuring that human factors are
considered in acquisitions, the Human Factors Division developed a
requirement in FAA's 1993 acquisition policy\11 that all new
acquisition projects include a human factors plan. Such a plan was
to (1) describe how considerations of human factors should be applied
and (2) document how a piece of equipment or a process should perform
when operated as expected by the end-users.
However, on April 1, 1996, in response to new legislation exempting
FAA from most federal procurement statutes, FAA implemented the
Federal Aviation Administration Acquisition Management System, which
superseded FAA's 1993 acquisition policy. According to the initial
guidance provided for this new system, human factors may be formally
considered at an earlier stage in the acquisition process than
previously, but this early consideration is not required.
Furthermore, the extent to which human factors should be considered
is not specified in the system's guidance, nor is a separate plan for
human factors required. There is no requirement for integrated
product teams to obtain recommendations from human factors
specialists.
--------------------
\9 Although at least 12 of the 14 integrated product teams are
currently functioning, only 1 has met FAA's requirements for a draft
implementation plan, received training, and collocated its members.
For more information, see our forthcoming report entitled Aviation
Acquisition: A More Comprehensive FAA Strategy Is Needed for
Cultural Change (GAO/RCED-96-159).
\10 Integrated product teams were established by a memorandum of
agreement between the associate administrators for airports, air
traffic services, regulation and certification, and research and
acquisitions (Apr. 14, 1995).
\11 FAA Order 1810.1F, Mar. 19, 1993.
OPERATIONS AND HUMAN
FACTORS
-------------------------------------------------------- Letter :3.3.2
According to some FAA human factors specialists, considering human
factors is key to improving the safety of aviation operations. In
1990, the FAA Administrator testified before the Congress that the
agency's objective in aviation safety is zero accidents.\12 The
following year, the Administrator testified that human error was the
most serious impediment to FAA's achieving that goal.\13 He said that
FAA planned to accentuate its consideration of human factors in all
of its programs, from training to procurement.
To help reach its goal of zero accidents in aviation operations, FAA,
in 1995, created a staff Office for System Safety. This office is
headed by the Assistant Administrator for System Safety, who reports
directly to the FAA Administrator. The objective of this office is
to proactively determine potential sources of accidents and prevent
them from occurring. The Assistant Administrator for this office has
indicated that human factors will be an important part of his
office's work.
Although the Human Factors Division administers FAA's research on
human factors, some of which is directly concerned with safety, its
staff are not involved in some applications of human factors to
safety. For example, the Office of Regulation and
Certification--responsible for aircraft certification, safety
inspections, and flight operational safety--plans to strengthen its
emphasis on human factors by hiring at least one specialist, rather
than rely on the specialists in the Human Factors Division.
According to the Associate Administrator for Regulation and
Certification, the specialists in the Human Factors Division do not
have the expertise needed to apply considerations of human factors to
developing requirements for regulation and certification.
--------------------
\12 Hearings Before the Subcommittee on the Department of
Transportation and Related Agencies Appropriations for 1991, House
Committee on Appropriations (Feb. 22, 1990).
\13 Hearings Before the Subcommittee on the Department of
Transportation and Related Agencies Appropriations for 1992, House
Committee on Appropriations (Feb. 19, 1991).
FAA'S RESEARCH ON HUMAN FACTORS
------------------------------------------------------------ Letter :4
The Human Factors Division is responsible for identifying
aviation-related issues in research on human factors and for
allocating and coordinating FAA's resources for internal and external
research on human factors.
IDENTIFYING RESEARCH ISSUES
---------------------------------------------------------- Letter :4.1
To identify aviation-related issues in research on human factors, the
Human Factors Division consults with FAA units and other members of
the aviation community. To develop its initial objectives for
research on human factors, FAA participated in a task force in April
1989, sponsored by the Air Transport Association of America. This
task force identified a number of significant research topics, which
FAA incorporated into the National Plan for Civil Aviation Human
Factors.\14 This plan--developed by the Human Factors Division in
conjunction with the Department of Defense, NASA, industry, and
academia--includes a framework that categorizes research on the basis
of five priorities, or "thrusts," and provides guidelines for
initiating and managing research on human factors in aviation. (See
app. II for a description of each priority and a listing of the
ongoing projects under each.) Besides participating in the task
force, the Human Factors Division has worked with the aviation
community to develop research issues by participating in conferences
and workshops. In comparing FAA's processes to the aviation
community's, we found that FAA not only looks to the aviation
community but the aviation community also often looks to FAA to focus
attention on particular research issues. For example, FAA sponsored
a national conference in 1995 on the challenge of approaching zero
accidents.
In addition, the Human Factors Division identifies research issues
that the aviation community may not. For example, by managing the
research sponsored by FAA units, the Human Factors Division is able
to identify research needs that may apply to other FAA units and the
aviation community as a whole.
According to the Assistant Administrator, the newly created Office of
System Safety will proactively seek to identify safety issues that
may indicate the need for additional research on human factors. For
example, this office has assumed responsibility from the Office of
Aviation Safety for an ongoing project to develop methods for
extracting information on human factors from FAA's existing sources
of data. However, according to the Assistant Administrator, this
office has not yet developed a research agenda. While staff from the
office have met with personnel from the Human Factors Division, no
joint activities have been established and no plans have been
developed for interactions between the two units.
Although the Human Factors Division identifies FAA's needs for
research on human factors, at least one operating unit is also
independently identifying and executing its own research needs. The
Office of Regulation and Certification identifies research issues on
the basis of its needs and determines what organization will conduct
the research. Specifically, the Associate Administrator for
Regulation and Certification has established a Human Factors Task
Force to review existing literature; obtain information from avionics
manufacturers, operators, and industry technical groups; and conduct
simulations. The task force was not chartered to initiate research;
however, it may make recommendations leading to research on human
factors. The Human Factors Division was involved neither in
determining the need for the task force nor in planning its work.
The possibility exists that the task force's recommendations could
lead the Office of Regulation and Certification to initiate research
duplicating the work of the Human Factors Division. Thus, FAA would
be deprived of the opportunity to leverage resources for research.
--------------------
\14 Published by FAA in Mar. 1995, this document is a revision of a
draft plan published by FAA in Nov. 1990.
ALLOCATING AND COORDINATING
RESOURCES FOR RESEARCH
---------------------------------------------------------- Letter :4.2
Although the Human Factors Division is primarily responsible for
allocating and coordinating FAA's resources for internal and external
research on human factors, FAA's other units are not required to
coordinate their research with the division, whether their research
is performed internally, by the units themselves, or externally,
through interagency agreements or through contractors.
INTERNAL ALLOCATION AND
COORDINATION
-------------------------------------------------------- Letter :4.2.1
Starting in 1995, the Office of Aviation Research made the Human
Factors Division responsible for allocating most of the agency's
Research, Engineering, and Development funds for research on human
factors--nearly $28 million.\15 In fiscal year 1995, the Human
Factors Division funded research projects in support of FAA's
acquisition ($5 million), regulation and certification ($12.5
million), and air traffic services ($10.5 million) programs.
The Human Factors Division has also assumed the responsibility for
funding contracts or grants for research on human factors at entities
such as FAA's Civil Aeromedical Institute (CAMI) located in Oklahoma
City, FAA's Technical Center near Atlantic City, NASA, the Department
of Transportation's Volpe Transportation Center, and other
institutions.\16 Previously, when its research on human factors was
funded solely by its operating units, FAA provided no centralized
planning for and oversight of its core research on human factors.
Now that the Human Factors Division is coordinating FAA's funding for
research (conducted by CAMI, FAA's Technical Center, NASA, the Volpe
Transportation Center, and other institutions), it is constructing a
combined database of ongoing research projects, which should give
greater visibility to FAA's research on human factors and permit
closer monitoring of the research projects that the agency has
funded. As a part of its research administration, the Human Factors
Division also monitors whether scientific and technical principles
are being applied to the research it funds.
Some FAA units may not be coordinating their research on human
factors with the Human Factors Division. For example, some
integrated product teams may be conducting such research through
contractors, but FAA has no mechanism to ensure that the information
developed by a private contractor for one team is made available to
another contractor addressing similar issues for another team. Thus,
because the FAA units that sponsor their own research on human
factors are not required to coordinate their work with that of other
units or to inform the Human Factors Division about their research,
the possibility of duplication exists.
--------------------
\15 Funding for research and other work on human factors can come
from other sources, including funding for acquisition projects, which
is provided through the Facilities and Equipment account.
\16 External recipients of grants, such as NASA, can, in turn,
provide funding for research to be conducted by other institutions or
individuals.
EXTERNAL COORDINATION
-------------------------------------------------------- Letter :4.2.2
The Human Factors Division has memoranda of agreement or
understanding with NASA and the Department of Defense. According to
officials in both the Human Factors Division and NASA, a beneficial
result of their coordination is that NASA has not duplicated research
being conducted by the division. In addition, the Human Factors
Division contracts with NASA to conduct some of its research on human
factors in areas where NASA has more experience and/or expertise.
FAA also contracts with the Department of Defense to conduct research
on human factors. While much of Defense's research is specific to
defense needs, Defense officials indicated that using the framework
articulated in the National Plan for Civil Aviation Human Factors
will enable the Department to better coordinate its research on human
factors with FAA's work in similar areas.
CONCLUSIONS
------------------------------------------------------------ Letter :5
The organizational structure for FAA's work on human factors is still
evolving. Therefore, it is too soon to evaluate the effectiveness of
the agency's procedures for incorporating the consideration of human
factors throughout FAA and for monitoring the quality of the agency's
work on human factors. Nonetheless, we have found that some FAA
units are not coordinating their research with the Human Factors
Division, although this division is, currently, primarily responsible
for allocating and coordinating FAA's resources for internal and
external research on human factors. Without agencywide coordination
of the research on human factors, the potential for duplication
exists and the opportunity to leverage the agency's research dollars
by combining related projects is diminished.
RECOMMENDATION
------------------------------------------------------------ Letter :6
To reduce the possibility of duplication and maximize the opportunity
to leverage resources for research on human factors, we recommend
that the Secretary of Transportation direct the Administrator, FAA,
to ensure that all units within FAA coordinate their research through
the agency's Human Factors Division.
AGENCY COMMENTS
------------------------------------------------------------ Letter :7
We provided copies of a draft of this report to the Department of
Transportation (DOT) and FAA. We met with officials from the Office
of the Secretary of Transportation, including the Chief of the Audit
Liaison Division, and FAA officials, including the Special Assistant
to the Associate Administrator for Regulation and Certification and
the Chief Scientist and Technical Advisor on Human Factors, who
generally agreed with the report's findings and recommendation. They
provided us with information clarifying FAA's formal consideration of
human factors in the agency's new acquisition process; we
incorporated this information into the text as necessary. According
to the Office of Regulation and Certification, the possibility that
its Task Force on Human Factors would recommend research duplicating
the work of the Human Factors Division is minimal because the
research might be administered by the Human Factors Division.
However, the Human Factors Division is concerned that, without
adequate coordination, the task force could initiate future research
that might duplicate the division's work. FAA indicated that the
Office of Regulation and Certification is taking steps to hire a
human factors specialist whose first duty will be to develop, in
conjunction with the Human Factors Division, a documented process for
coordinating research. Unless FAA ensures that research will be
administered through the Human Factors Division or until the agency
establishes a documented process for coordinating research, we
continue to believe that the possibility of duplication exists. DOT
expressed concern about our discussion of FAA's practice of not
reviewing the quality of the agency's work on human factors, noting
that quality is difficult to assess. While we agree that assessing
quality is difficult, we continue to believe that scientific and
technical standards are available for assessing the quality of the
agency's work on human factors. We further believe that adherence to
such standards is important to ensure the usefulness of the work's
results.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :8
To determine how FAA has incorporated the consideration of human
factors into its research, acquisition, and safety programs, we
examined FAA's organizational structure and reviewed FAA's policy
orders, formal guidance, and strategies for compiling and applying
information on human factors. We interviewed FAA officials in the
research and acquisitions, regulation and certification, and air
traffic services units, but we did not discuss the consideration of
human factors in the airports and civil aviation security units
because of time constraints. To determine the processes that FAA
uses to identify issues in aviation-related research on human factors
and compare these processes to those of the aviation community, we
reviewed FAA's plans and research abstracts, interviewed agency
officials, and contacted members of the aviation community. To
determine how FAA allocates and coordinates resources internally and
externally, we interviewed FAA, NASA, and Defense officials and other
members of the aviation community and reviewed the legislative
requirements for these activities. Because FAA's work on human
factors was not centralized, we relied on data from the Human Factors
Division on activities in the Research, Engineering, and Development
budget. However, we were not able to obtain similar information for
the work on human factors supported through other FAA accounts
because such information is not available. We conducted our review
from September 1995 through June 1996 in accordance with generally
accepted government auditing standards.
We are sending copies of this report to the Secretary of
Transportation and the FAA Administrator. We will also make copies
available to others on request. Please call me at (202) 512-3650 if
you or your staff have any questions about this report.
Sincerely yours,
Gerald L. Dillingham
Associate Director, Transportation
Issues
DEFINITION OF HUMAN FACTORS
=========================================================== Appendix I
The human factors discipline is a scientific and technical approach
for designing, operating, and maintaining systems. The goal of this
approach is to improve the efficiency and reliability of systems by
enhancing the integration of these systems' components. These
components generally consist of the facilities and equipment, rules
and regulations, human operators, and environment (physical,
economic, political, and social) in which they operate. Thus, the
human factors discipline tries to optimize the interactions between
the components of a system.
To achieve its goal, the human factors discipline relies on research
that combines human sciences and systems engineering. In aviation,
the application of human factors research focuses on the complex
connections between (1) the members of the flight crew, (2) the
flight crew and the aircraft they pilot, (3) the flight crew and the
air traffic controllers, (4) the air traffic controllers and their
equipment, and (5) the rules, regulations, laws, and standard
operating procedures that govern aviation operations. Table I.1
illustrates human factors issues in selected aviation incidents.
Table I.1
Selected Aviation Incidents Illustrating
Human Factors Issues
Specific potential
Type(s) of human human factors
Incident Possible cause factors issue(s) issue(s)
-------------- -------------------- -------------------- --------------------
An airplane The crew was not Computer/ The crew could have
crashed into a familiar with human (1) misinterpreted
mountain in sophisticated new interface the vertical speed/
bad weather.\a flight control flight path angle
equipment requiring display on the
accurate flight control
interpretation and computer or (2)
operation. entered the wrong
data.
Crew Communication and
resource coordination between
management the captain and the
first officer could
have been poor.
Training Both the captain and
first officer might
have had limited
experience with this
type of aircraft.
Air A last-moment air
traffic traffic control
control/flight approach procedure
deck might have
integration distracted the
crew's attention
from the aircraft's
position in relation
to the airport and
to the altitude/
descent rate.
An airplane The pilot disengaged Computer/ The pilot did not
crashed while two computerized human fully understand the
executing a safety features: an interface safety features'
low-level pass autothrottle and an functions: The
with a sharp alpha floor autothrottle
15-degree protection function. maintains a
pull-up at specified speed and
full thrust.\b the alpha floor
protection function
prevents the engine
from stalling.
An aircraft An antivibration Maintenance error A maintenance
hydraulic clamp on an engine- technician forgot to
system failed mounted hydraulic install the clamp.
during tube was missing.
flight.\c
Many changes FAA did not Design FAA should have
were made initially determine established
during the the systems' performance
development of operational baselines for the
major air requirements. systems being
traffic developed.
control
systems,
resulting in
rework.\d
An airplane The pilot did not Selection The pilot may not
crashed achieve a have possessed the
because the satisfactory level skills needed to
pilot did not of performance, become competent,
take proper despite remedial despite training.
corrective training.
action, even
though the
first officer
appropriately
advised him.\e
Crew The pilot and first
resource officer had little
management experience flying
together, and the
first officer may
not have known that
the pilot's skills
were inadequate.
--------------------------------------------------------------------------------
\a French Transport Ministry officials, as quoted in an article
appearing in Aviation Week and Space Technology (Jan. 3, 1994). We
did not verify the accuracy of the facts presented in this article.
\b Steven M. Casey, Set Phasers on Stun and Other True Tales of
Design, Technology, and Human Error, Santa Barbara: Aegean
Publishing Co., 1993.
\c R. Curtis Graeber and David A. Marx, "Reducing Human Error in
Aircraft Maintenance Operations," Seattle: Boeing Commercial
Airplane Group (Presented at the Flight Safety Foundation's 46th
Annual International Air Safety Seminar, Nov. 8-11, 1993).
\d Former FAA contractor.
\e American Eagle officials, as quoted in an article in U.S.A. TODAY
(Sept. 27, 1995). We did not verify the accuracy of the facts
presented in this article.
HUMAN FACTORS RESEARCH AREAS AND
ONGOING RESEARCH PROJECTS
========================================================== Appendix II
FAA's framework for research on human factors is organized into five
broad areas: (1) human-centered automation, (2) information
management and display, (3) selection and training, (4) human
performance assessment, and (5) bioaeronautics.
HUMAN-CENTERED AUTOMATION
-------------------------------------------------------- Appendix II:1
Human-centered automation research focuses on the role of the
operator and the effects of using automation to assist humans in
accomplishing their assigned tasks with greater safety and
efficiency. The research in this area is designed to identify and
apply knowledge of the relative strengths and limitations of humans
in an automated environment. It investigates the implications of
computer-based technology for the design, evaluation, and
certification of controls, displays, and advanced systems.
AREAS OF ONGOING RESEARCH
------------------------------------------------------ Appendix II:1.1
-- Automation, Advanced Technology, Controls and Display Design
-- Advanced User Systems Interface
-- Human Factors Design, Integration, Evaluation
-- General Aviation: Development and Assessment of Cockpit Display
Automation
-- Automation Transition Analysis: Impact on Understanding of
Flight Data Information
-- Human Factors Considerations in the Operations Control Center
INFORMATION MANAGEMENT AND
DISPLAY
-------------------------------------------------------- Appendix II:2
Research conducted under this area seeks to improve safety and
performance by addressing the presentation and transfer of
information among components in the national airspace system (NAS),
including controllers' workstations, the flight deck, operational and
airway facilities, and all the interfaces in between.
AREAS OF ONGOING RESEARCH
------------------------------------------------------ Appendix II:2.1
-- Determine the Appropriate Allocation of Authority and Functions
Between the Flightdeck and Air Traffic Control (ATC)
-- Develop the Required Methods, Tools, and Guidelines for
Integration of Flight Deck/ATC Components Into the NAS
-- Enhance Flightdeck/ATC Information Transfer and Management
-- Decrease the Frequencies and Consequences of Flight Deck/ATC
Errors
-- Pilot-ATC Communications: Datalinked Communications
SELECTION AND TRAINING
-------------------------------------------------------- Appendix II:3
The National Airspace System's efficiency and effectiveness are
enhanced through research to understand the relationship between
human abilities and the performance of aviation tasks; to enhance the
measures and methods for predicting future job/task performance; to
develop a scientific basis for designing training programs, devices,
and aids; to define criteria for assessing future training
requirements; and to identify new ways for selecting aviation system
personnel. The recipients of research findings on selection and
training are flight crews, air traffic controllers, airways
facilities systems management personnel, aircraft maintenance
technicians, airport security personnel, and others in the aviation
community who contribute to safety and efficiency through staffing
and training decisions.
AREAS OF ONGOING RESEARCH
------------------------------------------------------ Appendix II:3.1
-- Selection, Training, Certification, and Staffing of ATC
Personnel
-- Model Advanced Qualification Program (AQP)
-- Integrated Measures of CRM and Technical Performance in AQP
-- Airman Training and Selection
-- Airway Facilities Systems Model for Assessment, Recruitment, and
Training (AF SMART) Program
-- Integrated Digital Video Debrief Station
-- AQP Database Development
-- Advanced Technology in Training, Job Aiding, and Documentation
-- Airplane Simulator and Flight Training Device Transfer of
Performance
-- Job Satisfaction Surveys: Measurement, Content, Validity, and
Linkages to Policy
-- Validation of the Air Traffic Control Specialist Pre-Training
Screen (ATCS/PTS)
-- Development and Evaluation of Managerial Selection Systems
-- Evaluation of the Air Traffic Control Specialist Collegiate
Training Initiative (CTI)
-- Validation of AF Technician Post-Hire Assessments
-- Evaluation of the Effectiveness of the 16PF ATCS Applicant
Screen Procedure
-- Assessment of PC-Based Flight Simulation Devices
-- Certification and Validation Standards
HUMAN PERFORMANCE ASSESSMENT
-------------------------------------------------------- Appendix II:4
Research in this area is designed to improve the understanding of
human performance capabilities and limitations in aviation and the
means to measure them. Individuals' cognitive and interpersonal
skills, teams' characteristics, and organizational factors directly
shape the safety and efficiency of aviation operations. This
research will provide information to improve safety and productivity
through better equipment design, training, and system performance.
AREAS OF ONGOING RESEARCH
------------------------------------------------------ Appendix II:4.1
-- Automated Analysis of Machine Measured Performance
-- Human Performance in Inspection
-- Basic Scientific Information on Factors Impacting Controller
Performance
-- Pilot-ATC Communication: Identification of Human Factors
Associated With Effective Transfer of Information
-- Crew Resource Management (CRM) in Aircraft Maintenance and
Inspection
-- Air Crew Performance Measurement
-- Assessing Automation Impacts on Controller/Sector Performance
and Safety
-- Aviation System Safety Monitoring
-- Organizational and Environmental Factors Affecting Controller
Performance
-- Basic Scientific Knowledge of Human Performance Factors
-- Models of Aeronautical Decision-Making
-- Color Vision Deficiency and Use of Advanced Color-Coded Displays
-- Assessment of ATCS Crew Performance: Development and Validation
-- Readiness to Perform (RTP) Test Validation
-- Glare Vision Testing in the Certification of Pilots
-- Human Factors of Performance and Pilot Aging
-- Assessing Automated ATC Systems Through the Use of NAS Data
-- Organizational Impact of New Technologies on Airway Facilities
Performance
-- Human Factors Considerations in the Use of Nondestructive Test
(NDT) Equipment
-- CAMI Cabin Safety Database
-- Shiftwork in Controllers of Varying Age
-- Factors in Aircraft Accident Rates (Utilizing the Consolidated
Database)
BIOAERONAUTICS
-------------------------------------------------------- Appendix II:5
This area, which focuses on the bioengineering, biomedicine, and
biochemistry associated with performance and safety, seeks to enhance
personal performance and safety by maximizing the health and
physiological integrity of crews and passengers.
AREAS OF ONGOING RESEARCH
------------------------------------------------------ Appendix II:5.1
-- Biomedical and Clinical Support to the Federal Air Surgeon
-- Biological Effects of Low-Level Nonionizing Radiation
-- Effects of Hypoxia Induced at Altitudes of 10,000 and 12,500
Feet
-- Minimum Design Standards/Guidance for EMS Helicopter Medical
Interiors
-- Development and Enhancement of Simulation Software for Aircraft
Crash Protection
-- Impaired Performance: Impact of Prescription Drugs on
Psychomotor Responses
-- Drug Usage and Accident Investigation: Postmortem Toxicology of
Prescription Drugs
-- Development of Standards and Testing Procedures for Use of
Medical Equipment Onboard Aircraft
-- Effects of Over-the-Counter Drugs on Complex Tasks
-- Medical and Toxicological Factors of Accident Investigation
-- Cabin Air Quality
-- Safety of Beta-Blockers, ACE Inhibitors, etc., in Treatment of
Hypertension in Civilian Pilots
-- Aircraft Accidents--Role of Inflight Incapacitation: Causes of
Death in Potentially Survivable Situations
-- Improved Oxygen Masks Systems
-- Fatigue and Performance: Contribution of Hypoxia (Below 12,500
ft.) in General Aviation Pilots
-- Alcohol and Drugs (Legal and Illegal) as Factors in Aviation
Accidents: Forensic Toxicology
-- Cataract and Alternatives for Therapy: Consequences for Airman
Performance
-- Determination of Postmortem Ethanol Production in Aviation
Accidents
-- Cognitive Function Testing in the Medical Evaluation of Airmen
-- Specialty Ophthalmic Lens Use by En Route Air Traffic
Controllers
-- Aircraft Occupant Safety: Compatibility of Human Anatomy and
Biodynamics With Cabin Design and Safety Procedures
-- Human Factors and Performance: Effects of Alcohol Lower than
Allowable Under Present Regulations
-- Cosmic (Solar Particle and Galactic) Radiation Hazards at Air
Carrier Flight Altitudes
-- Cabin Safety: Aircraft Systems, Emergency Procedures, Survival
Equipment
-- Enhancing Human Protection and Survival in Civil Aviation
-- Aircraft Seats, Restraints, and Interior Systems: Crash Injury
Protection
-- Evaluation of Breathing Equipment and Oxygen System for Civil
Air Crew and Passengers
-- Cognitive and Behavioral Analysis of Operational Errors
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
RESOURCES, COMMUNITY, AND ECONOMIC
DEVELOPMNENT DIVISION, WASHINGTON,
D.C.
John H. Anderson, Jr.
Richard R. Scott
E. Jerry Seigler
Marnie S. Shaul
ATLANTA FIELD OFFICE
Veronica O. Mayhand
*** End of document. ***