Aviation Safety: Research Supports Limited Use of Personal Computer
Aviation Training Devices for Pilots (Letter Report, 07/12/1999,
GAO/RCED-99-143).

Last year, 621 people died in about 1,900 general aviation accidents.
The National Transportation Safety Board estimates that 87 percent of
all fatal general aviation accidents are caused by pilot error,
especially when pilots who do not have appropriate instrument training
fly with poor visibility, such as during bad weather. To reduce general
aviation accidents, the Federal Aviation Administration (FAA) has been
exploring ways to strengthen training for general aviation pilots. One
possible enhancement is the use of special personal computers and
software called personal computer-based aviation training devices, which
can be used for up to 10 hours of instrument training. FAA's decision to
allow the use of these devices has sparked debate. Some contend that
pilots trained with these devices will actually be less skilled. Others
argue that pilots trained with the devices are better trained at lower
cost. This report (1) describes the process and information FAA used to
approve the use of personal computer devices for 10 hours of instrument
training and (2) discusses what is known about the training
effectiveness of these devices and their long-term impact on a pilot's
ability to fly safely. FAA's decision to allow the use of computer-based
devices for instrument flight training took more than six years to be
finalized and was based on two major research studies and input from
aviation industry representatives. The two major research studies
generally support the use of computer-based devices for training.
Similarly, most experts GAO spoke with saw some training value from the
use of devices and did not believe that they were likely to degrade
aviation safety. GAO recommends that FAA gather more information on the
long-term safety issues associated with computer-based devices and
previously approved flight training devices.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  RCED-99-143
     TITLE:  Aviation Safety: Research Supports Limited Use of Personal
	     Computer Aviation Training Devices for Pilots
      DATE:  07/12/1999
   SUBJECT:  Flight training
	     Aircraft pilots
	     Computer assisted instruction
	     Aviation
	     Transportation research
	     Transportation safety
	     Data collection
	     Safety standards

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  This text was extracted from a PDF file.        **
** Delineations within the text indicating chapter titles,      **
** headings, and bullets have not been preserved, and in some   **
** cases heading text has been incorrectly merged into          **
** body text in the adjacent column.  Graphic images have       **
** not been reproduced, but figure captions are included.       **
** Tables are included, but column deliniations have not been   **
** preserved.                                                   **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

    United States General Accounting Office GAO                Report
    to Congressional Requesters July 1999          AVIATION SAFETY
    Research Supports Limited Use of Personal Computer Aviation
    Training Devices for Pilots GAO/RCED-99-143 GAO    United States
    General Accounting Office Washington, D.C. 20548 Resources,
    Community, and Economic Development Division B-280735 July 12,
    1999 The Honorable John J. Duncan Chairman, Subcommittee on
    Aviation Committee on Transportation and Infrastructure House of
    Representatives The Honorable Thomas W. Ewing House of
    Representatives In 1998, general aviation had 1,907 accidents,
    with 621 fatalities. The National Transportation Safety Board
    estimates that 87 percent of all fatal general aviation accidents
    are caused by pilot error, especially when pilots who do not have
    appropriate instrument training fly when visibility is poor, such
    as during bad weather. To reduce the occurrence of general
    aviation accidents, the Federal Aviation Administration (FAA) has
    been exploring a number of means to enhance the training of
    general aviation pilots. One possible enhancement is the use of
    new technologies for the training that occurs on the ground. For
    over 40 years, general aviation student pilots have used flight
    training devices to help them learn how to fly using an aircraft's
    instruments alone. These flight-training devices resemble an
    aircraft's cockpit and are often constructed with actual airplane
    instruments; they can be used by student pilots to substitute for
    up to 20 of the 40 hours of airplane training required by FAA to
    obtain an instrument rating. The instrument rating permits a pilot
    to fly when visibility is poor. In May 1997, FAA also approved the
    use of special personal computers, controls, and software called
    personal computer-based aviation training devices (PCATD), which
    can be used for up to 10 hours of instrument training.1 FAA's
    decision to allow the use of PCATDs has sparked debate. Some
    assert that pilots trained with these devices will be less
    skilled, thereby compromising aviation safety. Others argue that
    pilots trained with the devices are actually better trained at
    lower cost. As a result of this controversy, you asked us to (1)
    describe the process and information FAA used in deciding to
    approve the use of personal 1"Personal Computer-Based Aviation
    Training Device," or PCATD, is FAA's term for computer-based
    devices that have been approved for use, or qualified for credit
    hour training, by FAA officials. FAA requires PCATDs to be
    qualified, approved, and used in connection with an integrated
    ground and flight instrument training curriculum. Training using
    PCATDs should include the procedural tasks listed in FAA's
    Advisory Circular 61-126. In this report, we refer to these
    devices as computer-based devices before they were approved for
    use in training and PCATDs once the devices were approved for use
    by FAA. PCATDs describes one of the four FAA-approved devices.
    Page 1                   GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device B-280735 computer devices for 10 hours of
    instrument training and (2) discuss what is known about the
    training effectiveness of these devices and their long-term impact
    on a pilot's ability to fly safely. To respond to these
    objectives, we interviewed federal officials and others familiar
    with FAA's decision and reviewed FAA's supporting documentation;
    conducted a comprehensive literature search on the use of these
    and similar computer-based devices, on instrument training, and on
    general aviation safety; reviewed about 700 studies and articles;
    analyzed and summarized the most relevant data-based literature;
    interviewed government, academic, and private sector flight
    instruction experts on the use of the devices; and identified
    other issues related to the devices' potential impacts on aviation
    safety. Results in Brief    FAA's decision to allow the use of
    computer-based devices for instrument flight training took over 6
    years to be made final and was based on two major research
    studies, FAA's professional judgment, and input from aviation
    industry representatives. Because the two studies did not address
    the appropriate number of hours of training on these devices,
    FAA's decision to allow 10 hours was based on its professional
    judgment and industry input. FAA had earlier relied on its
    professional judgment, rather than empirical studies, to support
    its approval of up to 20 credit hours for the devices now known as
    flight training devices. A University of Illinois research study
    is planned to assess the appropriate number of credit hours for
    the use of these devices in an instrument training course. The two
    major research studies generally support the use of computer-based
    devices for training. Despite some methodological limitations,
    these are the most complete controlled studies to date on the
    training effectiveness of computer-based devices. One study shows
    that the use of computer-based devices may modestly reduce the
    training time spent in an airplane, while the other shows that the
    training effects of computer-based devices and previously approved
    flight training devices may not differ greatly. Other studies we
    reviewed also generally supported the use of computer-based
    devices in training. Similarly, the majority of the experts we
    interviewed saw some training value from the use of the devices
    and did not believe that they were likely to reduce aviation
    safety. Although most experts did not speculate on the appropriate
    number of credit hours that should be granted for the devices,
    several disagreed with FAA's decision to allow any credit hours.
    We found no empirical evidence, however, on the long-term safety
    impact of the devices, their potential safety benefits, or the
    long-term safety impact or benefits of the currently Page 2
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 approved flight training devices. Moreover, FAA does not
    currently collect the data needed to conduct future research on a
    possible link between the use of the devices and pilots' long-term
    safety records. This report recommends that FAA gather additional
    information on the long-term safety issues associated with
    computer-based devices and previously approved flight training
    devices. FAA will soon revise and computerize its pilot
    application form. This form could collect additional information
    at minimal cost on the use of pilot training devices and would
    facilitate future research. Background    To establish training
    requirements for pilots, FAA issues regulations for pilots, pilot
    schools, instructors, and pilot training equipment manufacturers,
    and provides guidance through advisory circulars, handbooks, and
    other types of informational material.2 Within FAA, several
    organizations regulate pilot training. For example, the Flight
    Standards Office is responsible for certifying and overseeing
    pilot schools, pilots, and pilot training equipment. Likewise,
    FAA's Civil Aeromedical Institute (CAMI) studies human performance
    factors in aviation, among other issues. Finally, FAA's National
    Simulator Program studies and evaluates the more sophisticated
    simulator equipment used in training pilots. Beginning students
    first learn to fly under FAA's visual flight rules, which
    generally require a minimum visibility of 3 miles, although most
    new pilots fly with much better visibility. Pilots then must
    obtain an instrument rating to fly in conditions of reduced
    visibility, such as in clouds or in poor weather. Flying when
    visibility is poor is more difficult because a pilot loses
    reference to the outside horizon and must rely completely on
    instruments to know the plane's spatial orientation. Instrument
    training is important to aviation safety because the vast majority
    of accidents involve poor weather and poor visibility. When a
    pilot without instrument training mistakenly flies into conditions
    of poor visibility, disorientation and loss of control can occur
    quickly; many fatal weather-related accidents occur when a pilot
    who is flying according to visual flight rules accidentally
    encounters poor visibility conditions that are beyond his or her
    abilities. According to experts, good instrument skills are
    critical to flying safely through bad weather. 2The regulation of
    pilots, pilot schools, and pilot training equipment by FAA is
    governed by title 14, C.F.R. parts 61, 91, 121, 135, and 141. Page
    3                    GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device B-280735 An instrument rating is
    considered difficult to obtain and normally requires additional
    training, an FAA written exam, and a practical test-called a
    "check ride"-with an FAA-designated examiner in an aircraft.
    Before taking these exams, pilots must take an additional 35 to 40
    hours of training, which may include up to 20 hours in a flight
    simulator or flight training device, with the remaining time in an
    aircraft.3 However, pilots typically take about 70 hours of
    training in an aircraft or training device before they are
    proficient enough to pass the tests needed to obtain their
    instrument rating. In May 1997, FAA issued an advisory circular
    approving the use of PCATDs for training credit toward the
    instrument rating.4 These aviation training devices are personal
    computers with specialized software and add-on equipment,
    including separate simulated radio and navigational controls to
    the right of the computer monitor, a throttle, a steering yoke
    under the computer screen, and rudder pedals on the floor. A fully
    equipped FAA-approved PCATD costs from $5,000 to $10,000. In
    comparison, a previously approved flight training device costs
    from $40,000 to $90,000.5 When training with either a PCATD or a
    flight training device, the instructor usually sits with a
    student, sometimes at a separate personal computer console where
    the instructor can plan, monitor, and control a student's flight.
    Figure 1 shows an FAA-approved PCATD. 3There are two types of
    pilot training schools. One is an FAA-approved school (Part 141)
    and others are nonapproved schools that still must meet FAA
    training requirements (Part 61). An FAA-approved school may be
    authorized to give its graduates written exams and check rides. In
    addition, an FAA-approved school can qualify pilots with fewer
    hours than can a nonapproved school. Many of the nonapproved pilot
    schools find it impractical to qualify for FAA certificates, but
    85 percent of all instrument ratings are granted for the training
    conducted at these schools. 4FAA Advisory Circular 61-126. 5In
    contrast, a full-motion simulator used by airlines to train
    commercial pilots can cost up to $20 million. Page 4
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 Figure 1: Personal Computer-Based Aviation Training
    Device Source: Aviation Supplies and Academics, Inc. Page 5
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 FAA's advisory circular spells out the conditions that an
    aviation training device must meet in order to be qualified by FAA
    as a PCATD for use in satisfying the instrument rating
    requirements. The advisory circular's guidelines are not mandatory
    and contain a voluntary reporting mechanism requesting information
    on students' PCATD use. FAA could have used its rule-making
    authority to approve PCATD use. It did not do so for several
    reasons, according to Flight Standards officials. First, rules are
    mandatory and advisory circulars are not, making any significant
    changes to mandatory rules more difficult and time-consuming than
    making any significant changes to advisory circulars. Second, in
    the past, the qualification of ground flight training devices in
    general aviation was handled by issuing advisory circulars.
    Finally, FAA officials wanted to facilitate the rapid introduction
    of the new PCATD technology without going through a lengthy rule-
    making process. Once FAA has qualified a PCATD-including its
    hardware, software, and accompanying equipment-the device can be
    used by a flight school or an individual flight instructor in an
    integrated ground and flight training curriculum for up to 10
    hours of flight instruction toward an instrument rating. FAA's
    Decision to Allow        FAA's decision to allow the use of
    computer-based devices and to grant the Use of Computer-Based
    credit hours evolved over many years. Since the 1980s, FAA had
    been Devices Was Based on Two       evaluating various designs of
    PCATDs but did not consider these early Studies, FAA's
    Professional    devices to be sophisticated enough for general use
    in instrument flight Judgment, and Industry         courses. As
    computer equipment advanced and training software was refined, FAA
    in 1991 began to seriously consider proposals from Input
    manufacturers and aviation schools to allow the use of computer-
    based devices for instrument flight training. Initially, FAA
    officials considered allowing the use of the devices for general
    instructional purposes only and not for students to earn flight
    hours toward an instrument rating. In 1992, Flight Standards
    officials proposed developing a research project to obtain
    information on the use of the devices because they were concerned
    that they needed empirical data to support the introduction of
    this new training technology. According to Flight Standards
    officials, FAA has never sponsored a study that evaluated the
    performance of previously approved flight training devices, which
    remain in wide use. FAA relied on its professional judgment rather
    than empirical studies to support the approval of up to 20 credit
    hours for flight training devices. Later in 1992, FAA provided a
    grant to Embry-Riddle Aeronautical University to study the use of
    computer-based devices. A principle Page 6             GAO/RCED-
    99-143 Personal Computer-Based Aviation Training Device B-280735
    objective of the study was to measure the transfer of instrument
    training knowledge to an aircraft for three groups of students
    trained using two different computer-based training devices and
    one approved flight training device. According to Flight Standards
    officials, the Embry-Riddle study was designed to provide them
    with information on whether the devices were useful and effective.
    In June 1993, while the Embry-Riddle research was ongoing, CAMI's
    Human Factors Research Laboratory submitted a research proposal to
    expand and participate in the study. This detailed approach was
    not approved, according to a Flight Standards official, because
    FAA at the time did not have the funds to support it. For the next
    year, CAMI officials continued to help explore possible research
    methodologies for qualifying the devices in a structured training
    curriculum that would provide some insight into determining if
    credit hours should be granted and what tasks the devices are best
    suited for in instrument training. However, at that time, FAA was
    not proposing to grant flight training credit hours for training
    on computer-based devices. When the Embry-Riddle study was
    completed in 1994, some FAA officials outside of Flight Standards
    believed that the study had significant limitations, was not
    properly designed, and did not fully answer important questions on
    whether computer-based devices should be approved. CAMI officials
    stated that the failure to include a control group that received
    training only in the aircraft seriously limited the usefulness of
    the study. In addition, officials in FAA's National Simulator
    Program stated that no meaningful conclusion could be drawn from
    the research. In June 1994, FAA sponsored a meeting of
    manufacturers and others in the aviation training industry to
    obtain their views on FAA's current and future direction in
    approving the use of computer-based devices. In addition, starting
    in 1994, FAA sponsored further research through a series of grants
    to the University of Illinois to study the training effectiveness
    of the devices and to answer many of the questions and concerns
    about their use that remained following the Embry-Riddle study.
    While FAA was evaluating the qualifications of computer-based
    devices for instrument training without credit hours, it received
    requests to allow some of the training hours done on the equipment
    to be credited toward the minimum hours of flight time in
    instrument training. Some manufacturers, for example, wanted FAA
    to consider granting credit hours in order to make their equipment
    more credible to their customers. Furthermore, in May 1995, the
    Aircraft Owners and Pilots Association's Air Safety Foundation
    formally petitioned FAA to allow training on Page 7
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 computer-based devices to count for training time toward
    an instrument rating and proposed a syllabus illustrating the
    integration of the devices into an instrument flight course.
    Although neither the Embry-Riddle nor the University of Illinois
    study addressed any appropriate number of flight training credit
    hours, FAA officials used their experience and professional
    judgment to support their decision and saw it as a reasonable
    compromise between the various proposals. Furthermore, FAA
    officials decided to add a voluntary reporting mechanism to the
    draft advisory circular designed to gather information to validate
    the approval and use of the devices with feedback from users.
    Finally, FAA officials told us that the required check ride by an
    FAA-designated examiner after a student's training would ensure
    that all instrument students were qualified to receive the
    instrument rating, no matter what equipment they used during
    training. Several flight training experts we interviewed, however,
    told us that the check ride is an imperfect measure of a pilot's
    ability, should not be seen as assurance of proper training, and
    is instead a snapshot of a pilot on a single day. These experts
    believe assurances on the quality of training devices, a quality
    curriculum, capable instructors, consistent testing by FAA-
    designated flight examiners, and long-term research on pilot
    safety and training are needed. In August 1995, FAA sponsored an
    additional meeting of aviation industry officials and presented a
    draft advisory circular that proposed, for the first time,
    granting 10 flight training credit hours. The draft advisory
    circular and all significant changes to the draft were submitted
    to over 60 groups, organizations, and other interested parties.
    For the next 12 months, FAA officials received numerous comments
    on the use of computer-based devices. During this time, several
    aviation industry officials suggested that FAA delay or withhold
    approval of a final advisory circular pending more research and
    the completion of the University of Illinois study. In September
    1996, FAA decided to delay approving the advisory circular until
    after the University of Illinois study was completed. In November
    1996, the University of Illinois issued its final report and in
    March 1997 briefed FAA officials on the results. The study
    concluded, in part, that computer-based devices can provide a
    worthwhile training benefit and can save a small portion of the
    aircraft time that would otherwise be needed. In January 1997, the
    National Air Transportation Association, after extensive
    discussion, recommended that FAA approve a draft advisory
    circular, including flight training credit hours for computer-
    based devices, as well as other recommendations. Page 8
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 In May 1997, FAA issued Advisory Circular 61-126, which
    created a separate category of training device-PCATD-rather than
    integrating PCATDs into the "ground training device" category
    already approved for credit hours in instrument training. These
    existing ground training devices were renamed flight training
    devices (FTD) to distinguish them from PCATDs. Older models of the
    new FTD category were grandfathered in and automatically approved
    by FAA for instrument credit hours. According to Flight Standards
    officials, the decision to approve 10 hours of flight credit was
    based on (1) the general utility of computer-based devices in
    instrument training, as shown by the Embry-Riddle and University
    of Illinois studies; (2) a wide variety of proposals, ranging from
    zero hours of credit to 20 hours; and (3) the input from
    manufacturers that they needed credit hours to sell their
    equipment. Two additional research studies are planned: One will
    try to assess the appropriate number of credit hours for PCATD use
    in an instrument course; the second will investigate the use of
    PCATDs to maintain instrument skills.6 Over the next several
    months, FAA officials qualified four PCATD devices, which included
    three software programs and several other manufacturers of
    controls, while finding an equal number unacceptable. As of June
    1999, no information had been sent to FAA under the advisory
    circular's voluntary reporting mechanism on the use of PCATDs. An
    official of the National Association of Flight Instructors
    estimated that very few of the 200 Part 141 flight schools now use
    PCATDs for instrument instruction. 6This University of Illinois
    research study is likely to be funded by the National Aeronautics
    and Space Administration's Ames Research Laboratory, which
    investigates human factors issues and funds research investigating
    general and commercial aviation safety. The study proposes to
    investigate, over 3 years, the progress of three groups of
    instrument students using PCATDs for 5, 10, and 15 hours of
    training in an instrument training course. A control group would
    receive all training in an airplane. The use of flight training
    devices will not be included in the study. An additional study is
    now examining the training value of PCATDs for maintaining pilots'
    skills and for substituting these devices for aircraft experience
    in maintaining instrument skills. According to a University of
    Illinois researcher, the airplane performance of 100 current
    instrument pilots will be tested and compared across four groups:
    A control group will receive no practice in instrument flight
    during a 6-month period; an airplane group will perform the six
    required approaches and holding patterns in the aircraft; a PCATD
    group will perform the same maneuvers in a PCATD; and a flight
    training device group will perform the same maneuvers in a flight
    training device. The performance of pilots on the final instrument
    proficiency check ride at the end of the 6-month experimental
    period will be compared with the performance on the initial
    instrument proficiencies check ride. Pilots may not receive
    recency credit for their practice on PCATDs. To meet FAA recency
    regulations toward maintaining current instrument skills, pilots
    must make at least six instrument approaches, holding procedures,
    and intercepting and tracking courses every 6 months. These
    maneuvers must be performed under either simulated or actual
    flight conditions; currently, flight training devices may be used
    to perform all six instrument currency approaches, but PCATDs may
    not. Page 9                   GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device B-280735 Research and Experts
    The two FAA-commissioned studies by the University of Illinois and
    Support Training             Embry-Riddle Aeronautical University
    are the most complete controlled studies to date on the training
    effectiveness of computer-based devices. Value of PCATDs, but
    Despite methodological limitations, the studies show that (1) the
    use of Devices' Impact on           computer-based devices may
    modestly reduce the training time spent in the airplane (Illinois)
    and (2) the training effects of computer-based Long-Term Safety Is
    devices and a previously approved flight training device may not
    differ Unknown                      greatly (Embry-Riddle). The
    additional studies we reviewed also supported the use of computer-
    based devices. None of these studies examined the issue of how
    many hours of training on a computer-based device should be
    allowed in an instrument training curriculum. Academic and other
    flight training experts generally believed that computer-based
    devices offer training value to pilots obtaining an instrument
    rating. Since PCATDs have only recently been approved and few
    pilots have used them in training, their long-term risks and
    benefits are not yet known. While several studies examined the
    training effectiveness of computer-based devices, none directly
    addressed their effects on general aviation safety. Safety experts
    we interviewed did not believe that the use of computer-based
    devices and the granting of credit hours pose a threat to general
    aviation safety by adversely affecting the skills and abilities of
    pilots who receive instrument training using the devices. However,
    several other issues that could affect the long-term ability of
    PCATD-trained pilots to fly safely have been raised. Research
    Generally           The Illinois study showed that the use of
    computer-based devices in an Supports Use of PCATDs in
    instrument course may modestly reduce the training time spent in
    the Training                     airplane-26.5 hours on the device
    during an instrument course reduced the amount of airplane
    training time needed by 3.9 hours. (See. fig. 2.) Page 10
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 Figure 2: University of Illinois Study: Hours to
    Proficiency in an Airplane        Hours Hours using PC Hours in
    airplane Notes: N = Number of students trained. The airplane used
    was a Beechcraft Sports/Sundowner. The computer-based device was
    an MDM FS-100, not FAA-approved as a PCATD. Source: GAO's analysis
    of University of Illinois data. The Embry-Riddle study provided a
    preliminary indication that there may not be very large
    differences between the training effects of the previously
    approved flight training devices and the recently approved
    computer-based devices. The training trials and time needed to
    reach proficiency for students using two different types of
    computer-based devices and for students using a previously
    approved flight training device are shown in figures 3 and 4. (See
    app. II for a more detailed summary of the two FAA-sponsored
    studies.) Page 11                   GAO/RCED-99-143 Personal
    Computer-Based Aviation Training Device B-280735 Figure 3: Embry-
    Riddle Study: Trials to Proficiency in an Airplane
    Trials Training device used Notes: N = Number of students trained.
    The airplane used was the Mooney 20J. The Elite PC was a computer-
    based device, an FAA-approved PCATD. The IFT PC was a computer-
    based device not approved by FAA as a PCATD. The Frasca FTD is a
    flight-training device approved by FAA. Source: GAO's analysis of
    Embry-Riddle Aeronautical University data. Page 12
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 Figure 4: Embry-Riddle Study: Hours to Proficiency in an
    Airplane          Hours Training device used Notes: N = Number of
    students trained. The airplane used was the Mooney 20J. The Elite
    PC was a computer-based device, an FAA-approved PCATD. The IFT PC
    was a computer-based device not approved by FAA as a PCATD. The
    Frasca FTD is a flight-training device approved by FAA. Source:
    GAO's analysis of Embry-Riddle Aeronautical University data. The
    additional studies we reviewed also found some value in using
    computer-based devices. Three studies found computer-based devices
    helped train beginning pilots.7 An additional study found that
    performance in an airplane test was better for a group of students
    trained on 7Gustavo Ortiz, "Effectiveness of PC-Based Flight
    Simulation," The International Journal of Aviation Psychology 4
    (3), p. 285, 1994; Kerry A Dennis and Don Harris, "Computer-Based
    Simulation as an Adjunct to Ab Initio Flight Training," The
    International Journal of Aviation Psychology 8 (3), pp. 261-276,
    1998; Jefferson M. Koonce, Steven L. Moore, and Charles J. Benton,
    "Initial Validation of a Basic Flight Instruction Tutoring System
    (BFITS)," Proceedings of the Eighth International Symposium on
    Aviation Psychology, Columbus, Ohio, 1995. Page 13
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    B-280735 computer-based devices than for a group trained on flight
    training devices.8 Experts See Training Value    The academic and
    flight training experts as well as the flight instructors in
    Computer-Based             we interviewed generally believed that
    PCATDs offer training value to Devices
    instrument students. Experts told us that training devices, if
    used properly, have the potential to be more effective than
    airplane training for certain uses; furthermore, training experts
    generally agree that a low-cost simulation device may be justified
    economically even if the training it provides is less effective
    than the training provided in an airplane. In addition, commercial
    airline training and military research support the value of
    computer-based training for certain training purposes. Several
    experts we interviewed believed that computer-based devices would
    become more effective as their technology continues to improve and
    as features are added to enhance their training effectiveness.
    Some experts believe that a variety of training equipment,
    including computer-based devices, can offer effective training if
    used properly. PCATDs are usually seen by experts as most
    appropriate for introducing equipment and procedures and for
    practicing certain maneuvers, flight scenarios, and emergency
    procedures. Flight instructors in one survey believed that
    computer-based devices were effective for teaching most tasks in
    an instrument course.9 Flight training experts at a major
    airline's training center told us that they and other airlines use
    a variety of training equipment in their operations, 8Sybil I.
    Phillips, Charles L. Hulin, and Paul J. Lamermayer, "Uses of Part-
    Task Trainers in Instrument Flight Training," University of
    Illinois, Proceedings of the Seventh International Symposium on
    Aviation Psychology, Columbus, Ohio, 1993. 9William F. Moroney,
    Steven Hampton, and David W. Biers, "Considerations in the Design
    and Use of Personal Computer-Based Aircraft Training Devices
    (PCATDs) for Instrument Flight Training: A Survey of Instructors,"
    University of Dayton, Ohio, and Embry-Riddle Aeronautical
    University, Daytona Beach, Florida, Proceedings of the Ninth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1997. According to another study (Brian K. Rogers, Capt., U.S. Air
    Force. "Microcomputer-Based Instrument Flight Simulation:
    Undergraduate Pilot Training Student Attitude Assessment,"
    December 1991, Human Resources Directorate, Aircrew Training
    Research Division, Williams Air Force Base, Arizona, AL-TR-1991-
    0039), surveyed students believe that computer-based devices have
    some training value. Almost all of the of U.S. Air Force flight
    students surveyed planned to use computer-based instrument
    simulators to replace at least some of their "chair-flying," or
    rehearsing procedures from a chair placed in front of a life-size
    photo of the airplane's cockpit. Over half said they would replace
    at least 75 percent of their chair-flying with computer-based
    simulation. The same study found increased student interest in
    using computer-based simulation as instrument training was
    expanded to include multiple procedures in real-time practice.
    Another study (D. Gopher, M. Weil, and T. Bareket, "Transfer of
    skill from a computer game trainer to flight," Human Factors, 36
    (3) (1994), pp. 387-405) found that military flight students who
    played a complicated computer game performed better in later
    airplane training. Page 14                  GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device B-280735 ranging
    from $20 million full-motion simulators to simple personal
    computer-based training, depending on whether their goal is to
    introduce a pilot to a new procedure or piece of equipment, to
    practice complete simulated flights, or to serve an intermediate
    purpose. For example, these airline training officials believed
    that simple personal computer equipment can introduce a pilot to a
    new navigation system or component more effectively than a full-
    motion simulator with its many competing demands for a pilot's
    attention. Safety and Academic         According to safety experts
    from the National Transportation Safety Board Experts Generally Do
    Not    and from an industry safety association, the use of
    computer-based Believe PCATDs Pose a       devices and the
    granting of credit hours do not pose a threat to general Safety
    Risk                 aviation safety by adversely affecting the
    skills and abilities of pilots who receive instrument training
    using the devices. In general, academic and other flight training
    experts believe that the devices offer training value to pilots
    who want to obtain an instrument rating. According to these
    experts, good instrument and navigational skills are vital in
    avoiding bad weather and surviving it when necessary; two experts
    we interviewed believe that the low cost and wide availability of
    PCATDs could help maintain or increase the general level of
    pilots' instrument skills, which are critical to safe flying.
    Long-Term Safety Impact     We found no empirical evidence on the
    long-term safety impact of of PCATDs Is Unknown        computer-
    based devices, their potential safety benefits, or the long-term
    safety impact or benefits of the currently approved flight
    training devices. Many of the experts we interviewed believed
    either that computer-based devices were not likely to reduce
    aviation safety or that they offer long-term safety benefits.
    However, several experts noted that the potential for long-term
    safety risks from computer-based devices used in training remains
    a possibility. Objections to their use for credit towards the
    training hours needed for an instrument rating generally revolve
    around whether using a computer-based device in instrument
    training is more likely to reduce or increase an instrument
    pilot's margin of safety. Several issues were raised in our
    interviews with experts concerning PCATD safety. Among these were
    potential risks from the lack of physical similarity between the
    device and the airplane and associated issues, possible benefits
    to training from their use, and the lack of research showing that
    one hour on a training device equals one hour of training in an
    airplane. Some of these issues appear to support the value of
    PCATDs; others raise questions about their value; and the effects
    of other issues on Page 15             GAO/RCED-99-143 Personal
    Computer-Based Aviation Training Device B-280735 the value of
    PCATDs are unclear. Although not fully resolved, these issues may
    be relevant to future research. (See app. III for a summary of
    these issues.) FAA does not currently track how many hours an
    instrument pilot trains on a PCATD. However, FAA is now
    computerizing this information and, for unrelated reasons, plans
    to revise the form on which pilots apply for the instrument and
    other ratings. FAA's application for the instrument rating could
    easily and inexpensively be changed to include a box on the types
    of training devices a pilot has used, particularly since the form
    (FAA Form 8710-1) will soon be revised and its data computerized.
    With information on the training devices used by a particular
    pilot, along with other data that FAA now maintains or will soon
    maintain electronically, researchers will be better able to
    determine, within a few years, the safety of pilots trained using
    computer-based devices. These data will enable researchers to link
    accidents or incidents with the type of instrument training the
    pilots had received. While this type of examination would present
    some statistical challenges, experts we interviewed said that it
    would be possible to identify any large safety risks posed by
    computer-based devices. Conclusions    Commercial aviation safety
    depends on many factors, including the training of general
    aviation pilots, who sometimes share common airspace and airports
    with the flying public. Because most accidents in general aviation
    are caused by pilot error, and many accidents occur when pilots
    without an instrument rating encounter bad weather or poor
    visibility and unexpectedly have to rely on their instruments,
    effective training that teaches, reinforces, and maintains safe
    instrument flying skills is vital. To the extent that the use of
    computer-based devices may be of training value to instrument
    pilots-which research supports and experts believe-the devices do
    not pose identifiable safety risks. The low cost and wide
    availability of computer-based devices could improve the general
    level of instrument training among pilots, according to some
    experts, and help to maintain or improve instrument pilots'
    skills, especially for students and pilots who would not otherwise
    have access to a training device. Several experts told us that
    FAA's check rides, which are required for a pilot to obtain an
    instrument rating, may not be sufficient indicators of a student's
    training, capabilities, and ability to fly safely. However, we
    found no studies on the link between a pilot's training equipment
    or flying proficiency at the time of training and that pilot's
    Page 16            GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device B-280735 long-term ability to fly safely.
    Collecting information on pilots' training equipment and safety
    records would permit research in the future and could help to
    uncover any safety problems related to the use of computer-based
    and other training devices. Recommendations    As a first step
    toward determining whether computer-based devices pose safety
    risks to general aviation in relation to other training methods,
    the Secretary of Transportation should direct the FAA
    Administrator to collect information from pilot applications for
    instrument ratings on how many hours students trained on PCATDs
    and flight training devices. This information can be used by FAA
    and others to study the relationship between the use of training
    devices and safety. FAA's revision to its rating application form
    allows the additional information to be obtained at minimal cost.
    Agency Comments    We provided a draft copy of this report to the
    Office of the Secretary of Transportation and the Federal Aviation
    Administration (FAA) for review and comment. We discussed the
    report with FAA officials, including the Deputy Associate
    Administrator for Regulation and Certification. They generally
    agreed with the draft report. FAA officials also indicated that
    they concurred with and would implement our recommendation as soon
    as possible. They also provided technical corrections, which we
    incorporated into the report as appropriate. Scope and          We
    reviewed FAA's decision to allow the use of PCATDs by reviewing
    the Methodology        data the agency used to support its
    decision and interviewing a variety of FAA and aviation industry
    officials. We also reviewed research on the use of computer-based
    devices in training and interviewed a variety of experts on the
    potential safety impacts of their use. Appendix I provides our
    detailed scope and methodology. We conducted our review from June
    1998 through June 1999 in accordance with generally accepted
    government auditing standards. As arranged with your offices,
    unless you announce its contents earlier, we plan no further
    distribution of this report until 7 days after the date of this
    report. At that time, we will make copies available to the
    Secretary of Page 17            GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device B-280735 Transportation and the
    Administrator, FAA. We will also make copies available to others
    on request. If you have any questions about this report, please
    contact me at (202) 512-3650. Major contributors to this report
    were Karen Bracey, David Ehrlich, and John Rose. Gerald L.
    Dillingham Associate Director, Transportation Issues Page 18
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    Page 19      GAO/RCED-99-143 Personal Computer-Based Aviation
    Training Device Contents Letter
    1 Appendix I
    22 Scope and Methodology Appendix II
    25 Summaries of Two         University of Illinois Study
    25 Embry-Riddle Aeronautical University Study
    26 Major FAA-Sponsored Studies Appendix III
    28 Other Issues Related     Views on the Lack of Physical
    Similarity Between PCATDs and                        28 Aircraft
    Instrument Panels and Controls to PCATD Safety Selected
    Bibliography
    41 Figures                  Figure 1: Personal Computer-Based
    Aviation Training Device                          5 Figure 2:
    University of Illinois Study: Hours to Proficiency in an
    11 Airplane Figure 3: Embry-Riddle Study: Trials to Proficiency in
    an Airplane                 12 Figure 4: Embry-Riddle Study: Hours
    to Proficiency in an                           13 Airplane Figure
    III.1: Approved Flight Training Device
    29 Figure III.2: Instrument Panel Photo of Cessna 172
    32 Figure III.3: PCATD Depiction of Cessna 172 Instruments
    33 Figure III.4: Flight Path Screen
    35 Figure III.5: Screen to Set Failures
    36 Figure III.6: Screen to Set Weather Conditions
    37 Figure III.7: Example of Grandfathered Flight Training Device
    39 Abbreviations CAMI           Civil Aeromedical Institute FAA
    Federal Aviation Administration FTD            flight training
    device PCATD          personal computer-based aviation training
    device Page 20               GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device Page 21      GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device Appendix I Scope
    and Methodology To examine the Federal Aviation Administration's
    (FAA) decision allowing the use of personal computer-based
    aviation training devices (PCATD), we reviewed the data the agency
    used to support its decision to issue Advisory Circular 61-126,
    reviewed other studies available on the topic, and interviewed a
    variety of experts within and outside government. In particular,
    we reviewed the supporting data FAA used in making its decision,
    including a 1994 study by Embry-Riddle Aeronautical University and
    a 1996 study by the University of Illinois on the effectiveness of
    PCATD training. We reviewed FAA's files, including archived e-mail
    messages, a chronology of events, and supporting information and
    documents. We met with FAA officials from Flight Standards, both
    in Washington, D.C., and at an Illinois Flight Standards District
    Office, as well as with FAA officials from Human Factors, the
    Office of Aviation Medicine, the Civil Aeromedical Institute
    (CAMI) in Oklahoma City, and the National Simulator Program in
    Atlanta. We asked several recognized academic and training experts
    to review our plans at several points during our research. Each of
    these experts has extensive academic, research, and training
    experience in aviation, including the fields of aviation
    psychology and human factors; their views are theirs alone rather
    than those of their institutions. These experts were Dr. Barry
    Strauch of the National Transportation Safety Board; Dr. Dee
    Andrews of the U.S. Air Force Research Laboratory, Mesa, Arizona;
    and Professor (Emeritus) Stanley Roscoe of the University of
    Illinois at Urbana-Champaign and New Mexico State University. In
    addition, we interviewed flight training, aviation psychology,
    human factors, military research, and aviation safety experts in
    academia and the military, as well as at the National
    Transportation Safety Board and the National Aeronautics and Space
    Administration. We interviewed flight instructors, association and
    airline officials, the manufacturers of each of the approved
    PCATDs, and the manufacturers of flight training devices and
    control equipment. In particular, we interviewed representatives
    of the following organizations: the National Association of Flight
    Instructors; the National Air Transport Association; the
    University Aviation Association; the Airplane Owners and Pilots
    Association and its affiliated Air Safety Foundation; two flight
    schools in the Chicago area; two flight schools in the Washington,
    D.C., area; and the chief flight instructors of the University of
    Illinois and Embry-Riddle Aeronautical University at Daytona
    Beach, and other selected flight instructors. Among the equipment
    Page 22            GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device Appendix I Scope and Methodology
    manufacturers and distributors we interviewed were Aviation
    Supplies & Academics, Inc.; Aviation Teachware Technologies;
    Jeppesen Sanderson; and Frasca International, Inc. Among the
    academic experts we contacted, in addition to our advisory panel,
    were the authors of both the University of Illinois and Embry-
    Riddle Aeronautical University studies. At FAA, we spoke to
    officials in Flight Standards; the National Simulator Center
    (Atlanta); the Civil Aeromedical Institute (Oklahoma City); Human
    Factors technical scientists (Washington, D.C.); and FAA's Flight
    Standards District Office (West Chicago, Illinois). We also spoke
    to representatives of the U.S. Air Force, the U.S. Navy, the
    National Transportation Safety Board, and the National Aeronautics
    and Space Administration (Ames, California). To identify other
    studies relevant to FAA's decision, we obtained articles cited in
    recent studies. We also conducted additional literature searches
    of the collections at two major aviation libraries and a
    computerized literature search using several bibliographic
    databases and including the following keywords and phrases:
    personal computer aviation training device, PCATD,
    accident/incident, flight training device, ground training device,
    part task trainers, aviation safety, pilot training, air safety.10
    Our computerized literature search identified about 700 studies
    and articles. Of these, we selected 68 dealing with personal
    computers, simulation issues, or aviation safety, including all
    the citations that appeared to involve, discuss, or present
    empirical research in these areas. Thirty-nine abstracts were
    further selected as relevant enough to obtain full text copies,
    including all those that appeared to involve empirical research on
    personal computer-based flight training. From all our sources,
    including electronic and nonelectronic literature searches, we
    found eight studies involving computer-based device effectiveness,
    and four of these met all of the following criteria: (1) examined
    computer-based flight simulation effectiveness; (2) compared
    computer-based device training to airplane-only training
    performance or to training with another approved training device;
    (3) had been conducted using an experimental design or survey,
    including random assignment; and (4) used data from an experiment
    not summarized elsewhere in our report (i.e., the two studies
    cited by FAA in its decision-one at Embry-Riddle Aeronautical
    University and one at the University of Illinois). In addition, we
    reviewed studies and 10The databases searched included the
    National Technical Information Service and Transportation Research
    Information Services; DIALOG, from which the following databases
    were searched: Dissertation Abstracts Online, GPO Monthly Catalog,
    PsychINFO, IAC Aerospace/Defense Markets, Aerospace Database,
    McGraw-Hill Publications, Periodical Abstracts Plus text, DIALOG
    Defense Newsletters, Business Dateline, IAC Newsletter Database;
    and the NEXIS database. Page 23                 GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device Appendix I Scope
    and Methodology articles on general aviation flight safety that
    did not directly mention either computer-based simulation or
    training. Many articles in this field are published in journals
    with limited circulation and in the proceedings of civilian and
    military conferences. Though these can be difficult to identify
    from conventional bibliographic databases, we attempted through
    these combined methods to obtain all those studies examining
    computer-based flight training that used empirical methods. We
    conducted our review from June 1998 through June 1999 in
    accordance with generally accepted government auditing standards.
    Page 24            GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device Appendix II Summaries of Two Major FAA-
    Sponsored Studies This appendix summarizes the studies by the
    University of Illinois and the Embry-Riddle Aeronautical
    University. University of Illinois    In the University of
    Illinois study,11 researchers compared the performance Study
    of students who received some training on a PCATD before their
    training in an airplane with the performance of a group of
    students trained entirely in an airplane. University of Illinois
    aviation students who were already private pilots and had no
    previous instrument coursework were randomly assigned to one of
    two groups for their instrument flight-training course. After this
    study's coursework, passing students took FAA check rides to
    attempt to obtain their instrument rating. Performance measures
    were available for 74 students; roughly half the students were in
    each group. Students were tested to demonstrate proficiency, and
    the time and number of trials were recorded for each student. The
    computer-based device used was Model FS-100 by MDM, a model not
    currently approved for use as a PCATD by FAA toward flight
    training credit hours; the airplane used was Beechcraft
    Sports/Sundowner, which has a single engine, a fixed-pitch
    propeller, and fixed landing gear. Students' performance in the
    airplane was not substantially different for those trained partly
    using a computer-based device and those trained solely in an
    airplane. However, the data showed small, but statistically
    significant, savings in the total airplane time needed to reach
    proficiency in the airplane by students trained on the PCATD
    device. Students using this device spent, on average, 3.9 fewer
    hours in the airplane than those who took their entire training in
    the airplane. Airplane-only students averaged 51.3 total hours to
    complete the course, while students using the PCATD averaged 47.4
    hours in an airplane and 26.5 on the device. This study was not
    designed to determine the amount of loggable time FAA should grant
    for PCATD use, according to its authors. It did not include a
    student group trained on a currently approved flight training
    device. Such a comparison group is of interest because the 10
    hours currently permitted for PCATD instrument training may be
    substituted for part of the 15 to 20 hours that can be earned
    using a flight training device. The study's authors believe that
    in an actual instrument course, where experimental controls would
    not require the use of the PCATD for all course lessons, the 3.9
    hour savings of time in the airplane could be achieved with fewer
    PCATD hours 11Henry L Taylor, Gavan Lintern, Charles L. Hulin,
    Donald Talleau, Tom Emanuel, and Sybil Phillips; "Transfer of
    Training Effectiveness of Personal Computer-Based Aviation
    Training Devices," University of Illinois; November 1996, Final
    Technical Report ARL-96-3-/FAA-96-2, prepared for the Federal
    Aviation Administration, Oklahoma City, OK, Contract DTFA-94-G-044
    Page 25                 GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device Appendix II Summaries of Two Major FAA-
    Sponsored Studies by targeting the use of the device to those
    early course lessons where training transfer is greatest. The
    study does not support a conclusion that a "negative transfer"-or
    learning interference caused by using PCATDs-occurred when the
    students trained on the personal computer device switched to the
    airplane. In the few instances in which this group of students
    needed more trials to demonstrate proficiency in the airplane than
    the group without PCATD training, the differences were so small
    that they were not statistically significant. Differences of the
    magnitude observed could occur even if the two groups of students
    had been given exactly the same training. While the study does not
    support a conclusion that negative transfer occurred on some
    tasks, it also does not rule out the possibility that it did
    occur. Small amounts of negative transfer in an individual task
    could have existed without being detected by statistical tests.
    Embry-Riddle        The Embry-Riddle study measured the flight
    performance of students Aeronautical        trained using PCATDs
    and compared their performance with that of students trained using
    a previously approved flight training device.12 All University
    Study    the students had previously taken navigation and
    instrument coursework. Embry-Riddle students who volunteered for
    the study were randomly assigned to receive ground-based training
    using one of two PCATDs or using the flight training devices.
    Students used the assigned ground-training devices during the
    first part of their training and continued their instrument flight
    training in an airplane. Instructors were randomly assigned to
    students and taught students in each of the three groups. In the
    airplane, the performances of the three groups of students were
    measured by the total amount of time and the total number of
    trials students required to demonstrate proficiency in each of
    eight maneuvers. Performance measures were available for a total
    of 66 students. The PCATDs used were the Elite (20 students
    finished) and the IFT Pro 5.1 (26 students finished); both devices
    had 15-inch screens, 486/66MHz computers, and similar control
    features. The flight training device used was the Frasca 141 (20
    students finished). The airplane used was a Mooney 20J single-
    engine plane. The study provided a preliminary indication that
    there may not be very large differences in the ground flight
    training provided by the approved 12Steven Hampton, William
    Moroney, Tom Kirton, and David W. Biers, "The Use of Personal
    Computer-Based Training Devices in Teaching Instrument Flying,"
    Embry-Riddle Aeronautical University, June 15, 1994. Prepared for
    the Federal Aviation Administration, CAAR 15471931, Grant No. 92-
    G-0015. Page 26                 GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device Appendix II Summaries of Two Major
    FAA-Sponsored Studies flight training devices and the two PCATDs
    studied. No statistically significant differences were found in
    the airplane performance of the groups trained on the three
    devices, as measured by trials and hours to meet FAA's established
    practical test standards. The study did not and was not intended
    to address the issue of how many hours of training on PCATDs FAA
    should grant for an instrument rating, according to one of the
    study's authors. The study also did not show how much, if any,
    training received on each of the ground-based devices was
    transferred to the airplane. The performance of a group of
    students who received all their training in an airplane would be
    required before an estimate of how much less time in an airplane
    students receiving training on a PCATD or flight training device
    might need. Because no empirical studies have been conducted on
    the effectiveness of training on a flight training device compared
    with airplane-only training for instrument students, this study's
    comparative data cannot be used to conclude that PCATDs provide
    effective instrument training. Page 27             GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device Appendix III
    Other Issues Related to PCATD Safety We identified a number of
    other issues from interviews with safety, industry, and academic
    experts that could affect the long-term use of computer-based
    devices. These issues are the lack of the devices' physical
    similarity to actual airplanes and the potential for associated
    problems from negative learning from simulation-that is, a pilot's
    reverting to incorrectly learned behaviors in an emergency or
    under stress-and the possibility that the smaller-sized
    instruments on the devices could interfere with a pilot's
    instrument scan. Possible benefits from PCATDs include their
    availability and use in teaching concepts and procedures, such as
    the flexibility they provide by permitting a pilot to practice on
    a particular type of airplane, to see a graphic display of
    performance, and to fly under varying scenarios of instrument
    failure, system failure, and weather conditions. Several
    simulation experts believe that credit hours should not be granted
    for PCATD use, that the two major studies do not show the training
    effectiveness of either PCATDs or flight training devices, and
    that there is no evidence that one hour in any training device
    equals one hour in an airplane, particularly for the older,
    grandfathered flight training devices. Views on the Lack of    One
    of the main concerns expressed by those opposed to granting credit
    Physical Similarity     for instrument training time on PCATDs is
    that PCATDs do not have sufficient similarity, or physical
    fidelity, to an airplane's instrument panel and Between PCATDs and
    controls.13 For example, PCATDs have display screens showing
    instruments Aircraft Instrument     that are smaller than they
    actually are in the cockpit, and the placement of some PCATD
    switches and dials represented on the display screen are in a
    Panels and Controls     physically different position than in the
    aircraft-alongside or below a desktop computer monitor. Flight
    training devices, in contrast, can include more instruments-often
    actual aircraft instruments-that can be mounted on a dashboard
    panel larger than a computer screen, permitting greater physical
    similarity to an aircraft. On the other hand, flight training
    devices represent a standard aircraft configuration, while PCATD
    software can be easily changed to represent a specific aircraft.
    Figure III.1 shows an approved flight training device. 13The
    literature discusses at least three different kinds of fidelity,
    including physical fidelity-whether the simulation "looks right;"
    functional fidelity-whether the simulation "acts right;" and
    psychological fidelity-whether the simulation "feels right." Page
    28                  GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device Appendix III Other Issues Related to
    PCATD Safety Figure III.1: Approved Flight Training Device Note:
    This is a previously approved flight training device with an
    optional visual system (Frasca 131). Source: Frasca International.
    Page 29                 GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device Appendix III Other Issues Related to
    PCATD Safety The potential impact of these differences between
    PCATDs and flight training devices on pilot training is not clear.
    Some experts and representatives of the flight training device
    industry believe that devices that are less similar to the
    airplane and have lower physical fidelity could interfere with
    learning. Some simulation experts believe that the more the
    simulator looks, feels, and "flies" like the airplane, the greater
    its training value will be, particularly for experienced pilots.
    However, other experts and many research studies suggest that high
    physical fidelity is not essential for a device to have training
    value; that students using low-fidelity devices often perform as
    well as those using more complex, higher-fidelity devices; and
    that the effectiveness of a training device depends on how the
    device is used in a training curriculum.14 Negative transfer-
    learning that can interfere with the performance of a task rather
    than improve it-can occur when a familiar instrument or knob is
    placed in a dramatically different position in a training device
    than in an aircraft. One expert on aviation psychology told us
    that negative transfer from PCATDs would be unlikely but possible.
    A pilot's switching from a training device to an airplane, or from
    one airplane to another, according to interviews with aviation
    psychologists, would rarely involve locating control yokes, knobs,
    or switches that are completely reversed. According to one expert,
    however, if a pilot trains in a plane with the retractable landing
    gear switch on the left side of the cockpit and the flap switch on
    the right and then flies another plane with the landing gear
    switch on the right and the flap switch on the left, negative
    transfer of learning could occur if a pilot were to confuse the
    two switches. In general, the academic, private, military, and
    airline flight training experts we interviewed did not believe
    that the issue of PCATDs' similarity to the airplane presented
    safety problems. Several safety experts noted that previously
    approved flight training devices do not typically represent one
    particular type of aircraft and often include instrument panels
    that differ from those on the plane in which the student will
    train and later fly; students will have to adjust from the
    training device to a somewhat different airplane. While
    familiarity with a particular type of plane is an important safety
    consideration, few students will fly only one type of plane and
    therefore will have to adjust, using training devices, to
    different 14Gavan Lintern, S.N. Roscoe, J.M. Koonce, and L.D.
    Segal, "Transfer of Landing Skill in Beginning Flight Training,"
    Human Factors 32, (1990), pp. 319-327; Gavan Lintern, Henry L.
    Taylor, Jefferson M. Koonce, and Donald Talleur, "An Incremental
    Transfer Study of Scene Detail and Field of View Effects on
    Beginning Flight Training," Proceedings of the Eighth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1995, p. 737. Page 30                  GAO/RCED-99-143 Personal
    Computer-Based Aviation Training Device Appendix III Other Issues
    Related to PCATD Safety instrument displays, a different feel to
    the controls, and different performance characteristics. Another
    potential danger could occur when a pilot, in an emergency,
    automatically reverts to incorrectly learned responses, skills, or
    procedures. This stressful reversion, according to some industry
    representatives, could be the result of the lower physical
    fidelity of PCATDs. Other experts dispute this, noting that
    instrument students are already pilots licensed to fly in good
    visibility conditions and already have some experience using
    airplane instrumentation; one expert noted that incorrect and
    dangerous initial instruction can take place in an airplane as
    well as while using a PCATD. Another expert said that quality
    instruction is more important than the type of training device,
    but this type of problem would be difficult to detect. Several
    experts were also concerned that the smaller size of PCATD
    instruments-usually about three-fourths actual size-coupled with
    the smaller size of the PCATD computer monitor screen, could
    affect the quality of a pilot's instrument scan-a critical skill
    to retain constant spatial orientation by using instruments
    alone.15 In poor visibility, a pilot must quickly scan the correct
    instruments that are most important to completing various
    maneuvers, as well as to maintain level flight. Instrument scan
    may also be impaired if PCATD-displayed instruments suffer from
    lower fidelity than flight training devices, some PCATD critics
    believe. Figures III.2 and III.3 show an actual instrument panel
    and the PCATD version of that panel. 15Spatial orientation is
    knowing the position of the plane in space and its speed,
    direction, and up, down, and sideways orientation. A blindfolded
    passenger in a plane-even an experienced pilotquickly loses the
    ability to accurately sense whether a plane is climbing,
    descending, or turning, in part because of the confusing multiple
    sensations of acceleration, deceleration, and turning from
    vestibular and other cues. G-forces and even a turning of the head
    during a turn or climb can cause equally misleading illusions. A
    pilot flying relying on visual cues can be tragically mistaken by
    subtle optical illusions under certain conditions that might be
    prevented by the better use of instruments to confirm the plane's
    situation. Page 31                  GAO/RCED-99-143 Personal
    Computer-Based Aviation Training Device Appendix III Other Issues
    Related to PCATD Safety Figure III.2: Instrument Panel Photo of
    Cessna 172 Source: The Cessna Aircraft Company. Page 32
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    Appendix III Other Issues Related to PCATD Safety Figure III.3:
    PCATD Depiction of Cessna 172 Instruments Source: Aviation
    Teachware Technologies While instrument and panel size could be
    potential problems for a pilot's instrument scan, experts
    responded to this potential problem in several ways. Several noted
    that a pilot's instrument scan must be altered to read different
    airplanes' instrument panels and so may be aided by practice on a
    PCATD that resembles the particular plane he or she flies because
    primary instruments would be correctly spaced although not full-
    sized. A flight instructor noted that larger computer monitors are
    becoming less expensive and that larger screens can increase the
    size of a PCATD's instrument display. A safety expert noted that
    whether an instrument is an actual airplane instrument or
    displayed on a video screen is not important. Other experts say
    that clarity on a video screen can be very good-most commercial
    airliners now display instruments on video screens in "glass
    cockpits." Another expert said that while very few general
    aviation Page 33               GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device Appendix III Other Issues Related
    to PCATD Safety aircraft now have glass cockpits, they will become
    more common in the future. Views on the Advantages    Experts we
    interviewed saw the main value of PCATDs in teaching of PCATDs
    procedures and concepts, rather than the complete set of skills
    needed to fly. Many experts and flight instructors believe that
    isolating a single procedure on a training device can help a
    student focus on that lesson and learn more effectively,
    particularly when introduced to a new topic, without the
    distractions of simultaneously flying the airplane. In general,
    the experts we interviewed believed that the potential safety
    advantages of PCATDs outweigh their potential risks. Among these
    possible benefits are the flexibility of PCATD software to mimic
    the instrument configuration and performance of a variety of
    airplanes; permitting the pilot to fly a half-dozen approaches in
    an hour compared with perhaps one in an airplane; allowing the
    pilot to see and save a chart of a flight as a learning tool; and
    allowing the simulation of systems failures, specific instrument
    failures, and a variety of weather conditions, such as wind speed
    and level of visibility. Figure III.4 shows a PCATD diagram of a
    flight path, figure III.5 shows a PCATD screen to set instrument
    failures and figure III.6 shows a PCATD screen to set weather
    conditions. Page 34             GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device Appendix III Other Issues Related
    to PCATD Safety Figure III.4: Flight Path Screen Source: Aviation
    Teachware Technologies. Page 35               GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device Appendix III
    Other Issues Related to PCATD Safety Figure III.5: Screen to Set
    Failures Source: Aviation Supplies & Academics, Inc. Page 36
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    Appendix III Other Issues Related to PCATD Safety Figure III.6:
    Screen to Set Weather Conditions Source: Aviation Teachware
    Technologies. Some safety experts also see the wider availability
    and use of relatively inexpensive PCATDs as a benefit, since most
    instrument students are not likely to have easy access to a flight
    training device.16 The wider use of PCATDs for continuing training
    could potentially improve the overall level of general aviation
    pilots' instrument skills and judgment, which in turn could
    improve pilot safety. Views on Granting Training
    According to several experts, PCATDs have a role in training but
    that Credit for PCATDs                      students using PCATDs
    should not be granted credit hours for instrument coursework.
    These experts believe that neither of the two major studies showed
    that PCATDs had achieved a level of training effectiveness so that
    one hour of device training equals one hour in the airplane;
    however, FAA 16According to PCATD manufacturers, many PCATDs are
    sold to individuals either for practice at home or possibily for
    individual Part 61 instruction. Page 37                  GAO/RCED-
    99-143 Personal Computer-Based Aviation Training Device Appendix
    III Other Issues Related to PCATD Safety assumes that one hour in
    the training device equals one hour in the airplane. These experts
    question whether the performances of the PCATD students in the
    Embry-Riddle Aeronautical University and University of Illinois
    studies were due to the PCATD, training received in the course
    curriculum or in the airplane itself during the check ride. They
    said that the previously approved flight training devices had also
    not demonstrated their effectiveness. These experts believed that
    both PCATDs and flight training devices should be included under
    one set of standards and that a device should not be approved for
    instrument training credit hours until it shows that (1) one hour
    in the device is as effective for training as one hour in the
    airplane and (2) the number of credit hours now permitted are the
    appropriate amounts that should be approved for instrument
    training. According to these and several academic flight training
    and safety experts, PCATDs are superior to the older,
    grandfathered flight training devices that are still approved for
    the full 15 to 20 hours of instrument flight training as well as
    to meet instrument recency (currency) requirements.17 Figure III.7
    shows an older flight training device that was grandfathered in
    for use as a flight training device. 17These older model flight
    training devices may be mechanically driven desktop models that
    are not computer-driven. Page 38                 GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device Appendix III
    Other Issues Related to PCATD Safety Figure III.7: Example of
    Grandfathered Flight Training Device Note: This is an ATC 610
    tabletop device. Source: Air Safety Foundation. Despite anecdotal
    evidence that there has been little difference in course
    completion hours between instrument students who are trained to
    proficiency in the airplane and those trained partly using flight
    training devices, we found no research showing that this level of
    instrument training effectiveness has been documented either for
    PCATDs or approved flight training devices. However, a 1971 study
    of beginning flight students found that an older training device,
    the Link GAT-1, transferred 100 percent of 11 hours of training
    value to the student's airplane performance, with a training
    effectiveness ratio of one-to-one; in other words, under certain
    conditions for beginning students, one hour in a flight training
    device can save one hour of time the student would Page 39
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    Appendix III Other Issues Related to PCATD Safety otherwise need
    in the airplane to reach proficiency. In a 1973 study, also on
    beginning flight students, airplane training time savings appeared
    to be greater than one-to-one in the first few hours of training,
    where one hour in the device saved more than one hour in the
    airplane and appeared to decline in effectiveness to less than
    one-to-one as time in the flight training device rose to 11
    hours.18 According to one expert, the assumption of one-to-one
    substitution of training device hours for airplane hours, though
    important, is secondary to other concerns because few students
    complete their instrument coursework in only the minimum required
    35 to 40 hours but instead average about 70 total hours to reach
    proficiency. 18H.K. Povenmire and S.N. Roscoe, "An Evaluation of
    Ground-Based Flight Trainers in Routine Primary Flight Training."
    Human Factors (1971), 13 (2), pp. 109-116; _________," Incremental
    Transfer effectiveness of a ground-based general aviation
    trainer." Human Factors (1973). 15 (6), pp. 534-542. Page 40
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    Selected Bibliography Amdor, S.L., F.W. Isley, and Byron J.
    Pierce. "Computer-Based Instruction/Simulator Program for Fighter
    Lead-In Training: Feasibility Research." July 1988. Air Force
    Human Resources Laboratory, Air Force Systems Command, Brooks Air
    Force Base, Texas, AFHRL-TP-87-64. Andrews, Dee H., Lynn A.
    Carroll, and Herbert H. Bell. "The Future of Selective Fidelity in
    Training Devices," March 1996, Human Resources Directorate,
    Aircrew Training Research Division, Mesa, Arizona, AL/HR-TR-1995-
    0195. Barber, Sarah. "From PCs to Flight Simulators: Cost
    Effective Aircrew Training Devices for Aeronautical Decision
    Making and Crew Resource Management." The Ohio State University,
    Proceedings of the Ninth International Symposium on Aviation
    Psychology. Columbus, Ohio, 1997 Benton, Charles J., Paul
    Corriveau, Jefferson M. Koonce, and William C. Tirre. "Development
    of the Basic Flight Instruction Tutoring System (BFITS)," January
    1992, Final Technical Paper for Period September 1989-December
    1991, Air Forces Systems Command, Brooks Air Force Base, Texas,
    AL-TP-1991-0060. Beringer, Dennis B., and Howard C. Harris, Jr.
    "Assessment of Cockpit Navigation and Memory Aids Using PC-Based
    Flight Simulation," Civil Aeromedical Institute, Federal Aviation
    Administration, Oklahoma City, Oklahoma. Proceedings of the Eighth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1995. Beringer, Dennis B. "Use of Off-The-Shelf PC-Based Flight
    Simulators for Aviation Human Factors Research," April 1996, Civil
    Aeromedical Institute, Federal Aviation Administration, U.S.
    Department of Transportation, Oklahoma City, Oklahoma, DOT/FAA/AM-
    96-15. Burki-Cohen, Judith, Nancy N. Soja, and Thomas Longridge.
    "Simulator Platform Motion- The Need Revisited," The International
    Journal of Aviation Psychology 8 (3), 293-317. Caro, Paul W. "Some
    Current Problems in Simulator Design, Testing and Use," March
    1977, Professional Paper 2-77, Air Force Office of Scientific
    Research, Air Force Systems Command, Bolling Air Force Base,
    HumRRO-PP-2-77. Page 41           GAO/RCED-99-143 Personal
    Computer-Based Aviation Training Device Selected Bibliography
    Collins, William E., and Carolyn S. Dollar. "Fatal General
    Aviation Accidents Involving Spatial Disorientation: 1976-1992,"
    August 1996, Civil Aeromedical Institute, Federal Aviation
    Administration, U.S. Department of Transportation, Oklahoma City,
    Oklahoma, DOT/FAA/AM-96-21 Cordell, Captain Tom. "Computer-Based
    Procedural Training," United Airlines, Proceedings of the Sixth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1991. Dennis, Kerry A., and Don Harris. "Computer-Based Simulation
    as an Adjunct to Ab Initio Flight Training," The International
    Journal of Aviation Psychology 8 (3) (1998) pp. 261-276, Emanuel,
    Tom W., Jr., Henry L. Taylor, Charles L. Hulin, Gavan Lintern,
    Sybil I. Phillips, and Donald A. Talleur. "Development of an
    Experimental Flight Syllabus for Testing PC Computer-Based
    Aviation Training Devices, "University of Illinois, Proceedings of
    the Eighth International Symposium on Aviation Psychology,
    Columbus, Ohio, 1995, p. 1198 Emanuel, Tom W., Jr. "PC Computer-
    Based Aviation Training Devices, Curriculum Development, and
    Student Affect," University of Illinois, Proceedings of the Ninth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1997, p. 1198. Etem, Kamil, and Marcia Patten. "Communications-
    Related Incidents in General Aviation Dual Flight Training,"
    Proceedings of the Ninth International Symposium on Aviation
    Psychology, Columbus, Ohio, 1997, p. 1204. Giffin, Walter C., and
    Thomas H. Rockwell. "A Methodology for Research on VFR Flight into
    IMC," Proceedings of the Fourth International Symposium on
    Aviation Psychology, Columbus, Ohio, 1987, p. 278. Gopher, D., M.
    Weil, & T. Bureket. "Transfer of skill from a computer game
    trainer to flight," Human Factors 36 (3) (1994), pp. 387-405.
    Hampton, Steven. "PC-Based Simulation: An Alternative for Teaching
    Instrument Flying Skills," Embry-Riddle Aeronautical University,
    Daytona Beach, Florida, in PC-Based Instrument Flight Simulation
    A First Collection of Papers, A. Robert Sadlowe, editor. Presented
    at the Winter Annual Meeting of the American Society of Mechanical
    Engineers, December 1-6, 1991. Page 42                  GAO/RCED-
    99-143 Personal Computer-Based Aviation Training Device Selected
    Bibliography William Moroney, Tom Kirton, and David W. Biers. "The
    Use of Personal Computer-Based Training Devices in Teaching
    Instrument Flying: A Comparative Study," June 15, 1994, Prepared
    for the Federal Aviation Administration, CAAR 15471931, Grant No.
    92-G-0015 [Summarized in this report.]. Koenig, Robert L.
    "Standard Computer Hardware and Software Configured to Produce
    Useful Flight Simulator in Study." Human Factors & Aviation
    Medicine 43, No. 4 (July-August 1996), Flight Safety Foundation.
    Koonce, Jefferson M., "Predictive Validity of Flight Simulators as
    a Function of Simulator Motion." Human Factors 21 (2) (1979), pp.
    215-223. Koonce, Jefferson M., and William J. Bramble, Jr.
    "Personal Computer-Based Flight Training Devices." The
    International Journal of Aviation Psychology 8 (3), (1998), pp.
    277-292. _____, Steven L. Moore, and Charles J. Benton. "Initial
    Validation of a Basic Flight Instruction Tutoring System (BFITS),"
    Proceedings of the Eighth International Symposium on Aviation
    Psychology, Columbus, Ohio, 1995. Lintern, Gavan. "Transfer of
    Landing Skill after Training with Supplementary Visual Cues,"
    Human Factors 22 (1), (1980), pp. 81-88. _____, Roscoe, S.N.,
    Koonce, J.M., and Segal, L.D. "Transfer of Landing Skill in
    Beginning Flight Training." Human Factors 32, (1990), pp. 319-327.
    _____, Henry L. Taylor, Jefferson M. Koonce, and Donald Talleur.
    "An Incremental Transfer Study of Scene Detail and Field of View
    Effects on Beginning Flight Training," Proceedings of the Eighth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1995, p. 737. Lombardo, D.A. "Effectiveness of Computer-Based
    Flight Simulation," in B.M. Barker (Ed.), Collegiate Aviation
    Review, University Aviation Fall Conference, Atlanta, 1993.
    Moroney, William F., Steven Hampton, and David W. Biers.
    "Considerations in the Design and Use of Personal Computer-Based
    Aircraft Training Devices (PCATDs) for Instrument Flight Training:
    A Survey of Instructors," University of Dayton, Ohio, and Embry-
    Riddle Page 43                  GAO/RCED-99-143 Personal Computer-
    Based Aviation Training Device Selected Bibliography Aeronautical
    University, Daytona Beach, Florida, Proceedings of the Ninth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1997. Moroney, William F. and Brian W. Moroney, "Utilizing a
    Microcomputer Based Flight Simulation in Teaching Human Factors in
    Aviation," Proceedings of the Human Factors Society 35th Annual
    Meeting, 1991, p. 523 "Nall Report: 1997." Airplane Owners and
    Pilots' Association Air Safety Foundation, Frederick, Maryland.
    O'Hare, David, and Stanley Roscoe. Flightdeck Performance: The
    Human Factor. Ames, Iowa: Iowa State University Press; 1990.
    O'Hare, David, and David Chalmers. "Risk Factors for Aviation
    Accidents and Incidents: A Nationwide Study," Proceedings of the
    Ninth International Symposium on Aviation Psychology, Columbus,
    Ohio, 1997, p. 1352. Ortiz, Gustavo. "Effectiveness of PC-Based
    Flight Simulation." The International Journal of Aviation
    Psychology 4 (3), p. 285, 1994. _____, Dominik Kopp, and Thomas
    Willenbucher. "Instrument Training Using a Computer-Based
    Simulation Device at Luftansa Airlines," Andrews University,
    Berrien Springs, Michigan, and Luftansa Airlines, Bremen, Germany,
    Proceedings of the Eighth International Symposium on Aviation
    Psychology, Columbus, Ohio, 1995. _____, Phillips, Sybil I.,
    Charles L. Hulin, and Paul J. Lamermayer. "Uses of Part-Task
    Trainers in Instrument Flight Training." University of Illinois,
    Proceedings of the Seventh International Symposium on Aviation
    Psychology, Columbus, Ohio, 1993. _____, Henry L. Taylor, Gavan
    Lintern, Charles L. Hulin, Tom W. Emanuel, Jr.; Donald A. Talleur.
    "Developing Performance Measures for Evaluating Personal Computer-
    Based Aviation Training Devices Within an FAR Part 141 Pilot
    Training School," Proceedings of the Eighth International
    Symposium on Aviation Psychology, Columbus, Ohio, 1995, p. 1204-
    1209. Pierce, Rodney, and Dr. Richard S. Jensen. "PC Based
    Assessment and Training of Aviation Decision Making Skills, The
    Ohio State University, Page 44                  GAO/RCED-99-143
    Personal Computer-Based Aviation Training Device Selected
    Bibliography Columbus, Ohio, Proceedings of the Ninth
    International Symposium on Aviation Psychology, Columbus, Ohio,
    1997. Povenmire, H. K., & S. N. Roscoe. "An evaluation of ground
    based flight trainers in routine primary flight training." Human
    Factors 13 (1971),.pp. 109-116. _____. "Incremental transfer
    effectiveness of a ground-based general aviation trainer. "1973.
    Human Factors 15, pp. 534-542. Rogers, Brian K., Capt., U.S. Air
    Force. "Microcomputer-Based Instrument Flight Simulation:
    Undergraduate Pilot Training Student Attitude Assessment,"
    December 1991, Human Resources Directorate, Aircrew Training
    Research Division, Williams Air Force Base, Arizona, AL-TR-1991-
    0039. Rolfe, Dr. J.M., J.R. Cook, and Squadron Leader C.G. Durose.
    "Knowing What We Can Get from Training Devices: Substituting a
    Little Arithmetic for a Measure of Emotion," Proceedings of the
    Third International Symposium on Aviation Psychology, Columbus,
    Ohio, 1985, p. 617. Roscoe, Stanley N. "Simulator Qualification:
    Just as Phony as it Can Be," Illiana Aviation Sciences Limited,
    International Journal of Aviation Psychology 1, No. 4 (1991), p.
    335-339. _____.( ed)., Aviation Psychology. Ames, Iowa: Iowa State
    University Press; 1980. Salas, Eduardo, Clint A. Bowers, and Lori
    Rhodenizer. "It is Not How Much You Have but How You Use It:
    Toward a Rational Use of Simulation to Support Aviation Training."
    International Journal of Aviation Psychology 8 (3) (1998), pp.
    197-208 Svoboda, J.V., Ruth M. Heron, and H. Weinberg. "Emergency
    Manoevre Pilot Training in a Low-Cost Flight Simulator,"
    Proceedings of the Seventh International Symposium on Aviation
    Psychology, Columbus, Ohio, 1993, p. 721. Talleur, Donald, Henry
    L. Taylor, Gavan Lintern, Charles L. Hulin, Tom W. Emanuel, Jr.,
    and Sybil I. Phillips. "Effectiveness of an Integrated Instrument
    Training Syllabus Using a Personal Computer and a Certified Page
    45                  GAO/RCED-99-143 Personal Computer-Based
    Aviation Training Device Selected Bibliography Ground Trainer,"
    University of Illinois, Proceedings of the Ninth International
    Symposium on Aviation Psychology, Columbus, Ohio, 1997. Taylor,
    Henry L., Gavan Lintern, Charles L. Hulin, Donald Talleur, Tom
    Emanuel, and Sybil Phillips; "Transfer of Training Effectiveness
    of Personal Computer-Based Aviation Training Devices," November
    1996, Final Technical Report ARL-96-3-/FAA-96-2, prepared for the
    Federal Aviation Administration, Oklahoma City, OK, Contract DTFA-
    94-G-044 [Summarized in this report.]. Wiener, Earl L. "Life in
    the Second Decade of the Glass Cockpit," Keynote Address, Seventh
    International Symposium on Aviation Psychology, Proceedings of the
    Seventh International Symposium on Aviation Psychology, Columbus,
    Ohio, 1993, p. 1. Williams, Kevin W., and Robert E. Blanchard.
    "Development of Qualification Guidelines for Personal Computer-
    Based Aviation Training Devices," February 1995, Final Report,
    Civil Aeromedical Institute, Federal Aviation Administration, U.S.
    Department of Transportation, Oklahoma City, Oklahoma, DOT/FAA/AM-
    95-6. Williams, Kevin W.,"Qualification Guidelines for Personal
    Computer-Based Aviation Training Devices: Instrument Training,"
    February 1996, Final Report, Civil Aeromedical Institute, Federal
    Aviation Administration, U.S. Department of Transportation,
    Oklahoma City, Oklahoma, DOT/FAA/AM-96-8. _____ (ed.) "DRAFT
    Proceedings of the Joint Industry-FAA Conference on Development
    and Use of PC-Based Aviation Training Devices: June 16-17, 1994,
    Oklahoma City, OK," July 1994, Civil Aeromedical Institute,
    Federal Aviation Administration, U.S. Department of
    Transportation, Oklahoma City, Oklahoma. Williams, Donna M.,
    Robert W. Holt, and Deborah A. Boehm-Davis. "Training for Inter-
    Rater Reliability: Baselines and Benchmarks," Proceedings of the
    Ninth International Symposium on Aviation Psychology, Columbus,
    Ohio, 1997, pp. 514-520. Wilt, Donna Forsyth. "Clueless in the
    Cockpit: Situation Awareness of Instrument Flight Students,"
    Proceedings of the Ninth International Symposium on Aviation
    Psychology, Columbus, Ohio, 1997, p. 1464. (348100)    Page 46
    GAO/RCED-99-143 Personal Computer-Based Aviation Training Device
    Ordering Information The first copy of each GAO report and
    testimony is free. Additional copies are $2 each. Orders should be
    sent to the following address, accompanied by a check or money
    order made out to the Superintendent of Documents, when necessary.
    VISA and MasterCard credit cards are accepted, also. Orders for
    100 or more copies to be mailed to a single address are discounted
    25 percent. Orders by mail: U.S. General Accounting Office P.O.
    Box 37050 Washington, DC  20013 or visit: Room 1100 700 4th St. NW
    (corner of 4th and G Sts. NW) U.S. General Accounting Office
    Washington, DC Orders may also be placed by calling (202) 512-6000
    or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each
    day, GAO issues a list of newly available reports and testimony.
    To receive facsimile copies of the daily list or any list from the
    past 30 days, please call (202) 512-6000 using a touchtone phone.
    A recorded menu will provide information on how to obtain these
    lists. For information on how to access GAO reports on the
    INTERNET, send an e-mail message with "info" in the body to:
    [email protected] or visit GAO's World Wide Web Home Page at:
    http://www.gao.gov PRINTED ON     RECYCLED PAPER United States
    General Accounting Office           Bulk Rate Washington, D.C.
    20548-0001     Postage & Fees Paid GAO Permit No. G100 Official
    Business Penalty for Private Use $300 Address Correction Requested

*** End of document. ***