    Aviation Safety: Oversight of Foreign Code-Share Safety Program
    Should Be Strengthened (05-AUG-05, GAO-05-930).

    U.S. airlines are increasingly relying on code-share partnerships
    with foreign carriers to provide additional sources of revenue.
    Code-sharing is a marketing arrangement in which an airline
    places its designator code on a flight operated by another
    airline and sells and issues tickets for that flight. To
    determine whether the foreign code-share partners of U.S.
    airlines meet an acceptable level of safety, in 2000, the
    Department of Transportation (DOT) established the Code-Share
    Safety Program, which requires U.S. airlines to conduct safety
    audits of their foreign code-share partners as a condition of
    code-share authorization. GAO's objective was to assess the
    federal government's efforts to provide reasonable assurance of
    safety and security on foreign code-share flights. GAO reviewed
    (1) the extent to which DOT's code-share authorization process is
    designed to consider safety and security, (2) the Federal
    Aviation Administration's (FAA) management of the Code-Share
    Safety Program, and (3) the implementation of the program by
    airlines and the results.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-05-930
        ACCNO:   A32157
      TITLE:     Aviation Safety: Oversight of Foreign Code-Share Safety
    Program Should Be Strengthened
         DATE:   08/05/2005
      SUBJECT:   Air transportation
    	     Airline security
    	     Airlines
    	     Audit reports
    	     Aviation
    	     Aviation security
    	     Internal controls
    	     International cooperation
    	     International relations
    	     Program evaluation
    	     Program management
    	     Safety regulation
    	     Safety standards
    	     Transportation safety
    	     Transportation security
    	     Code-Share Safety Program
    	     FAA International Aviation Safety
    	     Assessment Program

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-05-930

         * Report to the Ranking Democratic Member, Committee on Transportation
           and Infrastructure, House of Representatives
              * August 2005
         * AVIATION SAFETY
              * Oversight of Foreign Code-Share Safety Program Should Be
                Strengthened
         * Contents
              * Results in Brief
              * Background
              * DOT's Office of International Aviation Relies on FAA for Safety
                Assessments and TSA for Security Assessments to Authorize
                Code-Share Arrangements
                   * DOT Considers Information from Several Sources to Authorize
                     Code-Share Arrangements
                   * FAA Response Reflects Assessments of Foreign Civil Aviation
                     Authorities and Safety Audit Reviews
                   * FAA Reviews U.S. Airlines' Methodologies for Auditing Their
                     Foreign Code-Share Partners
                   * TSA Clearances Reflect Security Assessments of Foreign
                     Airlines That Provide Direct Service to the United States,
                     Its Territories, and Certain Foreign Airports
                   * DOD Provides Additional Safety Oversight of Foreign Airlines
              * Code-Share Safety Program Incorporates Auditing Standards, but
                FAA's Oversight of the Program Lacks Certain Management Controls
                   * Code-Share Safety Program Incorporates Selected Government
                     Auditing Standards
                   * FAA's Reviews of the Safety Audit Reports Lacked Certain
                     Management Controls
              * U.S. Airlines Are Auditing Foreign Partners' Safety, and Partners
                Are Taking Corrective Action, but Documentation of Corrective
                Actions Is Often Lacking
                   * U.S. Airlines' Safety Audits Primarily Identified Findings
                     in Foreign Partners' Flight Operations and Maintenance and
                     Engineering
                   * Airlines Submitted Compliance Statements to FAA but Did Not
                     Always Document the Closure of Findings
                   * U.S. Airlines Also Monitor the Safety of Their Foreign
                     Partners between Audits
                   * Use of International Safety Audit Program May Address Some
                     Weaknesses in the U.S. Safety Audit Program
                   * Most U.S. Airline Officials Believe the Code-Share Safety
                     Program Provides a Reasonable Assurance of Safety
              * Conclusions
              * Recommendations
              * Agency Comments
         * Objectives, Scope, and Methodology
         * U.S. Carriers and Their Foreign Code-Share Partners
         * Comments from DHS
         * GAO Contacts and Staff Acknowledgments
              * GAO Contacts
              * Staff Acknowledgments

                     United States Government Accountability Office

    Report to the Ranking Democratic

    Member, Committee on Transportation and Infrastructure, House of Representatives

    August 2005

    AVIATION SAFETY

         Oversight of Foreign Code-Share Safety Program Should Be Strengthened

                                           a

    AVIATION SAFETY

    Oversight of Foreign Code-Share Safety Program Should Be Strengthened

      What GAO Found

    In considering U.S. airlines' requests to establish code-share
    arrangements with foreign carriers, DOT's Office of International Aviation
    reviews, among other things, any safety and security objections from FAA
    and TSA. FAA assesses the safety of foreign civil aviation authorities and
    reviews reports of the safety audits that U.S. carriers have conducted of
    their foreign airline partners. From fiscal years 2000 through 2004, DOT
    (1) authorized U.S. airlines to establish or maintain code-share
    arrangements with foreign carriers 270 times and (2) did not suspend any
    arrangements because of known safety concerns. According to FAA, however,
    U.S. airlines occasionally have decided not to pursue code-share
    arrangements with foreign carriers because they expected FAA would object,
    and FAA sometimes puts its reviews of proposed code-share arrangements on
    hold if the agency has safety concerns. FAA and TSA did not object to any
    of the authorizations during that period for safety or security reasons.
    Although not involved in the code-share authorization process, the
    Department of Defense (DOD) reviews the safety of foreign airlines that
    transport DOD personnel. For their separate programs, FAA and DOD are
    reviewing many of the same safety audit reports on foreign carriers.

    The Code-Share Safety Program, which calls for U.S. airlines to conduct
    periodic safety audits of their foreign code-share partners, incorporates
    selected government auditing standards involving independence,
    professional judgment, and competence. However, FAA's reviews of the
    safety audit reports lacked management controls for reviewers'
    qualifications, documenting the closure of safety audit findings,
    verifying corrective actions taken in response to findings, and
    documenting reviews.

    Eight U.S. airlines with foreign code-share partners have implemented the
    DOT program by conducting safety audits of their foreign partners.
    According to our review of a random sample of audit reports that FAA
    reviewed from fiscal years 2000 through 2004, the largest numbers of
    safety findings identified were in the categories of (1) flight operations
    and (2) maintenance and engineering. GAO estimates that for 68 percent of
    the findings, the documentation was insufficient to demonstrate that the
    findings were closed or were resolved. Airlines are beginning to adopt a
    new safety audit program that requires the documentation of findings and
    corrective actions.

    Source: FAA.

    United States Government Accountability Office

    Contents

      Letter 1

    Results in Brief 4 Background 7 DOT's Office of International Aviation
    Relies on FAA for Safety Assessments and TSA for Security Assessments to
    Authorize Code-Share Arrangements 12 Code-Share Safety Program
    Incorporates Auditing Standards, but FAA's Oversight of the Program Lacks
    Certain Management Controls 23

    U.S. Airlines Are Auditing Foreign Partners' Safety, and Partners Are
    Taking Corrective Action, but Documentation of Corrective Actions Is Often
    Lacking 31 Conclusions 40 Recommendations 41 Agency Comments 42

    Appendixes

                              Appendix I: Appendix II: Appendix III: Appendix IV:

                         Objectives, Scope, and Methodology 44

                 U.S. Carriers and Their Foreign Code-Share Partners 50

    Comments from DHS 53

    GAO Contacts and Staff Acknowledgments 54 GAO Contacts 54 Staff
    Acknowledgments 54

    Table 1: Selected Government Auditing Standards Incorporated in

    Table

    the Code-Share Safety Program

    Figure 1: Percentage of U.S. Carriers' Active Code-Share

    Figures

    Partnerships, by World Region, as of May 2005 8 Figure 2: Aviation
    Accident Rates by World Region during 2004 11 Figure 3: Code-Share
    Authorization Process 13 Figure 4: FAA's Process for Responding to DOT's
    Office of

    International Aviation 14 Figure 5: Percentage of Findings by Audit
    Category 33

    Contents

                                     Abbreviations

    DHS        Department of Homeland Security
    DOD        Department of Defense
    DOJ        Department of Justice
    DOT        Department of Transportation
    FAA        Federal Aviation Administration
    GSA        General Services Administration
    IASA          International Aviation Safety Assessment
    IATA           International Air Transport Association
    ICAO         International Civil Aviation Organization
    IOSA       IATA Operational Safety Audit
    SARS       Severe Acute Respiratory Syndrome
    TSA             Transportation Security Administration

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    A

    United States Government Accountability Office Washington, D.C. 20548

    August 5, 2005

    The Honorable James L. Oberstar Ranking Democratic Member Committee on
    Transportation and Infrastructure House of Representatives

    Dear Mr. Oberstar:

    As the U.S. airline industry strives to improve its financial condition,
    many carriers are increasing their focus on international service to
    provide needed sources of revenue. To help expand their global market
    reach, U.S. airlines have established an increasing number of code-share
    arrangements with foreign airlines, and, as of May 2005, eight U.S.
    airlines had established foreign code-share partnerships. Code-sharing is
    a marketing arrangement in which an airline places its designator code1 on
    a flight operated by another airline and sells and issues tickets for that
    flight. U.S. carriers must obtain authorization for foreign code-share
    operations from the Department of Transportation (DOT).

    In 2000, DOT's Office of the Secretary and Federal Aviation Administration
    (FAA) established the Code-Share Safety Program to ensure that the foreign
    code-share partners of U.S. airlines meet an acceptable level of safety.
    The program was established in response to safety concerns that arose
    after SwissAir Flight 111 crashed off the shores of Nova Scotia in 1998,
    killing 229 passengers and crewmembers, including 53 Americans. At that
    time, SwissAir was a foreign partner of U.S.-based Delta Air Lines. Before
    the Code-Share Safety Program was established, DOT reviewed the potential
    competitive and economic impact of U.S. airlines' code-share arrangements
    with foreign airlines, but regarding safety considerations, relied on the
    oversight by the home country. Now, under the Code-Share Safety Program,
    U.S. airlines must conduct periodic safety audits of their foreign
    code-share partners as a condition of code-share authorization. DOT
    designed the Code-Share Safety Program to rely on U.S. airlines to audit
    the safety of their foreign code-share partners because FAA does not have
    the authority to inspect foreign aircraft that do not enter the United
    States. The DOT program is similar to a safety audit program that the

    1Designator codes are two-letter codes assigned by the International Air
    Transport Association, the international airline association, to the
    world's airlines.

    Department of Defense (DOD) created in 1999 for foreign carriers that
    transport DOD personnel.

    You asked that we review the measures that the federal government is
    taking to provide reasonable assurance of safety and security when
    passengers travel on flights operated by foreign code-share partners of
    U.S. airlines. To do so, we addressed the following questions:

         o To what extent is DOT's authorization of U.S. airlines' code-share
           arrangements with foreign airlines designed to consider safety and
           security?
         o How well has FAA managed the Code-Share Safety Program?
         o To what extent have U.S. airlines implemented the Code-Share Safety
           Program, and what have been the results of their efforts?

    To determine how safety and security are considered in DOT's authorization
    of U.S. airlines' code-share arrangements with foreign airlines,2 we
    reviewed the process that DOT follows in authorizing code-share
    arrangements, Code-Share Safety Program guidelines, applicable aviation
    safety standards, and related legal authorities. Our review covered safety
    audit reports on foreign carriers that FAA reviewed and U.S. airlines'
    code-share partnerships with foreign carriers that DOT has authorized
    since February 2000, when the Code-Share Safety Program was established,
    through the end of fiscal year 2004. We also examined how FAA's
    assessments of foreign civil aviation authorities are used in the
    code-share approval process. During our review of the Code-Share Safety
    Program, we found that DOD had a similar program designed to ensure the
    safety of foreign airlines that transport DOD personnel. Because our
    objective was to determine how the federal government is providing safety
    oversight of foreign airlines, we included DOD in our review. However,
    because DOD does not play a direct role in the authorization of foreign
    code-share arrangements, this report focuses on the DOT Code-Share Safety
    Program. Moreover, because security is a component of assessing airline
    safety, we determined what the Transportation Security Administration
    (TSA), the Department of Homeland Security agency

    2This report pertains only to U.S. airlines' code-share arrangements with
    foreign airlines for scheduled air service and not for charter or cargo
    air service.

    Page 2 GAO-05-930 Aviation Safety

    responsible for aviation security, has done to provide security
    information to FAA and DOD for their safety reviews of foreign airlines.

    Because the Code-Share Safety Program establishes an audit program, to
    assess how well FAA has managed the program, we reviewed whether the
    program's design conformed with selected standards identified in
    Government Auditing Standards.3 In addition, because we were evaluating
    the management of a government program, we examined how FAA applied
    certain management controls in its reviews of the safety audit reports for
    the Code-Share Safety Program using Standards for Internal Control in the
    Federal Government.4 We discussed with FAA officials how they conducted
    the assessments of foreign civil aviation authorities and with TSA
    officials how they assessed the security of certain foreign carriers and
    airports.

    Finally, to determine the extent to which U.S. airlines have implemented
    the Code-Share Safety Program and the results, we interviewed safety
    officials at the eight U.S. airlines that were participating in the
    Code-Share Safety Program5 and reviewed a random sample of reports of
    safety audits the airlines had conducted of their foreign code-share
    partners. We did not determine whether the airlines complied with
    international aviation safety standards but reviewed the safety audit
    reports to determine what types of findings were identified and whether
    corrective actions were documented. We also interviewed officials from the
    International Air Transport Association (IATA), the trade association that
    represents air carriers worldwide, to determine how IATA's recent
    introduction of a new safety audit program for its members may affect the
    Code-Share Safety Program. We conducted our review in Arlington, Virginia;
    Atlanta; Denver; Eagan, Minnesota; Ft. Worth; Houston; Phoenix;
    Pittsburgh; Seattle; Washington, D.C.; and Montreal, Canada; from August
    2004 through August 2005 in accordance with generally accepted government
    auditing standards. Appendix I provides additional information on our
    methodology.

    3GAO, Government Auditing Standards, GAO-03-673G (Washington, D.C.: June
    2003).

    4GAO, Standards for Internal Control in the Federal Government,
    GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999) and GAO, Internal
    Control Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.:
    August 2001).

    5Those airlines are Alaska Airlines, America West Airlines, American
    Airlines, Continental Airlines, Delta Air Lines, Northwest Airlines,
    United Airlines, and US Airways.

    Page 3 GAO-05-930 Aviation Safety

    Results in Brief

    In considering U.S. airlines' requests to establish code-share
    arrangements with foreign carriers, DOT's Office of International Aviation
    reviews, among other things, any safety and security objections from FAA
    and TSA. FAA assesses the safety of foreign civil aviation authorities and
    reviews reports of the safety audits that U.S. carriers have conducted of
    their foreign airline partners. According to DOT officials, on 270
    occasions from February 2000-when the Code-Share Safety Program was
    established- through the end of fiscal year 2004, DOT authorized or
    reauthorized U.S. airlines to establish or maintain code-share
    arrangements with foreign carriers and did not suspend any arrangements
    because of known safety concerns. However, according to FAA, U.S. airlines
    occasionally have decided not to pursue code-share arrangements with
    foreign carriers because they expected FAA would object, and on four
    occasions during that period, U.S. airlines suspended their code-share
    arrangements with foreign carriers because FAA was questioning the
    capabilities of the civil aviation authorities under which the foreign
    carriers were operating. In addition, FAA sometimes puts its reviews of
    proposed code-share arrangements on hold if the agency has safety
    concerns. Code-share arrangements may be periodically reauthorized, based
    on the terms of the initial authorization. To maintain code-share
    authorization, U.S. airlines are to conduct safety audits of their foreign
    code-share partners every 2 years. FAA provided DOT's Office of
    International Aviation with memorandums indicating that it did not object
    to any of the proposed arrangements from February 2000 through the end of
    fiscal year 2004. DOT's Office of International Aviation also receives
    security information from TSA, which assesses the security of foreign
    airlines that provide direct service to the United States and its
    territories and certain foreign airports. TSA provided security clearances
    for all proposed code-share arrangements from fiscal years 2000 through
    2004 for which it had information on the foreign carriers. TSA does not
    assess the security of foreign carriers that do not provide direct service
    to the United States and its territories because it lacks that authority.
    Twenty-nine of U.S. airlines' foreign code-share partners, or about
    one-third, do not provide direct service to the United States and its
    territories and therefore have not been assessed for security by TSA. DOD
    does not authorize code-share arrangements, but like FAA, it reviews the
    U.S. airlines' safety audit reports of their foreign code-share partners.
    We found that DOD and FAA are reviewing many of the same audit reports and
    that TSA was not providing its foreign airport security assessments to
    DOD.

    The Code-Share Safety Program incorporates selected government auditing
    standards involving auditors' independence, professional judgment, and
    competence. However, FAA's management of the program did not incorporate
    certain controls relating to establishing reviewers' qualifications,
    documenting the closure of safety audit findings, verifying corrective
    actions taken in response to the findings, and documenting its reviews.
    For example, FAA has not established the qualifications needed for agency
    staff who review the safety audit reports. In addition, FAA has not
    provided its reviewers or the airlines with definitions of
    "safety-critical" findings that the airlines must resolve immediately or
    "nonsafety-critical findings" that can be resolved after the audit is
    closed. Some airline officials told us they would like FAA to provide a
    definition of "safetycritical" findings that must be resolved immediately.
    Furthermore, in reviewing the safety audit reports, FAA frequently is not
    documenting its reviews. As a result, it cannot be determined which
    corrective actions that FAA verified were implemented by the foreign
    carriers. FAA's lack of documentation about its reviews of the audit
    reports could impede trend analyses and comparisons of findings and
    prevents determining whether FAA reviewed those findings.

    The eight U.S. airlines have implemented the Code-Share Safety Program by
    conducting safety audits of their foreign code-share partners to determine
    whether the partners comply with international aviation safety standards.
    According to our review of a random sample of 149 audit reports, the
    largest numbers of safety findings that the U.S. airlines identified in
    auditing their foreign partners during fiscal years 2000 through 2004 were
    in the categories of (1) flight operations, which govern the activities of
    the pilots, including training, and (2) maintenance and engineering, which
    involves the oversight of activities to maintain, repair, and overhaul
    aircraft, aircraft engines, and parts. The U.S. airlines followed the
    program's guidelines by submitting written statements from their safety
    directors to FAA affirming that their foreign airline partners had
    complied with international aviation safety standards. However, we
    estimate that 68 percent of the audit findings lacked complete
    documentation that corrective action had been taken. Without such
    documentation, FAA lacks evidence that the identified safety issues have
    been corrected. Furthermore, because FAA has not developed a definition of
    "safetycritical" findings, which FAA requires the airlines to resolve
    immediately, we could not identify those findings and, thus, were not able
    to determine if corrective action was documented. Increasingly, the
    airlines are adopting a new international audit program, which requires
    the documentation of identified findings and associated corrective
    actions. FAA accepted this new international audit program as a
    methodology that would meet the Code-Share Safety Program guidelines. To
    the extent that the airlines substitute the new international audit
    program for their previous audit methodologies, they may improve their
    documentation of resolved findings and associated corrective actions
    because this audit program requires such documentation. Most U.S. airline
    officials said they believe the Code-Share Safety Program provides
    reasonable assurance of safety or is effective. One airline official, for
    example, said that the program has been effective because some foreign
    airlines, seeking to become code-share partners of

    U.S. airlines, have restructured programs, rewritten manuals, and
    instituted new management techniques.

    We are making three recommendations to DOT and one to TSA. To improve the
    effectiveness of the program, we are recommending that DOT revise the
    Code-Share Safety Program guidelines to define "safety-critical" and
    "nonsafety-critical" audit findings, so that FAA reviewers and the
    airlines know which types of findings must be corrected immediately and
    which ones can be resolved later. In addition, we are recommending that
    FAA implement controls for reviewers' qualifications, corrective action
    verification, and review documentation. Furthermore, because DOD and FAA
    are reviewing many of the same audit reports, we recommend that FAA
    explore with DOD potential opportunities to reduce duplication of efforts.
    Finally, because security is an important component of assessing airline
    safety, to improve DOD's oversight of foreign carriers that transport DOD
    personnel, we are recommending that TSA develop a process for routinely
    coordinating with DOD regarding information on the security of foreign
    airports.

    In commenting on a draft of this report, overall, DOT generally concurred
    with our findings and agreed to consider our recommendations. DHS agreed
    with our recommendation regarding TSA. DOD and DOT provided some technical
    clarifications, which we incorporated into this report as appropriate. In
    addition, FAA provided general comments on the Code-Share Safety Program.

    Background

    Each year, several million passengers travel on foreign airlines that have
    established code-share arrangements with U.S. air carriers.6 Code-sharing
    is a marketing arrangement in which an airline places its designator code
    on a flight operated by another airline and sells and issues tickets for
    that flight. On foreign code-share routes, U.S. airlines and their foreign
    partners each place their respective designator code on flights operated
    by the other airline. Passengers can purchase one ticket from a U.S.
    airline that can include flight segments covered by one or more foreign
    partner airlines.7 Air carriers throughout the world form code-share
    alliances to strengthen or expand their market presence or ability to
    compete. Through code-sharing, U.S. airlines can offer seamless service to
    additional international destinations without incurring the expense of
    establishing their own operations to those locations. Moreover, airline
    officials said that code-share arrangements with foreign airlines have
    become important sources of revenue.8 According to FAA, international
    markets are viewed as more attractive growth markets by mainline carriers
    because of more limited competition from low-cost carriers and greater
    profitability.

    In recent years, U.S. airlines have established an increasing number of
    code-share arrangements with foreign carriers to expand their service
    markets.9 As of May 2005, eight U.S. airlines had established 108
    arrangements to place their designator codes on 85 different foreign
    carriers, up from six U.S. airlines that had established 39 arrangements
    to place their designator codes on 38 different foreign carriers in fiscal
    year

    6According to information provided by the eight U.S. airlines
    participating in the Code-Share Safety Program, about 6.5 million tickets
    were purchased from them in 2004 for travel on flights operated by their
    foreign code-share partners. (See app. I for more information on how this
    information was compiled.)

    7U.S. and foreign airlines have established three main types of code-share
    arrangements: (1) the U.S. airline places its designator code on the
    foreign carrier's flight, (2) the foreign airline places its designator
    code on the U.S. carrier's flight, and (3) the U.S. and the foreign
    airline each place their designator code on the other's flights.

    8U.S. airline officials said that their reimbursement agreements with
    foreign carriers were based on route mileage, fare class, and selling
    commissions.

    9Some U.S. airlines code-share with foreign airlines on selected routes
    only, while others code-share with foreign airlines on all of their
    routes.

    2000.10 As shown in figure 1, the majority of U.S. airlines' code-share
    arrangements are with European airlines, representing over half, followed
    by airlines from Asia and the Pacific, accounting for nearly a quarter of
    the arrangements. Appendix II lists the U.S. carriers and their foreign
    code-share partners.

    Figure 1: Percentage of U.S. Carriers' Active Code-Share Partnerships, by
    World Region, as of May 2005

    2%

    Africa

    3%

    Canada

    5%

    Middle East

    Latin America and Carribean

    Asia and Pacific

    Europe

    Source: FAA.

    Note: Percentages do not total to 100 because of rounding.

    In 1998, SwissAir Flight 111, which was a code-share flight with
    U.S.-based Delta Air Lines, crashed off the shores of Nova Scotia, killing
    229 passengers, including 53 Americans. Following that accident, the DOT
    Inspector General reviewed aviation safety under international code-share

    10DOT provided information on the number of U.S. airlines that had placed
    their designator codes on foreign carriers' flights in fiscal year 2000
    but indicated that the information was compiled informally and was not
    official.

    Page 8 GAO-05-930 Aviation Safety

    agreements and issued a report in 1999 recommending, among other things,
    that DOT develop and implement procedures requiring U.S. airlines to
    conduct safety audits of foreign carriers as a condition of authorization
    of code-share passenger services.11 Also in 1999, legislation was
    introduced in the House of Representatives that would have statutorily
    required U.S. airlines to audit the safety of their foreign code-share
    partners.12 Although that legislation was not enacted, in 2000, DOT's
    Office of the Secretary and FAA established the Code-Share Safety Program,
    which included the development of guidelines for U.S. carriers to follow
    in auditing the safety of their foreign code-share partners as a condition
    of DOT's authorization of code-share passenger services.

    DOD's safety audit program, called the Commercial Air Transportation
    Quality and Safety Review Program, expanded another program that DOD
    established in 1986 to check the safety of charter aircraft transporting
    its personnel, after an Arrow Air charter airplane transporting U.S.
    military personnel crashed in 1985, killing 256 passengers and crew. In
    1986, Congress passed Public Law 99-661, which created a Commercial
    Airlift Review Board and prohibits DOD from contracting with an air
    carrier unless it meets certain safety standards and submits to a
    technical safety evaluation. A 1999 memorandum of understanding between
    DOD and the Air Transport Association, a U.S. airline industry
    association, allows DOD to review the safety audits that U.S. airlines
    have conducted of their foreign airline partners.

    DOD is a major customer of airlines that have established code-share
    arrangements through its participation in the General Services
    Administration's (GSA) city-pairs program, under which the government
    negotiates service contracts for all federal government employees,
    including military personnel, to save the government money on air travel.
    The program requires federal employees and military personnel to fly with
    carriers under such contracts when they travel on government business. DOD
    is required to review the safety of all airlines that provide scheduled
    service to its personnel under the GSA city-pairs program, which include

    U.S. airlines' foreign code-share partners. DOD's program also has the

    11DOT Office of the Inspector General, Aviation Safety Under International
    Code-Share Agreements, Report No. AV-1999-138 (Washington, D.C.: Sept. 30,
    1999).

    12Aviation Codeshare Safety Act, H.R. 2024, 106th Congress (1999),
    introduced by Representative James Oberstar.

    Page 9 GAO-05-930 Aviation Safety

    effect of having the airlines comply with DOD requirements if they want to
    maintain the GSA contracts.

    The safety of foreign carriers is also a concern because aviation accident
    rates vary considerably from one region of the world to another. According
    to data compiled by IATA, an international airline association, during
    2004, the North American region had the lowest aviation accident rate
    (0.29 hull losses13 per million flight segments), while the Middle East
    had the highest

    (5.32 hull losses per million flight segments).14 Africa had the second
    highest rate, followed by South America, the Asia-Pacific region, and
    Europe. These accident rates are shown in figure 2.

    13According to Aviation Safety Network, a database devoted to aviation
    accident and safety information and used by FAA in monitoring accident
    histories, a hull loss occurs when airplane damage is beyond economic
    repair and may include events in which (1) the airplane is missing, (2)
    the search for the wreckage has been terminated without success, or

    (3) the airplane is substantially damaged and inaccessible. IATA's data on
    hull losses are for western-built jet aircraft. The 2004 world aggregate
    aviation accident rate compiled by IATA was 0.78 hull losses per million
    flight segments.

    14IATA's regional boundaries for this information are not necessarily
    geographic. For example, Mexico is included in the South American region.

    DOT Considers Information from Several Sources to Authorize Code-Share
    Arrangements

    program, DOD reviews many of the same safety audit reports on foreign
    airlines that FAA reviews for the Code-Share Safety Program.

    To authorize a code-share arrangement between a U.S. and a foreign
    airline, DOT must find that the arrangement is in the public interest.15
    Under the DOT guidelines, this public interest finding includes a
    determination of the foreign carrier's level of safety and the economic
    impact of the arrangement. Before authorizing a code-share arrangement,
    DOT's Office of International Aviation obtains (1) a memorandum of "no
    objection" from FAA, based on its review of the safety audits and other
    safety information available to FAA; (2) a clearance from DOT's Office of
    Policy on aspects of security involving the foreign carrier, including
    information from TSA; and (3) a clearance from DOT's Office of Aviation
    Analysis and Office of the General Counsel concerning the code-share
    arrangement's competitive impact on the airline industry.16 The Office of
    International Aviation also obtains advice from the Department of Justice
    on potential antitrust issues.17 According to DOT officials, on 270
    occasions, from February 2000 through the end of fiscal year 2004, DOT
    authorized or reauthorized U.S. airlines to establish or maintain
    code-share arrangements with foreign carriers and did not suspend any
    arrangements

    15DOT is required to consider the public interest in authorizing
    code-share arrangements under 49 U.S.C. 41309. Although the law does not
    provide a specific definition of public interest in this regard, leaving
    the determination to the discretion of the Secretary of Transportation,
    the determination is guided by the public interest standards of 49 U.S.C.
    40101 as well as by the department's precedents in this area.

    16According to DOT's Office of Aviation Analysis, since 2000, it has
    cleared the competitive aspects of all U.S. airlines' code-share
    arrangements with foreign airlines. Before providing a clearance, this
    office reviews code-share applications for the degree of competition
    involved, the terms of the financial agreements between the U.S. airlines
    and their foreign partners, and the competitive impact on the carriers
    involved and the region affected.

    17According to the Justice Department's Antitrust Division, since 2000, it
    has not advised DOT against approving any U.S. airlines' unimmunized
    code-share arrangements with foreign carriers because of antitrust
    concerns (unimmunized code-share arrangements are those for which the
    Department of Justice has not provided immunity from antitrust actions).

    FAA Response Reflects Assessments of Foreign Civil Aviation Authorities
    and Safety Audit Reviews

    during that time.18 However, FAA officials also said that U.S. airlines
    have occasionally decided not to pursue code-share arrangements with
    foreign airlines because they expected FAA would object. Code-share
    arrangements may be periodically reauthorized based on the terms of the
    initial authorization. Figure 3 shows the DOT code-share authorization
    process.

    Figure 3: Code-Share Authorization Process
                                FAA reviews safety audit
                                reports, civil aviation
                                 authority assessments,
    U.S. carrier                 and other information.
    files
    a code-share  DOT's Office   DOT's Office of   DOT's Office
    application   of             Policy reviews    of
    with DOT's    International  security          International
    Office of     Aviation       information from  Aviation      DOT's Office
    International requires       TSA. DOT's Office receives      of
    Aviation.     information on of Aviation       clearances    International
    U.S. carrier  safety,        Analysis and      from FAA, DOT Aviation
    conducts      security,      Office of the     policy and    decides
    safety audit  competitive    General Counsel   economic      whether to
    on potential  impact, and    review            analysis      authorize the
    foreign       potential      competitive       units, and    code-share
    code-share    antitrust      impact            antitrust     arrangement.
    partner.      implications   information.      advice from
                     to help make   Justice           the Justice
                     its decision.  Department        Department.
                                    reviews potential
                              antitrust implications.

                Source: DOT's Office of International Aviation and FAA.

    DOT's Office of International Aviation solicits the views of FAA on the
    safety aspect of its code-share authorization decision because of FAA's
    technical expertise in that area. FAA reviews reports of the safety audits
    that U.S. carriers have conducted on the foreign carriers and other safety
    information available to FAA, including its assessments of the
    capabilities of the relevant foreign civil aviation authorities. FAA
    provided DOT's Office of International Aviation with memorandums of "no
    objection" on all foreign airlines being considered for code-share
    authorization during fiscal years 2000 through 2004. According to FAA
    officials, if FAA has safety

    18According to a DOT official, in some cases, the same code-share partners
    requested and received multiple approvals during this period (adding
    routings, for example), so the total number of authorizations granted by
    the Office of International Aviation exceeds the total number of safety
    audit reports that FAA reviewed.

    Page 13 GAO-05-930 Aviation Safety

    concerns, it puts a hold on its review of the proposed code-share
    arrangement, allowing time for the safety issues to be resolved; and on
    four occasions, from February 2000 through September 2004, U.S. airlines
    suspended their code-share arrangements with foreign carriers because FAA
    was questioning the capabilities of the civil aviation authorities under
    which the foreign carriers were operating. Figure 4 shows FAA's process
    for providing information to DOT's Office of International Aviation on
    U.S. airlines' applications to establish code-share arrangements with
    foreign carriers.

    Figure 4: FAA's Process for Responding to DOT's Office of International Aviation

    Under the Code-Share Safety Program guidelines, DOT authorizes a U.S.
    airline's code-share arrangement with a foreign carrier only if the
    foreign airline is from a country that is compliant with applicable
    international aviation safety standards under FAA's International Aviation
    Safety Assessment (IASA) program.19 Under IASA, FAA reviews the
    capabilities of foreign civil aviation authorities by checking their
    compliance with standards established by the International Civil Aviation
    Organization

    19FAA's authority to conduct the IASA program is facilitated through the
    requirements of 14

    C.F.R. part 129.11 (a), which require in part that operations within the
    United States be in compliance with International Civil Aviation
    Organization (ICAO) annex 6 part 1 (aircraft operations).

    Page 14 GAO-05-930 Aviation Safety

    (ICAO), a United Nations aviation organization.20 Under IASA, FAA assigns
    countries' civil aviation authorities either a category 1 rating-meets
    ICAO standards-or a category 2 rating-does not meet ICAO standards.21
    During the IASA process, FAA personnel, typically from various
    international field offices, conduct on-site assessments of civil aviation
    authorities for compliance with ICAO standards in eight areas: (1) primary
    aviation legislation, (2) aviation regulations, (3) organization of the
    civil aviation authority, (4) adequacy of the technical personnel, (5)
    technical guidance,

    (6) licensing and certification, (7) records of continuing inspection and
    surveillance, and (8) resolution of safety issues. Each country with
    carriers serving, or wishing to serve, the United States in their own
    right or as part of a code-share arrangement with a U.S. airline must
    first have an assessment under the IASA program.

    Although FAA's plan is to reassess the category for each foreign civil
    aviation authority every 2 years, FAA officials said that this activity
    occurred less frequently because of a larger-than-anticipated number of
    reassessments and constraints on the agency's resources. FAA data indicate
    that 67 of the 100 foreign civil aviation authorities in the IASA program,
    or about two-thirds, have not been assessed within the last 4 years.
    According to FAA, some countries were not assessed within the last 4 years
    because available data indicated that their rating categorization remained
    valid. FAA data also show that from January 1, 2000, through May 1, 2005,
    FAA assessed or reassessed-because of safety oversight concerns-the
    capabilities of 33 foreign civil aviation authorities, 6 of which were
    assessed more than once.22 Of the 42 countries' civil aviation authorities
    under which the foreign code-share partners of U.S. airlines are
    operating, 16 have required an IASA assessment or reassessment since 2000
    and 26 have not.

    20The 1944 Chicago Convention on aviation safety led to the establishment
    of ICAO, the United Nations organization that develops standards and
    recommended practices for aviation safety and security, and outlined the
    rights and responsibilities of civil aviation authorities. ICAO's 18
    annexes delineate internationally agreed-upon standards that signatories
    to the Convention (i.e., civil aviation authorities) must meet. Annex 1
    (personnel licensing), annex 6 (aircraft operations), and annex 8
    (aircraft airworthiness) serve as primary sources of international
    aviation safety standards.

    21The IASA program and the Code-Share Safety Program use the same ICAO
    standards (annexes 1, 6, and 8) as evaluation criteria.

    22Five of the six countries were ones that FAA had first given an IASA
    category 2 rating and later upgraded to category 1.

    IASA results, along with the safety audits that U.S. airlines conduct of
    their foreign code-share partners, are FAA's principle measures of the
    level of safety of the foreign carriers. According to the guidelines, the
    level of oversight and regulation that an airline receives from its
    regulatory authority is an important factor in assessing its safety. For
    this reason, DOT authorizes U.S. airlines' code-share arrangements only
    with foreign airlines that are from IASA category 1 countries. As of May
    2005, FAA had assigned IASA category 1 ratings to 71 countries' civil
    aviation authorities and IASA category 2 ratings to 28; 94 other countries
    had not yet been categorized, generally because no carriers from those
    countries had applied to provide direct service to the United States.

    DOT's Office of International Aviation will not authorize a code-share
    application, and FAA will not review the safety audit report if flights
    that are intended to carry a U.S. carrier's designator code would be
    operated by a foreign carrier from a country with an IASA category 2
    rating. If a U.S. airline is seeking to establish a code-share arrangement
    with a foreign carrier that is from a country that does not have an IASA
    rating, FAA normally conducts the assessment before DOT's Office of
    International Aviation considers the application. When FAA lowers a
    country's IASA rating from category 1 to category 2, DOT's Office of
    International Aviation contacts any U.S. airline that has a code-share
    partnership with an airline from that country to advise the U.S. airline
    of the lowered IASA rating so that the U.S. carrier can promptly remove
    its code from any passenger flights operated by that airline, according to
    agency officials. While DOT indicated that it could, at its option, order
    the removal of U.S. airlines' designator codes under these circumstances,
    in practice, DOT has not needed to pursue that option because, when the
    airlines have learned about an IASA category change affecting their
    service, they have removed their operating codes from the foreign carrier.
    On four occasions since 2000, U.S. airlines have suspended their
    code-share arrangements with foreign airlines because FAA was questioning
    the capabilities of the civil aviation authorities under which the foreign
    airlines were operating.23 The program guidelines allow DOT to consider,
    on a case-by-case basis, continuing to authorize a U.S. airline's
    code-share arrangement with a foreign carrier that

    23In three cases, the U.S. airlines suspended their code-share
    arrangements before FAA had lowered the respective IASA ratings from
    category 1 to category 2. In another case, a U.S. airline temporarily
    suspended its code-share arrangement with a foreign airline after FAA
    questioned the capabilities of that airline's civil aviation authority,
    but FAA did not lower that country's IASA rating. After FAA resolved its
    questions, the U.S. airline resumed its code-share arrangement with that
    foreign carrier.

    Page 16 GAO-05-930 Aviation Safety

    FAA Reviews U.S. Airlines' Methodologies for Auditing Their Foreign
    Code-Share Partners

    is from a country with an IASA rating that has been lowered from category
    1 to category 2. According to FAA, this case-by-case language was included
    to enable DOT's Office of International Aviation to accord U.S. airlines a
    limited degree of flexibility needed to effectuate an orderly shutdown of
    their code-share services. However, DOT officials told us that they will
    not authorize the continuation of a code-share arrangement beyond the
    needs of such an orderly shutdown.

    FAA will not review a U.S. airline's safety audit report on a foreign
    carrier until FAA has reviewed and accepted the airline's audit
    methodology. According to the program guidelines, the U.S. airlines'
    safety audit methodologies should incorporate ICAO standards on personnel
    licensing, aircraft operations, aircraft airworthiness, and security.24
    The guidelines also describe how the U.S. airlines should conduct their
    safety audits, including what qualifications the auditors should possess,
    how the system for reporting and correcting findings should be devised,
    what audit results are satisfactory, how a safety monitoring system should
    be established, and how frequently audits should be conducted. At the same
    time, FAA officials said they provide the airlines with some flexibility
    in designing their audit programs, as long as the programs address all of
    the relevant ICAO standards. FAA reviewed and accepted an audit program
    for each of the eight U.S. airlines to participate in the Code-Share
    Safety Program. In designing their audit methodologies, some U.S. airlines
    include other standards and best practices, such as ones developed by DOD,
    in addition to the ICAO standards and recommended practices in the DOT
    program guidelines. Moreover, to audit the safety of their foreign
    code-share partners, six U.S. airlines have begun using standards from a
    new international safety audit program developed by IATA called the IATA
    Operational Safety Audit (IOSA), which incorporates the ICAO standards,
    plus many additional industry best practices. IOSA was developed by IATA
    to improve global airline safety and promote audit efficiency by reducing
    redundant audits. In 2004, FAA accepted the IOSA program as a methodology
    that would meet the Code-Share Safety Program guidelines.

    24The Chicago Convention provides that signatories to the Convention
    (countries), and thus the airlines under their oversight, must meet all
    the ICAO annexes containing international aviation safety standards.
    Including the United States, 188 countries are signatories to the
    Convention.

    Page 17 GAO-05-930 Aviation Safety

    Under the Code-Share Safety Program guidelines, after the U.S. airlines
    have completed the audits and the foreign airlines have taken all
    corrective actions, the U.S. airlines' safety directors (or similar
    officials) should provide written statements to FAA, known as compliance
    statements, affirming that the audits were conducted in accordance with
    the guidelines and that the foreign carriers meet the applicable ICAO
    standards. According to an FAA official, U.S. airlines filed compliance
    statements for all of the audit reports that FAA reviewed on foreign
    carriers.25 The guidelines also indicate that to maintain their continued
    code-share authorizations, U.S. airlines should audit the safety of their
    foreign code-share partners and submit compliance statements to FAA every
    2 years. We found that, for 12 out of 256 audit reports that FAA reviewed
    from February 2000 through the end of fiscal year 2004, FAA granted the
    U.S. airlines extensions of time to submit compliance statements because
    delays had resulted from the outbreak of Severe Acute Respiratory Syndrome
    (SARS), the U.S. airline planned to cancel the code-share arrangement, or
    the foreign carrier needed more time to implement corrective actions. FAA
    generally granted the extensions for between 1 and 3 months, during which
    time the code-share arrangements continued.

    25We reviewed FAA's files and did not find compliance statements that U.S.
    airlines had filed for 25 of the 256 safety audit reports that FAA
    reviewed from February 2000 through September 2004.

    Page 18 GAO-05-930 Aviation Safety

    TSA Clearances Reflect Security Assessments of Foreign Airlines That
    Provide Direct Service to the United States, Its Territories, and Certain
    Foreign Airports

    Since 2000, DOT's Office of Intelligence and Security and Office of
    Policy26 have provided security clearances to DOT's Office of
    International Aviation for all U.S. airlines' proposed code-share
    arrangements with foreign airlines. DOT's Office of Policy receives
    security information on certain foreign carriers and foreign airports from
    TSA,27 which assesses the security of foreign airlines that provide direct
    service to the United States and its territories, as well as to certain
    foreign airports. TSA provided security clearances for all proposed
    code-share arrangements, from fiscal years 2000 through 2004, for which it
    had information on the foreign carriers. Because it lacks the authority,28
    TSA does not assess the security of other foreign carriers that do not
    provide direct service to the United States and its territories.
    Twenty-nine, or about one-third, of the 85 foreign code-share partners of
    U.S. airlines do not provide service to the United States and its
    territories and therefore have not been assessed for security by TSA.

    DOT has also authorized U.S. airlines' code-share arrangements with
    foreign airlines that serve many foreign airports that TSA has not
    assessed for security. As a result, passengers traveling on foreign
    code-share partners of U.S. airlines may be traveling to certain foreign
    airports that could have security risks. TSA has the authority to assess
    the security of a foreign airport (1) served by U.S. airlines, (2) from
    which a foreign carrier serves the United States and its territories, or
    (3) that "poses a high risk of introducing danger to international air
    travel." Also, TSA can assess "other foreign airports the Secretary of
    Homeland Security considers appropriate." TSA has assessed the security of
    the foreign airports from which domestic and foreign airlines provide
    direct service to the United States and its territories. However, in
    addition to the foreign airports that provide direct service to the United
    States and its territories, the foreign

    26DOT's Office of Policy now includes security policy staff, formerly
    housed within DOT's Office of Intelligence and Security, which provided
    the security clearance information for foreign airlines.

    27Before fiscal year 2002, when TSA was created and began conducting the
    security assessments, FAA's Office of Civil Aviation Security was
    responsible for conducting them.

    28TSA has authority under 49 U.S.C. 44906 to regulate security aspects of
    foreign air carriers that provide service to the United States. TSA
    indicated that it does not have any special regulatory authority or
    requirements for the code-share partners of U.S. airlines. TSA requires
    foreign carriers in their operations to and from airports in the United
    States to adhere to security measures that are similar to those TSA
    requires of the U.S. carriers serving the same airports.

    code-share partners of U.S. airlines serve other foreign airports. TSA
    officials indicated they have begun to assess the security of other
    foreign airports. DOT has not always had comprehensive data on which
    foreign airports are being served by the foreign code-share partners, so
    we were unable to determine how many foreign airports have not undergone
    TSA security assessments.29 For one U.S. airline for which we had complete
    foreign code-share route information,30 we determined that the foreign
    partners served 128 foreign airports that did not provide direct service
    to the United States and its territories, and some of these 128 had yet to
    undergo TSA security assessments.

    In assessing the security of foreign airports,31 TSA rates them in
    categories and assesses airports in those categories as appropriate. DOT's
    Office of International Aviation, which receives TSA's security ratings
    through DOT's Office of Policy, authorizes code-share arrangements for
    U.S. airlines with foreign carriers that serve foreign airports. According
    to DOT security officials, it is not a problem to authorize code-share
    arrangements with foreign airlines regardless of category because all
    airports must meet ICAO security standards32 and are assessed
    appropriately.33 Moreover, officials from TSA and DOT noted that both U.S.
    and foreign airlines can be required to implement additional security
    measures at those airports. For example, the TSA officials described an
    instance in which a bombing in a Middle Eastern country resulted in the
    implementation of additional security measures at an airport in that
    country. TSA officials said that because that airport met ICAO security
    standards, TSA had to rely on increased security measures voluntarily
    implemented by the carriers to help mitigate the threat in that area.

    29In some cases, DOT's documentation on foreign code-share routes refers
    to "destinations" or "points" within a foreign country without specifying
    which ones.

    30We chose this U.S. airline because it has established code-share
    arrangements with one of its foreign partners for nearly all its flights.

    31Under 49 U.S.C. 44907, TSA is required to assess the effectiveness of
    the security measures maintained at foreign airports from which a foreign
    carrier serves the United States.

    32ICAO's annex 17 covers security standards.

    33 TSA also indicated that regardless of category, airlines from ICAO
    signatory countries meet ICAO security standards and are assessed as
    appropriate.

    DOD Provides Additional Safety Oversight of Foreign Airlines

    While not involved in DOT's code-share authorization process, DOD reviews
    the safety of certain foreign airlines, thereby providing an additional
    layer of federal oversight. The DOD Commercial Air Transportation Quality
    and Safety Review Program is focused on ensuring that the airlines DOD
    contracts with-to transport DOD personnel-meet applicable safety
    standards. DOD requires U.S. airlines to audit the safety of their foreign
    code-share partners every 2 years, on the basis of ICAO standards, and
    monitor the safety of their foreign partners between safety audits. In
    addition, DOD considers FAA's IASA ratings of foreign civil aviation
    authorities in determining whether to allow foreign carriers to fly on GSA
    city-pair routes.

    DOD requires that foreign airlines be assessed on the basis of standards
    that DOD developed called Quality and Safety Requirements, which are
    focused on system safety processes.34 According to a DOD official, these
    DOD standards include safety processes that are not ICAO requirements,
    which form the basis of the DOT program. A DOD official said, for example,
    that DOD requires airlines to have a safety audit program that analyzes
    and assesses trends of safety information, including feedback from crew
    members, for the purpose of enhancing safety, which is not an ICAO
    standard. Although DOT can suspend code-share authorizations for safety
    reasons, DOD can cancel, at any time, contracts with airlines that
    transport DOD personnel if it determines that they are not sufficiently
    safe. Between audits, DOD takes certain steps to monitor the safety of
    foreign carriers that FAA does not take, such as conducting semi-annual
    evaluations that include requiring foreign carriers that DOD contracts
    with to complete questionnaires about their safety. DOD does not consider
    TSA's security assessments of foreign airports in its review. DOD
    officials said that they were unaware of TSA's foreign airport assessments
    and would like TSA to provide the information for DOD to consider as part
    of its reviews.

    We found that DOD and FAA review many of the same safety audit reports on
    foreign airlines. During fiscal years 2001 through 2004,35 DOD and FAA
    reviewed 203 of the same reports of safety audits that U.S. airlines had
    conducted of their foreign code-share partners. In reviewing these same

    34DOD looks at system safety as a means of reducing risk through early
    identification, analysis, elimination, and control of hazards.

    35DOD had data on the reports that it reviewed from fiscal years 2001
    through 2004, so we were unable to compare which reports FAA and DOD
    reviewed during fiscal year 2000.

    Page 21 GAO-05-930 Aviation Safety

    reports, DOD and FAA reached the same conclusions about the safety of the
    foreign carriers involved. Because DOD and FAA are reviewing many of the
    same audit reports, the DOT and DOD safety programs are duplicating some
    efforts. In its 1999 report, the DOT Inspector General recommended that,
    in establishing a safety program on foreign code-share partners of U.S.
    airlines, FAA and DOT's Office of the Secretary work closely with DOD to
    maximize the use of limited resources, avoid duplication, and establish
    protocols for exchanging information about the carriers' safety
    assessments. A DOD official said that he communicates frequently with FAA
    Code-Share Safety Program officials, and that DOD has a full-time liaison
    in FAA's Flight Standards Service, who meets weekly with FAA officials.36
    However, FAA officials said that although DOD requests IASA reports on
    certain countries, FAA does not routinely communicate with DOD on its
    safety audit reviews of foreign carriers, and no set criteria spell out
    the circumstances under which FAA and DOD should communicate information
    on the safety of U.S. airlines' foreign code-share partners.

    When we discussed the possibility of reducing duplicative safety reviews
    with FAA and DOD officials, an FAA official said he did not consider their
    reviews to be duplicative because FAA and DOD have different objectives.
    The FAA official said that FAA is reviewing the reports from the
    perspective of a regulator, focusing on the carriers' compliance with ICAO
    standards. Furthermore, the FAA official questioned whether FAA or DOD
    could assume each others' responsibilities and report to different
    departments. A DOD official also said the potential for duplication should
    be considered from the perspective of DOD's and FAA's different objectives
    in conducting their reviews. The DOD official said that DOD's objective is
    to ensure that its requirements for transporting DOD personnel are being
    met. Another DOD official said that FAA and DOD are not duplicating their
    efforts because neither agency has the expertise to conduct its reviews
    from the other agency's perspective.

    36In a 2002 report on FAA and DOD responses to aviation safety concerns,
    we found that FAA and DOD had gaps in their formal communication process,
    which caused delays in bringing critical safety information to the
    attention of key officials. GAO, Aviation Safety: FAA and DOD Response to
    Similar Safety Concerns, GAO-02-77 (Washington, D.C.: Jan. 22, 2002). In
    that report, we recommended that FAA and DOD develop a memorandum of
    agreement for exchanging aviation safety-related information and research.
    In January 2004, FAA and DOD signed a memorandum of agreement for
    exchanging aviation safety-related information and research.

    Code-Share Safety Program Incorporates Auditing Standards, but FAA's
    Oversight of the Program Lacks Certain Management Controls

    The Code-Share Safety Program incorporates selected government auditing
    standards involving independence, professional judgment, and competence.
    According to FAA officials, FAA and DOT's Office of the Secretary worked
    with the airline industry to recommend that the Code-Share Safety Program
    guidelines incorporate these standards. Government auditing standards
    provide an overall framework for ensuring that auditors be independent and
    exercise judgment, competence, and quality control and assurance in
    planning, conducting, and reporting on their work.37 However, FAA's
    management of the program did not incorporate certain internal controls,38
    which the Office of Management and Budget requires federal managers to use
    in assessing the effectiveness and efficiency of operations.39 These
    controls are related to establishing reviewers' qualifications,
    documenting the closure of safety audit findings, verifying corrective
    actions taken in response to the findings, and documenting reviews.

    Code-Share Safety Program Incorporates Selected Government Auditing
    Standards

    The Code-Share Safety Program guidelines recommend that the airlines
    incorporate certain government auditing standards in their safety audit
    reviews. FAA has reviewed the methodologies that the U.S. airlines follow
    in auditing the safety of their foreign code-share partners, which
    incorporate these auditing standards. Ensuring independence is critical,
    for example, because the U.S. airlines generally audit the safety of their
    foreign code-share partners themselves. Although we did not assess the
    airlines' compliance with the independence standard, U.S. airline
    officials told us that they ensure independence by separating their safety
    and marketing departments organizationally to prevent any possible
    influence from the marketing staff on the safety audit results. In
    addition, safety officials at the

    U.S. airlines participating in the Code-Share Safety Program indicated
    that other airline departments do not have any input into their safety
    audit results. Moreover, some airline safety officials said they were not
    aware of the specific financial arrangements involved in their airlines'
    code-share

    37GAO, Government Auditing Standards, GAO-03-673G (Washington, D.C.: June
    2003).

    38GAO, Standards for Internal Control in the Federal Government,
    GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999) and GAO, Internal
    Control Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.:
    August 2001).

    39Office of Management and Budget, OMB Circular A-123, Management's
    Responsibility for Internal Control, (Washington, D.C.: 1995).

    Page 23 GAO-05-930 Aviation Safety

    partnerships. The program guidelines allow the U.S. airlines to employ
    personnel or hire outside experts as consultants (contractors) to conduct
    the safety audits. FAA officials said they are not concerned about
    allowing the U.S. airlines to use their own employees to conduct the
    safety audits because of the importance to the airlines of conducting
    sound safety audits to limit the liability associated with establishing
    code-share arrangements with foreign airlines. Table 1 lists the program
    guidelines that incorporate the auditing standards.

     Table 1: Selected Government Auditing Standards Incorporated in the Code-Share
                                     Safety Program

                          Auditing standard Program guidelines

    Independence: The audit organization and auditor should be free from
    personal, external, and organizational impairments to independence.

    The guidelines indicate that the U.S. airlines' safety auditors should
    have organizational independence to perform the audits and be free to
    objectively report to the airlines' senior management.

    The guidelines indicate that U.S. airline safety directors (or similar
    officials) should submit compliance statements to FAA affirming that their
    foreign code-share partners meet international aviation safety standards.

    The guidelines indicate that the safety auditors should have no financial
    interest in or family affiliation with the foreign code-share partner
    airlines.

    Professional judgment: Professional judgment should be The guidelines
    indicate that the safety audit report's content should be used in planning
    and performing audits and in reporting the independent, objective, fair
    and constructive, free of vagueness or results, including exercising
    reasonable diligence to maintain ambiguity, and supported by competent and
    relevant audit evidence. the highest degree of integrity, objectivity, and
    independence.

    Competence: Auditors should have the knowledge, skills, The guidelines
    indicate that the safety auditors should have the and experience necessary
    for their work, and these qualifications needed to conduct the safety
    audits and analyze the elements should be addressed in the hiring process
    and findings, including relevant training, experience in conducting safety
    through continuous development. audits, and knowledge of international
    aviation safety standards.

    FAA's Reviews of the Safety Audit Reports Lacked Certain Management
    Controls

    Source: Government Auditing Standards and Code-Share Safety Program guidelines.

    We found that FAA's reviews of the safety audit reports lacked certain
    management controls-including establishing reviewers' qualifications,
    verifying corrective actions, and documenting the reviews-but did employ
    some management controls for monitoring and measuring performance.
    Management controls are the continuous processes and sanctions that
    federal agencies are required to use to provide reasonable assurance that
    their goals, objectives, and missions are being met. These controls should
    be an integral part of an agency's operations and include a continuous
    commitment to identifying and analyzing risks associated with achieving
    the agency's objectives, establishing program goals and evaluating
    outcomes, and creating and maintaining related records.

    Effective management controls require that personnel possess and maintain
    a level of competence that allows them to accomplish their assigned
    duties. In addition, management must identify the knowledge and skills
    needed for various jobs, provide needed training, and obtain a workforce
    that has the skills that match those necessary to achieve organizational
    goals. However, we found that FAA has not established competence criteria
    and qualifications for the personnel who review the airlines' safety audit
    reports. As a result, the FAA staff who are reviewing the audit reports
    have different backgrounds and training, which may lead to differing
    interpretations of the standards. The FAA headquarters official who has
    reviewed a large number of the safety audit reports has aviation
    experience as a military pilot and is trained as an ISO 9000 auditor40 but
    is not trained as an FAA inspector and was hired in an administrative
    capacity. Two other FAA headquarters staff who review the audit reports
    have been trained as aviation safety inspectors. Furthermore, five FAA
    field inspectors who are conducting many of the reviews41 have not had
    training in IOSA, which six U.S. airlines in the Code-Share Safety Program
    are now using as standards to audit the safety of their foreign code-share
    partners. As a result of inspectors not having this training, this could
    impede FAA's review of the safety audits based on those standards.
    Moreover, the Code-Share Safety Program manager was transferred to a new
    position in February 2005, leaving the position vacant since that time. As
    of June 2005,

    40ISO 9000 is a certification process developed by the International
    Organization for Standardization, which develops standards for business,
    government, and technology based on quality management principles. An FAA
    official said that the agency uses the ISO 9000 standards for the
    Code-Share Safety Program for data collection and retention, continuous
    improvement efforts, customer satisfaction, and performance measurement.
    This official said that, for example, the ISO 9000 standards are used to
    ensure consistency in how code-share applications are processed, that the
    needed information is collected and provided to management to review the
    applications, and that performance goals are achieved.

    41Although FAA program staff based in Washington, D.C., review the
    majority of the audit reports, FAA field safety inspectors review some of
    them, because staffing is limited for this program. FAA provided
    information indicating that one full-time-equivalent (FTE) position was
    dedicated for the Code-Share Safety Program during fiscal years 2000,
    2002, 2003, and 2004, and two FTEs were dedicated during fiscal year 2001.
    In addition, FAA indicated that five inspectors helped conduct safety
    audit reviews during those years on a part-time basis. FAA estimated that
    the time these five inspectors spent reviewing audit reports represented
    about 4 percent of an FTE's time per year. FAA also indicated that about
    2.3 FTEs were used on the IASA program per year during fiscal years 2000
    through 2004, including the time spent by field office staff.

    FAA had not authorized this position to be filled and has denied a request
    for another full-time staff position dedicated to the program. Since the
    program manager's departure, other staff in FAA's International Programs
    and Policy Office, which administers the Code-Share Safety Program, have
    reviewed the safety audit reports in addition to performing their regular
    duties. FAA program officials said that since the program manager was
    transferred to another position, U.S. airlines must wait 3 to 4 weeks for
    FAA to review their safety audits of foreign carriers, compared with
    waiting 1 day to 2 weeks before his transfer, and that U.S. airlines now
    must bring all of their safety audit reports to FAA in Washington, D.C.,
    for review-a change that could hinder FAA's review of documentation, such
    as safety monitoring systems, that may be located at the airlines'
    facilities.42 An FAA management official said that because FAA's Flight
    Standards Service,43 of which the Code-Share Safety Program is a part,
    imposed a hiring freeze in January 2005 for budgetary reasons, only
    critical positions are being replaced. The official said that because the
    vacant position for the Code-Share Safety Program was not considered to be
    critical, it was not filled.

    Effective management controls also require the establishment of policies
    and procedures to verify that corrective actions have been taken in
    response to identified problems. According to FAA and airline officials,
    FAA staff review each audit report for about 2 to 4 hours, identifying any
    areas that need further clarification or resolution. Although FAA staff
    review the reports of all audits that U.S. airlines have conducted of
    their foreign code-share partners, normally they only spot check whether
    findings that were identified during the audit were resolved. According to
    an FAA safety official, FAA relies on the U.S. airlines' compliance
    statements, signed by the airlines' safety directors, which affirm that
    the audits were conducted in accordance with the guidelines and that the
    foreign carriers met the applicable ICAO standards, as proof that all
    findings have been resolved. However, FAA's reliance on the compliance
    statements may not provide an effective management control to ensure that
    corrective actions have been taken in response to audit findings. For
    example, we found that FAA provided a memorandum of no objection to DOT's
    Office of International Aviation about a foreign code-share partner that,
    according to an official from its U.S. partner, had not implemented all

    42FAA officials said that before the Code-Share Safety Program manager was
    transferred to a new position, U.S. airlines brought the safety audit
    reports of their foreign code-share partners to FAA in Washington, D.C.,
    for review 90 percent of the time.

    43FAA's Flight Standards Service is part of the Office of Aviation Safety.

    Page 26 GAO-05-930 Aviation Safety

    of the corrective actions needed to resolve the findings. The safety audit
    identified dozens of findings, many of which were also found in a second
    audit 2 years later and, according to the airline, subsequently corrected.
    Furthermore, because FAA has not provided its reviewers or the airlines
    with a standard definition of "safety-critical" findings that must be
    corrected before the audit can be closed, it is unknown whether these open
    findings were safety critical. Moreover, the reasonableness of leaving
    open dozens of safety audit findings is questionable, as is FAA's reliance
    on the airlines' compliance statements as proof that all corrective
    actions have been made. Although the U.S. carrier temporarily suspended
    the code-share arrangement with this foreign carrier, FAA officials said
    the suspension occurred because of FAA's concern about the safety
    oversight of that foreign airline's civil aviation authority, not because
    of the number of audit findings or their lack of closure.

    FAA uses compliance statements, which are based on the safety audit
    results, as reasonable assurance that the foreign airlines meet ICAO
    safety standards. However, FAA's reliance on compliance statements may not
    provide such assurance because FAA has accepted compliance statements as
    proof that the carriers met ICAO safety standards, even in situations when
    it questioned the audit results. For example, FAA provided memorandums of
    no objection to DOT's Office of International Aviation that were based on
    safety audits conducted by one airline contractor over a 4-year period,
    many of which did not identify any findings,44 even though an FAA official
    told us that he had discussed with the airline FAA's concern about the
    number of audits that did not identify any findings.45

    44We found that over half of the contractor's audit reports had no
    findings. By comparison, we estimate that 25 percent of all audits were
    closed with no findings.

    45The airline stopped using this contractor in 2003, after which the
    airline conducted the safety audits itself. The airline indicated that its
    decision to begin conducting the audits using its own resources was based
    on the opportunity to lower costs while expanding its auditing checklists
    to include elements from the IOSA program. In addition, the airline
    indicated that using its resources provided an improved opportunity to
    communicate with its counterparts at the code-share airline and exchange
    recommended safety practices. An official from the airline said that the
    airline shared FAA's concern about the relatively low number of findings
    contained in the contractor's audit reports, but explained that the
    airline had no basis for comparison since it was not conducting the audits
    itself. The official said that now that the airline has conducted several
    safety audits itself, there have been only a few instances when findings
    impacting the safety of flight have been discovered. In addition, the
    official said that many times findings are identified and resolved before
    the audit team concludes the audit and that most of the recent findings
    involve lack of documentation for established practices.

    According to the Code-Share Safety Program guidelines, U.S. airlines
    should not submit compliance statements to FAA until all corrective
    actions have been completed; the statements should not be predicated on
    future actions that are planned to be completed. However, FAA officials
    said that they allow "nonsafety-critical" findings identified during the
    audit, such as deficiencies in personnel training and omissions in
    manuals, to be addressed later. Because FAA has not provided the airlines
    with a standard definition of "safety-critical" findings that must be
    corrected before the audit can be closed, airlines could interpret the
    term inconsistently in documenting and resolving corrective actions. An
    FAA official indicated that developing a definition of safety critical
    would be difficult and time consuming. An aviation safety expert we
    consulted said that a definition of safety critical would require
    considerable study and criteria development because situations can be
    critical to safety in many ways. He added that a well-trained and
    experienced aviation safety inspector could identify a safety-critical
    situation. However, this same expert suggested that, as a quality
    assurance measure, FAA select several audits each year and check the
    underlying documentation in depth. Similarly, the DOT Inspector General
    recommended in 1999 that FAA conduct comprehensive audits of a sample of
    safety audits to confirm that carriers have applied agreed-upon standards
    and procedures in conducting the audits.46 However, even if FAA were to
    conduct such comprehensive audits, without a definition of safety-critical
    findings, the agency would still lack assurance that safety-critical
    findings were identified and resolved.

    FAA indicated that from August 2003 through July 2004,47 18 of the 50
    audit reports on foreign airlines it reviewed were returned to U.S.
    carriers for further action and 4 were placed on hold pending the outcome
    of IASA reviews; the other 31 foreign carriers received memorandums of no
    objection. Furthermore, FAA officials said that, according to anecdotal
    information from some U.S. carriers, too many safety concerns were
    identified during some safety audits for the carriers to proceed with
    applications for code-share authorization. However, FAA officials said
    they do not know how many times the safety audits have prevented airlines
    that pose safety concerns from becoming code-share partners with U.S.
    airlines.

    46DOT Office of the Inspector General, Aviation Safety Under International
    Code-Share Agreements, Report No. AV-1999-138 (Washington, D.C.: Sept. 30,
    1999).

    47FAA did not maintain data on the number of audit reports that it
    reviewed and placed on hold for other years.

    Page 28 GAO-05-930 Aviation Safety

    In addition, effective management controls require that documentation be
    created and maintained to provide evidence of executing approvals,
    authorizations, verifications, and performance reviews. FAA devised a
    checklist for agency staff to complete while reviewing safety audit
    reports to check for compliance with the program guidelines, record
    information about findings, or report irregularities. FAA officials said
    that the checklist was developed to establish and maintain consistency in
    reviewing the audit reports. However, we found that the checklist did not
    consistently document what actions FAA took when reviewing the airlines'
    audit reports, which findings it reviewed, and which corrective actions it
    verified were implemented. For example, in some cases, the checklist
    provided information about the closure of findings, but in other cases, no
    information was recorded about closure. FAA officials said that portions
    of the checklist may be left blank until the FAA reviewer has completed
    discussions with the airline and answered all of the concerns to his or
    her satisfaction, at which time the FAA reviewer will note that no
    irregularities were found. Officials said that in such cases, the
    checklist would not capture this process. However, not completing this
    information could hinder future reviews of the same airline by impeding
    comparisons between audits. Furthermore, because FAA often lacked
    documentation that it had verified the closure of findings, we were unable
    to determine how frequently FAA may have provided memorandums of no
    objection on foreign carriers that had not implemented all corrective
    actions in response to the findings, as occurred in the example discussed
    earlier.

    Effective management controls also include monitoring to assess the
    quality of performance over time. Management controls generally should be
    designed to ensure ongoing monitoring during normal operations and include
    regular management and supervisory activities, comparisons,
    reconciliations, and other actions people take in performing their duties.
    FAA officials said that the manager of the International Programs and
    Policy Division, which is responsible for administering the Code-Share
    Safety Program and is part of FAA's Flight Standards Service, is briefed
    by the Code-Share Safety Program staff on the results of their safety
    audit reviews before a recommendation is made to the Director of Flight
    Standards to sign the memorandums of no objection that are sent to DOT's
    Office of International Aviation. This procedure allows the International
    Programs and Policy Division manager to monitor the results and the
    decision-making processes involved. In addition to reviewing the audit
    reports, FAA monitors the safety of foreign carriers through other sources
    of information. FAA officials said they also review any accident and
    incident information from aviation safety databases, company financial
    histories, ICAO reports on the countries' civil aviation authorities,48
    media reports, ramp inspection results,49 and information from FAA
    international field offices about their inspections of foreign aircraft
    when these aircraft enter the United States.50

    According to the Code-Share Safety Program guidelines, the U.S. airlines
    participating in the program should have a process to monitor the safety
    of their foreign code-share partners on an ongoing basis, and FAA should
    review this monitoring process. FAA officials said they have reviewed the
    monitoring systems at seven of the eight U.S. airlines participating in
    the program.51 However, FAA had not documented its reviews of the
    monitoring systems, so we were unable to verify that activity.
    Furthermore, safety officials at three of the eight U.S. airlines said FAA
    had not reviewed their monitoring systems.52 Without an FAA review,
    deficiencies in these monitoring systems might not be identified.

    FAA does not maintain information on the types and frequencies of audit
    findings to provide a means of comparing the findings from initial and
    recurrent audits of the same airline, or perform trend analysis that could
    help identify problems across airlines or fleets. Trend analyses would be
    useful for monitoring, on an ongoing basis, the effectiveness of FAA's
    internal quality control system. FAA officials said the checklists are not
    used for tracking or trend analysis and that FAA does not formally examine
    either the safety problems occurring most often or the geographic areas

    48ICAO's Universal Safety Oversight Audit Program regularly audits
    countries that have signed the Chicago Convention of 1944, called ICAO
    Contracting States, to determine the status of the states' implementation
    of safety oversight and relevant ICAO standards and recommended practices,
    associated procedures, guidance material, and safety-related practices.

    49FAA conducts annual ramp inspections of foreign carriers entering the
    United States or monthly inspections (called the special emphasis list)
    when the results of inspections are repeatedly poor or the airline's
    country of operation is placed in IASA category 2.

    50Under 14 C.F.R. 129, FAA may inspect foreign airlines when they enter
    the United States, but FAA does not inspect foreign aircraft outside the
    United States. The Chicago Convention limits FAA and other civil aviation
    authorities to regulating or auditing foreign airlines within their own
    country's airspace.

    51An FAA official said that FAA had not observed the eighth U.S. airline's
    monitoring system because it was a new participant in the Code-Share
    Safety Program.

    52An official at another U.S. airline said that FAA had observed its
    system to track open safety audit findings but not its entire monitoring
    system.

    U.S. Airlines Are Auditing Foreign Partners' Safety, and Partners Are
    Taking Corrective Action, but Documentation of Corrective Actions Is Often
    Lacking

    where problems are occurring most frequently. However, the officials said
    that the FAA program manager does want to have a general idea of the types
    of problems being found, and the checklist provides this information
    informally. According to one FAA official, the purpose of the checklist is
    to ensure that the DOT guidelines are met, rather than to create a
    database of findings. In our view, not maintaining such documentation
    could impede analyses of trends and comparisons of findings, as well as
    limit opportunities for assessing risks and prevents determining whether
    FAA reviewed those findings.

    Establishing performance measures is another component of effective
    management controls. The Government Performance and Results Act of 1993
    requires agencies to, among other things, set strategic and annual
    performance goals, and measure and report on performance toward these
    goals. Management controls play a significant role in helping managers
    achieve those goals. FAA has established certain performance goals for the
    Code-Share Safety Program, including reviewing at least 40 safety audit
    reports during fiscal year 2004. FAA exceeded this goal by completing 57
    reviews. In addition, FAA set a performance goal of meeting with major

            U.S.
                    air carriers to request feedback on the Code-Share Safety
                    Program. FAA met this goal in 2004. The eight U.S. airlines
                    participating in the Code-Share Safety Program have conducted
                    the safety audits of their foreign code-share partners and
                    have monitored the safety of their code-share partners
                    between audits, as specified under the guidelines. Through
                    those audits, the U.S. airlines have identified numerous
                    safety issues associated with their foreign partners'
                    operations. After completing the audits, the U.S. airlines
                    have submitted written statements to FAA affirming their
                    foreign code-share partners' compliance with ICAO standards,
                    as specified under the guidelines. However, the U.S. airlines
                    have not always documented the implementation of actions
                    taken in response to the findings. Many airlines are now
                    moving to adopt the international safety audit program, IOSA,
                    which contains procedures that would help to ensure that
                    corrective actions implemented in response to audit findings
                    are documented. Most

    U.S.
               airline officials said they believe the Code-Share Safety Program
               provides reasonable assurance of safety or is effective, but some
               officials also suggested various changes in its administration.

    U.S. Airlines' Safety Audits Primarily Identified Findings in Foreign
    Partners' Flight Operations and Maintenance and Engineering

    The U.S. airlines participating in the Code-Share Safety Program have been
    assessing the safety of their foreign code-share partners at least every 2
    years, as the guidelines specify.53 We estimate, based on the results of
    our sample of 149 randomly selected safety audit reports, that there are
    2,047 findings among the audits that the eight U.S. airlines conducted of
    foreign carriers, which FAA reviewed from February 2000 through September
    2004.54 The program guidelines define a finding as an instance in which
    "the performance of the standard does not meet the established criteria"
    under ICAO standards. We estimate that 75 percent of the audits of foreign
    carriers that the eight U.S. airlines conducted of foreign carriers and
    that FAA reviewed from February 2000 through September 2004 contained at
    least one finding. Airline officials told us that most findings related to
    a lack of documentation. Documentation is important to ensure the
    implementation of management controls, which should appear, for example,
    in management directives and operating manuals. However, we found that
    many of the safety audit findings were broader in scope than a lack of
    documentation and extended to a lack of underlying policies and
    procedures. We further estimate that findings related to deficiencies in
    policies and procedures accounted for 23 percent55 of all findings.

    The audits reviewed the carriers' compliance in eight major categories
    (organization, flight operations, flight dispatch, maintenance and
    engineering, cabin operations, cargo and dangerous goods, ground handling,
    and security).56 As shown in figure 5, the findings spanned all eight
    categories, but the largest numbers were in two categories: (1) flight
    operations, which govern the activities of the pilots, including training,
    and

    (2) maintenance and engineering, which involves the oversight of
    activities

    53As discussed earlier, for 12 of the 256 audit reports that FAA reviewed
    from February 2000 through September 2004, FAA granted the U.S. airlines
    extensions of time to submit compliance statements. In some cases, U.S.
    airlines conducted safety audits of their foreign partners more frequently
    than every 2 years.

    54We are 95 percent confident that the actual number of findings lies
    between 1,450 and 2,643. All percentage estimates from the sample of
    audits have sampling margins of error of plus or minus 10 percentage
    points or less unless otherwise noted. All numerical estimates other than
    percentages have margins of error of plus or minus 10 percent of the value
    of those estimates unless otherwise noted. See appendix I for additional
    information on our sampling methodology.

    55The 95 percent confidence interval surrounding this estimate ranges from
    15 to 36 percent.

    56Most of the audit reports that we reviewed were organized into these
    eight categories.

    Airlines Submitted Compliance Statements to FAA but Did Not Always
    Document the Closure of Findings

    to maintain, repair, and overhaul aircraft, aircraft engines, and parts.
    In the flight operations category, the findings included a lack of drug
    and alcohol testing policies and a lack of documentation on flight time
    and rest requirements for flight personnel. In the maintenance and
    engineering category, one common type of finding related to the
    maintenance and calibration of tools and supplies, which could affect
    safety.

    Figure 5: Percentage of Findings by Audit Category

    Flight operations

    3%

    Unknown

    4%

    Security

    Cabin operations

    Cargo and dangerous goods

    Organization

    Ground handling

    Flight dispatch

    Maintenance and engineering

    Source: GAO analysis of the safety audit reports.

    Note: Percentages do not total to 100 because of rounding.

    After U.S. airlines completed their audits, their safety directors
    submitted statements to FAA affirming their foreign code-share partners'
    compliance with ICAO standards. FAA officials said they rely on these
    compliance statements as the primary evidence that the foreign code-share
    partners of

    U.S. airlines have resolved all safety-critical findings. However, on the
    basis of our review of a sample of the audit reports, we estimate that,
    for 68 percent of the identified findings,57 the documentation was
    insufficient to demonstrate that the findings had been closed or were
    resolved. Specifically, the documentation either failed to indicate at
    least one of the following three elements:58 (1) what corrective action
    was taken, (2) who accepted the corrective action, and (3) when the
    corrective action was accepted or the documentation was insufficient to
    determine whether the findings were closed. An estimated 28 percent59 of
    the audit reports that contained findings had at least one finding that
    lacked all three elements documenting corrective actions.60 The Code-Share
    Safety Program guidelines do not indicate that U.S. airlines should have
    documentation available for FAA's review to provide evidence of what
    corrective action was taken, who accepted the action, and when the action
    occurred in response to the findings identified in audits of their foreign
    code-share partners.61

    We asked the eight U.S. airlines participating in the Code-Share Safety
    Program what types of systems they were using to track any findings that
    were not resolved when the safety audit was complete. We found that three
    of the U.S. airlines were using computer systems to track the closure of
    such open findings; three other airlines had computer systems that could
    track the closure of findings, but their foreign partners had no open
    findings; and two airlines indicated that they did not have systems to
    track open findings because their foreign partners did not have any open
    findings. At one U.S. airline that was using a computer system to track
    open findings, officials said that a computer malfunction resulted in the
    loss of 6

    57These findings were not categorized as being either safety critical or
    nonsafety critical. Furthermore, because FAA had not developed a
    definition of safety-critical findings, which FAA requires airlines to
    resolve immediately, we could not determine how many safety-critical
    findings lacked complete documentation of corrective action.

    58We identified these three elements as sufficient evidence that the
    finding was resolved.

    59The 95 percent confidence interval surrounding this estimate ranges from
    16 percent to 44 percent.

    60An estimated 64 percent of the audit reports that contained findings had
    at least one finding that lacked at least one element documenting
    corrective actions.

    61The program guidelines indicate that the U.S. carriers should ensure
    that their foreign partners have processes in place that identify "types
    of problems that may occur from common or special circumstances" and that
    corrective action takes into account, among other things, "the existence
    of documentation of the changes made to analyze the effectiveness of the
    corrective action."

    months of data.62 An official from this airline said that before 2004, the
    airline coordinated closure of any findings directly with the contractor.
    When asked to produce this information, the airline did not have finding
    closure documentation available for audits conducted before 2004. This
    contractor said that although his firm was asked a few times by the U.S.
    carrier to check on the closure of audit findings by its foreign partner,
    the

    U.S. airline was responsible for tracking the closure of findings.

    Airlines also lacked documentation on the closure of findings in part
    because an unknown number of findings were closed on-site during the
    audits and not documented. The FAA program manager said he discouraged
    closing out findings on-site without documentation during the audits
    because it does not leave an audit trail about what findings were
    identified. Documentation provides a record of the execution of management
    controls which, in this situation, relate to the implementation of
    corrective actions.

    We estimate that 25 percent of the audits were closed with no findings
    identified. According to an FAA official, audits that identify no findings
    are questionable because the airlines must comply with so many
    requirements under either ICAO or IOSA standards. One U.S. airline used a
    contractor to conduct 31 of the audits of foreign airlines in our sample
    from 1999 through 2003, over half of which identified no findings.63 As
    described earlier, an FAA official told us that he had discussed with the
    airline FAA's concern about the number of audits conducted by the
    contractor that did not identify any findings. The FAA official also said
    that he helped the airline revise its approach to conducting the audits as
    a part of its internal evaluation program. The contractor told us that it
    is common for the safety audits not to identify findings because the
    airlines have prepared for the audit, and the audit findings are sometimes
    resolved on the spot. The contractor also said that his firm often
    recommended best practices that the foreign carriers could implement, but
    these recommendations did not relate to violations of ICAO standards and,
    thus, were not considered to be

    62An official from this airline said that although the electronic data
    were lost, the airline still had the paper documentation of the
    information that was contained in the computer system. This official also
    said that the airline had implemented a new tracking system, which he said
    was an improvement over the previous system.

    63From 2000 to 2004, six of the eight U.S. airlines used contractors to
    audit the safety of their foreign code-share partners, although three used
    them for only one audit. U.S. airline officials said they sometimes used
    contractors because of convenience (e.g., when contractors were located
    near the foreign airlines' operations).

    U.S. Airlines Also Monitor the Safety of Their Foreign Partners between
    Audits

    findings. Furthermore, this contractor said that a representative from the

    U.S. airline, who accompanied the contractor's auditors on the audits,
    kept the U.S. airline informed.

    The eight U.S. airlines participating in the Code-Share Safety Program
    have processes to monitor the safety of their foreign code-share partners
    on an ongoing basis, including their accident and incident rates,
    financial condition, equipment age, labor issues, and other issues, as
    called for in the program guidelines. Safety officials from the eight U.S.
    airlines said that, to their knowledge, no fatal accidents had occurred on
    their foreign code-share routes since the Code-Share Safety Program began
    in 2000. We observed the systems and information sources that each U.S.
    airline used for monitoring. Airline officials showed us, for example,
    safety questionnaires that they sent to their code-share partners between
    formal safety audits, news subscription services, and aviation safety Web
    sites. Some airline officials also said they occasionally made on-site
    visits to monitor their partners' safety. The airlines also indicated that
    they monitor any accident and incident data for their code-share partners.
    According to a safety official at one U.S. airline, a carrier's past
    accident and incident record does not conclusively prove that a safety
    problem exists, but it can be an indicator of other deficiencies, such as
    gaps in training. Some officials from airlines that are part of global
    alliances also said that they share safety information about their mutual
    foreign code-share partners. Four U.S. airlines had created computer
    databases to maintain this monitoring information while the other four
    maintained paper files.

    Use of International Safety Audit Program May Address Some Weaknesses in
    the

    U.S. Safety Audit Program

    As U.S. airlines and their foreign code-share partners begin to use IOSA-a
    new safety audit program developed by IATA-some of the weaknesses that we
    observed in the Code-Share Safety Program may be addressed, and

    U.S. airlines may receive other benefits. Increased use of IOSAs may help
    to ensure that audit findings are resolved and corrective actions
    implemented. IOSA requires that findings that are identified during the
    audit be documented, excluding those that are corrected immediately
    on-site during an audit.64 In addition, IOSA requires documentation of
    closure

    64According to IATA, when an airline being audited implements immediate
    corrective action while the audit team is still on site, the audit team is
    permitted to exclude the recording of a finding only if it is able to
    verify full implementation of comprehensive and permanent corrective
    action.

    Page 36 GAO-05-930 Aviation Safety

    for findings, including the three elements we identified-(1) a description
    of the corrective actions taken, (2) who accepted the corrective actions,
    and (3) when the corrective action was accepted-as well as the reasoning
    used by the auditing organization to clear the findings. As noted,
    documentation of one or more of these elements was missing, or it could
    not be determined if elements were missing for an estimated 68 percent of
    the audit findings.

    Six of the eight U.S. airlines use IOSA standards to audit the safety of
    their foreign code-share partners, one may do so in the future, and one
    does not plan to use the standards to audit the safety of its foreign
    code-share partner.65 Moreover, according to some airline officials, U.S.
    airlines have a financial incentive to encourage their foreign code-share
    partners to undergo IOSAs because the auditing costs66 are shifted from
    the U.S. airline to its foreign partner. However, not all U.S. airlines
    plan to require IOSAs of their foreign code-share partners.67 For example,
    officials from one U.S. airline said that IOSAs may be too expensive for
    some small foreign carriers. Similarly, officials at another U.S. airline
    said that IOSAs are applicable to airlines with large fleets and major
    processes but may not be practical for smaller airlines. Officials at a
    third U.S. airline said they preferred to continue conducting the safety
    audits themselves, rather than using an auditing organization selected by
    IATA, because they wanted the assurance of examining their partners'
    operations in person, rather than relying on an external organization.

    Finally, increased use of IOSAs may help standardize aviation safety
    auditing and streamline FAA's review of audit reports. Under the IOSA
    program, the airlines can obtain the audit results of their mutual
    code-share partners. Of the eight U.S. airlines with foreign code-share
    partners, six share 18 of the same foreign code-share partners. FAA
    recently allowed

    U.S. airlines to submit for review audit reports that other U.S. airlines
    had

    65One U.S. airline was using an auditing organization accredited by IATA,
    which is an independent business unit of that U.S. airline.

    66An IATA official said the average cost of conducting a safety audit of a
    foreign airline was between $50,000 and $70,000.

    67IATA originally set a goal of having all 265 of its members undergo
    IOSAs by January 2006, but it later revised that to having 140 airlines
    undergo IOSAs by the end of 2005. As of June 2005, 88 IOSA audits had been
    completed worldwide, including 66 IATA members. In May 2005, IATA's Board
    of Governors decided in principle that all IATA members will have to
    undergo IOSA.

    Most U.S. Airline Officials Believe the Code-Share Safety Program Provides
    a Reasonable Assurance of Safety

    conducted on a shared foreign code-share partner.68 Some U.S. airlines, as
    members of global airline alliances, plan to share their audit reports of
    foreign partners and reduce duplicative audits.69 The IOSA program should
    make it easier for airlines that are not in such alliances to share audit
    reports.70 Increased sharing of the reports could reduce the number of
    safety audits that the U.S. airlines would need to conduct of their
    foreign partners and could thus reduce the number of reports that FAA
    would need to review.

    Officials at most U.S. airlines participating in the Code-Share Safety
    Program told us they believe that the program provides reasonable
    assurance of safety concerning their foreign code-share partners or is
    effective. One airline official described the program as an "ingenious
    technique" that has had the effect of raising aviation safety standards
    worldwide by ensuring that safety issues will be resolved. This airline
    official said that some foreign airlines, seeking to become code-share
    partners of U.S. airlines, have restructured programs, rewritten manuals,
    and instituted new management techniques-evidence, he said, of the
    program's effectiveness. Another U.S. airline official said that, without
    the Code-Share Safety Program, U.S. airlines might not conduct safety
    audits of their foreign code-share partners. An official at another U.S.
    airline said the Code-Share Safety Program is a means to ensure that a
    carrier meets minimum ICAO-based international aviation safety standards
    and that the IOSA program creates a baseline of auditing standards to be
    followed worldwide. However, the official said that a safety audit,
    whether conducted by an auditing organization selected by IOSA or a U.S.
    airline, is only a snapshot of the carrier for the period in which the
    audit is

    68FAA requires that the audit be conducted using an FAA-accepted
    methodology and that the airline that is relying on the audit report
    submit a compliance statement to FAA.

    69Five of the eight U.S. airlines participating in the Code-Share Safety
    Program belong to global airline alliances with other U.S. airlines in the
    program. Continental, Delta, and Northwest belong to the Sky Team
    Alliance, and United Airlines and US Airways belong to the Star Alliance.

    70According to IATA, any interested party, such as an airline or
    regulator, may make a request through IATA to view the audit report of an
    airline that is on the IOSA registry, which lists the airlines that have
    undergone IOSA. After a requester "qualification and verification
    process," which includes the signing of a nondisclosure agreement and
    specific approval for release by the audited airline, the report is made
    available to the requesting party. The use of IOSA audit data by
    regulatory authorities was encouraged in November 2004, according to IATA.

    conducted. The airline official said that the carrier's actions before the
    audit or after the audit may differ and cannot be adequately evaluated
    until additional safety information is collected from the carrier between
    safety audits or until the next safety audit.

    An official at another U.S. airline participating in the Code-Share Safety
    Program said that although a safety audit provides a very good assessment
    of an airline's compliance with aviation safety standards, it does not
    guarantee the safety of the carrier's operations. This official added that
    even if a safety audit were conducted on a carrier monthly, it would not
    guarantee that the carrier would never have an accident. Furthermore, an
    official at another U.S. airline said that the Code-Share Safety Program
    is not necessarily required to provide reasonable assurance of safety
    concerning the foreign code-share partners of U.S. airlines and that the
    airline does not necessarily believe that formal, FAA-approved safety
    audits are the only way to gain such assurance. This airline official said
    that U.S. airlines should not be required to conduct safety audits of
    foreign airlines that are operating out of countries that FAA rated as
    IASA category 1 and that U.S. airlines should be able to choose whether to
    conduct safety audits in countries that FAA has rated as IASA category 2
    or has not rated. This airline official added that while the U.S. airline
    may continue to audit its partners on its own, it does not believe that
    FAA should oversee this process. However, an FAA IASA program official
    told us that the IASA program focuses on the capabilities of the foreign
    civil aviation authorities and does not ensure the safety of any carriers
    operating in IASA category 1 countries. This FAA official also said that
    inconsistencies in aviation safety oversight can exist throughout the
    world, even in countries with "higher" standards, and that some countries
    exceed ICAO standards, while others do not.

    A safety official at one U.S. airline said he believed that the Code-Share
    Safety Program guidelines should be made regulations. Although officials
    from DOT's Office of International Aviation and FAA said that making the
    program regulatory is not needed because it is working well, this airline
    safety official said that making the program regulatory would allow
    requirements to be applied more evenly to all airlines participating in
    the program. This airline official added that DOT is requiring the
    guidelines to be followed and therefore they are regulations in practice.
    A safety official at another U.S. airline questioned why DOT requires
    "guidelines" to be followed. He said that if DOT wants "rigid compliance"
    with the guidelines, it should make the program regulatory. A safety
    official at a third U.S. airline said the program's requirements should be
    standardized, noting that, for example, FAA was inconsistent about its
    requirements for reviewing auditors' qualifications. An aviation safety
    expert we consulted also said that the program should be made regulatory,
    observing that both the Code-Share Safety Program and IASA suffer from a
    "lack of regulatory teeth" and that making them regulatory would provide
    clarity to the DOT requirements, which he said are "mere policies." At the
    same time, this expert said that although the program is not regulatory,
    the Code-Share Safety Program guidelines clearly lay out what is expected
    of the airlines and set the standards that must be met. He added that
    under the guidelines,

    U.S. airlines are held accountable for the safety of their foreign
    code-share partners.

    Finally, officials at two airlines said that they would like FAA to
    provide a definition of safety critical or to define when an audit is
    considered to be closed so that it would be clear which findings must be
    resolved before closing an audit and submitting a compliance statement. As
    noted, FAA officials said that they allow nonsafety-critical findings
    identified during the audits to be addressed after the code-share
    arrangement is authorized.

    The safety of foreign code-share partners of U.S. airlines is important

    Conclusions

    because several million people fly on those foreign carriers using tickets
    purchased from U.S. airlines each year. Under the Code-Share Safety
    Program, the U.S. airlines are auditing the safety of their foreign
    code-share partners and identifying safety concerns, which the foreign
    carriers are addressing. However, FAA's reviews of the safety audit
    reports lack management controls for establishing reviewers'
    qualifications, verifying corrective actions, and documenting the reviews.
    FAA, for example, has not established the qualifications needed to review
    safety audit reports, and FAA field inspectors, who are reviewing many of
    the safety audit reports, have not been trained in the IOSA
    program-potentially impeding FAA's review of audits that were conducted
    using those standards. In addition, the program guidelines do not provide
    clear direction to the U.S. airlines and FAA reviewers on which concerns
    are critical to safety and must be addressed before DOT's Office of
    International Aviation will authorize or reauthorize a code-share
    arrangement. Without a definition of safety-critical concerns and complete
    documentation of the closure of findings, FAA lacks clear criteria for
    responding to requests from DOT's Office of International Aviation about
    the safety of foreign carriers and lacks assurance that safety-critical
    concerns have been addressed. Furthermore, FAA is not using effective
    management controls when it fails to document its reviews of the airlines'
    safety audit reports. Without complete documentation, a determination
    cannot be made of what actions FAA took when reviewing the reports, which
    findings it reviewed, and which corrective actions it verified were
    implemented. Because documentation on FAA's verification of the closure of
    findings was often lacking, we were unable to determine how frequently FAA
    may have failed to object to the authorization of code-share arrangements
    with foreign carriers that had not implemented all corrective actions in
    response to the findings. FAA also has not implemented a DOT Inspector
    General's recommendation that it conduct a comprehensive examination of a
    sample of audit reports to verify the underlying documentation.
    Furthermore, FAA's not collecting and tracking safety audit findings is an
    obstacle to conducting trend analysis or spotting anomalies.

    The airlines' increasing adoption of the IOSA program as a worldwide
    safety auditing standard is likely to change how FAA conducts its safety
    reviews of foreign code-share partners of U.S. airlines. Moreover, IOSA
    requires that actions to correct all findings, except those that are
    corrected during an audit, be documented-a requirement that is lacking in
    FAA's program. However, the adoption of the IOSA program is likely to be
    gradual, given that, as of June 2005, 66 of IATA's 265 members had
    completed the program.

    Finally, although DOD and FAA officials said they have different program
    objectives, the two federal agencies are nevertheless duplicating efforts
    by reviewing many of the same audit reports. In addition, DOD is not
    receiving the foreign airport security assessment information from TSA
    that DOT is receiving. TSA's information would provide DOD with more
    complete data for its safety reviews.

    To improve the safety oversight of foreign code-share operations, we

    Recommendations

    recommend that the Secretary of Transportation direct the FAA
    Administrator to implement the following three recommendations:

    1. Revise the Code-Share Safety Program guidelines to improve the
    effectiveness of the program and the clarity of the procedures that the
    airlines should follow in documenting and closing out safety audit
    findings. Because the audit guidelines indicate that the airlines should
    not submit compliance statements until all corrective actions have been
    completed, but FAA is allowing the airlines to resolve "nonsafetycritical"
    findings later, FAA should consider either following that guideline or
    defining "safety-critical" audit findings, so that the airlines and FAA
    reviewers know which types of findings must be corrected

    before submitting the compliance statements.

    ines' safety audit reports;
           identifying ways to document its reviews of the airlines' safety audit
           reports; increasing the scrutiny of audit reports that have an
           unusually high or low number of findings, periodically selecting a
           sample of safety audits to conduct a comprehensive review of the
           underlying documentation collected; and collecting and analyzing
           information on the audit findings for the foreign code-share partners
           of U.S. airlines so that the data can be more easily quantified and
           analyzed to spot possible trends and anomalies, should FAA decide such
           analyses are needed.
    .

    We provided drafts of this report to the Department of Homeland Security,

    Agency Comments

    (DHS), DOD, and DOT. DHS provided written comments, agreeing with our
    recommendation regarding TSA. DHS's comments are reprinted in appendix
    III. DOD provided no comments on our findings or recommendations. DOD and
    DOT provided some technical clarifications, which we incorporated into
    this report as appropriate. We received comments from DOT officials,
    including FAA's Deputy Associate Administrator for Aviation Safety. FAA
    generally agreed with the report and agreed to consider our
    recommendations. In addition, FAA provided comments on the Code-Share
    Safety Program, emphasizing that it is a collaborative effort between
    DOT's Office of the Secretary, FAA, and the air carriers. FAA officials
    also said that the program established guidelines for approving
    international code-share operations, with the intent of encouraging the
    highest possible levels of safety for international code-share operations.
    According to FAA, the program outlines the necessary steps that U.S. air
    carriers must follow in seeking approval from DOT to conduct code-share
    operations with foreign air carriers. The officials added that the
    Code-Share Safety Program charges U.S. air carriers with the primary
    responsibility for ensuring that their foreign code-share partners comply
    with applicable international aviation standards.

    As agreed with your office, unless you announce the contents of this
    report earlier, we plan no further distribution until 30 days from the
    date of this letter. At that time, we will send copies of this report to
    interested congressional committees; the Secretary of Transportation; the
    Administrator of FAA; the Secretary of Defense; the Secretary of Homeland
    Security; and the Assistant Secretary of Homeland Security for the
    Transportation Security Administration. Copies will also be available to
    others upon request and at no cost on GAO's Web site at www.gao.gov.

    If you or your staff have any questions about this report, please call me
    at (202) 512-2834 or dillinghamg@gao.gov. Contact points for our Offices
    of Congressional Relations and Public Affairs may be found on the last
    page of this report. GAO staff who made major contributions to this report
    are listed in appendix IV.

    Sincerely yours,

    Gerald L. Dillingham, Ph.D. Director, Physical Infrastructure Issues

    Appendix I

                           Objectives, Scope, and Methodology

    Our objective was to review the measures that the federal government is
    taking to provide reasonable assurance of safety and security when
    passengers travel on flights operated by the foreign code-share partners
    of

    U.S. airlines. To accomplish this, we reviewed (1) the extent to which the
    Department of Transportation's (DOT) authorization of U.S. airlines'
    code-share arrangements with foreign airlines is designed to consider
    safety and security, (2) how well FAA has managed the Code-Share Safety
    Program, and (3) the extent to which U.S. airlines have implemented the
    Code-Share Safety Program, and the results of their efforts.

    To determine how safety and security are considered in DOT's authorization
    of U.S. airlines' code-share arrangements with foreign airlines, we
    interviewed officials at DOT's Office of International Aviation, Federal
    Aviation Administration (FAA), Transportation Security Administration
    (TSA), and the Department of Justice (DOJ) and reviewed the Code-Share
    Safety Program guidelines and related program documentation, applicable
    international aviation safety standards, and relevant legal authorities.
    Our review covered the U.S. airlines' code-share partnerships with foreign
    carriers that DOT authorized from February 2000, when the Code-Share
    Safety Program began, through fiscal year 2004.1 At DOT, we interviewed
    the officials who decide whether to authorize such partnerships about the
    authorization process, their sources of information, and how often they
    authorize the partnerships. To gain a better understanding of the
    authorization process and the information considered, we also reviewed a
    sample of code-share applications that U.S. airlines had filed to
    establish code-share partnerships with foreign carriers. Our sample
    consisted of one randomly selected application filed by each of the eight
    U.S. airlines participating in the Code-Share Safety Program.

    We also interviewed DOT security officials about how they provide security
    clearances for foreign carriers and how often they have provided those
    clearances for code-share authorization. Because TSA was the source of
    aviation security information for DOT, we interviewed TSA officials about
    how they assess the security of foreign airlines and airports. We also
    reviewed data from TSA about the results and frequency of its security
    assessments of foreign airports and related legal authorities. Based on
    our understanding of the data through interviews with TSA officials, we
    determined that the data were sufficiently reliable for our purposes. In

    1Our review pertained only to U.S. airlines' code-share arrangements with
    foreign carriers for scheduled air service, and not for charter or cargo
    air service.

    Page 44 GAO-05-930 Aviation Safety Appendix I Objectives, Scope, and
    Methodology

    addition, we interviewed DOT officials who review the competitive aspects
    of the code-share arrangements about how they conduct their reviews and
    how often they have provided those clearances for code-share
    authorization. Because these DOT officials received advice from DOJ on
    potential antitrust issues involving the code-share partnerships, we also
    interviewed DOJ officials who provided that advice about their process and
    sources of information.

    At FAA, we interviewed officials about how they assess the capabilities of
    foreign civil aviation authorities through the International Aviation
    Safety Assessment (IASA) program and how those assessments relate to the
    Code-Share Safety Program. We also analyzed data on the results and
    frequency of IASA reviews since the Code-Share Safety Program was
    initiated. Based on our understanding of the data through interviews with
    FAA officials, we determined that the data were sufficiently reliable for
    our purposes. We reviewed documentation that FAA staff had prepared when
    they reviewed the airlines' safety audit reports to determine how they
    documented their reviews. We also discussed with FAA officials how often
    FAA provided memorandums of no objection to DOT's Office of International
    Aviation to support U.S. airlines' applications for code-share
    arrangements with foreign carriers.

    Because the Code-Share Safety Program was designed to assess foreign
    airlines' compliance with aviation safety standards established by
    International Civil Aviation Organization (ICAO), we interviewed ICAO
    officials about the standards, related international aviation safety
    issues, and the ICAO Universal Safety Oversight Audit Program, which
    assesses the capabilities of countries' civil aviation authorities. In
    addition, because many airlines are planning to use a new international
    safety audit program-the International Air Transport Association's (IATA)
    Operational Safety Assessment (IOSA) program-to assess the safety of their
    foreign partners, we interviewed IATA officials about how the program was
    developed, how airlines plan to implement it, and how it could affect the
    Code-Share Safety Program. For background information on how aviation
    safety varies internationally, we obtained data from IATA on aviation
    accident rates for different world regions. We did not review the
    reliability of IATA's aviation accident data because we used this
    information only for background purposes. We also interviewed officials
    from the Air Transport Association-a U.S. airline association-about its
    involvement in establishing the DOD safety audit program and its views on
    the Code-Share Safety Program and FAA's IASA program. Finally, because we
    found during our review that DOD had also established a program for
    reviewing the

    Appendix I Objectives, Scope, and Methodology

    safety of foreign carriers, we interviewed DOD officials about the design
    and implementation of its program. In addition, we obtained information
    about the safety audit reports that DOD had reviewed from fiscal year 2001
    through fiscal year 2004 and the results, which we compared with the
    results of those that FAA reviewed. We also discussed with FAA and DOD
    officials the extent to which they have coordinated their efforts.

    To determine how well FAA has managed the Code-Share Safety Program, we
    evaluated whether DOT's Office of the Secretary and FAA incorporated
    selected government auditing standards in the program's design and whether
    FAA effectively used management controls in reviewing the safety audit
    reports. Because the Code-Share Safety Program establishes an audit
    program, we reviewed whether the program's design, as reflected in the
    program guidelines, conforms to certain standards identified in Government
    Auditing Standards.2 We reviewed selected general standards3 that are
    contained in Government Auditing Standards (independence, professional
    judgment, and competence) to assess the program's design. Although we
    examined the audit methodologies that the

    U.S. airlines had developed and submitted to FAA for review, we did not
    review them for conformance with government auditing standards because FAA
    had already conducted this review as a condition of accepting the U.S.
    airlines' participation in the program. In addition, because we were
    evaluating the management of a government program, we examined FAA's
    application of management controls, which is synonymous with the term
    "internal controls," in its reviews of the safety audit reports using
    Standards for Internal Control in the Federal Government.4 We selected the
    management controls that were applicable to FAA's review of the audit
    reports for establishing reviewers' qualifications, verifying corrective
    actions, documenting the reviews, and monitoring and measuring
    performance. We also reviewed the recommendations contained in a 1999 DOT
    Office of the Inspector General report on aviation safety under
    international code-share agreements to determine whether and to what

    2GAO, Government Auditing Standards, GAO-03-673G (Washington, D.C.: June
    2003).

    3We did not include the quality control and assurance general standard
    because it mainly relates to the process of peer review.

    4GAO, Standards for Internal Control in the Federal Government,
    GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999) and GAO, Internal
    Control Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.:
    August 2001).

    Page 46 GAO-05-930 Aviation Safety Appendix I Objectives, Scope, and
    Methodology

    extent the report's recommendations-about how a code-share safety audit
    program should be designed-were implemented.

    To determine the extent to which U.S. airlines have implemented the
    Code-Share Safety Program and the results, we interviewed officials at the
    eight

    U.S.
               airlines that were participating in the program about how they
               were assessing the safety of their foreign partners and reviewed a
               sample of the reports. We drew a stratified random probability
               sample of 153 reports of audits conducted by U.S. airlines of
               their foreign code-share partners. This sample was drawn from a
               population of documentation maintained by FAA for the 242 audit
               reports that the agency had reviewed from February 2000 through
               September 2004. Of these 153 sampled audits, 2 were out of scope
               because the airlines withdrew them from consideration and 2 were
               in scope, but we did not complete our reviews of these reports. We
               ultimately collected information for 149 in-scope audits. With
               this probability sample, each audit report in the study population
               had a positive probability of being selected, and that probability
               could be computed for any audit. We stratified the population into
               nine groups on the basis of the U.S. airline conducting the audit,
               and further, for some of those airlines, whether the foreign
               airlines being audited were code-share partners with more than one

    U.S.
               airline or whether FAA's records of its reviews of the audit
               reports contained comments about the findings. Each sampled audit
               was subsequently weighted in the analysis to statistically account
               for all of the audits in the study population, including those
               that were not selected. During our audit work, three airlines
               provided information about a total of 14 additional audit reports
               that, according to the airlines, FAA had reviewed. These 14 audits
               were not included in the population from which we drew our sample
               because FAA's files did not contain information about them.
               Estimates generated in this report pertain only to the 242 audit
               reports that, according to FAA's files, the agency reviewed.

    Because we followed a probability procedure based on random selections,
    our sample is only one of a large number of samples that we might have
    drawn. Since each sample could have provided different estimates, we
    express our confidence in the precision of our particular sample's results
    in 95-percent confidence intervals. These are intervals that would contain
    the actual population values for 95 percent of the samples we could have
    drawn. As a result, we are 95-percent confident that each of the
    confidence intervals in this report will contain the true values in the
    study population. All percentage estimates from the sample of audits have
    sampling margins of error of plus or minus 10 percentage points or less
    unless otherwise noted. All numerical estimates other than percentages
    have margins of

    Appendix I Objectives, Scope, and Methodology

    error of plus or minus 10 percent of the value of those estimates or less
    unless otherwise noted.

    We did not determine whether the airlines complied with international
    aviation safety standards. However, we performed a content analysis of the
    audit reports in our sample to determine what types of safety findings
    were identified regarding the foreign carriers. We recorded the findings
    and grouped them into eight categories: (1) organization, (2) flight
    operations,

    (3)
               flight dispatch, (4) maintenance and engineering, (5) cabin
               operations,

                    (6)
                            cargo and dangerous goods, (7) ground handling, and
                            (8) security- because the reports were generally
                            organized into those categories. We then further
                            divided those eight categories into at least six
                            issue subcategories. Two coders independently
                            categorized each finding, and any coding
                            disagreements were resolved between the coders or by
                            a third reviewer. During our review of the audit
                            reports, we also attempted to determine whether
                            corrective actions taken in response to the findings
                            were documented. To accomplish this, we looked for
                            evidence of (1) what corrective action was taken, (2)
                            who accepted the corrective action, and

    (3)
               when the corrective action was accepted. We considered these three
               elements to be sufficient evidence of documentation after
               observing how some airlines had documented the closure of findings
               and by reviewing Government Auditing Standards, which indicate
               that auditors should examine whether recommendations from previous
               audits have been implemented, and from Standards for Internal
               Control in the Federal Government, which require management to
               determine whether proper actions have been taken in response to
               findings and audit recommendations.

    In addition to reviewing the audit reports at the airlines, we interviewed
    safety officials (typically the safety directors) at all eight U.S.
    airlines participating in the Code-Share Safety Program about how they
    assess the safety of their foreign code-share partners, including how they
    plan, carry out, and close the audits, as well as monitor the safety of
    their foreign partners between audits.5 We also observed the monitoring
    systems that

    5An independent business unit of one of the eight U.S. airlines
    participating in the Code-Share Safety Program is an auditing organization
    selected by IATA to conduct IOSAs. We had also interviewed staff from that
    organization about how the audits were conducted and their qualifications.

    Page 48 GAO-05-930 Aviation Safety Appendix I Objectives, Scope, and
    Methodology

    they had implemented, as the program guidelines require, and sources of
    information that they used to monitor the safety of their foreign
    code-share partners. In addition, we asked the U.S. airline safety
    officials about their program-related interactions with FAA and DOT's
    Office of International Aviation, whether and how they believe the program
    could be improved, and what they thought about the implications of the
    airlines' increased adoption of IOSA by as an international aviation
    safety audit program. We also obtained the views of an aviation safety
    expert about the Code-Share Safety Program. We selected this expert
    because of his experience in aviation safety, which included helping to
    design FAA's IASA program. Because some airlines had used contractors to
    conduct safety audits of their foreign code-share partners, we interviewed
    one contractor who said that he had conducted or helped to conduct safety
    audits for five of the eight U.S. airlines in the Code-Share Safety
    Program about how his firm conducted the audits and the qualifications of
    his staff.

    Finally, for background information on the extent to which passengers are
    traveling on foreign code-share partners of U.S. airlines, we asked the
    eight

    U.S. airlines to provide such data from 2000 through 20046 using the same
    methodology, which was based on the number of tickets that the U.S.
    airlines sold for travel on their foreign code-share partners. For
    example, if a U.S. airline sold a single ticket for travel that included
    one or more foreign code-share partner flight segments, this ticket was
    counted once. If a U.S. airline sold separate tickets for travel that
    included more than one foreign code-share partner flight segment, each
    flight segment was counted as a separate ticket. Some airlines could not
    provide data for all 4 years, but all eight U.S. airlines were able to
    provide data for 2004, which we reported. We did not independently verify
    this information provided by the airlines because it was used only for
    background purposes.

    6Annual data are in calendar years unless noted otherwise.

    Appendix II

    U.S. Carriers and Their Foreign Code-Share Partners

                        U.S. carrier Foreign code-share partner

                               Alaska (1 partner) Helijet

                   America West (1 partner) Royal Jordanian Airlines

    American (24 partners) Aer Lingus Aero Caribe Air Pacific BA CitiExpress
    British Airways Cathay Pacific China Eastern EVA Airways Finnair Gulf Air
    Iberia JAL JALways JetConnect Limited LACSA LAN Chile Lan Express Mexicana
    Airlines Qantas SN Brussels Swiss International Air Lines TACA TAM -
    Linhas Aereas Turkish Airlines

    Contintental (18 partners) AeroLitoral Aeromexico Air Europa Air France
    Alitalia Brit Air COPA CSA Czech Emirates EVA Airways flybe.British
    European KLM Cityhopper KLM Exel Airlines KLM Royal Dutch Airlines Korean
    Airlines Maersk Air TAP Air Portugal Virgin Atlantic

    Appendix II

    U.S. Carriers and Their Foreign Code-Share Partners

                             (Continued From Previous Page)

                        U.S. carrier Foreign code-share partner

    Delta (21 partners) AeroLitoral Aeromexico Air France Air Jamaica Alitalia
    Express Alitalia Team Avianca Brit Air China Airlines China Southern
    CityJet CSA Czech El Al Emirates flybe.British European KLM Royal Dutch
    Airlines Korean Airlines Malev Hungarian Airlines Regional Royal Air Maroc
    South African Airways

    Northwest (11 partners) Aeromexico Air Alps Air France Alitalia CSA Czech
    KLM Cityhopper KLM Exel KLM Royal Dutch Airlines Korean Airlines Malev
    Express Malev Hungarian Airlines

    Appendix II

    U.S. Carriers and Their Foreign Code-Share Partners

                             (Continued From Previous Page)

                        U.S. carrier Foreign code-share partner

                                  United (23 partners)

    AC Jazz Air Canada Air China Air Dolomiti Air Japan Air New Zealand Air
    Nippon All Nippon Airways (ANA) Asiana Austrian British Midland (BMI) LOT
    Polish Lufthansa Lufthansa Cityline Nakanihon Airlines PrivatAir
    (Switzerland) PrivatAir (Germany) SAS Thai Airways The Fair, Inc. Tyrolean
    Varig Virgin Blue

    US Airways (9 partners) AeBal (Aerolineas de Baleares) Air Dolomiti
    BahamasAir British Midland (BMI) Eurowings Lufthansa Lufthansa CityLine
    Spanair Winward Island Airways

    Source: FAA.

    Note: Data as of May 2005.

                                      Appendix III

                                   Comments from DHS

    Appendix IV

                         GAO Contacts and Staff Acknowledgments

    Gerald Dillingham (202) 512-2834

    GAO Contacts

    Teresa Spisak (202) 512-2834

    In addition to the above individuals, Elizabeth Eisenstadt, Jessica A.
    Evans,

    Staff

    Brandon Haller, Bob Homan, David Hooper, Casey Keplinger, Elizabeth A.
    Marchak, Sara Ann Moessbauer, Mark Ramage, and Sidney Schwartz made key
    contributions to this report.

    GAO's Mission

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts GAO
    Reports and newly released reports, testimony, and correspondence on its
    Web site. To

    have GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

                                 Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    Contact:

    To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm

    E-mail: fraudnet@gao.gov

    Federal Programs Automated answering system: (800) 424-5454 or (202)
    512-7470

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Relations
    Washington, D.C. 20548

    Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

    Public Affairs

    U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548
    *** End of document. ***