    Aviation Safety: FAA's Safety Oversight System Is Effective but
    Could Benefit from Better Evaluation of Its Programs' Performance
    (17-NOV-05, GAO-06-266T).

    The U.S. commercial aviation industry has an extraordinary safety
    record. However, when passenger airlines have accidents or
    serious incidents, regardless of their rarity, the consequences
    can be tragic. The Federal Aviation Administration (FAA) works to
    maintain a high level of safety through an effective safety
    oversight system. Keys to this system are to: (1) establish
    programs that focus resources on areas of highest safety risk and
    on mitigating risks; (2) provide training and communication to
    ensure that inspectors can consistently carry out the agency's
    oversight programs; and (3) have processes and data to
    continuously monitor, evaluate, and improve the numerous
    oversight programs that make up the safety oversight system. This
    statement focuses on these three key areas and is based on recent
    GAO reports on FAA's inspection oversight programs, industry
    partnership programs, enforcement program, and training program.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-266T
        ACCNO:   A41697
      TITLE:     Aviation Safety: FAA's Safety Oversight System Is
    Effective but Could Benefit from Better Evaluation of Its
    Programs' Performance
         DATE:   11/17/2005
      SUBJECT:   Airlines
    	     Data integrity
    	     Employee training
    	     Inspection
    	     Program evaluation
    	     Regulatory agencies
    	     Risk assessment
    	     Safety regulation
    	     Safety standards
    	     Training utilization
    	     Transportation safety

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-266T

    United States Government Accountability Office

    GAO Testimony

    Before the Subcommittee on Aviation, Committee on Commerce, Science, and
    Transportation, U.S. Senate

    Not to Be Released Before10:00 a.m. EST

    Thursday, November 17, 2005 AVIATION SAFETY

        FAA's Safety Oversight System Is Effective but Could Benefit from Better
                        Evaluation of Its Programs' Performance

    Statement for the Record by
    Gerald L. Dillingham, Ph.D., Director
    Physical Infrastructure Issues

    GAO-06-266T

    [IMG]

    Thursday, November 17, 2005

    AVIATION SAFETY

    FAA's Safety Oversight System Is Effective but Could Benefit from Better
    Evaluation of Its Programs' Performance

                                     What GAO Found

    FAA's safety oversight system includes programs that focus on risk
    identification and mitigation through a risk-based system safety approach,
    leveraging of resources through designee and partnership relationships,
    and enforcement of safety regulations, but the benefits of these programs
    are not being fully realized. For example, FAA's system safety approach
    includes the addition of a program that emphasizes risk identification to
    its traditional inspection program for overseeing some airlines, which is
    not based on risk. However, it is likely that the benefits of this
    approach could be enhanced if the inspection workload was not as heavily
    oriented to the traditional inspection program's non-risk based
    activities. FAA leverages its resources through its designee programs, in
    which designated individuals and organizations perform about 90 percent of
    certification-related activities, and through its industry partnership
    programs, which are designed to assist the agency in receiving safety
    information. An outgrowth of FAA's inspection process is its enforcement
    program, which is intended to ensure industry compliance with safety
    regulations. However, GAO has expressed concerns that this program may not
    be as effective as it could be in deterring violations.

    FAA has made training an integral part of its safety oversight system, but
    several actions could improve the results of its training efforts,
    including ensuring that inspectors are well-trained in FAA's system safety
    approach and have sufficient knowledge of increasingly complex aircraft
    and systems to effectively identify safety risks. FAA has established
    mandatory training requirements for its workforce and designees. We have
    reported that FAA has generally followed effective management practices
    for planning, developing, delivering, and assessing the impact of its
    technical training for safety inspectors.

    GAO has found inadequate evaluative processes and limitations with data
    for FAA's inspection programs, designee programs, industry partnership
    programs, and enforcement program. For example, FAA lacked requirements or
    criteria for evaluating its designee programs. In another example, FAA's
    nationwide enforcement database is not as useful as it could be because of
    missing or incomplete historical information about enforcement cases.

    FAA's Safety Oversight System Includes Inspections, Training, and Other
    Activities

                     United States Government Accountability Office

    Mr. Chairman and Members of the Subcommittee:

    The U.S. commercial aviation industry, with less than one fatal accident
    per 5 million flights from 2002 through 2005, has an extraordinary safety
    record. This record is a result of the efforts of the Federal Aviation
    Administration (FAA), airlines, manufacturers, and others in the aviation
    industry to maintain one of the safest aviation systems in the world.
    However, when passenger airlines have accidents or serious incidents,
    regardless of their rarity, the consequences can be tragic, as a single
    accident can result in hundreds of deaths. In order to maintain a high
    level of safety, it is critical to have a safety oversight system that is
    comprehensive, efficient, and effective and can provide an early warning
    of hazards that can lead to accidents. It is equally important to have a
    skilled, well-trained workforce to implement and monitor this safety
    oversight system. FAA's workforce of about 3,200 inspectors stationed at
    more than 100 field offices throughout the world is responsible for
    carrying out the agency's processes to certify, inspect, and enforce
    safety regulations for all aspects of the aviation industry, including the
    aircraft and its component parts, over 100 commercial airlines, about
    5,000 aircraft repair stations, and hundreds of thousands of pilots and
    mechanics. FAA augments its inspector workforce with nearly 13,600
    designated organizations and individuals (designees) that conduct the more
    routine aspects of industry oversight, such as administering tests to
    pilots and mechanics as part of their certification requirements, and
    augments the safety information that it obtains from inspections with
    industry partnership programs. Keys to making this safety oversight system
    work are to: (1) establish programs that focus resources on areas of
    highest safety risk and on mitigating risks; (2) provide training and
    other communication to ensure that inspectors maintain the skills and
    knowledge to consistently carry out the agency's oversight programs; and
    (3) have processes and data to continuously monitor, evaluate, and improve
    the numerous oversight programs that make up the safety oversight system.
    This statement focuses on these three key areas of FAA's "early warning
    system" and is based on our recent reports on FAA's inspection oversight
    programs, industry partnership programs, enforcement program, and training
    program. We will also discuss our recommendations that FAA has not fully
    addressed in these areas.

    In summary:

    o 	FAA's aviation safety oversight system includes programs that focus on
    risk identification and mitigation through a system safety approach, the
    leveraging of resources, and enforcement of safety regulations, but the

    benefits of these programs are not being fully realized. FAA's system

    safety approach has many strengths, including the addition of a program

    that emphasizes risk identification to its traditional inspection program
    for

    overseeing non-legacy airlines,1 which is not based on risk. However, it
    is

    likely that the benefits of this approach could be enhanced if the

    inspection workload for non-legacy airlines was not still heavily oriented

    to the traditional inspection program's non-risk based activities. FAA

    leverages resources through its designee programs, in which designees

    perform about 90 percent of certification-related activities, thus
    allowing

    FAA to better concentrate its limited staff resources on the most safety

    critical functions. However, concerns about the consistency and adequacy

    of designee oversight that FAA field offices provide have been raised by

    experts and other individuals we interviewed. FAA also leverages its

    resources through industry partnership programs, which are designed to

    assist the agency in receiving safety information. For example, FAA

    encourages voluntary reports of safety violations by responding to them

    by issuing a warning letter rather than a fine or other legal sanction.
    FAA's

    enforcement program, which is an outgrowth of its inspection process, is

    intended to ensure industry compliance with safety regulations and is

    another important element of its safety oversight system. FAA's policy for

    assessing legal sanctions against entities or individuals that do not
    comply

    with aviation safety regulations is intended to deter future violations.

    However, we found that recommendations for sanctions are sometimes

    reduced on the basis of factors that are not associated with the merits of

    the case, and the economic literature on deterrence suggests that the goal

    of preventing future violations is weakened when the penalties for

    violations are lowered for reasons not related to the merits of the case.

    o 	FAA has made training an integral part of its safety oversight system,
    but several actions could improve the results of its training efforts.
    FAA's use of a risk-based system safety approach to inspections requires
    inspectors to apply data analysis and auditing skills to identify and
    control potential risks. Therefore, it is important that inspectors are
    well-trained in this approach and have sufficient knowledge of
    increasingly complex aircraft and systems to effectively identify safety
    risks. FAA has established mandatory training requirements for its
    workforce as well as designees.

    1We refer to all passenger airlines that are not in FAA's Air
    Transportation Oversight System (ATOS) as non-legacy airlines. The seven
    "legacy" airlines and eight other airlines are overseen through ATOS. The
    air carriers in the ATOS program are Alaska; American; Continental; Delta;
    Northwest; United; American Eagle; Champion; ExpressJet; SkyWest;
    Southwest; Trans States; FedEx; United Parcel Service; and US Airways,
    which recently merged with America West.

    We have reported that FAA has generally followed effective management
    practices for planning, developing, delivering, and assessing the impact
    of its technical training for safety inspectors, although some practices
    have yet to be fully implemented. For example, in developing its training
    curriculum for inspectors, FAA followed effective management practices,
    such as developing courses that support changes in inspection procedures
    resulting from regulatory changes or agency initiatives. On the other
    hand, FAA develops technical courses on an ad hoc basis rather than as
    part of an overall curriculum for each type of inspector, such as
    inspectors of operations or cabin safety, because the agency has not
    systematically identified the technical skills and competencies each type
    of inspector needs to effectively perform inspections. FAA has recognized
    the need for improvements to its training program in this and other areas.

    o 	It is important for FAA to have effective evaluative processes and
    accurate nationwide data for its numerous safety oversight programs so
    that program managers and other officials have assurance that the safety
    programs are having their intended effect. Such processes and data are
    especially important because FAA's workforce is so dispersed throughout
    the world-with thousands of staff working out of more than 100 offices
    worldwide-and because FAA's use of a system safety approach represents a
    cultural shift from its traditional inspection program. Evaluation is
    important for understanding if the cultural shift has effectively
    occurred. Our most recent work has shown the lack of evaluative processes
    and limitations with data for FAA's inspection program for non-legacy
    airlines, designee programs, industry partnership programs, and
    enforcement program. For example, we found that FAA lacked requirements or
    criteria for evaluating its designee programs. In another example, FAA's
    enforcement policy calls for the assessment of sanctions that would
    potentially deter future violations. However, FAA lacks an evaluative
    process, so it is not known whether the agency's enforcement practices,
    such as at times reducing sanctions, may weaken any deterrent effect that
    would be expected from such sanctions. Furthermore, FAA's ability to
    evaluate its programs is hindered by the lack of useful nationwide data.
    For example, FAA's nationwide enforcement database is not as useful as it
    could be because of missing or incomplete historical information about
    enforcement cases.

    o 	In order to help FAA fully realize the benefits of its safety oversight
    system, we have made a number of recommendations to address the weaknesses
    that we identified in our reviews. These recommendations have not been
    fully implemented, although in some cases FAA has taken steps towards
    addressing them. Evaluative processes and relevant data are particularly
    important as FAA works to change its culture by incorporating

    Background

    a system safety approach into its oversight, and we have recommended that
    FAA develop continuous evaluative processes for its oversight of nonlegacy
    airlines, its designee programs, and its enforcement program, and
    systematically assess inspectors' technical training needs. In addition,
    FAA's nationwide databases are in need of improvements in their
    comprehensiveness and ease of use. We have recommended that FAA improve
    the consistency and completeness of its designee and enforcement
    databases. Continuous improvements in these areas are critical to FAA's
    ability to have a robust "early warning system" in order to maintain one
    of the safest aviation systems in the world.

    FAA's safety oversight system is made up of a number of programs for
    airlines and other entities. Safety oversight programs for airlines
    provide for their initial certification, periodic surveillance, and
    inspection. Since 1985, FAA has used National Work Program Guidelines
    (NPG), its traditional inspection program for airlines, as a primary means
    of ensuring that airlines comply with safety regulations. In NPG, an FAA
    committee of program managers identifies an annual minimum set of required
    inspections that are to be undertaken to ensure that airlines are in
    compliance with their operating certificates. In 1998, the agency
    implemented the Air Transportation Oversight System (ATOS), which
    currently oversees the nation's largest 15 commercial airlines and cargo
    carriers, with the goal of eventually including all commercial passenger
    and cargo airlines in it. ATOS emphasizes a system safety approach that
    extends beyond periodically checking airlines for compliance with
    regulations to the use of technical and managerial skills to identify,
    analyze, and control hazards and risks. For example, under ATOS,
    inspectors develop surveillance plans for each airline, based on data
    analysis and assessment of risks, and adjust the plans periodically based
    on inspection results. However, the agency has been delayed in placing a
    significant number of other passenger airlines in ATOS, resulting in 99
    passenger airlines, which we refer to as non-legacy airlines, continuing
    to be overseen through NPG, a process that is not risk-based or system
    safety oriented. In 2002, FAA added the Surveillance and Evaluation
    Program (SEP) to the NPG inspection program to incorporate principles of
    ATOS into its oversight of non-legacy passenger airlines. The two programs
    are used together to establish the number of annual inspections for
    non-legacy airlines. (Appendix 1 describes each inspection program.)
    Figure 1 illustrates some typical activities covered during inspections.

    Figure 1: FAA's Safety Inspections Cover a Wide Range of Activities

    Source: FAA.

    Note: As a workforce, FAA inspectors conduct a wide variety of
    inspections, including ensuring that pilots are qualified to operate
    aircraft, inspecting aircraft for safety, and overseeing FAA-certified
    mechanics.

    FAA's safety oversight programs for other aspects of the aviation
    industry-including manufacturers of aircraft and aircraft parts, repair
    stations, flight schools, aviation maintenance technician schools, pilots,

    and mechanics-involve certification, surveillance, and inspection by FAA's
    safety inspectors, engineers, flight surgeons, and designated
    representatives. FAA authorizes about 13,400 private individuals and about
    180 organizations (called "designees") to act as its representatives to
    conduct many safety certification activities, such as administering flight
    tests to pilots, inspecting repair work by maintenance facilities,
    conducting medical examinations of pilots, and approving designs for
    aircraft parts. These designees are grouped into 18 different programs and
    are overseen by three FAA offices-Flight Standards Service, Aerospace
    Medicine, and Aircraft Certification Service-all of which are under the
    Office of Aviation Safety (see fig. 2).

    Figure 2: FAA Offices That Manage the Different Designee Programs and
    Numbers of Designees (as of May 2004)

    Since 1990, FAA has emphasized gaining compliance from the aviation
    industry through cooperative means by establishing industry partnership
    programs with the aviation community that allow participants, such as
    airlines and pilots, to self-report violations of safety regulations and
    help identify safety deficiencies, and potentially mitigate or avoid fines
    or other legal action. For example, the Voluntary Disclosure Program
    encourages the self-reporting of manufacturing problems and safety
    incidents by

    FAA's Safety Oversight System Focuses on Risk Identification and
    Mitigation Through System Safety, Leveraging of Resources, and Enforcement
    of Safety Regulations, but Benefits Are Not Being Fully Realized

    participants that can include air carriers and repair stations. Appendix
    II describes the industry partnership programs.

    When violations of statutory and regulatory requirements are identified
    through inspections, through the partnership programs in certain cases, or
    through other methods, FAA has a variety of enforcement tools that it may
    use to respond to them, including administrative actions (such as issuing
    a warning notice or a letter of correction that includes the corrective
    actions the violator will take) and legal sanctions (such as levying a
    fine or suspending or revoking a pilot or other FAA-issued certificate).

    In recent reports, we found that FAA's safety oversight system has
    programs that focus on risk identification and mitigation through a system
    safety approach, the leveraging of resources, and enforcement of safety
    regulations, but that the benefits of these programs are not being fully
    realized. In our recent report on FAA's oversight of non-legacy airlines,
    we found that the focus on risk identification through the addition of SEP
    has many strengths and allows for enhancing the efficiency of FAA's
    oversight activities.2 Rather than relying on NPG's customary method of
    conducting a set number of inspections of an airline's operations, SEP
    emphasizes a system safety approach of using risk analysis techniques. SEP
    allows for the efficient use of inspection staff and resources by
    prioritizing workload based on areas of highest risk, and it includes a
    requirement that inspectors verify that corrective actions have occurred.
    For example, FAA has developed risk assessment worksheets for SEP that are
    aligned with key airline systems that guide inspectors through identifying
    and prioritizing risks. The worksheets guide inspectors to organize the
    results of their previous inspections and surveillance into a number of
    areas such as flight operations and personnel training in order to
    identify specific risks in each area and target the office's resources to
    mitigating those risks. The development of a system safety approach
    addresses a longstanding concern by us that FAA did not have a methodology
    for assessing airline safety risks so that it could target limited
    inspection resources to high-risk conditions.3 Another strength of SEP,
    consistent with findings in

    2GAO, Aviation Safety: System Safety Approach Needs Further Integration
    into FAA's Oversight of Airlines, GAO-05-726 (Washington, D.C.: Sept. 28,
    2005).

    3GAO, Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA
    in Identifying and Responding to Risks, GAO/RCED-98-6 (Washington, D.C.:
    Feb. 27, 1998); GAO, Aviation Safety: FAA Needs to More Aggressively
    Manage Its Inspection Program, GAO/T-RCED-92-25 (Washington, D.C.: Feb. 6,
    1992).

    our past reports, is that SEP relies on teams of inspectors, which are
    generally more effective than individual inspectors in their ability to
    collectively identify concerns.4

    However, the benefits of FAA's system safety approach for the inspection
    of non-legacy airlines could be enhanced by a more complete implementation
    of SEP and addressing other challenges. The inspection workload for
    non-legacy airlines is still heavily oriented to the NPG's nonrisk based
    activities. For example, as shown in table 1, from fiscal years 2002
    through 2004, 77 percent of inspection activities required for the top 25
    non-legacy airlines in terms of the number of enplanements were identified
    through NPG, and the remaining percentage of inspection activities were
    identified based on risk through SEP. Although inspectors can replace
    NPG-identified activities with SEP-identified activities that they deem
    constitute a greater safety risk, we found that FAA inspectors interpret
    agency emphasis on NPG as discouraging this practice. In order to ensure
    that all inspectors who oversee non-legacy airlines have a complete and
    timely understanding of the agency's policies relating to the inspection
    process, we recommended in September 2005 that FAA improve communication
    with and training of inspectors in this area.

    Table 1: SEP-and NPG-Initiated Required Inspections for the Top 25
    Non-legacy Airlines, Fiscal Years 2002-2004

                  Type of inspection      2002   2003     2004              Total
                       SEP-initiated     1,261  1,567     927         3,755 (23%)
                       NPG-initiated     5,470  3,623    3,338       12,431 (77%)
                               Total     6,731  5,190    4,265      16,186 (100%)

    Source: GAO analysis of FAA information.

    Another way that FAA attempts to enhance the efficiency of its oversight
    activities is through its designee programs. We reported that FAA
    maximizes its resources by allowing designees to perform about 90 percent
    of certification-related activities, thus allowing FAA to better
    concentrate its limited staff resources on the most safety-critical
    functions.5 For example, while designees conduct routine certification
    functions, such as approvals of aircraft technologies that the agency and

    4GAO/RCED-98-6; GAO/T-RCED-92-25.

    5GAO, Aviation Safety: FAA Needs to Strengthen the Management of Its
    Designee Programs, GAO-05-40 (Washington, D.C.: Oct. 8, 2004).

    designees have had previous experience with, FAA focuses on new and
    complex aircraft designs or design changes. In addition, the use of
    designees expands FAA's access to technical expertise within the aviation
    community. For the aviation industry, the designee programs enable
    individuals and organizations to obtain required FAA certifications-such
    as approvals of the design, production, and airworthiness of aircraft-in a
    timely manner, thus reducing delays and costs to the industry that might
    result from scheduling direct reviews by FAA staff. For example, officials
    from an aircraft manufacturer told us that the use of designees has added
    significantly to the company's ability to enhance and improve daily
    operations by decreasing certification delivery time and increasing the
    flexibility and utilization of company resources. In addition, designees
    are convenient to the aviation industry due to their wide dispersal
    throughout the United States.

    However, concerns about the consistency and adequacy of designee oversight
    that FAA field offices provide have been raised by experts and other
    individuals we interviewed. For example, designees and industry officials
    that we spoke with indicated that FAA's level of oversight and
    interpretation of rules differ among regions and among offices within a
    region, which limits FAA's assurance that designees' work is performed
    uniformly in accordance with FAA's standards and policy. Experts also
    ranked this issue as a top weakness.6 Table 2 shows the top five
    weaknesses identified by our experts. Experts also made a number of
    suggestions to strengthen the designee program, including clearly defining
    and following agency criteria for selecting designees and increasing
    penalties for designees found to violate standards or who do not exercise
    proper judgment. To improve management control of the designee programs,
    and thus increase assurance that designees meet FAA's performance
    standards, we recommended that FAA develop mechanisms to improve the
    compliance of FAA program and field offices with existing policies and
    incorporate, as appropriate, suggestions from our expert panel. In
    response to our recommendations, FAA is planning, among other

    6We identified 62 aviation experts with knowledge and expertise in FAA's
    designee programs, who participated on a Web-based panel that provided the
    group's views on the strengths and weaknesses of the designee programs and
    ways to improve the programs. The experts included designees, FAA
    inspectors and engineers, independent experts and university academics,
    and private sector and aviation industry associations. We obtained the
    experts' views by employing an iterative and controlled feedback process
    for obtaining individual views and then allowing each participant to
    respond to the entire panel's comments.

    things, to form a team to identify and share best practices for overseeing
    designee programs.

    Table 2: Experts' Ranking of Top 5 Oversight Weaknesses

                                    Ranking Weakness

    FAA offices level of oversight and interpretation of rules are
    inconsistent.

    Inactive, unqualified, or poor performing designees are not identified and
    removed expeditiously.

    It is difficult to terminate poor performing designees.

    Inadequate surveillance and oversight of designees.

            FAA has not made oversight of designees a high enough priority.

    Source: GAO analysis of expert panel information.

    Note: Rankings based on responses from 62 experts and the frequency of
    responses indicating a "great" or "very great" weakness.

    FAA also leverages its resources through its industry partnership
    programs. These partnership programs are designed to assist the agency in
    receiving safety information, including reports of safety violations.
    According to FAA officials, the Aviation Safety Action Program, Aviation
    Safety Reporting Program, and Voluntary Disclosure Reporting Program
    augment FAA's enforcement activities and allow FAA to be aware of many
    more safety incidents than are discovered during inspections and
    surveillance. In addition, the Flight Operational Quality Assurance
    Program provides safety information in the form of recorded flight data
    from participating airlines. FAA has established some management controls
    over its partnership programs, such as procedures to track actions taken
    to correct safety incidents reported under the programs, but the agency
    lacks management controls to measure and evaluate the performance of these
    programs, an issue that we will discuss later in the testimony.

    FAA's enforcement process, which is intended to ensure industry compliance
    with safety regulations, is another important element of its safety
    oversight system. FAA's policy for assessing legal sanctions against
    entities or individuals that do not comply with aviation safety
    regulations is intended to deter future violations. FAA has established
    some management controls over its enforcement efforts, with procedures
    that provide guidance on identifying regulated entities and individuals
    that are subject to inspections or surveillance actions, determining
    workload priorities on the basis of the timing and type of inspection to
    be performed, detecting violations of safety regulations, tracking the
    actions

    that are taken by the entities and individuals to correct the violations
    and achieve compliance with regulations, and imposing punitive sanctions
    or remedial conditions on the violators. These procedures provide FAA
    inspectors, managers, and attorneys with a process to handle violations of
    safety regulations that are found during routine inspections.

    However, we found that the effect of FAA's legal sanctions on deterrence
    is unclear, and that recommendations for sanctions are sometimes changed
    on the basis of factors that are not associated with the merits of the
    case. We found that from fiscal years 1993 through 2003, attorneys in
    FAA's Office of the Chief Counsel authorized a 52 percent reduction in the
    civil monetary penalties assessed from a total of $334 million to $162
    million. FAA officials told us that the agency sometimes reduces sanctions
    in order to prioritize attorneys' caseloads by closing the cases more
    quickly through negotiating a lower fine. Economic literature on
    deterrence suggests that although negative sanctions (such as fines and
    certificate suspensions) can deter violations, if the violator expects
    sanctions to be reduced, he or she may have less incentive to comply with
    regulations. In effect, the goal of preventing future violations is
    weakened when the penalties for present violations are lowered for reasons
    not related to the merits of the case. In addition, FAA lacks management
    controls to measure and evaluate its enforcement process, which we discuss
    later in this testimony.

    FAA's use of a risk-based system safety approach to inspections requires
    inspectors to apply data analysis and auditing skills to identify,
    analyze, assess, and control potential hazards and risks. Therefore, it is
    important that inspectors are well-trained in this approach and have
    sufficient knowledge of increasingly complex aircraft, aircraft parts, and
    systems to effectively identify safety risks. It is also important that
    FAA's large cadre of designees is well-trained in federal aviation
    regulations and FAA policies. FAA has made training an integral part of
    its safety inspection system and has established mandatory training
    requirements for its workforce as well as designees. FAA provides
    inspectors with extensive training in federal aviation regulations;
    inspection and investigative techniques; and technical skills, such as
    flight training for operations inspectors. The agency provides its
    designees with an initial indoctrination that covers federal regulations
    and agency policies, and refresher training every 2 to 3 years.

    We have reported that FAA has generally followed effective management
    practices for planning, developing, delivering, and assessing the impact
    of

    FAA Has Made Training an Integral Part of Its Safety Oversight System but
    Several Actions Could Improve Results

    its technical training7 for safety inspectors, although some practices
    have yet to be fully implemented.8 In its planning activities for
    training, FAA has linked technical training efforts to its goal of safer
    air travel and has identified technical proficiencies needed to improve
    safety inspectors' performance in meeting this goal. For example, FAA's
    Offices of Flight Standards and Aircraft Certification have identified
    gaps in several of the competencies required to conduct system safety
    inspections, including risk assessment, data analysis, systems thinking,
    and designee oversight. According to FAA, it is working to correct these
    gaps. We have also identified gaps in the training provided to inspectors
    in the Office of Flight Standards who oversee non-legacy airlines, and
    have recommended that FAA improve inspectors' training in areas such as
    system safety and risk management to ensure that these inspectors have a
    complete and timely understanding of FAA's inspection policies. We have
    identified similar competency gaps related to designee oversight. For
    example, FAA does not require refresher training concerning designee
    oversight, which increases the risk that staff do not retain the
    information, skills, and competencies required to perform their oversight
    responsibilities. We recommended that FAA provide additional training for
    staff who directly oversee designees.9 We did not identify any specific
    gaps in the competencies of designees.10 In prioritizing funding for
    course development activities, FAA does not explicitly consider which
    projects are most critical. Figure 3 describes the extent to which FAA
    follows effective management practices in planning training.

    7We define technical training as training in aviation technologies. FAA
    includes in its definition of technical training topics such as system
    safety and risk analysis, inspector job skills, data analysis, and
    training in software packages.

    8GAO, Aviation Safety: FAA Management Practices for Technical Training
    Mostly Effective; Further Actions Could Enhance Results, GAO-05-728
    (Washington, D.C.: Sept. 7, 2005). We compared FAA's management of its
    inspector technical training efforts with effective management practices
    outlined in GAO, Human Capital: A Guide for Assessing Strategic Training
    and Development Efforts in the Federal Government, GAO-04-546G
    (Washington, D.C.: Mar. 1, 2004).

    9GAO-05-40.

    10However, many experts on our panel indicated it was of high or highest
    importance to ensure standard training of designees within specific
    specialties to improve the consistency of their work, and to increase the
    number of subject-matter workshops for designees.

    Figure 3: Extent that FAA Follows Effective Management Practices in
    Planning Technical Training

    In developing its training curriculum for inspectors, FAA also for the
    most part follows effective management practices, such as developing
    courses that support changes in inspection procedures resulting from
    regulatory changes or agency initiatives. On the other hand, FAA develops
    technical courses on an ad hoc basis rather than as part of an overall
    curriculum for each inspector specialty-such as air carrier operations,
    maintenance, and cabin safety-because the agency has not systematically
    identified the technical skills and competencies each type of inspector
    needs to effectively perform inspections. Figure 4 describes the extent to
    which FAA follows effective management practices in developing training.

    Figure 4: Extent that FAA Follows Effective Management Practices in
    Developing Technical Training

    aThis management practice is not specifically identified in our assessment
    guide. However, a management approach that assesses training needs
    holistically rather than on a course-by-course basis can provide for a
    more systematic assessment of whether and how training will help meet
    organizational needs.

    In delivering training, FAA has also generally followed effective
    management practices. (See fig. 5.) For example, FAA has established clear
    accountability for ensuring that inspectors have access to technical
    training, developed a way for inspectors to choose courses that meet job
    needs and further professional development, and offers a wide array of
    technical and other courses. However, both FAA and its inspectors
    recognize the need for more timely selection of inspectors for technical
    training. In addition, FAA acknowledges the need to increase communication
    between inspectors and management with respect to the training program,
    especially to ensure that inspectors have bought into the system safety
    approach to inspections.

    Figure 5: Extent that FAA Follows Effective Management Practices in
    Delivering Technical Training

    FAA offers numerous technical courses from which inspectors can select to
    meet job needs. However, from our survey of FAA's inspectors, we estimate
    that only about half think that they have the technical knowledge needed
    for their jobs.11 FAA officials told us that inspectors' negative views
    stem from their wanting to acquire proficiencies that are not as crucial
    in a system safety environment. We also found a disparity between
    inspectors and FAA concerning the receipt of requested training. We
    estimated that 28 percent of inspectors believe that they get the
    technical training that they request. However, FAA's records show that FAA
    approves about 90 percent of these requests, and inspectors are making
    good progress in receiving training. Over half of the inspectors have
    completed at least 75 percent of technical training that FAA considers
    essential. FAA officials told us that inspectors' negative views on their
    technical knowledge and the training they have received stem from their
    not accepting FAA's move

    11Because of the statistical survey techniques we employed in surveying
    FAA's inspectors, we are 95 percent confident that the results we present
    are within 4.6 percentage points of the results that we would have
    obtained if we had surveyed all 3,000 inspectors. That is, we are 95
    percent confident that had we surveyed all inspectors, between 48 and 57
    percent of them would have told us that, to a great or very great extent,
    they have the technical knowledge to do their jobs. All percentage
    estimates from the survey have a margin of error of plus or minus 4.6
    percentage points or less, unless otherwise noted.

    to a system safety approach. That is, the inspectors are concerned about
    acquiring individual technical proficiency that is not as crucial in a
    system safety environment. Given that it has not completed assessing
    whether training for each inspector specialty meets performance
    requirements, FAA is not in a position to make definitive conclusions
    concerning the adequacy of inspector technical training.

    FAA also generally followed effective management practices in evaluating
    training. The agency requires that each training course receive a
    systematic evaluation every 3 years to determine if the course is up to
    date and relevant to inspectors' jobs, although training officials noted
    that many courses have yet to undergo such an evaluation. However, FAA
    collects limited information on the effectiveness of training, and its
    evaluations have not measured the impact of training on FAA's mission
    goals, such as reducing accidents. Training experts acknowledge that
    isolating performance improvements resulting from training programs is
    difficult for any organization. (See fig. 6.)

    Figure 6: Extent that FAA Follows Effective Management Practices in
    Evaluating Its Training Program

    While FAA follows many effective management practices in its training
    program, the agency also recognizes the need for improvements, including
    (1) systematically assessing inspectors' needs for technical and other
    training, (2) better timing of technical training so that inspectors
    receive it when it is needed to perform their jobs, and (3) better linking
    the training provided to achieving agency goals of improving aviation
    safety. FAA has begun to act in these areas, and we believe that if
    effectively implemented, the actions should improve the delivery of
    training and ultimately improve aviation safety. Therefore, it is
    important for FAA to follow through with its efforts. As a result, we
    recommended in September 2005, among other things, that in order to ensure
    that inspector technical training needs are identified and met in a timely
    manner, FAA systematically assess inspectors' technical training needs,
    better align the timeliness of training to when inspectors need the
    training to do their jobs, and gain inspectors' acceptance for changes
    made or planned to their training.

    It is important that both FAA's inspection workforce and FAA-certified
    aviation mechanics are knowledgeable about increasingly complex aircraft,
    aircraft parts, and systems. While we did not attempt to assess the
    technical proficiency that FAA's workforce requires and will require in
    the near future, FAA officials said that inspectors do not need a
    substantial amount of technical training courses because inspectors are
    hired with a high degree of technical knowledge of aircraft and aircraft
    systems. They further indicated that inspectors can sufficiently keep
    abreast of many of the changes in aviation technology through FAA and
    industry training courses and on-the-job training. However, in its
    certification program for aviation mechanics, we found that FAA standards
    for minimum requirements for aviation courses at FAA-approved aviation
    maintenance technician schools and its requirements for FAA-issued
    mechanics certificates do not keep abreast with the latest technologies.
    In 2003, we reported that those standards had not been updated in more
    than 50 years.12 We recommended that FAA review the curriculum and
    certification requirements and update both. FAA plans to make changes in
    the curriculum for FAA approved aviation maintenance technicians that
    reflect up-to-date aviation technologies and finalize and distribute a
    revised Advisory Circular in March 2006 that describes the curriculum
    changes. FAA then plans to allow the aviation industry time to implement

    12GAO, Aviation Safety: FAA Needs to Update the Curriculum and
    Certification Requirements for Aviation Mechanics, GAO-03-317 (Washington,
    D.C.: Mar. 6, 2003).

    FAA Has Evaluated Some Safety Programs, but the Lack of Evaluative Systems
    and Nationwide Data Impedes FAA's Ability to Continuously Monitor Its
    Safety Programs

    the recommended curriculum changes before changing the requirements for
    FAA-issued mechanics certificates.

    It is important for FAA to have effective evaluative processes and
    accurate nationwide data on its numerous safety oversight programs so that
    program managers and other officials have assurance that the safety
    programs are having their intended effect. Such processes and data are
    especially important because FAA's workforce is so dispersed
    worldwide-with thousands of staff working out of more than 100 local
    offices-and because FAA's use of a risk-based system safety approach
    represents a cultural shift from its traditional inspection program.
    Evaluation is important to understanding if the cultural shift has
    effectively occurred. Our most recent work has shown the lack of such
    processes and limitations with data for FAA's inspection programs for
    non-legacy airlines, designee programs, industry partnership programs, and
    enforcement program. In response to recommendations that we have made
    regarding these programs, some improvements are being made. On the
    positive side, as we mentioned earlier, our most recent work found that
    FAA generally follows effective management practices in evaluating
    individual technical training courses.

    FAA has not evaluated its inspection oversight programs for non-legacy
    airlines-which include SEP and NPG-to determine how the programs
    contribute to the agency's mission and overall safety goals, and its
    nationwide inspection database lacks important information that could help
    it perform such evaluations-such as whether risks identified through SEP
    have been mitigated. In addition, the agency does not have a process to
    examine the nationwide implications of or trends in the risks that
    inspectors have identified through their risk assessments-information it
    would need to proactively determine risk trends at the national level on a
    continuous basis. FAA's evaluation office instead conducts analyses of the
    types of inspections generated under SEP by airline and FAA region,
    according to FAA. We recommended that FAA develop a continuous evaluative
    process for activities under SEP and link SEP to the performance-related
    goals and measures developed by the agency, track performance toward these
    goals, and determine appropriate program changes. FAA is considering our
    recommendation, but its plan to place the remaining non-legacy airlines in
    the ATOS program by the end of fiscal year 2007 might make this
    recommendation unnecessary, according to the agency. Since FAA's past
    efforts to move airlines to ATOS have experienced delays, we believe that
    this recommendation is still valid.

    We also found that FAA lacked requirements or criteria for periodically
    evaluating its designee programs. In 2004, we reported that the agency had
    evaluated 6 of its 18 designee programs over the previous 7 years and had
    plans to evaluate 2 more, although it had no plans to evaluate the
    remaining 10 programs because of limited resources.13 FAA conducted these
    evaluations on an ad hoc basis usually at the request of headquarters
    directors or regional office managers. In addition, we found that FAA's
    oversight of designees is hampered, in part, by the limited information on
    designees' performance contained in the various designee databases.14
    These databases contain descriptive information on designees, such as
    their types of designations and status (i.e., active or terminated). More
    complete information would allow the agency to gain a comprehensive
    picture of whether staff are carrying out their responsibilities to
    oversee designees. To improve management control of the designee programs,
    and thus increase assurance that designees meet the agency's performance
    standards, we recommended that FAA establish a process to evaluate all
    designee programs and strengthen the effectiveness of its designee
    databases by improving the consistency and completeness of information in
    them. To address our recommendations, FAA expects to develop a plan to
    evaluate all designee programs on a recurring basis and intends to
    establish a team that will examine ways to improve automated information
    related to designees.

    In addition, we found that FAA does not evaluate the effects of its
    industry partnership and enforcement programs to determine if stated
    program goals, such as deterrence of future violations, are being
    achieved. For example, little is known about nationwide trends in the
    types of violations reported under the partnership programs or whether
    systemic, nationwide causes of those violations are identified and
    addressed. Furthermore, FAA's enforcement policy calls for inspectors and
    legal counsel staff to recommend or assess enforcement sanctions that
    would potentially deter future violations. However, without an evaluative
    process, it is not known whether the agency's practice of generally
    closing cases with

    13GAO-05-40.

    14These databases are Flight Standards Service's Program Tracking and
    Reporting Subsystem and National Vital Information Subsystem, Aircraft
    Certification Service's Designee Information Network, and Office of
    Aerospace Medicine's Airmen Medical Certification Information Subsystem.

    administrative actions rather than legal sanctions15 and at times reducing
    the amount of the fines, as mentioned earlier in this testimony, may
    weaken any deterrent effect that would be expected from sanctions.

    FAA's ability to evaluate the impact of its enforcement efforts is also
    hindered by the lack of useful nationwide data. FAA inspection offices
    maintain independent, site-specific databases because they do not find the
    nationwide enforcement database-the Enforcement Information System
    (EIS)-as useful as it could be because of missing or incomplete historical
    information about enforcement cases. As a result of incomplete data on
    individual cases, FAA inspectors lack the complete compliance history of
    violators when assessing sanctions. We recommended that FAA develop
    evaluative processes for its enforcement activities and partnership
    programs and use them to create performance goals, track performance
    towards those goals, and determine appropriate program changes. We also
    recommended that FAA take steps to improve the usefulness of the EIS
    database by enhancing the completeness of enforcement information. FAA
    expects to address some of these issues as it revises its enforcement
    policy, which is expected to be issued later in fiscal year 2006. In
    addition, FAA has established a database workgroup that is developing
    long-and short- term solutions to address the problems with EIS.

    Recommendations We Have Made to Improve FAA's Safety Oversight System

    In order to help FAA fully realize the benefits from its safety oversight
    system, we have made a number of recommendations to address weaknesses
    that we identified in our reviews. These recommendations have not been
    fully implemented, although in some cases FAA has taken steps towards
    addressing them. Evaluative processes and relevant data are particularly
    important as FAA works to change its culture by incorporating a system
    safety approach into its oversight, and we have recommended that FAA
    develop continuous evaluative processes for its oversight programs for
    non-legacy airlines, its designee programs, and its industry partnership
    and enforcement programs, and systematically assess inspectors' technical
    training needs. In addition, FAA's nationwide databases are in need of
    improvements in their comprehensiveness and ease of use. Without
    comprehensive nationwide data, FAA does not have the information needed to
    evaluate its safety programs and have assurance

    15We found that during fiscal years 1993 through 2003, FAA closed about 53
    percent of the nearly 200,000 enforcement actions with administrative
    actions (such as warning notices). About 28 percent of the actions were
    closed with legal sanctions (such as fines) and about 18 percent were
    closed with no enforcement action.

    that they are having the intended results. We have recommended that FAA
    improve the completeness of its designee and enforcement databases.
    Continuous improvements in these areas are critical to FAA's ability to
    have a robust "early warning system" and maintain one of the safest
    aviation systems in the world.

    Contacts and 	For further information on this testimony, please contact
    Dr. Gerald Dillingham at (202) 512-2834 or by email at
    dillinghamg@gao.gov.

    Acknowledgments 	Individuals making key contributions to this testimony
    include Brad Dubbs, Phillis Riley, Teresa Spisak, and Alwynne Wilbur.

    Appendix I: Description of FAA's Inspection Programs

    Table 3 describes the Federal Aviation Administration's (FAA) three
    inspection processes for overseeing airlines: Air Transportation Oversight
    System (ATOS), National Work Program Guidelines (NPG), and Surveillance
    and Evaluation Program (SEP). Many of the elements of ATOS, such as the
    use of data to identify risks and the development of surveillance plans by
    inspectors, are incorporated in the SEP process. The NPG process, in
    contrast, is not focused on the use of data and relies on an established
    set of inspections that are not risk based.

                    Table 3: Various Elements of ATOS, NPG, and SEP

    ATOS NPG SEP

    Description of  o  Focuses on safety vulnerabilities  o  Focuses on
    inspectors  o  Focuses on inspectors program rather than regulatory
    compliance completing a prescribed conducting a risk assessment

    o  Analysts and inspectors review airline number of inspection activities
    of various areas

    data to identify areas of safety risk  o  Primarily based on checking  o
    Inspectors review data to

    o 	Inspectors develop surveillance plans airline compliance with identify
    areas of safety risk and for each airline, based on data regulations use
    Flight Standards Safety

    analysis and assessment of risks,  o  Relies on inspectors' expertise
    Analysis Information Center and adjust the plans periodically to identify
    trends and risks and the Safety Performance based on inspection results
    Analysis System as analytical

    tools

    o 	Inspectors develop surveillance plans for each airline, based on data
    analysis and assessment of risks, and adjust plans periodically based on
    inspection results

    o 	Inspectors can also verify that planned NPG activities meet the
    surveillance needs for a particular year

    Type of commercial Legacy commercial airlines Non-legacy commercial
    airlines Non-legacy commercial airlines
    passenger airline
    inspected

    Frequency of Continuous safety        Periodic; regular     Periodic;
                        oversight              inspections are  inspections are
    inspections                     established annually by established during
                                                        an FAA      meetings held
                                       headquarters committee  at least twice a
                                                               year using risk-
                                                                 based criteria

    Approximate number           585               1,100b               1,100b
    of aviation safety
    inspectors
    conducting
    inspections a

    Number of                       13c                 99                  99
    commercial
    passenger airlines
    under the program b

    Source: GAO and FAA

    aAs of July 2005.

    bThere are a total of about 1,100 inspectors for both the NPG and SEP
    inspections.

    cFedEx and United Parcel Service, two cargo air carriers, are also in the
    ATOS program. This number reflects the recent merger of US Airways and
    America West, which were both in ATOS prior to the merger.

    Appendix II: Description of FAA's Partnership Programs

    Aviation Safety Action Program (ASAP)

    Year Established: 1997

    Participation: Participants include employees of air carriers and repair
    stations that have entered into a memorandum of understanding with the
    Federal Aviation Administration (FAA). The memoranda can cover employee
    groups, such as pilots, maintenance employees, dispatchers, or flight
    attendants. Each employee group is covered by a separate memorandum of
    understanding. As of June 2004, FAA had accepted 54 memoranda of
    understanding and received over 80,000 ASAP reports, which may or may not
    include safety violations, according to FAA officials.

    Purpose: ASAP seeks to improve aviation safety through the voluntary
    self-reporting of safety incidents under the procedures set forth in the
    memorandum of understanding. Under the program, FAA does not take
    enforcement action against employees who voluntarily self-reported safety
    violations for reports that are sole-source (the report is the only way
    FAA would have learned about the incident) and will pursue administrative
    action only for reports that are not sole-source. Incidents that involve
    alcohol, drugs, criminal activity, or an intentional disregard for safety
    are not eligible for self-reporting under ASAP.

    Process: Each memorandum of understanding is a voluntary partnership
    between FAA, the airline, and an employee group. Although employee groups
    are not always included, FAA encourages their participation. The
    memorandum of understanding ensures that employees who voluntarily
    disclose FAA safety violations in accordance with the procedures and
    guidelines of ASAP will receive administrative action or no action in lieu
    of legal enforcement action.

    Once a memorandum of understanding is approved, employees can begin
    reporting violations that fall under the agreement. When a violation
    occurs, an employee notifies the Event Review Committee, which includes
    representatives from FAA and the airline or the repair station and
    generally includes the appropriate employee association. The committee
    must be notified in writing within the time limit specified in the
    memorandum of understanding. The committee then determines whether to
    accept the report under the ASAP program. If the report is accepted (it
    meets the acceptance criteria in the memorandum and does not involve
    criminal activity, substance abuse, controlled substances, or alcohol),
    then the committee determines the action to take. That action may include
    remedial training or administrative action, but it will not include a
    legal sanction.

    Results: FAA does not know the overall program results because it does not
    have a national, systematic process in place to evaluate the overall
    success of ASAP. However, FAA cites examples that describe ASAP's
    contribution to enhanced aviation safety. These examples include
    identifying deficiencies in aircraft operations manuals, airport
    equipment, and runways. In July 2003, FAA's Compliance and Enforcement
    Review recommended that FAA evaluate the use and effectiveness of this
    program.

    Aviation Safety Reporting Program (ASRP)

    Year Established: 1975

    Participation: Participants are all users of the national airspace system,
    including air traffic controllers and employees of air carriers and repair
    stations.

    Purpose: The program is designed to improve aviation safety by offering
    limited immunity for individuals who voluntarily report safety incidents.
    ASRP was founded after TWA Flight 514 crashed on approach to landing in
    December 1974 after the crew misinterpreted information on the approach
    chart. This accident occurred only 6 weeks after another plane experienced
    the same error.

    Process: The National Aeronautics and Space Administration (NASA)
    administers this program. When a safety incident occurs, a person may
    submit a form and incident report to NASA. There are four types of forms
    that can be submitted to NASA: (1) Air Traffic Control, (2) General
    Reports (includes Pilots), (3) Flight Attendants, and (4) Maintenance
    Personnel.

    At least two aviation safety analysts read these forms and the incident
    reports that accompany them. The analysts at NASA screen the incident
    reports for urgent safety issues, which will be marked for immediate
    action to the appropriate FAA office or aviation authority. NASA analysts
    also edit the report's narrative to eliminate any identifying information.
    In addition, each report has a tear-off portion, which is separated and
    returned to the individual who reported the incident as a receipt of the
    incident report's acceptance into the ASRP. When a safety violation that
    has been previously reported under ASRP comes to the attention of FAA, the
    agency issues a legal sanction, which is then waived. Reports that would
    not be eligible to have a legal sanction waived include deliberate
    violations, violations involving a criminal offense, or accident; reports
    filed by participants who have committed a violation of federal aviation

    regulations or law within the last 5 years and reports filed later than 10
    days following an incident.

    Results: While FAA and NASA do not know the overall program results
    because they do not have a formal national evaluation program to measure
    the overall effectiveness of the program, the agencies widely disseminate
    information generated from the program to aircraft manufacturers and
    others. ASRP reports are compiled into a database known as the Aviation
    Safety Reporting System. When a potentially hazardous condition is
    reported, such as a defect in a navigational aid or a confusing procedure,
    NASA will send a safety alert to aircraft manufacturers, the FAA, airport
    representatives, and other aviation groups. The database is used for a
    monthly safety bulletin that includes excerpts from incident reports with
    supporting commentary by FAA safety experts. NASA officials estimate that
    the bulletin is read by over 150,000 people. In addition, individuals and
    organizations can request a search of the database for information on
    particular aircraft aviation safety subjects, including human performance
    errors and safety deficiencies. Further, NASA has used the database to
    analyze operational safety issues, such as general aviation incidents,
    pilot and controller communications, and runway incursions.

    Flight Operational Quality Assurance (FOQA)

    Year Established: 1995

    Participation: Participants include air carriers that equip their
    airplanes to record flight data. As of March 2004, 13 airlines had
    FAA-approved FOQA programs, and approximately 1,400 airplanes were
    equipped for the program.

    Purpose: FOQA is designed to enhance aviation safety through the analysis
    of digital flight data generated during routine flights.

    Process: Air carriers that participate in the program equip their aircraft
    with special acquisition devices or use the airplanes' flight data
    recorders to collect data and determine if the aircraft are deviating from
    standard procedures. These data include engine temperatures, descent rate,
    and deviations from the flight path. When the aircraft lands, data are
    transmitted from the aircraft to the airline's FOQA station, where they
    are analyzed for flight trends and possible safety problems.

    Once the data are transmitted to the FOQA ground station, the data are
    extracted and analyzed by software programs. The FOQA data are combined
    with data from maintenance databases, weather conditions, and

    other safety reporting systems, such as ASAP, in order to identify trends
    in flight operations. The analysis typically focuses on events that fall
    outside normal boundaries specified by the manufacturer's operational
    limitations and the air carrier's operational standards.

    FOQA data are collected and analyzed by individual air carriers. The data
    on safety trends are made available to FAA in an aggregated form with no
    identification of individual carriers. According to FAA officials, air
    carriers do not want to release this data to any outside party (including
    FAA) because of concerns that the data could then be publicly released.
    Air carriers pay for the special flight data recorders that can record
    FOQA data, which cost approximately $20,000 each. Although this can be an
    expensive investment for some air carriers, most newer aircraft models
    come with the data recorder built into the airplane. The International
    Civil Aviation Organization (ICAO) has recommended that airlines from
    member countries implement a FOQA program. FAA has notified ICAO that the
    program will remain voluntary in the United States.

    Results: Although FAA has no formal national evaluation program to measure
    the overall results or effectiveness of FOQA programs, FAA cites examples
    that describe FOQA's contribution to enhanced aviation safety. For
    example, one FOQA program highlighted a high rate of descent when
    airplanes land at a particular airport. On the basis of the information
    provided from FOQA, air traffic controllers at the airport were able to
    develop alternative approach procedures to decrease the rate of descent.

    Voluntary Disclosure Reporting Program (VDRP)

    Year Established: 1990

    Participation: Participants include air carriers, repair stations, and
    production approval holders.1

    Purpose: FAA initiated the program to promote aviation safety by
    encouraging the voluntary self-reporting of manufacturing, and quality
    control problems and safety incidents involving FAA requirements for
    maintenance, flight operations, drug and alcohol prevention programs, and
    security functions.

    1A production approval holder is an entity that holds a certificate,
    approval, or authorization from FAA to manufacture aircraft, aircraft
    engines, propellers, and related parts and articles.

    Process: Upon discovering a safety violation, participants can voluntarily
    disclose the violation to FAA within 24 hours. The initial notification
    should include a description of the violation, how and when the violation
    was discovered, and the corrective steps necessary to prevent repeat
    violations. Within 10 days of filing the initial notification to FAA, the
    entity is required to provide a written report that cites the regulations
    violated, describes how the violation was detected, provides an
    explanation of how the violation was inadvertent, and provides a
    description of the proposed comprehensive fix. FAA may pursue legal action
    if the participant discloses violations during, or in anticipation of, an
    FAA inspection.

    The violation must be reported immediately after being detected, must be
    inadvertent, must not indicate that a certificate holder is unqualified,
    and must include the immediate steps that were taken to terminate the
    apparent violation. If these conditions are met, and the FAA inspector has
    approved the comprehensive fix, then the FAA inspector will prepare a
    letter of correction and the case is considered closed with the
    possibility of being reopened if the comprehensive fix is not completed.

    Results: FAA does not know the overall program results because it does not
    have a process to measure the overall effectiveness of the program
    nationwide. A 2003 internal FAA report recommended that the agency
    evaluate the use and effectiveness of this program.

                        Selected GAO Reports on Aviation Safety

    Aviation Safety: System Safety Approach Needs Further Integration into
    FAA's Oversight of Airlines. GAO-05-726. Washington, D.C.: September 28,
    2005.

    Aviation Safety: FAA Management Practices for Technical Training Mostly
    Effective; Further Actions Could Enhance Results. GAO-05-728. Washington,
    D.C.: September 7, 2005.

    Aviation Safety: Oversight of Foreign Code-Share Safety Program Should Be
    Strengthened. GAO-05-930. Washington, D.C.: August 5, 2005.

    Aviation Safety: FAA Needs to Strengthen the Management of Its Designee
    Programs. GAO-05-40. Washington, D.C.: October 8, 2004.

    Aviation Safety: Better Management Controls are Needed to Improve FAA's
    Safety Enforcement and Compliance Efforts. GAO-04-646. Washington, D.C.:
    July 6, 2004.

    Aviation Safety: Information on FAA's Data on Operational Errors at Air
    Traffic Control Towers. GAO-03-1175R. Washington, D.C.: September 23,
    2003.

    Aviation Safety: FAA Needs to Update the Curriculum and Certification
    Requirements for Aviation Mechanics. GAO-03-317. Washington, D.C.: March
    6, 2003.

    Aviation Safety: FAA and DOD Response to Similar Safety Concerns.
    GAO-02-77. Washington. D.C.: January 22, 2002.

    Aviation Safety: Safer Skies Initiative Has Taken Initial Steps to Reduce
    Accident Rates by 2007. GAO/RCED-00-111. Washington, D.C.: June 30, 2000.

    Aviation Safety: FAA's New Inspection System Offers Promise, but Problems
    Need to Be Addressed. GAO/RCED-99-183. Washington, D.C.: June 28, 1999.

    Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA in
    Identifying and Responding to Risks. GAO/RCED-98-6. Washington, D.C.:
    February 27, 1998.

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    The Government Accountability Office, the audit, evaluation and

    GAO's Mission 	investigative arm of Congress, exists to support Congress
    in meeting its constitutional responsibilities and to help improve the
    performance and accountability of the federal government for the American
    people. GAO examines the use of public funds; evaluates federal programs
    and policies; and provides analyses, recommendations, and other assistance
    to help Congress make informed oversight, policy, and funding decisions.
    GAO's commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no
    costObtaining Copies of is through GAO's Web site (www.gao.gov). Each
    weekday, GAO postsGAO Reports and newly released reports, testimony, and
    correspondence on its Web site. To

    have GAO e-mail you a list of newly posted products every afternoon,
    goTestimony to www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone 	The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
    Orders for 100 or more copies mailed to a single address are discounted 25
    percent. Orders should be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    To Report Fraud, Contact:

    Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

    E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
    424-5454 or (202) 512-7470

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
    512-4400Congressional U.S. Government Accountability Office, 441 G Street
    NW, Room 7125 Relations Washington, D.C. 20548

    Public Affairs 	Paul Anderson, Managing Director, AndersonP1@gao.gov (202)
    512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

                               PRINTED ON RECYCLED PAPER
    *** End of document. ***