    National Transportation Safety Board: Preliminary Observations on
    the Value of Comprehensive Planning, and Greater Use of Leading
    Practices and the Training Academy (24-MAY-06, GAO-06-801T).

    The National Transportation Safety Board (NTSB) is a relatively
    small agency that plays a vital role in transportation safety and
    has a worldwide reputation for investigating accidents. With a
    staff of about 400 and a budget of $76.7 million in fiscal year
    2006, NTSB investigates all civil aviation accidents in the
    United States, and significant accidents in railroad, highway,
    marine, and pipeline; and issues safety recommendations to
    address issues identified during accident investigations. To
    support its mission, NTSB built a training academy, which opened
    in 2003 and provides training to NTSB investigators and others.
    It is important that NTSB use its resources efficiently to carry
    out its mission and maintain its preeminence. This testimony,
    based on ongoing work for this committee, addresses the extent to
    which NTSB follows leading practices in selected management
    areas, addresses challenges in completing accident investigations
    and closing safety recommendations, and generates sufficient
    revenues to cover costs at its academy.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-801T
        ACCNO:   A54732
      TITLE:     National Transportation Safety Board: Preliminary
    Observations on the Value of Comprehensive Planning, and Greater
    Use of Leading Practices and the Training Academy
         DATE:   05/24/2006
      SUBJECT:   Agency missions
    	     Best practices
    	     Best practices methodology
    	     Education or training costs
    	     Employee training
    	     Financial management
    	     Investigations by federal agencies
    	     Performance management
    	     Strategic planning
    	     Transportation safety

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-801T

                     United States Government Accountability Office

    Testimony

    GAO

        Before the Subcommittee on Aviation, Committee on Commerce, Science and
                              Transportation, U.S. Senate

    For Release on Delivery   NATIONAL TRANSPORTATION
    Expected at 9:00 a.m. EDT
    Wednesday, May 24, 2006
                                 SAFETY BOARD
                                 Preliminary Observations on the Value of
                                 Comprehensive Planning, and Greater Use of
                                 Leading Practices and the Training Academy
                                 Statement of Gerald L. Dillingham, Ph.D.
                                 Director, Physical Infrastructure Issues

    GAO-06-801T

    NATIONAL TRANSPORTATION SAFETY BOARD

    Preliminary Observations on the Value of Comprehensive Planning, and
    Greater Use of Leading Practices and the Training Academy

      What GAO Found

    NTSB has recently made progress in following leading management practices,
    but overall has a mixed record. For example, NTSB has improved its
    financial management by hiring a Chief Financial Officer and putting
    controls on its purchasing activities, which should address past problems
    with unapproved purchases. However, NTSB lacks a full cost accounting
    system, which would inform managers of the resources spent on individual
    investigations and provide data to balance office workload. NTSB has also
    begun to develop a performance management system that should eventually
    link each individual's performance to the agency's strategic goals and
    objectives. However, the performance management system will not be fully
    functional until NTSB has developed a strategic plan with results-oriented
    goals and objectives and specific strategies for achieving them, which are
    lacking in the current strategic plan. Other areas, such as human capital
    and communications, partially follow leading practices.

    While NTSB is accomplishing its accident investigation mission, it faces
    challenges that affect the efficiency of the report production and
    recommendation close out processes. NTSB routinely takes longer than 2
    years to complete major investigations. Several factors may affect the
    length of report production, including several revisions of draft reports
    through multiple layers of the organization. In addition, the processes
    for federal transportation agencies to implement NTSB's safety
    recommendations and for NTSB to change the status of recommendations are
    lengthy, paper-based, and labor intensive. While Department of
    Transportation officials have been working with NTSB to find acceptable
    means of implementing its recommendations, they cite the lengthy
    rule-making process as a challenge to speedy implementation.

    For fiscal years 2004 and 2005, NTSB's academy did not generate sufficient
    revenues to cover the costs of providing training. As a result, those
    portions of the academy's costs that were not covered by the revenues from
    tuition and other sources-approximately $6.3 million in fiscal year 2004
    and $3.9 million in fiscal year 2005-were offset by general appropriations
    to the agency. While NTSB has taken action to generate revenue from other
    sources, such as renting academy space for conferences, it does not have a
    marketing plan that seeks to optimize opportunities for additional
    revenues at the academy.

                     United States Government Accountability Office

    Mr. Chairman and Members of the Subcommittee:

    We are pleased to be here today to discuss the reauthorization of the
    National Transportation Safety Board (NTSB). NTSB is a relatively small
    agency that plays a vital role in transportation safety. With a staff of
    about 400 and a budget of $76.7 million in fiscal year 2006, NTSB is
    charged with investigating every civil aviation accident in the United
    States and significant accidents in the other modes-railroad, highway,
    marine, and pipeline-,determining the probable cause of these accidents
    and issuing recommendations to address safety issues identified during
    accident investigations. NTSB has gained a worldwide reputation as a
    preeminent agency in conducting transportation accident investigations.
    Since 1967, it has issued 1,340 major accident investigation reports, over
    130,000 brief accident reports, and made over 12,000 safety
    recommendations. To support its mission, NTSB built a training academy
    that opened in 2003 and provides training to NTSB investigators and other
    transportation safety professionals, including those from foreign
    countries. It is critical that the agency uses its resources in an
    efficient manner to carry out its safety mission and maintain its
    preeminent position. For this reason, you asked us to conduct a
    comprehensive review of NTSB's management functions such as strategic
    planning, human capital management, and mission-critical investigation
    activities. My testimony today is based on our ongoing work for you, and
    it addresses the extent to which NTSB is

    (1) following leading practices in selected management areas, (2)
    addressing challenges in completing accident investigations and closing
    safety recommendations, and (3) generating sufficient revenues to cover
    costs at its academy. We will be reporting additional results of our
    ongoing work to the committee later this year.

    In summary:

            * While NTSB has recently made progress in following leading
              management practices, its overall record is mixed. For example,
              NTSB generally follows leading practices in the area of financial
              management. Over the last several years, NTSB has hired a Chief
              Financial Officer and improved its financial management by putting
              controls on its purchasing activities, which should address past
              problems with unapproved purchases with government credit cards.
              However, NTSB lacks a full cost accounting system, which would
              inform managers of the resources spent on individual investigations
              and provide data to help assure balanced office workload. Other
              areas, such as performance management, human capital, and
              communications, partially follow leading practices. For example,
              NTSB has begun to develop a performance management system that
              should eventually link each
            * individual's performance to the agency's strategic goals and
              objectives. However, the performance management system will not be
              fully functional until NTSB has a strategic plan with
              results-oriented objectives and specific strategies for achieving
              them, which are lacking in the current strategic plan. In the area
              of human capital management, NTSB has recently developed a draft
              staffing plan that addresses the agency's skills and competencies
              needs and includes strategies to increase the number of
              investigators and thereby strengthen the agency's ability to carry
              out its transportation safety mission. However, the draft plan does
              not address organizational structure or the balance between
              supervisory and nonsupervisory positions. While NTSB has recently
              taken positive steps to improve communications from senior
              management to the staff-such as periodically sending e-mail to all
              staff to share information on new developments and policies-the
              agency does not regularly hold general staff meetings or undertake
              anonymous surveys to obtain employee feedback.
         o NTSB is accomplishing its accident investigation mission, but it faces
           challenges that affect the efficiency of the report production and
           recommendation close-out processes. In terms of accomplishing its
           mission, since its inception in 1966, NTSB has investigated over
           134,000 transportation accidents, and 82 percent of its
           recommendations have been implemented, or acceptable progress toward
           implementation has been made. However, investigations are
           often-sometimes necessarily- lengthy; NTSB routinely takes longer than
           2 years to investigate major accidents. Lengthy investigations,
           combined with lengthy processes for federal agencies to regulate and
           industries to implement NTSB's safety recommendations, can work
           against the goal of improving transportation safety. One factor that
           adds to the duration of investigations is that when new investigations
           are launched, inspectors are pulled from working on previous accidents
           to work on new ones. Other factors that may affect the duration of
           report production include the multiple revisions of draft
           investigation reports at different levels in the organizations and
           resource issues. NTSB has recently taken several actions that may help
           shorten report development time, such as reemphasizing its policy on
           holding report development meetings to obtain early buy-in on report
           messages and holding modal directors accountable for specific issuance
           dates. We also identified practices in certain offices, such as the
           use of a project manager or deputy investigator-in-charge to handle
           report production, which may improve the efficiency of report
           development if used by all modal offices as they all are similar in
           what they do. The processes for implementing NTSB's safety
           recommendations, and for NTSB to change the status of recommendations
           are also lengthy and labor intensive. As a result, unsafe conditions
           may continue to exist until federal transportation

    Page 2 GAO-06-801T

    agencies, and ultimately, transportation industries, fully implement the
    recommendations, and the extended period it takes to change the status of
    recommendations ties up NTSB's scarce resources. As of May 2006, 305 of
    the 852 open recommendations have been in open status for 5 years or more.
    While Department of Transportation (DOT) officials have been working with
    NTSB to find acceptable means of implementing its recommendations, they
    cite the lengthy rule-making process as a challenge to speedy
    implementation. In addition, the process that NTSB uses to change the
    status of safety recommendations is paper-based, labor intensive, and
    relies on a series of sequential reviews that can take months to complete.
    As a result, resources within NTSB are inefficiently used and DOT agency
    officials told us they remain unaware whether their response has been
    accepted or not accepted.

    o  For fiscal years 2004 and 2005, NTSB's academy did not generate
    sufficient revenues to cover the costs of providing training. As a result,
    those portions of the academy's costs that were not covered by the
    revenues from tuition and other sources-approximately $6.3 million in
    fiscal year 2004 and $3.9 million in fiscal year 2005-were offset by
    general appropriations to the agency. Although there is no statutory
    requirement that revenues from NTSB's academy generate sufficient revenues
    to cover the costs, NTSB was encouraged in the Senate report accompanying
    the Fiscal Year 2006 DOT Appropriations Act to be more aggressive in
    imposing and collecting fees to cover the costs. While NTSB has taken
    action to generate revenue from other sources, such as renting academy
    space for conferences, it does not have a business plan that seeks to
    optimize opportunities for additional revenues at the academy.
    Additionally, NTSB is missing opportunities to find other uses for academy
    space. For example, during fiscal year 2005, less than 10 percent of the
    total classroom space was used. About 14 percent of the academy students
    in fiscal year 2005 were NTSB employees. However, in 2006, NTSB employees
    are scheduled to take 97 percent of their requested training from sources
    other than the academy, such as DOT's Transportation Safety Institute. The
    academy is not utilized more by NTSB staff, in part, because the agency
    has not developed a core curriculum for its staff, which it could then
    offer at the academy. Furthermore, many academy courses are similar to
    those taught elsewhere, which may affect the agency's ability to attract
    non-NTSB students.

      Background

    NTSB was established in 1966 as an independent government agency located
    within the newly formed DOT.1 In 1974, Congress made NTSB completely
    separate from DOT.2 NTSB's principal responsibility is to promote
    transportation safety by investigating transportation accidents,
    determining the probable cause, and issuing recommendations to address
    safety issues identified during accident investigations. Unlike other
    transportation agencies, such as the Federal Aviation Administration
    (FAA), NTSB does not have the authority to promulgate regulations to
    promote safety, but makes recommendations in its accident reports and
    safety studies3 to other agencies that have such regulatory authority. The
    federal agencies that receive NTSB recommendations include the DOT's FAA,
    Federal Highway Administration (FHWA), Federal Motor Carrier Safety
    Administration (FMCSA), Federal Railroad Administration (FRA), Federal
    Transit Administration (FTA), National Highway Traffic Safety
    Administration (NHTSA), Pipeline and Hazardous Materials Safety
    Administration (PHMSA), and the U.S. Coast Guard. NTSB also makes
    recommendations to others, such as state transportation authorities and
    industries. As figure 1 indicates, NTSB has varying degrees of flexibility
    in its statutory mandate, as it pertains to initiating an investigation.
    By statute, NTSB has limited discretion in deciding which aviation
    accidents to investigate and the greatest amount of discretion to
    investigate highway accidents.

    1

    Department of Transportation Act, P.L. 89-670, Oct. 15, 1966.

    2

    Independent Safety Board Act, P.L. 93-633, Title III, 1974.

    3

    NTSB conducts safety studies as a result of identifying safety concerns
    rather than as a result of specific accidents.

    Page 4 GAO-06-801T

    Figure 1: NTSB's Investigative Policy by Mode

    NTSB is comprised of a five member board-a chairman, vice chairman, and
    three members-appointed by the President with the advice and consent of
    the Senate.4 The chairman is NTSB's chief executive and

    4

    Not more than three members may be appointed from the same political
    party. At least three members are appointed on the basis of technical
    qualification, professional standing, and demonstrated knowledge in
    accident reconstruction, safety engineering, human factors, transportation
    safety, or transportation regulation.

    Page 5 GAO-06-801T

    administrative officer. As of March 2006, the board was supported by a
    staff of 396, which includes 210 investigators assigned to four modal
    offices-aviation; highway; marine; and rail, pipeline, hazardous
    materials. (See fig. 2.) The agency is headquartered in Washington, D.C.,
    and maintains 10 field offices nationwide and a training academy in
    Ashburn, Virginia, in suburban Washington, D.C. In recent years, the
    agency has shrunk in size due to budget constraints, which it has largely
    dealt with by using attrition to downsize the staff. In 2003, NTSB had 438
    full time employees compared with the current level of 396. During the
    same period, the number of full-time investigators decreased from 234 to
    210. NTSB's modal offices vary in size, with the aviation office having
    125 employees; the rail, pipeline, and hazardous materials office having
    38; the highway office having 30; and the marine office having 16
    employees as of May 2006. An additional 42 employees work in the Office of
    Research and Engineering, which provides technical, laboratory,
    analytical, and engineering support for the modal investigation offices.
    For example, it is responsible for interpreting data recorders, creating
    accident computer simulations, and publishing general safety studies.
    NTSB's budget increased from $62.9 million in fiscal year 2001 to $76.7
    million in fiscal year 2006, or about 22 percent. After adjusting for
    inflation, this represents an increase of about 9 percent. The President
    has requested $79.6 million for NTSB in fiscal year 2007.

    Figure 2: NTSB's Organization

    Since 1966, NTSB has investigated over 124,000 aviation accidents and over
    10,000 surface transportation accidents. Figure 3 shows the total number
    of aviation investigations that NTSB has undertaken over the past 6 years
    and the degree to which NTSB was involved in the investigations. NTSB
    lacks the resources to conduct on-scene investigations of all aviation
    accidents. As a result, for general aviation accidents, NTSB delegates the
    gathering of on-scene information to FAA investigators, as allowed by
    statute.5 In these limited investigations, FAA sends the accident
    information to NTSB, and NTSB then determines a probable cause for the
    accident. In addition, NTSB participates in the investigations of foreign
    aviation accidents in conformance with Annex 13 of the International Civil
    Aeronautics Organization Treaty. These investigations involve a U.S.
    carrier or U.S.-built aircraft, or occur at the request of a foreign
    government. NTSB aviation investigators told us that there is often
    significant value in participating in such investigations; the findings
    often have safety implications for U.S. carriers, since most foreign
    airlines use

    5

    49 U.S.C Sec. 1132(c).

    U.S.-made aircraft, engines, and other parts and multiple foreign air
    carriers operate within the United States.

    Figure 3: NTSB Involvement in Aviation Accidents, Fiscal Years 2000-2005

    Note: Limited investigations are those in which NTSB delegates the
    gathering of on-scene information to FAA inspectors.

    Page 9 GAO-06-801T

      NTSB Has Made Recent Progress in Following Leading Management Practices, But
      Overall Record Remains Mixed

    Through our work government wide we have identified a number of key
    functional areas and leading practices in areas that are important for
    managing an agency. This testimony focuses on NTSB's performance in five
    key functional areas-strategic planning, performance management, human
    capital, financial management, and communications-and how NTSB's practices
    compare to leading practices in those areas. As illustrated in figure 4,
    NTSB generally is following leading practices in financial management,
    only minimally following leading practices in strategic planning, and has
    mixed results for the other functions. Much of NTSB's progress toward
    following leading practices is due to recent management initiatives. The
    report we will be issuing later this year will provide additional
    information on NTSB's performance relative to these five management
    functions, as well as information technology, acquisition management
    (including the agency's use of contracting), knowledge management, and
    capital decisionmaking.

    Figure 4: Extent to Which NTSB Is Following Leading Practices in Selected
    Management Areas

        NTSB's Strategic Plan Lacks Certain Performance-Based Elements and
        Performance Management Plans Closely Follow Leading Practices but are not
        Fully Functional

    The Congress and the President have encouraged better management of
    federal agencies by means such as results-oriented strategic planning, but
    NTSB's strategic plan generally does not follow performance-based
    practices. Without effective short- and long-term planning, federal
    agencies risk delivering programs and services that may or may not meet
    the nation's most critical needs. The Government Performance and Results
    Act of 1993 (GPRA)6 and guidance contained in the Office of Management and
    Budget's (OMB) Circular A-11, provide performance-based strategic planning
    guidelines. GPRA was intended to achieve several broad purposes, including
    improving federal program effectiveness, accountability, and service
    delivery, and enhancing congressional decision making by providing more
    objective information on program performance. GPRA requires federal
    agencies to develop strategic plans in which they define their missions,
    establish results-oriented goals, and identify the strategies that will be
    needed to achieve those goals. For instance, GPRA requires strategic plan
    updates at least every 3 years, and requires that agencies set objectives
    and goals that are specific outcomes that the organization wishes to
    accomplish (called outcome-related objectives).

    To its credit, in December 2005, NTSB issued a strategic plan for the
    years 2006 through 2010, which was the first time the agency had a
    strategic plan in 6 years. In developing that plan, senior agency
    officials told us that they modeled their plan on examples from other
    federal agencies with similar structure and mission, such as the Federal
    Communications Commission. We compared NTSB's strategic plan to selected
    elements required by GPRA. (See fig. 5.)

    6Pub. L. No. 103-62.

         Figure 5: Extent to Which NTSB's Strategic Plan Follows GPRA Elements

                          GPRA elements                     Follows GPRA elements
    Mission statement
    General goals and objectives
    Approaches or strategies to achieve goals and
    objectives
    Relationship between general goals and annual goals
    External factors
    Program evaluations
    Five year time frame
    Stakeholder involvement

    Follows GPRA elements

    Source: GAO's analysis of NTSB data.

    While NTSB's 5-year strategic plan has a mission statement, four general
    goals and related objectives, and mentions key factors, such as declining
    resources, that could affect the agency's ability to achieve those goals,
    the plan lacks a number of key elements-including information about the
    operational processes; skills and technology; and the human, capital, and
    information resources-required to meet the goals and objectives. In
    addition, the goals and objectives lack sufficient specificity to know
    whether they have been achieved. One goal states "NTSB will maintain its
    response capacity for investigation of accidents and increase its analysis
    of incidents." An objective of that goal is to "continuously assess the
    most robust and efficient approaches to accident investigation." Although
    such a goal is important for the safety of the transportation industry,
    this and the other three goals and related objectives are not measurable.
    As a result, it will be difficult for NTSB and others to determine if the
    goals have been achieved.

    In addition, the plan lacks specific strategies for achieving those goals.
    According to GPRA, the strategies should include a description of the
    operational processes, skills and technology, and the resources required
    to meet the goals and objectives. Since NTSB's strategic plan lacks such a
    description, it does not align staffing, training, or other human resource
    management to strategic goals. That is, the plan does not explicitly
    explain how NTSB will use its resources to meet its mission and goals.
    While the

    Page 12 GAO-06-801T

    plan explains that each program office has its own objectives linked to
    the agency's goals and objectives, the plan contains no information to
    understand how each office contributes to those goals and objectives. In
    addition, NTSB's strategic plan does not describe how the performance
    goals contained in the annual performance plan are related to the general
    goals and objectives in the strategic plan, as required by GPRA.

    GPRA also requires federal agencies to provide a description in their
    strategic plans of the program evaluations used in establishing or
    revising general goals and objectives and a schedule for future program
    evaluation. NTSB's strategic plan lacks this information. As a result of
    having no program evaluations, it is unclear how or whether NTSB reviews
    its efforts to identify strengths it can maximize and weaknesses it should
    address. In developing a strategic plan, GRPA requires agencies to consult
    with Congress and other stakeholders. We have previously reported that
    other stakeholders of federal agencies include state and local
    governments, other federal agencies, interest groups, and agency
    employees. NTSB's strategic plan does not mention consultation with any
    stakeholders in its development. Furthermore, board members and agency
    staff told us that they had no involvement in the development of the
    strategic plan. Some current and past board members additionally stated
    that they believed that their involvement would be beneficial in providing
    a strategic vision for the agency. NTSB's senior management told us they
    expect to revise the strategic plan in the near future and contacted us
    regarding assistance to develop a more comprehensive, results-oriented
    plan as part of this study.

    NTSB has begun to develop a performance management system that should
    eventually link each individual's performance throughout the agency to the
    agency's strategic goals and objectives. We have reported that performance
    management systems are crucial for agencies because if developed properly
    they allow employees to make meaningful contributions that directly
    contribute to agency goals.7 NTSB has developed a comprehensive
    performance management plan for Senior Executive Series (SES) employees
    that links individual performance to strategic goals. Furthermore, the
    plan states that NTSB will link performance management with the agency's
    results-oriented goals and set and communicate individual and
    organizational goals and expectations.

    GAO, Results Oriented Cultures: Creating a Clear Linkage between
    Individual Performance and Organizational Success, GAO-03-488 (Washington,
    D.C.: March 14, 2003).

    Page 13 GAO-06-801T

        NTSB's Staffing Plan Is a Step in the Right Direction, but the
        Organizational Structure Has Not Been Reviewed

    This plan establishes individual performance criteria and the appraisal
    process. The appraisal process defines performance standards and explains
    performance elements that determine individual ratings. Because NTSB
    recognizes in this plan the importance of aligning organizational
    performance with individual performance and contributions to the agency's
    mission, the performance management plan is a step in the right direction.

    Along with the SES plan, NTSB issued in August 2005 a performance plan for
    its overall workforce, which includes some elements of linking individual
    performance to organizational goals. However, without having
    results-oriented goals in the strategic plan itself, neither of the two
    performance management plans are fully functional. That is, until NTSB's
    goals are more fully articulated in the strategic plan, it will be
    impossible for staff to know whether their performance contributes to
    meeting those goals. As with the strategic plan, NTSB staff was not
    involved in the development of the performance plan, and there was no
    mechanism for employee feedback after the plan was initially developed.
    Employee involvement provides greater assurance that policies are accepted
    and implemented because employees had a stake in their development.

    NTSB developed a draft agencywide staffing plan in December 2005 that
    follows several leading practices but lacks a workforce deployment
    strategy that considers the organizational structure and its balance of
    supervisory and non-supervisory positions. Existing strategic workforce
    planning tools and models suggest that certain principles should be
    followed in strategic workforce planning, such as determining the agency's
    skills and competencies needs; involving stakeholders (e.g., management
    and employees) in the planning process; and developing succession plans to
    anticipate upcoming employee retirement and workforce shifts.8 Further, in
    workforce deployment, it is important to have human capital strategies to
    avoid excess organizational layers and to properly balance supervisory and
    nonsupervisory positions.9 NTSB's draft staffing plan addresses the
    agency's skills and competencies needs and includes strategies to deal
    with workforce shifts. For example, the staffing plan

    8

    GAO, A Model of Strategic Human Capital Management, GAO-02-373SP
    (Washington, D.C.: March. 15, 2002) and GAO, Human Capital: Key Principles
    for Effective Strategic Workforce Planning, GAO-04-39 (Washington, D.C.:
    December 11, 2003).

    9

    GAO, Executive Agency Management Diagnostic Survey (draft).

    Page 14 GAO-06-801T

    proposes to increase the number of investigative staff by 21, which will
    help with the agency's resource needs. In addition, while some
    stakeholders (i.e., managers) were involved in the planning process,
    employees were not included. As we mentioned previously in this testimony,
    employee input provides greater assurance that policies are accepted and
    implemented because employees have a stake in their development.

    To develop the staffing plan, each modal office director submitted to
    NTSB's Managing Director an ideal staff size for his office, including
    additional slots for investigators. The increase in investigative staff is
    consistent with requests by modal offices to enhance their ability to
    conduct their investigative mission. Managers told us that current
    staffing constraints inhibited their ability to conduct more accident
    investigations and indicated an increase in staff would be helpful. For
    example, directors of the highway and rail/pipeline offices told us they
    could not initiate investigations on more than two accidents at a time
    because they lacked sufficient investigative staff to do more.10 The modal
    office directors' request for staff resulted in a total agency allotment
    of 455 full time equivalents (FTEs) plus 20 co-op positions. The Managing
    Director reduced this number to 404, which corresponds to NTSB's current
    funding level of 395, allowing for attrition and turnover. The Managing
    Director's allocation resulted in a proposed increase of 21 investigators
    agencywide and a proposed reduction of certain staff positions to
    accommodate the increase in investigators. This increase in investigative
    staff is consistent with a recommendation by RAND Corporation, which
    evaluated NTSB's accident investigation process and workload in 1999.11 To
    help implement the realignment, senior managers told us that they would
    like to transition some existing administrative and support staff with
    appropriate background and training into investigator roles where
    possible. The draft plan set a target date of May 2006 to begin creating
    developmental opportunities for staff to transition to investigative roles
    and to develop reduction strategies for staff that fall outside the
    staffing plan.

    10

    Each investigative team initially consists of at least one
    investigator-in-charge and other technical support investigator positions
    based on the complexity of the accident.

    11

    RAND Institute for Civil Justice, Safety in the Skies: Personnel and
    Parties in NTSB Accident Investigations (Santa Monica, CA.: 2000).

    Page 15 GAO-06-801T

        NTSB Lacks a Strategic Approach to Training Staff

    Training is another key area of human capital management. It is important
    for agencies to develop a strategic approach to training its workforce,
    which involves establishing training priorities and leveraging investments
    in training to achieve agency results; identifying specific training
    initiatives that improve individual and agency performance; ensuring
    effective and efficient delivery of training opportunities in an
    environment that supports learning and change; and demonstrating how
    training efforts contribute to improved performance and results.12 NTSB
    has not developed a strategic training plan, nor has it identified the
    core competencies needed to support its mission and a curriculum to
    develop those competencies. As a result of not having a core curriculum
    that is linked in this manner, NTSB lacks assurance that the courses that
    staff take provide the technical knowledge and skills necessary for them
    to be competent for the type of work they perform.

        Financial Management Is Improved, but NTSB Lacks a Full Cost Accounting
        System

    Sound financial management is crucial for responsible stewardship of
    federal resources.13 In recent years, NTSB has made significant progress
    in improving its financial management. In March 2001, NTSB hired a Chief
    Financial Officer who has emphasized the importance of sound financial
    management based on best practices. Similar to private sector companies,
    government agencies are required to report their financial condition in
    publicly available financial statements. As a result of actions taken by
    NTSB, the agency received an unqualified or "clean" opinion from
    independent auditors on its financial statements for the fiscal years
    ending September 30 for the years 2003, 2004, and 2005. The audit report
    concluded that NTSB's financial statements presented fairly, in all
    material respects, the financial position, net cost, changes in net
    position, budgetary resources, and financing in conformity with generally
    accepted accounting principles for the three years. NTSB has also improved
    its purchasing and contracting activities after identifying problems in
    those areas in 1999. In 2001, DOT's Office of Inspector General (DOTIG)
    reviewed the agency's contracting and procurement activities and
    recommended that NTSB institute accountability and controls in its
    purchase card program as well as other purchasing activities. As a result

    12

    GAO, Human Capital: A Guide for Assessing Strategic Training and
    Development Efforts in the Federal Government, GAO-04-546G (Washington,
    D.C.: March 1, 2004).

    13

    GAO, Executive Guide: Creating Value through World-class Financial
    Management, GAO/AIMD-00-134 (Washington, D.C.: April 2000).

    Page 16 GAO-06-801T

    of this and another DOTIG audit,14 NTSB has taken a number of initiatives
    to improve its purchasing and contracting activities. For example, NTSB
    restructured its purchase card system and guidelines to address problems,
    such as unrestrained and unapproved purchases on government credit cards.
    NTSB hired a manager of the contracting function to manage the agency's
    acquisition function and implement the DOTIG recommendations. In our full
    report, we will analyze some of these initiatives in more detail.

    In 2000, RAND recommended that NTSB develop systems that would allow the
    agency to better manage its resources by permitting full-cost accounting15
    of all agency activities.16 To accomplish this, RAND recommended putting
    in place a timekeeping system, in which individual project numbers were
    assigned to each investigation and support activities such as training.
    With this information, project managers could better understand how staff
    resources were utilized and project workload could be actively monitored
    by the Managing Director. NTSB has begun to implement this recommendation
    by upgrading a software system in November 2005 that tracks employee
    annual leave and sick leave. However, the system is not being fully
    utilized to track the number of hours staff spend on each investigation.
    Also, this system is not used to track time staff spend in training or at
    conferences. As a result, RAND's previous conclusion that "NTSB managers
    have little information they can use to plan the utilization of staff
    resources or manage staff workloads properly" remains current.

    14DOTIG, Audit of the Purchase Card Program, FI-2005-072 (Washington,
    D.C.: Aug. 23, 2005) and Report on Financial Management Practices and
    Internal Controls, FI-2003-004 (Washington, D.C.: Dec. 11, 2002).

    15

    Cost accounting involves the accumulation and analysis of financial and
    nonfinancial data, resulting in the allocation of costs to organizational
    pursuits such as performance goals, programs, activities, and outputs.
    Nonfinancial data measure the occurrences of activities and can include,
    for example, the number of hours worked.

    16

    RAND Institute for Civil Justice, 2000.

        Communications from Senior Management to Staff have Increased and
        Communications Among Offices is Generally In Place, but Upwards
        Communications Mechanisms Are Lacking

    We have identified useful practices related to managing employees that
    include seeking and monitoring employee attitudes, encouraging two-way
    communication between employees and management, and incorporating employee
    feedback into new policies and procedures.17 In response to issues raised
    by NTSB employees in a governmentwide survey conducted by OPM in 2004,
    NTSB's senior management made changes to improve the way it is
    communicating information to staff. For example, the Managing Director
    periodically sends "management advisory" e-mail to all staff that share
    information such as policy changes or new developments at the agency.
    However, we found no formal processes that encouraged two-way
    communication, such as town hall meetings, regular staff meetings, or
    anonymous employee surveys; or incorporated employee feedback into
    policy-making.

    The 23 investigators and writer editors with whom we spoke had mixed views
    on the effectiveness of communications within the agency.18 The four
    investigators from one modal office that we spoke with told us that they
    are pleased to now hear about policy changes at the agency, but said that
    there is too much reliance on the Internet for these communications. They
    also told us that although they believe the increased communications are
    positive, they found it difficult to find the time to read the material
    and still conduct their regular investigative duties. The four
    investigators that we spoke with from another modal office agreed that
    staff meetings occur infrequently and that they do not receive information
    on new policies from their managers. Further, they said that new policies
    or agency issues are not discussed with staff prior to issuance, and there
    was no formal mechanism to provide feedback during the policies'
    development. In the past, regular formal meetings occurred between union
    leadership and senior NTSB management, which allowed for such input, but
    that practice ceased. Although formal communication processes from the
    staff level to management are lacking, informal e-mail communications do
    take place occasionally between staff and senior management.

    17

    GAO, Results-Oriented Cultures: Implementation Steps to Assist Mergers and
    Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2,
    2003).

    18

    We randomly selected 15 investigators and 8 writer editors evenly across
    the 4 modal offices and interviewed them to obtain their views on NTSB's
    processes. The views represent the particular individuals and are not
    representative of all NTSB investigators and writer editors.

    Page 18 GAO-06-801T

    Communication and collaboration across offices at all levels can improve
    an agency's ability to carry out its mission by providing opportunities to
    share best practices and helping to ensure that any needed input is
    provided in a timely manner. We found that communication and collaboration
    between the Research and Engineering office and the modal offices appears
    to be regular. This is shown by the inclusion of Research and Engineering
    staff as core members of major investigative teams. Also, our review of
    workload in the Research and Engineering office shows a large number of
    projects that support all modes, and a Research and Engineering manager
    told us that his office frequently interacts with investigative staff.

    In contrast, NTSB lacks processes that would allow investigators and
    writer editors to communicate across the modal offices regarding the
    investigative process and other issues, according to staff we spoke to.
    The four investigators that we spoke with from one modal office told us
    that they are isolated from the rest of the agency and that lessons
    learned are not shared across offices. The investigators from another
    modal office told us that they are on permanent teams that share the same
    priorities in completing accident analysis, which enhances communication
    and teamwork in the office. In addition, in previous years, all writer
    editors were located in one group and reported directly to the Managing
    Director. Now, each modal office has its own staff of writers and editors.
    While they have retained personal working relationships from when they
    were located in the same office, four of the eight writer editors we spoke
    with said that they no longer share information with each other
    regularly.19 As a result, efficiencies and lessons learned that
    investigators and writer editor staff in one office might develop might
    not be shared with other offices. However, NTSB officials pointed out that
    every 6 months writer editors have the opportunity to meet with the
    publications specialist for training and to exchange information.

    The writer editors held a conference in February 2004.

      NTSB Is Accomplishing Its Accident Investigation Mission, but Opportunities
      Exist to Gain Efficiencies

    While NTSB is accomplishing its accident investigation mission, it faces
    challenges that affect the efficiency of the report production and
    recommendation close-out processes. In terms of accomplishing its mission,
    since its inception, NTSB has investigated over 134,000 transportation
    accidents. Eighty-two percent of its recommendations have been "accepted,"
    a term NTSB uses to include recommendations that recipients have said they
    would implement as well as those that have already been implemented.
    Figure 6 shows that highway recommendations have the highest acceptance
    rate and marine recommendations have the lowest.

    Figure 6: Recommendations and Acceptance Rates of Recommendations by Mode,
                                     1967-May 2006

        NTSB Investigations Are Often Lengthy, in Part Because Investigators Must
        Launch New Investigations Before Completing Ongoing Investigations

    Investigations have four phases-the "launch," fact finding, analysis, and
    report production. After a report is issued and recommendations made, the
    progress of implementing the recommendations is tracked during a fifth
    close-out phase. Figure 7 describes these phases.

    Figure 7: Components of an NTSB Investigation and Recommendation Close Out

    a

    IIC is the "investigator in charge."

    b

    A group chairman is a technical specialist who is responsible for
    developing the facts and analysis for a particular area of an
    investigation.

    Investigations are often lengthy and sometimes necessarily so. NTSB
    routinely takes longer than 2 years to complete major aviation
    investigations. For example, the total time to complete major aviation
    investigations has increased from an average of about 1.25 years in 1996
    to an average of almost 3.5 years in 2006. (See fig. 8.) In 2004, NTSB
    contracted with Booz Allen Hamilton to examine and make recommendations to
    improve the report development process and the recommendation close-out
    process. Booz Allen Hamilton20 reported that the average time to complete
    major investigations across all the modes was either 1.8 months or 1.9
    months for 4 out of 5 years.21 Lengthy investigations, combined with
    lengthy processes for federal agencies to develop regulations based on
    those recommendations and industries to implement the recommendations can
    work against the goal of improving transportation safety.

         Figure 8: Average Duration of Major Aviation Investigations, 1996-2006

    Note: Several complex, lengthy investigations were completed in 2000,
    including the crash of TWA flight 800, which took over 4 years to
    complete.

    20 Booz Allen Hamilton, NTSB Organizational Process and Efficiency Study
    (Washington, D.C.: Aug. 12, 2004).

    21

    In fiscal years 1999 and 2002, Booz Allen Hamilton found that the average
    time to complete a major investigation was 1.8 years; in fiscal years 2001
    and 2003, the average time was 1.9 years; in fiscal year 2000 the average
    was 2.4, mainly due to several lengthy aviation investigations that took
    over 4 years to complete.

    Page 22 GAO-06-801T

    One factor that adds to the duration of investigations is that when new
    investigations are launched, investigators are pulled from working on
    previous accidents to work on new ones. For example, when a major
    commercial aviation accident occurs, an NTSB "go team" is dispatched from
    Washington D.C., usually within hours of notification of the accident. In
    such cases, the team members must leave the investigations they had been
    working on to begin fact-finding on the new accident. In the cases of rail
    and highway accidents, NTSB investigators must also arrive quickly on
    scene to gather information because the accident scenes will be cleared
    quickly so that traffic can resume. The manager of one department told us
    that all of his ongoing reports would be delayed by 2 months if a sudden
    launch were to occur. The number of major investigations that are ongoing
    for each mode is shown in figure 9.

     Figure 9: Number of Ongoing Major Investigations by Mode, As of February 2006

    Note: This table does not include limited aviation investigations, in
    which FAA has the primary factfinding role.

    Page 23 GAO-06-801T

        Writing and Report Production Is a Bottleneck in the Process

    Another reason for the expansive time frame for accident investigations is
    that reports receive multiple revisions at different levels in the
    organization, including the office directors and the Managing Director's
    office, prior to going to the board members for final voting and approval
    of the draft report. An investigation report typically goes through the
    following reviews: the modal office, the Office of Research and
    Engineering, the Executive Secretariat, the Office of Safety
    Recommendations, the Office of General Counsel, the deputy managing
    director, the Managing Director's office, and each board member and the
    Chairman. For any review, there may be multiple iterations. Eleven
    investigators and 6 writer editors told us that the review process often
    results in improved clarity for report recommendations.22 However,
    investigators and writer editors also told us that they believe the levels
    of management review and approval for written products are excessive. All
    eight writer editors agreed that the reviews by the Executive
    Secretariat's office, which services a quality assurance function, was a
    bottleneck for getting products approved. They told us that it is common
    for correspondence and other products to be delayed in this office for 1
    week or more, which they viewed as excessive. While it may be a reasonable
    expectation for short products, such as correspondence, to be reviewed in
    less than a week, that expectation may not be reasonable for reports. Booz
    Allen Hamilton confirmed multiple iterations of review as the draft was
    routed through numerous offices. On average Booz Allen Hamilton found that
    there were 7 levels of reviews within a given modal office that resulted
    in an average of 28 separate reviews. A senior NTSB official stated that
    the many levels of review were needed to get the appropriate perspectives
    from relevant offices that had been involved in report development, such
    as the Research and Engineering Office and Safety Recommendation Office.
    The official also noted that the process can be streamlined on a
    case-by-case basis in which the usual process of sequential reviews is
    replaced with concurrent reviews. The NTSB official told us that there are
    no explicit criteria for determining when the streamlined process could be
    used.

    NTSB staff with whom we spoke reported that resource issues contributed to
    other bottlenecks. For example, four writer editors pointed out that NTSB
    has only one final layout and typesetting person. As of May 2006, the
    final layout process had a backlog of approximately 10 reports that have

    22

    Booz Allen Hamilton, however, found that the logic and rationale for
    changes made during the review process were not transparent.

    Page 24 GAO-06-801T

        Certain Agency Practices May Help Shorten Report Development

    been approved for issuance at board meetings but have not yet been
    published. NTSB adopts about 2 reports a month and issues on average 4
    reports a month. In addition, some investigators have the perception that
    the workload of writer editors is another bottleneck. For example, one
    investigator told us that he submitted draft reports to the senior writer
    editor in September 2005, and as of April 2006, no additional writing had
    been done on his project. Writer editors from each modal office told us
    they typically worked on five or more products at one time.

    NTSB has recently taken several actions that, along with potentially
    better practices in one modal office, may help shorten report development
    time. First, in response to a recommendation by Booz Allen Hamilton to
    gain management's buy-in to the report message before writing the report
    and thereby reduce the number of review iterations, NTSB management has
    reemphasized its policy for report development meetings. NTSB has a
    long-standing order that calls for holding message development meetings
    with internal stakeholders who will be reviewing the report prior to
    report writing. According to a senior NTSB official, however, the agency
    had stopped following that policy before Booz Allen Hamilton conducted its
    study in 2004. The official further stated that subsequent to that
    recommendation, NTSB's managing director sent a memorandum reminding staff
    to follow the policy. While NTSB has no data on whether the message
    development meetings are actually taking place, officials told us that the
    managing director's recent emphasis on these meetings was resulting in
    more of them occurring than in previous years.

    Second, since the spring of 2005, NTSB has initiated production meetings
    with senior management with the goal of reducing the duration of
    investigations. These meetings occur every 2 weeks and focus on report
    development and production. NTSB modal directors are held accountable for
    a specific issuance date within a six month planning window prior to
    issuing a report. During the biweekly meetings, the directors discuss with
    NTSB's Managing Director and senior executives their progress and
    commitments to complete the investigations. The meetings result in a
    production schedule that is available for subsequent review. The modal
    directors stated that they believe the new system is effective in reducing
    the duration of investigations; however because these meetings began so
    recently, it is too early to evaluate their effectiveness.

    Third, the highway office-which has the swiftest rate of accident
    investigation completion-uses a concept called a "project manager," who
    serves as a supervisory writer editor and interface between the

    Page 25 GAO-06-801T

        Safety Recommendations Close-out Process Is Time Consuming for Several
        Reasons

    investigative staff and the writer editor staff. As a result, the project
    manager assumes some of the report development roles typically supported
    by the investigators-in-charge. In comparison, investigators-in-charge in
    the marine and rail, pipeline, and hazardous materials offices submit a
    draft report to the writer editor, who then edits and sometimes
    substantially rewrites the report. In aviation, investigators-in-charge do
    not write reports, but rather writer editors develop the final report from
    interim technical reports drafted by specialists on the team. Booz Allen
    Hamilton recommended that all modes use a project manager or deputy
    investigator-in-charge so that the expertise of staff can be used more
    fully. In addition, such a practice might alleviate some of the workload
    issues that writer editors face as they complete multiple reports. NTSB
    managers told us that they agree with this recommendation, but they have
    not implemented it or developed any milestones for implementation.

    Fourth, the highway safety office uses an incentive system for performance
    on developing reports. Booz Allen Hamilton reported the highway safety
    office rewards staff with a cash bonus for meeting key deadlines for
    producing accident reports. Again, the study recommended that the highway
    program be used as a model for the other modal offices. The study further
    recommended that the incentive program be slightly modified so that the
    incentives are based on delivering reports before deadlines, rather than
    meeting deadlines. In that way, the average time standard would be
    tightened and the overall report development time would be shortened.
    According to NTSB officials, they are currently examining how to implement
    improved awards and incentive programs that will result in improved
    quality and timeliness of report products.

    The processes for federal transportation agencies to implement NTSB's
    safety recommendations, and for NTSB to change the status of
    recommendations it has made, are also lengthy because of complex processes
    involving many players. As of May 2006, 305 of NTSB's 852 open
    recommendations had been open for 5 years or more. Lengthy processes for
    federal agencies to develop regulations to implement NTSB's safety
    recommendations and industries to comply can work against the goal of
    quickly improving transportation safety. In addition, the lengthy,
    paperbased process for changing the status of recommendations ties up
    NTSB's scarce resources.

    The length of time that NTSB recommendations remain open is due, in part,
    to challenges faced by federal transportation agencies in implementing
    those recommendations, particularly those that require

    Page 26 GAO-06-801T

    changes to federal regulations, which take many years to complete. DOT
    modal officials with whom we spoke cited a lengthy rule-making process,
    which includes budgeting and allocating resources to develop the proposed
    regulation, drafting and receiving comments on proposed rules, and waiting
    for the industry's subsequent response to implement the final rule. For
    example, TWA flight 800 crashed off Long Island in July 1996; NTSB issued
    safety recommendations pertaining to explosive fuel tanks in December
    1996. NTSB adopted the accident report with further recommendations to FAA
    to reduce flammable vapors in aircraft fuel tanks in 2000; FAA issued a
    notice of proposed rule to address this recommendation in November 2005;
    the comment period for the notice ended on March 23, 2006. Thus, 10 years
    after the crash, the final rule has not been issued. Federal
    transportation officials also said the failure to satisfy a cost-benefit
    analysis might impede the implementation of NTSB recommendations. Although
    NTSB is required to only consider the safety implications of its
    recommendations and not consider the cost factors, if a proposed
    regulation is not cost beneficial, it cannot be approved by OMB.

    Federal officials with whom we spoke at DOT, which receives the bulk of
    NTSB recommendations, indicated that they have been working with NTSB to
    find acceptable means of implementing recommendations. The
    process-recently called Safety With a Team-is designed for NTSB and
    federal agencies to work in cooperation to address open recommendations
    and implement needed safety improvements. NTSB and DOT officials told us
    that this process contributed to the closing of many recommendations.
    However, the process is not used with the Coast Guard, which has the
    lowest rate-74 percent-for accepting NTSB recommendations among the modes,
    as mentioned previously. According to a Coast Guard official we spoke
    with, the Coast Guard believes that it has an acceptable rate for closing
    NTSB recommendations and that it does not intend to act on recommendations
    that it deemed unnecessary.

    NTSB recognizes that open recommendations can have serious safety
    implications for the transportation industry. To spur implementation, the
    agency also publishes a "most wanted" list of what it considers the most
    serious safety concerns. For example, in 2000 NTSB added to its most
    wanted list the need to improve the safety of motor carrier operations.
    NTSB recommended that FMCSA prevent motor carriers from operating if they
    put vehicles with mechanical problems on the road or unqualified drivers
    behind the wheel. As recently as May 2006, NTSB issued an additional
    recommendation that FMCSA "establish a program to verify that motor
    carriers have ceased operations after the effective date of revocation of
    operating authority."

    Page 27 GAO-06-801T

    The process that NTSB uses to change the status of or close out safety
    recommendations is paper-based, labor intensive, and relies on a series of
    sequential reviews; this process can take between 6 and 12 weeks. As a
    result, NTSB is delayed in communicating with agencies on whether NTSB
    considers the actions that have been taken to address the recommendation
    are sufficient to accept the recommendation. Consequently, agencies remain
    unaware that their response has been accepted or not accepted. And in the
    case of DOT, this lack of information affects its ability to accurately
    report annually to Congress on the status of implementing NTSB's
    recommendations in all its modal administrations.23

    The process of closing recommendations is managed by NTSB's Safety
    Recommendation Office, which has responsibility for maintaining a
    recommendations database and administering the paper flow to change the
    status of recommendations. Adding complexity to the process-which NTSB
    calls the "mail control process"-is the fact that there are 12 separate
    categories of recommendations status. The 12 categories are listed in
    figure 10, which also shows the percentage of recommendations in each
    category as of May 1, 2006.

    49 USC Sec. 1135(d). NTSB pointed out that for those recommendations on
    the Most Wanted List, it specifically updates the list each November to
    ensure sufficient time for DOT to file its annual report to Congress.

    Page 28 GAO-06-801T

    Note: NTSB issued 12,471 recommendations from 1967 to May 1, 2006.

    The process begins when NTSB receives documentation from the
    recommendation recipient that would change the recommendation's status.
    The Safety Recommendation Office generates paper folders and supervises a
    process that is summarized in figure 11. This process involves multiple,
    sequential approvals starting from the Safety Recommendation Office, to
    the modal offices and Research and Engineering Office, to the Managing
    Director's office, to the board members for final approval. Since none of
    these reviews happen concurrently, some 150 folders are in process at any
    given time, according to the director of the Safety Recommendations
    Office. There are no electronic communications or

    Page 29 GAO-06-801T

    approvals throughout the process. In its study of NTSB, Booz Allen
    Hamilton identified this as an inefficient process. Officials at NTSB
    agree that efficiencies could be gained in this process and are
    considering eventually computerizing a number of processes such as this
    one. The agency expects to develop such plans after hiring a chief
    information officer later this year.

      NTSB's Academy Does Not Generate Sufficient Revenues to Cover Costs and Is Not
      Fully Utilized

    Although there is no statutory requirement that revenues from NTSB's
    academy would generate sufficient revenues to cover the costs, in July
    2005, NTSB was encouraged in the Senate report accompanying the Fiscal
    Year 2006 DOT Appropriations Act to be more aggressive in imposing and
    collecting fees to cover the costs.24 The academy generates revenues
    through tuition fees, space rental to other agencies for events such as
    conferences, and contracts with federal agencies that would allow them to
    use academy space for "continuity of operations" in emergency situations.
    To the extent that NTSB maximizes the use of the academy, it can produce
    additional revenues that may help cover costs.

    24

    Senate Report 109-109 accompanying P.L. 109-115, the Transportation,
    Treasury, the Judiciary, Housing and Urban Development, and Related
    Agencies Appropriations Act of 2006.

    Page 30 GAO-06-801T

        Academy Costs Have Exceeded Revenues

    For the first 2 full years of operation, fiscal years 2004 and 2005,
    NTSB's academy did not generate sufficient revenues to cover the costs of
    providing training, as shown in table 1. As a result, those portions of
    the academy's costs that were not covered by the revenues from tuition and
    other sources-approximately $6.3 million in fiscal year 2004 and $3.9
    million in fiscal year 2005-were offset by general appropriations to the
    agency. The salaries and other personnel related expenses associated with
    NTSB investigators and managers teaching at the academy, which would be
    appropriate to include in academy costs, are not included in table 1
    because NTSB told us that it does not choose to account for expenses in
    that manner. In addition, NTSB lacks a full cost-accounting system that
    would facilitate doing so. The table shows expenses directly associated
    with the academy and does not include an allocation of agency wide
    supporting services, such as the Managing Director's office, information
    technology, human resources, and legal support. Some of the expenses
    during these 2 years were one-time expenses-such as over $125,000 for
    furniture and equipment (included in table 1 as office supplies for fiscal
    year 2005) and $499,000 to move the wreckage of the TWA flight 800
    airplane from storage near the crash site in New York to the academy
    (included in the table as miscellaneous government contract services in
    fiscal year 2004). Space rental is a fixed annual expense of about $2.5
    million. When that fixed expense is excluded from academy expenses, the
    remaining operating expenses exceeded revenues by about $3.7 million in
    fiscal year 2004 and about $1.4 million the subsequent year.

    Table 1: Direct Expenses and Revenues for the NTSB Academy, Fiscal Years
    2004 and 2005 (unaudited)

    Percentage FY 2004 FY 2005 difference

                       Personnel related $1,011,716 $978,591 -3%

                              Travel $24,428 $56,912 133%

                        Space rentala $2,521,440 $2,500,896 -1%

                 Maintenance/repair of buildings $706,279 $238,203 -66%

    Miscellaneous government contract $2,204,880 $558,540 -75% services

                         Office supplies $12,939 $153,249 1084%

                      Miscellaneous expensesb $29,320 $28,887 -1%

                       Total expenses $6,511,002 $4,515,278 -31%

                         Earned revenue $258,760 $634,800 145%

                      Overall deficit -$6,252,242 -$3,880,478 -38%

    Deficit when space rental expense is excluded -$3,730,802 -$1,379,582 -63%

    Source: NTSB.

    a

    NTSB leases the academy facility from George Washington University under a
    20-year lease that will expire in 2021.

    b

    Miscellaneous expenses such as telephone, mail, and photography services
    and printing.

    In addition, while some courses presented during the first 2 years of
    academy operation did not recover the costs that NTSB attributes to them,
    revenues from other courses exceeded the cost. Of the 49 class sessions
    provided at the academy in fiscal years 2004 and 2005, revenues from 14
    sessions, all of which occurred in fiscal year 2005, did not recover their
    cost, while revenues from the remaining sessions exceeded the cost.25
    According to the academy's deputy manager, courses are only expected to
    generate enough revenues to offset the costs specifically attributed to
    the course, with some additional allocation for research and development
    of other programs and, if possible, other academy costs. Accordingly,
    tuition prices are determined by estimating those costs (such as course
    material, contracted instructors and their travel expenses) and dividing
    that cost by the projected class size. Costs such as the building lease,
    maintenance, building security, and academy personnel are not allocated to
    the costs of

    The revenue deficient for the 14 sessions totaled $54,279, and the revenue
    surplus for the two years totaled $307,203.

    Page 32 GAO-06-801T

    individual courses.26 In addition, consideration is given to setting
    tuition at a level that is competitive with similar courses by other
    institutions and that is not prohibitively high for prospective students
    from government agencies, according to the academy official.

    Other sources of revenue are needed for NTSB to be able to recover the
    full costs of the academy. For fiscal year 2004, over $12,000 in revenue
    (about 5 percent of total revenues) was collected from sources other than
    course fees to cover some of those costs. For fiscal year 2005, the
    revenue from other sources increased to over $91,000 (about 14 percent of
    total revenues). Other sources of income during these 2 years included
    renting space to other organizations, such as the Society of Automotive
    Engineers, George Washington University, and the National Association of
    State Boating Law Administrators for meetings, conferences, and boat
    storage. In addition, NTSB has contracted with two agencies-the Federal
    Energy Regulatory Commission and the Virginia Circuit Courts-for
    continuity of operations. According to NTSB officials, it has explored
    this option with other organizations, but has not found others who will
    pay a yearly retainer for the service.27 While NTSB has taken action to
    generate revenue from other sources, it does not have a business plan or
    marketing strategy that seeks to optimize opportunities for additional
    revenues. According to the academy's deputy manager, NTSB plans to develop
    a business plan. The agency, however, has no timeframes for doing so.

    Our analysis of the academy lease indicates that NTSB has the flexibility
    to use the facility in other ways to generate revenues or potentially
    reduce costs. For example, the lease does not preclude NTSB from
    subletting unused space to other users. Since certain space is already
    configured as classrooms and the academy is located in an academic setting
    on George Washington University's suburban Virginia campus, it may be
    possible to market space to academic users. Furthermore, NTSB is not
    precluded by its academy lease or its lease for headquarters space in
    Washington, D.C., from relocating some headquarters staff to the Virginia
    facility. The lease

    26

    If the tuition fee is set by dividing the costs attributable to a course
    by the projected class size, the fee may not be competitive with fees
    charged by other institutions offering similar courses. In that case, the
    projected class size might not be attainable without lowering the tuition
    to a competitive level, with the result that fee revenues collected might
    not cover the attributable costs.

    27

    NTSB has a memorandum of understanding with GAO for the two agencies to
    reciprocate in providing continuity of operations. There is no annual fee
    associated with this agreement, only cost reimbursement after the first 14
    days of providing space.

    Page 33 GAO-06-801T

                   Academy Classrooms Are Significantly Underutilized

    for the office space in Washington, D.C., expires in 2011. Such a move,
    however, would incur one-time costs that include relocating staff, moving
    furniture and equipment, reconfiguring space and utilities as well as
    recurring travel costs for staff who must travel between the two
    locations. Such costs would have to be weighed against the reduced cost of
    leasing less space in Washington, D.C.

    NTSB has not maximized the use of the facility, which could generate
    additional revenues that may help cover costs.28 We estimate that,
    overall, less than 10 percent of the total classroom space was used during
    fiscal year 2005.29 As shown in figure 12, none of the five classrooms
    were used for 21 weeks in fiscal year 2005. In addition, at any given
    time, no more than three classrooms were in use. Figure 12 shows the days
    in which classroom space was used for 31 class sessions and 12 other
    events, such as workshops and seminars by organizations that rented the
    space during fiscal year 2005.

    28

    The academy facility contains five classrooms, a large warehouse that
    houses aircraft and other wreckage, eating and lounge areas, and office
    space for five employees who constitute NTSB's Washington field office.

    29

    We excluded federal holidays and the last week in December from our
    analysis. In some cases, courses used multiple classrooms. We lacked
    specific information on which courses used multiple classrooms. To account
    for that situation, we rounded up the percentage of space utilized. The
    use of multiple classrooms does not affect the information on the lack of
    using any classrooms for 21 weeks.

    Page 34 GAO-06-801T

    Note: Bars indicate classroom use. For example, two stacked bars indicate
    that two classrooms were in use on a particular day.

    While a relatively small percentage of the academy's students have been
    NTSB staff, the agency is taking efforts to increase their enrollment at
    the academy. About 20 percent of the academy's approximately 1,000
    students30 in fiscal year 2004 were NTSB staff, and about 14 percent of
    the 1,400 students in fiscal year 2005 were NTSB staff. Over the 2 years,
    about 400 NTSB students31 attended 38 of the 49 class sessions conducted
    at the academy during fiscal years 2004 and 2005. (See fig. 13) NTSB is
    making efforts to have staff more fully utilize the facility. In fiscal
    year 2004, 1 of 18 sessions was only for NTSB investigators; in fiscal
    year 2005, 5 of 31 sessions were only for NTSB investigators.32 While
    increasing the use of the academy by NTSB staff would reduce the costs of
    sending them to external training, it is important that NTSB not reduce
    the number of external, paying students in the process.

    30

    The total number of students is the sum of the participants in all
    classes. Individuals who attended more than one class at the academy were,
    therefore, counted multiple times.

    31

    Individuals that attend more than one class are counted multiple times.

    32

    These course sessions were Conducting Effective Technical Presentations;
    two sessions each of Media Training and Major Investigation Protocol and
    Processes; and a joint training class with the Federal Bureau of
    Investigation.

    Page 36 GAO-06-801T

    NTSB staff receive most of their training from outside the academy, which
    may be due to the courses lacking the subject matter that they require.
    Our analysis of staff training requests for fiscal year 2006 showed that
    97 percent of all training is expected to be from external sources and the
    remaining training from NTSB's academy. NTSB staff have requested external
    training being provided by organizations that include FAA's Transportation
    Safety Institute, the University of Southern California, the

    U.S. Department of Agriculture, and Kettering University for training in
    subjects such as human factors in aviation safety, turbine engine
    investigation, or automotive design and safety. Training requests cover
    other specialties such as helicopter training, flight training currency
    for pilots, technical writing, supervisory and management skills, and
    industry conferences. Investigators and writer editors with whom we spoke
    had positive views on the quality of academy training courses but provided
    several reasons for not taking further courses there. Ten of the 23
    investigators and writer editors we interviewed told us that they had
    taken (or taught) courses at the academy and thought the courses were
    excellent;33 none of the investigators and writer editors had anything
    negative to say about the quality of any academy course. However, none of
    the staff we talked with had plans to attend academy training in fiscal
    year 2007. One reason noted for this situation was the remoteness of
    Ashburn, Virginia, from their residences. Another reason was the lack of
    courses on new transportation technologies and the skills and competencies
    needed by an investigator-in-charge. Eight investigators told us that they
    find workshops by manufacturers, such as aircraft and automobile
    manufacturers, more valuable to their work than academy training.

    The academy is not utilized more by NTSB staff, in part, because the
    agency has not developed a core curriculum for its staff that could then
    be offered at the academy, as mentioned previously in this testimony. The
    academy only offers one course that is required for NTSB staff--a 2-week
    course on aviation accident investigation that is required for new NTSB
    investigator staff. The deputy manager of the academy told us that the
    academy plans to eventually offer more internal training covering subjects
    such as management skills, retirement, and computers.34 However, no
    milestones or specific plans have been established for that effort.

    Although most students at the academy are from outside NTSB, several
    factors can affect the agency's ability to attract additional outside
    students. First, the lack of a business or marketing plan may be affecting
    NTSB's ability to fully utilize the academy. Second, academy training is
    similar to training provided by other institutions. FRA, FAA, and PHMSA
    officials told us that their investigators do not attend NTSB training
    because similar training is provided in-house by DOT's Transportation
    Safety Institute. For example, an FAA investigator told us that new
    investigators take a basic accident investigation course at the
    Transportation Safety Institute and subsequently take mid-career followup
    courses there. Furthermore, our comparison of NTSB's fiscal year 2006
    curriculum with that of several other institutions that teach courses on
    accident investigations showed that other institutions offered courses
    similar to 12 of NTSB's 19 courses. For example, DOT's Transportation
    Safety Institute offers basic courses on aviation and bus accident

    33

    Our review of course evaluations for fiscal years 2004 and 2005 indicated
    high positive responses by students to the academy courses. The data
    lacked information for us to compare evaluations by NTSB students and
    non-NTSB students.

    34

    NTSB is considering contracting out more courses such as these.

    Page 38 GAO-06-801T

        Additional Issues Concerning the Academy

    investigations, and the University of Southern California offers a course
    on human factors related to accident investigations.

    You asked that we provide information concerning the academy's use of NTSB
    investigators as instructors and NTSB's compliance with the
    Anti-Deficiency Act, with regard to its accounting for its academy lease.
    Concerning the first issue, academy courses are taught by a combination of
    academy staff, NTSB investigators and managers, and contractors. Use of
    investigators as instructors is limited and is likely to have little
    impact on investigators' overall workload. During fiscal year 2005, 51
    NTSB investigators or managers taught at the academy. On average they
    spent an estimated 22 hours to prepare for and teach courses. (See fig.
    14.)

    Note: Other instructors include NTSB academy staff, volunteers, and
    contract instructors from outside NTSB. Data on the number of hours
    "other" instructors spent preparing to teach is not collected by NTSB.

    Finally, NTSB classified its lease for the academy as an operating lease
    rather than a capital lease. As a result, NTSB has been noncompliant with
    the Anti-Deficiency Act because it did not obtain budget authority for the
    net present value of the entire 20-year lease obligation at the time the
    lease agreement was signed in 2001. NTSB realized the error in 2003 and
    reported its noncompliance to Congress and the President. NTSB has
    proposed in the President's fiscal year 2007 budget to remedy this
    antideficiency act violation by inserting an amendment in their fiscal
    year 2007 appropriation that would allow NTSB to fund this obligation from
    their salaries and expense account through fiscal year 2020.

      Conclusions

    Mr. Chairman, we have developed several conclusions from our analysis of
    NTSB to date. To the credit of the current leadership at NTSB, much of the
    agency's progress toward following leading practices is due to recent
    management initiatives. The performance management plan, draft staffing
    plan, and implementation of controls over financial transactions are all
    positive steps. NTSB's progress in these areas will likely remain
    incomplete without additional actions, however. For example, without a
    more comprehensive strategic plan than it currently has, NTSB cannot align
    staffing, training, or other human resource management to its strategic
    goals or align its organizational structure and layers of management with
    the plan. NTSB will also likely miss opportunities to strengthen the
    management of the agency until it develops a strategic training plan for
    its employees, implements a full cost-accounting system, and improves
    communications within the agency.

    We have also concluded that, despite the many safety recommendations NTSB
    has made and seen implemented over the years of its existence,
    inefficiencies have resulted from the process that the agency uses to
    close out safety recommendations. In particular, the absence of a
    computerized documentation system and the sequential reviews that NTSB
    currently requires slow the process and prevent expedient delivery of
    information about recommendation status to affected agencies. Finally, in
    terms of its academy, NTSB is missing opportunities to increase the value
    of this asset. Without a comprehensive marketing plan, NTSB will likely be
    unable to efficiently attract users who would help pay the ongoing costs
    of the facility.

      Recommendations for Executive Action

    o

    To improve the efficiency of agency operations, we are making eight
    recommendations to the Chairman of the National Transportation Safety
    Board based on our completed work to date. To improve agency performance
    in the key functional management areas of strategic planning, human
    capital planning, financial management, and communications, we recommend
    that the Chairman implement the following three recommendations:

    Improve strategic planning by developing a revised strategic plan that
    follows performance-based practices; developing a strategic training plan
    that is aligned with the revised strategic plan and identifies skill gaps
    that pose obstacles to meeting the agency's strategic goals and curriculum
    that would eliminate these gaps; and aligning their organizational
    structure to

    Page 41 GAO-06-801T

    implement the strategic plan and eliminate unnecessary management layers.

         o Develop a full cost-accounting system that would track the amount of
           time employees spend on each investigation and in training.
              * Develop mechanisms that will facilitate communications from
                staff-level employees to senior management, including
                consideration of contracting out a confidential employee survey
                to obtain employee feedback on management initiatives.
              * To enhance the efficiency of the report development and
                recommendation close-out processes, we recommend that the
                Chairman take the following two actions:
         o Identify better practices in the agency and apply them to all modes.
           Consider such things as using project managers or deputy
           investigators-in-charge in all modes, using incentives to encourage
           performance in report development, and examining the layers of review
           to find ways to streamline the process, such as eliminating some
           levels of review and using concurrent reviews as appropriate.
              * Improve the efficiency of the review process for changing the
                status of recommendations by computerizing the documentation and
                implementing concurrent reviews.
              * To enhance the utilization of the academy and improve the ability
                to generate revenues that will cover academy costs, we recommend
                that the Chairman take the following three actions:
         o Develop a comprehensive marketing plan for the academy. The plan
           should consider such things as outreach to potential users, working
           with USDA and GSA to market it as classroom and conference space, and
           conducting market research for additional curriculum development. If
           ethical and conflict-of-interest issues can be addressed, the plan
           should also consider options for allowing transportation manufacturers
           to conduct company-sponsored symposia and technical training at the
           academy facility, which would benefit NTSB investigators in keeping up
           with new technologies. In addition the plan should consider the
           feasibility of subleasing a portion of the academy space.
         o Develop core investigator curriculum for each mode and maximize the
           delivery of that training at the academy.

    Page 42 GAO-06-801T

    o  Conduct a study to determine the costs and feasibility of moving
    certain functions from headquarters to the academy facility in preparation
    for the renegotiation of the headquarters lease, which expires in 2011.

    We obtained comments on a draft of this testimony from NTSB. NTSB's

    Managing Director concurred with our recommendations and provided
    clarifying comments and technical corrections, which we incorporated as
    appropriate. In addition, NTSB commented that the draft did not
    sufficiently distinguish improvements that have been made over the past
    year. We revised the testimony to more clearly distinguish those actions.

      Scope and Methodology

    To determine the extent to which NTSB is following leading practices in
    selected management areas, we reviewed past GAO work on leading management
    practices in the areas of strategic planning, performance management,
    human capital management, financial management, and communications. We
    interviewed NSTB board members, senior officials, managers, investigators,
    and writer editors regarding their experience with those practices at
    NTSB, and their perceptions of the effectiveness of those practices. We
    also determined NTSB's response to recommendations made by the DOTIG. We
    reviewed NTSB documents, including its strategic, staffing, and
    performance management plans; management advisory e-mail; and information
    regarding the current staffing levels; and employees' training plans for
    2006.

    To determine the extent to which NTSB is developing accident investigation
    reports and closing safety recommendations in an efficient manner, we
    interviewed NTSB investigators, writer editors, managers, and senior
    officials regarding the investigative process and their role in it. We
    randomly selected 15 of the 210 investigators and 8 writer editors evenly
    across the 4 modal offices. The views represent the particular individuals
    and are not representative of all NTSB investigators and writer editors.
    We reviewed policy guidance on the investigative process and the level of
    current and past investigation activity. We examined data on
    recommendations acceptance rates and close-out status from NTSB's
    recommendation database, and we determined that the data were sufficiently
    reliable for the objectives of this review. Additionally, we reviewed
    studies done by the Rand Corporation and Booz Allen Hamilton that examined
    NTSB's investigation process and determined the extent to which the agency
    had implemented their recommendations.

    To determine the extent to which NTSB is generating sufficient revenues to
    cover costs at its academy, we reviewed financial data on NTSB's academy,
    including the revenues and expenses for fiscal years 2004 and 2005. We
    reviewed the course curriculum of the academy, and compared it with
    classes offered by DOT's Transportation Safety Institute, Embry Riddle,
    the University of Southern California, and the Southern California Safety
    Institute. We examined data on the student makeup of academy classes and
    analyzed data on the preparatory and teaching time used by NTSB
    investigators who taught at the academy. We interviewed NTSB
    investigators, writer editors, and managers and senior officials at DOT's
    modal administrations regarding their current and planned use of the
    academy. Finally, we examined the lease for the academy to determine how
    NTSB may utilize the space.

    We conducted our review from December 2005 to May 2006 in accordance with
    generally accepted government auditing standards.

      Contacts and Acknowledgments

    For further information on this testimony, please contact Dr. Gerald
    Dillingham at (202) 512-2834 or by e-mail at dillinghamg@gao.gov.

    Individuals making key contributions to this testimony
    include Teresa Spisak, Colin Fallon, Eric Fielding, Tom Keightley, Maren
    McAvoy, Josh Ormond, and Jena Whitley.

        (540115)

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

      GAO's Mission

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site (www.gao.gov). Each weekday, GAO posts GAO
    Reports and newly released reports, testimony, and correspondence on its
    Web site. To

    have GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

                                 Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

      To Report Fraud, Contact:

    Web site: www.gao.gov/fraudnet/fraudnet.htm

      E-mail: fraudnet@gao.gov

    Federal Programs Automated answering system: (800) 424-5454 or (202)
    512-7470

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Relations
    Washington, D.C. 20548

    Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

      Public Affairs

    U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

        PRINTED ON

    RECYCLED PAPER
    *** End of document. ***