National Transportation Safety Board: Preliminary Observations on
the Value of Comprehensive Planning, and Greater Use of Leading
Practices and the Training Academy (24-MAY-06, GAO-06-801T).
The National Transportation Safety Board (NTSB) is a relatively
small agency that plays a vital role in transportation safety and
has a worldwide reputation for investigating accidents. With a
staff of about 400 and a budget of $76.7 million in fiscal year
2006, NTSB investigates all civil aviation accidents in the
United States, and significant accidents in railroad, highway,
marine, and pipeline; and issues safety recommendations to
address issues identified during accident investigations. To
support its mission, NTSB built a training academy, which opened
in 2003 and provides training to NTSB investigators and others.
It is important that NTSB use its resources efficiently to carry
out its mission and maintain its preeminence. This testimony,
based on ongoing work for this committee, addresses the extent to
which NTSB follows leading practices in selected management
areas, addresses challenges in completing accident investigations
and closing safety recommendations, and generates sufficient
revenues to cover costs at its academy.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-801T
ACCNO: A54732
TITLE: National Transportation Safety Board: Preliminary
Observations on the Value of Comprehensive Planning, and Greater
Use of Leading Practices and the Training Academy
DATE: 05/24/2006
SUBJECT: Agency missions
Best practices
Best practices methodology
Education or training costs
Employee training
Financial management
Investigations by federal agencies
Performance management
Strategic planning
Transportation safety
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-801T
United States Government Accountability Office
Testimony
GAO
Before the Subcommittee on Aviation, Committee on Commerce, Science and
Transportation, U.S. Senate
For Release on Delivery NATIONAL TRANSPORTATION
Expected at 9:00 a.m. EDT
Wednesday, May 24, 2006
SAFETY BOARD
Preliminary Observations on the Value of
Comprehensive Planning, and Greater Use of
Leading Practices and the Training Academy
Statement of Gerald L. Dillingham, Ph.D.
Director, Physical Infrastructure Issues
GAO-06-801T
NATIONAL TRANSPORTATION SAFETY BOARD
Preliminary Observations on the Value of Comprehensive Planning, and
Greater Use of Leading Practices and the Training Academy
What GAO Found
NTSB has recently made progress in following leading management practices,
but overall has a mixed record. For example, NTSB has improved its
financial management by hiring a Chief Financial Officer and putting
controls on its purchasing activities, which should address past problems
with unapproved purchases. However, NTSB lacks a full cost accounting
system, which would inform managers of the resources spent on individual
investigations and provide data to balance office workload. NTSB has also
begun to develop a performance management system that should eventually
link each individual's performance to the agency's strategic goals and
objectives. However, the performance management system will not be fully
functional until NTSB has developed a strategic plan with results-oriented
goals and objectives and specific strategies for achieving them, which are
lacking in the current strategic plan. Other areas, such as human capital
and communications, partially follow leading practices.
While NTSB is accomplishing its accident investigation mission, it faces
challenges that affect the efficiency of the report production and
recommendation close out processes. NTSB routinely takes longer than 2
years to complete major investigations. Several factors may affect the
length of report production, including several revisions of draft reports
through multiple layers of the organization. In addition, the processes
for federal transportation agencies to implement NTSB's safety
recommendations and for NTSB to change the status of recommendations are
lengthy, paper-based, and labor intensive. While Department of
Transportation officials have been working with NTSB to find acceptable
means of implementing its recommendations, they cite the lengthy
rule-making process as a challenge to speedy implementation.
For fiscal years 2004 and 2005, NTSB's academy did not generate sufficient
revenues to cover the costs of providing training. As a result, those
portions of the academy's costs that were not covered by the revenues from
tuition and other sources-approximately $6.3 million in fiscal year 2004
and $3.9 million in fiscal year 2005-were offset by general appropriations
to the agency. While NTSB has taken action to generate revenue from other
sources, such as renting academy space for conferences, it does not have a
marketing plan that seeks to optimize opportunities for additional
revenues at the academy.
United States Government Accountability Office
Mr. Chairman and Members of the Subcommittee:
We are pleased to be here today to discuss the reauthorization of the
National Transportation Safety Board (NTSB). NTSB is a relatively small
agency that plays a vital role in transportation safety. With a staff of
about 400 and a budget of $76.7 million in fiscal year 2006, NTSB is
charged with investigating every civil aviation accident in the United
States and significant accidents in the other modes-railroad, highway,
marine, and pipeline-,determining the probable cause of these accidents
and issuing recommendations to address safety issues identified during
accident investigations. NTSB has gained a worldwide reputation as a
preeminent agency in conducting transportation accident investigations.
Since 1967, it has issued 1,340 major accident investigation reports, over
130,000 brief accident reports, and made over 12,000 safety
recommendations. To support its mission, NTSB built a training academy
that opened in 2003 and provides training to NTSB investigators and other
transportation safety professionals, including those from foreign
countries. It is critical that the agency uses its resources in an
efficient manner to carry out its safety mission and maintain its
preeminent position. For this reason, you asked us to conduct a
comprehensive review of NTSB's management functions such as strategic
planning, human capital management, and mission-critical investigation
activities. My testimony today is based on our ongoing work for you, and
it addresses the extent to which NTSB is
(1) following leading practices in selected management areas, (2)
addressing challenges in completing accident investigations and closing
safety recommendations, and (3) generating sufficient revenues to cover
costs at its academy. We will be reporting additional results of our
ongoing work to the committee later this year.
In summary:
* While NTSB has recently made progress in following leading
management practices, its overall record is mixed. For example,
NTSB generally follows leading practices in the area of financial
management. Over the last several years, NTSB has hired a Chief
Financial Officer and improved its financial management by putting
controls on its purchasing activities, which should address past
problems with unapproved purchases with government credit cards.
However, NTSB lacks a full cost accounting system, which would
inform managers of the resources spent on individual investigations
and provide data to help assure balanced office workload. Other
areas, such as performance management, human capital, and
communications, partially follow leading practices. For example,
NTSB has begun to develop a performance management system that
should eventually link each
* individual's performance to the agency's strategic goals and
objectives. However, the performance management system will not be
fully functional until NTSB has a strategic plan with
results-oriented objectives and specific strategies for achieving
them, which are lacking in the current strategic plan. In the area
of human capital management, NTSB has recently developed a draft
staffing plan that addresses the agency's skills and competencies
needs and includes strategies to increase the number of
investigators and thereby strengthen the agency's ability to carry
out its transportation safety mission. However, the draft plan does
not address organizational structure or the balance between
supervisory and nonsupervisory positions. While NTSB has recently
taken positive steps to improve communications from senior
management to the staff-such as periodically sending e-mail to all
staff to share information on new developments and policies-the
agency does not regularly hold general staff meetings or undertake
anonymous surveys to obtain employee feedback.
o NTSB is accomplishing its accident investigation mission, but it faces
challenges that affect the efficiency of the report production and
recommendation close-out processes. In terms of accomplishing its
mission, since its inception in 1966, NTSB has investigated over
134,000 transportation accidents, and 82 percent of its
recommendations have been implemented, or acceptable progress toward
implementation has been made. However, investigations are
often-sometimes necessarily- lengthy; NTSB routinely takes longer than
2 years to investigate major accidents. Lengthy investigations,
combined with lengthy processes for federal agencies to regulate and
industries to implement NTSB's safety recommendations, can work
against the goal of improving transportation safety. One factor that
adds to the duration of investigations is that when new investigations
are launched, inspectors are pulled from working on previous accidents
to work on new ones. Other factors that may affect the duration of
report production include the multiple revisions of draft
investigation reports at different levels in the organizations and
resource issues. NTSB has recently taken several actions that may help
shorten report development time, such as reemphasizing its policy on
holding report development meetings to obtain early buy-in on report
messages and holding modal directors accountable for specific issuance
dates. We also identified practices in certain offices, such as the
use of a project manager or deputy investigator-in-charge to handle
report production, which may improve the efficiency of report
development if used by all modal offices as they all are similar in
what they do. The processes for implementing NTSB's safety
recommendations, and for NTSB to change the status of recommendations
are also lengthy and labor intensive. As a result, unsafe conditions
may continue to exist until federal transportation
Page 2 GAO-06-801T
agencies, and ultimately, transportation industries, fully implement the
recommendations, and the extended period it takes to change the status of
recommendations ties up NTSB's scarce resources. As of May 2006, 305 of
the 852 open recommendations have been in open status for 5 years or more.
While Department of Transportation (DOT) officials have been working with
NTSB to find acceptable means of implementing its recommendations, they
cite the lengthy rule-making process as a challenge to speedy
implementation. In addition, the process that NTSB uses to change the
status of safety recommendations is paper-based, labor intensive, and
relies on a series of sequential reviews that can take months to complete.
As a result, resources within NTSB are inefficiently used and DOT agency
officials told us they remain unaware whether their response has been
accepted or not accepted.
o For fiscal years 2004 and 2005, NTSB's academy did not generate
sufficient revenues to cover the costs of providing training. As a result,
those portions of the academy's costs that were not covered by the
revenues from tuition and other sources-approximately $6.3 million in
fiscal year 2004 and $3.9 million in fiscal year 2005-were offset by
general appropriations to the agency. Although there is no statutory
requirement that revenues from NTSB's academy generate sufficient revenues
to cover the costs, NTSB was encouraged in the Senate report accompanying
the Fiscal Year 2006 DOT Appropriations Act to be more aggressive in
imposing and collecting fees to cover the costs. While NTSB has taken
action to generate revenue from other sources, such as renting academy
space for conferences, it does not have a business plan that seeks to
optimize opportunities for additional revenues at the academy.
Additionally, NTSB is missing opportunities to find other uses for academy
space. For example, during fiscal year 2005, less than 10 percent of the
total classroom space was used. About 14 percent of the academy students
in fiscal year 2005 were NTSB employees. However, in 2006, NTSB employees
are scheduled to take 97 percent of their requested training from sources
other than the academy, such as DOT's Transportation Safety Institute. The
academy is not utilized more by NTSB staff, in part, because the agency
has not developed a core curriculum for its staff, which it could then
offer at the academy. Furthermore, many academy courses are similar to
those taught elsewhere, which may affect the agency's ability to attract
non-NTSB students.
Background
NTSB was established in 1966 as an independent government agency located
within the newly formed DOT.1 In 1974, Congress made NTSB completely
separate from DOT.2 NTSB's principal responsibility is to promote
transportation safety by investigating transportation accidents,
determining the probable cause, and issuing recommendations to address
safety issues identified during accident investigations. Unlike other
transportation agencies, such as the Federal Aviation Administration
(FAA), NTSB does not have the authority to promulgate regulations to
promote safety, but makes recommendations in its accident reports and
safety studies3 to other agencies that have such regulatory authority. The
federal agencies that receive NTSB recommendations include the DOT's FAA,
Federal Highway Administration (FHWA), Federal Motor Carrier Safety
Administration (FMCSA), Federal Railroad Administration (FRA), Federal
Transit Administration (FTA), National Highway Traffic Safety
Administration (NHTSA), Pipeline and Hazardous Materials Safety
Administration (PHMSA), and the U.S. Coast Guard. NTSB also makes
recommendations to others, such as state transportation authorities and
industries. As figure 1 indicates, NTSB has varying degrees of flexibility
in its statutory mandate, as it pertains to initiating an investigation.
By statute, NTSB has limited discretion in deciding which aviation
accidents to investigate and the greatest amount of discretion to
investigate highway accidents.
1
Department of Transportation Act, P.L. 89-670, Oct. 15, 1966.
2
Independent Safety Board Act, P.L. 93-633, Title III, 1974.
3
NTSB conducts safety studies as a result of identifying safety concerns
rather than as a result of specific accidents.
Page 4 GAO-06-801T
Figure 1: NTSB's Investigative Policy by Mode
NTSB is comprised of a five member board-a chairman, vice chairman, and
three members-appointed by the President with the advice and consent of
the Senate.4 The chairman is NTSB's chief executive and
4
Not more than three members may be appointed from the same political
party. At least three members are appointed on the basis of technical
qualification, professional standing, and demonstrated knowledge in
accident reconstruction, safety engineering, human factors, transportation
safety, or transportation regulation.
Page 5 GAO-06-801T
administrative officer. As of March 2006, the board was supported by a
staff of 396, which includes 210 investigators assigned to four modal
offices-aviation; highway; marine; and rail, pipeline, hazardous
materials. (See fig. 2.) The agency is headquartered in Washington, D.C.,
and maintains 10 field offices nationwide and a training academy in
Ashburn, Virginia, in suburban Washington, D.C. In recent years, the
agency has shrunk in size due to budget constraints, which it has largely
dealt with by using attrition to downsize the staff. In 2003, NTSB had 438
full time employees compared with the current level of 396. During the
same period, the number of full-time investigators decreased from 234 to
210. NTSB's modal offices vary in size, with the aviation office having
125 employees; the rail, pipeline, and hazardous materials office having
38; the highway office having 30; and the marine office having 16
employees as of May 2006. An additional 42 employees work in the Office of
Research and Engineering, which provides technical, laboratory,
analytical, and engineering support for the modal investigation offices.
For example, it is responsible for interpreting data recorders, creating
accident computer simulations, and publishing general safety studies.
NTSB's budget increased from $62.9 million in fiscal year 2001 to $76.7
million in fiscal year 2006, or about 22 percent. After adjusting for
inflation, this represents an increase of about 9 percent. The President
has requested $79.6 million for NTSB in fiscal year 2007.
Figure 2: NTSB's Organization
Since 1966, NTSB has investigated over 124,000 aviation accidents and over
10,000 surface transportation accidents. Figure 3 shows the total number
of aviation investigations that NTSB has undertaken over the past 6 years
and the degree to which NTSB was involved in the investigations. NTSB
lacks the resources to conduct on-scene investigations of all aviation
accidents. As a result, for general aviation accidents, NTSB delegates the
gathering of on-scene information to FAA investigators, as allowed by
statute.5 In these limited investigations, FAA sends the accident
information to NTSB, and NTSB then determines a probable cause for the
accident. In addition, NTSB participates in the investigations of foreign
aviation accidents in conformance with Annex 13 of the International Civil
Aeronautics Organization Treaty. These investigations involve a U.S.
carrier or U.S.-built aircraft, or occur at the request of a foreign
government. NTSB aviation investigators told us that there is often
significant value in participating in such investigations; the findings
often have safety implications for U.S. carriers, since most foreign
airlines use
5
49 U.S.C Sec. 1132(c).
U.S.-made aircraft, engines, and other parts and multiple foreign air
carriers operate within the United States.
Figure 3: NTSB Involvement in Aviation Accidents, Fiscal Years 2000-2005
Note: Limited investigations are those in which NTSB delegates the
gathering of on-scene information to FAA inspectors.
Page 9 GAO-06-801T
NTSB Has Made Recent Progress in Following Leading Management Practices, But
Overall Record Remains Mixed
Through our work government wide we have identified a number of key
functional areas and leading practices in areas that are important for
managing an agency. This testimony focuses on NTSB's performance in five
key functional areas-strategic planning, performance management, human
capital, financial management, and communications-and how NTSB's practices
compare to leading practices in those areas. As illustrated in figure 4,
NTSB generally is following leading practices in financial management,
only minimally following leading practices in strategic planning, and has
mixed results for the other functions. Much of NTSB's progress toward
following leading practices is due to recent management initiatives. The
report we will be issuing later this year will provide additional
information on NTSB's performance relative to these five management
functions, as well as information technology, acquisition management
(including the agency's use of contracting), knowledge management, and
capital decisionmaking.
Figure 4: Extent to Which NTSB Is Following Leading Practices in Selected
Management Areas
NTSB's Strategic Plan Lacks Certain Performance-Based Elements and
Performance Management Plans Closely Follow Leading Practices but are not
Fully Functional
The Congress and the President have encouraged better management of
federal agencies by means such as results-oriented strategic planning, but
NTSB's strategic plan generally does not follow performance-based
practices. Without effective short- and long-term planning, federal
agencies risk delivering programs and services that may or may not meet
the nation's most critical needs. The Government Performance and Results
Act of 1993 (GPRA)6 and guidance contained in the Office of Management and
Budget's (OMB) Circular A-11, provide performance-based strategic planning
guidelines. GPRA was intended to achieve several broad purposes, including
improving federal program effectiveness, accountability, and service
delivery, and enhancing congressional decision making by providing more
objective information on program performance. GPRA requires federal
agencies to develop strategic plans in which they define their missions,
establish results-oriented goals, and identify the strategies that will be
needed to achieve those goals. For instance, GPRA requires strategic plan
updates at least every 3 years, and requires that agencies set objectives
and goals that are specific outcomes that the organization wishes to
accomplish (called outcome-related objectives).
To its credit, in December 2005, NTSB issued a strategic plan for the
years 2006 through 2010, which was the first time the agency had a
strategic plan in 6 years. In developing that plan, senior agency
officials told us that they modeled their plan on examples from other
federal agencies with similar structure and mission, such as the Federal
Communications Commission. We compared NTSB's strategic plan to selected
elements required by GPRA. (See fig. 5.)
6Pub. L. No. 103-62.
Figure 5: Extent to Which NTSB's Strategic Plan Follows GPRA Elements
GPRA elements Follows GPRA elements
Mission statement
General goals and objectives
Approaches or strategies to achieve goals and
objectives
Relationship between general goals and annual goals
External factors
Program evaluations
Five year time frame
Stakeholder involvement
Follows GPRA elements
Source: GAO's analysis of NTSB data.
While NTSB's 5-year strategic plan has a mission statement, four general
goals and related objectives, and mentions key factors, such as declining
resources, that could affect the agency's ability to achieve those goals,
the plan lacks a number of key elements-including information about the
operational processes; skills and technology; and the human, capital, and
information resources-required to meet the goals and objectives. In
addition, the goals and objectives lack sufficient specificity to know
whether they have been achieved. One goal states "NTSB will maintain its
response capacity for investigation of accidents and increase its analysis
of incidents." An objective of that goal is to "continuously assess the
most robust and efficient approaches to accident investigation." Although
such a goal is important for the safety of the transportation industry,
this and the other three goals and related objectives are not measurable.
As a result, it will be difficult for NTSB and others to determine if the
goals have been achieved.
In addition, the plan lacks specific strategies for achieving those goals.
According to GPRA, the strategies should include a description of the
operational processes, skills and technology, and the resources required
to meet the goals and objectives. Since NTSB's strategic plan lacks such a
description, it does not align staffing, training, or other human resource
management to strategic goals. That is, the plan does not explicitly
explain how NTSB will use its resources to meet its mission and goals.
While the
Page 12 GAO-06-801T
plan explains that each program office has its own objectives linked to
the agency's goals and objectives, the plan contains no information to
understand how each office contributes to those goals and objectives. In
addition, NTSB's strategic plan does not describe how the performance
goals contained in the annual performance plan are related to the general
goals and objectives in the strategic plan, as required by GPRA.
GPRA also requires federal agencies to provide a description in their
strategic plans of the program evaluations used in establishing or
revising general goals and objectives and a schedule for future program
evaluation. NTSB's strategic plan lacks this information. As a result of
having no program evaluations, it is unclear how or whether NTSB reviews
its efforts to identify strengths it can maximize and weaknesses it should
address. In developing a strategic plan, GRPA requires agencies to consult
with Congress and other stakeholders. We have previously reported that
other stakeholders of federal agencies include state and local
governments, other federal agencies, interest groups, and agency
employees. NTSB's strategic plan does not mention consultation with any
stakeholders in its development. Furthermore, board members and agency
staff told us that they had no involvement in the development of the
strategic plan. Some current and past board members additionally stated
that they believed that their involvement would be beneficial in providing
a strategic vision for the agency. NTSB's senior management told us they
expect to revise the strategic plan in the near future and contacted us
regarding assistance to develop a more comprehensive, results-oriented
plan as part of this study.
NTSB has begun to develop a performance management system that should
eventually link each individual's performance throughout the agency to the
agency's strategic goals and objectives. We have reported that performance
management systems are crucial for agencies because if developed properly
they allow employees to make meaningful contributions that directly
contribute to agency goals.7 NTSB has developed a comprehensive
performance management plan for Senior Executive Series (SES) employees
that links individual performance to strategic goals. Furthermore, the
plan states that NTSB will link performance management with the agency's
results-oriented goals and set and communicate individual and
organizational goals and expectations.
GAO, Results Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success, GAO-03-488 (Washington,
D.C.: March 14, 2003).
Page 13 GAO-06-801T
NTSB's Staffing Plan Is a Step in the Right Direction, but the
Organizational Structure Has Not Been Reviewed
This plan establishes individual performance criteria and the appraisal
process. The appraisal process defines performance standards and explains
performance elements that determine individual ratings. Because NTSB
recognizes in this plan the importance of aligning organizational
performance with individual performance and contributions to the agency's
mission, the performance management plan is a step in the right direction.
Along with the SES plan, NTSB issued in August 2005 a performance plan for
its overall workforce, which includes some elements of linking individual
performance to organizational goals. However, without having
results-oriented goals in the strategic plan itself, neither of the two
performance management plans are fully functional. That is, until NTSB's
goals are more fully articulated in the strategic plan, it will be
impossible for staff to know whether their performance contributes to
meeting those goals. As with the strategic plan, NTSB staff was not
involved in the development of the performance plan, and there was no
mechanism for employee feedback after the plan was initially developed.
Employee involvement provides greater assurance that policies are accepted
and implemented because employees had a stake in their development.
NTSB developed a draft agencywide staffing plan in December 2005 that
follows several leading practices but lacks a workforce deployment
strategy that considers the organizational structure and its balance of
supervisory and non-supervisory positions. Existing strategic workforce
planning tools and models suggest that certain principles should be
followed in strategic workforce planning, such as determining the agency's
skills and competencies needs; involving stakeholders (e.g., management
and employees) in the planning process; and developing succession plans to
anticipate upcoming employee retirement and workforce shifts.8 Further, in
workforce deployment, it is important to have human capital strategies to
avoid excess organizational layers and to properly balance supervisory and
nonsupervisory positions.9 NTSB's draft staffing plan addresses the
agency's skills and competencies needs and includes strategies to deal
with workforce shifts. For example, the staffing plan
8
GAO, A Model of Strategic Human Capital Management, GAO-02-373SP
(Washington, D.C.: March. 15, 2002) and GAO, Human Capital: Key Principles
for Effective Strategic Workforce Planning, GAO-04-39 (Washington, D.C.:
December 11, 2003).
9
GAO, Executive Agency Management Diagnostic Survey (draft).
Page 14 GAO-06-801T
proposes to increase the number of investigative staff by 21, which will
help with the agency's resource needs. In addition, while some
stakeholders (i.e., managers) were involved in the planning process,
employees were not included. As we mentioned previously in this testimony,
employee input provides greater assurance that policies are accepted and
implemented because employees have a stake in their development.
To develop the staffing plan, each modal office director submitted to
NTSB's Managing Director an ideal staff size for his office, including
additional slots for investigators. The increase in investigative staff is
consistent with requests by modal offices to enhance their ability to
conduct their investigative mission. Managers told us that current
staffing constraints inhibited their ability to conduct more accident
investigations and indicated an increase in staff would be helpful. For
example, directors of the highway and rail/pipeline offices told us they
could not initiate investigations on more than two accidents at a time
because they lacked sufficient investigative staff to do more.10 The modal
office directors' request for staff resulted in a total agency allotment
of 455 full time equivalents (FTEs) plus 20 co-op positions. The Managing
Director reduced this number to 404, which corresponds to NTSB's current
funding level of 395, allowing for attrition and turnover. The Managing
Director's allocation resulted in a proposed increase of 21 investigators
agencywide and a proposed reduction of certain staff positions to
accommodate the increase in investigators. This increase in investigative
staff is consistent with a recommendation by RAND Corporation, which
evaluated NTSB's accident investigation process and workload in 1999.11 To
help implement the realignment, senior managers told us that they would
like to transition some existing administrative and support staff with
appropriate background and training into investigator roles where
possible. The draft plan set a target date of May 2006 to begin creating
developmental opportunities for staff to transition to investigative roles
and to develop reduction strategies for staff that fall outside the
staffing plan.
10
Each investigative team initially consists of at least one
investigator-in-charge and other technical support investigator positions
based on the complexity of the accident.
11
RAND Institute for Civil Justice, Safety in the Skies: Personnel and
Parties in NTSB Accident Investigations (Santa Monica, CA.: 2000).
Page 15 GAO-06-801T
NTSB Lacks a Strategic Approach to Training Staff
Training is another key area of human capital management. It is important
for agencies to develop a strategic approach to training its workforce,
which involves establishing training priorities and leveraging investments
in training to achieve agency results; identifying specific training
initiatives that improve individual and agency performance; ensuring
effective and efficient delivery of training opportunities in an
environment that supports learning and change; and demonstrating how
training efforts contribute to improved performance and results.12 NTSB
has not developed a strategic training plan, nor has it identified the
core competencies needed to support its mission and a curriculum to
develop those competencies. As a result of not having a core curriculum
that is linked in this manner, NTSB lacks assurance that the courses that
staff take provide the technical knowledge and skills necessary for them
to be competent for the type of work they perform.
Financial Management Is Improved, but NTSB Lacks a Full Cost Accounting
System
Sound financial management is crucial for responsible stewardship of
federal resources.13 In recent years, NTSB has made significant progress
in improving its financial management. In March 2001, NTSB hired a Chief
Financial Officer who has emphasized the importance of sound financial
management based on best practices. Similar to private sector companies,
government agencies are required to report their financial condition in
publicly available financial statements. As a result of actions taken by
NTSB, the agency received an unqualified or "clean" opinion from
independent auditors on its financial statements for the fiscal years
ending September 30 for the years 2003, 2004, and 2005. The audit report
concluded that NTSB's financial statements presented fairly, in all
material respects, the financial position, net cost, changes in net
position, budgetary resources, and financing in conformity with generally
accepted accounting principles for the three years. NTSB has also improved
its purchasing and contracting activities after identifying problems in
those areas in 1999. In 2001, DOT's Office of Inspector General (DOTIG)
reviewed the agency's contracting and procurement activities and
recommended that NTSB institute accountability and controls in its
purchase card program as well as other purchasing activities. As a result
12
GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, GAO-04-546G (Washington,
D.C.: March 1, 2004).
13
GAO, Executive Guide: Creating Value through World-class Financial
Management, GAO/AIMD-00-134 (Washington, D.C.: April 2000).
Page 16 GAO-06-801T
of this and another DOTIG audit,14 NTSB has taken a number of initiatives
to improve its purchasing and contracting activities. For example, NTSB
restructured its purchase card system and guidelines to address problems,
such as unrestrained and unapproved purchases on government credit cards.
NTSB hired a manager of the contracting function to manage the agency's
acquisition function and implement the DOTIG recommendations. In our full
report, we will analyze some of these initiatives in more detail.
In 2000, RAND recommended that NTSB develop systems that would allow the
agency to better manage its resources by permitting full-cost accounting15
of all agency activities.16 To accomplish this, RAND recommended putting
in place a timekeeping system, in which individual project numbers were
assigned to each investigation and support activities such as training.
With this information, project managers could better understand how staff
resources were utilized and project workload could be actively monitored
by the Managing Director. NTSB has begun to implement this recommendation
by upgrading a software system in November 2005 that tracks employee
annual leave and sick leave. However, the system is not being fully
utilized to track the number of hours staff spend on each investigation.
Also, this system is not used to track time staff spend in training or at
conferences. As a result, RAND's previous conclusion that "NTSB managers
have little information they can use to plan the utilization of staff
resources or manage staff workloads properly" remains current.
14DOTIG, Audit of the Purchase Card Program, FI-2005-072 (Washington,
D.C.: Aug. 23, 2005) and Report on Financial Management Practices and
Internal Controls, FI-2003-004 (Washington, D.C.: Dec. 11, 2002).
15
Cost accounting involves the accumulation and analysis of financial and
nonfinancial data, resulting in the allocation of costs to organizational
pursuits such as performance goals, programs, activities, and outputs.
Nonfinancial data measure the occurrences of activities and can include,
for example, the number of hours worked.
16
RAND Institute for Civil Justice, 2000.
Communications from Senior Management to Staff have Increased and
Communications Among Offices is Generally In Place, but Upwards
Communications Mechanisms Are Lacking
We have identified useful practices related to managing employees that
include seeking and monitoring employee attitudes, encouraging two-way
communication between employees and management, and incorporating employee
feedback into new policies and procedures.17 In response to issues raised
by NTSB employees in a governmentwide survey conducted by OPM in 2004,
NTSB's senior management made changes to improve the way it is
communicating information to staff. For example, the Managing Director
periodically sends "management advisory" e-mail to all staff that share
information such as policy changes or new developments at the agency.
However, we found no formal processes that encouraged two-way
communication, such as town hall meetings, regular staff meetings, or
anonymous employee surveys; or incorporated employee feedback into
policy-making.
The 23 investigators and writer editors with whom we spoke had mixed views
on the effectiveness of communications within the agency.18 The four
investigators from one modal office that we spoke with told us that they
are pleased to now hear about policy changes at the agency, but said that
there is too much reliance on the Internet for these communications. They
also told us that although they believe the increased communications are
positive, they found it difficult to find the time to read the material
and still conduct their regular investigative duties. The four
investigators that we spoke with from another modal office agreed that
staff meetings occur infrequently and that they do not receive information
on new policies from their managers. Further, they said that new policies
or agency issues are not discussed with staff prior to issuance, and there
was no formal mechanism to provide feedback during the policies'
development. In the past, regular formal meetings occurred between union
leadership and senior NTSB management, which allowed for such input, but
that practice ceased. Although formal communication processes from the
staff level to management are lacking, informal e-mail communications do
take place occasionally between staff and senior management.
17
GAO, Results-Oriented Cultures: Implementation Steps to Assist Mergers and
Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2,
2003).
18
We randomly selected 15 investigators and 8 writer editors evenly across
the 4 modal offices and interviewed them to obtain their views on NTSB's
processes. The views represent the particular individuals and are not
representative of all NTSB investigators and writer editors.
Page 18 GAO-06-801T
Communication and collaboration across offices at all levels can improve
an agency's ability to carry out its mission by providing opportunities to
share best practices and helping to ensure that any needed input is
provided in a timely manner. We found that communication and collaboration
between the Research and Engineering office and the modal offices appears
to be regular. This is shown by the inclusion of Research and Engineering
staff as core members of major investigative teams. Also, our review of
workload in the Research and Engineering office shows a large number of
projects that support all modes, and a Research and Engineering manager
told us that his office frequently interacts with investigative staff.
In contrast, NTSB lacks processes that would allow investigators and
writer editors to communicate across the modal offices regarding the
investigative process and other issues, according to staff we spoke to.
The four investigators that we spoke with from one modal office told us
that they are isolated from the rest of the agency and that lessons
learned are not shared across offices. The investigators from another
modal office told us that they are on permanent teams that share the same
priorities in completing accident analysis, which enhances communication
and teamwork in the office. In addition, in previous years, all writer
editors were located in one group and reported directly to the Managing
Director. Now, each modal office has its own staff of writers and editors.
While they have retained personal working relationships from when they
were located in the same office, four of the eight writer editors we spoke
with said that they no longer share information with each other
regularly.19 As a result, efficiencies and lessons learned that
investigators and writer editor staff in one office might develop might
not be shared with other offices. However, NTSB officials pointed out that
every 6 months writer editors have the opportunity to meet with the
publications specialist for training and to exchange information.
The writer editors held a conference in February 2004.
NTSB Is Accomplishing Its Accident Investigation Mission, but Opportunities
Exist to Gain Efficiencies
While NTSB is accomplishing its accident investigation mission, it faces
challenges that affect the efficiency of the report production and
recommendation close-out processes. In terms of accomplishing its mission,
since its inception, NTSB has investigated over 134,000 transportation
accidents. Eighty-two percent of its recommendations have been "accepted,"
a term NTSB uses to include recommendations that recipients have said they
would implement as well as those that have already been implemented.
Figure 6 shows that highway recommendations have the highest acceptance
rate and marine recommendations have the lowest.
Figure 6: Recommendations and Acceptance Rates of Recommendations by Mode,
1967-May 2006
NTSB Investigations Are Often Lengthy, in Part Because Investigators Must
Launch New Investigations Before Completing Ongoing Investigations
Investigations have four phases-the "launch," fact finding, analysis, and
report production. After a report is issued and recommendations made, the
progress of implementing the recommendations is tracked during a fifth
close-out phase. Figure 7 describes these phases.
Figure 7: Components of an NTSB Investigation and Recommendation Close Out
a
IIC is the "investigator in charge."
b
A group chairman is a technical specialist who is responsible for
developing the facts and analysis for a particular area of an
investigation.
Investigations are often lengthy and sometimes necessarily so. NTSB
routinely takes longer than 2 years to complete major aviation
investigations. For example, the total time to complete major aviation
investigations has increased from an average of about 1.25 years in 1996
to an average of almost 3.5 years in 2006. (See fig. 8.) In 2004, NTSB
contracted with Booz Allen Hamilton to examine and make recommendations to
improve the report development process and the recommendation close-out
process. Booz Allen Hamilton20 reported that the average time to complete
major investigations across all the modes was either 1.8 months or 1.9
months for 4 out of 5 years.21 Lengthy investigations, combined with
lengthy processes for federal agencies to develop regulations based on
those recommendations and industries to implement the recommendations can
work against the goal of improving transportation safety.
Figure 8: Average Duration of Major Aviation Investigations, 1996-2006
Note: Several complex, lengthy investigations were completed in 2000,
including the crash of TWA flight 800, which took over 4 years to
complete.
20 Booz Allen Hamilton, NTSB Organizational Process and Efficiency Study
(Washington, D.C.: Aug. 12, 2004).
21
In fiscal years 1999 and 2002, Booz Allen Hamilton found that the average
time to complete a major investigation was 1.8 years; in fiscal years 2001
and 2003, the average time was 1.9 years; in fiscal year 2000 the average
was 2.4, mainly due to several lengthy aviation investigations that took
over 4 years to complete.
Page 22 GAO-06-801T
One factor that adds to the duration of investigations is that when new
investigations are launched, investigators are pulled from working on
previous accidents to work on new ones. For example, when a major
commercial aviation accident occurs, an NTSB "go team" is dispatched from
Washington D.C., usually within hours of notification of the accident. In
such cases, the team members must leave the investigations they had been
working on to begin fact-finding on the new accident. In the cases of rail
and highway accidents, NTSB investigators must also arrive quickly on
scene to gather information because the accident scenes will be cleared
quickly so that traffic can resume. The manager of one department told us
that all of his ongoing reports would be delayed by 2 months if a sudden
launch were to occur. The number of major investigations that are ongoing
for each mode is shown in figure 9.
Figure 9: Number of Ongoing Major Investigations by Mode, As of February 2006
Note: This table does not include limited aviation investigations, in
which FAA has the primary factfinding role.
Page 23 GAO-06-801T
Writing and Report Production Is a Bottleneck in the Process
Another reason for the expansive time frame for accident investigations is
that reports receive multiple revisions at different levels in the
organization, including the office directors and the Managing Director's
office, prior to going to the board members for final voting and approval
of the draft report. An investigation report typically goes through the
following reviews: the modal office, the Office of Research and
Engineering, the Executive Secretariat, the Office of Safety
Recommendations, the Office of General Counsel, the deputy managing
director, the Managing Director's office, and each board member and the
Chairman. For any review, there may be multiple iterations. Eleven
investigators and 6 writer editors told us that the review process often
results in improved clarity for report recommendations.22 However,
investigators and writer editors also told us that they believe the levels
of management review and approval for written products are excessive. All
eight writer editors agreed that the reviews by the Executive
Secretariat's office, which services a quality assurance function, was a
bottleneck for getting products approved. They told us that it is common
for correspondence and other products to be delayed in this office for 1
week or more, which they viewed as excessive. While it may be a reasonable
expectation for short products, such as correspondence, to be reviewed in
less than a week, that expectation may not be reasonable for reports. Booz
Allen Hamilton confirmed multiple iterations of review as the draft was
routed through numerous offices. On average Booz Allen Hamilton found that
there were 7 levels of reviews within a given modal office that resulted
in an average of 28 separate reviews. A senior NTSB official stated that
the many levels of review were needed to get the appropriate perspectives
from relevant offices that had been involved in report development, such
as the Research and Engineering Office and Safety Recommendation Office.
The official also noted that the process can be streamlined on a
case-by-case basis in which the usual process of sequential reviews is
replaced with concurrent reviews. The NTSB official told us that there are
no explicit criteria for determining when the streamlined process could be
used.
NTSB staff with whom we spoke reported that resource issues contributed to
other bottlenecks. For example, four writer editors pointed out that NTSB
has only one final layout and typesetting person. As of May 2006, the
final layout process had a backlog of approximately 10 reports that have
22
Booz Allen Hamilton, however, found that the logic and rationale for
changes made during the review process were not transparent.
Page 24 GAO-06-801T
Certain Agency Practices May Help Shorten Report Development
been approved for issuance at board meetings but have not yet been
published. NTSB adopts about 2 reports a month and issues on average 4
reports a month. In addition, some investigators have the perception that
the workload of writer editors is another bottleneck. For example, one
investigator told us that he submitted draft reports to the senior writer
editor in September 2005, and as of April 2006, no additional writing had
been done on his project. Writer editors from each modal office told us
they typically worked on five or more products at one time.
NTSB has recently taken several actions that, along with potentially
better practices in one modal office, may help shorten report development
time. First, in response to a recommendation by Booz Allen Hamilton to
gain management's buy-in to the report message before writing the report
and thereby reduce the number of review iterations, NTSB management has
reemphasized its policy for report development meetings. NTSB has a
long-standing order that calls for holding message development meetings
with internal stakeholders who will be reviewing the report prior to
report writing. According to a senior NTSB official, however, the agency
had stopped following that policy before Booz Allen Hamilton conducted its
study in 2004. The official further stated that subsequent to that
recommendation, NTSB's managing director sent a memorandum reminding staff
to follow the policy. While NTSB has no data on whether the message
development meetings are actually taking place, officials told us that the
managing director's recent emphasis on these meetings was resulting in
more of them occurring than in previous years.
Second, since the spring of 2005, NTSB has initiated production meetings
with senior management with the goal of reducing the duration of
investigations. These meetings occur every 2 weeks and focus on report
development and production. NTSB modal directors are held accountable for
a specific issuance date within a six month planning window prior to
issuing a report. During the biweekly meetings, the directors discuss with
NTSB's Managing Director and senior executives their progress and
commitments to complete the investigations. The meetings result in a
production schedule that is available for subsequent review. The modal
directors stated that they believe the new system is effective in reducing
the duration of investigations; however because these meetings began so
recently, it is too early to evaluate their effectiveness.
Third, the highway office-which has the swiftest rate of accident
investigation completion-uses a concept called a "project manager," who
serves as a supervisory writer editor and interface between the
Page 25 GAO-06-801T
Safety Recommendations Close-out Process Is Time Consuming for Several
Reasons
investigative staff and the writer editor staff. As a result, the project
manager assumes some of the report development roles typically supported
by the investigators-in-charge. In comparison, investigators-in-charge in
the marine and rail, pipeline, and hazardous materials offices submit a
draft report to the writer editor, who then edits and sometimes
substantially rewrites the report. In aviation, investigators-in-charge do
not write reports, but rather writer editors develop the final report from
interim technical reports drafted by specialists on the team. Booz Allen
Hamilton recommended that all modes use a project manager or deputy
investigator-in-charge so that the expertise of staff can be used more
fully. In addition, such a practice might alleviate some of the workload
issues that writer editors face as they complete multiple reports. NTSB
managers told us that they agree with this recommendation, but they have
not implemented it or developed any milestones for implementation.
Fourth, the highway safety office uses an incentive system for performance
on developing reports. Booz Allen Hamilton reported the highway safety
office rewards staff with a cash bonus for meeting key deadlines for
producing accident reports. Again, the study recommended that the highway
program be used as a model for the other modal offices. The study further
recommended that the incentive program be slightly modified so that the
incentives are based on delivering reports before deadlines, rather than
meeting deadlines. In that way, the average time standard would be
tightened and the overall report development time would be shortened.
According to NTSB officials, they are currently examining how to implement
improved awards and incentive programs that will result in improved
quality and timeliness of report products.
The processes for federal transportation agencies to implement NTSB's
safety recommendations, and for NTSB to change the status of
recommendations it has made, are also lengthy because of complex processes
involving many players. As of May 2006, 305 of NTSB's 852 open
recommendations had been open for 5 years or more. Lengthy processes for
federal agencies to develop regulations to implement NTSB's safety
recommendations and industries to comply can work against the goal of
quickly improving transportation safety. In addition, the lengthy,
paperbased process for changing the status of recommendations ties up
NTSB's scarce resources.
The length of time that NTSB recommendations remain open is due, in part,
to challenges faced by federal transportation agencies in implementing
those recommendations, particularly those that require
Page 26 GAO-06-801T
changes to federal regulations, which take many years to complete. DOT
modal officials with whom we spoke cited a lengthy rule-making process,
which includes budgeting and allocating resources to develop the proposed
regulation, drafting and receiving comments on proposed rules, and waiting
for the industry's subsequent response to implement the final rule. For
example, TWA flight 800 crashed off Long Island in July 1996; NTSB issued
safety recommendations pertaining to explosive fuel tanks in December
1996. NTSB adopted the accident report with further recommendations to FAA
to reduce flammable vapors in aircraft fuel tanks in 2000; FAA issued a
notice of proposed rule to address this recommendation in November 2005;
the comment period for the notice ended on March 23, 2006. Thus, 10 years
after the crash, the final rule has not been issued. Federal
transportation officials also said the failure to satisfy a cost-benefit
analysis might impede the implementation of NTSB recommendations. Although
NTSB is required to only consider the safety implications of its
recommendations and not consider the cost factors, if a proposed
regulation is not cost beneficial, it cannot be approved by OMB.
Federal officials with whom we spoke at DOT, which receives the bulk of
NTSB recommendations, indicated that they have been working with NTSB to
find acceptable means of implementing recommendations. The
process-recently called Safety With a Team-is designed for NTSB and
federal agencies to work in cooperation to address open recommendations
and implement needed safety improvements. NTSB and DOT officials told us
that this process contributed to the closing of many recommendations.
However, the process is not used with the Coast Guard, which has the
lowest rate-74 percent-for accepting NTSB recommendations among the modes,
as mentioned previously. According to a Coast Guard official we spoke
with, the Coast Guard believes that it has an acceptable rate for closing
NTSB recommendations and that it does not intend to act on recommendations
that it deemed unnecessary.
NTSB recognizes that open recommendations can have serious safety
implications for the transportation industry. To spur implementation, the
agency also publishes a "most wanted" list of what it considers the most
serious safety concerns. For example, in 2000 NTSB added to its most
wanted list the need to improve the safety of motor carrier operations.
NTSB recommended that FMCSA prevent motor carriers from operating if they
put vehicles with mechanical problems on the road or unqualified drivers
behind the wheel. As recently as May 2006, NTSB issued an additional
recommendation that FMCSA "establish a program to verify that motor
carriers have ceased operations after the effective date of revocation of
operating authority."
Page 27 GAO-06-801T
The process that NTSB uses to change the status of or close out safety
recommendations is paper-based, labor intensive, and relies on a series of
sequential reviews; this process can take between 6 and 12 weeks. As a
result, NTSB is delayed in communicating with agencies on whether NTSB
considers the actions that have been taken to address the recommendation
are sufficient to accept the recommendation. Consequently, agencies remain
unaware that their response has been accepted or not accepted. And in the
case of DOT, this lack of information affects its ability to accurately
report annually to Congress on the status of implementing NTSB's
recommendations in all its modal administrations.23
The process of closing recommendations is managed by NTSB's Safety
Recommendation Office, which has responsibility for maintaining a
recommendations database and administering the paper flow to change the
status of recommendations. Adding complexity to the process-which NTSB
calls the "mail control process"-is the fact that there are 12 separate
categories of recommendations status. The 12 categories are listed in
figure 10, which also shows the percentage of recommendations in each
category as of May 1, 2006.
49 USC Sec. 1135(d). NTSB pointed out that for those recommendations on
the Most Wanted List, it specifically updates the list each November to
ensure sufficient time for DOT to file its annual report to Congress.
Page 28 GAO-06-801T
Note: NTSB issued 12,471 recommendations from 1967 to May 1, 2006.
The process begins when NTSB receives documentation from the
recommendation recipient that would change the recommendation's status.
The Safety Recommendation Office generates paper folders and supervises a
process that is summarized in figure 11. This process involves multiple,
sequential approvals starting from the Safety Recommendation Office, to
the modal offices and Research and Engineering Office, to the Managing
Director's office, to the board members for final approval. Since none of
these reviews happen concurrently, some 150 folders are in process at any
given time, according to the director of the Safety Recommendations
Office. There are no electronic communications or
Page 29 GAO-06-801T
approvals throughout the process. In its study of NTSB, Booz Allen
Hamilton identified this as an inefficient process. Officials at NTSB
agree that efficiencies could be gained in this process and are
considering eventually computerizing a number of processes such as this
one. The agency expects to develop such plans after hiring a chief
information officer later this year.
NTSB's Academy Does Not Generate Sufficient Revenues to Cover Costs and Is Not
Fully Utilized
Although there is no statutory requirement that revenues from NTSB's
academy would generate sufficient revenues to cover the costs, in July
2005, NTSB was encouraged in the Senate report accompanying the Fiscal
Year 2006 DOT Appropriations Act to be more aggressive in imposing and
collecting fees to cover the costs.24 The academy generates revenues
through tuition fees, space rental to other agencies for events such as
conferences, and contracts with federal agencies that would allow them to
use academy space for "continuity of operations" in emergency situations.
To the extent that NTSB maximizes the use of the academy, it can produce
additional revenues that may help cover costs.
24
Senate Report 109-109 accompanying P.L. 109-115, the Transportation,
Treasury, the Judiciary, Housing and Urban Development, and Related
Agencies Appropriations Act of 2006.
Page 30 GAO-06-801T
Academy Costs Have Exceeded Revenues
For the first 2 full years of operation, fiscal years 2004 and 2005,
NTSB's academy did not generate sufficient revenues to cover the costs of
providing training, as shown in table 1. As a result, those portions of
the academy's costs that were not covered by the revenues from tuition and
other sources-approximately $6.3 million in fiscal year 2004 and $3.9
million in fiscal year 2005-were offset by general appropriations to the
agency. The salaries and other personnel related expenses associated with
NTSB investigators and managers teaching at the academy, which would be
appropriate to include in academy costs, are not included in table 1
because NTSB told us that it does not choose to account for expenses in
that manner. In addition, NTSB lacks a full cost-accounting system that
would facilitate doing so. The table shows expenses directly associated
with the academy and does not include an allocation of agency wide
supporting services, such as the Managing Director's office, information
technology, human resources, and legal support. Some of the expenses
during these 2 years were one-time expenses-such as over $125,000 for
furniture and equipment (included in table 1 as office supplies for fiscal
year 2005) and $499,000 to move the wreckage of the TWA flight 800
airplane from storage near the crash site in New York to the academy
(included in the table as miscellaneous government contract services in
fiscal year 2004). Space rental is a fixed annual expense of about $2.5
million. When that fixed expense is excluded from academy expenses, the
remaining operating expenses exceeded revenues by about $3.7 million in
fiscal year 2004 and about $1.4 million the subsequent year.
Table 1: Direct Expenses and Revenues for the NTSB Academy, Fiscal Years
2004 and 2005 (unaudited)
Percentage FY 2004 FY 2005 difference
Personnel related $1,011,716 $978,591 -3%
Travel $24,428 $56,912 133%
Space rentala $2,521,440 $2,500,896 -1%
Maintenance/repair of buildings $706,279 $238,203 -66%
Miscellaneous government contract $2,204,880 $558,540 -75% services
Office supplies $12,939 $153,249 1084%
Miscellaneous expensesb $29,320 $28,887 -1%
Total expenses $6,511,002 $4,515,278 -31%
Earned revenue $258,760 $634,800 145%
Overall deficit -$6,252,242 -$3,880,478 -38%
Deficit when space rental expense is excluded -$3,730,802 -$1,379,582 -63%
Source: NTSB.
a
NTSB leases the academy facility from George Washington University under a
20-year lease that will expire in 2021.
b
Miscellaneous expenses such as telephone, mail, and photography services
and printing.
In addition, while some courses presented during the first 2 years of
academy operation did not recover the costs that NTSB attributes to them,
revenues from other courses exceeded the cost. Of the 49 class sessions
provided at the academy in fiscal years 2004 and 2005, revenues from 14
sessions, all of which occurred in fiscal year 2005, did not recover their
cost, while revenues from the remaining sessions exceeded the cost.25
According to the academy's deputy manager, courses are only expected to
generate enough revenues to offset the costs specifically attributed to
the course, with some additional allocation for research and development
of other programs and, if possible, other academy costs. Accordingly,
tuition prices are determined by estimating those costs (such as course
material, contracted instructors and their travel expenses) and dividing
that cost by the projected class size. Costs such as the building lease,
maintenance, building security, and academy personnel are not allocated to
the costs of
The revenue deficient for the 14 sessions totaled $54,279, and the revenue
surplus for the two years totaled $307,203.
Page 32 GAO-06-801T
individual courses.26 In addition, consideration is given to setting
tuition at a level that is competitive with similar courses by other
institutions and that is not prohibitively high for prospective students
from government agencies, according to the academy official.
Other sources of revenue are needed for NTSB to be able to recover the
full costs of the academy. For fiscal year 2004, over $12,000 in revenue
(about 5 percent of total revenues) was collected from sources other than
course fees to cover some of those costs. For fiscal year 2005, the
revenue from other sources increased to over $91,000 (about 14 percent of
total revenues). Other sources of income during these 2 years included
renting space to other organizations, such as the Society of Automotive
Engineers, George Washington University, and the National Association of
State Boating Law Administrators for meetings, conferences, and boat
storage. In addition, NTSB has contracted with two agencies-the Federal
Energy Regulatory Commission and the Virginia Circuit Courts-for
continuity of operations. According to NTSB officials, it has explored
this option with other organizations, but has not found others who will
pay a yearly retainer for the service.27 While NTSB has taken action to
generate revenue from other sources, it does not have a business plan or
marketing strategy that seeks to optimize opportunities for additional
revenues. According to the academy's deputy manager, NTSB plans to develop
a business plan. The agency, however, has no timeframes for doing so.
Our analysis of the academy lease indicates that NTSB has the flexibility
to use the facility in other ways to generate revenues or potentially
reduce costs. For example, the lease does not preclude NTSB from
subletting unused space to other users. Since certain space is already
configured as classrooms and the academy is located in an academic setting
on George Washington University's suburban Virginia campus, it may be
possible to market space to academic users. Furthermore, NTSB is not
precluded by its academy lease or its lease for headquarters space in
Washington, D.C., from relocating some headquarters staff to the Virginia
facility. The lease
26
If the tuition fee is set by dividing the costs attributable to a course
by the projected class size, the fee may not be competitive with fees
charged by other institutions offering similar courses. In that case, the
projected class size might not be attainable without lowering the tuition
to a competitive level, with the result that fee revenues collected might
not cover the attributable costs.
27
NTSB has a memorandum of understanding with GAO for the two agencies to
reciprocate in providing continuity of operations. There is no annual fee
associated with this agreement, only cost reimbursement after the first 14
days of providing space.
Page 33 GAO-06-801T
Academy Classrooms Are Significantly Underutilized
for the office space in Washington, D.C., expires in 2011. Such a move,
however, would incur one-time costs that include relocating staff, moving
furniture and equipment, reconfiguring space and utilities as well as
recurring travel costs for staff who must travel between the two
locations. Such costs would have to be weighed against the reduced cost of
leasing less space in Washington, D.C.
NTSB has not maximized the use of the facility, which could generate
additional revenues that may help cover costs.28 We estimate that,
overall, less than 10 percent of the total classroom space was used during
fiscal year 2005.29 As shown in figure 12, none of the five classrooms
were used for 21 weeks in fiscal year 2005. In addition, at any given
time, no more than three classrooms were in use. Figure 12 shows the days
in which classroom space was used for 31 class sessions and 12 other
events, such as workshops and seminars by organizations that rented the
space during fiscal year 2005.
28
The academy facility contains five classrooms, a large warehouse that
houses aircraft and other wreckage, eating and lounge areas, and office
space for five employees who constitute NTSB's Washington field office.
29
We excluded federal holidays and the last week in December from our
analysis. In some cases, courses used multiple classrooms. We lacked
specific information on which courses used multiple classrooms. To account
for that situation, we rounded up the percentage of space utilized. The
use of multiple classrooms does not affect the information on the lack of
using any classrooms for 21 weeks.
Page 34 GAO-06-801T
Note: Bars indicate classroom use. For example, two stacked bars indicate
that two classrooms were in use on a particular day.
While a relatively small percentage of the academy's students have been
NTSB staff, the agency is taking efforts to increase their enrollment at
the academy. About 20 percent of the academy's approximately 1,000
students30 in fiscal year 2004 were NTSB staff, and about 14 percent of
the 1,400 students in fiscal year 2005 were NTSB staff. Over the 2 years,
about 400 NTSB students31 attended 38 of the 49 class sessions conducted
at the academy during fiscal years 2004 and 2005. (See fig. 13) NTSB is
making efforts to have staff more fully utilize the facility. In fiscal
year 2004, 1 of 18 sessions was only for NTSB investigators; in fiscal
year 2005, 5 of 31 sessions were only for NTSB investigators.32 While
increasing the use of the academy by NTSB staff would reduce the costs of
sending them to external training, it is important that NTSB not reduce
the number of external, paying students in the process.
30
The total number of students is the sum of the participants in all
classes. Individuals who attended more than one class at the academy were,
therefore, counted multiple times.
31
Individuals that attend more than one class are counted multiple times.
32
These course sessions were Conducting Effective Technical Presentations;
two sessions each of Media Training and Major Investigation Protocol and
Processes; and a joint training class with the Federal Bureau of
Investigation.
Page 36 GAO-06-801T
NTSB staff receive most of their training from outside the academy, which
may be due to the courses lacking the subject matter that they require.
Our analysis of staff training requests for fiscal year 2006 showed that
97 percent of all training is expected to be from external sources and the
remaining training from NTSB's academy. NTSB staff have requested external
training being provided by organizations that include FAA's Transportation
Safety Institute, the University of Southern California, the
U.S. Department of Agriculture, and Kettering University for training in
subjects such as human factors in aviation safety, turbine engine
investigation, or automotive design and safety. Training requests cover
other specialties such as helicopter training, flight training currency
for pilots, technical writing, supervisory and management skills, and
industry conferences. Investigators and writer editors with whom we spoke
had positive views on the quality of academy training courses but provided
several reasons for not taking further courses there. Ten of the 23
investigators and writer editors we interviewed told us that they had
taken (or taught) courses at the academy and thought the courses were
excellent;33 none of the investigators and writer editors had anything
negative to say about the quality of any academy course. However, none of
the staff we talked with had plans to attend academy training in fiscal
year 2007. One reason noted for this situation was the remoteness of
Ashburn, Virginia, from their residences. Another reason was the lack of
courses on new transportation technologies and the skills and competencies
needed by an investigator-in-charge. Eight investigators told us that they
find workshops by manufacturers, such as aircraft and automobile
manufacturers, more valuable to their work than academy training.
The academy is not utilized more by NTSB staff, in part, because the
agency has not developed a core curriculum for its staff that could then
be offered at the academy, as mentioned previously in this testimony. The
academy only offers one course that is required for NTSB staff--a 2-week
course on aviation accident investigation that is required for new NTSB
investigator staff. The deputy manager of the academy told us that the
academy plans to eventually offer more internal training covering subjects
such as management skills, retirement, and computers.34 However, no
milestones or specific plans have been established for that effort.
Although most students at the academy are from outside NTSB, several
factors can affect the agency's ability to attract additional outside
students. First, the lack of a business or marketing plan may be affecting
NTSB's ability to fully utilize the academy. Second, academy training is
similar to training provided by other institutions. FRA, FAA, and PHMSA
officials told us that their investigators do not attend NTSB training
because similar training is provided in-house by DOT's Transportation
Safety Institute. For example, an FAA investigator told us that new
investigators take a basic accident investigation course at the
Transportation Safety Institute and subsequently take mid-career followup
courses there. Furthermore, our comparison of NTSB's fiscal year 2006
curriculum with that of several other institutions that teach courses on
accident investigations showed that other institutions offered courses
similar to 12 of NTSB's 19 courses. For example, DOT's Transportation
Safety Institute offers basic courses on aviation and bus accident
33
Our review of course evaluations for fiscal years 2004 and 2005 indicated
high positive responses by students to the academy courses. The data
lacked information for us to compare evaluations by NTSB students and
non-NTSB students.
34
NTSB is considering contracting out more courses such as these.
Page 38 GAO-06-801T
Additional Issues Concerning the Academy
investigations, and the University of Southern California offers a course
on human factors related to accident investigations.
You asked that we provide information concerning the academy's use of NTSB
investigators as instructors and NTSB's compliance with the
Anti-Deficiency Act, with regard to its accounting for its academy lease.
Concerning the first issue, academy courses are taught by a combination of
academy staff, NTSB investigators and managers, and contractors. Use of
investigators as instructors is limited and is likely to have little
impact on investigators' overall workload. During fiscal year 2005, 51
NTSB investigators or managers taught at the academy. On average they
spent an estimated 22 hours to prepare for and teach courses. (See fig.
14.)
Note: Other instructors include NTSB academy staff, volunteers, and
contract instructors from outside NTSB. Data on the number of hours
"other" instructors spent preparing to teach is not collected by NTSB.
Finally, NTSB classified its lease for the academy as an operating lease
rather than a capital lease. As a result, NTSB has been noncompliant with
the Anti-Deficiency Act because it did not obtain budget authority for the
net present value of the entire 20-year lease obligation at the time the
lease agreement was signed in 2001. NTSB realized the error in 2003 and
reported its noncompliance to Congress and the President. NTSB has
proposed in the President's fiscal year 2007 budget to remedy this
antideficiency act violation by inserting an amendment in their fiscal
year 2007 appropriation that would allow NTSB to fund this obligation from
their salaries and expense account through fiscal year 2020.
Conclusions
Mr. Chairman, we have developed several conclusions from our analysis of
NTSB to date. To the credit of the current leadership at NTSB, much of the
agency's progress toward following leading practices is due to recent
management initiatives. The performance management plan, draft staffing
plan, and implementation of controls over financial transactions are all
positive steps. NTSB's progress in these areas will likely remain
incomplete without additional actions, however. For example, without a
more comprehensive strategic plan than it currently has, NTSB cannot align
staffing, training, or other human resource management to its strategic
goals or align its organizational structure and layers of management with
the plan. NTSB will also likely miss opportunities to strengthen the
management of the agency until it develops a strategic training plan for
its employees, implements a full cost-accounting system, and improves
communications within the agency.
We have also concluded that, despite the many safety recommendations NTSB
has made and seen implemented over the years of its existence,
inefficiencies have resulted from the process that the agency uses to
close out safety recommendations. In particular, the absence of a
computerized documentation system and the sequential reviews that NTSB
currently requires slow the process and prevent expedient delivery of
information about recommendation status to affected agencies. Finally, in
terms of its academy, NTSB is missing opportunities to increase the value
of this asset. Without a comprehensive marketing plan, NTSB will likely be
unable to efficiently attract users who would help pay the ongoing costs
of the facility.
Recommendations for Executive Action
o
To improve the efficiency of agency operations, we are making eight
recommendations to the Chairman of the National Transportation Safety
Board based on our completed work to date. To improve agency performance
in the key functional management areas of strategic planning, human
capital planning, financial management, and communications, we recommend
that the Chairman implement the following three recommendations:
Improve strategic planning by developing a revised strategic plan that
follows performance-based practices; developing a strategic training plan
that is aligned with the revised strategic plan and identifies skill gaps
that pose obstacles to meeting the agency's strategic goals and curriculum
that would eliminate these gaps; and aligning their organizational
structure to
Page 41 GAO-06-801T
implement the strategic plan and eliminate unnecessary management layers.
o Develop a full cost-accounting system that would track the amount of
time employees spend on each investigation and in training.
* Develop mechanisms that will facilitate communications from
staff-level employees to senior management, including
consideration of contracting out a confidential employee survey
to obtain employee feedback on management initiatives.
* To enhance the efficiency of the report development and
recommendation close-out processes, we recommend that the
Chairman take the following two actions:
o Identify better practices in the agency and apply them to all modes.
Consider such things as using project managers or deputy
investigators-in-charge in all modes, using incentives to encourage
performance in report development, and examining the layers of review
to find ways to streamline the process, such as eliminating some
levels of review and using concurrent reviews as appropriate.
* Improve the efficiency of the review process for changing the
status of recommendations by computerizing the documentation and
implementing concurrent reviews.
* To enhance the utilization of the academy and improve the ability
to generate revenues that will cover academy costs, we recommend
that the Chairman take the following three actions:
o Develop a comprehensive marketing plan for the academy. The plan
should consider such things as outreach to potential users, working
with USDA and GSA to market it as classroom and conference space, and
conducting market research for additional curriculum development. If
ethical and conflict-of-interest issues can be addressed, the plan
should also consider options for allowing transportation manufacturers
to conduct company-sponsored symposia and technical training at the
academy facility, which would benefit NTSB investigators in keeping up
with new technologies. In addition the plan should consider the
feasibility of subleasing a portion of the academy space.
o Develop core investigator curriculum for each mode and maximize the
delivery of that training at the academy.
Page 42 GAO-06-801T
o Conduct a study to determine the costs and feasibility of moving
certain functions from headquarters to the academy facility in preparation
for the renegotiation of the headquarters lease, which expires in 2011.
We obtained comments on a draft of this testimony from NTSB. NTSB's
Managing Director concurred with our recommendations and provided
clarifying comments and technical corrections, which we incorporated as
appropriate. In addition, NTSB commented that the draft did not
sufficiently distinguish improvements that have been made over the past
year. We revised the testimony to more clearly distinguish those actions.
Scope and Methodology
To determine the extent to which NTSB is following leading practices in
selected management areas, we reviewed past GAO work on leading management
practices in the areas of strategic planning, performance management,
human capital management, financial management, and communications. We
interviewed NSTB board members, senior officials, managers, investigators,
and writer editors regarding their experience with those practices at
NTSB, and their perceptions of the effectiveness of those practices. We
also determined NTSB's response to recommendations made by the DOTIG. We
reviewed NTSB documents, including its strategic, staffing, and
performance management plans; management advisory e-mail; and information
regarding the current staffing levels; and employees' training plans for
2006.
To determine the extent to which NTSB is developing accident investigation
reports and closing safety recommendations in an efficient manner, we
interviewed NTSB investigators, writer editors, managers, and senior
officials regarding the investigative process and their role in it. We
randomly selected 15 of the 210 investigators and 8 writer editors evenly
across the 4 modal offices. The views represent the particular individuals
and are not representative of all NTSB investigators and writer editors.
We reviewed policy guidance on the investigative process and the level of
current and past investigation activity. We examined data on
recommendations acceptance rates and close-out status from NTSB's
recommendation database, and we determined that the data were sufficiently
reliable for the objectives of this review. Additionally, we reviewed
studies done by the Rand Corporation and Booz Allen Hamilton that examined
NTSB's investigation process and determined the extent to which the agency
had implemented their recommendations.
To determine the extent to which NTSB is generating sufficient revenues to
cover costs at its academy, we reviewed financial data on NTSB's academy,
including the revenues and expenses for fiscal years 2004 and 2005. We
reviewed the course curriculum of the academy, and compared it with
classes offered by DOT's Transportation Safety Institute, Embry Riddle,
the University of Southern California, and the Southern California Safety
Institute. We examined data on the student makeup of academy classes and
analyzed data on the preparatory and teaching time used by NTSB
investigators who taught at the academy. We interviewed NTSB
investigators, writer editors, and managers and senior officials at DOT's
modal administrations regarding their current and planned use of the
academy. Finally, we examined the lease for the academy to determine how
NTSB may utilize the space.
We conducted our review from December 2005 to May 2006 in accordance with
generally accepted government auditing standards.
Contacts and Acknowledgments
For further information on this testimony, please contact Dr. Gerald
Dillingham at (202) 512-2834 or by e-mail at dillinghamg@gao.gov.
Individuals making key contributions to this testimony
include Teresa Spisak, Colin Fallon, Eric Fielding, Tom Keightley, Maren
McAvoy, Josh Ormond, and Jena Whitley.
(540115)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
Public Affairs
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON
RECYCLED PAPER
*** End of document. ***