-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO/RCED-00-258R		

TITLE:     Government Performance and Results Act: Information on FAA's 
Science Activities in DOT's Performance Report for Fiscal Year 1999, 
Performance Plan for FY 2001, and July 2000 Draft Strategic Plan

DATE:   08/18/2000 
				                                                                         
----------------------------------------------------------------- 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO/RCED-00-258R

FAA's Science Activities in DOT's Performance Report/
Plan United States General Accounting Office

Washington, DC 20548 Resources, Community, and

Economic Development Division

B- 285954 August 18, 2000 James L. Sensenbrenner Chairman, Committee on
Science House of Representatives

Subject: Government Performance and Results Act: Information on FAA's
Science Activities in DOT's Performance Report for Fiscal Year 1999,
Performance Plan for Fiscal Year 2001, and July 2000 Draft Strategic Plan

Under the Government Performance and Results Act (GPRA) of 1993, the
Department of Transportation (DOT) issued its first 5- year strategic plan
in 1997. That plan established five strategic goals related to safety,
mobility, economic growth and trade, the environment, and national security.
Since fiscal year 1999, DOT has also issued annual performance plans that
set annual performance goals and are intended to provide a direct link
between its long- term strategic goals and day- to- day activities. GPRA
further requires annual performance reports to report subsequently on the
degree to which those annual goals were met. In March 2000, DOT issued its
fiscal year 1999 performance report and fiscal year 2001 performance plan as
a single document, to which we refer hereafter as the performance report/
plan. In addition, DOT is revising its 5- year strategic plan and has made
drafts of that plan available to the Congress and other stakeholders for
comment and consultation. The revised plan is due to be issued on September
30, 2000, and would cover fiscal years 2000 through 2005. These documents
include information on the Federal Aviation Administration's (FAA) science
activities, such as the research and development of landing and navigational
aids and research concerning aviation medicine. To facilitate your review of
these documents and oversight of these activities, you asked us to provide
information on

ï¿½ how DOT's fiscal year 1999 performance report/ fiscal year 2001
performance plan addresses FAA's science activities;

ï¿½ the extent to which FAA's science activities in the President's fiscal
year 2001 budget request are linked to DOT's fiscal year 2001 performance
plan; and

ï¿½ for FAA's science activities, the extent to which DOT's fiscal year 2001
performance plan addresses weaknesses that we identified in previous
performance plans.

In addition, you asked us to determine the extent to which DOT's July 17,
2000, draft strategic plan complies with GPRA.

B- 285954 2 GAO/ RCED- 00- 258R FAA's Science Activities in DOT's
Performance Report/ Plan DOT's Performance Report/ Plan Includes

Goals and Strategies That Address FAA's Science Activities

DOT's performance report/ plan addresses FAA's science activities by
including information on these activities as explicit goals and as
strategies for achieving other goals. However, not all of FAA's science
activities are explicitly included as goals or strategies in the document,
and as a result, readers may have difficulty identifying these activities.

First, DOT has specific goals for some of FAA's science activities. For
example, DOT's fiscal year 2001 performance plan sets goals for the Wide
Area Augmentation System (WAAS), which is part of a new aviation
navigational system being developed by FAA. 1 To ensure that WAAS has the
necessary reliability for use as a precision navigational aid, FAA will
establish a panel of satellite navigation experts, termed the WAAS Integrity
Performance Panel (WIPP), to identify the changes needed to improve the
navigation system's reliability. The performance plan identifies the
following goals concerning WAAS: (1) complete the WIPP efforts by December
2000, (2) meet with aviation associations by February 2001 to discuss the
results of the WIPP effort, and (3) develop an updated project schedule for
WAAS by April 2001.

Since fiscal year 1999, DOT's performance plans have contained a goal
related to another one of FAA's science activities- developing aviation
security technology- to improve the detection rate for explosives and
weapons that may be carried through metal detectors and concealed in carry-
on baggage. DOT acknowledged that it did not meet its fiscal year 1999 goal
for detecting these dangerous objects carried through metal detectors, but
it believed that it had nearly met its goal for improving their detection in
carry- on baggage. However, we found flaws in FAA's methodology for
computing detection rates and determined that the goal was not met. 2 The
fiscal year 2001 performance plan also contains a goal to install 120
explosive- detection systems for screening checked baggage at U. S. airports
by September 30, 2000.

Second, DOT's performance report/ plan addresses some of FAA's science
activities as strategies to achieve broader goals. For example, DOT plans to
reduce the rate of fatal aviation accidents for commercial air carriers
through, among other things, (1) FAA's research program on aviation
medicine, which works to enhance cabin safety, and (2) FAA's research on
safety technology, which supports its regulatory program by studying such
areas as fire detection equipment and the prevention of engine failures. In
addition, the performance report/ plan discusses how FAA is coordinating its
aviation safety research efforts with the National Aeronautics and Space
Administration (NASA) through the development of an FAA/ NASA integrated
research plan.

1 WAAS consists primarily of a network of ground stations that receive,
process, and validate data from satellite- based global positioning systems
before transmitting these data to pilots. 2 Aviation Security: Long-
Standing Problems Impair Airport Screeners' Performance (GAO/ RCED00- 75,
June 28, 2000).

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 3 In addition, DOT includes “research and development
management” as part of its

corporate management strategies in the performance report/ plan. While
discussion of these strategies broadly encompasses aspects of FAA's science
activities, it does not specifically mention the activities. For example,
according to the performance report, in fiscal year 1999, DOT helped develop
the “National Transportation Science and Technology Strategy”
and related plans. In fiscal year 2001, DOT will update the “DOT
Transportation R& D Plan,” which addresses FAA issues, among other
things, and have the National Research Council conduct an annual peer review
of DOT's research and development program. A DOT official agreed that the
corporate management strategies section did not specifically mention FAA's
science activities, but noted that this section includes references to
other, more specific documents, such as the DOT Transportation R& D Plan.

Although the report/ plan reasonably addresses FAA's science activities, it
does not explicitly include all of FAA's science activities as goals or
strategies. For example, the science activity to develop the Local Area
Augmentation System, which is a navigational aid system, is not mentioned in
the performance report/ plan.

DOT's Performance Plan Is Linked to Its Budget Request for Fiscal Year 2001

DOT's fiscal year 2001 performance plan links FAA's science activities- and
other DOT activities- in its fiscal year 2001 budget request to the five
strategic goals contained in DOT's 1997 strategic plan. Appendix II of the
performance plan shows how the budget requested for each activity is
distributed among these strategic goals. The budget request for DOT includes
science activities in three of FAA's budget accounts. These activities and
accounts, along with the distribution of dollars by strategic goal, are
shown in table 1.

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 4

Table 1: FAA's Science Activities and Estimated Obligations, by Strategic
Goal, Fiscal Year 2001

Dollars in millions

Obligations by strategic goal Account and program activity Safety Mobility

Economic growth and

trade Environment National security

Operations

ï¿½ research and acquisitions

a $197 a a a Facilities and equipment

ï¿½ engineering, development, test and evaluation

$38 $571 a a a Research, engineering, and development

ï¿½ system development and infrastructure

a $25 a a a

ï¿½ capacity and air traffic management technology

a b a a a

ï¿½ weather a $28 a a a

ï¿½ aircraft safety technology $49 a a a

ï¿½ system security technology

a a a a $49

ï¿½ human factors and aviation medicine

$25 a a a a

ï¿½ environment and energy a a a $8 a a Does not apply.

b Rounds to less than $1 million. Source: DOT's 1999 Performance Report 2001
Performance Plan.

DOT's Fiscal Year 2001 Performance Plan Addresses Weaknesses Previously
Identified by GAO

For FAA's science activities, DOT's fiscal year 2001 performance plan
addresses weaknesses that we observed in previous annual plans. DOT's
performance plans for fiscal years 1999 and 2000 did not consistently
explain coordination strategies with outside organizations and include
annual performance goals and measures for addressing the management
challenges facing the Department. The fiscal year 2001 performance plan,
however, explains coordination strategies for annual performance goals
related to FAA's science activities. For example, to improve access to
airports in all types of weather, DOT's performance goal is to increase the
number of published global positioning system (GPS) airport approaches. The
performance plan explains that the basic enabling technology for precision
approaches is the GPS satellite navigation system developed and maintained
by the Department of Defense; the map information will be obtained from the
National Oceanic and Atmospheric Administration; and the Office of National
Geodetic Survey will survey airports to obtain information for locating
airport runways and obstacles near the flight paths for approaching them. In
addition, the performance plan addresses the management challenges that are
relevant to FAA's science activities- air traffic control

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 5 modernization and aviation safety and security- by including fiscal
year 2001

performance goals for each issue.

DOT's Draft Strategic Plan Complies With GPRA but Could Be Improved

DOT's July 17, 2000, draft strategic plan complies with GPRA's requirements
for (1) a mission statement, (2) long- term strategic goals, (3) strategies
for achieving the strategic goals, (4) a linkage between DOT's long- term
strategic goals and annual performance goals, (5) an identification of those
key external factors that could significantly affect the Department's
ability to achieve its strategic goals, and (6) a description of program
evaluations used in establishing or revising the strategic goals. In
addition, the draft plan contains information on cross- cutting functions
with other agencies, as suggested by OMB's guidance to agencies on preparing
and revising strategic plans. 3 Each of these seven areas had weaknesses
that could be improved by more closely following OMB's guidance. However,
some of these weaknesses have been addressed in a revised draft dated August
11, 2000.

Mission Statement GPRA requires that agencies' strategic plans include a
comprehensive mission statement that focuses on the agencies' major function
and operations. DOT's draft strategic plan does so. DOT's mission is to
“serve the United States by ensuring a safe transportation system that
furthers our vital national interests and enhances the quality of life of
the American people.” OMB's guidance states that the mission statement
(1) should focus on the agency's core programs and activities and (2) may
include a discussion of enabling or authorizing legislation. However, DOT's
mission statement does not follow the language in its enabling legislation,
which describes its purpose as developing transportation policies and
programs that, among other things, “contribute to providing fast,
safe, efficient, and convenient transportation at the lowest cost.”
The mission statement does not mention, for example, efficiency, speed, or
cost.

We made a similar observation concerning DOT's 1997 draft strategic plan. In
response to our prior observation, DOT revised its September 1997 strategic
plan to include language from its enabling legislation. However, that
language has been dropped from the fiscal year 2000 draft strategic plan.
According to a DOT official, the language was changed to make the mission
statement more concise and understandable to readers of the report,
including the public and DOT's employees and stakeholders.

Goals and Objectives GPRA requires an agency's strategic plan to contain
long- term strategic goals and objectives for its major functions and
operations. DOT's draft strategic plan meets

3 OMB, Circular No. A- 11, Part 2: Preparation and Submission of Strategic
Plans, Annual Performance Plans and Annual Program Performance Reports (July
2000).

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 6 this requirement- it includes six general goals (safety, mobility,
economic growth,

human and natural environment, national security, and organizational
excellence) 4 and a set of outcomes for each goal. OMB's guidance also calls
for an agency's strategic goals and objectives to be sufficiently precise to
direct and guide staff toward actions that fulfill that agency's mission. In
addition, the guidance states that strategic plans should be complete and
include all required plan elements. However, DOT's draft plan does not
specify which agencies or programs are expected to contribute to achieving
each of these six goals. We made the same observation concerning DOT's 1997
draft strategic plan. In response to our prior observation, the Department
included a table in its September 1997 strategic plan that listed the
agencies and programs that were expected to contribute significantly to each
goal. In response to our observation on the July 2000 draft strategic plan,
DOT added a reference in the draft plan to the programs listed in its annual
performance plan that are expected to contribute to achieving the strategic
goals. A DOT official explained that the Department did not want to
duplicate material that is in other GPRA reports.

Strategies to Achieve Goals and Objectives The draft plan meets GPRA's
requirement to describe the operational processes, skills, technology, and
resources required to meet the plan's goals and objectives. In addition,
OMB's guidance suggests that agencies' strategies for achieving strategic
goals and objectives include a brief description of the steps being taken to
resolve mission- critical management problems. DOT's draft strategic plan
includes the management challenges identified by DOT's Inspector General
(IG) and the management challenges we identified that coincided with the
IG's challenges. DOT's draft strategic plan discusses the management
challenges under each of the strategic goals, unlike the 1997 plan, in which
the management challenges were discussed in a separate section and appendix.
However, the draft strategic plan does not discuss aviation competition- a
management challenge we identified but the IG did not. According to a DOT
official, the draft strategic plan did not include this issue because we had
not updated the list of management challenges since issuing it in 1999. We
consider our challenges valid until they are updated, which is expected to
occur in 2001.

OMB's guidance further states that agencies should outline the process for
communicating goals and objectives throughout the agency and for assigning
accountability to managers and staff for achieving objectives. The draft
strategic plan does not mention these issues. By comparison, DOT's September
1997 strategic plan included a section that described the steps to
disseminate the strategic goals and objectives throughout DOT. In response
to our observation on the July draft strategic plan, DOT revised the draft
plan to include this information.

4 The first five goals were also contained in DOT's 1997 strategic plan; the
sixth goal- organizational excellence- is new in the 2000 draft strategic
plan.

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 7 Relationships Between Long- term Strategic Goals

and Annual Performance Goals GPRA requires that strategic plans describe the
relationship between the strategic goals and objectives in the strategic
plan and the performance goals and indicators in the annual performance
plan. The draft plan accomplishes this by including a table for each
strategic goal that lists the outcomes and performance measures that are
candidates for the performance plan but not the final selections. For
example, one outcome of DOT's national security goal is to reduce the flow
of migrants illegally entering the United States. A candidate performance
measure for that outcome is the rate at which undocumented migrants succeed
in entering the United States over maritime routes.

However, the draft plan does follow OMB's guidance to outline consistently
the relevance and use of annual performance goals in helping to determine
the achievement of certain strategic goals and objectives. The plan does not
(1) consistently include candidate performance measures that are clearly
relevant to the outcomes and are sufficient to measure the progress in
achieving the outcomes or (2) explain how the measures and outcomes are
related. For example, DOT expects to know whether it has achieved its
mobility outcome to increase access to transportation systems for the
individual user by using two candidate performance measures- the percentage
of key transit rail stations that are compliant with the Americans with
Disability Act (ADA) and the percentage of bus fleets that are ADAcompliant.
The draft plan provides no candidate measures that gauge the accessibility
of other transportation modes for all travelers. As another example, the
draft plan indicates that one outcome for its economic growth goal is to
ensure that the Producer Price Index (PPI) for transportation services grows
less rapidly than the overall PPI through the year 2005. Two candidate
measures clearly gauge this outcome- the percentage increase in the PPI for
transportation services and the percentage increase in the overall PPI.
Other candidate measures do not seem relevant on a national scale: the miles
of Appalachian Development Highway System completed, the percentage of
flights that are not subject to air traffic controlpreferred routes, 5 and
the number of days that critical waterways are closed due to ice. 6
According to a DOT official, the draft strategic plan represents a work in
progress, and DOT will continue to develop and improve performance measures
for its fiscal year 2002 performance plan and later plans.

5 DOT has published air traffic control (ATC)- preferred routes for many of
the nation's most heavily traveled air routes to aid air traffic controllers
and ensure accuracy in navigation. These routes can differ significantly
from the routes that pilots or flight planners would normally propose
between two cities. The aim of not assigning ATC- preferred routes is to
give increased flexibility to aircraft, which may translate into improved
scheduling efficiency and reduced flight miles.

6 DOT's 1999 Performance Report 2001 Performance Plan explains that the
measure for flight route flexibility has the potential to improve the
efficiency of aircraft operations in certain situations and that the closure
of waterways during the winter increases transportation costs substantially
and potentially overloads other transportation systems. Such explanations
help show the relevance of the performance measures to the outcomes.

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 8 Key External Factors

GPRA requires strategic plans to identify those key external factors beyond
an agency's control that could significantly affect the achievement of its
long- term strategic goals. The draft plan meets this requirement by listing
the applicable external factors for each strategic goal, but it does not
mention DOT's activities to mitigate these factors. Such information was
included in the September 1997 strategic plan, and we believed that it was
useful. However, OMB's guidance instructs agencies not to include
information on mitigating factors in their revised plans and suggests that
an agency may need to prepare an updated strategic plan if an external
factor begins to significantly affect its achievement of a goal.

OMB's guidance suggests that key external factors that could affect the
achievement of goals could include actions by the Congress. DOT's draft plan
did not indicate any pending legislation that could affect the achievement
of its goals; yet several major programs could be reauthorized during the
period covered by the plan. For instance, the Transportation Equity Act for
the 21 st Century is scheduled to be reauthorized in 2004, and the Federal
Railroad Safety Authorization Act, which expired in 1998, could be
reauthorized during the period covered by the strategic plan. By comparison,
DOT's September 1997 strategic plan listed legislation needed to support its
goals. In response to our observation on the July 2000 draft plan, DOT added
information on statutory reauthorizations that could occur during the period
covered by the strategic plan.

Program Evaluations GPRA calls for "a description of the program evaluations
used in establishing or revising general goals and objectives, with a
schedule for future program evaluations." OMB's guidance suggests that
completed program evaluations that were used in preparing an updated
strategic plan should be briefly described and the effect of these
evaluations on defining the strategic goals and objectives in the strategic
plan should be highlighted. DOT's draft plan describes the findings from
completed evaluations and lists the scheduled evaluations for each strategic
goal. The draft plan also explains the relationship of the completed
evaluations to specific strategies and outcomes for achieving the strategic
goals. For example, the draft plan describes an evaluation of the Coast
Guard's method for allocating boats used in fisheries law enforcement. The
Coast Guard expects to reallocate its boats as a result of the study. The
draft plan explains that this evaluation was considered in developing DOT's
strategy to ensure the readiness, availability, and coordination of
resources to respond to incidents of environmental degradation. A DOT
official said that focusing discussion of the performance evaluations at the
strategy and program level is appropriate because individual performance
evaluations are unlikely to affect the definition of Department- wide goals
and objectives.

Cross- cutting Functions OMB guidance states that the "strategic plans of
agencies participating in a crosscutting program should each describe the
interface between their related programs,

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 9 and outline how individual agency efforts synergistically support
common

endeavors." As noted earlier, OMB's guidance also states that strategic
plans should be complete and include all required plan elements. For each
strategic goal, DOT's draft plan includes a section on cross- cutting
programs that lists their subgoals and the agencies involved. In particular,
the draft plan includes this information for several science programs, such
as aviation safety research. However, the draft plan does not describe the
relationships among the programs nor each agency's specific contribution
toward the goal. For example, under the safety goal, the draft plan
identifies a subgoal to reduce recreational boating fatalities by promoting
safe boating practices. In addition, the draft plan lists the agencies
involved in accomplishing this goal: DOT's U. S. Coast Guard, the U. S. Army
Corps of Engineers, the National Park Service, The Boat U. S. Foundation,
the National Safe Boating Council, and the National Association of State
Boating Law Administrators. No further details are provided. By comparison,
DOT's 1999 Performance Report 2001 Performance Plan indicates for the same
goal that the U. S. Army Corps of Engineers and the National Park Service
manage many recreational lakes that are used by boaters and the Coast Guard
works with states and local governments and safety organizations such as The
Boat U. S. Foundation and the U. S. Power Squadrons to provide boating
education and training programs. However, a DOT official explained that to
keep the strategic plan concise, the Department has not duplicated material
that is in the annual performance plans.

Objectives, Scope, and Methodology

Our objectives were to provide information on (1) how DOT's fiscal year 1999
performance report/ fiscal year 2001 performance plan addresses FAA's
science activities; (2) the extent to which FAA's science activities in the
President's fiscal year 2001 budget request are linked to DOT's fiscal year
2001 performance plan; (3) for FAA's science activities, the extent to which
DOT's fiscal year 2001 performance plan addresses weaknesses that we
identified in previous performance plans; and (4) the extent to which DOT's
July 17, 2000, draft strategic plan complies with GPRA. To address the first
three objectives, we reviewed DOT's 1999 Performance Report 2001 Performance
Plan with respect to FAA's science activities. We defined FAA's science
activities as those that were included in the following three FAA budget
accounts:

ï¿½ engineering, development, test and evaluation activities, such as landing
and navigational aid programs and free flight, 7 which are included in FAA's
facilities and equipment account;

ï¿½ research and acquisitions activities, such as executive oversight and
policy direction for research acquisitions activities, which are included in
FAA's operations account; and

7 “Free flight” is an new system of air traffic management that
moves from the present highly structured rules and procedures for air
traffic operations to a more flexible system, in which decisions for
conducting flight operations will be based on collaboration between FAA and
pilots.

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 10

ï¿½ FAA's entire account for research, engineering, and development, which
includes activities such as human factors and aviation medicine, system
security technology, and weather research.

To address the first objective, we identified examples of FAA's science
activities that were mentioned and those that were not mentioned in DOT's
performance report/ plan. To address the second objective, we compared DOT's
fiscal year 2001 performance plan to the President's fiscal year 2001 budget
request for DOT. To address the third objective, we compared DOT's fiscal
year 2001 performance plan to our observations on the Department's previous
performance plans. 8 To address the fourth objective, we reviewed GPRA and
OMB's guidance on developing strategic plans. We also interviewed an
official in DOT's Office of Policy, which prepares the strategic plan. In
addition, we relied on our knowledge of DOT's September 1997 strategic plan
and its development. 9 It is important to recognize that the final strategic
plan is not due to the Congress until September 30, 2000. Thus, our findings
reflect a “snapshot” of DOT's draft strategic plan at this time.
We recognize that developing a strategic plan is a dynamic process and that
DOT is continuing to revise this plan. We conducted our review from July
through August 2000 in accordance with generally accepted government
auditing standards.

Agency Comments

We provided a draft of this report to the Department of Transportation for
review and comment. We met with Department officials, including the Special
Assistant to the Deputy Secretary, who agreed with the information presented
in this report. They provided technical clarifications, which we
incorporated into the report as appropriate.

8 Results Act: Observations on the Department of Transportation's Annual
Performance Plan for Fiscal Year 1999 (GAO/ RCED- 98- 180R, May 12, 1998)
and Results Act: Observations on the Department of Transportation's Fiscal
Year 2000 Performance Plan (GAO/ RCED- 99- 153, May 7, 1999).

9 We assessed the draft strategic plan that DOT provided to the Congress for
consultation on July 2, 1997, and reported our findings in Results Act:
Observations on the Department of Transportation's Draft Strategic Plan
(GAO/ RCED- 97- 208R, July 30, 1997). We also assessed DOT's final strategic
plan that was submitted to OMB and the Congress on September 30, 1997 (see
Managing for Results: Agencies' Annual Performance Plans Can Help Address
Strategic Planning Challenges (GAO/ GGD98- 44, Jan. 30, 1998).

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 11 ----

As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days after
the date of this report. At that time, we will send copies of this report to
the Honorable Rodney E. Slater, Secretary of Transportation, and will make
copies available to others on request. If you or your staff have any
questions about this report, please call me at (202) 512- 2834. Key
contributors to this report were Katherine Siggerud and Teresa Spisak.

Sincerely yours, Phyllis F. Scheinberg Associate Director, Transportation

Issues

B- 285954

GAO/ RCED- 00- 258R FAA's Science Activities in DOT's Performance Report/
Plan 12 (348241)
*** End of document ***