Aviation Rulemaking: Incomplete Implementation Impaired FAA's
Reform Efforts (11-JUL-01, GAO-01-950T).
This testimony discusses the results of GAO's review of the
Federal Aviation Administration's (FAA) rulemaking process and
ways to improve its efficiency. GAO found that the time FAA took
to formally initiate a rule in response to a congressional
mandate or a National Transportation Safety Board recommendation
varied widely. Between fiscal year 1995 and fiscal year 2000, FAA
initiated most such rules within two years, but some rules were
initiated many years later. Over the entire six-year period of
GAO's review, FAA's median time for the final rule phase--about
15 months--was comparable to that of four other federal
regulatory agencies. Over a shorter, more recent period, FAA took
longer to complete this phase. In 1998, FAA developed reforms
that were designed to address problems affecting the pace of
rulemaking, including the timing of management's involvement, the
administration of the process, and human capital management
issues. However, the reforms have not yet shortened the
rulemaking process because they have not been fully or
effectively implemented. Shifting priorities, some brought about
by external events and some by internal circumstances, have
continued to delay the pace of rulemaking. In addition, difficult
policy issues have sometimes remained unresolved late in the
process, and management has retained several layers of internal
review. Rulemaking participants are not clear about their job
responsibilities, and FAA's updated rulemaking information system
contains consistent data on only the highest-priority rules.
Recommended human capital management initiatives, including the
establishment of a system for measuring and evaluating
performance and creating performance incentives, have not yet
been implemented. This report summarizes a July report
(GAO-01-821).
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-950T
ACCNO: A01387
TITLE: Aviation Rulemaking: Incomplete Implementation Impaired
FAA's Reform Efforts
DATE: 07/11/2001
SUBJECT: Agency proceedings
Air transportation operations
Transportation safety
Regulatory agencies
FAA Flight Operational Quality Assurance
Program
FAA Integrated Rulemaking Management
Information Systems
Federal Agency Major Rules
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-950T
AVIATION RULEMAKING
Incomplete Implementation Impaired FAA's Reform Efforts Statement of Gerald
L. Dillingham, Ph. D.
Director, Physical Infrastructure Issues
United States General Accounting Office
GAO Testimony Before the Subcommittee on Aviation, Committee on
Transportation and Infrastructure, House of Representatives
For Release on Delivery Expected at 2: 00 p. m. EDT Wednesday July 11, 2001
GAO- 01- 950T
1 Mr. Chairman and Members of the Subcommittee:
We are pleased to be here today to discuss the results of our work for you
on the Federal Aviation Administration?s (FAA) rulemaking process and ways
to improve its efficiency. As you know, FAA develops rules, or regulations,
to enhance aviation safety and security and to promote the efficient use of
airspace. In doing so, it must balance opposing pressures. On the one hand,
rulemaking is time- consuming and complex, requiring careful consideration
of the impact of proposed rules on individuals, industries, the economy, and
the environment. On the other hand, threats to public safety and the rapid
pace of technological development in the aviation industry demand timely
action.
Over the past 40 years, numerous reports have documented and assessed the
causes of delays in FAA?s rulemaking process. In 1996, the Congress enacted
legislation that established time frames for publishing significant rules-
that is, rules whose potential impact on the economy and affected parties
meets certain criteria established by the Office of Management and Budget.
In 1997, FAA conducted a study on ways to improve the rulemaking process,
and in 1998, it began implementing reforms to address the factors that
affect the pace of rulemaking and respond to the congressional time frames.
You asked that we review the efficiency of FAA?s rulemaking in light of
these reforms. Accordingly, in our report to you today 1 and in this
statement, we address (1) the time frames for FAA?s rulemaking, (2) the 1998
reforms and their effects on the pace of FAA?s rulemaking, and (3) the
effectiveness of the reforms in addressing the factors that affect the pace
of rulemaking.
Before I summarize the results of our work, let me briefly describe the
phases of rulemaking, the rules, and the time frames we used for our
analysis. As shown in figure 1, rulemaking consists of a preliminary period
prior to initiation and two formal phases
1 Aviation Rulemaking: Further Reform Is Needed to Address Long- standing
Problems (GAO)- 01- 821, July 2001).
2 separated by a public comment period. During the preliminary period, FAA
studies the
issue and decides whether to initiate rulemaking. The first, or proposed
rule, phase extends from the initiation of rulemaking through the
publication for public comment of a proposed rule in the Federal Register.
The second, or final rule, phase begins at the close of the public comment
period and concludes with the publication of a final rule in the Federal
Register.
Figure 1: FAA?s Rulemaking Process for Significant Rules
Source: Based on FAA?s Rulemaking Manual, Dec. 1998.
3 To analyze the time frames for FAA?s rulemaking, we looked at the history
of 76
significant rules that constituted the majority of FAA?s workload of
significant rules from fiscal year 1995 through fiscal year 2000. When we
could identify a starting point for the preliminary period- namely, the date
of a congressional mandate or an NTSB recommendation- we analyzed how long
FAA took to initiate rulemaking. To evaluate the effects of FAA?s 1998
reforms on the pace of the agency?s rulemaking, we compared the median times
that FAA took to complete both the proposed rule phase and the final rule
phase during the 3- year period before the reforms and the 3- year period
after them. We also determined how well FAA met rulemaking time frames
established by statute or its own guidance. Finally, to assess the
effectiveness of FAA?s reforms in addressing the factors that affect the
pace of rulemaking, we surveyed FAA rulemaking staff, interviewed rulemaking
management and staff, and observed rulemaking projects. Let me turn now to
the results of our work.
In summary: The time FAA took to formally initiate a rule in response to a
congressional mandate or an NTSB recommendation varied widely. Between
fiscal year 1995 and fiscal year 2000, FAA initiated most such rules within
2 years, but some rules were initiated many years later. After formally
initiating a rule, FAA took a median time of approximately 2-� years to
complete the rulemaking, although 20 percent of the rules took 10 years or
longer to complete. Over the entire 6- year period of our review, FAA?s
median time for the final rule phase- about 15 months- was comparable to
that of four other federal regulatory agencies. Over a shorter, more recent
period, FAA took longer to complete this phase. From October 1996, when the
Congress established a 16- month statutory requirement for completing the
final rule phase, through March 2001, FAA completed this phase within 16
months for fewer than half of its rulemaking projects.
FAA?s 1998 reforms were designed to address problems affecting the pace of
rulemaking, including the timing of management?s involvement, the
administration of the process, and human capital management issues. However,
the reforms have not yet shortened the
4 rulemaking process. The median times FAA took to complete both the
proposed rule
phase and the final rule phase increased in the 3- year period since the
reforms, even though FAA published fewer rules then than it did during the
3- year period before the reforms.
FAA?s 1998 reforms have not improved the efficiency of the rulemaking
process primarily because they have not been fully or effectively
implemented. Shifting priorities, some brought about by external events and
some by internal circumstances, have continued to delay the pace of
rulemaking. In addition, difficult policy issues have sometimes remained
unresolved late in the process, and management has retained several layers
of internal review. Rulemaking participants are not clear about their job
responsibilities, and FAA?s updated rulemaking information system contains
consistent data on only the highest- priority rules. Recommended human
capital management initiatives, including the establishment of a system for
measuring and evaluating performance and creating performance incentives,
have not yet been implemented.
To strengthen FAA?s rulemaking process, we are recommending management?s
earlier and more effective participation in setting priorities and making
policy decisions; better use of information management systems; and the
implementation of human capital strategies to measure and evaluate
performance and create performance incentives for participants.
FAA?s Rulemaking Time Frames Varied
Between fiscal year 1995 and fiscal year 2000, FAA formally initiated
rulemaking actions within 6 months for about 60 percent of the congressional
mandates and about 33 percent of the NTSB safety recommendations calling for
such actions. However, FAA took more than 5 years for about one- fourth of
the mandates and one- third of the recommendations. Two examples illustrate
the wide variation in FAA?s rulemaking time frames. FAA responded within 1
month to a 1999 NTSB recommendation that flight data recorders be required
on Boeing 737 aircraft, but the agency took more than 7 years to respond to
an NTSB recommendation calling for the use of child safety seats. Figure 2
5 shows the time FAA took to initiate the rulemaking process in response to
20
congressional mandates and 12 NTSB recommendations.
Figure 2: Time Elapsed Between 20 Congressional Mandates and 12 NTSB Safety
Recommendations and FAA?s Initiation of the Rulemaking Process
Note: The time until initiation was measured from the date the legislation
containing a mandate was enacted or the date a safety recommendation was
issued to the initiation date identified in FAA?s rulemaking information
system. No rules in response to congressional mandates were initiated from
24 months up to 60 months after the mandate. Due to rounding, totals may not
add up to 100 percent.
Source: GAO?s analysis of FAA data.
From fiscal year 1995 through fiscal year 2000, FAA completed 29 significant
rules, taking from less than 1 year to almost 15 years for the entire
rulemaking process. The median time for completing a rule was 2-� years, but
six rules took 10 years or more to complete. The median time for the
proposed rule phase was approximately 20 months, and the median time for the
final rule phase was about 15 months. FAA?s median time for the final rule
phase was comparable to that of four other federal regulatory agencies- the
Animal and Plant Health Inspection Service (APHIS), the Environmental
Protection Agency (EPA), the Food and Drug Administration (FDA), and the
National Highway Transportation Safety Administration (NHTSA). Except for
APHIS, which completed
6 this phase for all of its significant rules within 2 years, the agencies
generally completed
this phase for two- thirds to three- fourths of their significant rules
within that time frame. The Federal Aviation Reauthorization Act of 1996
established a 16- month time frame for FAA?s final rule phase. From October
1996 through March 2001, FAA met this deadline for 7 of the 18 significant
rules it completed during this period.
FAA?s Reforms Have Not Expedited Rulemaking as Intended
FAA?s 1998 reforms established organizational structures, guidance, and
systems to improve the efficiency of the rulemaking process. To strengthen
management?s ability to coordinate and set priorities, resolve policy
questions, and streamline the review process, FAA established a rulemaking
steering committee to determine rulemaking priorities and a rulemaking
management council to manage the rulemaking process. In addition, FAA
developed a new rulemaking manual that suggested time frames for each step
in the rulemaking process and included a system for prioritizing rulemaking
projects. FAA also planned to limit reviews to those that added value and
proposed delegating more responsibility for rulemaking decisions to
rulemaking teams.
To strengthen the administration of the rulemaking process, FAA implemented
reforms designed to clarify the roles and responsibilities of rulemaking
staff, improve the monitoring of rules and the management of rulemaking
documents, and ensure ongoing evaluation of the process. The new rulemaking
manual outlined the roles and responsibilities for each member of the
rulemaking team. A new system, the Integrated Rulemaking Management
Information System, was designed to consolidate the functions of the
existing project- tracking and document- management systems. To ensure
ongoing evaluation of the rulemaking process, FAA developed quality
standards and established teams responsible for monitoring and improving the
quality of rulemaking documents and recommending improvements in the process
to the rulemaking management council. Finally, FAA?s 1997 study identified a
need for human capital management strategies to increase accountability
through training, evaluating, and rewarding the rulemaking staff.
7 FAA?s 1998 reforms have not expedited the rulemaking process as intended.
In fact, the
median time FAA took to complete the entire process increased from about 30
months in the 3- year period before the reforms (fiscal years 1995- 1997) to
38 months in the 3- year period since the reforms (fiscal years 1998- 2000).
Furthermore, the median times FAA took for both the proposed rule phase and
the final rule phase increased since FAA began implementing the reforms, as
shown in figure 3.
Figure 3: Median Time FAA Took to Process Significant Proposed and Final
Rules for Periods Before and Since FAA?s Reforms
Source: GAO?s analysis of FAA and Federal Register data.
Figure 3 also shows that FAA has published fewer rules since 1998. In the 3-
year period before the reforms, FAA completed 18 significant rules, compared
with 11 in the 3- year period since the reforms.
Among its reforms, FAA established its own time frames for the rulemaking
process: 450 days for the proposed rule phase and 310 days for the final
rule phase. While these time frames were not an applicable standard for
rulemaking efforts that predated the reforms,
8 we found that FAA met these time frames more often before the reforms than
afterwards. For example, it met its time frame for the proposed rule phase
for 47 percent of its rulemaking projects in the 3- year period before the
reforms but for only 19 percent of its projects in the 3- year period since
the reforms. This is shown in figure 4.
Figure 4: Percentage of FAA?s Significant Proposed and Final Rules Published
Before and Since FAA?s Reforms That Met Time Frames Suggested in FAA?s
Reform Guidance
Note: The time frame suggested in FAA?s guidance for developing proposed
rules is 450 days (about 15 months) while the time frame suggested for final
rules is 310 days (about 10 months and 10 days).
Source: GAO?s analysis of FAA data.
Incomplete Implementation Has Limited Effectiveness of Reforms
FAA?s 1998 reforms have not been fully implemented, and the efficiency of
the process has not improved, according to our survey of FAA rulemaking
staff, interviews with FAA rulemaking management and staff, and observations
of specific rulemaking projects. Fewer than 20 percent of the staff we
surveyed thought that FAA had made the changes necessary to improve the
process, and only about 20 percent believed that the process
9 had become more efficient and effective in the last 2 years. Our
interviews and
observations supported the staff?s views. Three problems associated with
management?s involvement in rulemaking continued to slow the process. First,
an excessive number of top- priority rules impaired FAA?s ability to
allocate resources effectively, contributing to delays. The number of
projects on FAA?s top- priority list grew from 35 in February 1998,
immediately after the reforms, to 49 by March 2001. According to the
Director of the Office of Rulemaking, FAA can effectively manage only about
30 to 35 rulemaking projects. Changes in the informal relative ranking of
?top? priorities created similar problems. According to 83 percent of the
rulemaking staff that responded to our survey, changing priorities caused
delays. Second, unresolved policy issues continued to slow the rulemaking
process, as our study of a rulemaking effort related to FAA?s Flight
Operational Quality Assurance (FOQA) Programs showed. Third, multiple layers
of review within the agency continued to delay the process after the
reforms. We found that for five of six projects approved after the reforms,
team members? decisions were reviewed and approved by the directors of the
team members? offices, their immediate managers, and other managers. These
multiple reviews occurred even though FAA concluded in 1997 that they
fostered a lack of accountability in the rulemaking process that, in turn,
led to milestones that were unrealistic or ignored because final
responsibility for a project was unclear. According to the vast majority of
the rulemaking team members responding to our survey (82 percent), layers of
review interfere with the timely processing of rules.
Problems related to the administration of the rulemaking process have also
persisted since the 1998 reforms, continuing to contribute to delays.
Although the reforms were designed to address confusion over roles and
responsibilities, only about 40 percent of our survey respondents agreed
that their roles and responsibilities were clearly understood. Rulemaking
officials said, for example, that although the rulemaking manual clarified
the specific role of the legal reviewer, legal reviewers sometimes continued
to focus on nonlegal issues.
10 FAA?s rulemaking management information system consolidated the functions
of the
agency?s former project- tracking and document- management systems, but the
system was not being used to its full potential. First, FAA used the
project- tracking portion of the automated system only for its ?A? list of
priority projects, which included 24 significant rules. The system was
missing complete and accurate data for FAA?s other significant rulemaking
projects, limiting FAA?s ability to monitor and evaluate its rulemaking
efforts. FAA rulemaking officials said they did not have the resources
available to complete, correct, or update records of rules that were not
being actively worked on from the agency?s ?A? list of rules. Second, the
document- management portion of the automated system was limited in its
usefulness because it had not been fully implemented across all offices
involved in rulemaking. For example, only 1 of 27 rulemaking staff outside
the Office of Rulemaking who were working on four significant rules included
in our review had used the automated system and that use occurred on only 1
day in February 1998.
The 1998 reforms explicitly called for the establishment of systems to
evaluate the new rulemaking process, but FAA has not fully implemented a
continuous improvement or a quality review program. For example, rulemaking
teams were to complete ?lessons
learned? evaluations to identify problem areas and opportunities to improve
the entire rulemaking process. However, we found that since 1998, FAA has
not documented any evaluations. The continuous improvement team was also
expected to review sample rulemaking documents and perform periodic quality
assurance reviews. None of these quality review functions has been
accomplished.
FAA has generally not implemented planned human capital management reforms,
including rulemaking training, performance measurement, evaluation, and
rewards for efficiency. For example, only about half of our survey
respondents agreed that they had received the training they needed to
perform their jobs. Moreover, despite an agencywide effort to link
performance with rewards in April 2000, FAA management has not established
agencywide systems to measure and reward performance in rulemaking according
to the quality or timeliness of the process. As a result, fewer than half of
our
11 survey respondents (about 48 percent) indicated that senior management
holds team
members accountable when teams do not meet milestones. Only 20 percent
indicated that senior management is held accountable when teams do not meet
milestones.
Finally, the Office of Rulemaking has not developed a separate rulemaking
award system as recommended. Not surprisingly, only 8 percent of our survey
respondents agreed that their offices provide incentives based on the
milestones of the rulemaking process.
Conclusion
In conclusion, Mr. Chairman, we recognize that rulemaking is a complex and
timeconsuming endeavor. We have not yet seen whether FAA?s rulemaking
reforms can effectively reduce delays in the process because many of the
initiatives have not been fully implemented. FAA?s attention to elements
critical to achieving desired results- limiting the number of top- priority
rules and the extent to which established priorities are shifted; maximizing
the use of evaluation systems to establish baseline data and measure the
agency?s progress; and implementing performance incentives for meeting time-
related goals- would facilitate more effective implementation of the
reforms.
Recommendations for Executive Action
In our report, we recommend that the Secretary of Transportation direct the
FAA Administrator to take a number of actions to improve the efficiency of
FAA?s rulemaking process. These actions include the following:
* Reduce the number of top- priority projects to a manageable number and
provide the resources rulemaking teams need to focus their efforts and
manage projects to timely completion. To do this, FAA must, among other
things, hold managers accountable for making policy decisions as early as
possible in the rulemaking process and hold team members accountable for
limiting their reviews to established criteria.
12
* Ensure that the information systems used to track and coordinate
rulemaking contain current, complete, and accurate data on the status of all
significant rulemaking projects and that the proposed continuous improvement
program and the resulting information are used to identify problems in the
process and potential solutions.
* Establish a human capital management strategy for rulemaking that includes
training, performance measures, and incentives that promote timeliness and
quality in rulemaking.
Contacts and Acknowledgement
For future contacts regarding this testimony, please contact Gerald L.
Dillingham at (202) 512- 3650. Individuals making key contributions to this
testimony were Chris Keisling, Sarah Lovering, Jason Schwartz, and Alwynne
Wilbur.
(540001)
*** End of document. ***