The Government Performance and Results Act: 1997 Governmentwide
Implementation Will be Uneven (Chapter Report, 06/02/97, GAO/GGD-97-109).
Pursuant to a legislative requirement, GAO reviewed federal agencies'
implementation of the Government Performance and Results Act of 1993
(GPRA), including the prospects for compliance by federal agencies
beyond those participating as pilots under the act, focusing on: (1) the
status of the act's implementation efforts; (2) significant challenges
confronting executive agencies in their efforts to become more
results-oriented; and (3) ongoing efforts to integrate program, cost,
and budget information into a reporting framework that allows for fuller
consideration of resource allocations, operational costs, and
performance results.
GAO noted that: (1) GPRA's implementation has achieved mixed results
which will lead to highly uneven governmentwide implementation in the
fall of 1997; (2) while agencies are likely to meet the upcoming
statutory deadlines for producing initial strategic plans and annual
performance plans, those documents will not be of a consistently high
quality or as useful for congressional and agency decisionmaking as they
could be; (3) the Office of Management and Budget (OMB) selected over 70
performance planning and reporting pilots that far exceeded the number
required by GPRA and that should provide a rich body of experience for
agencies to draw on in the future; (4) the experiences of some of GPRA's
pilot agencies and related efforts by nonpilot agencies showed that
significant performance improvements were possible, even in the short
term, when an agency adopted a disciplined approach to setting
results-oriented goals, measuring its performance, and using performance
information to improve effectiveness; (5) however, the reported examples
of substantial performance improvements were relatively few; (6) one set
of challenges to effectively implementing GPRA arises from the
complications of government structure and from program proliferation;
(7) others involve methodological difficulties in identifying
performance measures or the lack of data needed to establish goals and
assess performance; (8) the following are among the challenges that GAO
observed: (a) overlapping and fragmented crosscutting program efforts
present the logical need to coordinate efforts to ensure that goals are
consistent and, as appropriate, that programs efforts are mutually
reinforcing; (b) the often limited or indirect influence that the
federal government has in determining whether a desired result is
achieved complicates the effort to identify and measure the discrete
contribution of the federal initiative to a specific program result; (c)
the lack of results-oriented performance information in many agencies
hampers efforts to identify appropriate goals and confidently assess
performance; (d) instilling within agencies an organizational culture
that focuses on results remains a work in progress across the federal
government; and (e) linking agencies' performance plans directly to the
budget process may present significant difficulties; and (9) GAO
believes that GPRA's success or failure should not be judged on whether*
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-97-109
TITLE: The Government Performance and Results Act: 1997
Governmentwide Implementation Will be Uneven
DATE: 06/02/97
SUBJECT: Reporting requirements
Congressional/executive relations
Strategic planning
Mission budgeting
Federal agency reorganization
Interagency relations
Accountability
Agency missions
IDENTIFIER: Senior Executive Service
SES
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Committees
June 1997
THE GOVERNMENT PERFORMANCE AND
RESULTS ACT - 1997 GOVERNMENTWIDE
IMPLEMENTATION WILL BE UNEVEN
GAO/GGD-97-109
Results Act
(410135)
Abbreviations
=============================================================== ABBREV
ARC - Appalachian Regional Commission
ARL - Army Research Laboratory
BEP - Bureau of Engraving and Printing
CFO - Chief Financial Officers
CPDF - Civilian Personnel Data File
DLA - Defense Logistics Agency
DOD - Department of Defense
EDA - Economic Development Administration
EPA - Environmental Protection Agency
FAA - Federal Aviation Administration
FASAB - Federal Accounting Standards Advisory Board
FDA - Food and Drug Administration
GM - General Management
GMRA - Government Management Reform Act
GPRA - Government Performance and Results Act
GS - General Schedule
HHS - Department of Health and Human Services
HUD - Department of Housing and Urban Development
IRS - Internal Revenue Service
JFMIP - Joint Financial Management Improvement Program
MBO - Management by Objectives
NCS - National Cemetery System
NHTSA - National Highway and Traffic Safety Administration
NPR - National Performance Review
OCSE - Office of Child Support Enforcement
OMB - Office of Management and Budget
ONDCP - Office of National Drug Control Policy
OPM - Office of Personnel Management
OSHA - Occupational Safety and Health Administration
PPBS - Planning-Programming-Budgeting System
R&D - Research and Development
SES - Senior Executive Service
SSA - Social Security Administration
USDA - Department of Agriculture
VA - Department of Veterans Affairs
ZBB - Zero-Based Budgeting
Letter
=============================================================== LETTER
B-277071
June 2, 1997
The Honorable Fred Thompson
Chairman
The Honorable John Glenn
Ranking Minority Member
Committee on Governmental Affairs
United States Senate
The Honorable Ted Stevens
Chairman
The Honorable Robert C. Byrd
Ranking Minority Member
Committee on Appropriations
United States Senate
The Honorable Pete V. Domenici
Chairman
The Honorable Frank R. Lautenberg
Ranking Minority Member
Committee on Budget
United States Senate
The Honorable Dan Burton
Chairman
The Honorable Henry A. Waxman
Ranking Minority Member
Committee on Government Reform
and Oversight
House of Representatives
The Honorable Bob Livingston
Chairman
The Honorable David Obey
Ranking Minority Member
Committee on Appropriations
House of Representatives
The Honorable John R. Kasich
Chairman
The Honorable John M. Spratt, Jr.
Ranking Minority Member
Committee on Budget
House of Representatives
This report responds to the requirement in the Government Performance
and Results Act of 1993 that we report to Congress on the
implementation of the Act, including prospects for compliance by
federal agencies beyond those participating as pilots under the Act.
Our specific objectives for this report were to (1) assess the status
of the Act's implementation efforts; (2) identify significant
challenges confronting executive agencies in their efforts to become
more results-oriented; and (3) describe ongoing efforts to integrate
program, cost, and budget information into a reporting framework that
allows for fuller consideration of resource allocations, operational
costs, and performance results.
We are sending copies of this report to other appropriate
congressional committees and executive branch agencies, including the
Director of the Office of Management and Budget and other interested
parties. We also will make copies available to others on request.
If you or your staff have any questions concerning this report, I can
be reached at (202) 512-2637. The major contributors to this report
are listed in appendix II.
L. Nye Stevens
Director, Federal Management
and Workforce Issues
EXECUTIVE SUMMARY
============================================================ Chapter 0
PURPOSE
---------------------------------------------------------- Chapter 0:1
A consensus that the nation's persistent federal deficit must be
addressed and that the effectiveness of federal programs must improve
substantially has spurred widespread efforts in the executive branch
and Congress to dramatically change the way the federal government is
managed. During the 1990s, Congress enacted a broad statutory
framework, with the Government Performance and Results Act of 1993 as
its centerpiece, to improve the management and accountability of
federal agencies. In essence, the Act, which is referred to as
"GPRA" or "the Results Act," seeks to shift the focus of federal
management and decisionmaking away from a preoccupation with the
activities that are undertaken to a focus on the results of those
activities as reflected in citizens' lives.
Congress understood that the management changes required to
effectively implement the Results Act would not come quickly or
easily. The Act therefore included a phased implementation approach
that began in fiscal year 1994 with pilot projects on the Act's
performance planning and reporting requirements. Under the Results
Act, GAO is to report to Congress on the implementation of the Act,
including the prospects for compliance by executive agencies beyond
those that participated in the pilot phase. This report culminates
GAO's efforts to meet that mandate. GAO's specific objectives for
this report were to (1) assess the status of the Results Act's
implementation efforts; (2) identify significant challenges
confronting executive agencies in their efforts to become more
results oriented; and (3) describe ongoing efforts to integrate
program, cost, and budget information into a reporting framework that
allows for fuller consideration of resource allocations, operational
costs, and performance results.
BACKGROUND
---------------------------------------------------------- Chapter 0:2
The Results Act is intended to improve the efficiency and
effectiveness of federal programs by establishing a system to set
goals for program performance and to measure results. Specifically,
the Act requires executive agencies to prepare multiyear strategic
plans, annual performance plans, and annual performance reports. As
a starting point, the Act requires virtually every executive agency
to develop a strategic plan, covering a period of at least 5 years
forward from the fiscal year in which it is submitted. These
strategic plans are to include an agency's mission statement, general
goals and objectives, and the strategies that the agency will use to
achieve those goals and objectives. Agencies are to consult with
Congress and solicit the views of other stakeholders in the
development of those plans and to submit the first strategic plans to
the Office of Management and Budget (OMB) and Congress by September
30, 1997.
Next, the Results Act requires executive agencies to develop annual
performance plans covering each program activity set forth in the
agencies' budgets. The first annual performance plans, covering
fiscal year 1999, are to be provided to Congress after the
President's budget is submitted to Congress in 1998. The annual
performance plan is to contain an agency's annual goals, the measures
that the agency will use to gauge its performance toward meeting
those goals, and the resources the agency will need to meet its
goals. Finally, the Results Act requires executive agencies to
prepare annual reports on program performance for the previous fiscal
year. The performance reports are to be issued by March 31 each
year, with the first (for fiscal year 1999) to be issued by March 31,
2000. In each report, an agency is to compare its performance
against its goals, summarize the findings of program evaluations
completed during the year, and describe the actions needed to address
any unmet goals.
The Results Act established requirements for pilot projects so that
agencies could gain experience in using key provisions of the Act and
provide lessons for pilots and for other agencies. The Act required
OMB to designate at least 10 agencies to participate in the first set
of pilot projects during fiscal years 1994 through 1996, which
focused on the preparation of annual performance plans and reports.
A second set of pilot projects in managerial accountability and
flexibility were to propose waivers of administrative procedural
requirements and controls for one or more of the major functions and
operations of the agency to determine the effects of providing
federal managers with increased managerial flexibility in exchange
for greater accountability for performance. OMB was to designate at
least five agencies from the first set of pilot projects to test
managerial accountability and flexibility during fiscal years 1995
and 1996. A third set of pilot projects, which are scheduled in the
Act for fiscal years 1998 and 1999, are to test performance
budgeting--i.e., the presentation of the varying levels of
performance that would result from different budget levels.
To prepare this report, GAO drew on a large body of work it had done
in recent years on the Results Act and on related goal-setting,
performance measurement, and accountability concepts. GAO also
surveyed a sample of civilian managers at the general schedule (GS)
and general management (GM) levels, GS/GM-13, through Senior
Executive Service (SES) levels in 24 major agencies. These 24
agencies accounted for over 99 percent of the federal government's
net outlay for fiscal year 1996. The sample was stratified by
whether the manager was SES or non-SES and by whether the manager was
working in a selected agency or agency component that was designated
as a Results Act pilot. Of the approximately 1,300 managers GAO
surveyed, it received usable responses from about 72 percent.
The survey results are statistically generalizable to the 24 agencies
included in the survey. The survey data reported throughout this
report are the estimated percentages of how officials would have
responded had the entire universe of eligible officials been
surveyed. In general, percentages reported for the entire sample
have confidence intervals ranging from + 5 percentage points to + 12
percentage points. In other words, if all managers and supervisors
in the 24 agencies included in GAO's population had been surveyed,
the chances are 95 out of 100 that the results obtained would not
differ from the sample estimate, in the most extreme case, by more
than + 12 percentage points.
RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3
GAO's work shows that the Results Act's implementation to this point
has achieved mixed results, which will lead to highly uneven
governmentwide implementation in the fall of 1997. While agencies
are likely to meet the upcoming statutory deadlines for producing
initial strategic plans and annual performance plans, GAO found that
those documents will not be of a consistently high quality or as
useful for congressional and agency decisionmaking as they could be.
On a more promising note, OMB selected over 70 performance planning
and reporting pilots that far exceeded the number required by the
Results Act and that should provide a rich body of experience for
agencies to draw on in the future.
GAO also found that the experiences of some of the Results Act pilot
agencies, such as the Social Security Administration (SSA), and
related efforts by nonpilot agencies, such as the Veterans Health
Administration, showed that significant performance improvements were
possible--even in the short term--when an agency adopted a
disciplined approach to setting results-oriented goals, measuring its
performance, and using performance information to improve
effectiveness. However, the reported examples of substantial
performance improvements were relatively few, and many agencies did
not appear to be well positioned to provide in 1997 a
results-oriented answer to the fundamental Results Act question:
What are we accomplishing?
GAO's review of efforts to date under the Results Act have shown that
to effectively implement the Act, agencies face a variety of
significant challenges, some of which will not be resolved quickly.
One set of challenges arises from the complications of government
structure and from program proliferation. Others involve
methodological difficulties in identifying performance measures or
the lack of data needed to establish goals and assess performance.
The following are among the challenges that GAO observed.
Overlapping and fragmented crosscutting program efforts, such as
student loan or economic development programs, present the
logical need to coordinate efforts to ensure that goals are
consistent and, as appropriate, that program efforts are
mutually reinforcing. GAO found that overlapping and fragmented
program efforts can undermine efforts to establish clear
missions and goals. In addition to the problem of overlapping
and fragmented programs, agencies are challenged in setting
goals because those goals often must reflect a balance of
competing policy priorities. For example, GAO has reported that
the effectiveness of the Forest Service had been compromised
because it had not reconciled conflicts concerning its role in
regulating the use of its lands (e.g., promoting timber sales
versus protecting wildlife). Striking the right balance is a
continuing and difficult challenge because forging the political
consensus needed to create and sustain a program often results
in that program having competing and/or broadly stated goals.
The often limited or indirect influence that the federal government
has in determining whether a desired result is achieved
complicates the effort to identify and measure the discrete
contribution of the federal initiative to a specific program
result. GAO work has shown that measuring the federal
contribution is particularly challenging for regulatory
programs; scientific research programs; and programs that
deliver services to taxpayers through third parties, such as
state and local governments.
The lack of results-oriented performance information in many
agencies hampers efforts to identify appropriate goals and
confidently assess performance. Even when data exist, GAO has
consistently found that the quality of agencies' performance
data is often questionable due to several factors, including the
need to rely on third parties to provide data.
Instilling within agencies an organizational culture that focuses
on results remains a work in progress across the federal
government. According to GAO's survey, about 57 percent of
managers said that their agencies' top leadership demonstrated a
strong commitment to achieving results to a great or very great
extent. In contrast, however, only about 16 percent said that
program changes by management above their levels were based on
results-oriented performance information to a great or very
great extent. This suggests that despite the strength of
managers' perceptions about top leadership commitment, the
extent to which leaders are demonstrating their commitment by
using performance information for decisionmaking is not as
great.
Linking agencies' performance plans directly to the budget process,
through the Results Act requirement to base the annual program
performance goals on the budget's program activity structure,
may present significant difficulties. GAO's analysis has shown
that the extent to which the budget's program activity structure
can be directly linked to a results-oriented performance
framework varies widely among activities, and GAO's work has
suggested that adjustments and accommodations in the program
activity structure may be needed. GAO's work also has suggested
that reaching agreement on such changes between Congress and the
executive branch will be a time-consuming and difficult process
that will take more than one budget cycle to resolve.
The Results Act is the cornerstone of a series of initiatives that
are intended to provide a comprehensive framework for integrating
program, cost, and budget information. GAO notes that improved
financial reporting and auditing required by the Chief Financial
Officers Act (CFO) Act should strengthen the reliability of cost and
performance information. GAO also notes that the Federal Accounting
Standards Advisory Board (FASAB) has developed a new set of reporting
concepts and accounting standards that underpin OMB's guidance to
agencies on the form and content of their agencywide financial
statements. These standards include a new reporting model for
federal agencies geared to providing users with consolidated
performance, cost, and other information. These standards also
include cost accounting standards that became effective beginning
with fiscal year 1997 and are the first set of standards that are to
account for the full costs of federal programs. GAO work has shown
that developing the necessary information systems to gather, process,
and analyze the needed program and cost information will be a
substantial undertaking for most federal agencies because of
agencies' long-standing weaknesses in the use of information
technology. To help address problems in agencies' use of information
technology, Congress enacted the Paperwork Reduction Act of 1995 and
the Clinger-Cohen Act of 1996.
Addressing some of the challenges highlighted by GAO, such as
crosscutting program efforts and balances among competing priorities,
will raise significant policy issues for Congress and the
administration to consider, some of which will likely be very
difficult to resolve. In this regard, GAO believes that the Results
Act's success or failure should not be judged on whether contentious
policy issues are fully resolved; rather, judgment of the success or
failure of the Results Act should turn on the extent to which the
information produced through the Results Act's goal-setting and
performance measurement practices--once those practices are
successfully implemented--helps inform policy decisions and improve
program management.
PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4
PERFORMANCE IMPROVEMENTS HAVE
BEEN MADE, BUT OVERALL RESULTS
ACT IMPLEMENTATION EFFORTS ARE
MIXED
---------------------------------------------------------- Chapter 0:5
Executive agencies, OMB, and Congress are undertaking efforts to
implement the Results Act. Over 70 federal organizations--in some
cases, entire agencies such as SSA and the Internal Revenue
Service--participated in the performance planning and reporting pilot
phase. GAO's work indicated that in addition to overseeing those
pilots, OMB has achieved some success working with agencies to seek
agreement on strategic and annual goals and performance measures and
to increase the prominence of performance information in the
executive branch budget process. For example, when GAO reviewed
fiscal year 1996 OMB budget documents, it found that more attention
had been placed on performance-related issues than in the previous
fiscal year. Congress, too, has shown a growing interest in and
support for the governmentwide implementation of the Results Act.
For example, the House Majority has established teams consisting of
staff from various committees to lead its strategic plan consultation
efforts. These teams have been reaching out to agencies to review
and comment on agencies' strategic plans.
Most important, GAO found some agencies have reported improved
results due to the Results Act or related efforts. For example, GAO
has reported that the Veterans Health Administration improved
services to veterans by more rigorously assessing the results of the
medical care it provides to the nation's veterans. In particular,
the Veterans Health Administration reported that it used performance
information to target the most important improvement opportunities
and thereby lowered the mortality rate for cardiac procedures by an
average of 13 percent over the last 8 years. In another example,
involving SSA's national toll-free 800 telephone number to handle
citizen inquiries, SSA used customer satisfaction and other
performance information to identify and make program changes,
including providing additional staff to handle phone calls. As a
result, the busy rate decreased from 49 to 34 percent, and the
percentage of calls answered within 5 minutes increased from 74 to 83
percent from fiscal year 1995 to fiscal year 1996. GAO notes that
these and other performance improvements strongly suggest that the
basic goal-setting and performance measurement model used by the
Results Act, if successfully implemented, will be an important tool
for improving federal management and performance.
Although some agencies reported that significant performance
improvements have been made, GAO noted that, in general, few agencies
have shown substantial performance improvements thus far. These
limited reports of improvements may be understandable at this early
stage of the Results Act's implementation. However, GAO's work also
indicates that agencies do not appear to have consistently in place
the critical performance measurement processes that would allow them
to make future improvements. For example, GAO reported in January
1997 that the Department of Housing and Urban Development's Public
Housing Management Assessment Program did not collect important
information needed to manage and assess its results. The program is
to assess the performance of local housing authorities by measuring
factors such as the numbers of outstanding work orders and
uncollected rents. However, the system does not measure other
factors, such as housing quality, that are essential for assessing
the results that housing authorities are achieving, as well as for
determining which housing authorities are performing well or poorly.
The situation at the Department of Housing and Urban Development
appears to be typical. According to GAO's survey, fewer than
one-third of managers in the agencies included in its survey reported
that results-oriented performance measures exist for their programs
to a great or very great extent. The existence of other types of
performance measures needed to successfully manage programs also was
reported as low. For example, of the total managers reporting the
existence of such measures to a great or very great extent, 38
percent of the managers reported the existence of measures of
outputs, 32 percent reported the existence of customer satisfaction
measures, 31 percent reported the existence of measures of product or
service quality, and 26 percent reported the existence of measures of
efficiency.
On a more positive note, while still viewed as low, significantly
more managers reported in GAO's survey that results-oriented and
other performance measures existed to a greater extent currently than
3 years ago. For example, GAO asked managers about the current
situation as compared with what they recollected 3 years ago
concerning the existence of results-oriented performance measures and
measures of customer satisfaction. The percentage of managers who
reported the existence of these measures to a great or very great
extent was higher currently by 13 and 21 percentage points,
respectively. Although this increase is a very favorable
development, GAO notes that the overall low level still is a matter
for concern.
Obviously, it is not sufficient merely to measure current
performance. The Results Act envisions that performance information
will be used to make decisions and better manage programs. In that
regard, about one-fifth of federal managers from agencies included in
GAO's survey reported that results-oriented performance information
was used to a great or very great extent to develop their agencies'
budgets. Managers reported even lower levels in the use of
results-oriented performance information to help make funding
decisions, legislative changes, or program changes. However,
managers rated the extent of use of results-oriented performance
information currently as somewhat higher than they rated the extent
of use 3 years ago. The reported limited availability of
results-oriented performance measures, combined with the still
infrequent use of such information to help make key decisions,
suggests to GAO that many agencies may have a difficult time
developing meaningful results-oriented measures in time for fiscal
year 1999 performance plans that are due to OMB this fall.
KEY CHALLENGES REMAIN TO
EFFECTIVE IMPLEMENTATION OF THE
RESULTS ACT
---------------------------------------------------------- Chapter 0:6
GAO found that agencies are confronting a variety of difficult
challenges that have limited the implementation of the Results Act.
These challenges include establishing clear agency missions and
strategic goals, especially when program efforts are overlapping or
fragmented; measuring performance, particularly when the federal
contribution to a result is difficult to determine; generating the
results-oriented performance information needed to set goals and
assess progress; instilling a results-oriented organizational culture
within agencies; and linking performance plans to the budget process.
The experiences of pilot agencies and related efforts by nonpilot
agencies suggest that these challenges will not be quickly or easily
resolved.
ESTABLISHING CLEAR AGENCY
MISSIONS AND STRATEGIC GOALS
REMAINS A CHALLENGE
-------------------------------------------------------- Chapter 0:6.1
Attempting to reach a reasonable degree of consensus on agencies'
missions and establishing clear and precise strategic goals are
likely to raise contentious policy issues that may not be easily
resolved. Past GAO work has shown that as Congress and the executive
branch have responded to new national needs and problems, many
different agencies have been given responsibility for addressing the
same or similar national issues. Of the 18 national mission areas
displayed in the federal budget, 14 were addressed by more than 1
executive branch department or major agency in fiscal year 1996. In
some cases, this shared responsibility demonstrates that addressing a
national need requires the efforts of more than one agency.
However, past GAO work has suggested that some important federal
program areas have suffered from overlap and fragmentation.
Overlapping and fragmented program efforts can frustrate program
customers, waste scarce resources, and limit the overall
effectiveness of the federal effort. For example, GAO reported in
1995 on the Department of Education programs that provided loans and
grants to students to help finance their higher education. GAO found
that although the student loan and Pell grant programs provided the
majority of federal financial aid to students for postsecondary
education, another 22 smaller programs were targeted to specific
segments of the postsecondary school population, such as prospective
students from disadvantaged families or women and minorities who are
underrepresented in graduate education. These 22 programs were
collectively funded at $1.1 billion for fiscal year 1995. GAO
concluded that these smaller grant programs could be considered
candidates for consolidation--with other larger programs or among
themselves--with no adverse impact on students' access to
postsecondary education. GAO also found that the federal government
could anticipate administrative savings of 10 percent each year, or a
total of $550 million in budget authority (adjusted for inflation)
over 5 years.
In addition, GAO notes that many agencies confront competing policy
objectives, which are a natural by-product of the complex social and
political environment in which programs are created, funded, and
managed. For example, GAO reported in April 1997 that the Forest
Service had increasingly shifted the emphasis of its efforts from
producing timber to sustaining wildlife. This shift was taking place
in reaction to requirements in planning and environmental laws and
their judicial interpretation--reflecting changing public values and
concerns--together with social, ecological, and other factors.
However, GAO noted that the demand for recreation was also expected
to grow and may increasingly conflict with efforts aimed at
sustaining wildlife and producing timber. GAO found that the
disagreement both within the Forest Service and among key external
stakeholders, including Congress, on how the Forest Service is to
resolve conflicts or make choices among competing uses on its lands
had seriously undermined its efforts to establish the goals and
performance measures needed to ensure accountability. GAO concluded
that until general agreement is reached, the Forest Service's
decisionmaking will continue to be inefficient and ineffective.
LACK OF FEDERAL CONTROL OVER
RESULTS COMPLICATES
PERFORMANCE MEASUREMENT
EFFORTS
-------------------------------------------------------- Chapter 0:6.2
GAO's past work shows that agencies must grapple with some difficult
analytic and technical challenges in developing results-oriented
performance measures. The variety of influences beyond direct
federal control that may determine whether, and the degree to which,
a federal effort is successful is a major challenge to agencies'
efforts to measure performance. Moreover, the end result of a
program may occur long after the federal intervention, which makes it
difficult to measure progress annually. Because of issues like
these, performance measurement efforts, such as those for scientific
research, regulatory, and intergovernmental programs, are
particularly challenging.
For example, GAO found that determining the impact of economic
development programs has been a daunting task because of the numerous
external forces--including broad national economic trends and the
assistance that communities may receive from state and local
governments and the private sector--that may contribute to local
economic development. Separating out the effects of federal program
efforts can be extremely difficult, as GAO observed in a 1996 review
on economic development programs, because it would require, first,
documentation that there had been some improvement in a targeted
area; second, linkage of specific program elements to actual economic
changes; and third, measurement of the growth stemming from other
influences on the economy of the targeted area in order to isolate
the impact that could be attributed to the economic development
program.
GAO found that although such performance measurement efforts
constitute a substantial undertaking, they have the potential to
provide important information for decisionmakers. For example, one
study of the effectiveness of the Department of Commerce's Economic
Development Administration (EDA) programs found that income in the
counties that received EDA funding grew significantly faster than
income in the counties that received no aid. However, when EDA's
programs and factors unrelated to EDA were considered simultaneously,
the study found that EDA's programs had a very small effect on income
growth rates during the period that the aid was received and had no
significant effect in the 3 years after the aid ceased. The study
found that EDA's programs could explain only a small part of the
difference in these growth rates between the two groups of counties.
GAO found that agencies are exploring a number of approaches for
addressing difficulties in developing results-oriented performance
measures. These approaches include undertaking program evaluations
to try to isolate program impact, such as in the EDA case already
discussed; using measures of intermediate results; employing a range
of measures to assess whether programs are making progress; and
working with stakeholders to seek agreement on appropriate measures.
ABSENCE OF QUALITY
RESULTS-ORIENTED PERFORMANCE
INFORMATION HAMPERS EFFORTS
TO SET GOALS AND ASSESS
PERFORMANCE
-------------------------------------------------------- Chapter 0:6.3
GAO has found that a lack of results-oriented performance information
to use as a baseline complicates agencies' efforts to set appropriate
improvement targets. Department of Veterans Affairs officials told
GAO that some of their results-oriented measures for a Loan Guaranty
program were new and that baseline data were not available on those
measures. Consequently, they did not have data on past performance
to use in setting some of the program's fiscal year 1998 goals. In
some of these cases, the Department indicated in its fiscal year 1998
budget submission that those goals were "to be determined." In
another example, Department of Agriculture officials told GAO that
they eliminated some performance measures that had been part of their
Results Act pilot's annual performance plan because they did not have
a way to collect data on those measures. Lacking these data, they
did not have an informed basis on which to set goals.
Past GAO work also has shown that agencies face a variety of
challenges to collecting valid and reliable performance information,
including ensuring that standard definitions are used when measuring
performance. For example, GAO reported in 1996 on management
challenges that the Department of Health and Human Service's Office
of Child Support Enforcement (OCSE) faced because of a lack of
comparable performance data across state and local jurisdictions.
OCSE officials said that discrepancies resulting from differences in
the way the states and local jurisdictions defined what constitutes a
child support enforcement case had contributed to the difficulty of
uniformly measuring state performance. To address these
discrepancies, OCSE worked with state and local authorities to
develop standard data definitions for key child support enforcement
terms, including a definition for what constitutes a child support
enforcement case, and incorporated the use of standardized
definitions for measuring state performance. Together, they also
developed measures to assess state performance in obtaining support
orders and agreed that states would use the OCSE-established
definition of a child support enforcement case to report these data.
MAKING NEEDED CULTURAL
CHANGES IS A WORK IN
PROGRESS ACROSS AGENCIES
-------------------------------------------------------- Chapter 0:6.4
According to GAO's survey, federal managers rated the commitments of
top leadership to achieving results as higher currently than they did
for 3 years ago. However, federal managers' responses to GAO's
survey also suggested that more progress is needed for agencies to
develop and sustain cultures that focus on results. For example,
when GAO asked federal managers about the extent to which they or
supervisors at their levels had the authority they needed to help
their agencies accomplish their strategic goals, the managers did not
perceive that they had more such authority currently than they
recalled having 3 years ago.
Significantly, for managers from the Results Act pilots that GAO was
able to isolate for its sample, GAO's survey found these managers'
perception of the extent of their authority currently was much lower
than their perception of the situation 3 years ago. For example, 56
percent of SES managers from selected Results Act pilots reported
that managers at their level had authority to help the agency
accomplish its strategic goals to a great or very great extent 3
years ago. Fewer SES managers, 40 percent, reported having authority
to a comparable extent currently--a difference of 16 percentage
points.
These survey results suggest to GAO that as agencies implement the
Results Act and strive to become more results oriented, they need to
pay special attention to ensuring that key managers have the
authority they need to achieve intended results. In passing the
Results Act, Congress recognized that if federal managers were to be
held accountable for program results, they would need the authority
and flexibility to achieve those results. Congress also understood
the importance of affording federal program managers the freedom to
be innovative and creative and to marshal resources to achieve
results. Indeed, Congress has provided agencies with additional
authority in the key area of procurement that allows for more
flexibility in managing their programs. In addition, the
administration's National Performance Review has sought the
empowerment of employees to achieve results as one of its major
initiatives.
INTEGRATING PERFORMANCE
INFORMATION INTO THE BUDGET
WILL LIKELY PROVE DIFFICULT
-------------------------------------------------------- Chapter 0:6.5
The annual performance plans that agencies are to develop under the
Results Act provide the opportunity, if successfully implemented, to
assist congressional and executive branch resource allocation
decisionmaking by providing direct linkages between the performance
information contained in performance plans and agency budget
requests. Under the Results Act, the annual performance goals are to
be based on the program activities listed in the president's budget.
GAO has observed that although budgeting is inherently an exercise in
political choice, the Results Act is based on the premise that budget
decisions should be more clearly linked to expectations about program
performance.
In a 1997 report on past federal initiatives in performance
budgeting, GAO noted that an agency's program activities generally
result from negotiations between the agency, OMB, and the relevant
congressional appropriations subcommittees. Program activity
structures, which represent programmatic, process, organizational, or
other orientations, therefore depend on the needs, interests, and
experiences of agencies, OMB, and Congress. Because of this wide
variability, the suitability of current program activity structures
for the Results Act performance planning and measurement may vary.
GAO's discussions with agency officials and congressional staff as
part of its review of past performance budgeting initiatives
highlighted the differing expectations held by the two branches of
government on the use of the budget's program activity structure as a
basis for planning and measuring agencies' performance.
Congressional staff were generally comfortable with existing activity
structures and questioned whether changes would frustrate
congressional oversight. The staff generally viewed these structures
as fundamental to congressional oversight of agency activities, and
thus they viewed changes with apprehension and concern. However,
some agency officials saw program activity structures as secondary to
planning. Therefore, where current program activity structures
proved unsuitable for planning purposes, these officials viewed
change in program activity structures as inevitable and appropriate.
However, they noted that negotiating changes with Congress could
prove difficult and time-consuming.
AN AUGMENTED REPORTING
FRAMEWORK HOLDS PROMISE FOR
STRENGTHENING FEDERAL
DECISIONMAKING AND
ACCOUNTABILITY
---------------------------------------------------------- Chapter 0:7
GAO's past work indicates that integrated accountability reports have
the potential of further strengthening congressional and executive
branch decisionmaking and agencies' accountability. The FASAB
accountability reporting concepts and accounting standards--building
on CFO Act requirements--are to bring together program performance
information with audited financial information to provide
congressional and other decisionmakers with a more complete picture
of the results, operational performance, and the costs of agencies'
operations. For the first time, decisionmakers are to be provided
with annual "report cards" on the costs, management, and
effectiveness of federal agencies. The FASAB cost accounting
standards hold a similar promise to improve decisionmaking, if
successfully implemented. These standards are to provide
decisionmakers with information on the costs of all resources used,
including the costs of services provided by others to support
activities or programs. Such information would allow for comparisons
of the costs of various programs and activities with their
performance outputs and results.
Congress, in enacting the Results Act, the CFO Act, and information
technology reform legislation, sought to create a more focused,
results-oriented management and decisionmaking process within both
Congress and the executive branch. GAO has noted that these laws
were intended to respond to a need for accurate, reliable information
for congressional and executive branch decisionmaking, information
that had been badly lacking. GAO observed that implemented together,
these laws provide a powerful framework for developing fully
integrated information about agencies' missions and strategic
priorities, results-oriented performance goals that flow from those
priorities, performance data to show the achievement (or not) of
those goals, the relationship of information technology investments
to the achievement of performance goals, and accurate and audited
financial information about the costs of achieving mission results.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:8
GAO has long supported the goal-setting, performance measurement, and
results-based accountability concepts embodied in the Results Act.
As part of GAO's statutory mandate under the Results Act, and at the
request of congressional committees, GAO monitored the performance
planning and reporting pilot phase of the Results Act, highlighted
the key implementation issues needing additional attention as those
issues were identified, and made suggestions to Congress and agencies
as appropriate. Moreover, the GAO reports that examined management
issues in individual agencies contained recommendations, as
appropriate, to improve the management of those agencies. Overall,
GAO found nothing in the work that it did specifically for this
report that would lead it to alter its position in support of the
concepts in the Results Act or the recommendations it has made to
specific agencies. Therefore, GAO is making no new recommendations
in this report.
AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:9
GAO sent a draft of this report to the Director of OMB for comments.
On May 21 and 22, 1997, senior OMB officials provided technical
comments that were incorporated where appropriate.
INTRODUCTION
============================================================ Chapter 1
A consensus that the nation's persistent federal deficit must be
addressed and that the effectiveness of federal programs must improve
substantially has spurred widespread efforts under way in federal
agencies and Congress to dramatically change the way the federal
government is managed. Although proposals to reduce the deficit are
prompting difficult policy debates over the role of the federal
government, better management of federal programs and activities can
provide part of the solution to the government's fiscal problems.
But the hard work of managing downsized federal agencies is made more
difficult when those agencies lack goals that focus on results;
cost-effective strategies to achieve those goals; sound measures of
performance; and reliable, accurate, and timely information needed
for decisionmaking.
Our work over many years repeatedly has found that agencies lack
these basic underpinnings of well-run organizations. In enacting the
Government Performance and Results Act (GPRA) of 1993, which is
referred to as "GPRA" or "the Results Act," Congress expressed
frustration that congressional and executive branch decisionmaking
was often hampered by the lack of good information on the results of
federal program efforts. However, Congress also recognized that
improving management in the federal sector will not be easy or come
quickly.
CONGRESS HAS ENACTED A
STATUTORY FRAMEWORK FOR
IMPROVING FEDERAL MANAGEMENT
AND DECISIONMAKING
---------------------------------------------------------- Chapter 1:1
In the 1990s, Congress put in place a statutory framework to address
the long-standing weaknesses in federal operations, improve federal
management practices, and provide greater accountability for
achieving results. This framework included as its essential elements
financial management reform legislation, information technology
reform legislation, and the Results Act.
In enacting this framework, Congress sought to create a more focused,
results-oriented management and decisionmaking process within both
Congress and the executive branch. These laws seek to improve
federal management by responding to a need for accurate, reliable
information for congressional and executive branch decisionmaking,
information that has been badly lacking in the past, as much of our
work has demonstrated. Implemented together, these laws provide a
powerful framework for developing fully integrated information about
agencies' missions and strategic priorities, the results-oriented
performance goals that flow from those priorities, performance data
to show the achievement (or not) of those goals, the relationship of
information technology investments to the achievement of performance
goals, and accurate and audited financial information about the costs
of achieving mission results.
FINANCIAL MANAGEMENT REFORM
LEGISLATION
-------------------------------------------------------- Chapter 1:1.1
The primary financial management reform legislation Congress enacted
is the Chief Financial Officers Act of 1990 (CFO) Act, as expanded by
the Government Management Reform Act of 1994 (GMRA). This law
provides the basis for identifying and correcting financial
management weaknesses that have cost the federal government billions
of dollars and leave it vulnerable to waste, fraud, and
mismanagement. The expanded CFO Act spelled out a long overdue and
ambitious agenda to help the government resolve its lack of timely,
reliable, useful, and consistent financial information. First, 24
major executive agencies, covering over 99 percent of the federal
government's net outlay for fiscal year 1996, are required to prepare
and have audited financial statements for their entire operations,
beginning with fiscal year 1996. Second, every year starting with
fiscal year 1997, the Secretary of the Treasury is required to
produce a governmentwide financial statement in coordination with the
Director of the Office of Management and Budget (OMB), and the
statement is to be audited by the Comptroller General of the United
States. With successful implementation, the audited financial
statements required by the CFO Act, as expanded by GMRA, will provide
congressional and executive branch decisionmakers with the financial
and program cost information that they have not previously had.
INFORMATION TECHNOLOGY
REFORM LEGISLATION
-------------------------------------------------------- Chapter 1:1.2
Information technology reform legislation, including the Paperwork
Reduction Act of 1995 and the Clinger-Cohen Act of 1996, is based on
the best practices used by leading public and private organizations
to more effectively manage information technology.\1 Even though
agencies have obligated billions of dollars--$145 billion in the last
6 years alone--to build up and maintain their information technology
resources, the benefits frequently have been disappointing. Federal
technology projects often experience schedule slippages, incur
extensive cost overruns, and fail to provide promised performance
improvements.
We have noted that the sound application and management of
information technology to support strategic program goals must be an
important part of any serious attempt to improve agency mission
performance, cut costs, and enhance responsiveness to the public.\2
Under the information technology reform laws, agencies are to better
link their technology plans and information technology use to their
programs' missions and goals. To do this, agencies are to, among
other things, (1) involve senior executives in information management
decisions; (2) establish senior-level Chief Information Officers who
are to, among other things, evaluate information technology programs
on the basis of applicable performance measurements; (3) impose
much-needed discipline on technology spending; (4) redesign
inefficient work processes; and (5) use performance measures to
assess technology's contribution to achieving mission results.
--------------------
\1 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology--Learning from Leading
Organizations (GAO/AIMD-94-115, May 1994).
\2 High-Risk Areas: Actions Needed to Solve Pressing Management
Problems (GAO/T-AIMD/GGD-97-60, Mar. 5, 1997).
GOVERNMENT PERFORMANCE AND
RESULTS ACT
-------------------------------------------------------- Chapter 1:1.3
The landmark Results Act is the centerpiece of an integrated
statutory framework, which includes reform legislation in the areas
of financial management and information technology. The Act focuses
on clarifying missions, setting program goals, and measuring
performance toward achieving those goals. It emphasizes managing for
results and pinpointing opportunities for improved performance and
increased accountability. The purposes of the Act are to:
"(1) improve the confidence of the American people in the capability
of the Federal Government, by systematically holding Federal agencies
accountable for achieving program results;
"(2) initiate program performance reform with a series of pilot
projects in setting program goals, measuring program performance
against those goals, and reporting publicly on their progress;
"(3) improve Federal program effectiveness and public accountability
by promoting a new focus on results, service quality, and customer
satisfaction;
"(4) help Federal managers improve service delivery, by requiring
that they plan for meeting program objectives and by providing them
with information about program results and service quality;
"(5) improve congressional decisionmaking by providing more objective
information on achieving statutory objectives, and on the relative
effectiveness and efficiency of Federal programs and spending; and
"(6) improve internal management of the Federal Government."\3
Congress intended for the Act to improve the effectiveness of federal
programs by fundamentally shifting the focus of management and
decisionmaking away from a preoccupation with staffing and activity
levels to a broader focus on the results of federal programs. As
measures of their performance, agencies have traditionally used the
amount of money spent to support their programs, the number of staff
employed, or the number of tasks completed. But at a time when the
value of many federal programs is undergoing intense public scrutiny,
an agency that uses and reports only these measures has not answered
the defining question of whether these programs have produced real
results. Under the Results Act, virtually every executive agency is
required to ask itself some basic questions: What is our mission?
What are our goals, and how can we achieve them? How can we measure
our performance? How will we use performance information to make
improvements?
--------------------
\3 P.L. 103-62, sec. 2.
THE RESULTS ACT ESTABLISHES
SPECIFIC REQUIREMENTS
---------------------------------------------------------- Chapter 1:2
The Results Act requires executive agencies to set goals, measure
performance, and report on the degree to which goals were met.
Because Congress understood that most agencies would need to make
significant management changes to effectively implement the Act,
requirements were to be phased in over several years. Thus, the
implementation of the Act began with a set of pilot projects to be
designated by OMB so that selected agencies could gain experience in
using key provisions of the Act and provide lessons for other
agencies. These pilots were to focus on
the development of agency performance plans, which provide the
direct linkage between an agency's long-term goals (commonly
referred to as strategic goals) and what its managers and
employees do day-to-day; and performance reports, which are the
feedback to managers, policymakers, and the public as to what
was actually accomplished for the resources expended;
managerial accountability and flexibility, which is the granting of
waivers from certain nonstatutory administrative procedural
requirements and controls in return for greater management
accountability for performance; and
performance budgeting, which is intended to provide Congress with
information on the direct relationship between proposed program
spending and expected program results and the anticipated
effects of varying spending levels on results.
The Act requires the Director of OMB to report to Congress and the
president on the results of the pilot projects. The first report was
to cover pilot projects in performance planning and reporting and
managerial accountability and flexibility and was due no later than
May 1, 1997. This report was to include, among other things, an
assessment of the benefits and costs of pilot performance plans and
reports and any significant difficulties experienced by the pilot
agencies in preparing proposed waivers.\4 OMB is to prepare a second
report to Congress and the president on the performance budgeting
pilots. This report is due no later than March 31, 2001.
--------------------
\4 The Government Performance and Results Act: Report to the
President and the Congress from the Director of the Office of
Management and Budget, May 19, 1997.
AGENCIES ARE TO SET
STRATEGIC DIRECTION,
IDENTIFY ANNUAL PERFORMANCE
GOALS, AND REPORT ON
PERFORMANCE
-------------------------------------------------------- Chapter 1:2.1
The Results Act is designed to improve the efficiency and
effectiveness of federal programs by establishing a system to set
goals for program performance and to measure results. Specifically,
the Act requires executive agencies to prepare multiyear strategic
plans, annual performance plans, and annual performance reports.
MULTIYEAR STRATEGIC PLANS
------------------------------------------------------ Chapter 1:2.1.1
The Results Act requires virtually every executive agency to develop
strategic plans covering a period of at least 5 years forward from
the fiscal year in which it is submitted and to update those plans at
least every 3 years.\5 Agencies' first strategic plans are to be
submitted to Congress and the Director of OMB by September 30, 1997.
The strategic plans are to (1) include the agencies' mission
statements; (2) identify long-term general goals and objectives, also
known as strategic goals; (3) describe how the agencies intend to
achieve those goals through their activities and through their human,
capital, information, and other resources; and (4) explain the key
external factors that could significantly affect the achievement of
those goals. Under the Act, strategic plans are the starting point
for agencies to set annual performance goals and to measure program
performance in achieving those goals. Consequently, strategic plans
are also to include a description of how long-term general goals will
be related to annual performance goals as well as a description of
the program evaluations that agencies used to establish their
long-term general goals and a schedule for subsequent evaluations.
As part of the strategic planning process, agencies are required to
consult with Congress and solicit the views of other
stakeholders--those governmental and nongovernmental entities
potentially affected by, or interested in, the agencies' activities.
--------------------
\5 The Results Act applies to agencies as defined in 5 U.S.C.
306(f), which generally covers executive departments, government
corporations, and independent establishments.
ANNUAL PERFORMANCE PLANS
------------------------------------------------------ Chapter 1:2.1.2
Building on the decisions made as part of the strategic planning
process, the Results Act requires executive agencies to develop
annual performance plans covering each program activity set forth in
the agencies' budgets.\6 The first annual performance plans, covering
fiscal year 1999, are to be submitted to OMB this fall and to
Congress after the President's budget in 1998. The plan is to
contain an agency's annual performance goals, which the agency is to
use in order to gauge its progress toward accomplishing its strategic
goals. The plan also is to include the performance measures that the
agency will use to gauge progress toward the achievement of its
annual performance goals and the resources the agency will need to
meet its goals. Finally, the plan is to discuss how the agency will
verify the resulting performance data.
The Act authorizes agencies to apply for managerial flexibility
waivers in their annual performance plans. Agencies' authority to
request waivers of nonstatutory administrative procedural
requirements and controls is intended to provide federal managers
with more flexibility to structure agency systems to better support
performance goals. An example of increased flexibility would be to
allow an organization to recapture unspent operating funds because of
increased efficiencies and then to use these funds to purchase new
equipment or expand employee training. Another example might involve
delegating more authority to line managers to make procurement
decisions.
OMB is to use the performance plans that agencies submit to develop
an overall federal government performance plan. OMB is to submit
this governmentwide plan each year to Congress with the president's
budget. According to the Senate Committee on Governmental Affairs
report on the Act, the overall federal government performance plan is
to present to Congress a single cohesive picture of the federal
government's annual performance goals for the fiscal year.\7 The
first overall plan is due with the President's fiscal year 1999
budget.
--------------------
\6 The term "program activity" refers to the listings of projects and
activities in the Appendix portion of the Budget of the United States
Government. Program activity structures are intended to provide a
meaningful representation of the operations financed by a specific
budget account.
\7 S. Rep. No. 58, 103d Cong. 1st Sess. (1993).
ANNUAL PERFORMANCE
REPORTS
------------------------------------------------------ Chapter 1:2.1.3
Finally, the Results Act requires each executive agency to prepare
annual reports on program performance for the previous fiscal year.
The first performance reports for fiscal year 1999 are due to
Congress and the president no later than March 31, 2000, and
subsequent reports are due by March 31 for the years that follow. In
each report, an agency is to review and discuss its performance
compared with the performance goals it established in its annual
performance plan. When a goal is not met, the agency is to explain
in the report the reasons the goal was not met; plans and schedules
for meeting the goal; and, if the goal was impractical or not
feasible, the reasons for that and the actions recommended.
According to the Senate committee report on the Act, actions needed
to accomplish a goal could include legislative, regulatory, or other
actions. If an agency finds a goal to be impractical or not
feasible, it is to include a discussion of whether the goal should be
modified.
In addition to evaluating the progress made toward achieving its
annual goals, an agency's program performance report is to evaluate
the agency's performance plan for the fiscal year in which the
performance report was submitted. Thus, in their fiscal year 1999
performance reports that are due by March 31, 2000, agencies are
required to evaluate their performance plans for fiscal year 2000 on
the basis of their reported performance in fiscal year 1999. This
evaluation is to help show how an agency's actual performance is
influencing its performance plan. The report also is to include (1)
the summary findings of program evaluations completed during the
fiscal year covered by the report and (2) the use and effectiveness
of any of the Results Act managerial flexibility waivers that an
agency received.
Agencies also are to include baseline and trend data in annual
performance reports to help ensure that their reports are complete
and that performance is viewed in context. Such data can show
whether performance goals are realistic given the past performance of
an agency. Such data can also assist users of reports to draw more
informed conclusions than they would by comparing only a single
year's performance against an annual goal, because users of reports
can see improvements or declines in an agency's performance over
prior years.\8 For fiscal years 2000 and 2001, agencies' reports are
to include data on the extent to which their performance achieved
their goals, beginning with fiscal year 1999. For each subsequent
year, agencies are to include performance data for the year covered
by the report and 3 prior years. Congress recognized that in some
cases not all the performance data will be available in time for the
required reporting date. In such cases, agencies are to provide
whatever data are available with a notation as to their incomplete
status. Subsequent annual performance reports are to include the
complete data as part of the trend information.
--------------------
\8 GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).
THE RESULTS ACT USED A
PHASED IMPLEMENTATION
APPROACH TO BUILD EXPERIENCE
AND PROVIDE LESSONS
-------------------------------------------------------- Chapter 1:2.2
The Results Act required OMB to designate at least 10 agencies to
participate in the first set of pilot projects, which were to focus
on the preparation of program performance plans and reports. The
selected agencies were to reflect a representative range of
government functions and capabilities in measuring and reporting
program performance. Also, the Act required these pilot projects to
use strategic plans during 1 or more years of the pilot period, which
ran from fiscal years 1994 through 1996.
Congress recognized that managerial accountability for results is
linked to managers having sufficient flexibility, discretion, and
authority to accomplish desired results. Thus, the Act specified a
second set of pilot projects in managerial accountability and
flexibility. These pilot projects were to propose waivers of
administrative procedural requirements and controls for one or more
of the major functions and operations of the agency to determine the
effects of providing federal managers with increased managerial
flexibility in exchange for the greater accountability for
performance. At least five agencies from the first set of pilot
projects were to be designated by OMB to test managerial
accountability and flexibility during fiscal years 1995 and 1996.
Finally, the Results Act called for a third set of pilot projects for
fiscal years 1998 and 1999 to test performance budgeting--the
presentation of varying levels of performance that would result from
different budget levels--before requiring it statutorily. OMB is to
select at least five agencies, at least three of which had experience
developing performance plans during the first pilot phase, to test
performance budgeting. By March 31, 2001, OMB is to report to
Congress and the president on the pilot test of performance
budgeting. OMB's report is to assess the feasibility of performance
budgeting as part of the president's budget, recommend whether
legislation requiring performance budgets should be proposed, and
identify any other recommended changes to the requirements of the
Act.
THE RESULTS ACT INCORPORATES
LESSONS FROM PAST INITIATIVES
TO IMPROVE PUBLIC SECTOR
PERFORMANCE
---------------------------------------------------------- Chapter 1:3
The Results Act is the most recent federal government reform
initiative designed to better align spending decisions with expected
results. Through acquiring better information on the relative
efficiency and effectiveness of federal programs and spending, the
Act seeks to help federal managers improve program performance. We
recently reported on the key design elements and approaches of the
Act and compared them with those of past federal initiatives that
sought to link resources with results--broadly known as performance
budgeting.\9 Since 1950, the federal government has attempted four
governmentwide performance budgeting initiatives: (1) reforms
flowing from the first Hoover Commission in its efforts to downsize
the post-World War II government; (2) the
Planning-Programming-Budgeting-System (PPBS) that President Johnson
began in 1965, which introduced a decisionmaking framework to the
executive branch budget formulation process by presenting and
analyzing choices among long-term policy objectives and alternative
ways of achieving them; (3) Management by Objectives (MBO) that
President Nixon initiated in 1973, which sought to link agencies'
stated objectives to their budget requests and put in place a process
to hold agency managers responsible for achieving agreed-upon outputs
and results; and (4) Zero-Based Budgeting (ZBB) that President Carter
initiated in 1977, which required agencies to set priorities on the
basis of the program results that could be achieved at alternative
spending levels, one of which was to be below current funding. Even
though these reforms were generally perceived as having fallen short
of their stated goals, they contributed to the evolution of
performance-based measurement and budgeting in the federal
government.
The Results Act incorporates critical lessons learned from these
previous efforts.
Where past efforts failed to link executive branch performance
planning and measurement with congressional resource allocation
processes, the Act requires explicit consultation between the
executive and legislative branches on agency strategic plans.
Where past initiatives devised unique performance information
formats often unconnected to the structures used in
congressional budget presentations, the Act requires agencies to
plan and measure performance using the program activities listed
in their budget submissions.
Where past initiatives were generally unprepared for the
difficulties associated with measuring the results of federal
programs and often retreated to simple output or workload
measures, the Act states a preference for results-oriented
measures while recognizing the need to develop a range of
measures, including output and, in some cases, nonquantitative
measures.
--------------------
\9 Performance Budgeting: Past Initiatives Offer Insights for GPRA
Implementation (GAO/AIMD-97-46, Mar. 27, 1997).
LEADING ORGANIZATIONS SHOW A
WAY
---------------------------------------------------------- Chapter 1:4
The federal government's efforts to become more results oriented are
consistent with a broad national and international trend among public
organizations. According to the Senate committee report on the
Results Act, use of performance measurement was a growing trend in
state governments. The report further noted that work done by the
Organization for Economic Cooperation and Development suggested that
several foreign governments may have been 5 to 10 years ahead of the
United States in the development of performance measurement, which
was a basic part of their broader efforts to better manage for
results. More specifically, these foreign governments set out broad
themes to (1) define goals clearly, develop measures, and report on
progress; and (2) give managers the flexibility to manage for results
by providing them the tools and incentives to act.
At the request of the Senate Committee on Governmental Affairs and
the House Committee on Government Reform and Oversight, we reported
on a number of state and foreign governments that were successfully
pursuing management reform and becoming more results-oriented.\10 We
studied the state governments of Florida, Oregon, Minnesota, North
Carolina, Texas, and Virginia; and the foreign governments of
Australia, Canada, New Zealand, and the United Kingdom. Much like
the federal government, a number of these governments were
dissatisfied with their public service. Many of them were called
upon to improve performance while simultaneously reducing costs.
Some faced economic challenges, such as economies dominated by
government spending and high budget deficits. Each of these
organizations set its agenda for management reform according to its
own environment, needs, and capabilities. Yet, despite their
differing approaches to reform, these organizations were seeking to
become more results oriented.
We were asked by the Senate Committee on Governmental Affairs and the
House Committee on Government Reform and Oversight whether the
experiences of these and other public organizations could yield
worthwhile lessons for executive agencies as they attempt to
implement the Results Act. In response to that request, we issued
the Executive Guide: Effectively Implementing the Government
Performance and Results Act.\11 The Executive Guide identifies a set
of key steps and associated practices that leading federal and other
public sector organizations have used to successfully implement
reform efforts that are consistent with the Results Act.
Accompanying the discussion of each practice is a case illustration
of a federal agency that has made progress in incorporating that
practice into its operations.
Taken together, the key steps and practices drawn from the
organizations we studied provide a useful framework to assist
Congress and the executive branch as they work to implement the Act.
These steps were to (1) define mission and desired outcomes, or
results; (2) measure performance to gauge progress; and (3) use
performance information as a basis for decisionmaking. In taking
these steps, leading organizations also found that certain leadership
practices, such as devolving operational authority and creating
incentives for managers and staff to focus on results, were central
to making the changes needed for the organizations to become more
results oriented. Figure 1.1 illustrates the key steps and
associated practices.
Figure 1.1: Key Steps and
Critical Practices for
Effectively Implementing GPRA
(See figure in printed
edition.)
Source: Executive Guide .
(See figure in printed
edition.)
--------------------
\10 Managing for Results: State Experiences Provide Insights for
Federal Management Reforms (GAO/GGD-95-22, Dec. 21, 1994); and
Managing for Results: Experiences Abroad Suggest Insights for
Federal Management Reforms (GAO/GGD-95-120, May 2, 1995).
\11 GAO/GGD-96-118, June 1996.
OBJECTIVES, SCOPE, AND
METHODOLOGY
---------------------------------------------------------- Chapter 1:5
As required by the Act, our overall objective was to report to
Congress on the implementation of the Act, including the prospects
for compliance by executive agencies beyond those participating as
pilot projects. More specifically, our objectives for this report
were to (1) assess the status of the Results Act implementation
efforts; (2) identify significant challenges confronting executive
agencies in their efforts to become more results oriented; and (3)
describe ongoing efforts to integrate program, cost, and budget
information into a reporting framework that allows for fuller
consideration of resource allocations, operational costs, and
performance results. As agreed with the Senate Committee on
Government Affairs and the House Committee on Government Reform and
Oversight, our strategy was to partially fulfill our legislative
requirement by reporting regularly on the Act and related initiatives
throughout the pilot phase of performance planning, measurement, and
reporting. This report culminates our efforts to meet this mandate.
To meet each of our objectives, we relied on a wide range of
management work we have done for the reports, testimonies, and other
products we issued to congressional committees, OMB, and executive
agencies on the progress of agencies' efforts to become results
oriented. These products include, most recently, the Executive
Guide: Effectively Implementing the Government Performance and
Results Act (GAO/GGD-96-118, June 1996); GPRA Performance Reports
(GAO/GGD-96-66R, Feb. 14, 1996); GPRA: Managerial Accountability
and Flexibility Pilot Did Not Work As Intended (GAO/GGD-97-36, Apr.
10, 1997); Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997); Performance
Budgeting: Past Initiatives Offer Insights for GPRA Implementation
(GAO/AIMD-97-46, Mar. 27, 1997); and numerous congressional
testimonies. This work also included detailed examinations of
related federal management improvement efforts, in particular the
National Performance Review (NPR). In addition, we used products we
have issued in recent years on the critical management issues
confronting specific agencies, such as the Forest Service and the
Department of Energy; program efforts that cut across several
agencies, such as federal land management, food safety, and early
childhood development programs; and, most prominently, our
examination of 25 high-risk areas--those federal areas most
vulnerable to waste, fraud, and mismanagement.\12 The Related GAO
Products section at the end of this report lists selected reports and
congressional testimonies that we issued on the Results Act
implementation and related results-oriented management initiatives in
pilot and nonpilot agencies.
We reviewed OMB circulars, memoranda, and general guidance to
executive agencies on a wide range of the Results Act implementation
activities, such as the preparation and submission of strategic plans
and examples of performance measurement. For selected pilot
agencies, we reviewed performance plans and performance reports. For
these and nonpilot agencies that we selected to reflect a
representative range of federal functions, we also reviewed draft
strategic plans and other applicable management documents. To
supplement our existing information, we reviewed the American Society
of Public Administration's (ASPA) case studies on performance
management in the federal government and the National Academy of
Public Administration's (NAPA) assessment of the initial pilot
performance plans.\13
To further meet our first two objectives, we sent a questionnaire to
a random sample of 1,300 mid-level and upper level civilian managers
and supervisors working in the 24 executive branch agencies covered
by the CFO Act. These agencies represent about 97 percent of the
executive branch full-time workforce and cover over 99 percent of the
federal government's net outlay for fiscal year 1996. In reporting
this questionnaire data, when we use the term governmentwide, we are
referring to these 24 CFO Act agencies; and when we use the term
federal managers, we are referring to both managers and supervisors.
We drew our sample from the Office of Personnel Management's (OPM)
Civilian Personnel Data File (CPDF) as of June 30, 1996, using file
designators to identify individuals who perform management and
supervisory functions. We restricted our sample to individuals who
were in the United States due to the difficulty and time associated
with forwarding and receiving mail in some foreign countries.
This sample was stratified by whether the manager was SES or non-SES
and by whether the manager was working in an agency or agency
component that was designated as a Results Act pilot. The management
levels covered general schedule (GS) and general management (GM)
levels GS/GM-13 through the career Senior Executive Service (SES).
The identification of managers working in pilots was limited to
selection from the following 13 Results Act pilots: Defense
Logistics Agency, Army Research Laboratory, Social Security
Administration, Small Business Administration, Occupational Safety
and Health Administration, National Highway Traffic Safety
Administration, Forest Service, National Technical Information
Service, National Oceanographic and Atmospheric Administration,
Internal Revenue Service, U.S. Mint, Bureau of Engraving and
Printing, and the U.S. Customs Service's Office of Enforcement.
Our selection of pilots from which to sample managers was guided by
the following requirements: (1) unique CPDF agency component
identifier codes that corresponded with the scope or bounds of the
Results Act pilot, (2) the pilot designation was made before fiscal
year 1996, and (3) the pilot covered 350 or more employees. The
first restriction was needed to ensure that any managers who were
randomly selected for the pilot strata would be from an agency or
agency component that was encompassed in its entirety by the pilot.
Because many pilots were aligned according to agency functions or
processes and cut across multiple agency components, the CPDF
identifier codes could not be reliably used to ensure that a randomly
selected manager was working within the scope of the pilot. We also
excluded pilots that were designated in fiscal year 1996, because any
significant initiatives would have been fairly recent and may not
have been sufficiently implemented for any effects to be reflected in
questionnaire responses. Small-scale pilots covering fewer than 350
employees were excluded in order to allocate the sample more
efficiently, given the resources and time available to do the survey.
In general, our selected pilots, 12 of which were designated as
pilots in fiscal year 1994, contained a preponderance of pilots that
encompassed the whole agency or a major agency component.
Our questionnaire asked managers about their perceptions on such
topics as (1) agency mission and goals, (2) ways of measuring
performance, (3) the use of performance information now and 3 years
ago, (4) hindrances to measuring and using performance information,
(5) agency climate, (6) Results Act-related training, (7) knowledge
of the Act, (8) involvement in the Act, and (9) improvements seen or
expected as a result of implementation of the Act. Most of the items
on the questionnaire were closed-ended, meaning that depending on the
particular item, respondents could choose one or more response
categories or rate the strength of their perception on a 5-point
extent scale. After designing our questionnaire, we conducted
pretests at four federal agencies: the Department of Defense, the
Department of Commerce, the Department of Agriculture's Forest
Service, and the Department of Justice. In all, 32 federal managers
took part in these pretests. The questionnaire was revised after
each pretest on the basis of the feedback received.
The survey was initially sent out between November 27, 1996, and
January 3, 1997. Managers who did not respond to the initial
questionnaire were sent one follow-up questionnaire. Table 1.1
summarizes the disposition of the 1,300 surveys sent to managers
originally sampled.
Table 1.1
Analysis of Sample Dispositions
Number
-------------------------------------------------------------- ------
Total managers sampled 1,300
Selectees deleted from sample\a 47
Eligible sample 1,253
Unable to identify or locate sample selectee 22
Refused to participate 23
Questionnaires not returned 299
Questionnaires returned, not usable 4
Usable questionnaires returned 905
----------------------------------------------------------------------
\a Includes individuals who were outside the United States, retired,
separated, died, or otherwise left the agency or had some other
reason that excluded them from the population of interest.
We received usable responses from 72 percent of the eligible sample.
The response rate across strata ranged from 61 percent to 88 percent.
Although we did not test the validity of the respondents' answers or
the comments they made, we took several steps to check the quality of
our survey data. We reviewed and edited the completed
questionnaires, made internal consistency checks on selected items,
and checked the accuracy of data entry on a sample of surveys.
We followed up on a sample of the nonrespondents to assess whether
the views of the nonrespondent group differed from the views of those
who returned the survey. We selected a stratified, random sample of
101 nonrespondents to telephone and urge them to return their
questionnaires. If necessary, they were sent new questionnaires.
Nine of these nonresponse sample members returned completed
questionnaires before we contacted them. Of the remaining 92
individuals, we determined that 2 were no longer in the population of
interest and that various circumstances (e.g., maternity leave)
precluded participation by 4 others. Of the 86 individuals we did
contact, 51 returned completed, usable questionnaires; 16 refused; 1
returned the questionnaire too late for the data to be included in
the analysis; and 18 failed to return the questionnaire. We analyzed
the responses of the 51 individuals returning the questionnaire and
compared their responses on selected items to those received from all
other respondents. Our assessment indicated that across the selected
items and types of analyses performed, the responses of the follow-up
group did not demonstrate a sufficient degree of significant and
consistent differences to merit separate treatment in subsequent
analyses of the questionnaire data. The responses of the 51
individuals were included with all other responses. Appendix I
contains a copy of the questionnaire with weighted percentage of
managers responding to each item. Percents presented in this report
have been rounded to the nearest whole number.
The overall survey results are generalizable to the CFO Act agencies.
Results presented for the selected pilots are generalizable only to
those pilots, but not to all the other Results Act pilots. All
results are subject to some uncertainty or sampling error as well as
nonsampling error. Because a complex sample design was needed to
support a diverse set of planned analyses and sample size was
constrained by the resources and time available to do the survey, it
was not possible to employ a sample size that would consistently
limit overall sampling error to 5 percent or less at the 95 percent
confidence level. In general, percentages reported for the entire
sample have confidence intervals ranging from +5 percentage points to
+12 percentage points. In other words, if all CFO Act managers in
our population had been surveyed, the chances are 95 out of 100 that
the results obtained would not differ from our sample estimate, in
the most extreme case, by more than +12 percentage points. The
magnitude of sampling error will also vary in relation to the
particular groups or items being compared. As a result, the
magnitude of difference needed to attain statistical significance
between groups or items will also vary. In some instances, a
difference of a certain magnitude may be statistically significant;
in other instances, depending on the nature of the comparison being
made, a difference of equal or even greater magnitude may not be
statistically significant. We note throughout the report when
differences between groups are significant.
The objectives, scopes, and methodologies of the specific products
upon which this report is based are detailed in those products, and,
as appropriate, those products contained comments from the relevant
agencies. That work was done over the last several years. Our work
for this report was conducted between September 1996 and April 1997
in accordance with generally accepted government auditing standards.
We requested comments from the Director of OMB because of OMB's
responsibilities for designating pilots, overseeing the pilot
process, and generally leading the executive branch's Results Act
efforts. On May 21 and 22, 1997, senior OMB officials provided
technical comments. We have incorporated OMB's technical commments
where appropriate.
--------------------
\12 See, for example, High-Risk Series: An Overview (GAO/HR-97-1,
Feb. 1997).
\13 See the federal case studies developed at the initiative of ASPA
on selected agencies' strategic planning and performance measurement
efforts and Toward Useful Performance Measurement: Lessons Learned
from Initial Pilot Performance Plans Prepared Under the Government
Performance and Results Act, NAPA, Nov. 1994.
PROGRESS OF THE RESULTS ACT
IMPLEMENTATION HAS BEEN MIXED
============================================================ Chapter 2
Progress in implementing the Results Act has been mixed. The first
pilot--on performance planning and reporting--had far more
participants than required, although no participants were designated
for the second pilot on managerial accountability and flexibility,
and the third pilot on performance budgeting likely will be delayed
by at least a year. Still, OMB has taken a number of actions to
prepare for the implementation of the Act, such as expanding the
amount and prominence of performance information that is used during
the executive branch budget process. Congress has also taken actions
to demonstrate its commitment to the Act and reinforce to agencies
the importance that it places on the full and complete implementation
of the Act.
Overall, the experiences of the pilot agencies and related efforts in
nonpilot agencies show that substantial improvements in performance
are possible--even in the short term--when organizations adopt a
disciplined approach to setting goals, measuring performance, and
using performance information to improve effectiveness. However,
after 3 years of piloting the Act, the reported examples of
substantial performance improvements still are fairly isolated, and
many agencies do not appear to be well-positioned to provide a
results-oriented answer to the fundamental Results Act question:
What are we accomplishing?
Federal managers report that results-oriented performance measures
often do not exist. As a result, it is not surprising that managers
also report that results-oriented performance information frequently
is not used to make important decisions affecting their agencies and
programs, even when performance information is available.
THE NUMBER OF PERFORMANCE
PLANNING AND REPORTING PILOTS
EXCEEDED THE RESULTS ACT
REQUIREMENTS
---------------------------------------------------------- Chapter 2:1
The number of the performance planning and reporting pilots exceeded
the Act requirements, thereby giving a large number of organizations
a head start on the Results Act implementation and providing a rich
body of experience for nonpilot organizations to draw upon.\1 For the
first set of pilot projects, the Act required OMB to select at least
10 agencies to pilot the performance planning and reporting
requirements for 1 or more of an agency's major functions and
operations during fiscal years 1994 through 1996. However, a total
of 77 pilot projects, including both entire agencies--e.g., the
Defense Logistics Agency (DLA), the Internal Revenue Service (IRS),
and the Social Security Administration (SSA)--and programs--e.g., the
Department of Veterans Affairs (VA) Loan Guaranty Program,
participated at some point during this first pilot. When this part
of the pilot phase concluded at the end of fiscal year 1996, a total
of 68 pilots representing 28 agencies were project participants.
The pilots represented a wide range of government activities and
functions. For example, the first set of pilot projects included
military operations, such as those conducted by the Air Force Air
Combat Command in the Department of Defense (DOD); regulatory
programs, such as those conducted by the Occupational Safety and
Health Administration (OSHA) in the Department of Labor;
intergovernmental programs, such as those conducted by the Office of
Child Support Enforcement (OCSE) in the Department of Health and
Human Services (HHS); and businesslike functions, such as those
conducted by the U.S. Mint in the Department of the Treasury.
--------------------
\1 During the pilot phase, GAO and others issued various assessments
of pilot agencies. See, for example, GAO/GGD-96-66R, February 14,
1996, and the federal case studies developed at the initiative of
ASPA on selected agencies' strategic planning and performance
measurement efforts.
MANAGERIAL ACCOUNTABILITY AND
FLEXIBILITY PILOT EFFORT DID
NOT WORK AS INTENDED
---------------------------------------------------------- Chapter 2:2
The Results Act required OMB to select a second set of pilot projects
during fiscal years 1995 and 1996 in order to determine the effects
of providing managers of federal programs with increased managerial
flexibility in exchange for the greater accountability for
performance. The Act required OMB to designate at least five
agencies from the first set of pilot projects on performance planning
and reporting to participate in the second pilot on managerial
accountability and flexibility. However, we found that the
managerial accountability and flexibility pilot did not work as
intended.\2 OMB did not designate as pilot projects any of the 7
departments and 1 independent agency that submitted a total of 61
waiver proposals.
We found that three major factors contributed to the failure of the
managerial accountability and flexibility pilot to work as intended.
First, changes in federal management practices and laws that occurred
after the Act was enacted affected agencies' need for the managerial
flexibility waivers. For example, the Federal Workforce
Restructuring Act of 1994 established a new personnel ceiling for all
of the executive branch, which effectively limited OMB's ability to
waive agency personnel ceilings established in the budget. Second,
agencies could use other, less rigorous, means to obtain waivers from
administrative requirements, such as the administration's NPR
reinvention labs, which were designed to test ways that agencies
could improve their performance and customer service by reengineering
work processes and eliminating unnecessary regulations. Third,
unlike its proactive approach to the first set of pilots, OMB did not
work actively with agencies that were seeking to take part in the
managerial accountability and flexibility pilot. For example, OMB's
feedback to agencies concerning their waiver proposals for
consideration as pilots was very limited. This contrasted directly
with the approach that OMB took with the performance planning and
reporting pilots, for which it issued a summary assessment of the
1994 performance plans, including the strengths, weaknesses, and
additional actions agencies needed to take to improve these plans.
--------------------
\2 GPRA: Managerial Accountability and Flexibility Pilot Did Not
Work As Intended (GAO/GGD-97-36, Apr. 10, 1997).
PERFORMANCE BUDGETING PILOTS
LIKELY TO BE DELAYED
---------------------------------------------------------- Chapter 2:3
OMB is also to select a third set of pilot projects to test
performance budgeting for fiscal years 1998 and 1999. Under the
test, pilot agencies are to prepare performance budgets, which are to
provide Congress with information on the direct relationship between
proposed program spending and expected program results. The Senate
committee report on the Results Act said that the performance
budgeting pilots are to begin "only after agencies had sufficient
experience in preparing strategic and performance plans, and several
years of collecting performance data."\3 Because of this and because
it recognizes the importance of concentrating on governmentwide
Results Act implementation in fiscal year 1998, OMB has indicated
that these pilots will be delayed for at least a year. As envisioned
under the Act, performance budgets must show the effects on
performance of marginal changes in funding. According to OMB, few
agencies currently have either sufficient baseline performance or
financial information or the ability to use sophisticated analytic
techniques to calculate the effects on performance of marginal
changes in funding. According to OMB, delaying the performance
budgeting pilots will provide agencies time to develop needed
information and abilities.
--------------------
\3 S. Rep. No. 58, 103d Cong. 1st Sess. p. 38. (1993).
OMB ACTIONS TO PREPARE THE
EXECUTIVE BRANCH FOR THE
RESULTS ACT IMPLEMENTATION
---------------------------------------------------------- Chapter 2:4
In the nearly 4 years since the passage of the Results Act, OMB has
sought to expand the amount and prominence of performance information
within the executive branch budget process. For example, our 1995
review of budget documents and interviews with OMB staff 1 year after
OMB 2000--a major April 1994 reorganization and process change in
OMB--indicated that there was greater attention paid to agency
management issues and the use of performance information in the
fiscal year 1996 budget process than there had been in the fiscal
year 1995 process.\4 Whereas the fiscal year 1995 documents discussed
streamlining primarily in terms of the number of positions to be
eliminated, the fiscal year 1996 documents also included discussions
about how proposed staff reductions could affect the agency's
performance.
We reported that OMB continued to expand its focus on performance
measures and information during the fiscal year 1997 budget
preparation process. Between March and June 1995, OMB held a "Spring
Review" as a prelude for development of the fiscal year 1997 budget
that was intended to help OMB and agencies work together to identify
useful performance information. OMB conducted a "Summer Review" in
July 1996 that examined the status of agencies' strategic planning
efforts. Guidance from OMB to executive agencies stated that the
review would focus on the adequacy, relevance, and appropriateness of
their mission statements and their strategic goals.
The 1996 "Fall Review," according to OMB, was intended to produce
agreement between OMB and the agencies on the performance goals and
measures that agencies would include in their fiscal year 1999 annual
performance plans. OMB's most recent effort, which was under way in
the spring of 1997, was its strategic assessment initiative. The
purpose of this initiative, which focused on agencies' goals,
commitments, and main priorities, is to assist agencies in the
preparation of their strategic plans and annual performance plans
that are due in September 1997.
To help ensure the success of the Results Act, the CFO Council,
chaired by OMB's Deputy Director for Management, established a GPRA
Implementation Committee.\5 The Committee provides guidance and
information to CFOs and managers in the 24 agencies covered by the
CFO Act. The Committee, concluding that uncertainty or fear of
failure may immobilize an agency's efforts to implement the Act and
that its implementation is evolutionary in that proficiency comes
with time and experience, has been actively working to assist federal
managers with the implementation of the Act. As part of that effort,
the Committee published a set of guiding principles and key issues
for implementing the Act.\6 More recently, the Committee issued a
user guide to help federal managers meet the Results Act requirements
and, among other things, develop a framework for using performance
information in making budget decisions.\7 The Committee also has
undertaken a number of training efforts for federal managers,
including sponsoring a conference on the Act in the fall of 1996 and
developing a training package. The purpose of the training package,
which was provided to executive agencies, was to build awareness and
support for the Act among political appointees and senior career
staff.
--------------------
\4 Office of Management and Budget: Changes Resulting From the OMB
2000 Reorganization (GAO/GGD/AIMD-96-50, Dec. 29, 1995).
\5 The CFO Council was created by the CFO Act to provide the
leadership foundation necessary to effectively carry out the
responsibilities of executive agency CFOs.
\6 Implementation of the Government Performance and Results Act
(GPRA), A Report on the Chief Financial Officer's Role and Other
Issues Critical to the Governmentwide Success of GPRA, Chief
Financial Officers Council, GPRA Implementation Committee, May 1995.
\7 Integrating Performance Measurement into the Budget Process: A
User Guide to Multiple Ideas, Practices, Formats, Processes, Steps,
and One Model Tying Everything Together, Chief Financial Officers
Council, GPRA Implementation Committee, Jan. 21, 1997.
CONGRESSIONAL COMMITMENT TO THE
RESULTS ACT IS REFLECTED IN ITS
INCREASED INVOLVEMENT AND
ACTIVITIES
---------------------------------------------------------- Chapter 2:5
Congressional involvement is critical to sustain the momentum of the
Results Act implementation and to reinforce to agencies the
importance that Congress places on the successful and thorough
implementation of the Act. We testified in March 1996 that officials
in some pilot agencies believed Congress' interest in the Act was
limited since, for example, Congress seldom asked about the
implementation of the Act in their agencies.\8 However, as the date
for governmentwide implementation has approached, Congress has
signaled its strong commitment to the Act through periodic hearings
and other actions. Most prominently, on February 25, 1997, the
Speaker of the House, the House Majority Leader, the Senate Majority
Leader, and key committee chairmen from the House and the Senate sent
a letter to the Director of OMB. The letter underscored the
importance that the congressional Majority places on the
implementation of the Results Act, noted a willingness on the part of
Congress to work cooperatively with the administration, and
established expectations for consultations. Building on its
commitment to the Act, Congress has begun to consult with agencies on
their strategic plans. In the House, the Majority's consultation
effort is being led by teams consisting of staff from various
committees that are focusing on specific agencies. These teams have
been reaching out to agencies to review and comment on agencies'
strategic plans.
--------------------
\8 Managing for Results: Achieving GPRA's Objectives Requires Strong
Congressional Role (GAO/T-GGD-96-79, Mar. 6, 1996).
SOME AGENCIES REPORTED IMPROVED
PERFORMANCE
---------------------------------------------------------- Chapter 2:6
The Results Act pilot phase has shown that short-term, substantial
program improvements are possible when organizations adopt a
disciplined approach to setting goals, measuring performance, and
using performance information to improve effectiveness, although the
reported examples of substantial performance improvements still are
limited, as is to be expected at this early stage of implementation.
For example, as highlighted in our Executive Guide, the Coast Guard
reexamined its mission and shifted the focus of its marine safety
program from regulation to one that includes education.\9 This change
contributed to a significant decline in the towing industry fatality
rate--from 91 per 100,000 industry employees in 1990 to 27 per
100,000 in 1995. Similarly, the Veterans Health Administration
improved services to veterans by more rigorously assessing the
results of the medical care it provides to the nation's veterans. In
particular, the Veterans Health Administration reported that it used
performance information to target the most important improvement
opportunities and thereby lowered the mortality rate for cardiac
procedures by an average of 13 percent over the last 8 years.
We also have identified other examples of agencies that have reported
improvements to their operations. SSA has had a single, national
toll-free number as an alternative to field office contact for
conducting simple business since 1989. On the basis of customer
feedback indicating that this convenient telephone service was
important to the public, yet had persistently high busy rates, SSA
took several steps to increase public access, including doubling the
trained and available workforce to answer 800-number calls.
Primarily as a result of the workforce increase, the busy rate
decreased from 49 to 34 percent, and the percentage of calls that
were answered within 5 minutes increased from 74 to 83 percent from
fiscal years 1995 to 1996.\10
DLA shows how an agency can make substantial improvements in its
service to its customers and inventory management. DLA's
reengineering efforts for its delivery and supply processes focused
not only on saving time but also on lowering costs, two results
valued by its customers. DLA focused on improved partnerships
between suppliers and its customers, primarily DOD facilities.
According to DLA, this overall focus enabled it to reduce the
delivery time of nearly all medical supplies from 30 days in 1993 to
within the current 24 hours; at the same time, it reduced prices by
35 percent. DLA also reported that it is providing Internet ordering
of primarily electronics parts, which took from 35 to 40 days from
purchase to receipt prior to 1996 and now takes from 3 to 5 days,
with an estimated savings of $500 million over 10 years. DLA further
reported that it avoided inventory holding and storage costs of $95.7
million in fiscal year 1995, with a projection of an additional $353
million over the next 5 years. DLA's initiatives have had
departmentwide impact. We estimated in March 1997 that between
September 1991 and September 1996, DOD reduced its pharmaceutical,
medical, and surgical inventories and associated management costs by
about $714 million.\11
The Food and Drug Administration (FDA) is allowed to collect user
fees from drug companies seeking approval to market drugs under the
Prescription Drug User Fee Act of 1992. The law dedicates the
revenues to expediting FDA's reviews of human drug applications and
incorporated time-specific performance goals that FDA is to meet by
the end of fiscal year 1997. To satisfy these objectives, FDA
consulted with its stakeholders to determine appropriate performance
measures and target levels of performance and developed
output-oriented performance goals. FDA reported that the program had
exceeded its performance goals, improving the speed and efficiency of
the drug review process.\12 Specifically, the median time taken to
approve 53 applications for human drugs that have not been marketed
in the United States before was 14.3 months in fiscal year 1996. In
comparison, in 1993, which was the beginning of the program, FDA
approved only 25 applications, and the median approval time was about
23 months.
The U.S. Mint, as one part of its mission, manufactures and markets
numismatic and commemorative coins and products. Before 1994, the
Mint estimated it took from 8 weeks to about 27 weeks to fill
customers' orders for coins. Because of the number of customer
inquiries about the status of their orders, the Mint learned that the
time it took to fill orders was a problem. Subsequently, the Mint
set goals and reduced the time to fill orders. As of January 1997,
97 percent of the orders were filled within 4 weeks. Also, the time
for customers to get replacements has declined--from 47 to 7
days--from October 1995 to October 1996.
--------------------
\9 GAO/GGD-96-118, June 1996.
\10 Social Security Administration: Significant Challenges Await New
Commissioner (GAO/HEHS-97-53, Feb. 20, 1997).
\11 Defense Inventory Management: Problems, Progress, and Additional
Actions Needed (GAO/T-NSIAD-97-109, Mar. 20, 1997).
\12 Department of Health and Human Services: Management Challenges
and Opportunities (GAO/T-HEHS-97-98, Mar. 18, 1997).
MANAGERS REPORTED
RESULTS-ORIENTED PERFORMANCE
MEASURES AND THEIR USE WAS
LIMITED
---------------------------------------------------------- Chapter 2:7
Although agencies have reported selected instances where they have
used results-oriented performance information to make important
improvements in performance, federal managers reported to us that
overall, results-oriented performance measures often were not
available, and such information was not frequently used to help make
important decisions affecting their programs and agencies. According
to our survey, although 76 percent of federal managers governmentwide
reported that the programs, operations, or projects they were
involved in had performance measures, such measures typically were
not results oriented. That is, the measures failed to help answer an
essential question, "Is this program accomplishing its intended
results and contributing to the achievement of the agency's strategic
goals and mission?"
MANAGERS REPORTED LIMITED
RESULTS-ORIENTED PERFORMANCE
MEASUREMENT
-------------------------------------------------------- Chapter 2:7.1
In crafting the Results Act, Congress recognized that congressional
and agency decisionmaking often had been hampered by the absence of
sound information on the results of federal programs. In the past,
to the extent performance information was collected, agencies
generally collected information primarily on such things as the
amount of dollars received and spent, staffing levels, and the number
of activities performed. Although such information is valuable for
program management and accountability, it does not present a complete
picture of the results of the federal effort. Our more recent work
has confirmed that results-oriented performance information still
does not exist as often as it should. For example, the Department of
Housing and Urban Development's (HUD) Public Housing Management
Assessment Program has a system that collects data, such as
outstanding work orders and uncollected rents, to measure the
performance of housing authorities. However, the system does not
collect data on housing quality or the quality of maintenance, which
are essential for assessing the results that housing authorities are
achieving and for determining which housing authorities are
performing well or poorly.\13
Our survey of federal managers suggests that these are not isolated
occurrences of a lack of performance information in federal agencies.
Table 2.1 shows that, according to our survey, 32 percent of federal
managers said that, to a great or very great extent, they have the
types of performance measures that would demonstrate whether their
programs or operations were achieving their intended result. The
table also shows that 38 percent or less of federal managers reported
having, to a great or very great extent, other important
performance-related information, such as efficiency and quality
measures.
Table 2.1
Percent of Federal Managers Who Reported
Their Programs Have Various Performance
Measures and Recollect They Had Such
Measures 3 Years Ago
3 years Difference
Measures that ago Currently \a
------------------------------------ -------- ---------- ----------
would demonstrate to someone outside 19 32 +13
their agencies whether or not they
are achieving their intended
results
tell whether or not they are 11 32 +21
satisfying their customers
tell about the quality of the 19 31 +12
products or services they provide
tell how many things they produce or 27 38 +11
services they provide
tell if they are operating 17 26 + 9
efficiently
----------------------------------------------------------------------
\a All differences are statistically significant.
Source: GAO survey data.
Despite the lower percentage of federal managers reporting that their
programs or operations have performance-related measures to a great
or very great extent, there where statistically significant increases
in the percent of federal managers reporting the existence of these
types of measures over what they recollected the situation to have
been 3 years ago. For example, when asked to recollect what the
situation was 3 years ago, 19 percent of federal managers reported
that, to a great or very great extent, they had results-oriented
measures, compared to 32 percent who reported that they had such
measures today. This represents a 13 percentage point change over
what federal managers perceived the situation to have been 3 years
ago, suggesting that results-oriented performance information, which
is essential to the success of the Results Act, is becoming more
widely available.
According to federal managers' responses to our survey, it appears
that agencies are making the greatest change in developing measures
for assessing customer satisfaction. When asked to recollect what
the situation was 3 years ago, 11 percent of federal managers
governmentwide reported that, to a great or very great extent, they
had measures for assessing customer satisfaction. However, 32
percent of federal managers reported that, to a great or very great
extent, they currently had such measures. This represents a 21
percentage point change. This reported change in agencies' efforts
to measure customer satisfaction may be a reflection of agencies'
efforts to comply with Executive Order No. 12862, September 1993,
which required agencies to establish and implement customer service
standards to guide their operations.
--------------------
\13 Public Housing: HUD Should Improve the Usefulness and Accuracy
of its Management Assessment Program (GAO/RCED-97-27, Jan. 29,
1997).
MANAGERS REPORTED LIMITED
USE OF RESULTS-ORIENTED
PERFORMANCE INFORMATION
-------------------------------------------------------- Chapter 2:7.2
Given the limited reported availability of results-oriented
information, it is not surprising that federal managers also reported
limited use of such information to make decisions affecting their
programs and agencies, as shown in table 2.2.
Table 2.2
Percent of Federal Managers Reporting
Extent to Which Results-Oriented
Performance Information Is Used
Currently and Compared to 3 Years Ago
Results-oriented
performance 3 years
information ago Currently Difference
---------------------- ---------------------- ---------- ----------
used by the agency to 16 21 +5\a
help develop its
budget
used as the basis for 14 20 +6\a
funding decisions
used as the basis for 9 13 +4\b
legislative changes
used as the basis for 12 16 +4\a
program changes by
management above
respondent
----------------------------------------------------------------------
\a Statistically significant.
\b Approaches statistical significance.
Source: GAO survey data.
Table 2.2 shows that federal managers reported some improvements over
what they recollected the situation to have been 3 years ago.
Although these improvements are statistically significant for most of
the uses shown in the table, these improvements have been modest.
The reported use of results-oriented performance information to help
make key decisions was not high even among federal managers who
reported that they had performance measures that demonstrate their
programs are achieving intended results, to a great or very great
extent. At most, 37 percent these federal managers reported that
results-oriented performance information was used to a great or very
great extent. Table 2.3 shows these results.
Table 2.3
Percent of Federal Managers Reporting
Use of Performance Information to a
Great or Very Great Extent Among
Managers With Performance Measures
Demonstrating Achievement of Results to
a Great or Very Great Extent
Percent of federal
managers reporting
use to a great or
very great
Results-oriented performance information extent\a
-------------------------------------------------- ------------------
used by the agency to help develop its budget 37
used as the basis for funding decisions 34
used as the basis for legislative changes 21
used as the basis for program changes by 31
management above respondent
----------------------------------------------------------------------
\a All percentages of use are statistically significant and higher
compared to respective percentages of use for all other managers.
Source: GAO survey data.
CONCLUSIONS
---------------------------------------------------------- Chapter 2:8
The Results Act performance planning and reporting pilot phase
provided many agencies with important experience in meeting key
requirements of the Act. These experiences, along with OMB's efforts
to work with agencies on the Act, suggest that most agencies are
likely to meet the upcoming statutory deadlines for producing initial
strategic plans and annual performance plans. More importantly, the
experiences of the pilot agencies and related efforts by nonpilot
agencies show that substantial improvements in performance are
possible when organizations adopt a disciplined approach to setting
goals, measuring performance, and using performance information to
improve effectiveness.
However, the pilot phase also underscored how far organizations still
have to progress in the development and use of results-oriented
performance information. Although there has been progress over the
last 3 years, agencies still have not developed the information
necessary to determine whether their programs are accomplishing their
intended results. Even in those instances where agencies have
results-oriented information, agencies are generally not using the
information to a great extent to make decisions affecting their
programs. The subsequent chapters will detail that agencies face a
number of formidable challenges in establishing clear missions and
results-oriented goals, measuring their performance, building the
organizational cultures needed to sustain a focus on results, and
linking performance plans to the budget process.
UNCLEAR AGENCY MISSIONS AND GOALS
WILL CHALLENGE RESULTS ACT
IMPLEMENTATION
============================================================ Chapter 3
Reaching a reasonable degree of consensus on coordinating the
missions and goals of individual agencies that are involved in
crosscutting program areas and balancing individual agencies'
multiple strategic goals can raise contentious policy issues that may
not be easily resolved. The Results Act, with its focus on results
and requirement for consultation, can help Congress and the executive
branch clarify over the long term these management issues confronting
federal agencies.
Our work has shown that Congress' and the executive branch's approach
to addressing national issues has, in many cases, resulted in overlap
and fragmentation in federal program efforts. Coordinating these
program efforts will take time and may, in some instances, require
legislative action. Moreover, many agencies confront competing
demands that are a natural by-product of the complex social and
political environment in which programs are created, funded, and
managed. Balancing these demands will continue to be an ongoing
process for federal agencies.
Efforts under the Results Act will not answer questions about whether
the crosscutting program efforts should be consolidated or
reorganized or about the best balance for competing priorities.
Those are policy decisions that should appropriately consider many
factors, including program performance. However, congressional
consultations on agencies' strategic plans provide an ongoing
opportunity for Congress and the executive branch to work together to
identify the extent and potential consequences of overlap and
fragmentation in federal program efforts and to identify the
performance trade-offs associated with striking different balances of
competing demands.
CROSSCUTTING FEDERAL PROGRAMS
POSE CHALLENGE FOR RESULTS ACT
IMPLEMENTATION
---------------------------------------------------------- Chapter 3:1
Although federal programs have been designed for different purposes
or targeted for different population groups, coordination among
federal programs with related responsibilities is essential to
efficiently and effectively meet national concerns. Uncoordinated
program efforts can waste scarce funds, confuse and frustrate program
customers, and limit the overall effectiveness of the federal effort.
A focus on results, as envisioned by the Results Act, implies that
federal programs contributing to the same or similar results should
be closely coordinated to ensure that goals are consistent, and, as
appropriate, program efforts are mutually reinforcing. This means
that federal agencies are to look beyond their organizational
boundaries and coordinate with other agencies to ensure that their
efforts are aligned. Although some initiatives are under way in this
regard, fully coordinating federal programs will take time and
require sustained attention.
Our work has shown that as the federal government has responded over
time to new needs and problems, many federal agencies have been given
responsibilities for addressing the same or similar national
issues.\1 Of the 18 federal missions or areas of national need
described as budget functions, 14 were addressed by more than 1
executive branch department or major agency in fiscal year 1996, such
as health, international affairs, and justice. Six of the national
missions, including education, income security, and commerce and
housing credit were addressed by six or more executive branch
departments and major agencies. Some of this shared responsibility
was intended to recognize that addressing some issues from a national
perspective would necessarily involve more than one federal agency or
more than one approach. For example, we have reported that
multiagency initiatives to address environmental concerns recognize
that many programs can have an impact on the environment.\2
However, in many program areas--such as in food safety, employment
training, early childhood development, at-risk and delinquent youth
programs, federal land management, and national laboratories--our
work suggested that significant overlap and fragmentation existed in
the federal response to national needs and problems.\3 For example,
in 1995 we testified on the Department of Education's programs that
provided loans and grants to students to help finance their higher
education.\4 We found that although the student loan and Pell grant
programs provided the majority of federal financial aid to students
for postsecondary education, another 22 smaller programs were
targeted to specific segments of the postsecondary school population,
such as prospective students from disadvantaged families or women and
minorities who are underrepresented in graduate education. These 22
programs were collectively funded at $1.1 billion for fiscal year
1995. We concluded that these smaller grant programs may be
considered candidates for consolidation--with other larger programs
or among themselves--with no adverse impact on students' access to
postsecondary education and that the federal government could
anticipate administrative savings of 10 percent each year, or a total
of $550 million in budget authority (adjusted for inflation) over 5
years.\5
Similarly, we reported in 1995 on the federal programs that assisted
urban communities and their residents through a complex system
involving multiple federal agencies.\6 At that time, the Catalog of
Federal Domestic Assistance listed 342 economic development-related
programs administered by 13 agencies.\7 These programs were in the
areas of housing, economic development, and social services. We
reported that considered individually, many of these programs may
have made sense. However, the proliferation of federal programs and
the lack of coordination among agencies could impose a burden on
local organizations that attempted to piece together programs to
serve their communities. In particular, the neighborhood
organizations we studied found it burdensome to manage multiple
programs with individual funding streams, application requirements,
and reporting expectations.
--------------------
\1 Budget Issues: Fiscal Year 1996 Agency Spending by Budget
Function (GAO/AIMD-97-95, May 13, 1997); and Government
Restructuring: Identifying Potential Duplication in Federal Missions
and Approaches (GAO/T-AIMD-95-161, June 7, 1995).
\2 Restoring the Everglades: Public Participation in Federal Efforts
(GAO/RCED-96-5, Oct. 24, 1995); and Ecosystem Management:
Additional Actions Needed to Adequately Test a Promising Approach
(GAO/RCED-94-111, Aug. 16, 1994).
\3 Government Reorganization: Issues and Principles
(GAO/T-GGD-AIMD-95-166, May 17, 1995); At-Risk and Delinquent Youth:
Multiple Federal Programs Raise Efficiency Questions (GAO/HEHS-96-34,
Mar. 6, 1996); Federal Land Management: Streamlining and
Reorganization Issues (GAO/T-RCED-96-209, June 27, 1996); and Federal
R&D Laboratories (GAO/RCED/NSIAD-96-78R, Feb. 29, 1996).
\4 Department of Education: Information on Consolidation
Opportunities and Student Aid (GAO/T-HEHS-95-130, Apr. 6, 1995); and
Department of Education: Opportunities to Realize Savings
(GAO/T-HEHS-95-56, Jan. 18, 1995).
\5 Addressing The Deficit: Budgetary Implications of Selected GAO
Work for Fiscal Year 1998 (GAO/OCG-97-2, Mar. 14, 1997).
\6 Economic Development Programs (GAO/RCED-95-251R, July 28, 1995);
and Community Development: Comprehensive Approaches and Local
Flexibility Issues (GAO/T-RCED-96-53, Dec. 5, 1995).
\7 The Catalog of Federal Domestic Assistance is a governmentwide
compendium of federal programs, projects, services, and activities
coordinated by OMB and compiled by the General Services
Administration.
EFFORTS ARE UNDER WAY TO
COORDINATE CROSSCUTTING
PROGRAMS
-------------------------------------------------------- Chapter 3:1.1
In recent years, a number of efforts have been undertaken to
coordinate federal programs to help ensure that national needs are
being effectively targeted. These efforts--some of which were
started before the Results Act--have shown that coordinating
crosscutting programs takes time and requires sustained attention.
Moreover, because of the statutory bases of crosscutting programs,
congressional involvement is often needed to integrate the federal
response to national needs.
Within the executive branch, federal agencies have initiated several
efforts to address crosscutting issues. As of March 1997, about 25
interagency groups had been formed to discuss common concerns in
crosscutting issues, including the Federal Credit Policy Working
Group, the Research Roundtable, and the Interagency Regulatory
Reinvention Forum, among others.\8 Typically, these groups were
sponsored by a lead agency and provided a forum for agency officials
to discuss informally a wide range of concerns, such as goal-setting
and performance measures, but not full coordination of program
efforts. The Federal Credit Policy Working Group worked on
appropriate goals and performance measures for credit programs. The
Research Roundtable produced a paper on the challenges of developing
and reporting performance measures for research programs. The
Interagency Regulatory Forum discussed common concerns among
regulatory agencies, including identifying customers and conducting
customer service surveys. Although these agency forums have provided
an important opportunity for agencies to work together to address
common concerns in goal-setting and performance measurement, they
have not generally attempted to coordinate crosscutting program
efforts. Thus, despite these initiatives, during its review of major
portions of agencies' strategic plans in the summer of 1996, OMB
concluded that on the whole, there was little sign of significant
coordination among agencies.
Addressing the overlap and fragmentation in federal program efforts
often requires congressional involvement. In particular, legislative
action can ensure that the goals and measures of existing programs
are coordinated to meet federal needs. For example, Congress
incorporated Healthy People objectives for the year 2000 into
national legislation. Started in 1979, Healthy People is a series of
outcome-based public health objectives and measures developed and
updated each decade by the U.S. Public Health Service in
consultation with other federal agencies, state governments, and
national organizations. The year 2000 objectives succeeded the 1990
health objectives that were set in 1980. The Public Health Service
has started planning for the 2010 health objectives. Over time,
Congress has required three federal programs to incorporate Healthy
People objectives for the year 2000 into their goals; and in one of
those programs, it has linked resource allocations to Healthy People
objectives. The first, in 1989, was the requirement that the
Maternal and Child Health Block Grant Program, Title V of the Social
Security Act, report on the progress made by states toward
accomplishing Healthy People objectives on maternal and child health.
The Indian Health Care Improvement Act, as amended in 1992, created
an annual reporting requirement for the Indian Health Service to
inform Congress on the health status of American Indians and Alaska
Natives. In addition, the 1992 authorization of the Preventive
Health and Health Services Block Grant program linked the state
grant-funded activities to Healthy People objectives.
However, the mere existence of a legislated coordinating mechanism
does not ensure success, and sustained effort is required to build
consensus on common goals and how best to coordinate approaches. For
example, the Office of National Drug Control Policy (ONDCP),
established in 1988 by law, is responsible for producing the national
drug control strategy and coordinating its implementation with other
federal agencies. Given the complexity of drug control efforts and
the fragmentation of the approach to the national drug control
strategy among more than 50 federal agencies, we have endorsed the
need for a central planning agency, such as ONDCP, to coordinate the
nation's efforts and recommended its reauthorization for a finite
period of time.\9 However, our work has shown that despite some
successes, international drug control efforts have not materially
reduced the availability of drugs in the United States.\10 One reason
was a lack of ways to tell whether or how well counternarcotics
efforts were contributing to the goals of the national strategy.
Consistent with the intent of the Results Act, we recommended that
ONDCP complete a long-term plan with meaningful performance measures
and multiyear funding needs that were linked to the goals and
objectives of the international drug control strategy. Such a plan
would provide managers and policymakers with the information to make
more informed decisions on prioritizing funding levels based on
performance and results. ONDCP concurred with our recommendation.
In February 1997, ONDCP proposed a 10-year strategy with five major
goals--including "break foreign and domestic drug sources of
supply"--as the basis for a long-term national effort and stated that
it will continue to work on a 5-year drug control budget concept.
ONDCP is making progress toward developing performance targets and
measures for each of its goals.
--------------------
\8 Information provided by Carl J. Metzger, Management Systems
International, Inc., Washington, D.C., March 1997.
\9 Drug Control: Reauthorization of the Office of National Drug
Control Policy (GAO/T-GGD-97-97, May 1, 1997); and Drug Control:
Observations on Elements of the Federal Drug Control Strategy
(GAO/GGD-97-42, Mar. 14, 1997).
\10 Drug Control: Long-Standing Problems Hinder U.S. International
Efforts (GAO/NSIAD-97-75, Feb. 27, 1997).
BALANCING GOALS IS COMPLICATED
BY COMPETING DEMANDS
---------------------------------------------------------- Chapter 3:2
Federal agencies often face a variety of competing priorities that
force congressional and executive branch decisionmakers to balance
public expectations, cost, quality, and other factors. These
multiple priorities, which in many cases are built into the intent
and design of an agency or program, constitute one of the primary
challenges to public sector goal-setting because priorities
continually need to be balanced; otherwise, one or two priorities may
be inappropriately overemphasized at the expense of others.
According to our survey, nearly half of the federal managers reported
that reconciling differing congressional views and the views of other
parties on an agency's mission and strategic goals has been, or will
be, somewhat to very difficult. A greater proportion of federal
managers from the Results Act pilots that we were able to isolate
from our survey reported difficulty, suggesting that as agencies gain
experience with the Results Act they will come to better appreciate
the need for, and difficulty of, reconciling competing views.
Fifty-eight percent of managers in selected Act pilots reported that
reconciling differing congressional views has been or will be
somewhat to very difficult, compared to 46 percent of federal
managers in all other agencies. Similarly, 59 percent of managers in
selected Results Act pilots reported that reconciling the views of
other parties has been or will be somewhat to very difficult,
compared to 48 percent of federal managers in all other agencies.
The differences between the managers in selected Results Act pilots
and federal managers in all other agencies for reconciling differing
congressional views and the views of other parties are statistically
significant.
We have identified numerous examples of agencies that can benefit
from using the Results Act to help them better balance competing
priorities and thereby improve their effectiveness. For example, we
reported in April 1997 that the Forest Service had been increasingly
shifting the emphasis of its efforts from producing timber to
sustaining wildlife.\11 This shift was taking place in reaction to
requirements in planning and to environmental laws and their judicial
interpretation--reflecting changing public values and
concerns--together with social, ecological, and other factors.
However, we noted the demand for recreation is also expected to grow
and may increasingly conflict with sustaining wildlife and producing
timber. We found that the disagreement both within the Forest
Service and among key external stakeholders, including Congress, on
how the Forest Service is to resolve conflicts or make choices among
competing uses on its lands had seriously undermined its ability to
establish the goals and performance measures needed to ensure its
accountability. Until general agreement is reached, we believe the
Forest Service's decisionmaking will continue to be inefficient and
ineffective.
Our ongoing work examining federal credit agencies identified
multiple competing priorities as a challenge in setting
results-oriented goals for those programs. Agency officials told us
that on the one hand, they are expected to increase program service
while also reducing program costs and minimizing default rates.
However, these programs were designed to offer credit to a population
that the private sector would consider high-risk. For example, to be
eligible for a rural housing loan, the borrower must be ineligible to
get credit from any other source. Therefore, the target population
may be more likely to default on a loan.
The need to identify and balance competing demands can be especially
complicated in cases where an agency lacks an integrated legislative
mission. For example, the Environmental Protection Agency (EPA) was
established in 1970 under a presidential reorganization plan in
response to concerns over protecting public health as well as air,
water, and land resources. Since then, as environmental threats were
identified, Congress gave EPA responsibility for implementing over a
dozen environmental laws. We reported in May 1995 that because it
did not have an overarching legislative mission and its environmental
responsibilities had not been integrated, EPA had not been able to
target its resources to reduce the greatest risks to human health and
the environment.\12 EPA has been working with state governments,
industry, environmental groups, and other interested parties since
1992 to select the most appropriate national environmental goals.
Building on these goals, EPA is also ranking the relative risk of
environmental problems to help identify the most appropriate goals
for the agency.
--------------------
\11 Forest Service Decision-Making: A Framework for Improving
Performance (GAO/RCED-97-71, Apr. 29, 1997).
\12 Environmental Protection: Current Environmental Challenges
Require New Approaches (GAO/T-RCED-95-190, May 17, 1995).
CONGRESSIONAL CONSULTATIONS
PROVIDE MECHANISM FOR
IDENTIFYING AND ADDRESSING
CROSSCUTTING ISSUES AND SEEKING
TO BALANCE PRIORITIES
---------------------------------------------------------- Chapter 3:3
The Results Act requirement that agencies consult with Congress in
developing their strategic plans presents an important opportunity
for congressional committees and the executive branch to work
together to identify the extent and the potential consequences of
overlap and fragmentation of federal program efforts.\13 As agencies
work with congressional committees on developing, and subsequently
updating, their strategic plans, they can identify legislative
changes that may be needed to clarify congressional intent as well as
legislative adjustments to better ensure an effective, coordinated
response to national issues. The consultation process should also be
helpful to agencies in addressing key management issues, such as
identifying multiple priorities, reaching a workable agreement on
priorities, and then establishing results-oriented goals to reflect
the balance struck among those priorities. The requirement for
consultations is also perhaps the Results Act's most significant
challenge, because consultations will likely raise contentious policy
issues that are inherent in the political process.
We testified in February 1997 that although no single set of best
practices has yet emerged for the Results Act consultation process,
making consultations most useful would require that both Congress and
agencies be actively engaged in the effort.\14 On the whole, the
House committee staff we spoke with stressed the very limited nature
of the meetings with agencies that had taken place through February
1997. Rather than consultations, House staff characterized the
meetings as briefings, preconsultations, or preliminary
consultations. As of April 30, 1997, however, some House Majority
committee staff we spoke with had expressed concern that in their
view, not enough agencies had provided draft strategic plans for
consultations. In particular, a number of agencies have told
congressional staff that draft plans will not be available until June
or early July. Congressional staff have said that an important
opportunity for Congress and the administration to clarify and seek
agreement on agencies' missions, goals, and strategies will be lost
if draft plans are not provided early enough to permit meaningful
reviews and consultations during the consultation process. For
example, although EPA provided a detailed outline of its strategic
plan, including a mission statement, goals, and strategic
"principles," completing a draft plan in early July may not leave
adequate time for EPA to obtain congressional input and fully
consider and incorporate congressional views before the final plans
are due in September 1997.
Although the Results Act's required consultation can help improve the
management of federal agencies, it will have a less direct role in
resolving difficult policy choices, because such choices should
appropriately consider many factors, including program performance.
For example, we have observed that transferring weapons production
from the Department of Energy to DOD, as was proposed by some, would
require careful consideration of many policy and management
issues.\15 In a survey of former Department of Energy executives and
experts on energy policy, some argued that because of the declining
strategic role of nuclear weapons, DOD might be better able to
balance resource allocations among nuclear and other types of weapons
if the weapons complex were completely under its control. Others
argued, however, that the need to maintain civilian control over
nuclear weapons outweighs any other advantages.
The Results Act can assist decisionmakers in making policy choices by
providing data on the management and performance implications of
different options. For example, data generated as part of the
Results Act can identify potential performance improvements that
might result from consolidating programs or shifting responsibilities
among agencies. Congress and the executive branch can then weigh
this information against policy considerations. Our work suggests
that ultimately a successful strategic planning effort will often
reflect hard choices by the agency, and there may be disagreement
between the agency and key stakeholders about particular goals or the
means by which the goals will be achieved. In this regard, an OMB
official has noted that a strategic plan that has achieved complete
agreement among all interested parties is likely to be at such a high
level of generality that the usefulness of the plan as a
decisionmaking tool would be fairly limited.
--------------------
\13 GAO/T-GGD-96-79, March 6, 1996.
\14 Managing For Results: Enhancing the Usefulness of GPRA
Consultations Between the Executive Branch and Congress
(GAO/T-GGD-97-56, Mar. 10, 1997).
\15 Department of Energy: Observations on the Future of the
Department (GAO/T-RCED-96-224, Sept. 4, 1996).
CONCLUSIONS
---------------------------------------------------------- Chapter 3:4
The Results Act, with its focus on results and requirement for
consultation, can help Congress and the executive branch address the
management implications of two of the seemingly intractable issues
confronting federal agencies: overlap and fragmentation in program
efforts that cut across agencies and the need to balance multiple
priorities within individual agencies. However, although the Results
Act provides a potentially effective vehicle for addressing these
issues, their existence also makes the prospects for the effective
governmentwide implementation of the Act uncertain in the near term.
Efforts under the Results Act can help identify the extent and
consequences of the overlap and fragmentation, but addressing such
problems will likely be a lengthy process, because Congress and the
executive branch will have to consider policy trade-offs to reach a
reasonable degree of consensus on the appropriate federal response to
national needs. Similarly, balancing multiple priorities will need
to be an ongoing process, because priorities change as new needs
arise.
The strategic planning consultation process is a starting point for
Congress and the executive branch to work through these different
policy issues. If successfully implemented, the Results Act will be
most helpful to decisionmakers in providing performance information
that identifies the management implications of different structures
and policy choices among competing demands.
DEVELOPING USEFUL RESULTS-ORIENTED
PERFORMANCE INFORMATION WILL BE AN
ONGOING DIFFICULTY
============================================================ Chapter 4
Agencies have encountered some difficult analytic and technical
challenges in developing useful results-oriented performance
information to use in planning their efforts and gauging progress
toward the achievement of their goals and missions. There are two
primary considerations for agencies in their development of such
information. The first is to identify those performance measures
that will be meaningful for the agency to use in gauging progress in
achieving its goals. The other is to collect reliable and valid data
on those measures so that program managers and other decisionmakers
have the data they need to effectively manage federal programs.
However, several factors--including a lack of data on
results-oriented performance measures; the influence of external
forces, such as emerging economic, social, and technological trends
and the role that third parties, such as state and local governments,
may have in determining whether program results are achieved; and the
long time frames sometimes needed before the results of agency or
program actions can be seen--have complicated agencies' ability to
set goals and identify results-oriented performance measures.
Agencies also are challenged in their collection of valid and
reliable data. Reasons for this challenge include different parties
using different definitions to measure performance and agencies'
reliance on outside parties to provide results-oriented performance
data. Some agencies are exploring approaches for addressing the
difficulty they have had in developing useful results-oriented
performance information.
SEVERAL FACTORS HAVE
COMPLICATED AGENCIES' ABILITY
TO SET GOALS AND IDENTIFY
RESULTS-ORIENTED PERFORMANCE
MEASURES
---------------------------------------------------------- Chapter 4:1
Several factors have complicated the degree to which agencies have
faced challenges in setting goals and identifying results-oriented
performance measures. These factors include a lack of data on
results-oriented performance measures; the influence of external
forces, such as emerging economic, social, and technological trends
and the role that third parties, such as state and local governments,
may have in determining whether program results are achieved; and the
long time frames sometimes needed before the results of agency or
program actions can be seen.
AGENCIES' LACK OF DATA
COMPLICATES EFFORTS TO SET
GOALS
-------------------------------------------------------- Chapter 4:1.1
Our work has shown that baseline and trend data on past performance
can help agencies set realistic goals for their programs given the
past performance of those programs. We have previously reported that
prior efforts to link resources to results were hampered by the
absence of systems to collect needed data.\1 Under PPBS, agencies
that attempted to gather results-oriented performance data found the
process to be far more difficult than expected. Agency officials
reported that developing the information and collection systems
envisioned by PPBS would take several years.
Because agencies often did not focus on having results-oriented
performance information in the past, they generally have not
collected such data. Therefore, they do not have all of the baseline
and trend data needed to set goals. For example, our reviews of the
National Park Service--whose mission is, in part, to preserve and
protect park resources--showed that baseline information about
natural and cultural resources was frequently incomplete or
nonexistent.\2 Consequently, it was difficult for Park Service
officials to determine whether the best management decisions about
those resources were being made. For example, for 70 years, the
Service has stocked nonnative fish in various lakes and waterways in
Yosemite National Park. However, officials said they knew little
about the types or numbers of fish in the lakes and waterways, as
well as other species inhabiting the park, including birds, badgers,
river otters, wolverines, and red foxes. By not collecting data on
those resources, officials realized that some of their decisions
about those resources were not effective in helping the Park Service
achieve intended results. At the time of our review, nonnative fish
outnumbered native rainbow trout by a 4-to-1 margin, and the stocking
had reduced the numbers of at least one federally protected species.
VA officials said that some of the results-oriented measures for
their agency's Loan Guaranty program were new and baseline data were
not available on those measures. Consequently, VA did not have data
on past performance to use in setting some of the program's fiscal
year 1998 goals. In some of these cases, VA indicated in its fiscal
year 1998 budget submission to OMB that those goals were "to be
determined." For example, according to VA, a key result of the
program is providing veterans with timely service in obtaining a VA
guaranteed loan. Performance measures include the percentage of
veterans satisfied with the time it took to process their loans and
the average time taken to process loans. Although VA set fiscal year
1998 goals for customer satisfaction with timeliness, VA reported
that goals for average processing time would need to be determined.
In another example, U.S. Department of Agriculture (USDA) officials
said they dropped some performance measures that had been part of
their Single Family Housing Results Act pilot's annual performance
plan because they did not have information systems to collect data on
those measures.\3 Such measures included those that would have
provided performance data on the number of loans made in targeted
geographic areas and the number of houses upgraded from substandard
to safe and sanitary in targeted areas. Lacking these data, USDA did
not have an informed basis on which to set goals.
In recent years, Congress has enacted landmark information technology
legislation--the Paperwork Reduction Act of 1995 and the
Clinger-Cohen Act of 1996--that, if successfully implemented, holds a
key to ensuring that agencies put in place the systems needed to
collect results-oriented performance data. These acts establish an
investment framework for having agencies better plan and manage their
technology efforts and link those efforts directly to the achievement
of agency program goals and mission.\4
--------------------
\1 GAO/AIMD-97-46, March 27, 1997.
\2 National Parks: Park Service Needs Better Information to Preserve
and Protect Resources (GAO/T-RCED-97-76, Feb. 27, 1997); and Park
Service: Managing for Results Could Strengthen Accountability
(GAO/RCED-97-125, Apr. 10, 1997).
\3 Information systems are a discrete set of information resources
and processes, automated or manual, organized for the collection,
processing, maintenance, use, sharing, or dissemination of
information. See GAO/AIMD-94-115, May 1994.
\4 For further information see: Information Technology Investment:
Agencies Can Improve Performance, Reduce Costs, and Minimize Risks
(GAO/AIMD-96-64, Sept. 30, 1996); Information Management Reform:
Effective Implementation Is Essential for Improving Federal
Performance (GAO/T-AIMD-96-132, July 17, 1996); and GAO/AIMD-94-115,
May 1994.
AGENCIES' RESULTS CAN BE
INFLUENCED BY EXTERNAL
FORCES
-------------------------------------------------------- Chapter 4:1.2
The efforts of federal agencies often are but one factor among many
external forces that may influence whether, and the degree to which,
their programs achieve their intended results. Many agencies have
been challenged to separate out the influence that program activities
have had on the achievement of program results when those results
also could have been influenced by external forces. This challenge
has complicated agencies' efforts to identify those performance
measures that will be meaningful for the agency to use in gauging
progress in achieving its goals.
Identifying meaningful performance measures has been a long-standing
problem for federal agencies. We have reported that agency officials
implementing the PPBS initiative, which was mandated governmentwide
by President Johnson in 1965, found it far more difficult than they
expected to relate program activities to a stated result or to
separate out other influences that might affect intended results.\5
For example, under the PPBS initiative, the Upward Bound program was
designed to increase skills and motivation for low-income high school
students. Agency officials had no way to isolate the program's
effect from other environmental influences that might also have
contributed to the success or failure of program participants.
The influence that external forces can have on the achievement of
program results has continued to challenge agencies' ability to
identify meaningful measures. Our work looking at the efforts of
economic development and our forthcoming report on regulatory
programs illustrates this challenge. For each of these types of
programs, isolating the federal contribution to the achievement of an
intended result has been exceedingly difficult and accordingly has
hindered agency efforts to identify meaningful performance measures.
In situations where the federal program effort is but one factor
among many external forces that may determine the degree to which an
intended result is achieved, determining the level of federal
influence on such achievement requires agencies to understand and
measure the nonfederal influence.
For example, in 1995 we reported on the existence of 342 economic
development-related programs.\6 Numerous external forces, including
broad national economic trends, contribute to local economic
development, and communities also may receive assistance from state
and local governments and the private sector. Therefore, determining
the impact of one of these programs on economic development has been
a daunting task. A persuasive study of a program's impact, as we
observed in our 1996 report,\7 would require three elements. First,
it would have to document that there had been some improvement in a
targeted area; second, it would have to link specific program
elements to actual economic changes; and third, it would have to
measure the growth stemming from other influences on the economy of
the targeted area in order to isolate the impact that could be
attributed to the economic development program.
Although such analysis entails a substantial undertaking, it can
potentially provide important information for decisionmakers. As
part of that 1996 review, we reported that one study of the
effectiveness of the Department of Commerce's Economic Development
Administration (EDA) programs found that income in the counties that
received EDA funding grew significantly faster than income in the
counties that received no aid. However, when the researchers
simultaneously considered EDA's programs and factors unrelated to
EDA, they found that EDA's programs had a very small effect on income
growth rates during the period that the aid was received and had no
significant effects in the 3 years after the aid ceased. EDA's
programs could explain only a small part of the difference in the
growth rates between the two groups of counties.
Regulatory agencies also have had difficulty sorting out the
interaction that external forces have had on the results that those
agencies were trying to achieve and accordingly have had difficulty
identifying meaningful performance measures. For example, OSHA
officials said they were having difficulty identifying measures to
assess program results because many other factors, such as business
cycles and the development of safer machines, affect the number of
workplace injuries each year more than OSHA's actions do. In
addition, a senior OSHA official said that worker safety data are
sometimes influenced by catastrophic events, such as bombings or
airplane crashes over which OSHA has no control.
Conversely, the less that program results are affected by external
forces, the greater will be the agency's influence in accomplishing
its intended results. Such agencies also will tend to have an easier
time identifying results-oriented performance measures that
demonstrate the extent to which the agency achieved its goals. For
example, the intended goals of VA's National Cemetery System (NCS)
include burying eligible veterans and their family members in
national cemeteries and maintaining the graves and their surroundings
as national shrines. Because NCS exerts a relatively high degree of
influence over program results, identifying results-oriented
performance measures is a fairly straightforward effort. Its fiscal
year 1996 results-oriented performance measures included the
percentage of veterans who chose a burial option provided by NCS
and/or who chose a VA headstone or marker. Because of NCS' high
degree of influence over program results, it also can more readily
pinpoint opportunities for managing its efforts and improving its
performance. To help it do so, NCS (1) conducted 14 focus groups at
5 locations throughout the country with specific customer groups who
had recent direct contact with the cemetery and (2) placed 20,000
visitor comment cards at 76 national cemeteries for visitors to
comment on their satisfaction with various aspects of cemetery
appearance and maintenance and what their priorities were.
Similarly, the Bureau of Engraving and Printing (BEP) primarily
produces currency and stamps in the volume and with the quality
required by two primary, immediate customers: the Federal Reserve
Board and the U.S. Postal Service. BEP receives no appropriations
from Congress and is self-financed by a revolving fund whereby the
Reserve Board and Postal Service reimburse BEP for its costs.
Because of its direct relationship with its customers, BEP has a
relatively high degree of influence over the achievement of its
intended results and was able to more readily identify
results-oriented performance measures. These measures, which BEP has
used for years, focus on customer service and include currency notes
delivered (in billions) and postage stamps delivered (in billions),
currency and postage stamp productivity measures, and customer
satisfaction measures.
--------------------
\5 GAO/AIMD-97-46, March 27, 1997.
\6 GAO/RCED-95-251R, July 28, 1995.
\7 Economic Development: Limited Information Exists on the Impact of
Assistance Provided by Three Agencies (GAO/RCED-96-103, Apr. 3,
1996).
LONG TIME FRAMES TO ACHIEVE
RESULTS HINDER ANNUAL
PERFORMANCE MEASUREMENT
-------------------------------------------------------- Chapter 4:1.3
Often it can take years before agencies see the results of their
programs' activities. This factor has made it difficult for agencies
to identify performance measures that will provide them with
information on the annual progress they are making toward achieving
program results. For example, OSHA officials said it was difficult
to track their agency's yearly progress in eliminating workplace
hazards when the impact of some regulatory actions could not be seen
for years. They said that the latency periods between exposure to a
hazardous substance, such as asbestos, and a resulting illness can be
20 years or more. Therefore, OSHA officials said, the results of any
actions OSHA takes to reduce exposures to those substances may be
equally long in coming. Officials at EPA also said it was difficult
to track the yearly progress of some of their agency's regulatory
actions because, for example, a significant lag usually existed
between the elimination of a chemical hazard and any corresponding
change in illness rates.
Our work on research and development (R&D) programs exemplified the
significance of the challenge agencies face in identifying meaningful
performance measures to use in assessing program results when
achievement of those results can take several years and the results
also can be affected by external forces.\8 Over $71 billion was
appropriated for federal R&D efforts in 1996. However, as we and
others have reported for almost 20 years, agencies with such programs
have encountered difficulty in identifying meaningful performance
measures, particularly quantitative measures, that can be used to
assess the results that the technological advancements of these
programs have on economic growth and the overall standard of living.
Because of the difficulties in identifying the impacts of research,
decisionmakers in the public and private sectors typically have
chosen to measure results using a variety of proxies. Three of the
most frequently cited quantitative measures are return on investment,
patents issued, and bibliometrics--i.e., the study of published data,
in particular, counts of citations. These measures imply a degree of
precision. However, they generally were not designed to measure the
long-term results of R&D programs and, therefore, are not easily
adaptable to such a purpose.
For example, return on investment is aimed at assessing the sales and
profits resulting from investments in R&D. However, long time
periods and the multiple research investments involved make the task
of calculating the return on basic research especially difficult.
Productivity growth may lag 20 years behind the first appearance of
research in the scientific community. A more serious impediment,
however, is the fact that results are often not directly traceable to
a specific research investment or may result from a combination of
such investments.
--------------------
\8 Measuring Performance: Strengths and Limitations of Research
Indicators (GAO/RCED-97-91, Mar. 21, 1997).
AGENCIES HAVE FACED CHALLENGES
COLLECTING RELIABLE AND VALID
RESULTS-ORIENTED PERFORMANCE
DATA
---------------------------------------------------------- Chapter 4:2
In passing the Results Act, Congress emphasized that the usefulness
of agencies' performance data depends, to a large degree, on the
reliability and validity of those data. However, the reliability and
validity of performance data are often questionable. From previous
reviews, we found that agencies have faced challenges in collecting
reliable and valid performance data. These challenges include
different parties using different definitions to measure performance
and agencies' reliance on outside parties to provide results-oriented
performance data.
USE OF DIFFERENT DEFINITIONS
FOR MEASURING PERFORMANCE
-------------------------------------------------------- Chapter 4:2.1
The lack of standard definitions for performance measurement data can
significantly hinder agencies' ability to use such data in planning
and reporting. For example, we reported challenges that OCSE faced
because of a lack of comparable performance data across state and
local jurisdictions.\9 OCSE officials said that discrepancies
resulting from differences in the way the states and local
jurisdictions defined what constitutes a child support enforcement
case had contributed to the difficulty of uniformly measuring state
performance. To address these discrepancies, OCSE worked with state
and local authorities to develop standard data definitions for key
child support enforcement terms, including a definition for what
constitutes a child support enforcement case, and incorporated the
use of standardized definitions for measuring state performance.
Together, they developed measures to assess state performance in
establishing paternities, obtaining support orders, and collecting
child support payments, using the established definition of a child
support enforcement case.
We also found that three economic development agencies we
reviewed--EDA, the Appalachian Regional Commission, and the Tennessee
Valley Authority's nonpower programs--computed a key performance
measure differently.\10 Specifically, each of these three agencies
uses "performance ratios" to measure the extent to which other
federal, state, local, or private investment in communities is
attracted to an economic development project as a result of the
agency's investment. The ratio is a comparison of total dollars
invested in or planned for an economic development project--including
the funding from other federal programs--with the dollars contributed
by the agency itself. However, we found that each agency, in
computing its ratio, defined "total dollars" differently and
calculated the ratio for only a portion of its programs.
Inconsistent definitions for what will be measured among crosscutting
programs can hamper efforts to compare the relative effectiveness of
individual programs.
--------------------
\9 Child Support Enforcement: Reorienting Management Toward
Achieving Better Program Results (GAO/HEHS/GGD-97-14, Oct. 25,
1996).
\10 GAO/RCED-96-103, April 3, 1996.
RELIANCE ON OUTSIDE PARTIES
FOR RESULTS-ORIENTED
PERFORMANCE DATA
-------------------------------------------------------- Chapter 4:2.2
Relying on parties outside of agencies for performance data has
created challenges for agencies to ensure the accuracy of such data.
For example, the Department of Education had faced difficulty in
obtaining accurate data from adult education programs administered by
states.\11 We reported that states are required to submit to the
Department of Education annual statistical performance reports giving
basic program information, such as number of students served, student
progress over the years, and various types of student achievement.
However, Education and state officials acknowledged serious problems
with the quality of the data contained in those reports. These
problems were based in part on double-counting or under-counting of
students in adult education programs.
As noted earlier, OCSE has had difficulty in obtaining comparable
state-reported data. OCSE also has faced challenges in ensuring that
states provide accurate data. In 1994, we recommended that OCSE
implement additional oversight tools for ensuring that the data
states submit are accurate and comparable.\12 In a subsequent report,
we found that OCSE has placed greater emphasis on its reviews of
state reporting systems, which analyze the procedures and systems
states use to accumulate, record, and report data.\13
However, in its reviews of 20 state reporting systems, OCSE found
that most of those states did not have reliable systems for reporting
data accurately and that improvements would be needed as it moves to
results-oriented management. In 1996, the Personal Responsibility
and Work Opportunity Reconciliation Act required OCSE to review and
validate the data states report on progress toward achieving program
goals.
--------------------
\11 Adult Education: Measuring Program Results Has Been Challenging
(GAO/HEHS-95-153, Sept. 8, 1995).
\12 Child Support Enforcement: Families Could Benefit From Stronger
Enforcement Program (GAO/HEHS-95-24, Dec. 27, 1994).
\13 GAO/HEHS/GGD-97-14, October 25, 1996.
APPROACHES TO IMPROVE THE
USEFULNESS OF RESULTS-ORIENTED
PERFORMANCE INFORMATION ARE
BEGINNING TO EMERGE
---------------------------------------------------------- Chapter 4:3
Some agencies are exploring approaches that begin to address the
difficulty they are having in developing useful results-oriented
performance information. Five approaches we identified include: (1)
using impact evaluations; (2) using intermediate performance
measures; (3) using a range of measures; (4) providing decisionmakers
with information on the reliability and validity of performance data;
and (5) working with stakeholders to identify and reach consensus on
the most meaningful measures for the program, data sources for those
measures, and data collection strategies.
USING IMPACT EVALUATIONS
-------------------------------------------------------- Chapter 4:3.1
During our recent review of analytic challenges that agencies have
faced in measuring their performance, agency officials described
using a variety of approaches, including techniques that are employed
in program evaluations, specifically those evaluating program impact,
to address the challenge of isolating the impact of a program on its
intended results.\14 The Results Act defines program evaluation as
"an assessment, through objective measurement and systematic
analysis, of the manner and extent to which federal programs achieve
intended objectives." Such assessments are to provide evaluative
information about whether, and in what important respects, a program
is working well or poorly and may address questions of program
implementation and impact. Notably, the impact evaluations described
by the agency officials were often employed at state or local levels
where the influence of other variables was either reduced or easier
to observe and control for. For example, because they were well
aware that the economy has a strong effect on a loan program's
performance, officials from one such program monitored changes in the
economy very closely at the regional level. By breaking down the
data to follow one regional economy at a time, these officials were
able to determine whether an increase in loan defaults in a given
region reflected a faltering economy or indicated some problem in the
program that needed to be followed up.
Although the Results Act does not require agencies to conduct impact
evaluations, it does require them to measure progress toward their
goals, identify how external forces might affect such progress, and
explain why a goal was not met. We previously reported that agencies
often need information on a program's impact on its results relative
to the impact of external factors to confidently attribute the
achievement of intended results to the program.\15 Further,
congressional and other decisionmakers will likely request such
information to help them make informed decisions.
We also have reported on how findings of program evaluations can be a
potentially critical source of information on the reasons an agency's
goals were not met and actions that could be taken to improve
performance.\16 However, we have reported that because of their
complexity, such evaluation studies may be costly, and many agencies
may not have staff with the skills to conduct program evaluations.\17
--------------------
\14 Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997).
\15 Program Evaluation: Improving the Flow of Information to the
Congress (GAO/PEMD-95-1, Jan. 30, 1995).
\16 GAO/GGD-96-66R, February 14, 1996.
\17 Program Evaluation Issues (GAO/OCG-93-6TR, Dec. 1992).
USING INTERMEDIATE MEASURES
-------------------------------------------------------- Chapter 4:3.2
As noted earlier, an agency's ability to identify meaningful
performance measures to use in planning its efforts and gauging
progress--especially annual progress--toward achieving its goals is
sometimes hindered because the intended results of the agency's
programs can be affected by various factors or require many years to
come to fruition. Our work has found that in such cases, agencies
may be able to use intermediate performance measures that provide
information on interim results to gauge their progress in meeting
their goals.\18
EPA's Office of Water has identified intermediate results, with
accompanying measures, for local estuary protection programs to use
in assessing their progress in meeting intended results. According
to EPA's Coastal Management Branch Chief, those programs are to
develop partnerships among all government agencies that oversee
estuary resources and the people who depend on those resources for
their livelihood and quality of life. The estuary programs generally
try to affect the behavior of government agencies, businesses,
households, and boaters toward achieving improvements in the quality
of estuaries and the health of organisms living in those estuaries.
According to EPA, because it may take years to see such improvements,
intermediate measures can provide information on the performance of
those programs on a more timely basis and are, therefore, important
for managing estuary protection efforts. EPA has developed
procedures for managers of local estuary programs to monitor and
report on program results and the progress in improving and
maintaining the quality of their estuary waters, according to the
official. Further, to the extent that many estuary programs use
similar procedures for assessing their progress, such information
also can provide a national perspective on progress in estuary
protection. EPA's Coastal Management Branch Chief said that most of
the 28 estuary programs have adopted or are considering this
approach.
EPA identified three sequential intermediate results--which are
referred to as first-, second-, and third-order results--that are
expected to affect changes in bay quality and the health of organisms
living in estuaries--which are referred to as the fourth-order, or
end, results of the program. Measures for first-order intermediate
results provide information on the extent to which desirable estuary
protection actions have been adequately implemented by governments,
businesses, or households. Measures for second-order intermediate
results provide information on the extent to which those actions have
led to reductions in pollutant discharges. Measures for third-order
intermediate results provide information on the extent to which water
or sediment quality has changed. Finally, measures for assessing the
fourth-order results of the program provide information on the extent
to which the health of fish, shellfish, other wildlife, habitat and
vegetation, and the region's economy has changed.
--------------------
\18 GAO/GGD-95-22, December 21, 1994.
USING A RANGE OF MEASURES
-------------------------------------------------------- Chapter 4:3.3
We found that in cases where agencies faced significant challenges
identifying results-oriented performance measures, they sometimes
used a range of measures to provide a more complete picture of agency
performance. For example, the Army Research Laboratory (ARL) found
that it could not predict the results or ultimate value of its
research. Therefore, it could not objectively and quantitatively
measure whether its research was achieving a specific intended
result. Instead, ARL devised a performance measurement approach that
is to make use of "three pillars": metrics, customer feedback, and
peer review. For example, its metrics included about 60 quantitative
measures that according to ARL provided useful information for
understanding the functional health of the organization and the
management of ARL. However, ARL cautioned that the resulting
performance information would not enable it to determine the real
quality and impact of its programs. Because of its lack of
results-oriented performance measures, ARL is relying more heavily on
customer surveys and peer review for information about the quality of
its efforts.
Our work on R&D programs found that although there are strengths to
using peer reviews, there are also limitations.\19 Peer review uses
technical experts to judge R&D results on the basis of the experts'
evaluation of the quality of research. We reported that although
peer review has been used extensively in the selection of proposed
research projects, it is subject to two serious shortcomings. First,
peer review is based on individuals' perceptions of quality, which
depend largely on the expertise of the selected experts. Thus, the
subjective nature of peer reviews makes performance results
vulnerable to bias. Frequently, a numerical rating scale, such as 1
for poor through 5 for excellent, is used to judge the quality of
research projects and the selection of proposed projects. However,
despite the appearance of precision conferred by a specific number,
the numbers represent the best of sometimes widely differing
judgments. Second, peer review can be expensive. For example, ARL
contracted for a peer review of its activities, which calls for
National Research Council technical assessment board reviews of
one-third to one-half of ARL's programs each year at a cost of
approximately $650,000 per year.
--------------------
\19 GAO/RCED-97-91, March 21, 1997.
PROVIDING DECISIONMAKERS
INFORMATION ON THE
RELIABILITY AND VALIDITY OF
PERFORMANCE DATA
-------------------------------------------------------- Chapter 4:3.4
As noted earlier, in passing the Results Act, Congress emphasized
that the usefulness of agencies' performance data depends, to a large
degree, on the reliability and validity of those data. Consequently,
the Act requires that agencies describe in their annual performance
plans the means to be used to verify and validate performance data.
We found that including such information in performance reports could
be equally important in providing assurance to report users of the
quality of the data.\20 The National Highway and Traffic Safety
Administration (NHTSA) included an appendix to its fiscal year 1994
Results Act pilot performance report that discussed the sources, and
in some cases the limitations, of the data it used to report on
performance. For example, NHTSA's accident data are based on police
reports, but various sources suggested that about half of the motor
vehicle crashes in the country were not reported to police. NHTSA
believed that the majority of these unreported crashes involved only
minor property damage and no significant injury. However, NHTSA's
report stated that it planned "to assess the unreported injury
problem." By incorporating a discussion of the limitations of its
accident data, NHTSA provided a context to assist the users of the
data in assessing NHTSA's performance.
--------------------
\20 GAO/GGD-96-66R, February 14, 1996.
WORKING WITH STAKEHOLDERS TO
REACH CONSENSUS ON MEASURES
AND DATA COLLECTION
-------------------------------------------------------- Chapter 4:3.5
Stakeholders can help agencies identify results-oriented performance
measures and the ways to collect needed performance data. For
example, in 1994, we reported that OCSE lacked essential management
tools to improve its responsiveness to the child support needs of
children and families and recommended actions OCSE could take in this
regard.\21 Since that time, OCSE has made progress in reorienting its
management of the program toward achieving intended program results
through greater stakeholder involvement and other means.\22 OCSE's
effort to involve states and other stakeholders took time and
concerted effort.
One of the first steps OCSE took was specifying the goals that it
expected states to achieve in such areas as paternities established
and collections received. However, state program officials strongly
objected to this mandate, because they did not have an opportunity to
participate in the planning process. OCSE then sought to obtain
wider participation from program officials at the federal, state, and
local levels of government through a joint planning process. During
the planning process, participants agreed that intended national
results would be based on the collective suggestions of the states
and consensus among participants. OCSE also established a
Performance Measures Work Group to identify statistical measures for
assessing state progress toward achieving national results. The work
group, which consisted of officials from the Department of Health and
Human Services' (HHS) Administration for Children and Families, OCSE,
and state and local child support enforcement programs, met several
times in 1995 and 1996 to discuss mutually acceptable performance
measures.
Officials in VA's Loan Guaranty program also discussed the importance
of involving stakeholders in identifying meaningful measures.
According to those officials, they had difficulty linking field
office activities to the intended results of the program as
established by headquarters staff. To address this, VA used input
from OMB and a performance measures work group that included both
headquarters and field office staff to reach consensus on the most
meaningful performance measures for the program and how data would be
collected on those measures. Also, key headquarters and field
managers were brought together to reach agreement on the goals to be
included in the program's fiscal year 1997 business plan.
Perhaps most important, congressional consultations on agencies'
strategic plans also can assist agencies in identifying the most
meaningful measures for the agencies' programs. A VA official
reported that most of their discussions with congressional staff have
included useful exchanges about the most meaningful performance
measures for assessing the results of VA's benefit programs. In
several cases, congressional staff proposed additional performance
measures for VA programs. For example, the staff suggested that VA's
Vocational Rehabilitation and Counseling program add measures to
assess the socioeconomic impact of the program on veterans, such as
the extent to which veterans have to rely on other government
assistance programs.
--------------------
\21 GAO/HEHS-95-24, December 27, 1994.
\22 GAO/HEHS/GGD-97-14, October 25, 1996.
CONCLUSIONS
---------------------------------------------------------- Chapter 4:4
The challenges confronting agencies as they seek to develop useful
results-oriented performance information are substantial,
long-standing, and will not be quickly or easily resolved. In cases
where results-oriented baseline and trend data on past performance do
not exist, agencies' will have difficulty setting realistic goals.
Many agencies also are grappling with how best to assess their
intended results in an environment where federal program efforts are
but one factor among many external forces that may influence whether
program results are achieved. Identifying results-oriented
performance measures will often require an agency to separate out the
influence on the achievement of its intended results from those of
forces external to it in order to isolate the agency's contribution
to the achievement of those results--an analytically daunting
challenge. Moreover, identifying such measures also is challenged by
the long lead times that may occur between a federal agency's actions
and the achievement of intended results.
Finally, once agencies have identified results-oriented performance
measures, collecting reliable and valid data on those measures poses
challenges for agencies. Reasons for these challenges include
different parties using different definitions to measure performance
and agencies relying on outside parties to provide results-oriented
performance data. Some agencies have begun exploring approaches to
address the difficulty in developing useful results-oriented
performance information. However, the nature of the challenges
agencies need to address and the time and effort needed to overcome
those challenges suggest that agencies will be hard-pressed to
consistently identify results-oriented measures in time for the first
annual performance plans to be submitted to OMB this fall. Those
agencies with the most direct influence on their results generally
will make the most progress in identifying such measures, while those
agencies with less influence will continue to struggle to identify
the most meaningful measures for informing decisionmakers in Congress
and the executive branch.
BUILDING AN ORGANIZATIONAL CULTURE
TO HELP CREATE AND SUSTAIN A FOCUS
ON RESULTS REMAINS A WORK IN
PROGRESS
============================================================ Chapter 5
Successful implementation of the Results Act in federal agencies will
depend on the degree to which those agencies create organizational
cultures that foster and maintain a focus on results at all levels.
The committed support of top agency leadership is critical to
building a results-oriented culture. Yet, according to the opinions
of the federal managers who responded to our survey, top agency
leadership is not fully committed to results-oriented management. In
addition, successful implementation of the Results Act will depend,
in part, on managers having the knowledge, skills, and abilities to
set goals, measure performance, and use performance information to
improve effectiveness. However, managers' responses to our survey
indicated that federal managers needed additional training on
critical results-oriented management-related tasks. A
results-oriented organization also strives to ensure that its
managers at all levels have the authority needed to accomplish goals
and provides those managers with the incentives and opportunities for
doing so. However, federal managers' responses to our survey also
suggest that more progress is needed in these areas as well.
TOP LEADERSHIP HAS NOT FULLY
DEMONSTRATED COMMITMENT TO
RESULTS-ORIENTED MANAGEMENT
---------------------------------------------------------- Chapter 5:1
For the Results Act to become part of an agency's culture, the agency
needs the sustained, demonstrated commitment of its top leadership to
initiate and insist on the use of results-oriented management
practices, to keep the agency focused on results, and to embed
related principles in the agency's approach to doing business.\1 By
showing managers and staff its commitment to achieving an agency's
goals, top agency leadership can encourage a focus on results. When
asked about the extent to which their agencies' top leadership
demonstrated a strong commitment to achieving results, federal
managers in our survey reported a significantly greater extent of
current top leadership commitment than they recalled existing 3 years
ago. According to our survey, about 44 percent of managers reported
that their agencies' top leadership demonstrated a strong commitment
to achieving results to a great or very great extent 3 years ago, and
57 percent of managers reported such commitment to a great or very
great extent currently, which is a 13 percentage point difference.
The percentage of federal managers reporting that top leadership
demonstrated strong commitment to a great or very great extent both
currently and 3 years ago was significantly higher for SES managers
than for non-SES managers. This suggests that the perception of top
leadership commitment to results is stronger for federal managers at
higher levels and that this perception has not reached managers at
all agency levels to the same extent. (See table 5.1.)
Table 5.1
Percent of Federal Managers Who Reported
That Their Agencies' Top Leadership
Demonstrated a Strong Commitment to
Achieving Results
Level of managers 3 years ago Currently
-------------------------------------- -------------- --------------
SES 60% 76%
Non-SES 42 56
----------------------------------------------------------------------
Source: GAO survey data.
Without top leadership making its commitment to results-oriented
management clear, the Results Act risks the danger that all
management reforms face: becoming a hollow, paper-driven exercise.
As we have said in our Executive Guide, leaders who integrate
results-oriented management into the culture and day-to-day
activities of their organizations will help avoid that danger.\2
One of the most meaningful demonstrations of leaders' commitment to
results-oriented management is the use of performance information to
make decisions. However, federal managers' perceptions about the
extent to which such use is occurring are not as strong as their
perceptions about top leadership's demonstrated commitment in
general. This suggests that despite the strength of managers'
perceptions about top leadership commitment, the extent to which
leaders are demonstrating their commitment by using performance
information for decisionmaking about their agencies' day-to-day
activities is not as great.
Compared to the 57 percent of managers who reported strong top
leadership commitment to a great or very great extent, only about 16
percent reported that changes by management above their levels to the
programs for which they were responsible were based on results or
outcome-oriented performance information to a great or very great
extent. When we compared managers who reported a great or very great
extent of strong top leadership commitment to managers who rated
leadership commitment from moderate to none, we found that 25 percent
of the managers who rated leadership commitment as great or very
great also reported that managers above their level made program
changes on the basis of performance information to a great or very
great extent. Only 5 percent of managers who rated the extent of
leadership commitment from moderate to none rated the extent of
managerial program changes based on performance information as great
or very great. Although the difference between these two groups is
statistically significant, in either case, only a minority of
managers perceived program changes based on performance information
occurring to a great or very great extent even when top leadership
commitment was perceived to be high.
Although still a minority, more federal managers said that the
individuals to whom they reported periodically reviewed with them the
results or outcomes of the programs for which they were responsible
than said that program changes were made based on performance
information. Both of these activities are indicators of the use of
performance information for management decisionmaking. About 42
percent of managers said that the individuals to whom they reported
periodically reviewed with them the results or outcomes of the
programs for which they were responsible to a great or very great
extent. As before, a comparison of managers who reported a great or
very great extent of strong top leadership commitment to managers who
rated leadership commitment from moderate to none showed a
significant difference. We found that 57 percent of the managers who
rated leadership commitment as great or very great also rated the
extent of periodic reviews of their programs' results or outcomes as
great or very great. Only 21 percent of managers who rated the
extent of leadership commitment from moderate to none rated the
extent of periodic reviews of their programs' results or outcomes as
great or very great.
Managers' responses regarding the extent to which changes to programs
were based on results and the extent to which individuals to whom
they reported reviewed program results with them indicate that as the
perception of strong top leadership commitment increased, the
perception of the use of performance information for decisionmaking
also increased. In addition, as their responses indicate, many
managers did not perceive that performance information was being used
to make such decisions to a great or very great extent.
--------------------
\1 GAO/GGD-96-118, June 1996.
\2 GAO/GGD-96-118, June 1996.
MANAGERS REPORTED NEEDING
ADDITIONAL TRAINING
---------------------------------------------------------- Chapter 5:2
In our work on identifying techniques that are needed to change an
organization's culture, we found that training was one of the most
important techniques used.\3 From our symposium on transforming the
civil service, we discerned that treating continuous learning as an
investment in success rather than as a cost to be minimized was a key
principle for managing people.\4 Investing in training and
professional development could help federal agencies meet changing
customer needs, keep skills up to date, and develop new personal and
organizational competencies. In work we did to identify insights for
Results Act implementation from past initiatives on performance
budgeting, we spoke with current and former executive branch
officials and legislative branch staff as well as individuals with
expertise in the Results Act, budgeting, and public administration.\5
These experts said that participants in implementing the Act needed
to acquire the skills to develop and use performance information.
One of the critical practices for implementing the Results Act that
we identified in our Executive Guide was the need to build expertise
for staff at all levels of the organization so that they are skilled
in strategic planning, performance measurement, and the use of
performance information in decisionmaking. \6
When asked whether, during the past 3 years, their agencies had
provided, arranged, or paid for training that would help them
accomplish several critical results-oriented management-related
tasks, the percentage of SES managers who responded "yes" was
significantly greater than that of non-SES managers. According to
our survey, 43 to 61 percent of SES managers and 17 to 38 percent of
non-SES managers responded yes to questions on training that would
help them accomplish each of several critical results-oriented
management-related tasks. The task for which the highest percentage
in general responded yes was for training on conducting strategic
planning. For this task, 61 percent of SES managers and 38 percent
of non-SES managers responded yes. However, for the remaining five
of the six tasks, fewer than 38 percent of non-SES managers responded
yes. Only 46 percent of SES managers and 17 percent of non-SES
managers responded yes regarding training on implementing the
requirements of GPRA. The results for training on each task for SES
and non-SES managers appear in table 5.2.
Table 5.2
Percent of Federal Managers Who Reported
That During the Past 3 Years Their
Agency Provided, Arranged, or Paid for
Training That Would Help Them to
Accomplish Specific Tasks
Tasks SES Non-SES
---------------------- ---------------------- ----------------------
Conduct strategic 61% 38%
planning
Set program 56 33
performance goals
Develop program 52 35
performance measures
Use program 43 30
performance
information to make
decisions
Link the performance 46 27
of program(s)/
operation(s)/
project(s) to the
achievement of agency
strategic goals
Implement the 46 17
requirements of GPRA
----------------------------------------------------------------------
Source: GAO survey data.
In addition, we asked federal managers to what extent they thought
they needed training or additional training to help them accomplish
each of these tasks. For each task, the percentage of non-SES
managers was significantly higher than the percentage of SES managers
reporting a need for training to a great or very great extent. Table
5.3 shows the results for training or additional training needed to a
great or very great extent by SES and non-SES managers for each task.
Table 5.3
Percent of Federal Managers Who Reported
Needing Training or Additional Training
to Accomplish Specific Tasks
Tasks SES Non-SES
---------------------- ---------------------- ----------------------
Conduct strategic 14% 28%
planning
Set program 15 32
performance goals
Develop program 20 40
performance measures
Use program 14 32
performance
information to make
decisions
Link the performance 17 33
of program(s)/
operation(s)/
project(s) to the
achievement of agency
strategic goals
Implement the 18 43
requirements of GPRA
----------------------------------------------------------------------
Source: GAO survey data.
Implementing the requirements of GPRA was the only task for which a
significant difference existed between the ratings of managers who
reported training that their agencies provided, arranged, or paid for
and managers who reported no such training. About 29 percent of
managers who reported the training in implementing the requirements
of GPRA also reported a great or very great need for additional
training for this task. However, more than half of managers who
reported no such training in implementing the requirements of the Act
reported a great or very great need for it. A need for training on
this task received the highest proportion of great and very great
responses from managers who reported no such training for any given
task.
Although responses to the need for training varied, many managers
believed that they needed training to a great or very great extent
for at least some of the tasks. For example, based on our survey, 43
percent of managers reported that they needed training to a great or
very great extent in at least two of the tasks listed.
Providing the skills to develop and use performance information may
well be a significant challenge to implementing the Results Act
because of declining resources and serious budgetary constraints.
Executive branch officials we spoke with as part of our review of
past initiatives on performance budgeting tended to think that the
Results Act was "the right thing to do." Thus, they believed that the
resources for needed training related to implementing the Act would
be found.\7 In an April 2, 1997, OPM-sponsored interactive broadcast
on the Results Act, OMB's Deputy Director for Management said he
recognized the difficult problem that federal agencies faced, given
the training and other capacity-building activities needed to measure
and evaluate program performance during this time of significant
budgetary constraint.\8 But he stressed that federal agencies do have
to devote some resources to these important activities to increase
the effectiveness of their programs, for which they use most of their
resources. Because of this environment, agencies will need to seek
new, creative, and less costly ways to build their capacities to
implement the Act in order to improve their programs. In our past
work, we found that one approach--which has been used at HHS and the
Department of the Interior--was to limit training costs by having
in-house coordinators lead training efforts and serve as mentors on
the Results Act.\9 Managers' responses to our survey questions on
training suggest that federal agencies' efforts to train managers on
results-oriented management-related tasks may best be focused on
non-SES managers. The responses also suggest that such training
especially include training on implementing the requirements of the
Results Act.
--------------------
\3 Organizational Culture: Techniques Companies Use To Perpetuate or
Change Beliefs and Values (GAO/NSIAD-92-105, Feb. 27, 1992).
\4 Transforming the Civil Service: Building the Workforce of the
Future--Results of a GAO-Sponsored Symposium (GAO/GGD-96-35, Dec.
20, 1995).
\5 GAO/AIMD-97-46, March 27, 1997.
\6 GAO/GGD-96-118, June 1996.
\7 GAO/AIMD-97-46, March 27, 1997.
\8 Office of Personnel Management Federal Human Resources Forum:
Interactive Broadcast on the Government Performance and Results Act
and How It Affects Every Federal Government Agency Manager and
Worker, April 2, 1997.
\9 Managing for Results: Status of the Government Performance and
Results Act (GAO/T-GGD-95-193, June 27, 1995).
MANAGERS REPORTED REDUCED
AUTHORITY TO ACHIEVE RESULTS
---------------------------------------------------------- Chapter 5:3
In crafting the Results Act, Congress recognized that if federal
managers were to be held accountable for program results, they would
need the authority and flexibility to achieve those results.
Congress also understood the importance of affording federal program
managers the freedom to be innovative and creative and to marshal
resources to achieve results. Congress has provided agencies with
additional authority in the key area of procurement that allows
agencies more flexibility in managing their programs. In addition,
one of the major initiatives of NPR has been the empowerment of
federal employees to get results.
Overall, when asked about the extent to which managers or supervisors
at their levels had the authority they needed to help their agencies
accomplish their strategic goals, federal managers did not perceive
that they had a greater extent of such authority than they recalled
having 3 years ago. Interestingly, for managers from selected
Results Act pilots, the perception of the extent of this authority
currently was significantly lower than the perception for 3 years
ago. In contrast, for managers who were not in selected pilots, the
perception of the extent of authority currently was essentially
unchanged from the perception for both 3 years ago. In addition, for
both 3 years ago and currently, the percentage of SES managers
reporting a great or very great extent of authority was significantly
higher than the percentage of non-SES managers. (See table 5.4.)
Table 5.4
Percent of Federal Managers Reporting
That Agency Managers/Supervisors at
Their Levels Had the Decisionmaking
Authority They Needed to Help the Agency
Accomplish Its Strategic Goals
3 years
Level of managers ago Currently Difference
------------------------------------ -------- ---------- ----------
SES
----------------------------------------------------------------------
Selected Results Act pilots 56% 40% -16%
All other agencies 53 52 -1
Non-SES
----------------------------------------------------------------------
Selected Results Act pilots 36 24 -12
All other agencies 29 31 +2
----------------------------------------------------------------------
Source: GAO survey data.
When asked about the extent to which managers or supervisors at their
levels were held accountable for the results of the programs for
which they were responsible, no such differences between managers
from selected Results Act pilots and managers from all other agencies
or between SES and non-SES managers were observed. However, more
managers overall rated the extent of accountability for results as
higher currently than for 3 years ago. For example, about 55 percent
of managers reported that managers or supervisors at their levels
were currently being held accountable to a great or very great extent
for the results of the programs for which they were responsible.
About 48 percent of managers reported having this perception for 3
years ago.
LESS THAN HALF OF FEDERAL
MANAGERS REPORTED POSITIVE
RECOGNITION FOR ACHIEVING
RESULTS
---------------------------------------------------------- Chapter 5:4
In crafting the Results Act, Congress recognized the need to create
incentives for managers to use results-oriented performance
information. The commitment demonstrated by Congress and top
leadership when they use results-oriented performance information to
make decisions is one of the greatest incentives for changing
agencies' cultures and managers' behavior to focus on achieving
intended results. In addition, agencies have traditionally rewarded
their employees formally through pay increases and other monetary and
nonmonetary awards for performance that helped agencies achieve their
goals. However, the current environment of constrained resources
challenges agency leaders to think creatively about informal
incentives, which need not be costly, such as positive recognition,
that they can use to encourage results-oriented management.
When asked about the extent to which employees in their agencies
currently received positive recognition and received such recognition
3 years ago for helping the agencies accomplish their strategic
goals, federal managers reported essentially no difference. However,
statistically significant differences existed between the perceptions
of SES and non-SES managers, both currently and for 3 years ago.
According to our survey, 48 percent of SES managers and 25 percent of
non-SES managers reported that, to a great or very great extent,
employees in their agencies currently received positive recognition
for helping the agencies accomplish their strategic goals. When
asked to recollect the situation 3 years ago, 43 percent of SES
managers and 27 percent of non-SES managers reported such
recognition.
In work we did on state governments' experiences in using
results-oriented management, we found that in Oregon and Minnesota,
managers and staff in some state agencies were reluctant to commit to
achieving results that they did not totally control because of
concerns that performance information would be used against them.\10
In our survey of federal managers, we listed 16 factors and asked
managers to rate each factor on the extent to which it hindered
measuring performance or using performance information for the
programs with which they were involved. One of these factors was
concern that performance information could be used against their
programs or agencies. When asked about this factor, only about 9
percent of managers reported it as a hindrance to a great or very
great extent--least often of the 16 factors included in the survey.
--------------------
\10 GAO/GGD-95-22, December 21, 1994.
INVOLVEMENT OF FEDERAL MANAGERS
IN RESULTS ACT-RELATED
ACTIVITIES IS VARIED
---------------------------------------------------------- Chapter 5:5
Congress expected that under the Results Act, managers throughout
federal agencies would have active roles in making their
organizations more results oriented. One of the stated purposes of
the Act was to help federal managers improve service delivery by
requiring that they plan for meeting program objectives and by
providing them with information about program results and service
quality. More recently, during OPM's April 1997 interactive
broadcast on the Results Act, the OMB Deputy Director for Management
underscored OMB's commitment to the effective implementation of the
Results Act.\11 He told federal managers that if they had not been
involved in strategic planning, they should be asking their agencies
why they had not been involved and saying that OMB wanted them to be
involved.
However, when we asked federal managers whether, during the past 3
years, they had been involved in a variety of GPRA-related activities
that incorporated a number of key practices needed to implement the
Act, they reported varied levels of involvement, depending on the
activity. Significant differences were especially evident for each
activity between SES managers' and non-SES managers' reported levels
of involvement. For example, according to our survey, 72 percent of
SES managers reported involvement in establishing long-term strategic
goals for their agencies--which is a key step in strategic
planning--while only 35 percent of non-SES managers reported
involvement in this activity. Table 5.5 shows the percentages of
both SES and non-SES managers as well as the overall percentages of
federal managers who reported that they had been involved in these
activities during the past 3 years.
Table 5.5
Percent of Federal Managers Reporting
That They Had Been Involved in GPRA-
Related Activities During the Past 3
Years
Activities SES Non-SES Overall
---------------- ---------------- ---------------- ----------------
Establishing 72% 35% 38%
long-term
strategic goals
for the agency
Reconciling 34 8 10
differing
congressional
views on the
agency's
mission and
strategic goals
Reconciling the 53 29 31
views of other
interested
parties on the
agency's
mission and
strategic goals
Developing ways 72 47 49
to measure
whether program
performance
goals are being
achieved
Gathering and 65 54 55
analyzing data
to measure
whether
programs are
meeting their
specific
performance
goals
Using measures 55 34 35
for program
performance
goals to
determine if
the agency's
strategic goals
are being
achieved
Publicly 44 18 20
reporting on
how well the
agency's
programs are
meeting their
specific
performance
goals
----------------------------------------------------------------------
Source: GAO survey data.
Of activities that showed significantly lower percentages of non-SES
managers reporting involvement as compared to SES managers, some are
activities where such differences might be expected--for example,
reconciling differing congressional views on the agency's mission and
strategic goals and reconciling the views of other interested parties
on the agency's mission and strategic goals. However, such
significant differences between SES and non-SES managers' reported
involvement were also evident for activities where involvement of
managers at all levels could reasonably be expected--for example,
developing ways to measure whether program performance goals are
being achieved and gathering and analyzing data to measure whether
programs are meeting their specific performance goals. However,
gathering and analyzing data to measure whether programs are meeting
their specific performance goals was the only one of these
GPRA-related activities for which at least half of managers overall,
including both SES and non-SES managers, reported being involved.
--------------------
\11 Office of Personnel Management Federal Human Resources Forum:
Interactive Broadcast on the Government Performance and Results Act
and How It Affects Every Federal Government Agency Manager and
Worker, April 2, 1997.
FEDERAL MANAGERS' VIEWS ABOUT
GPRA'S PAST AND FUTURE EFFECTS
ON AGENCY PROGRAMS ARE MIXED
---------------------------------------------------------- Chapter 5:6
When we asked federal managers about the extent to which their
agencies' efforts to implement GPRA to date had improved their
agencies' programs, 57 percent overall responded that they had not
been sufficiently involved in the Act in their agencies to have an
opinion, comprising 22 percent of SES managers and 60 percent of
non-SES managers. Of the 41 percent of managers who did express an
opinion on the extent to which their agencies' efforts to implement
GPRA to date had improved their agencies' programs, 42 percent
responded that the Act had improved programs to a moderate or greater
extent. Figure 5.1 shows the percentages of the federal managers who
did express an opinion for each extent level.
Figure 5.1: Distribution of
Federal Managers Expressing an
Opinion on the Extent to Which
Implementing GPRA to Date Had
Improved Their Agencies'
Programs
(See figure in printed
edition.)
Source: GAO survey data.
Governmentwide implementation of the Results Act is just beginning
for many federal agencies and programs and, as our earlier work has
shown, changing to a results-oriented culture can take many years.\12
Thus, the 57 percent of managers overall reporting that they had not
been sufficiently involved in GPRA to have an opinion on the extent
to which efforts to implement the Act to date had improved their
agencies' programs and the much greater proportion of non-SES
managers (60 percent) than SES managers (22 percent) reporting that
they had not been sufficiently involved to have an opinion are not
surprising. In addition, given the long-term challenges that a move
toward a results-oriented culture involves, it is reasonable to
expect that many federal managers are not yet seeing their agencies'
results improving as an effect of the Results Act.
When asked about the extent to which implementing GPRA could improve
their agencies' programs in the future, federal managers' opinions
were more optimistic. About 36 percent of managers overall did not
express an opinion, responding that they had no basis to judge. Of
the 62 percent of managers who did express an opinion, about 71
percent responded that implementing the Act could improve programs in
the future to a moderate or greater extent. Figure 5.2 shows the
percentages of the federal managers who did express an opinion for
each extent level.
Figure 5.2: Distribution of
Federal Managers Expressing an
Opinion on the Extent to Which
Implementing GPRA Could Improve
Their Agencies' Programs in the
Future
(See figure in printed
edition.)
Source: GAO survey data.
Still, many federal managers remained unconvinced or uncertain about
whether the Results Act could help their agencies' programs even
moderately.
--------------------
\12 GAO/T-GGD-95-193, June 27, 1995.
CONCLUSIONS
---------------------------------------------------------- Chapter 5:7
According to the opinions of federal managers, building the
organizational cultures necessary to create and sustain a
results-orientation in the federal government appears to be a work in
progress. In some ways, changes toward a culture that will allow a
focus on results appear to be occurring. For example, the
perceptions of both SES and non-SES managers about the extent to
which top leadership is demonstrating a strong commitment to
achieving results were higher currently than they were for 3 years
ago. However, we would expect to see federal managers' experiences
with and positive perceptions about results-oriented management
practices become more prevalent if the Results Act is to succeed.
Over the next several years, we would expect to see more federal
managers reporting positively on indicators, such as top management's
use of results to make decisions, agencies providing training on key
results-oriented management-related tasks, and managers' involvement
in GPRA-related activities. But just as importantly, we would expect
to see the gap between SES and non-SES managers regarding positive
experiences and perceptions about results-oriented management begin
to narrow if the Results Act is to work as intended. Agencies can
especially concentrate their efforts on areas where managers are not
perceiving or experiencing progress, such as on devolving
decisionmaking authority to managers throughout the agencies.
When both mid-level and upper level managers support and are involved
in the changes that the Results Act entails, the probability of
sustaining those changes will increase substantially. Providing
federal managers at all levels with greater authority and greater
opportunities to participate in results-oriented activities can give
them the experiences needed to increase their confidence in the
benefits of managing for results. When such indicators are more
widespread and pervasive among federal managers, agencies' commitment
to achieving results and their ability to do so should be enhanced
governmentwide.
EFFORTS TO INTEGRATE PERFORMANCE
INFORMATION WITH BUDGET DECISIONS
AND TO DEVELOP A NEW REPORTING
FRAMEWORK ARE PROMISING
============================================================ Chapter 6
Addressing some of the challenges discussed earlier in this report,
such as crosscutting program efforts and balances among competing
priorities, raises significant policy issues for Congress and the
administration to consider, and some issues will likely be very
difficult to resolve. Our work has indicated that the program
performance information that is to be generated under the Results Act
will become most useful to congressional and executive branch
decisionmakers when that information is systematically linked with
resource allocation decisions and reported within a framework that
allows for fuller assessments of results, operational performance,
and costs.
Although budgeting is inherently an exercise in political choice, the
Results Act is based on the premise that budget decisions should take
into consideration an agency's expectations about program
performance. Therefore, the Results Act's success or failure should
not be judged on whether contentious budget and other policy issues
are fully resolved; rather, the success or failure of the Results Act
will likely turn on the extent to which the information produced
through the Act's goal-setting and performance measurement practices,
once those practices are successfully implemented, helps Congress and
the executive branch make informed policy decisions and improve
program management.
Agencies are to provide the first annual performance plans, covering
fiscal year 1999, to OMB in the fall of 1997 and to Congress after
the President's budget is submitted in 1998. These plans are to
directly link executive branch performance planning with the budget
process. Agencies are to accomplish this link by basing their annual
program performance goals on their program activity structures as
listed in the President's budget. However, our recent report on
previous governmentwide initiatives to better align spending
decisions with expected performance--commonly referred to as
performance budgeting--found that the extent to which the budget's
program activity structures can be directly linked to a
results-oriented performance framework varies widely among program
activities. For Congress and the executive branch to reach agreement
on changes to these structures will be a time-consuming and difficult
process that will require more than one budget cycle to resolve.\1
Over the longer term, improved financial reporting and auditing as
required by the CFO Act is to strengthen the cost basis and
reliability of performance information. In this regard, reporting
concepts and standards developed by the Federal Accounting Standards
Advisory Board (FASAB) are intended to provide congressional and
other decisionmakers for the first time with annual "report cards" on
the costs, management, and effectiveness of federal agencies in order
to help strengthen federal decisionmaking and agencies'
accountability.\2 In addition, FASAB's new cost accounting standards
are intended to strengthen federal decisionmaking by requiring
agencies to provide relevant and reliable cost information that can
be used, for example, to relate the costs of various programs and
activities with their performance outputs and results.
--------------------
\1 GAO/AIMD-97-46, March 27, 1997.
\2 FASAB was created in October 1990 by the Secretary of the
Treasury, the Director of OMB, and the Comptroller General to
consider and recommend accounting principles for the federal
government.
THE RESULTS ACT SEEKS TO BETTER
ALIGN DESIRED RESULTS WITH
BUDGET DECISIONS
---------------------------------------------------------- Chapter 6:1
Congress intended for the Results Act's annual performance plans to
help Congress and the executive branch make informed decisions by
providing a simple, straightforward linkage among plans, budgets, and
performance results--a linkage that we have found did not exist in
the past. Our report on performance budgeting initiatives
demonstrated the importance of making this linkage if the Results
Act's goal of better aligning expected results with budget decisions
is to be met. We reported that past performance budgeting
initiatives, such as PPBS, which was initiated governmentwide in 1965
by President Johnson, generated unique performance information that
was often unconnected to the structures used in congressional budget
presentations. We found that consequently, congressional interest in
these initiatives quickly waned as plans and performance results
could not be directly linked to familiar oversight and budget
structures. In the end, we concluded that structural
incompatibilities meant that resource decisions were not linked to
the information generated by the past performance budgeting efforts.
The Results Act attempts to establish this linkage by requiring
agencies to base the goals in their annual performance plans on the
program activity structures used in their budget submissions and to
provide these performance plans to Congress each year after the
president's budget is submitted. Program activities are listings of
projects and activities contained in the president's budget and are
intended to provide a meaningful representation of agency operations.
The Results Act allows agencies to aggregate, disaggregate, or
consolidate program activities for performance planning purposes. We
noted in our performance budgeting report that program activity
structures are used to provide a relatively consistent framework for
OMB and congressional budget decisions. Congress also often uses the
program activity structures to form the basic unit of congressional
oversight for determining reprogramming thresholds.\3
We reported that an agency's program activities generally result from
negotiations between the agency, OMB, and the relevant congressional
appropriations subcommittees. Program activity structures,
therefore, typically are tailored to meet the specific needs of an
agency, OMB, and Congress. Thus, program activity structures
represent programmatic, process, organizational, or other
orientations depending on the needs, interests, and experiences of
agencies, OMB, and Congress. Moreover, program activities differ
from agency to agency; and within agencies, they differ from budget
account to budget account. For example, the "space station" program
activity in the National Aeronautics and Space Administration's
budget is a discrete program. In contrast, the "park management"
program activity under the National Park Service's budget represents
process and/or functional efforts.
Because of the wide variability across program activities, we noted
that the suitability of current program activity structures for the
Results Act's performance planning and measurement also will vary.
Agency officials we spoke to as part of our review of previous
performance budgeting initiatives highlighted the varying suitability
of their program activity structures for the Results Act's purposes.
One agency successfully worked through the performance planning
process using its existing program activities. A second agency had
program activities that reflected its organizational units--a
structural approach that is useful for traditional accountability
purposes, such as monitoring outputs and staff levels, but less
useful for results-oriented planning. This agency therefore found it
necessary to devise a separate planning structure and then crosswalk
performance goals back to the budget's program activities. Still
other agencies separated performance planning from program activity
structures, believing it necessary to first establish appropriate
program goals, objectives, and measures before considering the link
to the budget.
Our discussions with agency officials and congressional staff found
that they tended to view the need for and benefits of adjustments to
program activities from very different perspectives. Congressional
staff were generally comfortable with existing activity structures
and questioned whether changes would frustrate congressional
oversight. The staff generally viewed these structures as
fundamental to congressional oversight of agency activities; thus,
they viewed changes with apprehension and concern. However, some
agency officials saw program activity structures as secondary to
planning; thus, where current program activity structures proved
unsuitable for planning purposes, these officials viewed change in
program activity structures as inevitable and appropriate. However,
they noted that negotiating changes with Congress could prove
difficult and time-consuming.
--------------------
\3 Reprogramming is the shifting of funds within an appropriation to
purposes different from those contemplated at the time the
appropriation was requested and provided. Several appropriations
subcommittees use program activity structures to establish
reprogramming thresholds. If an agency needs to shift funds from one
activity to another above the threshold, it is expected to notify the
appropriate subcommittee.
INTEGRATED PERFORMANCE, COST,
AND MANAGEMENT REPORTS PROVIDE
BASIS FOR SUPPORTING KEY
DECISIONS
---------------------------------------------------------- Chapter 6:2
Our work has shown that congressional and executive branch
decisionmakers have been handicapped by the absence of reliable and
valid performance, program cost, and management information needed to
make well-informed decisions.\4 If successfully implemented, the
performance measurement and reporting requirements under the Results
Act will provide congressional and executive branch decisionmakers
with a wealth of information on the results of agencies' efforts. We
have testified that such information will be most useful to
congressional and executive branch decisionmakers if it is
consolidated with critical financial and program cost data in
financial statements, which agencies are to produce and have audited
under the CFO Act.\5 The CFO Act also sets expectations for agencies
to deploy modern systems to replace existing antiquated, often
manual, processes; develop better performance and cost measures; and
design results-oriented reports on the government's financial
condition and operating performance by integrating budget,
accounting, and program information.
The results-oriented reports will be most effective if they support a
wide range of decisions--including budget, policy, program
management, and accountability--that are routinely made by Congress
and the executive branch. The challenge of developing reports that
are specifically tailored to meet decisionmakers' needs and that are
based on appropriate accounting standards is being met by FASAB.
FASAB has put together a new set of accounting concepts and standards
based on a framework that financial reporting should demonstrate
accountability and provide information useful for planning,
controlling, and conducting the federal government's functions.
These concepts and standards underpin OMB's guidance to agencies on
the form and content of their agencywide financial statements.
The new accounting concept statements--the Objectives of Federal
Financial Reporting and Entity and Display--establish a new reporting
model for federal agencies that is geared to providing users with
information about budgetary integrity, operating performance,
stewardship, and systems and controls. Through this new reporting
model, Congress, agency decisionmakers, and the American public will
receive annual audited financial reports that are to
present a top-level overview discussion of the significance of the
financial information in accounting reports and commentary on
performance goals and results that are consistent with measures
an agency uses under the Results Act;
report, and thus make auditable, (1) uses of budgetary resources
that are made available by Congress, (2) obligations incurred,
(3) the balance of budgetary resources, and (4) outlays;
show total cost, earned revenues (if any), and net cost of agency
operations and programs in a manner that enables users to relate
costs to outputs and results;
report (1) unspent funds; (2) other operating assets available for
use in providing government goods, services, and benefits; and
(3) all liabilities incurred, with separate displays of those
that would require future funding by Congress; and
reflect the nation's assets and investments for which the federal
government has a stewardship responsibility.
As authorized by GMRA--which expanded the requirements of the CFO
Act--OMB is piloting accountability reports that are consistent with
the FASAB accountability reporting concept. OMB worked with six
agencies to pilot the development of accountability reports for
fiscal year 1995 and added an additional eight agencies to the pilot
test in fiscal year 1996. According to OMB, additional agencies will
produce accountability reports for fiscal year 1997. By seeking to
consolidate and integrate the separate reporting requirements of the
Results Act, the CFO Act, and other specified acts, the
accountability reports are to show the degree to which an agency met
its goals, at what cost, and whether the agency was well run.\6
Along with the new reporting model, FASAB developed cost accounting
standards that are to be effective beginning with fiscal year
1997--the first set of standards that are to account for the costs of
federal programs. These standards require agencies to develop
measures of the full costs of carrying out a mission or producing
products or services. Thus, decisionmakers are to have information
on the costs of all resources used, including the cost of services
provided by others to support activities or programs. Such
information would allow for comparisons of the costs of various
programs and activities with their performance outputs and results.
To help agencies implement the cost accounting standards and related
requirements set forth in the CFO Act and the Result Act, the Joint
Financial Management Improvement Program (JFMIP)--which develops the
requirements that agencies' financial management systems are to
meet--has issued an exposure draft, entitled "Managerial Cost
Accounting System Requirements."\7 This document is intended to
facilitate the acquisition and development of managerial cost
accounting systems by specifying the minimum information and
functional processing requirements agencies needed to accumulate and
assign cost data consistent with governmentwide guidance.
In addition, fiscal year 1997 will usher in the new requirements of
the Federal Financial Management Improvement Act of 1996, which will
help ensure greater attention to making much needed improvements in
underlying financial management systems. This act requires agencies'
financial management systems to comply with FASAB standards and JFMIP
system requirements.
Our work has shown that developing the necessary information systems
to gather, process, and analyze the needed program and cost
information will be a substantial undertaking for most federal
agencies because of agencies' persistent weaknesses in the use of
information technology.\8 To help address problems in agencies' use
of information technology, Congress enacted the Paperwork Reduction
Act of 1995 and the Clinger-Cohen Act of 1996. This information
technology reform legislation introduced requirements, in large part
based on GAO's research, that are intended to significantly improve
agencies' management, including how they select and manage
information technology resources. We have reported that together
with GPRA and the CFO Act, this reform legislation directs agencies
to implement a framework of modern technology management--one based
on practices followed by leading public sector and private sector
organizations that have successfully used technology to dramatically
improve performance and meet strategic goals.\9
--------------------
\4 See Information Resources: Summary of Federal Agencies'
Information Resources Management Programs (GAO/IMTEC-92-13FS, Feb.
13, 1992); Financial Management: CFO Act Is Achieving Meaningful
Progress (GAO/T-AIMD-94-149, June 21, 1994); Government Reform:
Using Reengineering and Technology to Improve Government Performance
(GAO/T-OCG-95-2, Feb. 2, 1995); Managing for Results: Critical
Actions for Measuring Performance (GAO/T-GGD/AIMD-95-187, June 20,
1995); and High-Risk Areas: Actions Need to Solve Pressing
Management Problems (GAO/T-AIMD/GGD-97-60, Mar. 5, 1997).
\5 Financial Management: Continued Momentum Essential to Achieve CFO
Act Goals (GAO/T-AIMD-96-10, Dec. 14, 1995).
\6 Managing For Results: Using GPRA to Assist Congressional and
Executive Branch Decisionmaking (GAO/T-GGD-97-43, Feb. 12, 1997).
\7 JFMIP is a joint initiative of OMB, GAO, the Department of the
Treasury and the Office of Personnel Management, which are working in
cooperation to improve financial management practices throughout the
federal government.
\8 See, most recently, High-Risk Series: Information Management and
Technology (GAO/HR-97-9, Feb. 1997).
\9 GAO/AIMD-94-115, May 1994 and Assessing Risks and Returns: A
Guide for Evaluating Federal Agencies' IT Investment Decision-making
(GAO/AIMD-10.1.13, Feb. 1997).
CONCLUSIONS
---------------------------------------------------------- Chapter 6:3
The Results Act's annual performance plans are to establish a direct
linkage between the goals in those plans and the congressional and
executive branch budget processes. This linkage between the Results
Act and budget decisions is key to Congress and the executive branch
ultimately being able to connect resources to results. Our work
reviewing the Results Act and previous performance budgeting efforts
suggests that due to the great variability among agencies' program
activity structures, Congress and the executive branch will likely be
challenged as they attempt to link performance goals with the
budget's program activity structures.
Therefore, the successful integration of performance goals developed
under the Results Act into budget decisionmaking may take several
annual budget cycles before Congress, OMB, and executive agencies are
able to make significant progress. We have noted that in the short
term, the challenges in reaching agreements on whether and how to
realign program activities may require agencies to develop crosswalks
between their performance goals and current program activity
structures. However, if long-term progress in this integration is
not achieved, the overall usefulness of the Results Act's performance
plans as a tool for improved congressional and executive branch
decisionmaking may be compromised.
Reports that present an agency's financial condition and the results
of its operations in an integrated way also hold promise for
enhancing the usefulness of performance information. We have noted
that such reports, which are independently audited, will help correct
the lack of complete and reliable information that has been a source
of concern for congressional and executive branch decisionmakers for
decades. Under FASAB's new cost accounting standards, agencies are
also required, for the first time, to provide decisionmakers with
measures of the full costs of carrying out a mission or providing a
service. If fully and effectively implemented, integrated
accountability reports can facilitate congressional and executive
branch decisionmaking by consolidating performance, cost, and
financial data in a single, user-friendly document.
(See figure in printed edition.)Appendix I
SURVEY ON RESULTS-ORIENTED
MANAGEMENT PRACTICES
============================================================ Chapter 6
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
MAJOR CONTRIBUTORS AND
ACKNOWLEDGEMENTS
========================================================== Appendix II
GENERAL GOVERNMENT DIVISION
-------------------------------------------------------- Appendix II:1
Michael Brostek, Associate Director, (202) 512-9039
J. Christopher Mihm, Acting Associate Director, (202) 512-3236
Joseph S. Wholey, Senior Advisor for Evaluation Methodology
Thomas M. Beall, Senior Social Science Analyst
Donna J. Byers, Senior Evaluator
William R. Chatlos, Senior Social Science Analyst
Joyce D. Corry, Senior Evaluator
Victoria M. O'Dea, Senior Evaluator
Kathleen M. Peyman, Senior Evaluator
Dorothy L. Self, Evaluator
Lisa R. Shames, Senior Evaluator
Kiki Theodoropoulos, Senior Evaluator
ACKNOWLEDGEMENTS
-------------------------------------------------------- Appendix II:2
In addition to those named above, the following individuals made
important contributions to this report: From the Accounting and
Information Management Division: Linda F. Baker, Senior Evaluator;
Michael J. Curro, Assistant Director; and Carolyn Yocum, Senior
Evaluator. From the General Government Division: James M. Fields,
Senior Social Science Analyst; Tracy J. Harris, Report Production
Assistant; Catherine M. Hurley, Computer Programmer Analyst; Stuart
M. Kaufman, Senior Social Science Analyst; Donna M. Leiss,
Communications Analyst; Susan Ragland, Senior Evaluator; Theodore H.
Saks, Senior Evaluator; Stephanie Shipman, Assistant Director;
Gregory H. Wilmoth, Senior Social Science-Analyst; and Ellen
Wineholt, Senior Evaluator. From the Office of Information
Management and Technology: Delaney Branch, Customer Service
Representative; Kenneth L. Cooper, Electronic Publishing Specialist;
Gail M. Traynham, Lead Electronic Publishing Specialist; and Wanda
L. Okoro, Customer Service Representative.
RELATED GAO PRODUCTS
============================================================ Chapter 1
Managing for Results: Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997).
Agencies' Strategic Plans Under GPRA: Key Questions to Facilitate
Congressional Review (GAO/GGD-10.1.16, May 1997).
GPRA: Managerial Accountability and Flexibility Pilot Did Not Work
As Intended (GAO/GGD-97-36, Apr. 10, 1997).
Performance Budgeting: Past Initiatives Offer Insights for GPRA
Implementation (GAO/AIMD-97-46, Mar. 27, 1997).
Measuring Performance: Strengths and Limitations of Research
Indicators (GAO/RCED-97-91, Mar. 21, 1997).
Managing for Results: Enhancing the Usefulness of GPRA Consultations
Between the Executive Branch and Congress (GAO/T-GGD-97-56, Mar. 10,
1997).
Managing for Results: Using GPRA to Assist Congressional and
Executive Branch Decisionmaking (GAO/T-GGD-97-43, Feb. 12, 1997).
DOD's GPRA Implementation (GAO/NSIAD/GGD-97-65R, Jan. 31, 1997).
Child Support Enforcement: Reorienting Management Toward Achieving
Better Program Results (GAO/HEHS/GGD-97-14, Oct. 25, 1996).
Information Technology Investment: Agencies Can Improve Performance,
Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept. 30, 1996).
Federal Downsizing: Better Workforce and Strategic Planning Could
Have Made Buyouts More Effective (GAO/GGD-96-62, Aug. 26, 1996).
Information Management Reform: Effective Implementation Is Essential
for Improving Federal Performance (GAO/T-AIMD-96-132, July 17, 1996).
Managing for Results: Key Steps and Challenges in Implementing GPRA
in Science Agencies (GAO/T-GGD/RCED-96-214, July 10, 1996).
Executive Guide: Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996).
Budget and Financial Management: Progress and Agenda for the Future
(GAO/T-AIMD-96-80, Apr. 23, 1996).
Budget Issues: Deficit Reduction and the Long Term
(GAO/T-AIMD-96-66, Mar. 13, 1996).
Managing for Results: Achieving GPRA's Objectives Requires Strong
Congressional Role (GAO/T-GGD-96-79, Mar. 6, 1996).
GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).
Transforming the Civil Service: Building The Workforce of the
Future, Results of A GAO-Sponsored Symposium (GAO/GGD-96-35, Dec.
20, 1995).
Financial Management: Continued Momentum Essential to Achieve CFO
Act Goals (GAO/T-AIMD-96-10, Dec. 14, 1995).
Financial Management: Momentum Must Be Sustained to Achieve the
Reform Goals of the Chief Financial Officers Act (GAO/T-AIMD-95-204,
July 25, 1995).
Managing for Results: Status of the Government Performance and
Results Act (GAO/T-GGD-95-193, June 27, 1995).
Government Restructuring: Identifying Potential Duplication in
Federal Missions and Approaches (GAO/T-AIMD-95-161, June 7, 1995).
Government Reorganization: Issues and Principles
(GAO/T-GGD/AIMD-95-166, May 17, 1995).
Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology (GAO/AIMD-94-115, May 1994).
*** End of document. ***