Managing for Results: An Agenda To Improve the Usefulness of Agencies'
Annual Performance Plans (Letter Report, 09/08/98, GAO/GGD/AIMD-98-228).

Pursuant to a congressional request, GAO summarized its reviews of
individual federal agency performance plans, focusing on opportunities
to improve the usefulness of future performance plans for
decisionmakers.

GAO noted that: (1) the agencies' first annual performance plans showed
the potential for doing performance planning and measurement as
envisioned by the Government Performance and Results Act to provide
decisionmakers with valuable perspective and useful information for
improving program performance; (2) however, overall, substantial further
development is needed for these plans to be useful in a significant way
to congressional and other decisionmakers; (3) most of the plans that
GAO reviewed contained major weaknesses that undermined their usefulness
in that they: (a) did not consistently provide clear pictures of
agencies' intended performance, (b) generally did not relate strategies
and resources to performance; and (c) provided limited confidence that
agencies' performance data will be sufficiently credible; (4) GAO
believe that Congress, the Office of Management and Budget (OMB), and
the agencies need to build on the experiences of the first round of
annual performance planning by working together and targeting key
performance issues that will help to make future plans more useful; (5)
most of the performance plans had at least some objective, quantifiable,
and measurable goals, but few plans consistently included a
comprehensive set of goals that focused on the results that programs
were intended to achieve; (6) the plans generally did not go further to
describe how agencies expected to coordinate their efforts with those of
other agencies; (7) most agencies' performance plans did not provide
clear strategies that described how performance goals would be achieved;
(8) the performance plans generally, provided listings of the agencies
current array of programs and initiatives but provided limited
perspective on how these programs and initiatives were necessary or
helpful for achieving results; (9) most of the plans did not adequately
describe the resources needed to achieve their agencies' performance
goals; (10) most annual performance plans provided only superficial
descriptions of procedures that agencies intended to use to verify and
validate performance data; and (11) the absence of program evaluation
capacity is a major concern, because a federal environment that focuses
on results depends on program evaluation to provide vital information
about the contribution of the federal effort.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GGD/AIMD-98-228
     TITLE:  Managing for Results: An Agenda To Improve the Usefulness 
             of Agencies' Annual Performance Plans
      DATE:  09/08/98
   SUBJECT:  Agency missions
             Data integrity
             Reporting requirements
             Strategic planning
             Program evaluation
             Congressional/executive relations
             Federal agency reorganization
             Interagency relations
             Accountability
IDENTIFIER:  Y2K
             GPRA
             Government Performance and Results Act
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to Congressional Requesters

September 1998

MANAGING FOR RESULTS - AN AGENDA
TO IMPROVE THE USEFULNESS OF
AGENCIES' ANNUAL PERFORMANCE PLANS

GAO/GGD/AIMD-98-228

Agencies' Annual Performance Plans

(410315/935258)


Abbreviations
=============================================================== ABBREV

  ATF - Bureau of Alcohol, Tobacco and Firearms
  CDC - Centers for Disease Control and Prevention
  CFO - Chief Financial Officer
  DI - Disability Insurance
  EPA - Environmental Protection Agency
  FEMA - Federal Emergency Management Agency
  GSA - General Services Administration
  HHS - Department of Health and Human Services
  HUD - Department of Housing and Urban Development
  IHS - Indian Health Service
  IG - Inspector General
  NASA - National Aeronautics and Space Administration
  NPS - National Park Service
  NRC - Nuclear Regulatory Commission
  NSF - National Science Foundation
  OMB - Office of Management and Budget
  OPM - Office of Personnel Management
  OASI - Old Age and Survivors Insurance
  SBA - Small Business Administration
  SSA - Social Security Administration
  USAID - U.S.  Agency for International Development
  VA - Department of Veterans Affairs

Letter
=============================================================== LETTER


B-280472

September 8, 1998

The Honorable Newt Gingrich
Speaker of the House of Representatives

The Honorable Richard K.  Armey
Majority Leader
House of Representatives

The Honorable John Kasich
Chairman, Committee on the Budget
House of Representatives

The Honorable Dan Burton
Chairman, Committee on Government Reform
 and Oversight
House of Representatives

The Honorable Bob Livingston
Chairman, Committee on Appropriations
House of Representatives

The Government Performance and Results Act of 1993 (the Results Act)
is the centerpiece of a statutory framework that Congress put in
place during the 1990s to strengthen federal decisionmaking and
accountability and to improve federal effectiveness and efficiency by
promoting a new focus on results, service quality, and customer
satisfaction.\1 The Results Act seeks to create this new focus by
establishing a system of strategic and annual performance planning to
set goals for program performance and to measure results. 

With the implementation of the Act's requirement for fiscal year 1999
annual performance plans, agencies across the government have, for
the first time, made a systematic attempt to establish performance
goals and related measures for a given year.  The plans are intended
to clearly inform Congress and the public of the annual performance
goals for agencies' major programs and activities, the measures that
will be used to gauge performance, the strategies and resources
required to achieve the performance goals, and the procedures that
will be used to verify and validate performance information.  Annual
performance plans are to assist congressional and executive branch
decisionmakers by providing a road map that links long-term strategic
goals and agency missions to daily operations and the results to be
achieved for a proposed level of resources.  The agencies' annual
performance plans are also to support the development of the
governmentwide performance plan that the Office of Management and
Budget (OMB) must submit to Congress with the President's Budget each
year. 

In January 1998, you requested that we review the fiscal year 1999
annual performance plans of the 24 agencies covered by the Chief
Financial Officers (CFO) Act, which account for about 98 percent of
the federal government's net outlays.  (See app.  I for a list of the
agencies.) The overall purpose of those reviews was to assess the
usefulness of agencies' plans for decisionmaking by determining the
extent to which the plans provide a clear picture of intended
performance, discuss strategies and resources to achieve performance
goals, and provide confidence that performance information will be
credible.  For our reviews, we organized the Results Act's criteria
under those three areas.  We briefed congressional staff on the
results of this work in April 1998, and, as agreed, we have issued
separate reports on each agency plan that we reviewed.  (See app. 
III for the list of reports on agencies' performance plans.) In
addition, we are issuing a separate report on our review of the
governmentwide performance plan required by the Results Act.\2

This report responds to your request for summary information on the
reviews of the individual agency performance plans.  The specific
objectives of this report were to (1) summarize our observations on
the agencies' performance plans and (2) identify opportunities to
improve the usefulness of future performance plans for
decisionmakers.  Appendix II provides additional information on our
scope and methodology for this review. 


--------------------
\1 Managing for Results:  The Statutory Framework for
Performance-Based Management and Accountability (GAO/GGD/AIMD-98-52,
Jan.  28, 1998). 

\2 The Results Act:  Assessment of the Governmentwide Performance
Plan for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept.  8, 1998). 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

Agencies' first annual performance plans showed the potential for
doing performance planning and measurement as envisioned by the
Results Act to provide decisionmakers with valuable perspective and
useful information for improving program performance.  For example,
almost all of the plans showed how agencies' missions and strategic
goals related to their performance goals.  However, overall,
substantial further development is needed for these plans to be
useful in a significant way to congressional and other
decisionmakers.  Most of the plans that we reviewed contained major
weaknesses that undermined their usefulness in that they (1) did not
consistently provide clear pictures of agencies' intended
performance, (2) generally did not relate strategies and resources to
performance, and (3) provided limited confidence that agencies'
performance data will be sufficiently credible.  Although all of the
plans contained valuable information for decisionmakers, their
weaknesses caused their usefulness to vary considerably within and
among plans.  We believe that Congress, OMB, and the agencies need to
build on the experiences of the first round of annual performance
planning by working together and targeting key performance issues
that will help to make future plans more useful. 

Most of the performance plans had at least some objective,
quantifiable, and measurable goals, but few plans consistently
included a comprehensive set of goals that focused on the results
that programs were intended to achieve.  Moreover, agencies did not
consistently follow OMB's guidance that goals for major management
problems be included in the plans.  Most significantly, most
agencies' plans did not include steps to address the Year 2000
computing problem.  On the other hand, agencies that included
information such as baseline or trend data for their performance
goals helped to underscore the potential usefulness of performance
plans.  Agencies' plans generally showed how the agencies' missions
and strategic goals were related to their annual performance goals
and covered all of the program activities\3 in the agencies' budget
requests.  In addition, many agencies took a needed first step of
identifying their crosscutting efforts, with some including helpful
listings of other agencies with which they shared a responsibility
for addressing similar national issues.  However, the plans generally
did not go further to describe how agencies expected to coordinate
their efforts with those of other agencies. 

Most agencies' performance plans did not provide clear strategies
that described how performance goals would be achieved.  The
performance plans generally provided listings of the agencies'
current array of programs and initiatives but provided limited
perspective on how these programs and initiatives were necessary or
helpful for achieving results.  The most useful plans presented
strategies that were clear and appeared logically related to annual
performance goals.  The better plans also discussed the external
factors that could influence the degree to which goals are achieved
and provided strategies to mitigate the negative factors or take
advantage of positive factors as appropriate.  On the other hand,
most of the plans did not adequately describe--or reference other
appropriate documents that describe--the capital, human, information,
and financial resources needed to achieve their agencies' performance
goals. 

Most annual performance plans provided only superficial descriptions
of procedures that agencies intended to use to verify and validate
performance data.  Moreover, in general, agencies' performance plans
did not include discussions of documented limitations in financial
and other information systems that may undermine efforts to produce
high-quality data.  Without such information, and strategies to
address those limitations, Congress and other decisionmakers cannot
assess the validity and reliability of performance information.  In
addition, we continue to be concerned about the lack of a capacity in
many federal agencies to undertake the program evaluations that will
be vital to the success of the Results Act.  The absence of program
evaluation capacity is a major concern, because a federal environment
that focuses on results--where federal efforts are often but one
factor among many that determine whether goals are achieved--depends
on program evaluation to provide vital information about the
contribution of the federal effort. 


--------------------
\3 The term "program activity" refers to the listing of projects and
activities in the appendix portion of the Budget of the United States
Government.  Program activity structures are intended to provide a
meaningful representation of the operations financed by a specific
budget account. 


   BACKGROUND
------------------------------------------------------------ Letter :2

In crafting the Results Act, Congress drew on the experiences of
foreign governments and state and local governments in the United
States and recognized that the results-oriented goal setting and
performance measurement requirements of the Results Act would
constitute a new way of doing business for many agencies.  Congress
also realized that the effective implementation of the Results Act
may take several years.  To advance this effort, the Results Act
provided for a series of pilot projects so that agencies could gain
experience and share lessons learned in implementing the key
provisions of the Results Act before its governmentwide
implementation.  One set of these pilot projects covered the Act's
annual performance planning and reporting provisions.  Over 70
federal organizations participated in this pilot phase, which covered
fiscal year 1994 through fiscal year 1996.  To further help agencies,
several Members of Congress asked us to develop--on the basis of the
experiences of leading foreign, state, and federal organizations--a
guide for agency managers to use to effectively implement the Act.\4

We observed in our June 1997 report on the implementation of the
Results Act and related performance-based management initiatives that
despite the rich body of experience the pilots provided, initial
governmentwide implementation of the Results Act would be highly
uneven.\5

We identified a series of daunting implementation challenges and
predicted that the initial set of agency strategic and annual
performance plans would not be of consistently high quality or as
useful for congressional and executive branch decisionmakers as they
could be. 

At the request of several members of the congressional leadership, in
May 1997 we issued a guide for congressional staff to use in
assessing agencies' strategic plans.\6 We subsequently reviewed draft
and September 30, 1997, strategic plans that agencies submitted to
Congress.  In our January 1998 summary report on our reviews of the
September plans, we highlighted three difficult planning challenges
that especially needed continued progress:  setting a strategic
direction, including establishing clear, results-oriented goals and
performance measures; coordinating crosscutting programs; and
ensuring the capacity to gather and use performance information.\7 We
suggested that agencies' annual performance plans could help address
these challenges. 

As a next step in our efforts to assist Congress and agencies in
effectively implementing the Results Act, we issued two related
guides--one for congressional decisionmakers and one for evaluators
and others interested in more detailed assessments--on assessing
annual performance plans.\8 These guides, developed with the
assistance of congressional staff, senior officials in agencies,
members of the CFO Council, and others, integrated criteria from the
Results Act, its legislative history, OMB's guidance for developing
the plans (OMB Circular A-11, part 2), and our work on implementation
of the Results Act.  The guides organize the Results Act's criteria
under three core questions that are aimed at ensuring that
performance plans are useful for decisionmaking.  The three core
questions are:  (1) To what extent does the agency's performance plan
provide a clear picture of intended performance across the agency? 
(2) How well does the agency's performance plan discuss the
strategies and resources the agency will use to achieve its
performance goals?  (3) To what extent does the agency's performance
plan provide confidence that its performance information will be
credible? 

We noted that as agencies and Congress gain experience in developing
and using annual performance plans, additional issues and questions
will emerge.  We therefore have committed to issuing a combined,
updated version of our congressional and evaluators' performance plan
guides reflecting those experiences and providing examples drawn from
the agencies' plans illustrating useful presentations.  An exposure
draft of that guide will be issued this fall. 


--------------------
\4 Executive Guide:  Effectively Implementing the Government
Performance and Results Act (GAO/GGD-96-118, June 1996). 

\5 The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

\6 Agencies' Strategic Plans under GPRA:  Key Questions to Facilitate
Congressional Review, Version 1 (GAO/GGD-10.1.16, May 1997). 

\7 Managing for Results:  Agencies' Annual Performance Plans Can Help
Address Strategic Planning Challenges (GAO/GGD-98-44, Jan.  30,
1998). 

\8 Agencies' Annual Performance Plans Under the Results Act:  An
Assessment Guide to Facilitate Congressional Decisionmaking, Version
1 (GAO/GGD/AIMD-10.1.18, Feb.  1998); and The Results Act:  An
Evaluator's Guide to Assessing Agency Annual Performance Plans,
Version 1 (GAO/GGD-10.1.20, Apr.  1998). 


   AGENCY PERFORMANCE PLANS DID
   NOT CONSISTENTLY PROVIDE CLEAR
   PICTURES OF INTENDED
   PERFORMANCE
------------------------------------------------------------ Letter :3

At the most basic level, an annual performance plan is to provide a
clear picture of intended performance across the agency.  Such
information is important to Congress, agency managers, and others for
understanding what the agency is trying to achieve, identifying
subsequent opportunities for improvement, and assigning
accountability.  We found that the plans did not consistently provide
the succinct and concrete statements of intended performance that are
needed to help guide decisions and subsequently assess actual
performance.  The plans generally were successful in showing how an
agency's mission and strategic goals were related to its performance
goals.  This is a very positive development because it provides a
basis for using the performance plans to track progress toward the
achievement of agencies' long-term strategic goals.  The plans were
much less successful in providing assurance that crosscutting program
efforts were sufficiently coordinated.  Agencies appear to be taking
the first step of identifying crosscutting efforts, with some
including helpful listings of other agencies with which they share
common goals.  However, few plans provided any descriptions of how
the agency will coordinate with other agencies regarding national
issues for which they share responsibility or reflected other
substantive coordination. 


      DEFINING EXPECTED
      PERFORMANCE
---------------------------------------------------------- Letter :3.1

Almost all of the annual performance plans that we reviewed contained
at least some objective, quantifiable, and measurable annual
performance goals--a key expectation of Congress in enacting the
Results Act.  Overall, however, the annual performance goals and
accompanying measures in the plans would need significant development
to improve the usefulness of the plans to congressional and other
decisionmakers.  Specifically, we found that the goals in the annual
performance plans often were not as results-oriented as they could
be; the relationship between goals and performance measures at times
was either neglected or obscured; and the plans did not consistently
set goals to address major management problems, as suggested in OMB
guidance.  On the other hand, some of the plans provided very helpful
baseline and trend data for performance goals.  Such information
allows users of plans to judge whether performance targets are
appropriate and reasonable based on past performance. 


      FOCUSING ON RESULTS
---------------------------------------------------------- Letter :3.2

Goals in the performance plans that we reviewed typically focused on
program outputs, such as the number of products and services
delivered by the agency.  The Results Act allows agencies to include
output goals in their plans, and such goals can provide important
information for agency managers to use in managing programs. 
However, the Act envisions that agencies' plans would contain goals
that focus on the results that programs are intended to achieve,
which is particularly important for policymakers.  We found that the
annual performance plans did not consistently contain such
results-oriented goals. 

For example, the Social Security Administration (SSA), which was
responsible for expenditures of about $400 billion in
1997--constituting one-fourth of the federal budget--did not
consistently have results-oriented performance goals.  For its
high-risk Supplemental Security Income program, SSA's plan included
output goals on the number of claims processed and the number of
nondisability redeterminations, but it did not include
results-oriented goals.  Likewise, for SSA's Old Age and Survivors
Insurance (OASI) and Disability Insurance (DI) programs, the
performance plan had output goals related to the number of
beneficiaries served, but it did not contain results-oriented goals
related to services these beneficiaries receive.  However, the
governmentwide performance plan's chapter on Social Security included
a results-oriented discussion of the effect of Social Security on
reducing poverty among the elderly in addition to addressing the
number of beneficiaries served by the OASI and DI programs.  Having
output-oriented goals will provide decisionmakers with important
information but will not directly provide a perspective on the degree
to which the program is accomplishing the results it is intended to
achieve. 

In crafting the Results Act, Congress recognized that for some types
of federal programs it may not be feasible for an agency to express
its performance goals in an objective, quantifiable, and measurable
form.  The Results Act therefore allows an agency to propose, and OMB
to authorize, that a goal be expressed in an alternative form, such
as by describing a minimally effective program and a successful
program.  Although few agencies used the alternative form of
measurement for fiscal year 1999, the experiences of the National
Science Foundation (NSF) suggested how alternative forms of
measurement could be employed.  The agency's performance plan used
such alternative descriptions to establish annual performance goals
for its scientific research and educational activities.  For example,
NSF's plan described annual success in addressing the agency's
strategic goal of promoting scientific discovery as occurring when
the agency's awards lead to important discoveries and new knowledge
within and across traditional disciplinary boundaries.  NSF's plan
described corresponding minimal effectiveness as occurring when there
is a steady stream of outputs of good scientific quality.  By
establishing definitions for successful and minimally effective
levels of performance, NSF's descriptions allowed the agency's
performance to be assessed, both by congressional and executive
branch decisionmakers and by expert reviewers that NSF plans to use. 
NSF could build on its approach to measurement by better explaining
what it means by such phrases as "important discoveries" and "steady
stream of outputs of good scientific quality." One way to do this
would be to provide examples of past discoveries that illustrate each
of the descriptive statements. 

In contrast to many agencies, the Department of Health and Human
Services' (HHS) Centers for Disease Control and Prevention (CDC) had
numerous concrete measurable results-oriented performance goals.  For
example, CDC had a results-oriented goal and measure to reduce the
incidence of congenital syphilis in the general population from the
1995 rate of 39 per 100,000 live births to less than 30 in fiscal
year 1999.  CDC's program outputs related to its goal, such as
targeted prenatal screenings for congenital syphilis, were presented
as the strategies CDC will use to achieve its end result rather than
as the programmatic end in itself.  Such a presentation suggests a
clear understanding of the relationships and differences between the
activities an agency undertakes and the results it hopes to achieve. 
In addition, when CDC used output-oriented performance measures in
some cases, it explained why it used such measures rather than more
results-oriented measures.  The section on chronic disease
prevention, for example, noted that health outcome measures for these
programs have been difficult to define for a number of reasons,
including the long latency period of chronic diseases like cancer and
heart disease. 


         RELATIONSHIPS BETWEEN
         GOALS AND MEASURES
-------------------------------------------------------- Letter :3.2.1

One of the major challenges that agencies face in moving from a focus
on the activities they undertake to results they are trying to
achieve is to develop performance measures that clearly and
sufficiently\9 relate to the performance they are meant to assess. 
At CDC, for example, the performance goal to reduce the incidence of
congenital syphilis was clearly and sufficiently represented by the
performance measure of reducing the occurrence of congenital syphilis
from 39 per 100,000 live births to less than 30 per 100,000 live
births. 

Far more typical were situations where the relationships between what
is being measured and desired results are not sufficient.  For
example, three of the Department of Labor's (Labor) performance goals
used the number of complaints received as measures of compliance with
worker protection and civil rights laws.  In one case, a decrease in
the number of discrimination complaints filed by federal grant
recipients and persons with disabilities in state and local
governments was used to indicate progress towards the goal of
ensuring that workplaces are fair for these groups.  Used alone, such
a measure is a questionable indicator of fairness in the workplace. 
A decrease in the number of complaints could also be a function of
lack of information, fear, a complainant's lack of confidence in
Labor's enforcement, or a tendency of agency managers to discourage
the filing of otherwise meritorious complaints.  An expanded or
improved enforcement program could produce an increase in complaints
as workers gain confidence in the enforcement agency.  Without other
independent measures that also demonstrate the existence of a fair
workplace, measuring the decrease in complaints may be insufficient. 

Also, programs often must achieve multiple goals or goals with
several dimensions that reflect such competing demands or priorities
as quality, timeliness, program cost, and outcomes.  Annual
performance plans that do not contain balanced sets of measures may
not sufficiently assess all aspects of a goal or multiple goals for
the agencies' programs.  One of the key priorities missed in many
plans was cost.  For example, the Office of Personnel Management's
(OPM) plan did not appear to have cost-based performance measures to
show how efficiently it performs certain businesslike operations,
such as the administration of health and retirement programs.  On the
other hand, the Department of Veterans Affairs' (VA) plan provided
both financial and nonfinancial measures for some of its program
areas.  For example, VA's performance plan contained measures that
addressed various program priorities, such as the accuracy and
timeliness of claims processing, unit costs of providing benefits and
services, and customer satisfaction with VA services. 

In addition, agencies' performance measures did not always have a
clearly apparent or commonly accepted relationship to the performance
goals.  At the Department of the Interior's (Interior) National Park
Service (NPS), some performance measures did not provide clear
definitions of the criteria that would be used to accurately assess
the performance.  For example, one of NPS' performance goals was to
ensure that 50 percent of the cultural landscapes on its Cultural
Landscapes Inventory were in good condition.  However, the plan did
not define "good condition" or make reference to where such a
definition could be found.  Without such a definition, neither the
precise relationship between the measure and the desired result nor
whether performance is being measured consistently from year to year
can be determined. 


--------------------
\9 A sufficient measure or set of measures, as used in our annual
performance plan guidance for evaluators, addresses the key
dimensions of a performance goal, reflects the significant priorities
of the relevant programs or activities, and has acceptable
reliability and validity. 


         ADDRESSING MAJOR
         MANAGEMENT PROBLEMS
-------------------------------------------------------- Letter :3.2.2

According to OMB guidance, an agency's annual performance plan should
also set performance goals to address major management problems that
are mission-critical or impede the agency's ability to meet its
programmatic goals.  We found, however, that agencies did not
consistently set goals to address major management problems.  Most
significantly, the governmentwide performance plan's first priority
objective is to ensure that agencies' business processes and
supporting systems function successfully in the year 2000 and beyond. 
Even though there were agency plans that otherwise acknowledged this
issue, most of the agencies' plans neglected to include steps to
address it.  For example, in the Small Business Administration's
(SBA) case, although its performance plan discussed actions SBA
planned to take to help small businesses deal with the Year 2000
problem, the plan did not discuss or provide information on SBA's
efforts to resolve the agency's own Year 2000 problems.  In addition,
the Interior Departmental Overview\10 section of its performance plan
listed ensuring that the Department's critical information systems
and processes are Year 2000-compliant by March 31, 1999, as a
strategic goal for the Department.  However, six of the eight
subagency plans did not address the problem. 

On the other hand, some agencies' plans included performance goals
and measures to show how the Year 2000 issue would be addressed.  OPM
had a fiscal year 1999 annual performance goal to ensure that OPM's
information technology systems would operate properly on and after
January 1, 2000.  One of OPM's measures for this goal was that the
agency's systems would meet or improve on the OMB-established
governmentwide target dates for Year 2000 compliance in that all
systems would be renovated by September 1998 and would be implemented
in a Year 2000-compliant environment by December 1998. 


--------------------
\10 Interior's annual performance plan consisted of (1) a
Departmental Overview with agencywide management goals, goals for
ecosystem initiatives, and goals for other programs not contained in
the subagency plans; and (2) sections for each of the eight Interior
bureaus. 


         BASELINE AND TREND
         PERFORMANCE DATA
-------------------------------------------------------- Letter :3.2.3

Agencies that go beyond the requirements of the Results Act and
include baseline or trend data for their performance goals in their
annual performance plans provide a more informative basis for
assessment of expected performance.  Reliable baseline and trend data
are helpful to providing congressional and other decisionmakers with
a context for assessing whether performance goals are reasonable and
appropriate and suggesting areas for questions about how planned
performance improvements will be achieved.  For example, the
Department of Commerce's (Commerce) annual performance plan generally
provided performance data from fiscal year 1997 when available, the
fiscal year 1998 goal, and the fiscal year 1999 goal.  As an
illustration, the performance plan showed that although the fiscal
year 1998 goals for the lead times and the accuracy of flash flood
warnings were the same as the actual levels achieved in fiscal year
1997, performance improvements were planned in both areas for fiscal
year 1999. 


         CONNECTING MISSION,
         GOALS, AND ACTIVITIES
-------------------------------------------------------- Letter :3.2.4

The annual performance plans that we examined were generally
successful in showing how the agencies' missions and strategic goals
were related to their performance goals.  Indicating such
relationships is important for showing how an agency will chart
annual progress toward the achievement of its long-term strategic
goals.  The Environmental Protection Agency's (EPA) performance plan
included one of the most effective presentations in this regard. 
This plan included the mission statement and goals from the strategic
plan and a section on the relationship between the two plans,
including changes in the strategic goals and objectives since the
strategic plan was issued.  In addition, the plan was primarily
organized by the strategic goals and objectives, with performance
goals, resources, strategies, and performance measures grouped by
strategic goal and objective. 

As required by the Results Act, agencies generally developed
performance plans that covered all program activities in their budget
requests--with many agencies establishing for the first time a direct
connection among plans, budgets, performance information, and the
related congressional resource allocation and oversight processes. 
Agencies used various approaches to make this connection.  Some
provided descriptions or tables that associated performance goals
with their existing program activities, and others took advantage of
the flexibility provided by the Results Act to aggregate,
disaggregate, and consolidate program activities to indicate
coverage.\11 For example, EPA's and the Department of
Transportation's performance plans showed a relationship between
performance goals and program activities by linking them to strategic
goals and/or objectives.  Whether the agencies used existing program
activities or aggregated, disaggregated, or consolidated program
activities, the most useful linkages indicated how funding from the
agency's program activities would be allocated to a discrete set of
performance goals. 

In contrast, the performance plans that associated one or more
performance goals with one or more program activities were less
informative in this regard.  This is because such associations made
it difficult to determine whether all activities were substantively
covered or understand how specific program activities were intended
to contribute to the agency's results.  For example, the SBA plan did
not convey which performance goals covered which program activities
or whether all of SBA's program activities were covered by
performance goals.  The Interior plan contained some goals for NPS
that were not associated with any program activities, even though the
goals apparently required some funding.  For example, NPS' goal of
improving the quality of its employee housing through removing,
replacing, or upgrading units was not related to any program
activity.  Although the Department of Housing and Urban Development's
(HUD) plan related some of the agency's program activities to its
goals, it did not cover all of HUD's program activities or explain
whether those activities were aggregated, disaggregated, or
consolidated.  For example, HUD did not explain which performance
goals covered the $310 million drug elimination grants for its
low-income housing program. 


--------------------
\11 OMB's Results Act guidance in Circular A-11 states that agencies
may aggregate, disaggregate, or consolidate the budgets' program
activities so that they align with the goals presented in the
performance plan.  Aggregation combines program activities within one
of an agency's budget accounts, disaggregation breaks a single
program activity in one budget account into two or more activities,
and consolidation combines some or all of the program activities in
two or more budget accounts to form a single program activity that
appears in the performance plan. 


      RECOGNIZING CROSSCUTTING
      EFFORTS
---------------------------------------------------------- Letter :3.3

Over the last several years, we have produced a body of work pointing
to mission fragmentation and overlap in a wide variety of federal
program areas.\12 Our work has shown that uncoordinated program
efforts can waste scarce funds, confuse and frustrate program
customers, and limit the overall effectiveness of the federal effort. 
We have suggested that agencies' efforts under the Results Act
provide a potentially effective vehicle for ensuring that
crosscutting program goals are consistent; strategies are mutually
reinforcing; and, as appropriate, progress is assessed through the
use of common performance measures. 

Last fall, when we reviewed agencies' strategic plans, we stressed
that coordinating crosscutting programs can be a difficult and
time-consuming process.  To underscore our concern, we highlighted
the issue as one of the most difficult planning challenges requiring
continued progress.  In our review of agencies' September 1997
strategic plans, we found that those plans provided better
descriptions of crosscutting programs and coordination efforts than
the agencies' draft strategic plans.  The most useful of the
strategic plans contained presentations that listed other agencies
involved in crosscutting program areas and outlined approaches to
coordinating such areas with those agencies.  We noted that such
presentations illustrated the magnitude of, and provided a foundation
for, the much more difficult work that lies ahead--undertaking the
substantive coordination that is needed to ensure that crosscutting
programs are effectively managed. 

Since then, agencies appear to have made uneven progress.  Our review
of agencies' annual performance plans suggested that the needed first
step is now being taken more consistently--the plans often identified
crosscutting efforts, and some included helpful listings of other
agencies with which responsibility for addressing similar national
issues is shared.  However, few plans attempted the more challenging
description of how the agencies expected to coordinate their efforts
with those of other agencies or reflected the existence of
substantive coordination. 

As an illustration, Commerce identified a number of other federal
agencies' programs that are related to Commerce's three strategic
themes and its bureaus' activities.  However, the agency's
performance plan did not indicate how Commerce would work with these
other agencies in addressing shared activities.  For example, the
plan associated 12 other federal agencies with Commerce's
International Trade Administration through the Trade Promotion
Coordinating Committee.  However, neither the plan nor the supporting
congressional budget justification documents explained how Commerce
can use its key role in chairing the Committee to accomplish
Commerce's strategic goal of implementing the national export
strategy.\13

Similarly, the National Aeronautical and Space Administration's
(NASA) performance plan also took the first step of identifying other
agency or international partners involved in specific efforts related
to NASA's efforts.  However, it did not discuss the extent to which
NASA had coordinated with other agencies in establishing the goals,
objectives, and associated performance targets.  For example, in
describing the objective of developing next-generation computational
design tools, the plan indicated that NASA's efforts were part of the
Federal High Performance Computing and Communications initiative. 
However, there was no discussion about whether NASA coordinated its
performance target of a 200-fold improvement with other federal
partners; nor was there an explanation of how NASA's effort will
contribute to the overall federal initiative, separately from the
contributions that other agencies will make. 

A few performance plans were more useful in that they discussed how
agencies expected to coordinate efforts with other agencies that have
similar responsibilities.  Similar to the most useful strategic
plans, such discussions underscored the magnitude of the coordination
work that lies ahead.  For example, Education's plan contained not
only an extensive list of other agencies with which the Department
shares a common result but also a general discussion of its
coordination efforts and plans.  For its strategic objective that
every state have a school-to-work system that increases student
achievement, improves technical skills, and broadens career
opportunities for all, Education's plan indicated that the agency
plans a coordination effort with Labor to jointly administer the
National School-to-Work Office Program and improve the management of
that program by aligning grant-making, audit, technical assistance,
and performance reporting functions.  Education can build on its
foundation by identifying (1) performance goals that reflect its
crosscutting programs, (2) how Education and other agencies will work
to ensure that program strategies are mutually reinforcing, and (3)
whether any common performance measures are to be used. 


--------------------
\12 Managing for Results:  Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug.  29, 1997). 

\13 See Export Promotion:  Issues for Assessing Governmentwide
Strategy (GAO/T-NSIAD-98-105, Feb.  26, 1998). 


   AGENCY PERFORMANCE PLANS
   GENERALLY DID NOT RELATE
   STRATEGIES AND RESOURCES TO
   PERFORMANCE
------------------------------------------------------------ Letter :4

In general, the annual performance plans did not provide sufficiently
complete discussions of the strategies and resources that agencies
will use to achieve their performance goals.  Discussions in the
plans of agency strategies, which can include program initiatives,
partnerships, and operational processes, frequently did not yield
clear understandings of how the strategies were to lead to improved
agency performance and the achievement of annual performance goals. 
Moreover, the plans often lacked complete discussions of the capital,
human, financial, and other resources that the agencies needed to
achieve their goals.  The absence of fully developed discussions
relating strategies and resources to goals undermined the usefulness
of the plans.  As a result, congressional and other decisionmakers do
not have complete information on which to judge the reasonableness of
an agency's proposed strategies and resources needed to achieve its
goals. 


      CONNECTING STRATEGIES TO
      RESULTS
---------------------------------------------------------- Letter :4.1

Most agencies' annual performance plans did not clearly describe how
the performance goals would be achieved.  The performance plans often
provided listings of the agencies' current array of programs and
initiatives but provided only limited perspective on how these
programs and initiatives were necessary to or helpful for achieving
results.  For example, the General Services Administration (GSA)
performance plan provided descriptive information on GSA's activities
as opposed to specific strategies for achieving performance goals. 
One of GSA's performance goals was to increase market share for its
vehicle fleet program.  Although the plan contained measures and
target levels for fiscal years 1998 and 1999, the accompanying
narrative gave little indication of how GSA intended to achieve the
target levels.  Instead, the plan provided general statements about
leveraging GSA's competitive pricing with broad market penetration
and government downsizing.  The plan offered no information on a
specific approach or strategy for how GSA would achieve broad market
penetration or take advantage of downsizing to meet the market share
target levels for its vehicle fleet program. 

Far less typical, but far more useful in our view, was the approach
taken by the Federal Emergency Management Agency (FEMA), which
generally presented strategies that were clear and appeared logically
related to annual performance goals.  For example, to achieve its
strategic goal of protecting lives and preventing the loss of
property, FEMA's performance plan contained a performance goal to
increase Project Impact communities in each state for its Project
Impact program, which is designed to promote predisaster mitigation. 
Strategies for achieving this goal included working with states and
federal agencies to identify candidate communities, providing grants
as seed funding, providing technical information, and monitoring
progress. 

We also observed that most agencies did not build on the work they
had done last fall in developing their strategic plans where they
were to identify factors external to the agency that would affect the
degree to which they achieve their strategic goals.  Such factors
could include economic, demographic, social, technological, or
environmental factors.  Assessments of external factors help agencies
and Congress judge the likelihood of an agency achieving its
strategic goals and the actions needed to better meet those goals. 
Similar to the situation with strategic goals, discussions of the
influence external factors can have on annual performance goals,
although not required by the Results Act, can provide important
context for understanding both the factors other than agency
performance that can affect whether goals are achieved and the
adequacy of the agencies' plans for mitigating negative factors and
taking advantage of positive ones. 

The value of including such an analysis in annual performance plans
is shown by SBA's plan.  SBA provided an informative discussion of
external factors, such as the economy and continued support from
stakeholder and program partners, that might affect the agency's
ability to achieve performance goals related to its strategic goal to
become a 21st century leading-edge financial institution.  Within
this context, the plan also included a discussion of actions SBA can
take to mitigate these factors. 


      CONNECTING RESOURCES TO
      STRATEGIES
---------------------------------------------------------- Letter :4.2

Most agencies' annual performance plans did not adequately describe
capital, human, information, and financial resources and relate them
to achieving performance goals.  Even in cases where the achievement
of performance goals seems to depend on increased staffing levels or
capital expenditures, such increases were not always described in the
plans.  For example, VA included a discussion concerning the
expansion and construction of its cemetery system, but it does not
identify the additional dollars, people, or equipment necessary to
achieve this goal. 


         INFORMATION TECHNOLOGY
-------------------------------------------------------- Letter :4.2.1

Addressing information technology issues in annual performance plans
is important because of technology's critical role in achieving
results, the sizable investment the federal government makes in
information technology (about $145 billion between 1992 and 1997),
and the long-standing weaknesses in virtually every agency in
successfully employing technology to further mission accomplishment. 
The vital role that information technology can play in helping
agencies achieve their goals was not clearly described in agency
plans.  In the absence of such discussions, the plans also generally
did not reference other appropriate documents that might contain
information on the agencies' technology plans.  The failure to
recognize the central role of technology in achieving results is a
cause of significant concern because, under the Paperwork Reduction
and Clinger-Cohen Acts, Congress put in place clear statutory
requirements for agencies to better link their technology plans and
information technology use to their missions and programmatic goals. 
Without at least some discussion of information technology, agency
plans are not complete, and their usefulness to congressional and
other decisionmakers is accordingly undermined. 

The Department of State's (State) and Interior's performance plans
were fairly typical of all agencies' plans in terms of the lack of
attention to technology issues.  Although the State plan discusses
upgrading the information technology infrastructure, along with other
resources, it does not address how such an upgrade would be used to
improve performance or help achieve specific performance goals.  At
Interior, with the exception of the United States Geological Survey's
plan, the subagency plans generally did not discuss how information
technology will be used to help achieve annual performance goals or
improve performance for long-term objectives. 


         LINKING FUNDING REQUESTS
         TO PERFORMANCE
-------------------------------------------------------- Letter :4.2.2

As discussed earlier in the report, most agencies developed
performance goals that cover the program activities in their budget
requests.  In addition, OMB Circular A-11 states that agencies should
display, by program activity, the funding level being applied to
achieve performance goals.  However, most agencies did not clearly
convey in their annual performance plans the amount of funding needed
to achieve a discrete set of performance goals.  For example, as
illustrated in figure 1, VA aggregated program activities under
specific budget accounts and consolidated these activities across
those budget accounts to align those accounts (and the underlying
program activities) with groupings of performance goals for two of
its business lines--(1) Education and (2) Vocational Rehabilitation
and Counseling.  Although this association established a relationship
between numerous program activities and numerous performance goals,
it did not identify the funding levels that are needed to achieve
discrete sets of goals.  As a result, the plan did not convey how
requested funds under the related program activities will be
allocated to achieve performance. 

   Figure 1:  Relationship Between
   VA's Performance Goals and
   Measures and the Program
   Activities for a Component's
   Programs

   (See figure in printed
   edition.)

   Source:  GAO analysis based on
   VA's fiscal year 1999 annual
   performance plan and Budget of
   the United States Government
   Fiscal Year 1999--Appendix.

   (See figure in printed
   edition.)

In contrast, by identifying how much funding is needed to support
discrete sets of performance goals and showing where that funding was
included in the agency's budget request, an agency's annual
performance plan can give Congress and other decisionmakers
information needed to relate decisions about desired performance with
decisions about funding levels.  A few agencies, such as EPA and some
components of the Department of Treasury (Treasury), proposed changes
to their program activity structures to better facilitate such an
allocation and establish a clearer connection between budgetary
decisionmaking and performance.  For example, EPA proposed a uniform
program activity structure across all of its budget accounts in which
each program activity represents one of the agency's strategic goals. 
Using this proposed program activity structure in its performance
plan, EPA showed by account the funding it is requesting to achieve
each strategic objective and the supporting annual performance goals. 
Figure 2 illustrates this relationship. 

   Figure 2:  Relationship Between
   EPA's Performance Goals and
   Program Activities

   (See figure in printed
   edition.)

   Source:  GAO analysis based on
   EPA's fiscal year 1999 annual
   performance plan and Budget of
   the United States Government
   Fiscal Year 1999--Appendix.

   (See figure in printed
   edition.)

Similarly, Treasury's Bureau of Alcohol, Tobacco and Firearms (ATF)
has changed its program activities to align them with its strategic
goals.  Because ATF's revised program activities reflect its
strategic goals, its performance plan clearly showed how ATF would
allocate its resources and the bureau's performance goals can be
readily and logically related to the program activity structure. 
Figure 3 shows the relationship described between performance goals
and program activities in ATF's plan. 

   Figure 3:  Relationship Between
   ATF's Performance Goals and
   Program Activities

   (See figure in printed
   edition.)

   Source:  GAO analysis based on
   the Department of the
   Treasury's fiscal year 1999
   annual performance plan and
   Budget of the United States
   Government Fiscal Year
   1999--Appendix.

   (See figure in printed
   edition.)


   PERFORMANCE PLANS PROVIDED
   LIMITED CONFIDENCE THAT
   PERFORMANCE DATA WOULD BE
   CREDIBLE
------------------------------------------------------------ Letter :5

Credible performance information is essential for accurately
assessing agencies' progress towards the achievement of their goals
and, in cases where goals are not met, identifying opportunities for
improvement or whether goals need to be adjusted.  Under the Results
Act, agencies' annual performance plans are to describe the means
that will be used to verify and validate performance data.  However,
the majority of the plans we reviewed provided only limited
confidence that performance information would be credible. 
Specifically, although most of the plans describe procedures for
verifying and validating performance information, these plans lack
specific details on the actual procedures the agencies will use.  In
addition, few plans include a discussion of the known limitations in
the agencies' existing data systems.  Of those plans that did include
such a discussion, almost none discussed the agency's strategies to
address the known limitations. 


      VERIFYING AND VALIDATING
      PERFORMANCE
---------------------------------------------------------- Letter :5.1

In our report on agencies' September 30 strategic plans,\14 we noted
that many agencies had long-standing and serious shortcomings in
their ability to generate reliable and timely performance data.  We
suggested that the annual performance plans provided an opportunity
to articulate how these shortcomings will be addressed.  Performance
plans can help do this by including a discussion intended to provide
confidence that the means agencies will use to verify and validate
performance information, such as audits, program evaluations,
independent external reviews, and internal controls, will yield
performance data of sufficient quality to support decisionmaking. 

Some performance plans, such as NASA's and SSA's, did not appear to
describe any verification and validation procedures that these
agencies used or expected to use to ensure that performance data are
sufficiently complete, accurate, and consistent.  Most of the plans
provided descriptions of procedures for verifying and validating
performance data, but these descriptions were often superficial.  For
example, although SBA's plan included a general discussion of
verification and validation procedures, it did not specify how the
agency would ensure that its performance data are credible.  Specific
verification and validation systems, related measures, and the
milestones for verification and validation generally are not cited in
the plans.  The Nuclear Regulatory Commission's (NRC) plan discussed
validating the list of primary systems and measuring levels of
satisfaction with the accuracy and availability of information in the
systems, but it did not discuss how NRC intends to actually validate
these data to ensure that they are accurate and complete. 

The performance plans of some other agencies, such as Labor,
mentioned only that their Inspectors General (IGs) would be
responsible for auditing the agencies' data systems.  Agencies that
expected to use the IGs generally suggested that the review of
performance data would be done as part of the annual financial audit
of the agency.  The IGs and external auditors can make important
contributions toward ensuring that performance data are valid, but
these contributions cannot substitute for management actions to
ensure that the data are sound.  Moreover, these plans generally did
not discuss whether agencies had coordinated with their IGs to
perform this work.  Agencies and IGs need to jointly determine how
the IGs' resources could best support verification and validation
efforts, given the IGs' continuing audit responsibilities.  In making
this determination, agencies need to carefully consider the most
appropriate means for verifying and validating performance
information. 

Education and HHS' Indian Health Service (IHS) proved to be
exceptions regarding the verification and validation of performance
data.  Their performance plans included a variety of specific and
credible procedures to ensure that Congress, agency managers, and
other decisionmakers will have performance information of sufficient
credibility to support decisions.  For example, Education's plan
included such procedures as a mix of audits, independent external
reviews, and program evaluations, as well as the scope and timing of
what its IG would be undertaking.  IHS' plan included such procedures
as performing editing checks, monitoring the reasonableness of data,
and developing software to allow for the transmission of data to a
centralized database. 

Separate from the issues associated with the need to ensure that
performance data are verified and valid, we continue to be concerned
about the lack of a capacity in many federal agencies to undertake
the program evaluations that will be vital to the success of the
Results Act.  In reviewing agencies' strategic plans, we found that
many agencies had not given sufficient attention to how program
evaluation will be used in implementing the Results Act and improving
performance.  More recently, we reported that agencies' program
evaluation capabilities would be challenged to meet the new demands
for information on program results.\15 We found that the resources
allocated to conducting program evaluations were small and unevenly
distributed across the 13 departments and 10 independent agencies we
surveyed for that report. 

The findings of that report are a major concern because a federal
environment that focuses on results--where federal efforts are often
but one factor among many that determine whether and if goals are
achieved--depends on program evaluations to provide vital information
about the contribution of federal efforts.  For example, the success
of the United States Agency for International Development (USAID) in
achieving its intended results is affected by many factors and
programs beyond USAID's control.  Development programs of the
international donor community and the governments and institutions
within the developing countries themselves all can have greater or
lesser influences on advancing social and economic development. 
USAID, as well as other agencies, can use program evaluations to help
isolate the degree to which its efforts are contributing to results
and what actions it specifically can take to better meet its goals. 


--------------------
\14 GAO/GGD-98-44, January 30, 1998. 

\15 GAO/GGD-98-53, April 24, 1998. 


      RECOGNIZING DATA LIMITATIONS
---------------------------------------------------------- Letter :5.2

In general, agencies' annual performance plans did not include
discussions of known data limitations and strategies to address them. 
Such limitations can be a significant challenge to performance
measurement.  Over the years, we and others have identified problems
with the financial and information systems at several agencies.  The
recent governmentwide financial statement audit further raised
concerns about the reliability of data.\16 The amount of progress
still needed to obtain high-quality financial data suggests the types
of challenges that agencies will face in obtaining high-quality
performance data. 

Agencies face particular challenges when they must rely on other
organizations to provide important performance information.  For
example, State's performance plan acknowledges that the agency is to
rely on data from external sources to measure performance.  However,
the plan did not describe how limitations in the quality of that data
would affect efforts to assess and improve performance.  These data
limitations included inconsistencies in data collection from location
to location; from year to year; or from one data source to another,
especially when data from more than one source must be combined to
measure performance.  The plan would be more useful if it recognized
and identified significant data limitations and their implications
for assessing performance. 

Education's performance plan contained a good example of an agency's
recognition of data limitations.  In the quality of performance data
section of its plan, Education stated that ensuring the accurate and
efficient collection of its student loan data is vital to achieving
one of its strategic goals.  However, we and Education's IG have
previously reported about the inadequacy of Education's student loan
data.\17 Because of these inadequacies, Education had been unable to
report on the Department's financial position in a complete,
accurate, and reliable manner.  In its performance plan, Education
acknowledged that its student aid delivery system has suffered from
significant data quality problems.  The plan outlined several steps
the Department plans to take to address these problems, including
improving data accuracy by establishing industrywide standards for
data exchanges, receiving individual student loan data directly from
lenders, expanding efforts to verify data reported to the National
Student Loan Data System, and preparing a systems architecture for
the delivery of federal student aid. 


--------------------
\16 Financial Audit:  1997 Consolidated Financial Statements of the
United States Government (GAO/AIMD-98-127, Mar.  31, 1998). 

\17 Department of Education:  Status of Actions to Improve the
Management of Student Financial Aid (GAO/HEHS-96-143, July 12, 1996);
and Student Financial Aid Information:  Systems Architecture Needed
to Improve Programs' Efficiency (GAO/AIMD-97-122, July 29, 1997). 


   OPPORTUNITIES TO IMPROVE FUTURE
   PLANS
------------------------------------------------------------ Letter :6

The fiscal year 1999 annual performance plans represent the first
attempt across all executive agencies to carry out the annual
performance planning called for under the Results Act.  Although
these plans collectively suggested that annual performance planning,
as established under the Act, can be a powerful device for better
informing congressional and executive branch decisionmaking,
substantial further development is needed before the plans will
consistently be able to support that goal.  In crafting the Results
Act, Congress understood--and similar foreign experiences
confirmed--that effectively implementing management changes of the
magnitude envisioned under the Act would take several cycles,
although each cycle should see marked improvements over the preceding
ones. 

Both OMB's fiscal year 1999 annual performance plan and the fiscal
year 1999 governmentwide performance plan contain commitments to
implement the Results Act.  The OMB annual performance plan includes
a goal to improve the performance of government programs by meeting
the statutory requirements of the Act.  The governmentwide
performance plan includes Results Act implementation as 1 of 22
priority management objectives that the administration will focus on
in 1999.  According to a senior OMB official, OMB incorporates
lessons learned from performance plans in the guidance it provides
agencies on developing future plans.  OMB has also committed to
reviewing agencies' subsequent plans to ensure that improvements and
appropriate changes are made.  On the basis of our review of the
agency plans, it appears that OMB can build on its commitment and
design and implement a broad, aggressive performance planning
improvement effort.  Specifically, our work suggests that giving
priority attention to the following key opportunities for improvement
will lead to the greatest increases in usefulness: 

  -- Better articulating a results-orientation.  Agency performance
     plans could be more useful if they more consistently
     incorporated results-oriented goals and showed more direct
     relationships among goals and measures.  More results-oriented
     agency goals will also facilitate understanding the
     relationships among agencies' efforts and planned contributions
     and goals included in the governmentwide performance plan.  The
     value of agencies' annual performance plans also could increase
     if the plans consistently included useful and informative
     baseline and trend data, as some agencies did in their 1999
     performance plans.  Such data provide decisionmakers with a
     context for assessing whether performance targets are
     appropriate and reasonable. 

The value of the performance plans could also be augmented if they
more fully included goals that addressed mission-critical management
issues (for example, historic problems in maximizing the use of
information technology).  Precise and measurable goals for resolving
mission-critical management problems are important to ensuring that
the agencies have the institutional capacity to achieve their more
results-oriented programmatic goals.  Consistently including goals in
individual agency plans to address mission-critical management issues
also will facilitate the integration of governmentwide and agency
performance planning processes.  Section IV of the governmentwide
annual performance plan is devoted to improving performance through
better management and lists the administration's priority management
objectives.  We found in our review of the governmentwide plan that
the clarity and effectiveness of OMB's discussion of the objectives
in that plan could be improved by a more integrated and focused
discussion of the strategies associated with the objectives.\18

Augmented agency performance plans can be helpful in this regard by
showing that agencies, where appropriate, are positioned to address
governmentwide priority management objectives, such as the Year 2000
computing crisis.  To facilitate this effort, we recommend in our
review of the governmentwide performance plan that OMB ensure that
the governmentwide management priorities and performance goals
contained in the governmentwide performance plan be reflected in
relevant agency performance plans. 

  -- Coordinating crosscutting programs.  Our work has suggested that
     program overlap and mission fragmentation are important issues
     that need to be addressed.  We also have noted, consistent with
     OMB's guidance, that a focus on results implies that
     crosscutting programs will be coordinated.  At the time of our
     reviews, many agencies' annual performance plans identified
     crosscutting efforts, with some listing other agencies with
     which they shared the same or similar result, but the
     substantive work of coordination was not yet apparent. 
     Specifically, few of the plans showed evidence of the work
     necessary to ensure that crosscutting programs have mutually
     reinforcing goals; complementary strategies; and, as
     appropriate, common performance measures.  Not surprisingly,
     given the amount of coordination that still needs to take place,
     in our review of the fiscal year 1999 governmentwide performance
     plan we found that substantial opportunities exist for enhancing
     the discussion of crosscutting efforts in that plan as well. 

By building on the initial progress that some agencies have made, the
usefulness of performance plans could be enhanced if all agencies
more consistently identified the results-oriented annual performance
goals that involve other federal agencies and set intermediate goals
that clarify the agency's specific contribution to the common result. 
Moreover, because of the still early state of coordination of
crosscutting programs, the more useful plans will continue to
describe relevant interagency coordination efforts. 

  -- Clearly showing how strategies will be used to achieve goals. 
     Although not explicitly required by the Results Act, the more
     useful annual performance plans discussed how the strategies and
     approaches would lead to results.  The listings of current
     programs and initiatives that often were included in agencies'
     plans are useful in providing an understanding of what agencies
     do.  Presentations that more directly explain how programs and
     initiatives achieve goals will be most helpful to Congress as it
     assesses the degree to which strategies are appropriate and
     reasonable.  Discussions of external factors and how different
     governing tools (for example, intergovernmental partnerships,
     performance-based contracts, financial credits) will be, or can
     be, used in achieving goals could further enhance the plans. 
     Such discussions could also assist in the development of a base
     of governmentwide information on the strengths and weaknesses of
     various tools in addressing differing public policy issues. 

  -- Showing performance consequences of budget decisions.  The
     Results Act was intended to help Congress develop a clearer
     understanding of what is being achieved in relation to the money
     being spent.  In the fiscal year 1999 performance plans,
     agencies generally covered all of the program activities in
     their budget requests.  However, most plans did not clearly
     convey the requested funding level associated with achieving a
     discrete set of performance goals and clearly identify where
     that funding was included in the structure of agencies' budget
     requests.  Agencies, OMB, and Congress can take advantage of
     three initiatives--budget and program activity changes, the
     implementation of cost accounting, and the initiation of
     performance budgeting pilots--to help ensure that performance
     plans better convey the performance consequences of budget
     decisions. 

Congress and OMB have clearly expressed a willingness to consider
changes in agencies' budget account and/or program activity
structures in future years to more clearly and readily relate
expected performance to funding requests.  For example, in its fiscal
year 1998 appropriations reports, the House Appropriations Committee
stated that it would consider any requests for program activity
changes that ensure that budget submissions display amounts requested
against program activity structures for which annual performance
goals and measures have been established.  Similarly, OMB's Circular
A-11 encouraged agencies to consider proposing changes to the budget
account structure to facilitate an understanding of performance. 

In addition to more closely linking expected performance to agency
budget requests, Congress, in crafting the Results Act expected that
agencies, whenever possible, would develop performance measures that
correlated the level of program activity with program costs, such as
costs per unit of result, costs per unit of service, or costs per
unit of output.  Agencies were expected to assign a high priority to
developing these types of unit cost measures.  The successful
implementation of the managerial cost accounting standards\19
recommended by the Federal Accounting Standards Advisory Board and
issued by OMB and GAO are vital to providing agencies the program
cost information needed to develop such performance measures. 

The Results Act also demonstrates Congress' interest in determining
the extent to which performance can be related to changes in funding
levels.  The Act requires the Director of OMB to designate at least
five federal agencies to participate in a 2-year pilot in performance
budgeting in which the budgets of those agencies will display varying
levels of performance that would result from different budgeted
amounts for one or more of an agency's major functions or
operations.\20 Agencies' progress in establishing reliable cost
accounting systems and allocating resources to performance goals,\21

as well as progress in defining goals and measuring performance, may
affect how OMB designs and determines participation in the
performance budgeting pilots.  More broadly, as agencies continue to
define relationships between performance planning and budget
structures, Congress, OMB, and agencies can explore whether changes
in budget presentations can provide agencies with needed flexibility
and accountability while ensuring appropriate congressional oversight
and control. 

  -- Building capacity within agencies to gather and use performance
     information.  Our work suggests that few agencies have adequate
     procedures in place to ensure that the performance data
     generated will be of sufficient quality to confidently make
     decisions.  The financial audits under the CFO Act--where only
     10 of the 24 CFO Act agencies have been able to obtain an
     unqualified opinion from independent auditors--have shown how
     far most agencies have to go to be able to generate reliable
     year-end financial information.  It is important that agencies
     continue to make progress on developing financial systems that
     can produce timely financial information throughout the year and
     work with their IGs to explore how the IGs can contribute to
     improving the credibility of performance data.  Moreover, our
     recent work continues to show that many agencies are not
     well-positioned to undertake the program evaluations that will
     be critical to identifying why goals are not met and determining
     the best improvement strategies.\22 The relatively limited level
     of agencies' evaluation capabilities suggests that evaluation
     resources will need to be carefully targeted and coordinated
     with nonfederal evaluation efforts to ensure that key questions
     about program results are adequately addressed. 

In our Executive Guide, we noted that leading results-oriented
organizations consistently strive to ensure that their day-to-day
activities support their organizational missions and move them closer
to accomplishing their strategic goals.\23 We reported that in
practice, these organizations see the production of a strategic
plan--that is, a particular document issued on a particular day--as
one of the least important parts of the planning process.  Annual
performance plans should be viewed the same way.  The performance
improvements expected under the Results Act will not occur because an
agency has issued strategic and annual performance plans.  Rather,
performance improvements occur when agency managers and external
decisionmakers use those documents and the planning and management
processes that underpin them. 

Because fiscal year 1999 is to mark the first year of governmentwide
implementation of the Results Act's annual performance planning
requirements, Congress and the agencies lack a common base of
experience for how the performance-based approach to management
envisioned under the Results Act can best be used to support
congressional and executive branch decisionmaking.  Building this
base of experience will require ensuring that performance-based
management is integrated into the way programs are managed and
decisions are made.  The importance of building this base of
experience also suggests that any successful effort to improve the
usefulness of agency performance plans will require the active
partnership of Congress, OMB, and the agencies because of the
potentially broad use of such plans both within Congress and the
executive branch. 


--------------------
\18 For a discussion of this section of the governmentwide plan see
GAO/AIMD/GGD-98-159, Sept.  8, 1998, p.  15. 

\19 Effective for fiscal year 1998, these standards are to provide
decisionmakers with information on the costs of all resources used
and the costs of services provided by others to support activities or
programs, thus allowing comparisons of the costs of various programs
and activities with their performance outputs and results. 

\20 OMB plans to solicit agencies this year to participate in these
pilots, and the Act makes OMB responsible for reporting and assessing
the pilots' results and making recommendations on whether the
agencies' techniques should be applied governmentwide. 

\21 In the case of crosscutting federal activities, this means
allocating resources both within and across agencies. 

\22 GAO/GGD-98-53, April 24, 1998. 

\23 GAO/GGD-96-118, June 1996. 


   CONCLUSIONS
------------------------------------------------------------ Letter :7

We found that on the whole, future annual performance plans would be
more useful if they provide clearer pictures of intended performance
across an agency, more fully articulate what strategies and resources
will be used to achieve goals and how those strategies and resources
will lead to improved performance, and provide much greater
confidence that performance information will be credible and useful
for decisionmaking.  OMB's efforts to identify lessons learned from
the agencies' fiscal year 1999 performance plans and its commitment
in the fiscal year 1999 governmentwide performance plan to review
agencies' subsequent plans to ensure improvements are made are a
first step.  However, the need for progress across all agencies and
the range of annual planning issues that need to be addressed
underscore the scope of the effort that lies ahead and suggest that a
more concerted, active, and specific improvement agenda needs to be
developed and put in place. 

Beyond improving the quality of agencies' written plans, experience
is needed in using those plans to inform congressional and executive
branch decisionmaking.  In this regard, it is vital that all agencies
begin implementing their fiscal year 1999 annual performance plans in
October 1998 and seek to prepare improved plans for fiscal year 2000. 
However, the limited experience with the use of the Results Act at
the federal level thus far suggests that targeting key program areas
for special congressional and executive branch attention can help
agencies develop a common base of experience in using Results Act
principles and processes to drive performance and management
improvements.  A coordinated OMB, congressional, and agency effort
could be helpful in three ways: 

  -- First, the effort could develop a body of specific examples that
     demonstrate where congressional and executive branch use of the
     performance-based approach to management and accountability
     contained in the Results Act helped to inform decisionmaking. 
     Initially, the evidence for this use will be seen in such areas
     as improved and more focused program management within agencies;
     more informed executive branch and congressional budget
     decisions, including the better alignment of performance
     planning and budgeting processes; and better information
     available and used as part of congressional authorization and
     oversight efforts.  Most importantly, over time, the use of
     performance plans should lead to substantial improvements in
     program performance. 

  -- Second, a common OMB, congressional, and agency focus on
     selected program areas also will aid in the development of a set
     of agreed-upon "best practices" in performance planning and the
     integration of results-oriented performance information into
     decisionmaking and management.  Including programs that
     represent a cross-section of service delivery mechanisms will
     provide insights into how the various tools of government (for
     example, regulation, direct service, intergovernmental funding,
     tax expenditures, loans, or loan guarantees) can be used
     individually and together to address public policy issues.  This
     will aid in building an understanding of how implementation of
     the Results Act may differ--such as in the nature of federal
     goals--depending on the specific characteristics of different
     tools. 

  -- Third, a common focus on using the Results Act to make decisions
     for selected program areas can also assist in the identification
     of experienced-based similarities and differences in the
     congressional and executive branch needs for the content of
     annual performance plans and any changes to OMB guidance and
     Results Act statutory requirements that may be necessary to
     better meet those needs. 


   RECOMMENDATION TO THE DIRECTOR
   OF OMB
------------------------------------------------------------ Letter :8

To fully implement OMB's commitment to evaluate its and agencies'
experience in developing the fiscal year 1999 performance plans and
to improve agencies' performance plans for the future, we recommend
that the Director, OMB, implement a concrete agenda aimed at
substantially enhancing the usefulness of agencies' performance plans
for congressional and executive branch decisionmaking.  The five key
opportunities for improvement that we identified--better articulating
a results orientation; coordinating crosscutting programs; clearly
showing how strategies will be used to achieve goals; showing
performance consequences of budget decisions; and building capacity
within agencies to gather and use performance information, including
program evaluation--can serve as core elements of the improvement
effort.  For example, OMB could work with agencies to ensure that
annual performance plans include presentations that more directly
explain how agency programs and initiatives will be achieved. 
Similarly, discussions of external factors and how different
governing tools (e.g., intergovernmental partnerships,
performance-based contracts, financial credits) will be, or can be,
used in achieving goals would help enhance the usefulness of
agencies' plans. 

To go beyond the formal requirements of the Results Act to issue
annual performance plans and performance reports, and to build a base
of experience for how the performance-based approach to management
envisioned under the Results Act can be used to improve program
results and support congressional and executive branch
decisionmaking, we also recommend that the Director of OMB work with
Congress and the agencies to identify specific program areas that can
be used as best practices.  This would help demonstrate the use and
benefits of results-oriented management and where concrete
information about program results contributes directly to executive
branch and congressional decisionmaking.  This effort could also
assist Congress in identifying and considering opportunities to merge
and expand results-oriented performance information into existing
authorization, oversight, and appropriations processes.  For the
effort to be most effective, several criteria should be used to
identify specific program areas, such as those program areas where
agreement exists between Congress and the administration that the
areas likely will be on legislative and oversight agendas; those
programs that have the most direct influence on meeting the central
Results Act purpose of improving citizens' confidence in government;
and programs needing priority management attention, such as those
listed in the fiscal year 1999 governmentwide performance plan. 


   AGENCY COMMENTS AND OUR
   EVALUATION
------------------------------------------------------------ Letter :9

On July 17, 1998, we provided a draft of this report to the Director
of OMB for comment.  We did not provide a draft to individual
agencies discussed in this report because the reports we prepared on
individual agency plans in response to your request were provided to
the relevant agencies for comment.  Those comments were reflected, as
appropriate, in the final versions of those reports. 

On August 19, 1998, we received OMB's written comments; see appendix
IV for a copy of the letter from the Acting Deputy Director for
Management.  On August 20, 1998, we received additional technical
comments from a senior OMB official; we have incorporated these
comments where appropriate.  OMB's August 19, 1998, letter includes
comments on both this report and our companion report on the fiscal
year 1999 governmentwide performance plans.\24 Our evaluation of OMB
comments on this report are provided below; OMB's comments on our
assessment of the governmentwide performance plan are discussed in
our companion report. 

OMB generally agreed with our observations and said that the report
was an expansive portrait of the fiscal year 1999 plans that
contained many useful suggestions.  OMB did, however, raise two
related issues about the report.  OMB commented that the report
predominantly focuses on what was included or lacking in the annual
performance plans rather than on how the plans would be used to
provide better services and products to the American public and to
improve the quality and nature of programming, funding, and
management decisions made within the executive branch and by
Congress.  OMB also commented that the report does not clearly
distinguish between major and secondary elements of an annual plan,
such as between factors that are and are not required by statute. 

We agree with OMB that the use of annual performance plans by
congressional and executive branch decisionmakers is the essential
indicator of the effective implementation of the Results Act.  As we
said in our report, performance improvements will not occur because
an agency issues a plan, but rather when plans--and the planning and
management processes that underpin them--are used by agency managers
and other decisionmakers.  However, as our report shows, significant
improvements are needed in agencies' plans before they will be useful
in a significant way to congressional and executive branch
decisionmakers.  Thus, we reported on the presence or absence of
characteristics that affect the usefulness of annual performance
plans and identified the key opportunities for improving those plans. 
Similarly, because we focused our review on the elements that are
most important to developing useful plans, we disagree with OMB's
comment that we did not distinguish between major and secondary
elements of performance plans.  Moreover, as we note in the report,
the major elements of our evaluation--goals and performance measures,
program strategies, and the existence of valid performance data--are
based on specific criteria set forth in the Results Act, the Act's
legislative history, or OMB's guidance to agencies on preparing
performance plans. 


--------------------
\24 GAO/AIMD/GGD-98-159, September 8, 1998. 


---------------------------------------------------------- Letter :9.1

We are sending copies of this report to the Minority Leader of the
House; the Ranking Minority Members of your Committees; Committee
Chairmen who requested our review of the fiscal year 1999
governmentwide annual plan and the Ranking Minority Members of their
respective Committees; other appropriate congressional committees;
and the Director, Office of Management and Budget.  We will also make
copies available to others on request. 

The major contributors to this report are listed in appendix V. 
Please contact L.  Nye Stevens on (202) 512-8676 or Paul L.  Posner
on (202) 512-9573 if you or your staff have any questions. 

L.  Nye Stevens
Director, Federal Management and
Workforce Issues
General Government Division

Paul L.  Posner
Director, Budget Issues
Accounting and Information Management
Division


AGENCIES COVERED BY THE CHIEF
FINANCIAL OFFICERS ACT
=========================================================== Appendix I

Department of Agriculture
Department of Commerce
Department of Defense
Department of Education
Department of Energy
Department of Health and Human Services
Department of Housing and Urban Development
Department of the Interior
Department of Justice
Department of Labor
Department of State
Department of Transportation
Department of the Treasury
Department of Veterans Affairs
Environmental Protection Agency
Federal Emergency Management Agency
General Services Administration
National Aeronautics and Space Administration
National Science Foundation
Nuclear Regulatory Commission
Office of Personnel Management
Small Business Administration
Social Security Administration
U.S.  Agency for International Development


SCOPE AND METHODOLOGY
========================================================== Appendix II

To summarize our observations on agencies' fiscal year 1999 annual
performance plans and to help us identify opportunities for agencies
to improve future performance plans, we analyzed the information
contained in our reviews of the annual performance plans of the 24
CFO Act agencies.  Our review of each of the agencies' performance
plans and our summary analysis of all 24 plans were based on our
guides for congressional and evaluator review of annual performance
plans.  For purposes of assessing the plans, we collapsed the Results
Act's requirements\25 for annual performance plans into the three
core questions that structure those guides.  The three questions are: 

  -- To what extent does the agency's performance plan provide a
     clear picture of intended performance across the agency? 

  -- How well does the performance plan discuss the strategies and
     resources the agency will use to achieve its performance goals? 

  -- To what extent does the agency's performance plan provide
     confidence that its performance information will be credible? 

We used the questions and associated issues contained in the guides
to help us identify strengths and weaknesses in the performance
plans, with a particular focus on assessing the overall usefulness of
the plans for congressional and other decisionmakers.  In doing our
summary analysis, we examined and classified our reviews of the
individual agency plans as related to the questions, issues, and
criteria in the guides to discern any themes or trends and then to
develop an overall, descriptive characterization of our observations
and judgments about the agencies' plans.  We also reviewed parts of
selected agencies' annual performance plans, as needed, to supplement
our analysis of our individual agency reviews and to elaborate
further on particular issues.  To further help us identify
opportunities for agencies to improve future performance plans, we
also drew on other related work, including our recent reports on
Results Act implementation.\26

We reviewed agency performance plans from February through June 1998
and did our work according to generally accepted government auditing
standards.  We requested comments from the Director of OMB on a draft
of this report.  OMB's comments are discussed in the "Agency Comments
and Our Evaluation" section in this report.  In addition, we provided
drafts of our individual reviews on agencies' plans to the relevant
agencies for comment.  The agencies' comments are reflected, as
appropriate, in our products on their respective plans. 


--------------------
\25 The Results Act requires annual performance plans to (1)
establish performance goals to define the level of performance to be
achieved by a program activity; (2) express those goals in an
objective, quantifiable, and measurable form unless an alternative
form is authorized; (3) briefly describe the operational processes,
skills and technology, and the human, capital, information, or other
resources required to meet the performance goals; (4) establish
performance measures to be used in assessing the progress the agency
is making in achieving the established performance goals; (5) provide
a basis for comparing actual program results with the established
performance goals; and (6) describe the means to be used to verify
and validate measured values. 

\26 GAO/GGD-97-109, June 2, 1997; and GAO/GGD-98-44, January 30,
1998. 


GAO PRODUCTS ON AGENCIES' FISCAL
YEAR 1999 ANNUAL PERFORMANCE PLANS
========================================================= Appendix III


   DEPARTMENTS
------------------------------------------------------- Appendix III:1

Results Act:  Observations on the U.S.  Department of Agriculture's
Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98R, June 11,
1998). 

Results Act:  Observations on the Department of Commerce's Annual
Performance Plan for Fiscal Year 1999 (GAO/GGD-98-135R, June 24,
1998). 

Results Act:  DOD's Annual Performance Plan for Fiscal Year 1999
(GAO/NSIAD-98-188R, June 5, 1998). 

The Results Act:  Observations on the Department of Education's
Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-172R, June 8,
1998). 

Results Act:  Observations on DOE's Annual Performance Plan for
Fiscal Year 1999 (GAO/RCED-98-194R, May 28, 1998). 

The Results Act:  Observations on the Department of Health and Human
Services' Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-180R,
June 17, 1998). 

Results Act:  Observations on the Department of Housing and Urban
Development's Fiscal Year 1999 Annual Performance Plan
(GAO/RCED-98-159R, June 5, 1998). 

Results Act:  Department of the Interior's Annual Performance Plan
for Fiscal Year 1999 (GAO/RCED-98-206R, May 28, 1998). 

Observations on the Department of Justice's Fiscal Year 1999
Performance Plan (GAO/GGD-98-134R, May 29, 1998). 

Results Act:  Observations on Labor's Fiscal Year 1999 Performance
Plan (GAO/HEHS-98-175R, June 4, 1998). 

The Results Act:  Observations on the Department of State's Fiscal
Year 1999 Annual Performance Plan (GAO/NSIAD-98-210R, June 17, 1998). 

Results Act:  Observations on the Department of Transportation's
Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-180R, May
12, 1998). 

Results Act:  Observations on Treasury's Fiscal Year 1999 Annual
Performance Plan (GAO/GGD-98-149, June 30, 1998). 

Results Act:  Observations on VA's Fiscal Year 1999 Performance Plan
(GAO/HEHS-98-181R, June 10, 1998). 


   INDEPENDENT AGENCIES
------------------------------------------------------- Appendix III:2

Results Act:  EPA's Annual Performance Plan for Fiscal Year 1999
(GAO/RCED-98-166R, Apr.  28, 1998). 

Results Act:  Observations on the Federal Emergency Management
Agency's Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-207R,
June 1, 1998). 

Results Act:  Observations on the General Services Administration's
Annual Performance Plan (GAO/GGD-98-110, May 11, 1998). 

Managing for Results:  Observations on NASA's Fiscal Year 1999
Performance Plan (GAO/NSIAD-98-181, June 5, 1998). 

Results Act:  NSF's Annual Performance Plan for Fiscal Year 1999
(GAO/RCED-98-192R, May 19, 1998). 

Results Act:  NRC's Annual Performance Plan for Fiscal Year 1999
(GAO/RCED-98-195R, May 27, 1998). 

Results Act:  Observations on the Office of Personnel Management's
Annual Performance Plan (GAO/GGD-98-130, July 28, 1998). 

Results Act:  Observations on the Small Business Administration's
Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-200R, May 28,
1998). 

The Results Act:  Observations on the Social Security
Administration's Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R,
June 9, 1998). 

The Results Act:  Observations on USAID's Fiscal Year 1999 Annual
Performance Plan (GAO/NSIAD-98-194R, June 25, 1998). 




(See figure in printed edition.)Appendix IV
COMMENTS FROM THE OFFICE OF
MANAGEMENT AND BUDGET
========================================================= Appendix III



(See figure in printed edition.)



(See figure in printed edition.)


MAJOR CONTRIBUTORS
=========================================================== Appendix V

GENERAL GOVERNMENT DIVISION

J.  Christopher Mihm, Associate Director, Federal Management and
Workforce Issues, (202) 512-8676
Joseph S.  Wholey, Senior Advisor for Evaluation Methodology
Alan M.  Stapleton, Assistant Director
Joyce D.  Corry, Evaluator-in-Charge
Thomas M.  Beall, Technical Advisor
Allen C.  Lomax, Senior Evaluator
Kiki Theodoropoulos, Senior Evaluator

ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION

Michael J.  Curro, Assistant Director
Laura E.  Castro, Senior Evaluator

ACKNOWLEDGEMENTS

The examples used in this report are drawn from the assessments of
the individual agency annual performance plans that were done by
staff across GAO.  Thus, in addition to the individuals noted above,
the staff who worked on the individual agency plan assessments also
made important contributions to this report.  These individuals are
identified in the separate reports on agency plans. 


*** End of document. ***