Agency Performance Plans: Examples of Practices That Can Improve
Usefulness to Decisionmakers (Letter Report, 02/26/99,
GAO/GGD/AIMD-99-69).

Pursuant to a congressional request, GAO: (1) identified and described
practices that were applied in some federal agencies' 1999 performance
plans that might, if consistently applied, improve the usefulness of all
agencies' annual performance plans; and (2) provided examples from
agencies' fiscal year 1999 performance plans that illustrate each
practice.

GAO noted that: (1) opportunities to improve the usefulness of annual
plans include: (a) creating a set of performance goals and measures that
addresses important dimensions of program performance and balances
competing priorities; (b) using intermediate goals and measure to show
progress or contribution to intended results; (c) including explanatory
information on the goals and measures; (d) developing performance goals
to address mission-critical management problems; (e) showing baseline
and trend data for past performance; (f) identifying projected target
levels of performance for multiyear goals; and (g) linking the goals of
component organizations to departmental strategic goals; (2) to
coordinate crosscutting programs, agencies should: (a) identify programs
that contribute to the same or similar results; (b) set complementary
performance goals to show how differing program strategies are mutually
reinforcing and establishing common or complementary performance
measures, as appropriate; and (c) describe--briefly or refer to a
separate document--planned coordination strategies; (3) to show how
strategies will be used to achieve goals, agencies should: (a) link
strategies and programs to specific performance goals and describe how
they will contribute to the achievement of those goals; (b) describe
strategies to leverage or mitigate the effects of external factors on
the accomplishment of performance goals; (c) discuss strategies to
resolve mission-critical management problems; and (d) discuss--briefly
or refer to a separate document--plans to ensure that mission-critical
processes and information systems function properly and are secures; (4)
to show performance consequences of budget and other resources
decisions, agencies should: (a) show how budgetary resources relate to
the achievement of performance goals; (b) discuss--briefly and refer to
the agency capital plan--how proposed capital assets (specifically
information technology investments will contribute to achieving
performance goals; and (c) discuss--briefly or refer to a separate
plan--how the agency will use its human capital to help achieve
performance goals; and (5) to build the capacity to gather and use
performance information, agencies should identify internal and external
sources for data.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GGD/AIMD-99-69
     TITLE:  Agency Performance Plans: Examples of Practices That Can 
             Improve Usefulness to Decisionmakers
      DATE:  02/26/99
   SUBJECT:  Reporting requirements
             Program evaluation
             Performance measures
             Strategic planning
             Private sector practices
             Interagency relations
             Data integrity
IDENTIFIER:  GPRA
             Government Performance and Results Act
             Pell Grant
             Federal Family Education Loan Program
             William D. Ford Federal Direct Loan Program
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Chairman, Committee on Governmental Affairs, U.S. 
Senate

February 1999

AGENCY PERFORMANCE PLANS -
EXAMPLES OF PRACTICES THAT CAN
IMPROVE USEFULNESS TO
DECISIONMAKERS

GAO/GGD/AIMD-99-69

GAO/GGD-99-69

Agency Performance Plans

(410311)


Abbreviations
=============================================================== ABBREV

  CFOAct - Chief Financial Officers Act
  GPRA - Government Performance and Results Act
  OMB - Office of Management and Budget
  ONDCP - Office of National Drug Control Policy
  OPM - Office of Personnel Management

Letter
=============================================================== LETTER


B-279902

February 26, 1999

The Honorable Fred Thompson
Chairman, Committee on Governmental Affairs
United States Senate

Dear Mr.  Chairman: 

The Government Performance and Results Act of 1993 seeks to improve
the effectiveness, efficiency, and accountability of federal programs
by establishing a system for agencies to set goals for program
performance and to measure results.  Under the Act, agencies are to
prepare annual performance plans to systematically provide
decisionmakers with information on the results to be achieved for a
proposed level of resources.  These plans are to reinforce the
connection between the long-term strategic goals outlined in
agencies' strategic plans and the day-to-day activities of their
program managers and staff.  In so doing, the annual performance
plans provide a basis for establishing accountability for results. 

Fiscal year 1999 marks the first time agencies across the federal
government were to prepare annual performance plans covering each
program activity in their budgets.\1 As requested by several members
of the congressional leadership, we reviewed the performance plans of
the 24 agencies covered by the Chief Financial Officers (CFO) Act. 
After briefing congressional staff, we issued separate reports on
each of the 24 agencies' fiscal year 1999 performance plans.\2 In
September 1998, we issued a report that summarized our observations
on the agencies' plans and noted that substantial further development
is needed for these plans to be useful in a significant way to
congressional and other decisionmakers.\3 That report identified
opportunities to improve the usefulness of future performance plans. 
These opportunities include (1) better articulating a results
orientation, (2) coordinating crosscutting programs, (3) clearly
showing how strategies will be used to achieve goals, (4) showing
performance consequences of budget decisions, and (5) building
capacity within agencies to gather and use performance information. 

As you requested, this report builds on those improvement
opportunities by (1) identifying and describing practices that were
applied in some plans that might, if consistently applied, improve
the usefulness of all agencies' annual performance plans; and (2)
providing examples from agencies' fiscal year 1999 performance plans
that illustrate each practice.  A forthcoming report that you also
requested will provide additional information and examples related to
one of those practices--showing the relationship of budgetary
resources to performance goals. 


--------------------
\1 The term "program activity" refers to the listing of projects and
activities in the appendix portion of the Budget of the United States
Government.  Program activity structures are intended to provide a
meaningful representation of the operations financed by a specific
budget account. 

\2 See appendix I for the list of reports on fiscal year 1999
performance plans. 

\3 Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 


   RESULTS
------------------------------------------------------------ Letter :1

Taken as a whole, some of the fiscal year 1999 annual performance
plans contain practices that, if consistently applied, should help
improve the usefulness of subsequent agency performance plans to
congressional and other decisionmakers.  These practices are shown in
table 1. 



                                     Table 1
                     
                     Summary of Opportunities to Improve the
                      Usefulness of Annual Plans and Applied
                                    Practices

Opportunities to improve
the usefulness of annual
plans                      Applied practices
-------------------------  -----------------------------------------------------
Better articulate a        1. Create a set of performance goals and measures
results orientation.       that addresses important dimensions of program
                           performance and balances competing priorities.

                           2. Use intermediate goals and measures to show
                           progress or contribution to intended results.

                           3. Include explanatory information on the goals and
                           measures.

                           4. Develop performance goals to address mission-
                           critical management problems.

                           5. Show baseline and trend data for past
                           performance.

                           6. Identify projected target levels of performance
                           for multiyear goals.

                           7. Link the goals of component organizations to
                           departmental strategic goals.

Coordinate crosscutting    8. Identify programs that contribute to the same or
programs.                  similar results.

                           9. Set complementary performance goals to show how
                           differing program strategies are mutually reinforcing
                           and establish common or complementary performance
                           measures, as appropriate.

                           10. Describe--briefly or refer to a separate
                           document--planned coordination strategies.

Show how strategies will   11. Link strategies and programs to specific
be used to achieve goals.  performance goals and describe how they will
                           contribute to the achievement of those goals.

                           12. Describe strategies to leverage or mitigate the
                           effects of external factors on the accomplishment of
                           performance goals.

                           13. Discuss strategies to resolve mission-critical
                           management problems.

                           14. Discuss--briefly or refer to a separate document-
                           -plans to ensure that mission-critical processes and
                           information systems function properly and are secure.

Show performance           15. Show how budgetary resources relate to the
consequences of budget     achievement of performance goals.
and other resource
decisions.                 16. Discuss--briefly and refer to the agency capital
                           plan--how proposed capital assets (specifically
                           information technology investments) will contribute
                           to achieving performance goals.

                           17. Discuss--briefly or refer to a separate plan--
                           how the agency will use its human capital to help
                           achieve performance goals.

Build the capacity to      18. Identify internal and external sources for data.
gather and use
performance information.   19. Describe efforts to verify and validate
                           performance data.

                           20. Identify actions to compensate for unavailable or
                           low-quality data.

                           21. Discuss implications of data limitations for
                           assessing performance.
--------------------------------------------------------------------------------
Source:  GAO analysis of agency performance plans. 

The applied practices and examples illustrating how these practices
were incorporated into certain agencies' fiscal year 1999 performance
plans are discussed in the individual sections accompanying this
report. 


   OBJECTIVES, SCOPE, AND
   METHODOLOGY
------------------------------------------------------------ Letter :2

At your request, the objectives of this review were to (1) identify
and describe practices to improve the usefulness of agencies' annual
performance plans and (2) provide examples from agencies' fiscal year
1999 performance plans that illustrate each practice. 

To meet both of our objectives, we contacted selected House and
Senate congressional staff--representing appropriation,
authorization, budget, and oversight committees that reviewed the 24
CFO Act agencies' annual plans--to solicit their views on the
features they found to be particularly useful in the plans.  We also
reviewed our individual and summary reports on agencies' fiscal year
1999 plans; the Results Act requirements for agency annual
performance plans; guidelines contained in the Office of Management
and Budget (OMB) Circular No.  A-11, Part 2; fiscal year 1999
performance plans of the 24 CFO Act agencies; our guides to assist
agencies and Congress to effectively implement the Results Act;\4 and
our work over the last several years examining agencies' efforts to
implement the Results Act.  Appendix II provides references to the
Results Act; OMB Circular No.  A-11, Part 2; and our recently issued
related products and is arranged by the sections discussed in this
report. 

We did our work from March 1998 to January 1999 in Washington, D.C.,
in accordance with generally accepted government auditing standards. 
We did not seek agency comments on this report because much of the
analysis was drawn from our review of the fiscal year 1999
performance plans on which the agencies had already commented. 
However, we asked officials in each of the agencies profiled to
verify the accuracy of the information presented on their respective
agencies' fiscal year 1999 performance plans, and we incorporated
their comments where applicable.  We did not independently verify the
accuracy of the information contained in the agencies' fiscal year
1999 performance plans. 


--------------------
\4 The Results Act:  An Evaluator's Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr.  1998); Agencies'
Annual Performance Plans Under the Results Act:  An Assessment Guide
to Facilitate Congressional Decisionmaking, Version 1
(GAO/GGD/AIMD-10.1.18, Feb.  1998); Agencies' Strategic Plans Under
GPRA:  Key Questions to Facilitate Congressional Review, Version 1
(GAO/GGD-10.1.16, May 1997); and Executive Guide:  Effectively
Implementing the Government Performance and Results Act
(GAO/GGD-96-118, June 1996).  See also GPRA Performance Reports
(GAO/GGD-96-66R, Feb.  14, 1996). 


---------------------------------------------------------- Letter :2.1


We are sending copies of this report to the Ranking Minority Member
of the Senate Committee on Governmental Affairs; the House Majority
Leader; the Chair and Ranking Minority Member of the House Committee
on Government Reform; other interested congressional committees; and
to the Director, Office of Management and Budget.  We will also make
copies available to others on request. 

The major contributors to this report are listed in appendix III.  If
you have any questions concerning this report, please contact me on
(202) 512-8676. 

Sincerely yours,

J.  Christopher Mihm
Associate Director, Federal Management and Workforce Issues


BETTER ARTICULATE A RESULTS
ORIENTATION
============================================================ Chapter 1

Annual performance plans could be more useful documents in helping to
guide decisions and, subsequently, assess actual performance if the
plans consistently showed how the agency was adopting a results
orientation.  Table 1.1 identifies practices that can help agencies
articulate a results orientation. 



                               Table 1.1
                
                 Applied Practices to Address a Results
                              Orientation

----------------------------------  ----------------------------------
A performance plan that better      1. Create a set of performance
articulates a results orientation   goals and measures that addresses
includes the following practices:   important dimensions of program
                                    performance and balances competing
                                    priorities.

                                    2. Use intermediate goals and
                                    measures to show progress or
                                    contribution to intended results.

                                    3. Include explanatory information
                                    on the goals and measures.

                                    4. Develop performance goals to
                                    address mission-critical
                                    management problems.

                                    5. Show baseline and trend data
                                    for past performance.

                                    6. Identify projected target
                                    levels of performance for
                                    multiyear goals.

                                    7. Link the goals of component
                                    organizations to departmental
                                    strategic goals.
----------------------------------------------------------------------
1.  Create a set of performance goals and measures that addresses
important dimensions of program performance and balances competing
priorities. 

Performance goals and measures that successfully address important
and varied aspects of program performance are key aspects of a
results orientation.  To illustrate, the Department of
Transportation's general goal to advance highway safety was included
in its performance plan and was accompanied by various goals and
measures to depict the complex performance they are intended to
assess.  For example, although the department's performance goals and
measures on the highway-related fatality rate and highway-related
injury rate are the primary gauges of safety, they are complemented
by other goals and measures that contribute to those results. 
Specifically, the plan includes a goal to increase the percentage
rate of seat belt usage by front seat occupants, a key element in
reducing overall highway-related fatalities and injuries. 

Federal programs are designed and implemented in dynamic environments
where competing program priorities and stakeholders' needs must be
balanced continuously and new needs must be addressed.\1 As a result,
programs are often forced to strike difficult balances among
priorities that reflect competing demands, such as timeliness,
service quality, customer satisfaction, program cost, and other
stakeholder concerns.  Sets of performance goals and measures could
provide a balanced perspective of the intended performance of a
program's multiple priorities.  In its performance plan, for example,
the Department of Commerce's National Oceanic and Atmospheric
Administration defined its expected performance to improve the
timeliness and accuracy of short-range environmental predictions that
have immediate impact on individuals and many sectors of the economy. 
The relationship between "timeliness" and "accuracy" can represent
competing demands because the accuracy of predictions could be
improved as the weather event comes closer to a vicinity.  However,
the closer the weather event to a vicinity, the shorter and less
timely the warning or lead time to the public in that vicinity.  For
fiscal year 1999, the agency set a lead time of 11 minutes for
tornado warnings with 70-percent accuracy.  The agency also reported
that it set a lead time of 10 minutes with 59-percent accuracy in
fiscal year 1997 and 10 minutes with 65-percent accuracy in fiscal
year 1998.  (See fig.  1.1.)

   Figure 1.1:  Excerpt From the
   Department of Commerce's Fiscal
   Year 1999 Annual Performance
   Plan Featuring the National
   Oceanic and Atmospheric
   Administration

   (See figure in printed
   edition.)

Program cost is another important factor to be considered.  In
crafting the Results Act, Congress expected that agencies, whenever
possible, would develop performance measures that correlate the level
of program activity with program costs, such as costs per unit of
result, costs per unit of service, or costs per unit of output. 
Agencies were expected to assign a high priority to developing these
types of unit cost measures.  The successful implementation of the
managerial cost accounting standards recommended by the Federal
Accounting Standards Advisory Board and issued by OMB and GAO are
vital to providing agencies the program cost information needed to
develop such performance measures.  Effective for fiscal year 1998,
these standards are to provide decisionmakers with information on the
costs of all resources used and the costs of services provided by
others to support activities or programs, thus allowing
decisionmakers to compare the costs of various programs and
activities with their performance outputs and results. 

2.  Use intermediate goals and measures to show progress or
contribution to intended results. 

Intermediate goals and measures, such as outputs or intermediate
outcomes, can be used to show progress or contribution to intended
results.  For instance, when it may take years before an agency sees
the results of its programs, intermediate goals and measures can
provide information on interim results.  Also, when program results
could be influenced by external factors, agencies can use
intermediate goals and measures to identify the programs' discrete
contribution to a specific result. 

As an example, the Department of Health and Human Services' Centers
for Disease Control and Prevention reported in its performance plan
that heart disease is the nation's number one killer among men and
women of all racial and ethnic groups.  However, because of the long
latency period of chronic diseases, such as heart disease, health
outcome measures do not provide a complete picture of the agency's
annual performance.  To address the burden of heart disease, the
Centers for Disease Control and Prevention intends, among other
things, to focus on the prevention of risk factors, such as tobacco
use.  The plan explained that comprehensive state programs have been
shown to be effective in reducing the prevalence of tobacco use.  To
show a nationwide reduction in morbidity and mortality attributable
to behavioral risk factors in tobacco use, the plan presented an
output measure to increase the number of state programs with tobacco
prevention capacities from 17 in 1996 to 30 in 1999.  The Centers for
Disease Control and Prevention intends to fund programs in all 50
states. 


3.  Include explanatory information on the goals and measures. 

Explanatory information in a performance plan can help to show the
relationship among results-oriented goals, measures, and program
outputs and services.  Such information is particularly important
when an agency uses intermediate goals or has several goals that need
to be balanced.  In its performance plan, the Centers for Disease
Control and Prevention linked its program objectives to its
performance goals and measures and to the Department of Health and
Human Services' strategic goals.  Also included was a rationale for
the use of process and output measures to inform the user of the plan
about limitations of the data and associated program constraints.  As
previously illustrated, the section in its performance plan on
chronic disease prevention noted that health outcome measures for
these programs have been difficult to define for a number of reasons,
including the long latency period of chronic diseases like heart
disease. 

4.  Develop performance goals to address mission-critical management
problems. 

Weaknesses in internal management processes and systems undermine the
achievement of program results.  Therefore, the value of performance
plans could be increased if they more fully included performance
goals to address mission-critical management problems that may exist. 
Discussing the most critical management problems--such as those
identified by the administration as a priority management objective
in the governmentwide performance plan, the agency's financial audit,
and other agency assessments made by external organizations,
including our recent performance and accountability and high-risk
series\2 --ensures that those management problems that would have the
greatest impact on results receive the most attention.  A specific
reference to those priority management objectives identified in the
governmentwide performance plan can help show an integrated approach
to their resolution. 

The Department of Education's annual performance plan referred to the
priority management objectives identified in the fiscal year 1999
governmentwide performance plan.  Specifically, the plan acknowledged
that Education's strategic objective on postsecondary student aid
delivery and program management relates to the governmentwide
performance plan's priority management objective on student aid
programs.  Education is responsible for the collection of more than
$150 billion in outstanding loans, and its data systems track 93
million student loans and 15 million Pell Grants.  One of Education's
measures for the Federal Family Education Loan and Direct Loan
programs was a decline in the percentage of borrowers leaving school
who default on their student loans within 2 years.  For the Federal
Family Education Loan program, the annual plan provided baseline and
trend data for the default rate and indicated that the rate declined
from 22.4 to 10.4 percent from fiscal years 1990 to 1995.  For fiscal
year 1999, Education's goal for this program is to reduce the default
rate to 10.1 percent of borrowers.  According to the plan, future
declines are likely to be steady but smaller because of the large
number of high-default schools that have already been eliminated from
the program. 

5.  Show baseline and trend data for past performance. 

With baseline and trend data, the more useful performance plans
provided a context for drawing conclusions about whether performance
goals are reasonable and appropriate.  Decisionmakers can use such
information to gauge how a program's anticipated performance level
compares with improvements or declines in past performance. 

In addition to baseline and trend data on past performance, the
Department of the Treasury's U.S.  Customs Service performance plan
showed both actual and planned levels of performance for fiscal year
1997.  For example, in presenting its goal to maximize trade
compliance, Customs reported the extent of actual and proposed
compliance with trade laws in key industries.  Specifically, in the
bearings industry, Customs reported that in fiscal year 1997, it
achieved an 85.6 percent level of compliance compared to a planned
level of 83 percent; in the automobile and truck parts industry, it
achieved an 82.5 percent compliance level compared to a planned level
of 86 percent.  The proposed performance goals for fiscal year 1999
in both industries are 89 percent.  (See fig.  1.2.)

   Figure 1.2:  Excerpt From the
   Department of the Treasury's
   Fiscal Year 1999 Annual
   Performance Plan Featuring the
   U.S.  Customs Service

   (See figure in printed
   edition.)

6.  Identify projected target levels of performance for multiyear
goals. 

Where appropriate, an agency can convey what it expects to achieve in
the long term by including multiyear performance goals in its
performance plan.  Such information can provide congressional and
other decisionmakers with an indication of the incremental progress
the agency expects to make in achieving results.  As a start, for
example, the Department of Veterans Affairs' performance plan has an
objective for fiscal year 2002 to increase by 20 percent the number
of patients (as opposed to visits) of its healthcare system over its
fiscal year 1997 baseline.  The plan shows that the department
projects achieving a 4.4 percent increase in patients treated in
fiscal year 1998 and an 8.6 percent increase in fiscal year 1999. 
(See fig.  1.3.)

   Figure 1.3:  Excerpt From the
   Department of Veterans Affairs'
   Fiscal Year 1999 Annual
   Performance Plan

   (See figure in printed
   edition.)

7.  Link the goals of component organizations to departmental
strategic goals. 

Linking component performance to departmental goals can provide a
clear, direct understanding of how the achievement of the components'
annual goals will lead to the achievement of the agency's strategic
goals.  The Centers for Disease Control and Prevention, a component
of the Department of Health and Human Services, linked its annual
performance goals to the departmentwide strategic goals.  For
example, the plan set an objective to reduce the case rate of
tuberculosis--an infectious disease that was nearly eliminated in the
United States in the mid-1980s, only to reemerge strongly in the
1990s with drug-resistant strains.  The plan linked that objective to
Health and Human Services' departmentwide goals to reduce major
threats to the health and productivity of all Americans.  Specific
performance goals and measures were included in the plan to reduce
the tuberculosis case rate, such as increase to 85 percent the
patients who will complete a course of curative treatment within 12
months of initiation of treatment. 


--------------------
\1 Performance and Accountability Series:  Major Management
Challenges and Program Risks:  A Governmentwide Perspective
(GAO/OCG-99-1, Jan.  1999). 

\2 Performance and Accountability Series:  Major Management
Challenges and Program Risks (GAO/OCG-99-22SET, Jan.  1999) and
High-Risk Series:  An Update (GAO/HR-99-1, Jan.  1999). 


COORDINATE CROSSCUTTING PROGRAMS
============================================================ Chapter 2

A focus on results implies that agencies will coordinate efforts in
significant program areas where responsibility for achieving results
is shared among agencies.  Agencies' annual performance plans can be
useful tools for identifying such crosscutting programs, setting
complementary performance goals, and describing planned coordination
strategies.  Table 2.1 identifies applied practices that can help
agencies address coordination of crosscutting programs. 



                               Table 2.1
                
                      Applied Practices to Address
                         Crosscutting Programs

----------------------------------  ----------------------------------
A performance plan that addresses   8. Identify programs that
coordination of crosscutting        contribute to the same or similar
programs includes the following     results.
practices:
                                    9. Set complementary performance
                                    goals to show how differing
                                    program strategies are mutually
                                    reinforcing and establish common
                                    or complementary performance
                                    measures, as appropriate.

                                    10. Describe--briefly or refer to
                                    a separate document--planned
                                    coordination strategies.
----------------------------------------------------------------------
8.  Identify programs that contribute to the same or similar results. 

Annual performance plans that include listings of programs and
agencies that share common results are useful in that they begin to
document the range and degree to which those federal programs have
shared responsibilities for achieving results.  These listings can
serve as the prelude to the substantive work and fuller discussion of
how agencies' contributions to the same or similar results are being
coordinated.  As a first step, the Department of Commerce included in
its performance plan a table identifying interagency activities
within which the department participates, such as formulating policy
proposals.  The three themes drawn from Commerce's mission statement
provide the organizational framework for the listing that includes
crosscutting activities by bureaus between Commerce and other federal
agencies.  (See fig.  2.1.)

   Figure 2.1:  Excerpt From the
   Department of Commerce's Fiscal
   Year 1999 Annual Performance
   Plan

   (See figure in printed
   edition.)

9.  Set complementary performance goals to show how differing program
strategies are mutually reinforcing and establish common or
complementary performance measures, as appropriate. 

After agencies have identified other agencies with which they have a
shared responsibility, the next step is to include in their
performance plans complementary performance goals to show how
differing program strategies are mutually reinforcing and establish
common or complementary performance measures, as appropriate.  Taking
this further, the usefulness of a performance plan to congressional
and other decisionmakers could be enhanced if an agency identifies
the results-oriented performance goals that involve other agencies
and sets intermediate goals that clarify the specific contribution
the agency makes to the common result. 

The Department of Health and Human Services' Indian Health Service
associated its annual performance goals to the governmentwide goals
of Healthy People.  Started in 1979, Healthy People is a series of
outcome-based public health objectives and measures developed and
updated each decade by the U.S.  Public Health Service in
consultation with other federal agencies, state governments, and
national organizations.  The objectives for 2000 succeeded the 1990
health objectives that were set in 1980.  The Public Health Service
has started planning for the 2010 health objectives, which are
scheduled to be released in January 2000. 

In developing its performance plan, the Indian Health Service
reviewed the appropriateness of the Healthy People objectives that
focused on the most significant health problems affecting American
Indians and Alaska Natives and the essential services that address
them.  The plan reported that virtually all of the Service's
performance goals support the Healthy People objectives for 2000. 
For example, the Indian Health Service's fiscal year 1999 goal to
ensure that the proportion of the American Indian and Alaska Native
female population over 40 years of age who have had screening
mammography is no lower than the fiscal year 1996 level relates to a
Healthy People objective on reducing the number of women who die from
breast cancer.  The plan also described the approach for meeting this
goal, as well as the data sources and baseline data. 

The performance plans for both the Department of Transportation and
the Department of Justice referenced the Office of National Drug
Control Policy (ONDCP) goal framework.  Established in 1988, the
principal purpose of ONDCP is to establish policies, priorities, and
objectives for the nation's drug control program.  In conjunction
with stakeholders from both the public and private sectors, ONDCP has
proposed a system to assess the effectiveness of the national drug
control strategy to reduce the use of drugs, their availability, and
the damaging consequences associated with drug use and trafficking. 
The strategic goals and performance measures form a framework that
shows how the efforts of more than 50 federal drug control agencies'
programs--along with state, local, and foreign governments and the
private sector--can contribute to the drug control strategy. 

Transportation's performance plan stated that the Coast Guard is
working with ONDCP to develop a performance goal and indicator
measuring the rate of illegal drug smuggling in noncommercial
maritime routes.  Justice's performance plan stated it will continue
to work with ONDCP and other agencies in developing and implementing
an interagency performance measurement system to help gauge the
effectiveness of the national drug control strategy. 

10.  Describe--briefly or refer to a separate document--planned
coordination strategies. 

The more useful performance plans not only identified crosscutting
efforts, they also described how agencies expected to coordinate
efforts with other agencies that have similar responsibilities.  When
interagency coordination is difficult because of disparate or even
conflicting missions across agencies, performance plans offer
agencies an opportunity to identify statutory or regulatory options
for better aligning disparate missions and for identifying
initiatives to avoid or mitigate conflicts over strategic and
performance goals. 

Because progress in coordinating crosscutting programs is still in
its infancy, discussions in performance plans of what is being done
can contribute to the development of a common base for understanding
the strengths and weaknesses of various approaches to coordination. 
The discussion in the annual performance plans can describe
opportunities for interagency coordination that are identified in
other agencies' strategic and performance plans, describe the status
of coordination efforts, and develop interagency strategies to
achieve shared outcome goals in the most efficient and effective
manner.  As a next step, annual performance plans could discuss how
agencies can cooperate in data collection and program evaluation. 

For example, the Department of Education's performance plan discussed
how it plans to coordinate its programs that share common results and
its efforts to cooperate with others to collect data and conduct
program evaluations.  For its strategic objective that schools are
strong, safe, disciplined, and drug-free, the performance plan
indicated that the agency will continue to pursue joint prevention
activities, such as truancy and hate crimes initiatives, with the
Department of Justice.  Education intends to develop a database for
the President's Report Card on School Violence with the Department of
Justice and make maximum use of data from other federal agencies. 
Education also plans to continue to cooperate on evaluation projects
with Justice and Health and Human Services.  (See fig.  2.2.)

   Figure 2.2:  Excerpt From the
   Department of Education's
   Fiscal Year 1999 Annual
   Performance Plan

   (See figure in printed
   edition.)


SHOW HOW STRATEGIES WILL BE USED
TO ACHIEVE GOALS
============================================================ Chapter 3

The more useful annual performance plans discussed how specific
strategies and programs will contribute to achieving performance
goals.  Agencies can also use their performance plans to articulate
their strategies to leverage or mitigate key external factors that
will affect program performance and strategies to address
mission-critical management problems that threaten the achievement of
program results.  Table 3.1 identifies applied practices that can
help agencies show how strategies will be used to achieve goals. 



                               Table 3.1
                
                   Applied Practices to Address Goal
                         Achievement Strategies

----------------------------------  ----------------------------------
A performance plan that shows how   11. Link strategies and programs
strategies will be used to achieve  to specific performance goals and
goals includes the following        describe how they will contribute
practices:                          to the achievement of those
                                    goals.

                                    12. Describe strategies to
                                    leverage or mitigate the effects
                                    of external factors on the
                                    accomplishment of performance
                                    goals.

                                    13. Discuss strategies to resolve
                                    mission-critical management
                                    problems.

                                    14. Discuss--briefly or refer to a
                                    separate document--plans to ensure
                                    that mission-critical processes
                                    and information systems function
                                    properly and are secure.
----------------------------------------------------------------------
11.  Link strategies and programs to specific performance goals and
describe how they will contribute to the achievement of those goals. 

The more useful annual performance plans linked the agencies'
strategies and programs to specific performance goals and described
how they will contribute to the achievement of those goals.  The
listings of current programs and initiatives that often were included
in agencies' fiscal year 1999 plans were useful for providing an
understanding of what agencies do.  However, presentations that more
directly explain how programs and initiatives achieve goals will be
most helpful to congressional and other decisionmakers in assessing
the degree to which strategies are appropriate and reasonable.  For
example, an agency might state in its performance plan that it will
administer grants to states.  In order to link this strategy to its
intended results, the agency should explain how administering grants
to states will help it achieve specific performance goals.  Over time
and as agencies gain experience in linking strategies to results,
agencies will be in a better position to understand and discuss
alternative strategies and program design and the relative
contributions these alternatives could make to results.  A subsequent
discussion of how different governing tools (for example,
intergovernmental partnerships, performance-based contracts,
financial grants and credits, and direct service provision by the
federal government) can be used in achieving goals could further
enhance the plans. 

The Department of Agriculture's Food Safety and Inspection Service
provided a detailed discussion of its strategy to achieve its
performance goal of reducing pathogens on raw meat and poultry
products.  The performance plan proposed to expand in-plant
inspections and laboratory testing of meat and poultry samples taken
from slaughtering plants.  Laboratory personnel will conduct expanded
testing of pathogens of samples taken from plants slaughtering
cattle, swine, chicken, and turkeys and use upgraded technology to
conduct inspections including stethoscopes, thermometers, and
computer software programs that are not currently available. 

The Federal Emergency Management Agency identified in its performance
plan a number of strategies relating to its Project Impact program,
which is designed to promote predisaster mitigation.  One performance
goal for this program is to increase the number of Project Impact
communities in each state by at least one.  The strategies identified
for achieving this goal included working with states and federal
agencies to identify candidate communities, providing grants as seed
funding, providing technical information, and monitoring progress. 

12.  Describe strategies to leverage or mitigate the effects of
external factors on the accomplishment of performance goals. 

The extent to which results are achieved is influenced by a number of
external factors, such as economic, social, and technological
developments; and the actions of numerous outside entities, including
state and local governments.  The Results Act requires agencies to
identify key external factors in their strategic plans that would
affect the degree to which they achieve their strategic goals.  The
value of performance plans can be augmented by including discussions
on the strategies agencies plan to take to leverage or mitigate the
influence these factors can have on achieving results.  Such
discussions can help congressional and other decisionmakers determine
if the agency has the best mix of program strategies in place to
achieve its goals, or if additional agency or congressional actions
are needed to better meet those goals. 

The Department of Agriculture's Food and Nutrition Service discussed
external factors as well as strategies to mitigate conditions.  The
annual performance plan included a performance goal to reduce food
stamp trafficking, which is the illegal sale of food stamps for cash. 
The plan cited the agency's dependence on grocery stores to properly
handle food stamps as an external factor that affects the agency's
ability to achieve this goal.  To mitigate this factor, the plan
noted that it has promulgated rules that participating stores must
follow and instituted a system of sanctions that may be applied to
stores that violate the rules. 

13.  Discuss strategies to resolve mission-critical management
problems. 

Performance plans that contain specific strategies to resolve
mission-critical management problems provide congressional and other
decisionmakers with an understanding of how the agency plans to
improve its management.  By discussing such activities, plans also
provide a basis for accountability.  As mentioned earlier, agencies
should set goals to address these management problems, such as those
identified by the administration as a priority management objective
in the governmentwide performance plan, the agency's financial audit,
and other agency assessments by external organizations, including our
performance and accountability and high-risk series.  Annual
performance plans can lay out concrete actions the agency plans to
resolve mission-critical management problems that threaten the
achievement of major program goals. 

As a first step, Commerce's performance plan identified major
management challenges raised by the Inspector General, such as the
department's financial management system, which is also included in
our performance and accountability series.  The plan stated that the
system, the Commerce Administrative Management System, is
substantially over its original estimated cost and continues to
experience performance shortfalls and schedule delays.  The plan
identified actions the department will take to reduce expenditures
and impose greater control over the project's direction, such as
implementing the system as a pilot at the Census Bureau. 

14.  Discuss--briefly or refer to a separate document--plans to
ensure that mission-critical processes and information systems
function properly and are secure. 

By including information technology issues in annual performance
plans, agencies underscore the critical role that technology plays in
achieving results.  Key information technology issues include the
ability to ensure that mission-critical information systems will
function properly after the year 1999 and are secure from intrusion
and misuse--two of the issues we have identified to be high risk for
the federal government.\1 Year 2000-induced failures of one or more
mission-critical systems may have a severe impact on agencies'
abilities to deliver critical services.  Agencies can use their
fiscal year 2000 performance plans, which become effective in October
1999, to discuss their final business continuity and contingency
plans to reduce the risk of Year 2000 business failures. 

As a start, the Social Security Administration reported in its
performance plan that it expects that by January 1999 all of its
mission-critical systems will have the ability to operate properly in
2000.  It reported that it is also preparing a Year 2000 contingency
plan that addresses how core business processes will be supported if
planned conversion activities experience unforeseen disruptions. 

Despite the sensitivity and criticality of federal information
systems, they are not being adequately protected from unauthorized
access.\2 Although advances in the use of electronic data promise to
streamline federal operations and improve the delivery of federal
services, security weaknesses can create serious pervasive risks for
the federal government, such as potential disclosure of sensitive
data, loss of assets worth billions of dollars due to fraud, and
disruption of critical operations.  The annual performance plan could
discuss the agency's ability to ensure the integrity and availability
of information resources supporting mission-critical operations and
the confidentiality of sensitive information. 

As a first step, the Department of Transportation's fiscal year 1999
performance plan stated that a key component of continued security of
its information system is the education of the workforce.  By the end
of fiscal year 1998, the plan stated that the agency will develop
information systems security training for various elements of the
workforce.  A fiscal year 1999 milestone is that 100 percent of
senior management, 75 percent of system administrators, and 60
percent of end users will be trained. 

In its performance plan, the Office of Personnel Management discussed
plans to enhance its information security program by conducting
internal and external evaluations of its systems, such as engaging
the assistance of the National Security Agency to review its security
capabilities and implementing appropriate recommendations to improve
its security.  In addition to training staff, the agency will have in
place a tested disaster recovery capability for general support and
major financial, benefits, and workforce information application
systems. 


--------------------
\1 High-Risk Series:  An Update (GAO/HR-99-1, Jan.  1999). 

\2 Information Security:  Serious Weaknesses Place Critical Federal
Operations and Assets at Risk (GAO/AIMD-98-92, Sept.  23, 1998). 


SHOW PERFORMANCE CONSEQUENCES OF
BUDGET AND OTHER RESOURCE
DECISIONS
============================================================ Chapter 4

The more useful annual performance plans related the performance
consequences of budget and other resource decisions by indicating how
the funding from agencies' program activities will be allocated to a
discrete set of performance goals.  Table 4.1 identifies applied
practices that can help agencies show performance consequences of
budget and other resource decisions. 



                               Table 4.1
                
                Applied Practices to Address Budget and
                           Resource Decisions

----------------------------------  ----------------------------------
A performance plan that shows       15. Show how budgetary resources
performance consequences of budget  relate to the achievement of
and other resource decisions        performance goals.
includes the following practices:
                                    16. Discuss--briefly and refer to
                                    the agency capital plan--how
                                    proposed capital assets
                                    (specifically information
                                    technology investments) will
                                    contribute to achieving
                                    performance goals.

                                    17. Discuss--briefly or refer to a
                                    separate plan--how the agency will
                                    use its human capital to help
                                    achieve performance goals.
----------------------------------------------------------------------
15.  Show how budgetary resources relate to the achievement of
performance goals. 

A key expectation of the Results Act is that Congress will gain a
clearer understanding of what is being achieved in relation to what
is being spent.  Therefore, plans are more useful when they convey
the requested funding level associated with achieving a discrete set
of performance goals and identify where that funding was included in
the structure of agencies' budget requests.  With such information,
decisionmakers can better compare planned levels of accomplishments
with the resources requested.  Decisionmakers can then consider how
significant changes from the requested amounts for these goals could
affect performance.  Our forthcoming report on this practice provides
additional information on the variety of approaches agencies used to
connect budget resources with results. 

The Results Act gives agencies the flexibility to aggregate,
disaggregate, or consolidate the budget's program activities so that
they align with the performance goals.  If an agency takes one of
these approaches, its performance plan will be more useful if it
includes a crosswalk that shows how the program activities in its
budget request relate to the program activities in its annual
performance plan.  In addition to this flexibility, an agency may
propose a change to its budget account or program activity structures
to facilitate an understanding of performance. 

The Department of Health and Human Services' Indian Health Service
has taken the first step in its performance plan to aggregate program
activities for the purposes of performance planning.  The plan
includes a crosswalk that aligns its budget activities with its
performance in four areas identified as Results Act (GPRA) aggregated
program activities:  Treatment; Prevention; Capital
Programming/Infrastructure; and Partnerships, Consultation, Core
Functions, and Advocacy.  The crosswalk indicates that from two
separate budget accounts--Services and Facilities--the Indian Health
Service consolidated program activities (6, 7, 8, and 9 from the
Services budget account and 21 and 22 from the Facilities budget
account) in the area of Prevention.  The crosswalk also shows that
budget program activities 21 and 22 support GPRA program activities
2, 3, and 4 (Prevention; Capital Programming/Infrastructure; and
Partnerships, Consultation, Core Functions, and Advocacy).  (See fig. 
4.1.) The Indian Health Service's intended performance in the
Prevention area includes such things as increasing childhood
immunization rates, reducing deaths by unintentional injuries, and
reducing childhood obesity. 

   Figure 4.1:  Excerpt From the
   Department of Health and Human
   Services' Fiscal Year 1999
   Annual Performance Plan
   Featuring the Indian Health
   Service

   (See figure in printed
   edition.)

The Environmental Protection Agency proposed changes to its program
activity structure to better align its budget resources with
performance goals.  The agency proposed a uniform program activity
structure across all of its accounts in which each program activity
represents one of the agency's strategic goals.  Using this realigned
program activity structure in its annual plan, the agency showed, by
account, the funding it requested in order to achieve each strategic
objective and the supporting annual performance goals.  For example,
it requested almost $22 million in fiscal year 1999 to meet its
strategic objective related to the reduction of acid rain.  This
request is associated with performance goals relating to emissions of
sulfur dioxide (SO2) and nitrogen oxides (NOx) that react in the
atmosphere and fall to earth as acid rain.  The request for the acid
rain strategic objective was related to three accounts from the
"clean air" program activity:  Environmental Program and Management,
Science and Technology, and State and Tribal Assistance Grants.  (See
fig.  4.2.)

   Figure 4.2:  Excerpt From the
   Environmental Protection
   Agency's Fiscal Year 1999
   Annual Performance Plan

   (See figure in printed
   edition.)

In addition to aligning budget resources with performance goals and
including trend data and target levels of performance, the Department
of the Treasury's Internal Revenue Service displayed in its
performance plan the budget amounts that corresponded with the
performance actually achieved in a given fiscal year.  Specifically,
the agency showed that for fiscal year 1997, it received budget
authority of almost $795 million in its Submission Processing
Activity to achieve a set of performance goals:  Improve Customer
Service, Increase Compliance, and Increased Productivity.  The
performance level that it achieved in fiscal year 1997 to Improve
Customer Service included, among other things, a timeliness refund
rate of 14.5 days for electronic filings, an improvement over the
rates in previous years of 15.5 days in fiscal year 1996 and 21 days
in fiscal year 1995.  (See fig.  4.3.)

   Figure 4.3:  Excerpt From the
   Department of the Treasury's
   Fiscal Year 1999 Annual
   Performance Plan Featuring the
   Internal Revenue Service

   (See figure in printed
   edition.)

16.  Discuss--briefly and refer to the agency capital plan--how
proposed capital assets (specifically information technology
investments) will contribute to achieving performance goals. 

The federal government's spending on major physical capital
investment is projected to total over $68 billion in fiscal year
1999.  Capital assets include information technology; equipment;
land; structures; and intellectual property, such as software.  With
federal agencies facing increasing demands to improve performance,
the importance of making the most effective capital acquisition
choices--choices that are linked to results--will also intensify.  A
number of laws are beginning to propel agencies toward improving
their capital decisionmaking practices.  For example, Title V of the
Federal Acquisition Streamlining Act of 1994 was designed to foster
the development of measurable cost, schedule, and performance goals
and incentives for acquisition personnel to reach these goals.  The
Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 put
in place statutory requirements for agencies to better link their
information technology plans and decisions to their missions and
programmatic goals. 

OMB has taken an active leadership role in seeking to link capital
decisionmaking to program results.  OMB developed a Capital
Programming Guide that provides agencies with the key elements for
producing effective plans and investments.\1 OMB's Guide drew on our
work on best practices used by leading private sector and state and
local governments, which was subsequently published.\2 Consistent
with these best practices, OMB has required agencies to submit 5-year
capital spending plans and justifications--thus encouraging a
longer-term consideration of agency capital needs and alternatives
for addressing them.  OMB's Guide provides a basic reference on
principles and techniques, including appropriate strategies for
analyzing benefits and costs, preparing budget justifications, and
managing capital assets once they are in place.  In addition, OMB has
worked closely with the President's Commission to Study Capital
Budgeting, which is expected to issue its report and recommendations
soon. 

In this regard, a more useful performance plan describes the
processes, technologies, and types of resources, including capital,
that are needed to achieve performance goals.  Further, performance
plans provide agencies with the opportunity to show how a proposed
capital asset will, for example, decrease costs, improve service
quality, or increase productivity.  For example, the Department of
the Treasury's U.S.  Customs Service performance plan explained that
continuing the acquisition and installation of the Land Border
automation equipment is needed to allow inspectors to perform more
careful screening and questioning of vehicle occupants, which should
help to achieve Customs' goal of improving its efficiency at
targeting arriving vehicles for enforcement purposes. 

17.  Discuss--briefly or refer to a separate plan--how the agency
will use its human capital to help achieve performance goals. 

In an era that demands improved performance at reduced costs, an
agency's success depends upon its ability to assemble the human
capital with the right blend of talents and skills.  Thus, human
capital planning must be an integral part of an organization's
strategic and program planning; human capital itself should be
thought of not as a cost to be minimized but as an asset to be
enhanced.  To show that the vital integration of human capital and
program planning is in place, the more useful annual performance
plans could discuss the size and composition--in terms of knowledge,
skills, and abilities--of the human capital needed to support the
achievement of performance goals and explain the workforce planning
methods by which these needs were determined.  Also, the more useful
plans could describe how strategies--in such areas as recruitment and
hiring, retention and separation, training and career development,
and employee incentives and accountability systems--could meet
workforce needs and support the achievement of annual performance
goals. 

The U.S.  Agency for International Development stated in its
performance plan that its direct-hire workforce has been trimmed by
hiring freezes, early retirements, and a reduction-in-force in 1996. 
In the wake of these events, the plan recognized that the agency must
identify critical skills needed to achieve its goals.  Its Office of
Human Resources is examining issues and changes in workforce planning
and is focusing, among other things, on improving basic personnel
operations, such as assignments and employee evaluations. 


--------------------
\1 Capital Programming Guide, Version 1, July 1997 (Executive Office
of the President, OMB). 

\2 Executive Guide:  Leading Practices in Capital Decision-Making
(GAO/AIMD-99-32, Dec.  1998). 


BUILD THE CAPACITY TO GATHER AND
USE PERFORMANCE INFORMATION
============================================================ Chapter 5

In order to successfully measure progress toward intended results,
agencies need to build the capacity to gather and use performance
information.  Under the Results Act, agencies are to discuss in their
annual performance plans how they will verify and validate the
performance information that they plan to use to show whether goals
are being met.  The usefulness of agencies' plans could increase if
they identify data sources, describe efforts to verify and validate
data, identify actions to compensate for unavailable data or
low-quality data, and discuss implications of data limitations for
assessing performance.  Table 5.1 identifies applied practices that
can help agencies describe their capacity to gather and use
performance information. 



                               Table 5.1
                
                      Applied Practices to Address
                    PerformanceInformation Capacity

----------------------------------  ----------------------------------
A performance plan that describes   18. Identify internal and external
an agency's capacity to gather and  sources for data.
use performance information
includes the following practices:   19. Describe efforts to verify and
                                    validate performance data.

                                    20. Identify actions to compensate
                                    for unavailable or low-quality
                                    data.

                                    21. Discuss implications of data
                                    limitations for assessing
                                    performance.
----------------------------------------------------------------------
18.  Identify internal and external sources for data. 

The usefulness of performance data ultimately depends on the degree
of confidence that users have in that data.  The more informative
annual performance plans can provide decisionmakers with confidence
in the agency's ability to report on its performance goals and
measures by identifying existing internal and external data sources. 
In those instances when performance data do not exist, a description
in the performance plan of efforts by the agency to identify relevant
data sources could inform decisionmakers of actions the agency is
taking in order to measure performance. 

The Department of Justice identified in its performance plan specific
sources of performance data for many of its performance measures. 
According to its plan, data are already collected and reported
through existing statistical series and internal Justice data
systems, such as the department's Bureau of Justice Statistics--the
primary source of justice statistics on crime, criminal offenders,
victims of crime, and the operation of the justice system at all
levels of government. 

As another example, the Small Business Administration recognized in
its performance plan its need for data from external sources, such as
the Federal Procurement Data System.  This system provides
information relating to the agency's goal to help increase the share
of federal procurement dollars awarded to small firms.  Specifically,
the Federal Procurement Data System provides the agency with data on
the percentage of federal procurement dollars awarded to small
businesses, women-owned businesses, and small disadvantaged
businesses. 

19.  Describe efforts to verify and validate performance data. 

Performance plans are to include descriptions of the procedures that
will be used to verify and validate the measured values of actual
performance.  The procedures should be credible and specific to
ensure that performance information is sufficiently complete,
accurate, and consistent to document performance and support
decisions on how best to manage programs.  Information in the
performance plan about the procedures to be used to verify and
validate the data--such as scope, methodology, and timing--can
reinforce an agency's ability to generate reliable and timely
performance data. 

To ensure the accuracy and reliability of reported data, the
Department of Justice stated in its performance plan that it will (1)
examine and analyze data to identify anomalies or inconsistencies,
(2) make comparisons to other data series measuring the same or
similar variables, (3) require the submitting entities to certify the
accuracy and completeness of data and to identify any data
limitations, (4) institute appropriate quality control checks, (5)
conduct periodic data audits, and (6) obtain independent audits of
its financial statements.  Justice expects that its component
organizations will independently pursue several of these initiatives,
consistent with their individual data requirements. 

As part of its efforts to underscore that program managers are to use
performance data to make decisions, the Department of Education has
decided to hold program managers accountable for the quality of the
performance data related to their programs.  In its performance plan,
the Department of Education established accountability for data
quality with its program managers as one of its strategies for
ensuring high-quality information.  Specifically, program managers
will attest that the data used for their program's performance
measurement are reliable and valid, or they will submit plans for
data improvement.  To validate program managers' assertions to the
quality of their data, Inspector General audits and program
evaluations will be used.  Education will support this strategy
through guidance to managers on developing and monitoring quality
data systems and using data to manage program performance. 

In addition to the procedures used to verify and validate performance
data, the use of program evaluations is important for assessing how
well a program is working, identifying the factors affecting
performance, and pinpointing improvement opportunities.  In its
performance plan, the Department of Transportation recognized that
program evaluation will help it set meaningful performance goals by
quantifying the relationship between program efforts and levels of
outcomes.  The plan further identified specific program evaluations
planned for fiscal years 1998 and 1999.  The department also noted
that it is developing a method to both plan and track program
evaluations, as well as to develop the skills and resources
sufficient to conduct evaluations throughout its organization. 

20.  Identify actions to compensate for unavailable or low-quality
data. 

When data are unavailable or existing data contain significant
limitations, providing credible data may require new information
systems or major changes to existing systems.  We recently reported,
for example, that some agencies developed or adopted standardized
data collection systems to obtain data not readily available to them
or to fill information gaps.\1 However, an agency's ability to
collect and generate data often needs to be weighed against several
factors, such as cost or the burden on respondents.  The more
informative annual performance plans provide a vehicle for discussing
where the agency struck such a balance and the implications for the
quality of the data--including accuracy, reliability, and
timeliness--and the cost of collection. 

The Department of Education recognized that to effectively report the
performance of the department in achieving its strategic plan goals
and objectives requires developing some new data systems and fixing
old ones.  For example, Education stated in its performance plan that
its student aid delivery system has suffered from significant data
quality problems.  The plan outlined several steps that Education
plans to take to address these problems, including improving data
accuracy by establishing industrywide standards for data exchanges,
receiving individual student loan data directly from lenders,
expanding efforts to verify data reported to the National Student
Loan Data System, and preparing a systems architecture for the
delivery of federal student aid. 

21.  Discuss implications of data limitations for assessing
performance. 

Performance plans could be more useful if they recognize and identify
significant data limitations and their implications for assessing
performance.  Plans can identify known data limitations from internal
and external sources, such as quality or timeliness of the data;
indicate what actions will be taken to compensate for the
limitations; and report when those actions will be taken.  The more
informative performance plans also identify implications for
decisionmaking arising from data limitations on reported results. 

The Department of Transportation stated in its performance plan that
one of the most significant limitations of both internal and external
data is timeliness.  One way the department will deal with this
limitation is to compile preliminary estimates from the portion of
data that is available in time to report on the performance measures. 
According to the plan, fatality data from the first 6 months of the
year could be compared with data from the first 6 months of the
previous year for an initial performance measurement. 


--------------------
\1 Managing for Results:  Measuring Program Results That Are Under
Limited Federal Control (GAO/GGD-99-16, Dec.  11, 1998). 


GAO PRODUCTS ON FISCAL YEAR 1999
PERFORMANCE PLANS
=========================================================== Appendix I

Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 

The Results Act:  Assessment of the Governmentwide Performance Plan
for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept.  8, 1998). 

Managing for Results:  Measuring Program Results That Are Under
Limited Federal Control (GAO/GGD-99-16, Dec.  11, 1998). 


   DEPARTMENTS
--------------------------------------------------------- Appendix I:1

Results Act:  Observations on the U.S.  Department of Agriculture's
Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-212R, June
11, 1998). 

Results Act:  Observations on the Department of Commerce's Annual
Performance Plan for Fiscal Year 1999 (GAO/GGD-98-135R, June 24,
1998). 

Results Act:  DOD's Annual Performance Plan for Fiscal Year 1999
(GAO/NSIAD-98-188R, June 5, 1998). 

The Results Act:  Observations on the Department of Education's
Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-172R, June 8,
1998). 

Results Act:  Observations on DOE's Annual Performance Plan for
Fiscal Year 1999 (GAO/RCED-98-194R, May 28, 1998). 

The Results Act:  Observations on the Department of Health and Human
Services' Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-180R,
June 17, 1998). 

Results Act:  Observations on the Department of Housing and Urban
Development's Fiscal Year 1999 Annual Performance Plan
(GAO/RCED-98-159R, June 5, 1998). 

Results Act:  Department of the Interior's Annual Performance Plan
for Fiscal Year 1999 (GAO/RCED-98-206R, May 28, 1998). 

Observations on the Department of Justice's Fiscal Year 1999
Performance Plan (GAO/GGD-98-134R, May 29, 1998). 

Results Act:  Observations on Labor's Fiscal Year 1999 Performance
Plan (GAO/HEHS-98-175R, June 4, 1998). 

The Results Act:  Observations on the Department of State's Fiscal
Year 1999 Annual Performance Plan (GAO/NSIAD-98-210R, June 17, 1998). 

Results Act:  Observations on the Department of Transportation's
Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-180R, May
12, 1998). 

Results Act:  Observations on Treasury's Fiscal Year 1999 Annual
Performance Plan (GAO/GGD-98-149, June 30, 1998). 

Results Act:  Observations on VA's Fiscal Year 1999 Performance Plan
(GAO/HEHS-98-181R, June 10, 1998). 


   INDEPENDENT AGENCIES
--------------------------------------------------------- Appendix I:2

Results Act:  EPA's Annual Performance Plan for Fiscal Year 1999
(GAO/RCED-98-166R, Apr.  28, 1998). 

Results Act:  Observations on the Federal Emergency Management
Agency's Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-207R,
June 1, 1998). 

Results Act:  Observations on the General Services Administration's
Annual Performance Plan (GAO/GGD-98-110, May 11, 1998). 

Managing for Results:  Observations on NASA's Fiscal Year 1999
Performance Plan (GAO/NSIAD-98-181, June 5, 1998). 

Results Act:  NSF's Annual Performance Plan for Fiscal Year 1999
(GAO/RCED-98-192R, May 19, 1998). 

Results Act:  NRC's Annual Performance Plan for Fiscal Year 1999
(GAO/RCED-98-195R, May 27, 1998). 

Results Act:  Observations on the Office of Personnel Management's
Annual Performance Plan (GAO/GGD-98-130, July 28, 1998). 

Results Act:  Observations on the Small Business Administration's
Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-200R, May 28,
1998). 

The Results Act:  Observations on the Social Security
Administration's Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R,
June 9, 1998). 

The Results Act:  Observations on USAID's Fiscal Year 1999 Annual
Performance Plan (GAO/NSIAD-98-194R, June 25, 1998). 


RELATED GUIDANCE ON ANNUAL
PERFORMANCE PLANS
========================================================== Appendix II

This is a compilation of related guidance, including the Results Act,
OMB Circular A-11, and GAO reports and is arranged by the sections
discussed in this report. 


   SECTION 1:  BETTER ARTICULATE A
   RESULTS ORIENTATION
-------------------------------------------------------- Appendix II:1

Government Performance and Results Act (Results Act), 5 U.S.C. 
306(c), 31 U.S.C.  1115(a)(1), 1115(a)(2), 1115(a)(4), and
1115(a)(5). 

Senate Committee on Governmental Affairs Report accompanying the
Results Act (Senate Report 103-58, June 16, 1993), pp.  15-16,
"Performance Plans"; pp.  27-28, "Federal Government Performance
Plan"; p.  28, "Agency Performance Plans"; p.  29, "Performance
Goals"; and pp.  29-30, "Performance Indicators."

OMB Circular A-11(1998), secs.  210.4, 220.1, 220.3, 220.4, 220.5(a),
220.5(b), 220.5(c), 220.5(f), 220.6(a), 220.6(b), 220.6(d), 220.7(a),
220.7(b), 220.7(c), 220.7(d), 220.7(f), 220.9(a), 220.9(b), 220.9(c),
220.9(d), 220.9(e), 220.13, 220.15, 220.16, 221.1(a), 221.1(b),
221.1(c), 221.2(a), 221.2(b), 221.2(c), 221.2(d), 221.2(e), and
221.3. 

Managing for Results:  Measuring Program Results That Are Under
Limited Federal Control (GAO/GGD-99-16, Dec.  11, 1998). 

Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 

The Results Act:  Assessment of the Governmentwide Performance Plan
for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept.  8, 1998). 

The Results Act:  An Evaluator's Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr.  1998). 

Agencies' Annual Performance Plans Under the Results Act:  An
Assessment Guide to Facilitate Congressional Decisionmaking, Version
1 (GAO/GGD/AIMD-10.1.18, Feb.  1998). 

The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

Managing for Results:  Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997). 

Measuring Performance:  Strengths and Limitations of Research
Indicators (GAO/RCED-97-91, Mar.  21, 1997). 

Executive Guide:  Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996). 

GPRA Performance Reports (GAO/GGD-96-66R, Feb.  14, 1996). 

Implementation of the Government Performance and Results Act (GPRA),
A Report on the Chief Financial Officer's Role and Other Issues
Critical to the Governmentwide Success of GPRA, Chief Financial
Officers Council, GPRA Implementation Committee, May 1995. 


   SECTION 2:  COORDINATE
   CROSSCUTTING PROGRAMS
-------------------------------------------------------- Appendix II:2

OMB Circular A-11(1998), secs.  220.7(e) and 220.9(f). 

Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 

The Results Act:  Assessment of the Governmentwide Performance Plan
for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept.  8, 1998). 

The Results Act:  An Evaluator's Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr.  1998). 

Agencies' Annual Performance Plans Under the Results Act:  An
Assessment Guide to Facilitate Congressional Decisionmaking, Version
1 (GAO/GGD/AIMD-10.1.18, Feb.  1998). 

Managing for Results:  Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug.  29, 1997). 

The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 


   SECTION 3:  SHOW HOW STRATEGIES
   WILL BE USED TO ACHIEVE GOALS
-------------------------------------------------------- Appendix II:3

Results Act, 31 U.S.C.  9703. 

Senate Committee on Governmental Affairs Report accompanying Results
Act (Senate Report 103-58, June 16, 1993), pp.  15-16, "Performance
Plans"; pp.  17-18, "Managerial Flexibility Waivers"; and pp.  34-36,
"Section 5.  Managerial Accountability and Flexibility."

OMB Circular A-11 (1998), secs.  220.5(d), 220.5(e), 220.9(b),
220.9(d), 220.11(a), 220.11(b), 220.11(c), 220.11(d), and 220.11(e). 

Year 2000 Computing Crisis:  A Testing Guide (GAO/AIMD-10.1.21, Nov. 
1998). 

Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 

The Results Act:  Assessment of the Governmentwide Performance Plan
for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept.  8, 1998). 

Year 2000 Computing Crisis:  Business Continuity and Contingency
Planning (GAO/AIMD-10.1.19, Aug.  1998). 

Executive Guide:  Information Security Management (GAO/AIMD-98-68,
May 1998). 

The Results Act:  An Evaluator's Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr.  1998). 

Executive Guide:  Measuring Performance and Demonstrating Results of
Information Technology Investments (GAO/AIMD-98-89, Mar.  1998). 

Agencies' Annual Performance Plans Under the Results Act:  An
Assessment Guide to Facilitate Congressional Decisionmaking, Version
1 (GAO/GGD/AIMD-10.1.18, Feb.  1998). 

Year 2000 Computing Crisis:  An Assessment Guide (GAO/AIMD-10.1.14,
Sept.  1997). 

The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

Managing for Results:  Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997). 

Business Process Reengineering Assessment Guide, Version 3
(GAO/AIMD-10.1.15, Apr.  1997). 

Privatization:  Lessons Learned by State and Local Governments
(GAO/GGD-97-48, Mar.  14, 1997). 

Executive Guide:  Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996). 


   SECTION 4:  SHOW PERFORMANCE
   CONSEQUENCES OF BUDGET AND
   OTHER RESOURCE DECISIONS
-------------------------------------------------------- Appendix II:4

Results Act, 31 U.S.C.  1115(a)(3) and 1115(c). 

Senate Committee on Governmental Affairs Report accompanying Results
Act (Senate Report 103-58, June 16, 1993), pp.  15-16, "Performance
Plans"; pp.  18-19, "Performance Budgeting"; pp.  29-30, "Performance
Indicators"; and p.  31, "Coverage of Program Activities."

OMB Circular A-11 (1998), secs.  220.1, 220.8(a), 220.8(b), 220.8(c),
220.8(d), 220.8(e), 220.9(d), 220.10(a), 220.10(b), 220.10(c),
220.11(a), 220.11(d), 221.1(a), 221.1(b), 221.1(c), and Part 3. 

OMB Capital Programming Guide, v.  1.0 (July 1997). 

Executive Guide:  Leading Practices in Capital Decision-Making
(GAO/AIMD-99-32, Dec.  1998). 

Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 

Performance Management:  Aligning Employee Performance With Agency
Goals at Six Results Act Pilots (GAO/GGD-98-162, Sept.  4, 1998). 

The Results Act:  An Evaluator's Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr.  1998). 

Acquisition Reform:  Implementation of Key Aspects of the Federal
Acquisition Streamlining Act of 1994 (GAO/NSIAD-98-81, Mar.  9,
1998). 

Budget Issues:  Budgeting for Capital (GAO/T-AIMD-98-99, Mar.  6,
1998). 

Executive Guide:  Measuring Performance and Demonstrating Results of
Information Technology Investments (GAO/AIMD-98-89, Mar.  1998). 

Agencies' Annual Performance Plans Under the Results Act:  An
Assessment Guide to Facilitate Congressional Decisionmaking, Version
1 (GAO/GGD/AIMD-10.1.18, Feb.  1998). 

The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

Performance Budgeting:  Past Initiatives Offer Insights for GPRA
Implementation (GAO/AIMD-97-46, Mar.  27, 1997). 

Assessing Risks and Returns:  A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, Version 1 (GAO/AIMD-10.1.13,
Feb.  1997). 

Information Technology Investment:  Agencies Can Improve Performance,
Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept.  30, 1996). 

Executive Guide:  Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996). 

GPRA Performance Reports (GAO/GGD-96-66R, Feb.  14, 1996). 

Transforming the Civil Service:  Building the Workforce of the
Future--Results of a GAO-Sponsored Symposium (GAO/GGD-96-35, Dec. 
26, 1995). 

Integrating Performance Measurement into the Budget Process, Chief
Financial Officers Council, GPRA Implementation Committee
Subcommittee Project, September 22, 1997. 

Federal Accounting Standards Advisory Board (FASAB) Volume 1 Original
Statements:  Statements of Federal Financial Accounting Concepts and
Standards, Statement of Federal Financial Accounting Standards No. 
1, Objectives of Federal Financial Reporting (GAO/AIMD-21.1.1, Mar. 
1997). 

FASAB Volume 1 Original Statements:  Statements of Federal Financial
Accounting Concepts and Standards, Statement of Federal Financial
Accounting Standards No.  4, Managerial Cost Accounting Standards
(GAO/AIMD-21.1.1, Mar.  1997). 

Budget and Financial Management:  Progress and Agenda for the Future
(GAO/T-AIMD-96-80, Apr.  23, 1996). 


   SECTION 5:  BUILD THE CAPACITY
   TO GATHER AND USE PERFORMANCE
   INFORMATION
-------------------------------------------------------- Appendix II:5

Results Act 31 U.S.C.  1115 (a)(6). 

Senate Committee on Governmental Affairs Report accompanying Results
Act (Senate Report 103-58, June 16, 1993), p.  30, "Verification and
Validation."

OMB Circular A-11(1998), secs.  220.6(a), 220.12(a), and 220.12(b). 

Managing for Results:  Measuring Program Results That Are Under
Limited Federal Control (GAO/GGD-99-16, Dec.  11, 1998). 

Managing for Results:  An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept.  8,
1998). 

The Results Act:  An Evaluator's Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr.  1998). 

Agencies' Annual Performance Plans Under the Results Act:  An
Assessment Guide to Facilitate Congressional Decisionmaking, Version
1 (GAO/GGD/AIMD-10.1.18, Feb.  1998). 

Managing for Results:  Regulatory Agencies Identified Significant
Barriers to Focusing on Results (GAO/GGD-97-83, June 24, 1997). 

The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

Managing for Results:  Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997). 

Measuring Performance:  Strengths and Limitations of Research
Indicators (GAO/RCED-97-91, Mar.  21, 1997). 

Executive Guide:  Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996). 

GPRA Performance Reports (GAO/GGD-96-66R, Feb.  14, 1996). 


MAJOR CONTRIBUTORS
========================================================= Appendix III

GENERAL GOVERNMENT DIVISION

Alan M.  Stapleton, Assistant Director
Lisa R.  Shames, Project Manager
Dorothy L.  Self, Senior Evaluator

ACKNOWLEDGMENTS

The examples used in this report are drawn from the assessments of
the individual agency annual performance plans that were done by
staff across GAO.  Thus, in addition to the individuals noted above,
the staff who worked on the individual agency plan assessments and
summary assessment also made important contributions to this report. 
These individuals are identified in the separate reports on agency
fiscal year 1999 plans. 


*** End of document. ***