Results Act: Observations on CFTC's Fiscal Year 2000 Annual Performance
Plan (Letter Report, 03/15/99, GAO/GGD-99-51).
Pursuant to a congressional request, GAO reviewed the Commodity Futures
Trading Commission's (CFTC) fiscal year 2000 annual performance plan,
focusing on whether CFTC complied with the requirements of the
Government Performance and Results Act of 1993.
GAO noted that: (1) CFTC actions show a good-faith effort and commitment
both to comply with the Results Act and meet congressional expectations
for information about the agency's performance goals, including how the
agency will accomplish the goals and measure their results; (2) however,
CFTC could improve future performance plans in several areas to make the
plans more useful to congressional and other decisionmakers; (3) CFTC
could improve its performance goals, measures, and targets to provide a
clearer picture of its intended performance; (4) CFTC could better
connect its mission, goals, and activities to demonstrate more fully how
it plans to chart annual progress toward achieving its strategic goals;
(5) CFTC could more fully address crosscutting efforts that involve
other federal agencies to better ensure that the overall effectiveness
of federal efforts is maximized; (6) CFTC could discuss in greater
detail the strategies and resources used to achieve its performance
goals to better enable decisionmakers to assess their reasonableness;
(7) CFTC should describe the specific means for verifying and validating
the quality of performance information and ensuring the reliability of
such information; (8) CFTC could link past accomplishments to
performance goals to enhance the plan's clarity; (9) although CFTC could
improve its plan in these areas, GAO has found that other federal
agencies, including federal financial regulators, could improve their
plans in many of the same areas; and (10) moreover it is important to
recognize that the Results Act anticipates that the process of
developing an effective planning process and performance plan could take
several planning cycles.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-99-51
TITLE: Results Act: Observations on CFTC's Fiscal Year 2000 Annual
Performance Plan
DATE: 03/15/99
SUBJECT: Performance measures
Agency missions
Strategic planning
Internal controls
Congressional/executive relations
Data integrity
Interagency relations
IDENTIFIER: GPRA
Government Performance and Results Act
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. This text was extracted from a PDF file. **
** Delineations within the text indicating chapter titles, **
** headings, and bullets have not been preserved, and in some **
** cases heading text has been incorrectly merged into **
** body text in the adjacent column. Graphic images have **
** not been reproduced, but figure captions are included. **
** Tables are included, but column deliniations have not been **
** preserved. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
gg99051 RESULTS ACT Observations on CFTC's Fiscal Year 2000 Annual
Performance Plan
United States General Accounting Office
GAO Report to the Honorable Thomas W. Ewing, Chairman,
Subcommittee on Risk
Management and Specialty Crops, Committee on Agriculture, House of
Representatives
March 1999
GAO/GGD-99-51
March 1999 GAO/GGD-99-51
United States General Accounting Office Washington, D. C. 20548
General Government Division
B-281743
Page 1 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual Performance
Plan
GAO March 15, 1999 The Honorable Thomas W. Ewing Chairman,
Subcommittee on Risk Management
and Specialty Crops Committee on Agriculture House of
Representatives
Dear Mr. Chairman: As part of your continuing effort to ensure
that the Government Performance and Results Act of 1993 (the
Results Act) achieves its full promise, you asked us to review the
fiscal year 2000 annual performance plan of the Commodity Futures
Trading Commission (CFTC) to determine whether it complies with
the requirements of the Results Act. In October 1998, we testified
on the results of our review, 1 and you subsequently asked that we
provide you with a report on our results.
This report provides our observations on CFTC's fiscal year 2000
annual performance plan. Successful implementation of a
performance- based management system, as envisioned by the Act,
represents a significant challenge that requires sustained agency
attention. Our observations are intended to assist CFTC in its
continuing efforts to improve its plan. As a result, our report
focuses on the areas in which CFTC could improve future plans not
on areas in which CFTC has already made significant progress.
CFTC actions show a good- faith effort and commitment both to
comply with the Results Act and meet congressional expectations
for information about the agency's performance goals, including
how the agency will accomplish the goals and measure their
results. However, CFTC could improve future performance plans in
several areas to make the plans more useful to congressional and
other decisionmakers. First, CFTC could improve its performance
goals, 2 measures, 3 and targets to provide a clearer
1 Results Act: Observations on CFTC's Annual Performance Plan
(GAO/T-GGD-99-10, Oct. 8, 1998). 2 Performance goals, as defined
in the Results Act, are levels of performance that are expressed
as tangible, measurable objectives against which actual
achievement can be compared; they include goals expressed as
quantitative standards, values, or rates.
3 Performance measures are tabulations, calculations, recordings
of activity or effort, or assessments of results that are compared
to an intended purpose with the objective of assessing progress
toward achieving performance goals. CFTC's plan uses the term
performance indicator instead of performance measure. We use the
term performance measure in this report. Results in Brief
B-281743 Page 2 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
picture of its intended performance. Second, CFTC could better
connect its mission, goals, and activities to demonstrate more
fully how it plans to chart annual progress toward achieving its
strategic goals. Third, CFTC could more fully address crosscutting
efforts that involve other federal agencies to better ensure that
the overall effectiveness of federal efforts is maximized. Fourth,
CFTC could discuss in greater detail the strategies and resources
used to achieve its performance goals to better enable
decisionmakers to assess their reasonableness. Fifth, CFTC should
describe the specific means for verifying and validating the
quality of performance information and ensuring the reliability of
such information. Finally, CFTC could link past accomplishments to
performance goals to enhance the plan's clarity.
Although CFTC could improve its plan in these areas, we have found
that other federal agencies, including federal financial
regulators, could improve their plans in many of the same areas. 4
Moreover, it is important to recognize that the Results Act
anticipates that the process of developing an effective planning
process and performance plan could take several planning cycles.
CFTC, an independent agency created by Congress in 1974,
administers the Commodity Exchange Act (CEA), as amended. 5 The
principal purposes of the CEA are to protect the public interest
in the proper functioning of the futures and option markets' price
discovery and risk- shifting functions. In administering the CEA,
CFTC is responsible for fostering the economic utility of the
markets by encouraging their efficiency, monitoring their
integrity, and protecting market participants from abusive trade
practices and fraud.
In recent years, agencies have moved toward a performance- based
approach to management. Congress enacted the Results Act in 1993
as part of a framework of reform legislation that included the
Chief Financial Officers Act and information technology
legislation, such as the ClingerCohen Act of 1996, to instill
performance- based management in the federal government. The
Results Act seeks to improve the management of federal programs,
including their efficiency and effectiveness, by establishing a
system under which agencies set goals for program performance and
measure their results. The Act is intended to shift the focus of
government decisionmaking and accountability away from a
preoccupation with
4 See the listing of related GAO products at the end of this
report. 5 7 U. S. C. 1- 25. Background
B-281743 Page 3 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
activities such as the number of market surveillance reports
prepared to a focus on the results of those activities, such as
protecting the economic functioning of the commodity futures and
option markets.
Under the Results Act, strategic plans are the starting point for
setting goals and measuring progress toward them. The Results Act
requires virtually every executive agency to develop a strategic
plan, covering a period of at least 5 years forward from the
fiscal year in which the plan is submitted. In September 1997,
CFTC formally submitted its fiscal years 1997- 2002 strategic plan
to Congress and the Office of Management and Budget (OMB). This
plan established three strategic goals: (1) protect the economic
functions of the commodity futures and option markets; (2) protect
market users and the public; and (3) foster open, competitive, and
financially sound markets.
The Results Act also requires a federal agency to prepare an
annual performance plan covering the program activities set out in
its budget. In establishing the requirement for a performance
plan, the Results Act establishes the first statutory link between
an agency's budget request and its performance- planning efforts.
The performance plan is to reinforce the connections between the
long- term strategic goals outlined in the agency's strategic plan
and the daily activities of program managers and staff. CFTC
submitted its initial performance plan for fiscal year 2000 to OMB
in September 1998. 6 It is the second performance plan that CFTC
has prepared under the Act.
Finally, the Results Act requires executive agencies to prepare
annual reports on program performance for the previous fiscal
year. The performance reports are to be issued by March 31 each
year, with the first report (for fiscal year 1999) to be issued by
March 31, 2000. In each report, the agency is to compare its
performance against its goals, summarize findings of program
evaluations completed during the year, and describe the actions
needed to address any unmet goals.
To develop our observations, we compared the CFTC fiscal year 2000
annual performance plan to the Results Act requirements, CFTC
fiscal year 1999 annual performance plan, and CFTC fiscal years
1997- 2002 strategic plan. We also discussed these plans with CFTC
staff in Washington, D. C. In addition, our observations were
based on our knowledge of the agency's
6 According to OMB guidance, initial performance plans are due to
OMB with the agencies' budget requests. Final annual performance
plans are to be sent to Congress, following the transmittal of the
President's budget. Scope and
Methodology
B-281743 Page 4 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
operations and programs; past reviews of CFTC; and results of our
work on other agencies' performance plans and the Results Act.
The criteria we used to assess whether CFTC's annual performance
plan complied with the requirements of the Results Act were the
Act itself; OMB guidance on preparing strategic and performance
plans (OMB Circular A- 11, Part 2); and our guidance on assessing
agency performance plans. 7 Specifically, we used the criteria and
questions contained in our guidance to determine whether the
performance plan met the requirements of the Act, identify
strengths and weaknesses in the plan, and assess the plan's
usefulness for congressional oversight. We also assessed the
plan's discussion of actual performance data for fiscal year 1998.
We did our work between September 1998 and February 1999 at CFTC
in Washington, D. C., in accordance with generally accepted
government auditing standards. We requested comments on a draft of
this report from CFTC. These comments are discussed near the end
of this letter and reprinted in appendix I.
The Results Act requires that annual performance plans (1)
establish performance goals that define the levels of performance
to be achieved; (2) express the performance goals in an objective,
quantifiable, and measurable form; (3) establish performance
measures to be used in assessing progress toward achieving the
goals; and (4) provide a basis for comparing actual program
results with the established goals. CFTC's performance plan takes
important steps toward addressing these requirements by, among
other things, providing performance goals, measures, and targets
that reflect CFTC's mission and long- term strategic goals.
However, the plan could provide a clearer picture of intended
performance and be of greater use in assessing progress toward
achieving intended performance if performance goals, measures, and
targets were improved. Specifically, CFTC could improve its plan
by (1) making its performance goals and measures more results-
oriented and providing goals for all measures; (2) making some
performance goals self- measuring 8 and others more objective; (3)
replacing, restating, or deleting certain 7 See Agencies' Annual
Performance Plans Under the Results Act: An Assessment Guide to
Facilitate Congressional Decisionmaking (GAO/ GGD/ AIMD- 10.1. 18,
Feb. 1998) and The Results Act: An Evaluator's Guide to Assessing
Agency Annual Performance Plans (GAO/ GGD- 10. 1. 20, Apr. 1998).
8 Self- measuring goals are expressed objectively and quantifiably
and thus do not require the use of additional measures. According
to OMB guidance, the Act allows an agency to define a performance
goal in a way that is not self- measuring. If a goal is not self-
measuring, one cannot determine whether the goal was achieved by
objectively comparing actual performance to the goal. Therefore,
for a goal that is not self- measuring, the plan must include one
or more performance measures that set out specific, measurable
values or characteristics. CFTC's Performance
Plan Could Provide a Clearer Picture of Intended Performance
B-281743 Page 5 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
performance measures; and (4) establishing baselines against which
annual targets could be compared.
Results- oriented, or outcome, goals and measures provide the
clearest picture of intended and actual performance. Such goals
and measures focus on the intended result, effect, or consequence
that is expected to occur from carrying out a program or activity.
We recognize that establishing outcome measures is particularly
challenging for regulatory agencies as they move from a focus on
the activities that they undertake to the results that they are
trying to achieve. However, a key shortcoming of CFTC's
performance plan is that it relies predominantly on output
measures that describe completed activities, not program results.
9 In our October 1997 testimony before this Subcommittee, we
highlighted a similar problem with CFTC's strategic plan. 10
CFTC's plan has 31 performance goals and 228 performance measures.
About two- thirds of the goals do not have outcome measures to
describe intended results. Of the plan's 228 measures, 184 are
output measures that capture program outputs such as the number of
meetings attended and number of reports and research projects
completed. These measures are weighted toward measuring the
quantity of completed activities, rather than the quality, cost,
or timeliness of performance outcomes. In addition, 15 of the
performance measures are input measures that capture workload
inputs such as the number of reports that CFTC received from
futures firms and exchanges. Although such information is helpful
for defining the amount of work that must be performed, it has
limited utility in conveying the expected outcome to result from
such work. As previously mentioned, the Results Act is intended to
shift the focus of government decisionmaking and accountability
away from a preoccupation with completed activities to a focus on
the results of such activities.
The focus of CFTC's performance plan on output measures appears to
flow from CFTC's strategy of deriving performance goals from
program activities. The agency's reliance on program activities to
develop performance goals may be counterproductive because it
appears to focus attention on completed activities and not on the
results of such activities. For example, one of the plan's program
activities is to monitor the Internet
9 OMB guidance emphasizes that outcome goals should be included in
a performance plan whenever possible. A performance plan should
include outcome goals when the related goals are to be achieved in
the year covered by the plan. The guidance notes that most plans
will supplement outcome goals with output measures. Output
measures can be the predominant goals and measures in an annual
performance plan under certain circumstances.
10 Results Act: Observations on CFTC's Strategic Plan (GAO/T-GGD-
98-17, Oct. 22, 1997). Goals and Measures Could
Be More Results- Oriented and Complete
B-281743 Page 6 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
and other media for fraudulent activities and other possible
violations of the CEA. The performance goal for this activity is
to aggressively identify, investigate, and take action against
individuals engaged in fraudulent Internet and media activities.
The measure for the goal is the number of referrals to enforcement
authorities generated from Internet and media monitoring an output
of the activity, not an outcome.
According to CFTC officials, the plan's performance goals are
basically restatements of their associated performance activities,
which support the plan's outcome objectives. The officials said
that a CFTC task force examined the plan's 50 program activities
last year and reduced the number of activities to 44 by modifying
and consolidating several of them. They told us that, in the
upcoming year, the task force will examine CFTC's performance
goals and targets with the objective of making them more results-
oriented.
CFTC could use the narrative sections of its performance plan that
discuss the agency's fiscal year 1998 accomplishments to identify
more resultsoriented performance goals. For example, the
Enforcement Program accomplishment section discusses the
effectiveness of its quick- strike ability the ability to file
injunctive actions quickly after detecting fraud to obtain, among
other things, timely injunctive relief and enhance the possibility
that customer funds will be recovered. This accomplishment section
describes cases, filed within days or weeks of CFTC's discovering
an illegal activity, that stopped fraud at an early stage and
preserved customer funds. CFTC's performance measures could be
made more results- oriented by replacing measures, such as reports
on activities related to bringing injunctive actions and
sanctioning violators, with more outcome- related measures, such
as the percentage of quick- strike cases filed within a certain
number of days of starting an investigation that resulted in
sanctions and the percentage of funds recovered.
CFTC could also learn from the plans of other federal financial
regulators that are attempting the transition to results- oriented
goals. For example, the National Credit Union Administration is
developing new outcome performance goals. One of its outcome goals
is to ensure that federally insured credit unions are adequately
capitalized. A performance goal is to reduce the percentage of
federally insured credit unions that are undercapitalized by 10
percent, from 372 to 335.
Finally, CFTC could also improve its performance plan and make it
more complete by providing performance goals for all of its
program activities that have performance measures. CFTC's plan has
15 program activities
B-281743 Page 7 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
that have no corresponding performance goals but have performance
measures. For example, no performance goal exists for the activity
of reviewing the adequacy of self- regulatory organization
disciplinary actions. Without performance goals, it is not clear
what performance is expected. As discussed below, opportunities
exist for CFTC to develop selfmeasuring performance goals for some
of these activities and their measures.
Although not required by the Results Act, CFTC could redefine some
performance goals so that they are self- measuring, thereby
reducing the number of measures and the complexity of the plan.
Currently, 29 of CFTC's 31 performance goals are stated as
abstract goals that is, as goals requiring that specific
performance measures be defined to assess progress toward their
achievement.
Performance goals that can be redefined so that they are self-
measuring generally have just one measure or two or more measures
that can be combined. CFTC's plan has nine activities or goals
that have only one performance measure, and opportunities exist to
redefine some of them in self- measuring terms. For example, the
performance goal for the activity of reviewing the adequacy of
self- regulatory organization disciplinary actions could be made
self- measuring by stating it the following way: On an annual
basis, review a defined percentage of self- regulatory
organization disciplinary actions to ensure compliance with CFTC
standards. This approach, which has been taken by other federal
financial regulators, such as the National Credit Union
Administration and the Federal Deposit Insurance Corporation,
reduces the amount of subjectivity in interpreting performance
goals and more clearly defines performance expectations. For
example, the Federal Deposit Insurance Corporation has the
following performance goal: Market 80 percent of a failing
financial institution's assets based on book value at the time of
resolution or within 90 days.
CFTC could also improve its performance plan by making its
performance goals more objective. According to OMB guidance, to
the extent possible, goals should not require subjective
considerations. CFTC's plan has 14 performance goals that require
subjective consideration in determining whether they have been
met. For example, one such goal is defined as follows: Bring
important cases (including matters involving ongoing conduct and
complex transactions) aggressively. The performance goal does not
define what an important case is or what it means to aggressively
bring a case. Although the goal has 11 performance measures
including the numbers of cases filed, counts, and settlements none
clearly define the terms important or aggressively. Some Goals
Could Be Made
Self- Measuring and Others Could Be Made More Objective
B-281743 Page 8 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
CFTC's performance plan has 31 performance goals with 228
performance measures to address them. CFTC could replace, restate,
or delete performance measures for certain performance goals to
enhance its ability to assess the progress made in achieving
performance goals.
First, CFTC could replace certain performance measures to better
capture key aspects of the performance the measures are trying to
assess. CFTC's plan has six performance goals that are partly
defined in terms of timeliness but do not have measures that
capture timeliness. One such goal is to review every designation
application and rule change request within 10 to 45 days. The
goal's performance measures are the number of designation
applications processed and the number of contract market rule
changes processed. Because neither measure captures the amount of
time it takes to process applications or changes, one option for
improving the plan is to replace these measures with one measure
that captures the percentage of applications and changes processed
in 45 days or less.
Second, CFTC could restate or delete certain performance measures,
because they do not appear to be clearly related to their
performance goals and/ or appear to be unduly affected by external
factors. For example, the number of active futures markets is used
to measure, in part, two goals: (1) Identify traders who can
influence futures prices and (2) determine whether traders are
influencing futures prices. Because the number of futures markets
is largely determined by exchanges and other external factors,
this number has little direct bearing on the two goals and, thus,
could be deleted as a measure. Another example of such a measure
is the number of requests for assistance from foreign authorities,
which is used, in part, to measure the following goal: Maximize
the enforcement program's effectiveness by cooperating with
domestic and foreign authorities in gathering information. This is
an input measure that captures action taken by foreign regulators,
not CFTC.
Third, CFTC could restate or delete certain performance measures
because the measures might have limitations that preclude them
from accurately capturing intended performance or might promote
unintended consequences. For example, one of the plan's
performance goals is to conduct important investigations and refer
potential violations to other authorities, as appropriate. The
performance measures for this goal include the number of documents
obtained through subpoena or inspection, number of witnesses from
whom testimony was taken, and number of witnesses interviewed.
These output- oriented measures could provide an incentive for
staff to conduct more interviews, take more testimonies, and
obtain more documents than necessary, which could add Certain
Performance
Measures Could Be Replaced, Restated, or Deleted
B-281743 Page 9 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
cost and time to investigations without necessarily contributing
commensurately to their success.
Although CFTC's performance plan includes annual targets for
performance goals, it could make such information more useful by
providing baselines, or a context, for assessing the
reasonableness and appropriateness of such targets. As we found in
our review of other performance plans, agencies that go beyond the
requirements of the Results Act and include baseline or trend data
for their goals provide a more informative basis for assessing
expected performance.
CFTC's performance plan provides many quantitative annual targets
for each fiscal year from 1998 through 2000. Many targets are set
at the same performance level as or at a lower performance level
than in prior fiscal years without an explanation. For example,
the performance measures for assessing sanctions under CFTC's
first strategic goal are the same in each year covered by the plan
for the number of cease- and- desist orders, registration
sanctions, and trading prohibitions. However, the bases for
setting the specific targets and the contributions of these
targets to the outcome objective are not readily apparent. 11
Without this contextual information, it is not clear whether
CFTC's output- oriented performance targets are reasonable and
appropriate.
Consistent with the Results Act, CFTC's performance plan attempts
to show the relationship between the agency's annual performance
goals and its fiscal years 1997 through 2002 mission and strategic
goals. To do so, the plan uses tables that connect each strategic
goal to its accompanying set of performance goals, measures, and
annual targets. The plan also ranks CFTC's outcome objectives by
dollars budgeted, which is a starting point for providing useful
information about its priorities. However, the plan could better
connect mission, goals, and activities by more fully demonstrating
how CFTC will chart annual progress toward achieving its long-
term strategic goals.
The performance plan's presentation of many separate program
areas; program activities; and performance goals, measures, and
targets makes it difficult to link CFTC's mission and strategic
goals to performance goals across the entire agency. The plan's 31
performance goals and 228 performance measures support 3 strategic
goals and 9 strategic objectives, covering 5 program areas.
Although the plan's ranking of outcome
11 The outcome objective is to foster futures and option markets
that accurately reflect the forces of supply and demand for the
underlying commodity and are free of disruptive activity.
Baselines Could Be
Established Against Which to Compare Annual Targets
Mission, Goals, and Activities Could Be Better Connected
B-281743 Page 10 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
objectives offers a useful perspective on CFTC's priorities, given
the high level of complexity, it is difficult to (1) identify the
agency's key priorities among the many goals and measures, (2)
differentiate efforts to meet these priorities, and (3) understand
what will be achieved if all of the performance goals are met.
Similarly, as we found with other agency performance plans, CFTC's
plan associates one performance goal with multiple program
activities and strategic goals. For example, the performance goal
assess sanctions that are remedial and deter violators is
associated with three different program activities and three
different strategic goals. Additionally, the measures and targets
for this performance goal differ with each program activity and
strategic goal. Associating one performance goal with multiple
program activities and strategic goals makes it difficult not only
to determine whether all activities are substantially covered but
also to understand how specific program activities are intended to
contribute to CFTC's strategic goals.
The Results Act seeks to ensure that crosscutting goals of federal
programs are consistent; strategies are mutually reinforcing; and,
as appropriate, progress is assessed through the use of common
performance measures. By accomplishing these goals, agencies could
avoid wasting scarce resources and could better ensure that the
overall effectiveness of federal efforts is maximized. OMB
guidance tasks performance plans with identifying those
performance goals that are being mutually undertaken with other
federal agencies in support of programs or activities of a
crosscutting nature. 12 It encourages agencies to go beyond
coordination and develop common performance goals and measures for
related programs. CFTC's plan recognizes the need to address
crosscutting efforts. However, the agency could address such
efforts in more substantive terms if it worked with the cognizant
federal agencies to develop performance goals and measures that
better reflect the nature and extent of their common efforts.
CFTC's performance plan briefly discusses CFTC's need to work with
other U. S. financial regulators through, among other means, the
President's Working Group on Financial Markets. 13 However, the
related
12 OMB Circular A- 11 guidance states that, at a minimum, the
performance plan should indicate those programs or activities that
are being undertaken with other agencies to achieve a common
purpose or objective. An agency should also review the performance
plans of other agencies participating with it in a crosscutting
program or activity to ensure that goals and measures are
consistent and harmonious.
13 The President's Working Group on Financial Markets was created
following the October 1987 stock market crash to address issues
concerning the competitiveness, integrity, and efficiency of the
Crosscutting Efforts
That Involve Other Agencies Could Be More Fully Addressed
B-281743 Page 11 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
performance goals and measures do not substantively address the
type of crosscutting performance that the Working Group was
created to address. For example, CFTC's performance goal covering
the Working Group is for CFTC to contribute to the performance of
the group. The measure for this goal is the number of meetings
attended, and the fiscal year 2000 target is two meetings. Neither
the goal nor its measure captures the substantive work of the
Working Group's efforts.
As supported by OMB guidance, CFTC could strengthen its
performance plan by participating with the members of the Working
Group and other federal regulators involved in crosscutting
programs to develop common performance goals and measures. For
example, the continued growth and development of the over- the-
counter (off- exchange) derivatives market has raised several
potential regulatory concerns that affect CFTC and other members
of the Working Group. In addition, the potential need to develop a
financial markets contingency plan to address the Year 2000
computerdating problem 14 could involve coordination among CFTC
and other federal financial regulators.
Additionally, CFTC's performance plan could be expanded to include
performance goals and measures related to other crosscutting
efforts, such as those identified in its fiscal year 2000 budget
justification. These efforts include CFTC's sharing of information
with other financial regulators; working with the U. S. Department
of Agriculture on a risk- management education program;
contributing to a Department of the Treasury initiative that
encourages global financial stability; and engaging in cooperative
enforcement efforts with the Department of Justice, Federal Bureau
of Investigation, Federal Reserve Board, Federal Trade Commission,
Securities and Exchange Commission, and U. S. Postal Inspection
Service.
financial markets. The Secretary of the Treasury chairs the group,
and other members include the chairs of CFTC, the Federal Reserve
System, and the Securities and Exchange Commission.
14 The Year 2000 computer- dating problem relates to the need for
computer systems to be changed to accommodate dates beyond the
year 1999.
B-281743 Page 12 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
Consistent with the Results Act, CFTC's performance plan provides
important information on how strategies and resources will be used
to achieve its goals. 15 However, the plan's discussion could be
expanded to better enable congressional and other decisionmakers
to judge the reasonableness and appropriateness of CFTC's
strategies and anticipated resource deployment. Specifically, the
plan could be strengthened by (1) describing more fully the
strategies needed to achieve performance goals, including those
for addressing both internal management challenges and external
factors, and (2) discussing in more detail the resources required
to achieve each performance goal.
Consistent with the Results Act and OMB guidance, CFTC's
performance plan attempts to address the strategies that CFTC will
use to achieve its performance goals. Aligning its discussion with
CFTC's strategic goals, the plan briefly describes the major
activities, or operational processes, of each program in relation
to its performance goals and measures. In a few cases, the plan
also discusses skills or technologies that programs will use in
relation to performance goals and measures. The plan could be made
more useful by providing additional information on the skills and,
if appropriate, technologies used in connection with operational
processes to achieve program goals.
CFTC's fiscal years 1997- 2002 strategic plan identifies several
internal management challenges, but CFTC's performance plan does
not include a strategy consisting of specific goals and measures
to address them. These challenges include diminishing resources,
recruiting and retaining qualified professionals, remaining
abreast of current technology, and remaining educated and informed
as innovation changes the industry. According to OMB guidance,
performance goals for management problems should be included in
the annual plan, particularly goals for those problems whose
resolution is mission- critical and which could impede achievement
of program goals. To better respond to the intent of the Results
Act, CFTC could add agencywide performance goals to the plan to
address these challenges or incorporate these challenges within
the plan's existing structure of performance goals and measures.
For example, CFTC could add a performance goal that covers
recruiting and retaining a highly qualified workforce to handle
complex issues facing the agency.
CFTC's performance plan also discusses some of the external
factors that can affect its performance goals, but it does not
identify the strategies that
15 Typically, strategies cover an agency's operational processes,
skills, and technology, while resources cover human, capital,
funding, and other resources. Strategies and
Resources Used to Achieve Goals Could Be Discussed in Greater
Detail
The Strategies Needed to Achieve Goals Could Be Described More
Fully
B-281743 Page 13 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
the agency plans to use to address them. In our testimony on
CFTC's fiscal year 1997- 2002 strategic plan, we highlighted this
area, noting that CFTC could strengthen its plan by describing how
external factors are linked to particular goals and how a
particular goal could be affected by external factors. Although
the Results Act does not require performance plans to discuss the
specific impact of external factors on achieving performance
goals, such information can provide a better context for assessing
CFTC's expected performance. Moreover, OMB guidance notes that
agencies may choose to include information on external factors in
their annual plan when the factors may bear directly on goal
achievement for the fiscal year covered by the plan.
According to OMB guidance, a performance plan should discuss the
resources that will be applied to achieve each of the agency's
performance goals. Using tables and graphics, CFTC's plan shows
the amount of budget funding and the number of full- time-
equivalent employees that will be needed to achieve each strategic
goal and related outcome objectives. CFTC's plan could be improved
by addressing OMB guidance that also calls for plans to describe
the resources required to achieve each performance goal.
At the October 1998 hearing, the CFTC Chairperson testified that
the agency is working to improve its capacity to account for
budgetary resources by activity, outcome objective, and
performance goal. She said that the agency's financial reporting
system was being modified to provide budget data on a program and
performance goal basis. She also said that CFTC plans to link its
financial reporting system to the activities of its performance
plan and, if successful, use it to formulate its fiscal year 2001
budget. Resources Required to
Achieve Performance Goals Could Be Discussed in Greater Detail
B-281743 Page 14 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
Although required by the Results Act and OMB guidance, CFTC's
fiscal year 2000 performance plan does not describe the procedures
that the agency will use to verify 16 and validate 17 that
performance information is sufficiently complete, accurate, and
consistent. These procedures can include internal controls over
data collection, maintenance, and entry. External assessments,
such as audits, evaluations, and peer reviews, are other important
procedures for verifying and validating performance information.
In addition, the plan does not, as required by the Act, discuss
the extent to which the performance information and the means for
collecting, maintaining, and analyzing it are reliable. CFTC's
performance plan should be expanded to address these requirements.
To assess progress toward achieving its goals, CFTC will need to
collect information on the over 200 performance measures in its
plan. For some of these measures, the amount of information to be
collected is voluminous and covers activities across CFTC
headquarters and regional offices. Errors can occur in collecting,
maintaining, processing, and reporting such information
potentially introducing bias and resulting in inaccurate estimates
of program performance. As a result, CFTC should have procedures
for ensuring that its performance information is free of
significant levels of error and that bias is not introduced. Such
procedures can include internal controls over data collection,
maintenance, and entry as well as audits, evaluations, and peer
reviews.
At the October 1998 hearing, the CFTC Chairperson testified that
the agency is adapting its Quarterly Objectives review a public
document that briefly describes each program's priorities and
accomplishments to match the annual performance plan's goals,
measures, and targets. According to the Chairperson, the goal is
to use the performance data in its Quarterly Objectives review to
evaluate whether CFTC is meeting its performance plan targets.
16 Verify means to check or test performance data to reduce the
risk of using data that contain significant errors. 17 Validate
means to ensure that the data are free of significant levels of
error and that bias is not introduced. The Means for
Verifying and Validating Performance Information and Ensuring Its
Reliability Should Be Described
B-281743 Page 15 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
Although not required by the Results Act or OMB guidance, CFTC's
performance plan presents its fiscal year 1998 accomplishments.
CFTC could improve the plan's clarity by linking past
accomplishments to performance goals. CFTC's plan provides a
narrative section under each strategic goal that discusses the
agency's fiscal year 1998 accomplishments by program. For example,
in the first strategic goal section, the plan discusses, among
other things, the major accomplishments of the Market
Surveillance, Analysis, and Research Program. These
accomplishments include conducting daily surveillance of 157
futures markets, collecting and analyzing about 7,980 large trader
reports, and preparing about 2,538 weekly surveillance reports. In
some cases, CFTC links the discussion of its past accomplishments
to specific outcome objectives. However, CFTC does not link the
accomplishments to the performance goals described in its fiscal
year 1999 annual performance plan. Without this linkage, it is
difficult to determine the progress that CFTC made in achieving
its performance goals and, in turn, strategic goals.
Our observations on CFTC's fiscal year 2000 annual performance
plan are intended to assist the agency in improving future plans
so that they can be more valuable tools for congressional and
other decisionmakers. Although we identified opportunities for
CFTC to improve its plan as we have done for many other federal
agencies the Results Act anticipated that the process of
developing an effective planning process and performance plan
could take several planning cycles. Additionally, the agency's
ongoing efforts reflect a commitment to fully complying with the
Act.
In the upcoming year, CFTC plans to examine its performance goals
to make them more results- oriented, use performance data to
evaluate whether performance plan targets are being met, and
improve its accounting system to track budgetary resources by
performance goals. To further enhance its plan, CFTC could focus
additional attention on (1) improving its performance goals,
measures, and targets to provide a clearer picture of intended
performance; (2) demonstrating more fully how progress toward
achieving strategic goals will be charted; (3) addressing
crosscutting efforts in more substantive terms; (4) linking
strategies and resources more clearly; and (5) providing
confidence that performance data will be sufficiently reliable.
CFTC could also enhance the plan's clarity by linking past
accomplishments to performance goals.
Written comments from CFTC on a draft of this report are contained
in appendix I. CFTC said it appreciated the suggestions and
recommendations made in our report and by others. It also said it
is Past Accomplishments
Could Be Linked to Performance Goals to Enhance the Plan's Clarity
Conclusions Agency Comments
B-281743 Page 16 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual
Performance Plan
committed to improving its future plans to make them as useful as
possible.
We are sending copies of this letter to Representative Gary A.
Condit, the Ranking Minority Member of your Subcommittee; and the
Honorable Brooksley Born, Chairperson of CFTC. We will also make
copies available to others on request.
This report was prepared under the direction of Cecile Trop,
Assistant Director. Other major contributors included Sharon
Caudle and Richard Tsuhara. Please contact me at (202) 512- 8678
or Ms. Trop at (312) 220- 7600 if you or your staff have any
questions.
Sincerely yours, Richard J. Hillman Associate Director, Financial
Institutions
and Markets Issues
Page 17 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual Performance
Plan
Appendix I Comments From the Commodity Futures Trading Commission
Page 18 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual Performance
Plan
Appendix I Comments From the Commodity Futures Trading Commission
Page 19 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual Performance
Plan
Related GAO Products
Page 20 GAO/GGD-99-51 CFTC's Fiscal Year 2000 Annual Performance
Plan
The Results Act: Observations on the Federal Reserve's 1998- 99
Biennial Performance Plan (GAO/GGD-99-9R, Nov. 9, 1998).
The Results Act: Observations on NCUA's Annual Performance Plan
(GAO/GGD-98-192R, Sept. 15, 1998).
The Results Act: Observations on FDIC's Annual Performance Plan
(GAO/GGD-98-190R, Sept. 15, 1998).
The Results Act: Observations on OCC's Annual Performance Plan
(GAO/GGD-98-189R, Sept. 15, 1998).
Managing for Results: An Agenda to Improve the Usefulness of
Agencies' Annual Performance Plans (GAO/ GGD/ AIMD- 98- 228, Sept.
8, 1998).
Results Act: Observations on Treasury's Fiscal Year 1999 Annual
Performance Plan (GAO/GGD-98-149, June 30, 1998).
Managing for Results: Experiences of Selected Credit Programs
(GAO/GGD-98-41, Feb. 19, 1998).
Managing for Results: Agencies' Annual Performance Plans Can Help
Address Strategic Planning Challenges GAO/GGD-98-44, Jan. 30,
1998).
The Results Act: Observations on Draft Strategic Plans of Five
Financial Regulatory Agencies (GAO/T-GGD-97-164, July 29, 1997).
Managing for Results: Regulatory Agencies Identified Significant
Barriers to Focusing on Results (GAO/GGD-97-83, June 24, 1997).
The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).
Managing for Results: Analytic Challenges in Measuring Performance
(GAO/ HEHS/ GGD- 97- 138, May 30, 1997).
Ordering Information The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be
sent to the following address, accompanied by a check or money
order made out to the Superintendent of Documents, when necessary.
VISA and MasterCard credit cards are accepted, also. Orders for
100 or more copies to be mailed to a single address are discounted
25 percent.
Order by mail: U. S. General Accounting Office P. O. Box 37050
Washington, DC 20013
or visit: Room 1100 700 4 th St. NW (corner of 4 th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders may also be placed by calling (202) 512- 6000 or by using
fax number (202) 512- 6061, or TDD (202) 512- 2537.
Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512- 6000 using a
touch- tone phone. A recorded menu will provide information on how
to obtain these lists.
For information on how to access GAO reports on the INTERNET, send
e- mail message with info in the body to:
info@ www. gao. gov or visit GAO's World Wide Web Home Page at:
http:// www. gao. gov
United States General Accounting Office Washington, D. C. 20548-
0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. G100
(233595)
*** End of document. ***