Results Act: Observations on Treasury's Fiscal Year 1999 Annual
Performance Plan (Letter Report, 06/30/1998, GAO/GGD-98-149).
The Government Performance and Results Act of 1993 requires federal
agencies to prepare annual performance plans that set goals against
which the agencies' actual performance can be measured. This report
provides GAO's observations on the Treasury Department's fiscal year
1999 annual performance plan that was submitted to Congress. GAO found
that Treasury's annual performance plan appropriately links its annual
performance goals and measures to its strategic goals. Although the plan
provides useful information for congressional decisionmakers and other
stakeholders, it does not fully present information that (1) reflects
the intended performance across the agency, (2) describes how strategies
relate to attainment of goals, and (3) assures readers that performance
results and data are credible. The plan GAO reviewed was Treasury's
first one under the Results Act. Developing a plan that fully meets all
the Act's criteria and related guidance will be a challenge because
developing measures and collecting reliable data in some important
areas, such as taxpayer burden, are difficult to do. Treasury's plan
could be enhanced by explicitly discussing the agency's strategy to
improve its performance measurement systems and data and by describing
Treasury's interim plans to measure performance in critical areas.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-98-149
TITLE: Results Act: Observations on Treasury's Fiscal Year 1999
Annual Performance Plan
DATE: 06/30/1998
SUBJECT: Performance measures
Accountability
Agency missions
Interagency relations
Strategic planning
Data integrity
Customs administration
Data collection
Congressional/executive relations
IDENTIFIER: GPRA
Government Performance and Results Act
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
GAO/GGD-98-149
Cover
================================================================ COVER
Report to the Secretary, Department of the Treasury
June 1998
RESULTS ACT - OBSERVATIONS ON
TREASURY'S FISCAL YEAR 1999 ANNUAL
PERFORMANCE PLAN
GAO/GGD-98-149
Treasury's Annual Performance Plan
(268856)
Abbreviations
=============================================================== ABBREV
ATF - Bureau of Alcohol, Tobacco, and Firearms
CFO - Chief Financial Officers
CFS - consolidated financial statements
FMS - Financial Management Service
FTE - full-time equivalent
IRS - Internal Revenue Service
IT - information technology
NTC - National Tracing Center
OCC - Office of the Comptroller of the Currency
OTS - Office of Thrift Supervision
OMB - Office Management and Budget
SSA - Social Security Administration
Letter
=============================================================== LETTER
B-280185
June 30, 1998
The Honorable Robert E. Rubin
The Secretary of the Treasury
Dear Mr. Rubin:
This report provides our observations on the Department of the
Treasury's fiscal year 1999 annual performance plan that was
submitted to Congress as required by the Government Performance and
Results Act of 1993 (Results Act). As you know, we were asked by
several Members of the House Majority Leadership to review the fiscal
year 1999 annual performance plans submitted by the 24 federal
agencies covered by the Chief Financial Officers (CFO) Act. In April
1998, we briefed the offices of our congressional requesters on
Treasury's plan, and we agreed with them that it would be useful to
provide our observations to you.
The Results Act requires federal agencies to prepare annual
performance plans covering the program activities set out in the
agencies' budgets, beginning with plans for fiscal year 1999. These
plans are to (1) establish performance goals to define levels of
performance to be achieved; (2) express those goals in an objective,
quantifiable, and measurable form; (3) briefly describe the
operational processes, skills, and technology and the human, capital,
information, or other resources required to meet the goals; (4)
establish performance measures for assessing the progress toward
achievement of the goals; (5) provide a basis for comparing actual
program results with the established goals; and (6) describe the
means to be used to verify and validate measured values.
For purposes of our review, these six requirements of the Results Act
for the annual performance plans were collapsed into the following
three core questions:
-- To what extent does the agency's performance plan provide a
clear picture of intended performance across the agency?
-- How well does the agency's performance plan discuss the
strategies and resources the agency will use to achieve its
performance goals?
-- To what extent does the agency's performance plan provide
confidence that its performance information will be credible?
These questions are contained in our February 1998 congressional
guide and our April 1998 evaluators' guide for assessing performance
plans, which we used to do our review.\1 These guides integrated
criteria from the Results Act, its legislative history, the Office of
Management and Budget's (OMB) guidance for developing performance
plans (OMB Circular A-11, Part 2), a December 1997 letter to OMB from
several congressional leaders, and our other guidance on
implementation of the Results Act. We used the criteria and
questions contained in the guides to help us determine whether
Treasury's plan met the requirements of the Act, to identify
strengths and weaknesses in the plan, and to assess the plan's
usefulness for executive branch and congressional decisionmakers.
As requested, we reviewed the Department of the Treasury's annual
performance plan for fiscal year 1999 that was submitted to Congress
in February 1998. Treasury's annual performance plan and budget
request for fiscal year 1999 has separate sections for each of the
Department's bureaus and offices.\2 We conducted a more in-depth
analysis of sections relating to the Internal Revenue Service (IRS);
United States Customs Service; Financial Management Service (FMS);
Bureau of Alcohol, Tobacco, and Firearms (ATF); Office of the
Comptroller of the Currency (OCC); and Office of Thrift Supervision
(OTS). These components, except for OCC and OTS, which do not
receive appropriated funds, represent about 88 percent of Treasury's
total budget request for fiscal year 1999.
We sent a draft of this letter to Treasury's Office of Strategic
Planning for review and comment. On May 28, 1998, Treasury officials
provided oral comments on that draft, which are discussed at the end
of this letter. We did our work from February 1998 through June
1998, according to generally accepted government auditing standards.
--------------------
\1 See Agencies' Annual Performance Plans Under the Results Act: An
Assessment Guide to Facilitate Congressional Decisionmaking
(GAO/GGD/AIMD-10.1.18, Feb. 1998, Version 1) and The Results Act:
An Evaluator's Guide to Assessing Agency Annual Performance Plans
(GAO/GGD-10.1.20, Apr. 1998, Version 1).
\2 Seventeen Treasury bureaus and major offices have annual
performance plans. Some smaller offices, including Joint Financial
Management Improvement Program and Community Adjustment and
Investment Program, also have plans.
BACKGROUND
------------------------------------------------------------ Letter :1
The Results Act is designed to improve the efficiency and
effectiveness of federal programs by establishing a system to set
goals for program performance and to measure results. Specifically,
the Act requires executive agencies to prepare multiyear strategic
plans, annual performance plans, and annual performance reports. The
strategic plan serves as the starting point and basic underpinning of
the performance-based management system and includes the agency's
mission statement and its long-term goals and objectives for
implementing the mission. Treasury submitted its first strategic
plan under the Results Act to Congress and the Director of OMB, as
required, by September 30, 1997. The annual performance plan links
the agency's day-to-day activities to its long-term strategic goals.
The first plans, covering fiscal year 1999, were submitted to OMB in
the fall of 1997 and to Congress after the President's budget in
February 1998. Finally, the first annual performance reports for
fiscal year 1999 are due to Congress and the President no later than
March 31, 2000. Performance reports are to include, among other
things, an evaluation of the agencies' progress toward achieving the
goals in their annual plans. These reports are to provide feedback
to federal managers, policymakers, and the public on the results
achieved each year.
The Treasury Department has responsibilities in key governmental
roles, including tax administrator, revenue collector, law enforcer,
and financial manager. Treasury also formulates and recommends
economic, financial, tax, and fiscal policies and manufactures coins
and currency. To carry out its diverse responsibilities, Treasury
houses more than a dozen bureaus and offices. For its fiscal year
1999 budget, Treasury requested about $12.301 billion and about
147,900 full-time equivalent (FTE) staff years.
Public sector organizations, like Treasury, are faced with demands to
be more effective and accountable for the results of their programs.
To meet such demands, Treasury began moving toward a
performance-based approach to management before the Results Act
requirements became mandatory. This is the third year that Treasury
has included performance goals derived from its strategic plan in its
budget request. Treasury's fiscal year 1999 performance plan under
Results Act requirements is combined with its budget request and
includes reports on performance goals for the past 2 fiscal years.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
Treasury's fiscal year 1999 annual performance plan partially meets
the criteria set forth in the Results Act and related guidance. One
of the strengths of the plan is that the annual performance goals and
measures are linked to the strategic goals in the bureaus' and
offices' strategic plans. Moreover, the plan generally provides a
clear connection between its performance goals and the program
activities in Treasury's budget request. With a few exceptions, the
plan covers each of Treasury's program activities as required by the
Results Act.
The plan could be improved to better meet the criteria set forth in
the Results Act and related guidance by presenting information on
performance goals and measures in a manner that would better reflect
intended or expected performance and achievements. While we
recognize that some output measures are necessary, we also believe
the plan could define Treasury's expected performance better if it
had more outcome goals and measures. Also, the plan does not
consistently include information across Treasury's bureaus and
offices on how the Department plans to coordinate its activities that
share a common purpose with activities in other agencies. IRS, for
example, is responsible for administering the tax code provisions
relating to several billion dollars of tax expenditures, such as the
earned income tax credit, the low-income housing credit, and the
research credit.\3 However, there is no discussion of how IRS intends
to coordinate with federal agencies that administer related direct
expenditure programs to develop performance goals pertaining to its
responsibilities.
The plan, which includes the budget justification, describes the
resources for carrying out the strategies to meet the criteria set
forth in the Results Act and related guidance. However, the
information in the plan on how the strategies relate to achieving the
goals did not always list strategies or adequately describe them.
Additional details on how Treasury plans to verify and validate
performance data, along with some discussion of how the effects of
data limitations are to be handled, would better assure Congress and
other stakeholders that the intended performance or results, if
achieved, are credible. We realize that developing measures and
collecting reliable data for some important areas of Treasury's
performance, such as taxpayer burden, are very difficult to do.
However, Treasury's plan could be enhanced by explicitly discussing
the Department's strategy to improve its performance measurement
systems and data and by describing Treasury's interim plans to
measure performance in critical areas.
We also believe that Treasury's plan would be more useful to Congress
and other stakeholders if it included performance goals to address
the significant management challenges and high-risk areas the
Department faces. The plan briefly acknowledges some of the major
management challenges and high-risk areas, but it does not have
performance goals that adequately address all of them.
--------------------
\3 Based on estimates by the Joint Committee on Taxation, the fiscal
year effect of tax expenditures in 1999 is $543.7 billion.
TREASURY'S PERFORMANCE PLAN
PROVIDES A PARTIAL PICTURE OF
INTENDED PERFORMANCE ACROSS THE
DEPARTMENT
------------------------------------------------------------ Letter :3
As the Results Act requires, the annual performance plan is to
provide a basis for an agency to compare actual results with
performance goals. To do this, the agency needs to set goals and
develop appropriate performance measures and show how it will use
them to assess performance across the agency. By showing the
relationship between the annual performance goals and the agency's
mission and strategic goals, an agency's performance plan can
demonstrate how the agency intends to make progress toward the
achievement of its strategic goals. An agency's performance plan
should also reflect and discuss the crosscutting nature of its
programs and how they will contribute to achieving performance
related to crosscutting functions.
DEFINING EXPECTED
PERFORMANCE
---------------------------------------------------------- Letter :3.1
Treasury's performance plan does not provide a succinct and concrete
statement of expected performance for subsequent comparison with
actual performance for several reasons. First, many of the annual
performance goals in Treasury's plan are necessarily abstract and not
directly measurable. IRS, for example, has established three
performance goals--improve customer service, increase compliance, and
increase productivity--for defining its intended performance. Each
of these broad goals is complemented with program-level measures to
assess progress toward achieving the three goals. IRS' performance
goal relating to improving customer service is particularly difficult
to quantify because achieving it implies that IRS can measure and
reduce taxpayer burden. IRS currently does not know how to
realistically measure taxpayer burden. Reliable data for measuring
burden do not exist because taxpayers normally do not track the time
they spend complying with their tax and filing obligations.
IRS recognizes the limitations of these goals on defining its
performance and is looking for alternatives. Because reducing
taxpayer burden affects IRS' ability to achieve its performance goals
and IRS' measure of taxpayer burden is not based on reliable data,
its performance measures based on burden may not be very useful.
However, devising ways to measure the burden that IRS influences and
developing reliable measures of taxpayer burden and the impact of
IRS' programs on burden will be challenging. IRS is not alone;
Treasury as a whole faces similar challenges.
Second, the quality of some measures in Treasury's plan could be
improved so that they directly relate to the performance goals. The
relationship between some measures and goals is not clear, making it
difficult to define the level of expected performance. Also, the
measures do not always appear to cover key aspects of performance.
Examples from OCC's plan illustrate this.
-- One of OCC's strategic goals is to "improve the efficiency of
bank supervision and reduc[e] burden by streamlining supervisory
processes." This strategic goal has three performance goals, and
each has a single indicator or measure. One such performance
goal is to "continue with the regulatory reinvention process to
improve efficiency and reduce unnecessary burden." The single
measure in the plan for this goal is "percentage [of] time
meeting the application processing time frames," with a
performance target of 95 percent in calendar year 1998.\4 This
measure only addresses application processing time frames and
does not clearly relate to the goal of continuing with the
regulatory reinvention process.
-- OCC has two measures for its performance goal to "support
efforts to foster a national bank charter that will effectively
compete with other financial service providers and continue to
meet the financial service needs of all types of customers."
However, these measures--"rating on customer satisfaction in
connection with the licensing process" and "average processing
time for analysis of customer complaints"--do not clearly relate
to the performance goal.
Third, Treasury's plan is also incomplete in that some of the
performance measures for its bureaus and offices are still being
developed and defined. For example, many IRS measures are coded
"TBD," or to be determined. For these proposed measures, IRS does
not have complete information, such as definitions, data sources,
level of detail, and data reliability. During fiscal year 1998, IRS
is working with OMB, a contractor, and others to develop a balanced
scorecard measurement system that is to evaluate IRS on customer
satisfaction, employee satisfaction, and business results.
Finally, many of the measures in Treasury's plan are output measures.
While output measures are expected to be in the plan, Treasury could
better convey its expected results and show how goals are to be
achieved by developing additional outcome measures and better
explaining how the outputs that are measured relate to the goals.
--------------------
\4 OCC operates on a calendar year basis.
CONNECTING MISSION, GOALS,
AND PROGRAM ACTIVITIES
---------------------------------------------------------- Letter :3.2
For the most part, the performance goals of the Department's bureaus
and offices are connected to their missions, strategic goals, and
program activities in the budget request. Specifically, the plan
contains tables that align the Departmentwide strategic goals, bureau
missions and strategic goals, and performance goals and measures.
However, the linkages between the program-level measures and
performance goals are not consistently clear.
IRS, for example, has tables that show the linkage between its
strategic goals or objectives and the Departmentwide strategic plan
and its performance goals and annual performance measures. However,
the plan does not discuss how the intended results of its many
performance measures will be assessed to indicate IRS' success in
achieving its performance goals. For example, the number of
individual refunds issued, paper processing accuracy rate, and number
of calls answered are 3 of the 19 performance measures under the goal
to improve customer service. The plan does not explain how any of
these measures should be rolled up to indicate progress toward
achieving the customer service goal. We recognize the difficulty IRS
faces in explaining this, especially since its performance goals are
necessarily abstract and not directly measurable. However, some
discussion of how IRS plans to evaluate progress toward achieving its
performance goals would help explain how the results of its
performance measures affect the attainment of its performance goals.
Although Customs' plan provides information to align its strategic
goals and performance goals, the information is not consistent.
Customs' plan has a table that shows the linkage between its
strategic goals and performance goals. Later in the plan, other
tables show the exact same strategic goals as performance goals; and
what are shown in the earlier table as performance goals are now
called performance measures.
The Results Act requires that annual performance plans identify
annual performance goals that cover all the program activities in the
agency's budget. Treasury's plan complies with this requirement, as
each component and major office generally has one or more performance
goals for each of the budget activities in the budget request. For
one new IRS budget activity relating to the earned-income tax credit
compliance initiative, the plan listed one performance
goal--overclaim rate--but the definition and targets for the goal
have not yet been determined. Also, IRS' budget activity,
"Modernization Investments," did not list any performance goals.
However, the plan noted that the performance measures are discussed
in a separate document relating to modernization proposals.
RECOGNIZING CROSSCUTTING
EFFORTS
---------------------------------------------------------- Letter :3.3
Treasury's performance plan could be improved if it better addressed
the crosscutting nature of its programs and how they will contribute
to achieving performance related to crosscutting functions.
Specifically, we found that Treasury's annual performance plan
generally did not identify performance goals that reflect activities
being undertaken to support crosscutting programs, and the plan does
not consistently address the crosscutting nature of its programs.
Treasury has responsibilities for functions and issues that involve
other agencies. As such, its plan should indicate how Treasury will
coordinate those programs with other federal programs having related
strategic or performance goals. In crosscutting program areas,
Treasury should present output goals and intermediate outcome goals
that would clarify its contribution to the intended outcomes of the
crosscutting program. This information would be helpful to Congress
and other stakeholders in identifying areas in which agencies should
be coordinating efforts to efficiently and effectively meet national
concerns. A focus on results, as envisioned by the Results Act,
implies that federal programs contributing to the same or similar
outcomes should be coordinated to ensure that goals are consistent
and that program efforts are mutually reinforcing.
Customs, for example, is involved in several crosscutting
activities--drug interdiction, counterterrorism, and investigations
of money laundering. These activities are recognized in Customs'
plan as crosscutting activities, but there is no clear evidence in
the plan that its fiscal year 1999 performance goals have been
coordinated with other agencies. The plan does mention some past
coordination efforts--such as between Customs and the Office of
National Drug Control Policy to develop measures for a strategy to
reduce the supply of narcotics. The plan did not clearly discuss the
results of those efforts or indicate whether Customs' fiscal year
1999 performance goals were based on them. However, Customs' plan
does mention coordination efforts with the Immigration and
Naturalization Service and the Department of Agriculture in
establishing performance goals to improve customer service when
processing passengers through ports of entry.
ATF's plan recognizes the role of other law enforcement agencies in
achieving the goals of contributing to a safer America, and the plan
mentions partnerships with various law enforcement agencies to
achieve its goals. However, the plan does not clearly indicate that
ATF coordinated with the other agencies in setting its fiscal year
1999 annual goals or targets.
FMS states that one part of its mission focuses on efforts to
increase the collection of delinquent debts owed the federal
government and that its success is achieved through such activities
as providing debt collection and management services to all federal
agencies and developing and implementing governmentwide debt
management policies. The debt collection program activity in FMS'
plan, for example, has a measure on the percentage of market share of
federal agencies with debt servicing requirements that have referred
their debts to FMS as required by the Debt Collection Improvement Act
of 1996 and another measure to increase governmentwide delinquent
nontax debt collections over the fiscal year 1995 baseline. However,
FMS does not provide any information to show how it plans to
coordinate with other agencies to achieve these goals.
IRS plays a role in administering tax code provisions pertaining to
several billions of dollars in tax expenditures, such as the
earned-income tax credit, the low-income housing credit, and the
research credit, and there is no discussion of these crosscutting
programs. IRS, too, shares responsibilities with other agencies,
such as the Social Security Administration (SSA), in processing and
reconciling information on employee wages and social security
benefits, but the plan does not explicitly discuss or describe
whether any performance goals were coordinated with SSA or other
agencies. Conversely, IRS' plan does state that its narcotics
conviction rate is dependent upon prosecutions within the Department
of Justice and that national priorities for criminal investigations
are determined, in part, by Justice.
TREASURY'S PERFORMANCE PLAN
DOES NOT COMPLETELY DISCUSS HOW
THE DEPARTMENT'S STRATEGIES AND
SOURCES WILL HELP IT ACHIEVE
ITS GOALS
------------------------------------------------------------ Letter :4
The Results Act requires that annual performance plans briefly
describe the strategies and resources the agency intends to use to
achieve its performance goals. We found that Treasury's performance
plan adequately discusses, with some exceptions, the resources to
support the achievement of its performance goals. The usefulness of
the plan, which includes the budget justification, would be enhanced
with a fuller description of how its strategies relate to achieving
the goals.
CONNECTING STRATEGIES TO
RESULTS
---------------------------------------------------------- Letter :4.1
Strategies to facilitate achieving performance goals include
activities such as administrative processes, training, and the
application of technology and efforts to improve efficiency and
effectiveness through approaches such as reengineering work
processes. We found that the information in Treasury's plan on how
strategies were connected to results did not always list strategies
and, in other cases, did not adequately describe the strategies. The
plan also does not consistently discuss how the strategies will help
the Department achieve its goals.
IRS provides an example where strategies relating to its goal to
"improve customer service" were clear and complete. The plan lists
nine strategies to enhance customer service and eight customer
service standards for related products and services. The strategies
and standards include improving the clarity of notices, forms, and
tax publications; increasing the hours for its telephone service;
opening district offices on Saturdays during the filing season;
providing additional telephone assistance to small businesses; and
creating citizen advocacy panels. The descriptions of the strategies
are succinct; and they outline methods that, if followed, should
enhance customer service.
In contrast, Customs' plan provides only a partial description of the
strategies it expects to use in fiscal year 1999 to achieve its
projected results. For example, Customs indicates that it plans to
improve drug interdiction results by focusing attention on areas of
increased vulnerability, exploiting intelligence leads, and improving
technology. However, Customs offered no strategies for its goals in
the revenue-producing and antimoney-laundering areas.
In addition, OCC's plan does not fully describe strategies to achieve
its performance goals. Those goals included general references to an
approach, such as streamlining, but OCC did not provide detailed
strategies for achieving the goals. In some cases, regulatory
requirements were mentioned as a means for achieving goals.
Although the Act does not require agencies' annual performance plans
to disclose how external factors might affect performance and
results, including this information in the plans would enhance their
overall usefulness as it would more fully describe Treasury's
potential to achieve the expected performance. Treasury's strategic
plan did mention some of the external factors that may affect its
ability to achieve its strategic goals. In our opinion, Treasury's
performance plan could be improved by more explicitly addressing how
external factors may affect strategies and intended results and
discussing how it will mitigate or use the identified conditions to
achieve its performance goals.
CONNECTING RESOURCES TO
STRATEGIES
---------------------------------------------------------- Letter :4.2
With some exceptions, the Treasury plan adequately discusses the
resources the Department will use to achieve its performance goals.
In addition to information on dollar amounts and staffing levels, the
plan frequently explains how the resources that Treasury is
requesting specifically contribute to one or more performance goals.
For example:
-- The IRS plan notes that its goals for improving the accuracy and
timeliness of tax return processing depend largely on the
agency's ability to use or acquire four specific information
systems.
-- The IRS plan also notes that the accomplishment of its
performance goal of "$290 million in increased collections" is
contingent upon completing the rollout of the Integrated
Collection System to its district and international offices and
obtaining an additional 57 FTEs to expand office hours and
conduct problem-solving days.
-- The Customs plan explains that continuing the acquisition and
installation of the Land Border automation equipment is needed
to allow inspectors to perform more careful screening and
questioning of vehicle occupants, which should help to achieve
Customs' goal of improving its efficiency at targeting arriving
vehicles for enforcement purposes.
-- The ATF plan explains that expanding its youth crime gun
interdiction initiative, including providing additional agents
for the program, would (1) "provide comprehensive crime gun
tracing by State and local law enforcement"; (2) "provide rapid,
high volume crime gun tracing and crime gun market analysis by
the National Tracing Center (NTC)"; and (3) "train ATF, State,
and local law enforcement personnel." As described, the
requested dollars and staffing would seem to contribute to
achieving ATF's performance targets for the number of persons
trained, the number of traces, and the average trace response
time.
Treasury's plan could be improved in some areas, however, with a more
thorough discussion of the resources required to achieve its
performance goals. For example, in the FMS plan, the resources
needed for accomplishing the performance goals are not always
evident. One of the measures in the "Payments" program activity, for
example, relates to increasing the number of states in which the
direct federal electronic benefits transfer system is available.
However, the FMS plan does not indicate the resources FMS intends to
use to accomplish this measure.
Treasury's performance plan does not consistently address the use of
information technology (IT) resources to achieve performance goals
across its bureaus and offices. The Departmental Offices'
performance plan includes a goal to "pursue and maintain fully
integrated financial systems Departmentwide by standardizing core
financial information into a Departmental data warehouse." However,
the plan does not include any strategy or approach to achieve this
goal. Similarly, one of Customs' goals is to "maximize trade
compliance through a balanced program of informed compliance,
targeted enforcement actions, and the facilitation of complying
cargo." However, in its description of its strategy to meet this
goal, Customs does not mention its major initiative to automate its
commercial operations, known as the Automated Commercial Environment,
or describe how this system will help achieve the goal.
TREASURY'S PERFORMANCE PLAN
DOES NOT PROVIDE SUFFICIENT
CONFIDENCE THAT THE
DEPARTMENT'S PERFORMANCE
INFORMATION WILL BE CREDIBLE
------------------------------------------------------------ Letter :5
Treasury's performance plan does not provide sufficient confidence
that its performance information will be credible because it does not
adequately describe procedures for verifying and validating
performance data or sufficiently discuss the ramifications of known
data limitations. The Results Act requires performance plans to
describe the procedures an agency will use to verify and validate its
performance measures. The descriptions of the procedures should also
identify any significant data limitations and discuss the impact they
may have on the credibility of the performance information.
VERIFYING AND VALIDATING
PERFORMANCE
---------------------------------------------------------- Letter :5.1
Treasury's performance plan does not adequately discuss procedures
for verifying and validating performance information that will ensure
that it is sufficiently complete, accurate, and consistent. Several
of Treasury's bureaus propose to use data from various information
systems to measure performance; but the plan does not adequately
discuss system controls or procedures for ensuring the reliability,
integrity, and security of the data.
Specifically, IRS often uses short descriptions, such as "excellent,"
"good," and "low," to describe the reliability of data for its
performance measures. These descriptions and other information on
IRS' measures do not adequately explain what general procedures are
to be used to control data quality and ensure accuracy. For example,
IRS describes the reliability of data it plans to use from its
Criminal Investigation Management Information System to determine its
narcotics and fraud conviction rates as "excellent." However, IRS'
performance plan does not describe procedures for verifying the
accuracy and completeness of the data. IRS indicates that the data
needed to determine the narcotics and fraud conviction rates come
from the Department of Justice, an external source, but it does not
comment on the credibility of Justice's data or its own data even
though it is aware that the credibility of the IRS data has been
questioned by a private research group.
In the past, we have identified obstacles IRS and Customs face as
they attempt to measure the performance of their programs. One area
of concern has been IRS' inability to adequately measure the
performance of some of its programs because of the lack of reliable
data to measure such key indicators as taxpayer compliance and
burden.\5 We have raised concerns that some of IRS' program-level
performance indicators need to be balanced with indicators designed
to measure whether taxpayers are treated properly.\6
Concerning Customs, we have pointed out that the agency has
traditionally measured the success of its drug interdiction efforts
by the resulting number of seizures, arrests, indictments, and
convictions.\7 These measures do not sufficiently cover key aspects
of performance. In addition, it is not clear whether an increase in
seizures indicates that Customs has become more effective or that the
amount of smuggling has increased and Customs is still seizing the
same percentage of drugs.
--------------------
\5 Tax Administration: Taxpayer Rights and Burdens During Audits of
Their Tax Returns (GAO/T-GGD-97-186, Sept. 26, 1997).
\6 Tax Administration: IRS Faces Challenges in Measuring Customer
Service (GAO/GGD-98-59, Feb. 23, 1998).
\7 Customs Service: Drug Interdiction Efforts (GAO/GGD-96-189BR,
Sept. 26, 1996).
RECOGNIZING DATA LIMITATIONS
---------------------------------------------------------- Letter :5.2
Data limitations can affect the credibility of performance
information. Treasury's performance plan falls short in identifying
data limitations and their implications for the reliability of the
performance information. The Departmental Offices propose to use the
dollar value of U.S. exports of goods and services to measure
progress toward a goal to "facilitate legitimate trade, enhance
access to foreign markets, and enforce trade agreements," but the
plan does not acknowledge any limitations in the data from the
Department of Commerce.
Customs' plan does not discuss additional efforts that are needed to
ensure the credibility of the data by which Customs' performance is
to be judged. This is important in several of Customs' programs
because one of its performance measures is the accuracy of key trade
statistics, and we have noted Customs' inability to generate reliable
trade data.\8
Customs has also expressed concerns about its ability to generate
reliable trade data. Its fiscal year 1997 trade compliance
measurement report states that "Concerns remained for the improper
classification of goods by importers potentially hindering
enforcement activity and skewing trade statistics."\9 Because some of
Customs' measures depend on narrative assessments based on input from
informant or intelligence operations (e.g., money-laundering systems
disrupted and changes in drug-smuggling organizations' behavior), the
plan could be improved by briefly describing efforts to ensure that
the data are credible.
Further, Customs' plan does not specifically mention weaknesses
related to ensuring that sensitive data maintained in its automated
systems are adequately protected from unauthorized access and
modification and that its core financial systems capture all
activities that occurred during the year and provide reliable
information for management to use in controlling operations. These
weaknesses could affect the reliability of Customs' performance data.
The FMS plan does not adequately identify weaknesses in computer
controls that could affect the reliability of data used to measure
performance. For example, based on our ongoing work on the central
banking function of FMS, which includes the payment and collection
activities, we identified weaknesses in the general controls over
some of FMS' computerized information systems that process receipts
and disbursement information for the government. These controls did
not provide adequate assurance that data files and computer programs
were fully protected from unauthorized access and modification.
--------------------
\8 Automated Export System: Prospects for Improving Data Collection
and Enforcement Are Uncertain (GAO/NSIAD-98-5, Nov. 14, 1997).
\9 U.S. Customs Service, Trade Compliance Measurement Report (Jan.
1998).
OTHER OBSERVATIONS
------------------------------------------------------------ Letter :6
When we commented on Treasury's strategic plan, we said that it could
be improved by explicitly addressing the Department's capacity to
measure progress toward achieving its goals.\10 We also said that
developing measures and collecting reliable data for some important
areas of Treasury's performance, such as taxpayer burden, are very
difficult to do. These issues are still concerns to us as Treasury's
performance plan does not adequately discuss the strategies the
Department plans to use to ensure that its measures of program
performance are reliable and that they will improve accountability
and support decisionmaking. These are challenges that Treasury faces
as it strives to better meet the criteria set forth in the Results
Act and related guidance.
We realize that these challenges are difficult and that some measures
and data, such as those pertaining to burden and compliance, will
take more time than others to develop. However, in such instances,
Treasury may need to devise and communicate the interim plans it will
use to measure performance in these critical areas. We believe that
Treasury's plan could be enhanced by explicitly discussing the
Department's strategy to improve its performance measurement systems
and data.
Treasury's plan should also include annual performance goals for
efforts to address its major management challenges. We believe that
Treasury's plan could be improved by including performance goals to
address the significant management challenges and high-risk areas the
Department faces. We found that the Treasury plan does not have
performance goals that adequately address the eight high-risk areas
we previously identified that affect Treasury operations.\11 For
example, one governmentwide high-risk area for Treasury is ensuring
that its computer systems will function properly after the century
date change, yet only two bureaus--OCC and OTS--include specific
performance goals related to the year 2000 computer date-change
issue. The Departmental Offices' plan has a year 2000 goal for
Treasury's systems in general and IRS' and FMS' plans did acknowledge
that the computer date change is a management issue.
Some of the other major management challenges that Treasury faces are
briefly acknowledged in the bureaus' and offices' plans. Treasury's
plan mentions the need to implement the Clinger-Cohen Act
requirements.\12 To fulfill these requirements, the Departmental
Offices' plan has a Treasurywide goal that calls for establishing IT
investment controls. The plan has one related "measure" for the
goal, which is "establishing IT investment controls and ensuring
[that] Treasury and all bureaus have established investment review
boards with defined, repeatable processes for project selection."
However, the plan does not include any discussion of strategies for
achieving this goal or how performance data will be used to
demonstrate improvements to agency programs.
Further, none of the bureaus' plans we reviewed in depth had related
performance goals for establishing IT investment controls. This is a
very important element of any investment strategy and the purpose for
establishing it at the Department level. To ensure that all ongoing
and new IT projects are considered by the investment review boards,
each of Treasury's bureaus and offices should have performance goals
that address IT investment controls in their respective plans.
Treasury's plan also mentions the requirement in the Government
Management Reform Act of 1994 (P.L. 103-356) that the Secretary of
Treasury is to prepare audited consolidated financial statements
(CFS) of the federal government beginning in the spring of 1998.
FMS, which is responsible for preparing the audited cfs, revised the
program activities in its fiscal year 1999 budget creating one on
governmentwide accounting and reporting that covers the cfs
requirement. For this activity, FMS' has one goal--to make the
federal government a model for financial management--and four related
measures such as the percent of agency reports for the cfs processed
by FMS within the established range for accuracy. However, there is
no discussion of how the 1999 proposed targets for the four
performance measures relate to being a model for financial
management.
--------------------
\10 Managing for Results: Agencies' Annual Performance Plans Can
Help Address Strategic Planning Challenges (GAO/GGD-98-44, Jan. 30,
1998).
\11 High-Risk Series: An Overview (GAO/HR-97-1, Feb. 1997).
\12 The purpose of the Clinger-Cohen Act (P.L. 104-106, Div. E) is
to improve the productivity, efficiency, and effectiveness of federal
programs through the improved acquisition, use, and disposal of
information technology resources. Among the act's requirements are
that agencies base decisions about IT investments on quantitative and
qualitative factors associated with the costs, benefits, and risks of
those investments and use performance data to demonstrate how well
the IT expenditures support improvements to agency programs through
measurements such as reduced costs, improved employee productivity,
and higher customer satisfaction.
CONCLUSIONS
------------------------------------------------------------ Letter :7
The Results Act seeks to improve the management of federal programs
by shifting the focus of decisionmaking from staffing and activity
levels to the results of federal programs. Annual performance plans,
as required by the Act, should establish linkages between the
long-term strategic goals outlined in agencies' strategic plans and
their day-to-day program activities. Treasury's annual performance
plan appropriately links its annual performance goals and measures to
its strategic goals. Although, the plan provides useful information
for congressional decisionmakers and other stakeholders, it did not
fully present information that reflects the intended performance
across the Department, describes how strategies relate to attainment
of goals, and assures readers that performance results and data are
credible.
The plan we reviewed was Treasury's first one under the Results Act.
Developing a plan that fully meets all the criteria of the Act and
related guidance will be a challenge because developing measures and
collecting reliable data for some important areas of Treasury's
performance, such as taxpayer burden, are very difficult to do.
Treasury's plan could be enhanced by explicitly discussing the
Department's strategy to improve its performance measurement systems
and data and by describing Treasury's interim plans to measure
performance in critical areas.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :8
On May 28, 1998, we obtained oral comments from Treasury's Director
of the Office of Strategic Planning and his staff on a draft of this
report. They said that Treasury generally agreed with our analysis
and provided comments to clarify its position. The officials said
that Treasury's fiscal year 1999 performance plan--the first such
plan required by the Results Act--is not its first plan. According
to the officials, Treasury has published performance plans in the
past and has publicly reported its performance results against the
plans for fiscal years 1996 and 1997, ahead of the Act's
requirements.
Treasury agreed with our concerns about the validity of its
performance data. Treasury pointed out that the validity of its
performance data and its capacity to regularly and accurately report
on performance are key challenges it needs to address. To this end,
they said Treasury's Office of Inspector General is planning to
identify critical information systems for inclusion in its annual
evaluation work plans; Treasury's bureaus are continuing to identify
and report where data are of questionable reliability; and the
Department is developing a performance reporting system to routinely
report the results of performance.
In the draft of this report that Treasury reviewed, we said that a
fuller description of strategies to achieve goals would be
beneficial. Treasury said that to keep the plan focused and useful,
a balance is needed on the amount of detailed information provided in
the plan. Further, Treasury said that since its plan is incorporated
in its budget request, congressional stakeholders can explore
specific strategies of interest during hearings and follow-up
questions. We agree that balance is needed in the amount of detailed
information provided in the plan. At the meeting, we clarified that
the plan did not always list strategies or adequately describe them.
We revised our report to reflect this, and we also made other
technical changes on the basis of Treasury's comments where
appropriate.
---------------------------------------------------------- Letter :8.1
We will send copies of this report to the Chairman and Ranking
Minority Members of interested congressional committees; the
Director, Office of Management and Budget; and other interested
parties. Copies will also be made available to others on request.
This report was prepared under the direction of Charlie W. Daniel,
Assistant Director. Please contact me or Mr. Daniel on (202)
512-9110 if you or your staff have any questions concerning this
report.
Sincerely yours,
James R. White
Associate Director, Tax Policy
and Administration Issues
RELATED GAO PRODUCTS
============================================================ Chapter 0
Tax Administration: IRS Faces Challenges in Measuring Customer
Service (GAO/GGD-98-59, Feb. 23, 1998).
Managing for Results: Agencies' Annual Performance Plans Can Help
Address Strategic Planning Challenges (GAO/GGD-98-44, Jan. 30,
1998).
The Results Act: Observations on the Department of the Treasury's
July 1997 Draft Strategic Plan (GAO/GGD-97-162R, July 31, 1997).
*** End of document. ***