IRS Modernization: IRS Should Enhance Its Performance Management
System (23-FEB-01, GAO-01-234).
The Internal Revenue Service (IRS) has made progress in the
challenging task of revamping its performance management system.
The goals, objectives, and measures that IRS has developed along
with its new strategic planning and budgeting process are
intended to integrate results-oriented management into IRS' daily
decision-making. However, as IRS officials acknowledge,
extensively revamping a performance management system is a
complex task that takes years to complete. GAO identified several
opportunities to enhance IRS' performance management system,
including clarifying goals and objectives, improving the linkages
between measures, objectives, and goals, and developing fewer but
more specific action items. Such enhancements could increase
managerial accountability and create stronger incentives for
frontline employees to achieve IRS' goals and objectives.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-234
ACCNO: 577865
TITLE: IRS Modernization: IRS Should Enhance Its Performance
Management System
DATE: 02/23/2001
SUBJECT: Performance measures
Program evaluation
Strategic information systems planning
Systems conversions
Tax administration systems
IRS Tax System Modernization Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-234
February 2001
IRS MODERNIZATION
IRS Should Enhance Its Performance Management System Report to Congressional
Requesters
United States General Accounting Office
GAO
Page i GAO- 01- 234 IRS' Performance Management System Letter 1
Results in Brief 3 Background 5 IRS Has Developed Goals, Objectives, and
Performance Measures 6 IRS Has Made Progress But Could Further Enhance Its
Performance Management System 12 Conclusions 18 Recommendations for
Executive Action 19 Agency Comments and Our Evaluation 19
Appendix I Objectives, Scope, and Methodology 22
Appendix II Assessment of the Characteristics of Operating Division
Strategic Goals 25
Appendix III Examples of Action Items Developed by IRS Organizational Units
27 Specific, Measurable, and Outcome- or Output- Oriented Action
Items 27 Unspecific, Unmeasurable, and Not Outcome- or Output- Oriented
Action Items 28
Appendix IV Comments From the Internal Revenue Service 29
Appendix V GAO Contacts and Staff Acknowledgments 32
Figures
Figure 1: Recommended Enhancements to IRS' Performance Management System 4
Figure 2: IRS' Agencywide Strategic Goals, Objectives, and
Performance Measures 8 Figure 3: W& I Strategic Goal and Operating
Objectives
Corresponding to the Agencywide Goal of “Service to Each
Taxpayer” 11 Contents
Page 1 GAO- 01- 234 IRS' Performance Management System
February 23, 2001 The Honorable Bill Thomas Chairman, Committee on Ways and
Means House of Representatives
The Honorable Amo Houghton Chairman, Subcommittee on Oversight Committee on
Ways and Means House of Representatives
In response to congressional and public criticism that it was emphasizing
revenue collection over taxpayer service, the Internal Revenue Service (IRS)
initiated a major modernization effort to make it more responsive to
taxpayers' needs. A key part of this modernization effort is developing a
new performance management system that lays out the agency's goals,
objectives, and performance measures and is intended to balance taxpayer
service and enforcement activities while fostering a productive work
environment. The Commissioner of Internal Revenue considers holding managers
in IRS' operating divisions 1 accountable for achieving these goals and
objectives critical to accomplishing IRS' mission. To an important extent,
the performance management system is expected to create the incentives that
influence the behavior of IRS employees and determine whether IRS' day- to-
day activities support the goals of modernization.
Recognizing that revamping a performance management system is a difficult,
long- term effort, you asked us to provide information on how IRS measures
its performance. Specifically, as agreed with your office, our objectives
were to (1) describe the status of IRS' efforts to develop agencywide and
operating division goals, objectives, and performance measures that support
IRS' mission and (2) assess these efforts and identify actions, if any, IRS
could take to enhance its performance management system.
1 The operating divisions are (1) Wage and Investment, serving individual
taxpayers; (2) Small Business and Self- Employed, serving fully or partially
self- employed individuals and small businesses with assets under $5
million; (3) Large and Mid- Size Business, serving businesses with assets
over $5 million; and (4) Tax Exempt and Government Entities, serving pension
plans, exempt organizations, and governments.
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 01- 234 IRS' Performance Management System
Our assessment is not a review of IRS' compliance with the requirements of
the Government Performance and Results Act (GPRA) of 1993. 2 Although IRS
expects to use certain features of its performance management system to
satisfy GPRA requirements, IRS developed the system primarily to meet its
internal management needs. As a result, our assessment focused on whether
the system met IRS' unique management needs. Our assessment is based on
characteristics that we found associated with goals, objectives, and
performance measures in the performance management literature, including
GPRA. 3 In particular,
? goals, which are broad statements of desired outcomes, should
? reflect the agency's priorities,
? provide a clear direction for future agency action,
? identify what impact or outcome will result from an agency's work, and
? form the basis for formulating clear objectives;
? objectives, which are targets that describe the end results a service or
program is expected to accomplish in a given period of time to meet goals,
generally should be
? clearly stated,
? specific,
? measurable,
? outcome or output oriented, and
? consistent with the agency's mission and goals; and
? performance measures or indicators, which are often selected after goals
and objectives are developed, should be
? logically and directly related to the goals and objectives,
? focus on expected outcomes or outputs, and
? capture information relevant to the goals and objectives. 2 P. L. 103- 62.
GPRA requires agencies to set goals, measure performance, and report on
their accomplishments in their annual performance plans and annual program
performance reports.
3 The publications we reviewed included: OMB Circular A- 11, Part 2,
Preparation and Submission of Strategic Plans, Annual Performance Plans, and
Annual Program Performance Reports (July 2000); Agencies' Strategic Plans
Under GPRA: Key Questions to Facilitate Congressional Review (GAO/ GGD- 10.
1. 16, May 1997); The Results Act: An Evaluators Guide to Assessing Agency
Annual Performance Plans (GAO/ GGD- 10. 1. 20, April 1998); Center for
Accountability and Performance, Performance Management: Concepts and
Techniques, 2nd ed. (1999); Robert S. Kaplan and David P. Norton, The
Balanced Scorecard: Translating Strategy Into Action (Boston, Mass.: Harvard
Business School Press, 1996); and National Partnership for Reinventing
Government, Balancing Measures: Best Practices in Performance Management
(August 1999).
Page 3 GAO- 01- 234 IRS' Performance Management System
Our work was done between August 1999 and December 2000 in accordance with
generally accepted government auditing standards. See appendix I for a
detailed description of our scope and methodology.
IRS has established three agencywide strategic goals and the corresponding
strategic objectives and has identified the strategic measures it plans to
use for measuring progress toward meeting the strategic objectives. Below
the agencywide level, the operating divisions developed their own strategic
goals that were to align with the agencywide goals. The operating divisions
also developed operational objectives to meet the division- level strategic
goals and operating measures to measure progress toward some of those
objectives. In addition, managers developed action items that detailed the
steps they planned to take to achieve operational objectives.
IRS has made progress in the challenging task of revamping its performance
management system. In addition to developing goals, objectives, and
performance measures, IRS has implemented a new strategic planning and
budgeting process intended to integrate the use of performance measures into
agency management.
However, as shown in figure 1, we identified several issues that IRS could
address at both the agencywide and division levels to enhance its
performance management system. Results in Brief
Page 4 GAO- 01- 234 IRS' Performance Management System
Figure 1: Recommended Enhancements to IRS' Performance Management System
Source: GAO.
At the agencywide level, the intent of the objective of “increase
fairness of compliance” is not clear, and performance measures do not
capture key information needed to gauge progress toward achieving three
strategic objectives. At the division level, three of 15 goals do not (1)
clearly articulate the divisions' future direction, (2) indicate the
expected impact of achieving the goal, or (3) provide a clear basis for
establishing objectives. Also, operating objectives are not specific,
measurable, or in all but two cases, outcome or output oriented; and a large
number of operating measures and indicators are not directly linked to
objectives. In addition, because IRS was introducing its new performance
management concepts, many of the action items developed by managers to meet
the fiscal year 2000 operating objectives were not specific, measurable, and
outcome or output oriented.
IRS recognizes that revamping its performance management system will be an
ongoing process because the inherent difficulties in implementing
Agencywide level
Operating division level
Improve the linkage between 2 strategic measures and related objectives
Develop strategic measure for one objective Clarify intent of "increase
fairness of compliance"
Strategic goals Strategic objectives Strategic measures Operating objectives
Operating measures
Develop objectives that are specific, measurable, and outcome oriented
Link operating measures more directly to operating objectives
Enhancement Enhancement Enhancement
Enhancements Strategic goals
Revise 3 goals to articulate agency's future direction, indicate expected
impact of achieving the goal, and provide a clear basis for developing
objectives
Enhancement
A
Page 5 GAO- 01- 234 IRS' Performance Management System
such a major change will require continuing adjustments. We are making
recommendations to the Commissioner of Internal Revenue on how IRS can
enhance its performance management system at both the agencywide and
division levels. In commenting on a draft of this report, the Commissioner
agreed with our recommendations at the division level. However, the
Commissioner raised some concerns with our recommendations to clarify one
strategic objective and add a measure for the strategic objectives
concerning customer service.
In response to congressional concerns that IRS was overemphasizing revenue
collection at the expense of fairness to taxpayers, the Commissioner of
Internal Revenue is seeking to refocus the agency toward an approach that
promotes voluntary compliance with the tax laws through better taxpayer
service while concentrating enforcement resources on those taxpayers who are
deliberately and willfully noncompliant.
IRS realized it needed to revamp its performance management system to
reflect this new focus within the constraints of two legislative
requirements on how it sets goals and measures performance. First, the
Government Performance and Results Act of 1993 requires federal agencies to
establish a hierarchy of goals, objectives, and performance measures
applicable to various organizational units within their agencies. Second,
the IRS Restructuring and Reform Act of 1998 4 directed IRS to establish
individual, group, and organizational goals and objectives, but prohibited
IRS from using tax enforcement results 5 to evaluate any IRS employee or to
impose or suggest production quotas or goals.
According to the performance management literature we reviewed, the purpose
of organizational goals, objectives, and measures is to enable an
organization to monitor how well it is achieving its mission. Strategic
goals provide direction to an organization's work, services, programs, and
activities and generally describe a desired outcome for the organization and
its programs. To achieve goals, organizations develop objectives. Objectives
are more specific than goals and are measurable targets that
4 P. L. 105- 206. 5 Tax enforcement results are outcomes produced when IRS
employees exercise their judgment in recommending or determining whether or
how IRS should pursue enforcement of the tax laws. Examples of tax
enforcement results are the number of liens, levies, and seizures issued;
the dollar amount assessed; the dollar amount collected; and the number of
fraud referrals. Background
Page 6 GAO- 01- 234 IRS' Performance Management System
describe the end results a service or program is expected to accomplish in a
given time period. Finally, organizations develop performance measures to
monitor progress toward their objectives and ultimately their goals.
Performance measures should be quantifiable measures, such as outcomes,
outputs, efficiency, or cost- effectiveness, and should be useful in
determining whether the organization is achieving its goals.
One framework for developing performance measures is the “balanced
measures approach,” which was adopted by IRS. This approach originated
in the private sector when companies concluded that overemphasizing short-
term financial objectives could actually be detrimental to their longterm
success. Instead, these companies began focusing on both financial and
nonfinancial aspects of performance, including areas such as product
convenience and quality, that engender the kind of customer satisfaction and
loyalty necessary for success in the long run.
At both the agencywide and operating division levels, IRS has established
goals and the corresponding objectives and has identified the performance
measures it plans to use for measuring progress toward meeting the goals and
objectives.
In 1998, the Commissioner of Internal Revenue replaced IRS' old mission
statement with a new one: “Provide America's taxpayers top quality
service by helping them understand and meet their tax responsibilities and
by applying the tax law with integrity and fairness to all.”
Using the balanced measures approach of emphasizing multiple goals, IRS
established three agencywide strategic goals in 1999 to reflect its new
approach to tax administration. The goals are
? “service to each taxpayer,”
? “service to all taxpayers,” and
? “productivity through a quality work environment.” IRS chose
the balanced measures framework for its new performance management system
because senior executives wanted to ensure that managers and employees
considered taxpayer needs in addition to revenue collection when making
decisions. IRS also established two or more strategic objectives for each
goal. IRS Has Developed
Goals, Objectives, and Performance Measures
IRS Established New Agencywide Strategic Goals, Objectives, and Measures to
Reflect Its New Mission Statement
Page 7 GAO- 01- 234 IRS' Performance Management System
In 1999, IRS developed and began using the results of its employee
satisfaction survey as a strategic measure for the objective of
“increase
employee job satisfaction.” In 2000, IRS identified and began
developing other strategic measures, such as taxpayer burden as a measure
for
“making filing easier.” These measures are in various stages of
development. For example, IRS expects to have taxpayer burden estimates for
the Wage and Investment Division in fiscal year 2001 and anticipates
developing estimates for the Small Business and Self- Employed Division in
fiscal year 2002. IRS is not sure when data on productivity and workload
will be available. The strategic goals, objectives, and performance measures
are shown in figure 2.
Page 8 GAO- 01- 234 IRS' Performance Management System
Figure 2: IRS' Agencywide Strategic Goals, Objectives, and Performance
Measures
Objective Objective Objective
Provide prompt, professional, helpful treatment to taxpayers in cases where
additional taxes may be due Make filing easier Provide first quality service
to each taxpayer needing help with his or her return or account
Goal Performance measure Strategic
goals Strategic performance
measures Strategic
objectives Service to
each taxpayer
Taxpayer burden (time and dollar costs)
Performance measure
Customer satisfaction/ dissatisfaction
Performance measure
Customer satisfaction/ dissatisfaction a
Page 9 GAO- 01- 234 IRS' Performance Management System
Source: IRS.
Performance measure Performance measure Performance measure Objective
Objective Objective Objective
Increase employee job satisfaction Increase fairness of compliance Increase
overall
compliance Hold agency employment stable while economy
grows and service improves Payment compliance Employee satisfaction
surveys Productivity/ workload
index
Performance measure Goal Goal
Service to all taxpayers Productivity through
a quality work environment
None identified
Performance measure Performance measure
Filing compliance Reporting compliance
a
Page 10 GAO- 01- 234 IRS' Performance Management System
In 2000, IRS implemented a new strategic planning and budgeting process that
provides the framework for developing goals, objectives, and measures at the
operating division level. Under this process, the operating divisions
developed program plans that identified the division's strategic goals,
which were to correspond to each of the agencywide strategic goals. Such
division- level strategic goals were to be tailored to the specific
characteristics and needs of individual taxpayer segments served by the
division. The program plans also identified the operating objectives, which
IRS calls operating priorities, intended to accomplish the division's
strategic goals.
For example, the Wage and Investment Division (W& I) concluded that most of
the taxpayers it deals with
? prepare their own tax returns;
? are highly compliant, with noncompliance often resulting from taxpayer
confusion;
? contact IRS only once a year; and
? receive refunds. W& I considered these characteristics when developing its
strategic goals and operational objectives. Figure 3 illustrates the linkage
between the agencywide goal of “service to each taxpayer” and
one of W& I's strategic goals and associated operating objectives, which
addresses improving taxpayer convenience and communication.
The program plans also identify three types of operating measures or
indicators that the operating divisions will use to measure their
performance. They are
? balanced measures, which includes customer satisfaction to align with
“service to each taxpayer,” business results (quality and
quantity) to align with “service to all taxpayers,” and employee
satisfaction to align with
“productivity through a quality work environment,” to measure
the overall health of the organizational unit;
? workload indicators, such as the number of returns processed, to project
the expected level of activity for the organizational unit, identify
resource needs, and justify resource requests; and
? performance indicators, such as the percentage of individual refunds
issued within 40 days of the date of the signed tax return, to measure how
well the organizational units are achieving program objectives.
At the time of our review, each division had developed these measures for at
least some of their organizational units. IRS' Strategic Planning and
Budgeting Process Provides the Framework for Operating Divisions' Goals,
Objectives, and Measures
Page 11 GAO- 01- 234 IRS' Performance Management System
Figure 3: W& I Strategic Goal and Operating Objectives Corresponding to the
Agencywide Goal of “Service to Each Taxpayer”
Source: IRS.
As the final step in this process, IRS requires managers to develop action
items that detail the steps they will take to implement the operational
objectives. IRS' fiscal year 2000 Operations Plan established six agencywide
priorities to promote consistency among action items. The six priorities
were to (1) implement balanced measures, (2) improve customer satisfaction,
(3) improve employee satisfaction, (4) improve the quality of work, (5)
identify and work with taxpayer segments likely to become noncompliant, and
(6) improve compliance by emphasizing nonenforcement activities.
At each organizational level down to the group level, managers and their
work groups developed action items that addressed each of these six
priorities.
Agencywide level
Operating division level
Service to each taxpayer
Strategic goal Strategic goal
Meet taxpayer demands for better traditional assistance services (access and
quality): phone, correspondence, and in- person
Agencywide level Operating objectives
Upgrade field assistance Organize, train, and
specialize customer service representatives Add trained phone assistors
Leverage partnerships Research customer segment
needs Expand alternative language
services
Division level Division level
a
Page 12 GAO- 01- 234 IRS' Performance Management System
IRS has made progress in the challenging task of revamping its performance
management system from one focused on revenue collection to one that better
balances taxpayer satisfaction, tax law compliance, and employee
satisfaction. However, IRS could make enhancements to its performance
management system by revising some goals, objectives, and measures at both
the agencywide and operating division level.
IRS intends to make its new performance management system the routine way
the agency is managed, which IRS believes will put it in a better position
to manage toward its strategic goals and objectives. Specifically, IRS
intends to use its performance measures to monitor and evaluate its
performance as part of its new planning and budgeting process and then use
the results of this review to reassess and revise its objectives and goals.
In addition, IRS considers the action items developed to implement
operational objectives a primary mechanism for getting employees at all
levels of the agency actively engaged in carrying out IRS' goals and
mission. Managers and their work groups are expected to develop action items
in response to problems they define and the causes and possible solutions
they identify from their analysis of the operating data. In this way, IRS
hopes to establish a “clear line of sight” between employees'
dayto- day activities and IRS' overall mission and strategic goals.
Because IRS has not yet completed a full cycle of its new planning and
budgeting process, it is too soon to evaluate the impact of this
resultsoriented process on performance. Nevertheless, the effort IRS is
making to integrate the use of performance measures into its day- to- day
management represents progress in using performance management to support
the new mission of the agency.
At the agencywide level, we identified three issues that IRS could address
to enhance the usefulness of its performance management system. The issues
include the (1) clarity of one strategic goal, (2) clarity of one strategic
objective, and (3) linkage between certain strategic measures and strategic
objectives.
With respect to the first issue, two of IRS' three strategic goals-”
service to each taxpayer” and “productivity through a quality
work environment”- give a relatively clear indication of IRS'
priorities, direction, and the potential impact its policies can be expected
to have on individual taxpayers and agency employees. However, IRS' goal of
“service
IRS Has Made Progress But Could Further Enhance Its Performance Management
System
IRS' Efforts to Integrate Performance Measures Into Management Represent
Progress in Supporting Its Mission
Issues to Be Addressed at the Agencywide Level
Page 13 GAO- 01- 234 IRS' Performance Management System
to all taxpayers” did not clearly convey its intent, which is to apply
the tax law with integrity and fairness so that taxpayers that do not comply
with the tax laws are not allowed to place a burden on those that do. During
the course of our review, we brought this matter to the attention of top IRS
executives, including the Commissioner of Internal Revenue. IRS agreed that
the goal did not clearly convey its intent. In February 2001, just before we
issued this report, IRS released its strategic plan for fiscal years 2000-
05, which showed the goal statement was modified to read “service to
all taxpayers through fair and uniform application of the law.” This
modification of the goal statement makes clearer that this goal covers how
IRS applies the tax laws to all taxpayers.
With respect to the second issue, IRS' strategic objectives are consistent
with its mission and with one exception- the objective of “increasing
fairness of compliance”- are clearly stated, specific, measurable, and
outcome or output oriented. As IRS recognizes, this fairness objective could
be subject to various interpretations and difficult to measure. For example,
fairness could mean each individual taxpayer is equally likely to be audited
regardless of such characteristics as occupation or income. Fairness could
also mean that IRS would target enforcement activities toward pockets of
noncompliance so that compliant taxpayers are less likely to be burdened by
audit activities. Rewording this objective to more clearly express the
desired result could help managers better focus their efforts to achieve
this objective.
With respect to the third issue, for six of its seven strategic objectives,
IRS has developed performance measures that are directly related to the
objectives, focus on expected outcomes or outputs, and capture relevant
information. However, for two of these six objectives, IRS' performance
measures do not capture all the information that is relevant to gauging
progress toward achieving the objectives. IRS has relied strictly on
taxpayer perceptions as measured by customer satisfaction surveys or
complaints to measure how well it is meeting its two strategic objectives of
“providing quality service to taxpayers who need help with their
returns or accounts” and “providing quality service to those who
may owe additional taxes.” Although customer satisfaction surveys and
complaints provide relevant information, they do not address an important
component of IRS' service- whether the taxpayer's problem was properly
handled within the context of the tax law. Elements of IRS' existing quality
management systems might provide the basis for developing such a measure.
The quality management systems currently include assessments such as whether
IRS employees properly computed tax liabilities or selected the most
appropriate payment method for collecting past- due taxes. In addition, IRS
did not have a performance measure for its strategic
Page 14 GAO- 01- 234 IRS' Performance Management System
objective of “increase fairness of compliance.” Without a
performance measure that is logically and directly related to this
objective, IRS cannot determine its progress toward meeting the objective.
At the operating division level, we identified three issues that IRS could
address to enhance the usefulness of its performance management system. The
issues include (1) the clarity with which goals articulate the divisions'
future direction, indicate the expected impact of achieving the goal, and
provide a clear basis for establishing objectives; (2) whether objectives,
which IRS calls operating priorities, are specific, measurable, and outcome
or output oriented; and (3) the linkage between operating measures and
operational objectives.
With respect to the first issue, our review of 15 division- level strategic
goals for three of four divisions 6 found that they reflected IRS'
agencywide goals and priorities. However, three of the goals did not
? articulate the divisions' future direction,
? indicate the expected impact of achieving the goal, or
? provide a clear basis for establishing objectives. These goals are (1)
“address underreporting, nonfiling, and abuse of trusts and pass-
throughs,” (2)” build a tax administration to effectively deal
with globalization,” and (3) “implement comprehensive issue
management strategy.” For example, the strategic goal to
“address underreporting, nonfiling, and abuse of trusts and pass-
throughs” is set within the broad context of increasing overall
compliance. However, it is vague and does not articulate the division's
future direction or a desired impact. And since the expected impact is not
indicated, the goal does not provide a good basis for formulating
objectives. In contrast, the strategic goal “achieve
early, accurate resolution of taxpayer accounts” focuses managers'
attention on the impact that is expected and provides a basis for setting
objectives to accomplish the goal. Appendix II provides a more detailed
analysis of the shortcomings of these the three goals.
With respect to the second issue, our review of division- level operational
objectives found that they were clearly stated and consistent with
divisionlevel goals and IRS' overall goals. However, none of the 72
objectives were
6 We reviewed the strategic goals for three of IRS' four operating
divisions- Wage and Investment, Small Business and Self- Employed, and Large
and Mid- Size Business. Issues to Be Addressed at
the Division Level
Page 15 GAO- 01- 234 IRS' Performance Management System
stated in terms that were specific and measurable- that is, they did not
include a time period, a numeric target, or a means to measure the
objective. Also, 70 of 72 were not outcome or output statements because they
did not include an expected result, a program impact, or a time period.
We recognize that formulating objectives that describe the desired end
result can be a challenging undertaking; however, objectives that are not
clearly articulated are less useful to managers responsible for
accomplishing division objectives. For instance, the objective
“consider
targeted notices and guidance” provides no indication of
? which notices or guidance are of interest,
? what is to be done with them,
? how to measure whether the objective has been met, or
? the expected impact of accomplishing this objective. In contrast, the
objective “centralize processing of OICs [offers- incompromise] to
improve quality, timeliness, and efficiency” indicates what is to be
done and the expected impact of meeting this objective.
With respect to the third issue, our review of the operating division's
program plans found that the operating divisions have identified balanced
measures, workload indicators, and performance indicators for many of their
functional programs. 7 However, for 63 of the 72 objectives, the operating
divisions had not linked operating measures and indicators directly to the
objective. Without such direct linkages, operating divisions will not be
able to measure the extent to which they are meeting their objectives.
For example, the W& I program plan identified expanding Spanish and second-
language services as an operational objective. However, W& I has not
developed measures specifically directed toward measuring the impact of its
actions, such as the customer satisfaction level of Spanish- speaking
taxpayers, or the extent to which they achieved this objective, such as
whether W& I increased the number of Spanish- speaking tax auditors. In
contrast, W& I did identify specific performance measures for its objective
of leveraging partnerships with volunteer and other organizations to
7 The work of each operating division is performed through functional
programs aligned with one of three general service areas: prefiling
services, filing and account services, and compliance services. W& I's
Taxpayer Assistance program, for example, is a prefiling service.
Page 16 GAO- 01- 234 IRS' Performance Management System
deliver return preparation services. As measures of its success in meeting
this objective, W& I will look to (1) reducing the number of compliance
personnel supporting field assistance, (2) increasing the number of hours
volunteers provide walk- in assistance, and (3) increasing return
preparation services provided by partnering organizations.
In fiscal year 2000, IRS began requiring managers to develop plans each year
that identify the actions they intend to take to support their operating
objectives. We evaluated the action items contained in the plans using the
same characteristics we used to evaluate objectives. We found that, while
clearly stated and consistent with IRS' mission, few action items were
specific, measurable, and outcome or output oriented. In addition, the large
number of action items reduced the likelihood that they would be tracked and
assessed. IRS recently issued guidance for developing individual performance
commitments that could be a model for better guidance on action plans.
We received and reviewed 268 action plans from 32 of IRS' 33 district
offices. Although the 6,398 fiscal year 2000 action items we reviewed were
clearly stated and consistent with IRS' mission, we found that 91 percent of
them (5, 847) were not specific, measurable, and outcome or output oriented.
As a result, IRS could find it difficult to hold managers accountable for
achieving their action items. The following examples illustrate how the
quality of the action items varied:
? “Use reports to identify improvement areas and develop actions to
address them.”
? “Branch will establish a quarterly review of 5 cases from each of
four pure general program groups. The five cases will be reviewed by a
manager from another group for adherence to the EQMS [Examination Quality
Management System] standards.”
The second action item is more specific, measurable, and outcome or output
oriented because it includes such information as who is responsible for
taking action, the frequency of the action, the number of actions, and the
anticipated outcome (adherence to quality standards). Appendix III provides
additional examples of action items.
IRS National Office officials were not surprised by our results. In fiscal
year 2000, their guidance steered managers away from developing specific and
measurable action items for at least two reasons. First, IRS lacked
statistically valid performance data, and therefore, IRS' training course on
balanced performance measures instructed managers not to establish In the
First Year of
Implementation, IRS' Managers Lacked Adequate Guidance for Developing Action
Items
Page 17 GAO- 01- 234 IRS' Performance Management System
quantitative measures for their action items. Second, IRS felt the need to
introduce the balanced measures concept to managers as quickly as possible,
even though operating divisions did not develop specific goals and
objectives until after the action plans were due. As a result, the action
items that divisions provided to their organizational units as a framework
to follow were vague and nonspecific. For example, the Examination
operations plan included the following action items:
? “Ensure the cultural change and philosophy supporting the protection
of taxpayer rights has been embraced by the organization,”
? “Support the transition to the new operating divisions,” and
? “Revise the Quality Review Program to emphasize uniformity and
consistency.”
In addition, although IRS guidance instructed managers to prepare brief,
focused plans with a few areas of emphasis, we found that each manager's
plan contained an average of 24 action items. This multitude of action items
could reduce the likelihood that managers would be focusing on those
activities that are critical to achieving the operating division's goals and
objectives. Furthermore, such a large number of action items might increase
the difficulty of holding managers accountable for achieving their action
items.
IRS officials told us they are taking steps to improve the quality of the
action items. These steps include
? developing training on the balanced measures system that is to include a
module covering the skills needed to analyze data to better identify action
items,
? developing a handbook on action planning intended to help managers analyze
customer satisfaction data,
? issuing a four- page brochure intended to help familiarize both IRS
employees and other interested parties with the purpose of the customer
satisfaction survey and how the data should be used to improve IRS
operations, and
? ensuring that the District Offices of Research and Analysis provide the
operating divisions with more research assistance.
Although these initiatives may help managers identify the areas that should
be targeted by action items, they do not cover how to formulate action items
in specific, measurable, and outcome or output oriented terms or how to
limit the number of action items to the critical few. Such guidance would
have to be provided by the operating divisions since,
Page 18 GAO- 01- 234 IRS' Performance Management System
under the new strategic planning and budgeting process, they will be
responsible for implementing the strategic plans.
Recently issued guidance on developing individual performance commitments, a
component of the employee evaluation process, could serve as a model for
guidance on developing action items. Performance commitments are intended to
better link IRS' performance appraisal process to the balanced measures and
set forth specific expected goals and results tied directly to the
organization's strategic objectives. The guidance identifies the key
features of a well- constructed commitment as
? spelling out specifically what is to be accomplished, not general tasks,
duties, or responsibilities of an ongoing job,
? describing realistically the actions to be taken and the expected
outcomes,
? maintaining a “line of sight” to the goals of the Service,
? including a clear timeframe for accomplishment,
? being achievable and capable of being monitored and assessed, and
? outlining required organizational resources or support. The key features
of a well- constructed commitment are similar to those that would be key
features of a good action item. Guidance laying out similar features, along
with examples of specific, measurable, and outcome- or output- oriented
action items, could help managers develop action items that would in turn
help employees carrying out IRS' day- today operations to better link their
activities to IRS' goals and objectives.
IRS has made progress in the challenging task of revamping its performance
management system. The goals, objectives, and measures that IRS has
developed along with its new strategic planning and budgeting process are
intended to integrate results- oriented management into IRS' daily decision-
making. However, as IRS officials acknowledge, extensively revamping a
performance management system is a complex task that takes years to
complete. We identified several opportunities to enhance IRS' performance
management system, including clarifying goals and objectives, improving the
linkages between measures, objectives, and goals, and developing fewer but
more specific action items. Such enhancements could increase managerial
accountability and create stronger incentives for frontline employees to
achieve IRS' goals and objectives. Conclusions
Page 19 GAO- 01- 234 IRS' Performance Management System
As IRS continues to refine its performance management system, we recommend
that the Commissioner of Internal Revenue take steps at both the agencywide
and division levels to enhance the system. At the agencywide level, these
steps are to
? clarify the strategic objective, “increase fairness of
compliance,” to more precisely express the desired result in ways that
can be measured, and
? add a measure of quality- that is, whether the taxpayers' problems are
correctly handled within the context of the law- to the agencywide
performance measures for the strategic objectives of providing quality
service to taxpayers who need help or who may owe additional taxes, and
provide a performance measure for the objective, “increase fairness of
compliance.”
At the division level, these steps are to
? clarify three strategic goals to better articulate IRS' future direction,
indicate the expected impact of achieving the goal, and provide a clear
basis for establishing objectives;
? revise or develop operating objectives to ensure that they are specific,
measurable, and outcome or output oriented; and
? revise or develop operational performance measures to ensure that the
measures are directly linked to operational objectives.
In addition, we recommend that IRS provide better guidance to unit managers
on how to develop action items that are few enough to focus employees'
attention and are specific, measurable, and outcome or output oriented.
We requested comments on a draft of this report from IRS. We obtained
written comments in a February 13, 2001, letter from the Commissioner of
Internal Revenue (see app. IV). In his letter, the Commissioner
characterized our assessment of IRS' implementation of its revamped
performance management system as fair and balanced. He said that he believes
IRS has made significant progress, but recognizes that IRS still faces many
challenges in establishing a lasting and effective system for measuring
agency performance. The Commissioner agreed with our recommendations at the
division level. However, the Commissioner raised some concerns with our
recommendations to clarify one strategic objective and add a measure for the
strategic objectives concerning customer service. Recommendations for
Executive Action Agency Comments and Our Evaluation
Page 20 GAO- 01- 234 IRS' Performance Management System
In commenting on our recommendation that IRS clarify its strategic objective
of “increasing fairness of compliance,” the Commissioner agreed
that it was ideal for objectives to be completely self- explanatory.
However, he said that he believed it was important to communicate to the
public that IRS valued fairness in conjunction with its compliance role. He
said that IRS intends to continue to elaborate on the intent of this
objective in its ongoing communications with taxpayers and IRS employees.
Further, the Commissioner stated that an aspect of IRS' current work on the
balanced measures system is to better understand the taxpayers' concept of
fairness and to develop suitable measures of fairness. We agree that
communicating to the public that IRS values fairness is important. However,
since IRS' new performance management system is intended to increase
accountability, we continue to believe that IRS needs to clarify this
objective statement so that taxpayers know specifically what to expect from
IRS in the area of fairness, IRS managers can better focus their efforts on
achieving this objective, and Congress can gauge IRS' progress in meeting
this objective.
In commenting on our recommendation that IRS use a quality measure in
addition to a customer satisfaction measure for the strategic goal of
providing quality customer service, the Commissioner agreed conceptually
that it is relevant to determine whether taxpayers' problems are properly
handled within the context of the law. The Commissioner focused his comments
on measuring customer satisfaction and expressed concern that weighting the
customer satisfaction score or otherwise combining it with a quality score
would diminish the customer view and devalue the importance of taxpayers'
opinions. We agree that a customer satisfaction score measures one very
important component of providing quality service to taxpayers. However,
taxpayers may not know whether their case was handled properly within the
context of the law. Therefore, we continue to believe that IRS needs to have
a separate score to measure the component of quality service that deals with
whether IRS handles taxpayer problems properly. It is not our intent that
IRS attempt to combine these two measures into one overall score of customer
service, and we have modified our recommendation to make this clearer.
The Commissioner agreed with our recommendations on improving IRS'
performance management system at the division level regarding the clarity of
goals, the specificity and measurability of objectives, and the linkage
between measures and objectives. He said that while the operating divisions
have already taken some steps in line with our report recommendations, IRS
will be addressing these issues on an ongoing basis and that refinements
will occur as IRS develops more experience with the
Page 21 GAO- 01- 234 IRS' Performance Management System
strategic planning and budgeting process. The Commissioner also agreed with
our recommendation to provide better guidance to unit managers on how to
develop action items and said IRS had already undertaken corrective action
in this area.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from
the date of this letter. At that time, we will send copies to the Ranking
Minority Member of the Subcommittee on Oversight; the Ranking Minority
Member of the Committee on Ways and Means; the Secretary of the Treasury;
and the Commissioner of Internal Revenue. We will also make copies available
to others on request.
Please contact me at (202) 512- 9110 or Ralph Block at (415) 904- 2150 if
you or your staff have any questions. Key contributors to this report are
acknowledged in appendix V.
Sincerely yours, James R. White Director, Tax Issues
Appendix I: Objectives, Scope, and Methodology
Page 22 GAO- 01- 234 IRS' Performance Management System
Our objectives were to (1) describe the status of IRS' efforts to develop
agencywide and operating division goals, objectives, and performance
measures that support IRS' mission and (2) assess these efforts and identify
actions, if any, IRS could take to enhance its performance management
system.
To describe IRS' efforts to develop goals, objectives, and performance
measures, we interviewed officials responsible for developing and
implementing IRS' new strategic planning and budgeting process. We reviewed
IRS' draft strategic plan for fiscal years 2000- 05. Because IRS' new
performance management system is intended to balance taxpayer service and
enforcement activities, we focused on the goals, objectives and performance
measures affecting enforcement employees. Therefore, we reviewed the
strategic and program plans prepared by the Wage and Investment, Small
Business and Self- Employed, and Large and Mid- Size Business Divisions
because examination and collection employees were generally reassigned to
one of these divisions during IRS' reorganization. In addition, we requested
the fiscal year 2000 action plans prepared by the district and each of the
examination and collection divisions and branches from IRS' 33 district
offices. We received 15 district plans, 56 division plans, and 197 plans
from 60 branch and 32 district offices.
To assess IRS' efforts in developing the components of its performance
management system, we conducted a literature search on performance measures
in general and balanced measures in particular; identified characteristics
associated with goals, objectives, and performance measures for use in
evaluating IRS' performance management system; and discussed IRS' approach
and methodology with IRS contractors involved in developing the performance
measures and analyzing the resulting data. We also interviewed IRS
headquarters officials and supervisors in the Northern California and
Kansas- Missouri District Offices to discuss their use of performance
measurement data and the action plans they developed. We selected these two
offices because of their proximity to the members of the audit team.
The following provides more information on the characteristics associated
with goals, objectives, and performance measures that we used to assess IRS'
performance management system and how we determined if those characteristics
were present.
Characteristics associated with strategic goals are that they should (1)
reflect the agency's priorities, (2) provide clear direction for future
agency Appendix I: Objectives, Scope, and
Methodology Objectives Scope and Methodology
Characteristics Used to Assess IRS' Performance Management System
Characteristics Associated With Goals
Appendix I: Objectives, Scope, and Methodology
Page 23 GAO- 01- 234 IRS' Performance Management System
action, (3) identify what impact or outcome will result from an agency's
work, and (4) form the basis for formulating clear objectives. Basically, we
were looking for statements that indicated how and why the goal should be
achieved. In assessing IRS' goals, we addressed the following types of
questions:
? Reflect the agency's priorities: Does the goal contribute toward achieving
IRS' mission of helping taxpayers understand and meet their tax
responsibilities and applying the tax law with integrity and fairness?
? Provide clear direction for future action: Does the goal establish a trend
for IRS' programs and activities?
? Identify impact or outcome: Does the goal indicate what effect achieving
the goal will have on taxpayers, overall compliance, or employees?
? Form the basis for formulating clear objectives: Does the goal indicate
the kinds of activities that managers and employees should focus on?
Characteristics associated with objectives and action items are that they
should be (1) clearly stated, (2) specific, (3) measurable, (4) outcome or
output oriented, and (5) consistent with the agency's mission and goals. We
used these characteristics to assess IRS' strategic objectives, operating
objectives, and action items. Basically, we were looking for statements that
would allow managers to be held accountable for their actions. In assessing
IRS' strategic objectives, operating objectives, and action items, we
addressed the following types of questions:
? Clearly stated: Is the objective or action item easy to understand? Does
it contain jargon that would be unfamiliar to IRS managers and employees?
? Specific: Does the objective or action item include details such as who is
responsible for taking action? Does the objective or action item include a
time period, an expected result, or a numeric target?
? Outcome or output oriented: Does the objective or action item include an
expected result, a program impact, or a time period?
? Measurable: Is the objective or action item quantifiable? Does the action
item include a time period, an expected result, a numeric target, or a means
to measure?
? Consistent with IRS' mission: Is the objective or action item consistent
with IRS' mission statement and goals?
Characteristics associated with performance measures are that they should be
(1) logically and directly related to the goals and objectives; (2) focus on
expected outcomes or outputs; and (3) capture information relevant to the
goals and objectives. In assessing IRS' performance measures, we addressed
the following types of questions: Characteristics Associated
With Objectives and Action Items
Characteristics Associated With Performance Measures
Appendix I: Objectives, Scope, and Methodology
Page 24 GAO- 01- 234 IRS' Performance Management System
? Logically and directly related: Does the measure address the important
components of the goals or objectives?
? Focus on expected outcomes or outputs: Does the measure address the impact
that IRS' activities have on taxpayers and employees or the output of IRS
programs?
? Capture relevant information: Is the measure providing information useful
to managers in assessing the extent to which the objective has been achieved
or the impact of IRS activities?
Appendix II: Assessment of the Characteristics of Operating Division
Strategic Goals
Page 25 GAO- 01- 234 IRS' Performance Management System
We assessed whether all of the 15 strategic goals established by the Wage
and Investment, Small Business and Self- Employed, and Large and Midsize
Business Divisions had the characteristics described in appendix I. We found
that all 15 reflected IRS' priorities. However, three of the goals did not
provide a clear direction for future action, identify the expected impact of
achieving the goal, or form the basis for establishing clear objectives. A
discussion of how we assessed these three goals follows.
“Address Underreporting, Nonfiling, and Abuse of Trusts and
PassThroughs”
This goal, established by the Small Business and Self- Employed Division, is
set within the context of increasing overall compliance. However, the word
“address” is vague and does not provide a clear indication of
the direction IRS will be taking in the future. For example,
“address” could mean increasing taxpayer education, outreach to
practitioners, or compliance monitoring. In addition, the goal statement
does not articulate what the expected impact will be- presumably, to reduce
the amount of unreported income that results from taxpayers' underreporting
income, failing to file tax returns, or using questionable trusts and pass-
throughs. The goal also does not provide a good basis for formulating
objectives for accomplishing it because it does not indicate how IRS will
deal with underreporting, nonfiling, and abuse of trusts and pass- throughs.
As a result, the goal statement does not communicate the activities IRS will
focus on in both the short term and the long term and the accomplishments
IRS is striving to achieve.
“Build a Tax Administration to Effectively Deal With
Globalization”
This statement, established by the Large and Mid- Size Business Division, is
attempting to articulate a goal for administering the tax laws in a global
economic environment where such factors as transnational operations, rapidly
changing technology, and changing business practices must be recognized.
However, the goal is broad and nonspecific. As with the prior example, the
phrase “build a tax administration” does not articulate the
direction IRS intends to take in the future. In addition, the goal does not
indicate the expected impact on taxpayers or overall compliance. For
example, the phrase “effectively deal with globalization” could
refer to identifying and responding to emerging trends quickly or to
developing technical competence and sophistication in interacting with
multinational corporations. The goal statement also does not indicate what
type of activities will be undertaken or the areas that should be improved
and so Appendix II: Assessment of the
Characteristics of Operating Division Strategic Goals
Appendix II: Assessment of the Characteristics of Operating Division
Strategic Goals
Page 26 GAO- 01- 234 IRS' Performance Management System
does not provide the basis for formulating objectives to accomplish the
goal.
“Implement Comprehensive Issue Management Strategy”
With this goal statement, IRS' Large and Mid- Size Business Division is
attempting to address the dissatisfaction of taxpayers related to the
examination of large corporate returns. However, as with the other goals,
the phrase “implement a comprehensive issue management strategy”
is too vague to indicate the future direction of IRS' programs and
activities. The statement also does not provide an indication of how
achieving this goal will impact taxpayers and overall compliance, focusing
instead on IRS' internal processes. In addition, the vagueness and
generality of the goal statement provide little basis for formulating
objectives. For example, if IRS has identified the factors contributing to
taxpayer dissatisfaction, the goal statement should articulate how IRS will
deal with those factors.
Appendix III: Examples of Action Items Developed by IRS Organizational Units
Page 27 GAO- 01- 234 IRS' Performance Management System
This appendix provides examples of action items developed by IRS managers in
the examination and collection functions. The action items are grouped in
two categories, those that were specific, measurable, and outcome or output
oriented, and those that were not.
The following action items were determined to be specific, measurable, or
outcome or output oriented.
? “Branch will establish a quarterly review of 5 cases from each of
four pure general program groups. The five cases will be reviewed by a
manager from another group for adherence to the EQMS standard.”
? “Specialty Group Managers (E & G [Estate and Gift], SEP [Special
Enforcement Program], Employment Tax) will ensure attorneys and examiners
receive information regarding the recommendations of Business Measures Task
Force Study within 60 days of issuance of report.”
? “Ensure all contact employees receive examination customer service
training by December 31, 1999.”
? “Train CEP [Coordinated Examination Program] and Specialty agents on
recommendations of Business Measures Task Force Study within 60 days of
issuance of report.”
? “Compile 9/ 30/ 99 data by 12/ 31/ 99 for baseline
compilations.”
? “Ensure all employees receive Customer Service Examination Specific
training by 12/ 31/ 99.”
? “CEB- 3 [Chief of Examination Branch 3] will attend one group
meeting each month to discuss employee feedback and elevate issues when
appropriate.”
? “Group manager will conduct field visits with revenue officers at
least once during the fiscal year.”
? “Complete balanced measures training 8530( A) by 12/ 31/ 99.”
? “Improve manager and employee understanding of auditing standards
through consistency reviews and at least one non- evaluative live case
review per examiner by 12/ 31/ 99.”
? “Managers and secretaries will not use the VMS [Voice Mail System]
but will answer the phones personally. R/ A [revenue agent], T/ A [tax
auditor], and ROE [revenue officer examiner] will use VMS system when not in
the office, but will leave dated and current messages for callers to
receive.” Appendix III: Examples of Action Items
Developed by IRS Organizational Units Specific, Measurable, and Outcome- or
Output- Oriented Action Items
Appendix III: Examples of Action Items Developed by IRS Organizational Units
Page 28 GAO- 01- 234 IRS' Performance Management System
In contrast to the above action items, the following were determined not to
be specific, measurable, and outcome or output oriented.
? “Support on- going Research Strategy Projects.”
? “Ensure employees have necessary tools to perform duties (on-
going).”
? “Implement/ follow the district's market segment plan.”
? “Identify improvement areas and implement actions to address
them.”
? “Ensure managers are correctly implementing the balanced
measures.”
? “Monitor group plans to ensure actions are being taken to improve
quality.”
? “Minimize neglected taxpayer groups.”
? “Emphasize one stop service in our examinations.”
? “Dialogue, coach, collaborate, and look forward when planning and
making decisions.”
? “Meet District/ Division due date or time line.”
? “Identify potential areas for outreach activities.”
? “Strive to understand and solve problems from the taxpayer's point
of view.”
? “Increase available help for those taxpayers who are calling in or
writing in for assistance.”
? “Use new IRM [Internal Revenue Manual] standards for operational
reviews (when available).”
? “Utilize all forms of communication available to facilitate keeping
employees informed.”
? “Use balanced measures as part of the decision making
process.”
? “Identify transition issues.”
? “Create a positive environment that encourages open communication
(e. g. coaching and mentoring).”
? “Monitor and balance between providing quality customer service
support and providing quality customer service to Collection taxpayers
across all branches.” Unspecific,
Unmeasurable, and Not Outcome- or Output- Oriented Action Items
Appendix IV: Comments From the Internal Revenue Service
Page 29 GAO- 01- 234 IRS' Performance Management System
Appendix IV: Comments From the Internal Revenue Service
Appendix IV: Comments From the Internal Revenue Service
Page 30 GAO- 01- 234 IRS' Performance Management System
Appendix IV: Comments From the Internal Revenue Service
Page 31 GAO- 01- 234 IRS' Performance Management System
Appendix V: GAO Contacts and Staff Acknowledgments
Page 32 GAO- 01- 234 IRS' Performance Management System
James R. White (202) 512- 9110 Ralph Block (415) 904- 2150
In addition to those named above, Benjamin Douglas, Suzy Foster, Jonda Van
Pelt, and John Zugar made key contributions to this report. Appendix V: GAO
Contacts and Staff
Acknowledgments GAO Contacts Acknowledgments
(268898)
The first copy of each GAO report is free. Additional copies of reports are
$2 each. A check or money order should be made out to the Superintendent of
Documents. VISA and MasterCard credit cards are also accepted.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail:
U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013
Orders by visiting:
Room 1100 700 4 th St., NW (corner of 4 th and G Sts. NW) Washington, DC
20013
Orders by phone:
(202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet
For information on how to access GAO reports on the Internet, send an email
message with “info” in the body to:
Info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov
Contact one:
? Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
? E- mail: fraudnet@ gao. gov
? 1- 800- 424- 5454 (automated answering system) Ordering Information
To Report Fraud, Waste, and Abuse in Federal Programs
*** End of document ***