Tax Administration: IRS Faces Challenges in Measuring Customer Service
(Letter Report, 02/23/98, GAO/GGD-98-59).
Pursuant to a congressional request, GAO reviewed the Internal Revenue
Service's (IRS) efforts to implement the Government Performance and
Results Act, focusing on: (1) IRS's system of performance measures; and
(2) the challenges IRS faces in developing and implementing performance
measures to gauge its efforts to reduce taxpayer burden through improved
customer service.
GAO noted that: (1) IRS is striving to develop and implement a
results-oriented performance measurement system that will meet the
requirements of the Results Act; (2) however, IRS faces some difficult
challenges in measuring the results of its efforts to reduce taxpayer
burden through improved customer service; (3) IRS has a three-tiered
system of performance measures; (4) at the highest level, IRS has a
mission effectiveness indicator, which is intended to measure the
agency's overall performance in collecting the proper amount of tax
revenue at the least cost or burden to the government and the taxpayer;
(5) the second level of indicators is intended to gauge IRS' progress in
meeting its strategic objectives to improve customer service, increase
taxpayer compliance, and increase its productivity; (6) to gauge its
progress in improving customer service, IRS uses five initial
indicators: (a) taxpayer burden cost for IRS to collect $100: (b)
initial contact resolution rate for taxpayer inquiries; (c) toll-free
telephone level of access; (d) tax law accuracy rate for taxpayer
inquiries; and (e) customer satisfaction rates (being developed at the
time of GAO's review); (7) the third level of indicators is intended to
measure the accomplishments of specific IRS programs or operations, such
as IRS' toll-free telephone operations; (8) IRS' 1997 Annual Performance
Plan had 30 program-level customer service measures, which measure such
things as the number of taxpayer calls answered and the average number
of calls answered per full-time employee; (9) although IRS is striving
to improve its overall performance measurement system, it faces some
difficult challenges as it develops and implements performance measures
to gauge its efforts to reduce taxpayer burden through improved customer
service; (10) the key challenges GAO identified are: (a) developing a
reliable measure of taxpayer burden; (b) developing measures that can be
used to compare the effectiveness of the various customer service
programs; and (c) refining or developing new measures that gauge the
quality of the services provided; (11) it is important that IRS obtain
stakeholder involvement to balance its efforts between assisting
taxpayers and enforcing compliance with the tax laws; (12) IRS
recognizes the limitations of its taxpayer burden measure and is looking
for alternatives; and (13) at the same time, IRS will be faced with
making decisions on how to minimize the costs of collecting data and
measuring results over time.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-98-59
TITLE: Tax Administration: IRS Faces Challenges in Measuring
Customer Service
DATE: 02/23/98
SUBJECT: Strategic planning
Agency missions
Taxpayers
Tax administration systems
Taxes
Customer service
Internal controls
Program evaluation
Accountability
IDENTIFIER: Government Performance and Results Act
GPRA
IRS Business Master Plan
IRS Problem Resolution Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman, Subcommittee on Oversight, Committee on Ways
and Means, House of Representatives
February 1998
TAX ADMINISTRATION - IRS FACES
CHALLENGES IN MEASURING CUSTOMER
SERVICE
GAO/GGD-98-59
Measuring Customer Service
(268758)
Abbreviations
=============================================================== ABBREV
ACS - Automated Collection System
BMP - Business Master Plan
FTE - full-time equivalent
IRS - Internal Revenue Service
MEI - Mission Effectiveness Indicator
OMB - Office of Management and Budget
PRP - Problem Resolution Program
TCMP - Taxpayer Compliance Measurement Program
TDA - Taxpayer Delinquent Account
TDI - Taxpayer Delinquency Investigation
TPS - Taxpayer Services
Letter
=============================================================== LETTER
B-274295
February 23, 1998
The Honorable Nancy L. Johnson
Chairman, Subcommittee on Oversight
Committee on Ways and Means
House of Representatives
Dear Chairman Johnson:
The Internal Revenue Service (IRS) defines its mission to be the
collection of the proper amount of federal taxes owed at the least
cost to taxpayers and the government. In fiscal year 1996, IRS
collected over a trillion dollars and had a budget of approximately
$7 billion. Although IRS spends millions of dollars on programs to
assist taxpayers in complying with the tax laws and meeting their tax
obligations, IRS has been criticized by the press, the public, and
Congress for its inability to meet the tax assistance needs of
taxpayers.
For years, IRS has been developing and revising a system of measures
to gauge its performance in achieving its mission, including reducing
taxpayer burden through improved customer service. Passage of the
Government Performance and Results Act in 1993 (Results Act) has led
IRS to begin refocusing its performance measures on the results of
its programs and operations. This is a difficult undertaking,
because it requires IRS to shift its focus of management and
decisionmaking away from a preoccupation with staffing, activity
levels, and tasks completed to a focus on the real difference its
programs make on taxpayers.
Recognizing the difficulty agencies face in meeting the requirements
of the Results Act, this report responds to your request that we
provide information on IRS' performance measures, particularly for
those dealing with customer service. Specifically, in this report,
our objectives are to (1) describe IRS' system of performance
measures and (2) identify any challenges IRS faces in developing and
implementing performance measures to gauge its efforts to reduce
taxpayer burden through improved customer service.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
IRS is striving to develop and implement a results-oriented
performance measurement system that will meet the requirements of the
Results Act. However, IRS faces some difficult challenges in
measuring the results of its efforts to reduce taxpayer burden
through improved customer service.
IRS has a three-tiered system of performance measures. At the
highest level, (mission level) IRS has a mission effectiveness
indicator, which is intended to measure the agency's overall
performance in collecting the proper amount of tax revenue at the
least cost or burden to the government and the taxpayer. The second
level of indicators (strategic level) is intended to gauge IRS'
progress in meeting its strategic objectives to improve customer
service, increase taxpayer compliance, and increase its productivity.
To gauge its progress in improving customer service, IRS uses five
indicators: (1) taxpayer burden cost for IRS to collect $100, (2)
initial contact resolution rate for taxpayer inquiries, (3) toll-free
telephone level of access, (4) tax law accuracy rate for taxpayer
inquiries, and (5) customer satisfaction rates (being developed at
the time of our review).
The third level of indicators (program level) is intended to measure
the accomplishments of specific IRS programs or operations, such as
IRS' toll-free telephone operations. IRS' 1997 Annual Performance
Plan had 30 program-level customer service measures, which measure
such things as the number of taxpayer calls answered and the average
number of calls answered per full-time employee.
Although IRS is striving to improve its overall performance
measurement system, it faces some difficult challenges as it develops
and implements performance measures to gauge its efforts to reduce
taxpayer burden through improved customer service. The key
challenges we identified are (1) developing a reliable measure of
taxpayer burden, including the portion that IRS can influence; (2)
developing measures that can be used to compare the effectiveness of
the various customer service programs; and (3) refining or developing
new measures that gauge the quality of the services provided.
Additionally, as IRS refines its strategic goals and related
measures, it is important that IRS obtain stakeholder involvement to
balance its efforts between assisting taxpayers and enforcing
compliance with the tax laws.
IRS recognizes the limitations of its taxpayer burden measure and is
looking for alternatives. However, devising ways to measure the
burden that IRS influences and overcoming the other challenges we
identified will not be easy for IRS as it strives to meet the
requirements of the Results Act. IRS will be faced with devising
reliable measures that are useful in improving agency and program
performance, improving accountability, and in supporting policy
decisionmaking. At the same time, IRS will be faced with making
decisions on how to minimize the costs of collecting data and
measuring results over time.
BACKGROUND
------------------------------------------------------------ Letter :2
According to IRS, the agency's overall mission is to collect the
proper amount of tax revenue at the least cost; serve the public by
continually improving the quality of its products and services; and
perform in a manner warranting the highest degree of public
confidence in IRS' integrity, efficiency, and fairness. Its
strategic objectives are to improve customer service, increase
compliance with the tax laws, and increase IRS' productivity.
Essentially, IRS is striving to encourage taxpayers to pay what they
owe, reduce taxpayers' cost to get answers to their questions and
prepare their tax returns, and reduce IRS' cost to collect federal
taxes.
In the mid-1980s, IRS' Strategic Business Plan first provided IRS'
mission statement, objectives, general strategies, and goals. IRS
created the Fiscal Year 1995-2001 Business Master Plan (BMP) to
incorporate IRS' vision and long-range objectives. The fiscal year
1996 BMP formalized IRS' "measures hierarchy," which was intended to
link IRS' mission, objectives, and annual performance goals with
respective programs. For fiscal year 1997, IRS replaced the BMP with
the Strategic Plan and Budget and the Annual Performance Plan. On
September 30, 1997, IRS released a new Strategic Plan that updates
its strategic measures. IRS expects to use these documents to
provide guidance to its field offices and to implement the Results
Act.
According to IRS, improving taxpayer service is one of its highest
priorities, and it has a variety of programs and operational units to
assist taxpayers in meeting their federal tax obligations. A primary
source of taxpayer assistance is IRS' 24 customer service centers,
which are to answer calls from taxpayers who have questions about the
tax laws, where to file returns, or the status of their accounts and
refunds. According to IRS, in fiscal year 1996, the centers answered
over 99 million taxpayer calls about tax law and procedures. Other
sources of assistance include IRS' walk-in sites, taxpayer education
and outreach programs, Problem Resolution Program, and Internet web
site. According to IRS, about 440 walk-in sites helped almost 6.4
million taxpayers in fiscal year 1996 with tax forms, questions about
their accounts, or preparing tax returns.
In addition, over the past several months, IRS has been working to
improve its measures and has consulted with many stakeholders. A
task force has been formed with representatives from Treasury and OMB
to develop a "balanced scorecard."
IRS' taxpayer education and outreach programs assist millions of
taxpayers at various community locations, often with the help of
volunteers and nonprofit organizations. For example, almost 12.7
million taxpayers in fiscal year 1996 received free tax information
and return preparation through IRS' Volunteer Income Tax Assistance,
Tax Counseling for the Elderly, and other outreach programs. The
Problem Resolution Program staff assists taxpayers who have such
problems as repeated unsuccessful attempts to resolve an issue or a
pending IRS enforcement action that might cause undue hardship, such
as the seizure of a taxpayer's property. Also, taxpayers may use the
Internet to obtain forms and instructions, publications, information
on tax topics, and press releases. According to IRS, its web site
had 73 million "hits" in fiscal year 1996.
IRS has had several efforts under way to improve its performance
measures. For example, the agency established the Measures Advisory
Group, comprising field and National Office executives, in part to
provide advice and recommendations on the agency's performance
measures. As the group suggested, IRS recently developed three new
performance measures for its customer service centers: (1) customers
successfully served per dollars expended, (2) dollars collected per
dollars expended, and (3) taxpayers gaining access to telephone
assistance as a percentage of demand. IRS' September 30, 1997,
Strategic Plan included the first measure as a strategic-level
indicator for increasing productivity and the third measure as a
strategic-level indicator for improving customer service. IRS plans
to use the second measure as a program-level indicator.
The Results Act requires federal agencies to measure the results of
their programs and operations. Agencies are expected to set goals,
measure performance, make needed improvements, and report results.
The Results Act required executive agencies, no later than September
30, 1997, to have developed strategic plans covering a period of at
least 5 years and have submitedt them to Congress and the Office of
Management and Budget (OMB). Strategic plans are intended to be the
framework for each agency's performance measurement system. The
Results Act also requires agencies to develop annual performance
plans that are intended to reinforce the link between strategic goals
and day-to-day activities. The first annual performance reports,
covering fiscal year 1999, are due by March 31, 2000.
Implementation of the Results Act requires adequate and reliable
performance measures that are useful in improving agency and program
performance, improving accountability, or supporting policy
decisionmaking. IRS recognizes that collecting such data can be
costly and difficult. As with other federal agencies, IRS will have
to balance the cost of data collection efforts against the need to
ensure that the collected data are complete, accurate, and consistent
enough to document performance and support decisionmaking at various
organizational levels.
In conjunction with developing the required strategic plans, federal
agencies are required to solicit views of other stakeholders to
clarify their missions and reach agreement on their goals. This
statutory requirement was, in part, designed to address instances
where Congress, the agency, and other interested parties may disagree
because of competing priorities.
OBJECTIVES, SCOPE, AND
METHODOLOGY
------------------------------------------------------------ Letter :3
Our objectives were to (1) describe IRS' system of performance
measures and (2) identify any challenges IRS faces in developing and
implementing performance measures to gauge its efforts to reduce
taxpayer burden through improved customer service.
To describe IRS' performance measures, we reviewed IRS' fiscal year
1997 Strategic Plan and Budget, including the Annual Performance
Plan; its updated September 30, 1997, Strategic Plan; and other
planning documents, including IRS' fiscal year 1996 Business Master
Plan and Business Review. We also interviewed the staff of the
National Director of Compliance Research and the National Director
and staff of the Strategic Planning Division, who are responsible for
developing the Annual Performance Plan; and officials in the Analysis
and Studies Division, who conduct the Business Review and are
responsible for establishing selected strategic measures.
To identify any challenges IRS faces in developing and implementing
performance measures to gauge its efforts to reduce taxpayer burden
through improved customer service, we reviewed IRS' strategic-level
measures to improve customer service in its September 30, 1997,
Strategic Plan for fiscal years 1997 through 2002; its fiscal year
1997 Strategic Plan; and selected program-level customer service
measures in its Annual Performance Plan for fiscal year 1997. Using
criteria drawn from the steps and critical practices set forth in
GAO's Executive Guide: Effectively Implementing the Government
Performance and Results Act (GAO/GGD-96-118, June 1996), we examined
the strategic-level and new customer service measures to determine if
they were based on sound methodologies and were useful in improving
agency and program performance and in supporting agency policy
decisionmaking. For example, we analyzed the strategic-level
taxpayer burden indicator to determine whether it (1) was linked to
the burden IRS can influence and the services it provides; and (2)
measured the full range of costs that taxpayers' incur, including the
costs they incur after they file their returns. Similarly, we
examined IRS' definition of initial contact resolution to determine
what services IRS measures and the contacts that are counted as
successful.
We reviewed the fiscal year 1997 Performance Plan to determine
whether IRS had comparable program-level indicators for the different
sources of assistance, including customer service centers, walk-in
sites, the Problem Resolution Program, the Education and Outreach
Program, and the Internet web site. We selected these five units or
programs because they are primary sources of assistance for taxpayers
who need help from IRS. Additionally, we examined IRS' definition of
"customers successfully served" to determine whether IRS considered
the quality of the service, such as how many times the taxpayers had
to call before being assisted.
We also interviewed IRS' National Office, Atlanta Service Center,
Southeast Region, Georgia District, and Nashville District officials
who were responsible for either developing, implementing, or using
the customer service performance measures to determine the status of
IRS' system of performance measures and to obtain an understanding of
the newly developed customer service measures.
We did our work from September 1996 through December 1997 in
accordance with generally accepted government auditing standards. We
requested comments on a draft of this report from the Commissioner of
Internal Revenue or his designated representative. Responsible IRS
officials, including Chief Management and Administration; the
National Director, Strategic Planning Division; and staff of the
Executive Officer for Customer Service provided oral comments and
factual clarifications in a January 21, 1998, meeting. We have
incorporated those comments in the report where appropriate. The
Commissioner of IRS provided us written comments on January 23, 1998,
which are discussed near the end of this report and reproduced in
appendix III.
IRS HAS THREE TIERS OF
PERFORMANCE MEASURES
------------------------------------------------------------ Letter :4
IRS' system of performance measures has three tiers: mission,
strategic, and program. IRS has 1 mission effectiveness measure, 3
strategic objectives with 9 measures, and 111 program measures, as
depicted in figure 1.\1 See appendix I for definitions of
mission-level, strategic-level, and selected customer service
program-level measures.
Figure 1: IRS' Performance
Measures
(See figure in printed
edition.)
Source: IRS' September 30, 1997, Strategic Plan for fiscal years
1997 through 2002 and its 1997 Annual Performance Plan.
--------------------
\1 The mission effectiveness and strategic measures are depicted as
illustrated in IRS' September 30, 1997, Strategic Plan for fiscal
years 1997 through 2002. The program measures were taken from IRS'
fiscal year 1997 Annual Performance Plan.
FIRST-TIER MEASURE IS TO
GAUGE OVERALL PERFORMANCE
---------------------------------------------------------- Letter :4.1
IRS' mission-level effectiveness indicator (MEI) is intended to
measure the agency's performance in accomplishing its primary mission
of collecting the proper amount of tax revenue at the least cost.
The MEI compares total revenue collected during a fiscal year, less
the cost of collecting the revenue (the sum of IRS' budget and
estimated taxpayers' costs), to the revenue that should have been
collected if all taxpayers had paid their full liability. With the
MEI, IRS has a mission-level performance indicator that includes the
taxpayer compliance rate, the cost or burden to taxpayers of
complying with the tax laws, and the cost of operating IRS.
SECOND-TIER MEASURES ARE TO
GAUGE PROGRESS IN ACHIEVING
STRATEGIC OBJECTIVES
---------------------------------------------------------- Letter :4.2
The second tier of measures includes nine performance indicators that
are intended to gauge IRS' progress in achieving its three strategic
objectives--improve customer service, increase compliance, and
increase productivity.
These three objectives link directly to the MEI, because improving
customer service reduces taxpayer burden, increasing compliance
increases the compliance rate, and increasing productivity reduces
IRS cost.
IMPROVE CUSTOMER SERVICE
-------------------------------------------------------- Letter :4.2.1
To improve customer service, IRS seeks to better serve the public,
reduce taxpayer burden, and increase public confidence in the tax
administration system. IRS seeks to improve taxpayer access, resolve
as many inquiries as possible on the first contact, and increase
customer satisfaction. IRS states that improving customer service
supports its mission to collect the proper amount of tax at the least
cost to taxpayers and IRS.
IRS uses five indicators to gauge its progress in improving customer
service: (1) taxpayer burden cost for IRS to collect $100, (2)
initial contact resolution rate, (3) toll-free telephone level of
access, (4) tax law accuracy rate for taxpayer inquiries, and (5)
customer satisfaction rates (being developed at the time our review).
According to IRS, the taxpayer burden measure is the principal
measure of its efforts to improve customer service.
INCREASE COMPLIANCE
-------------------------------------------------------- Letter :4.2.2
To increase compliance, IRS seeks to encourage and assist taxpayers
to file timely and accurate returns and to pay their taxes on time.
If taxpayers do not comply, IRS intends to take appropriate action to
force the taxpayers to comply. Also, to help improve customer
satisfaction, IRS intends to treat taxpayers with courtesy, fairness,
and professionalism.
IRS uses two indicators to gauge its progress in increasing
compliance: (1) total collection percentage and (2) total net
revenue collected. According to IRS, the principal measure of
taxpayer compliance is the total collection percentage, or the
comparison of the revenue IRS collects with the total tax liability.
INCREASE PRODUCTIVITY
-------------------------------------------------------- Letter :4.2.3
To increase productivity, IRS seeks to continually improve operations
and the quality of products and services it provides to customers
through reengineering and a highly skilled work force. IRS states
that accomplishing this objective will increase compliance, improve
customer service, and reduce the cost of tax administration.
To gauge its progress toward accomplishing this objective, IRS uses
two indicators: (1) budget cost to collect $100 and (2) customers
successfully served per dollars expended. According to IRS, its
principal productivity measure is the amount it spends to collect
$100 as measured by comparing IRS' budget to the revenues it
collects.
THIRD-TIER MEASURES ARE TO
GAUGE PROGRAM PERFORMANCE
---------------------------------------------------------- Letter :4.3
The third tier of measures--111 in all--is intended to gauge how well
specific IRS programs are performing. IRS' fiscal year 1997 Annual
Performance Plan had 16 submission processing measures, 30 customer
service measures, and 38 compliance measures. The plan also had 8
Service-wide measures for which all IRS executives and managers
shared responsibility and 19 other measures specific to such areas as
resource management and business operations.
Appendix II provides a complete list of IRS' 30 program-level
customer service performance measures.
IRS FACES CHALLENGES DEVELOPING
AND IMPLEMENTING CUSTOMER
SERVICE MEASURES
------------------------------------------------------------ Letter :5
IRS is striving to develop and implement a results-oriented
performance measurement system to meet the requirements of the
Results Act. However, IRS faces some difficult challenges in
measuring the results of its efforts to reduce taxpayer burden
through improved customer service. The key challenges we identified
are (1) developing a reliable measure of taxpayer burden, including
the portion that IRS can influence; (2) developing measures that can
be used to compare the effectiveness of the various customer service
programs; and (3) refining or developing new measures that gauge the
quality of the services provided. As IRS refines its customer
service measures, it must consider the costs of implementing the
measures, including the costs of collecting and analyzing data over
time.
DEVELOPING A RELIABLE
TAXPAYER BURDEN MEASURE
LINKED TO IRS SERVICES
---------------------------------------------------------- Letter :5.1
IRS' taxpayer burden indicator is intended to measure taxpayers' cost
for IRS to collect $100. IRS measures taxpayer burden by using a
model that estimates the time taxpayers spend on each tax form using
variables such as the number of lines on a tax form. The burden
estimate excludes the time and costs taxpayers face after they file
their tax returns, such as responding to IRS notices and audits.
Additionally, it is not linked to important IRS services to assist
taxpayers in meeting their tax obligations. As a result, IRS' burden
estimates may not reflect reductions in taxpayer burden that are
attributable to these services. The flaws in the burden measure also
limit the usefulness of IRS' mission effectiveness indicator, because
burden is a key component of this indicator. IRS recognizes the
limitations of its burden measure and is looking for alternatives.
IRS calculates its burden indicator by using a model developed by
Arthur D. Little, Inc. The model estimates the time a taxpayer
spent on each tax form using variables, such as the number of lines
on the form, number of words and pages in the related instruction
booklet, and the number of references to the Internal Revenue Code.
IRS then converts this total time estimate to dollar costs by
multiplying the total time by IRS' estimate of the value of a
taxpayer's hour.
IRS' burden model excludes the time and costs taxpayers incur after
tax forms are filed, such as the time taxpayers spend inquiring about
the status of a tax refund or responding to notices, examinations, or
other IRS-initiated compliance activities. In a recent draft issue
paper, IRS identified several other shortcomings in the model,
including weaknesses in the underlying assumptions of the model. For
example, the model uses the number of lines on a form to estimate the
form's burden, even though additional lines may make the form simpler
or easier to understand. Also, the draft issue paper said that the
1983 data underlying the model are outdated and cited methodological
errors in the development of the model.
The limitations of the taxpayer burden measure affect IRS' mission
effectiveness indicator because taxpayer burden is one of the four
elements of this indicator. The indicator compares total revenue
collected during a fiscal year, less the cost of collecting the
revenue (the sum of IRS' budget and estimated taxpayers' costs) with
the revenue that should have been collected if all taxpayers had paid
their full liability. The usefulness of IRS' overall measure is
questionable considering the limitations of the taxpayer burden
measure.\2
IRS recognizes the limitations of the current methodology for
measuring burden and in 1995 sought help in developing an improved
methodology for measuring all facets of taxpayer burden.
Specifically, IRS issued a request for proposals seeking contractors
to develop an approach for measuring taxpayer burden, including the
burden after forms are filed. However, according to IRS, no
contractors were interested in doing the work.
The lack of response to IRS' request may reflect the difficulty of
measuring overall compliance burden. In our December 9, 1994,
testimony before the Subcommittee on Oversight, House Committee on
Ways and Means, we discussed the difficulties of measuring taxpayer
burden and reported that a reliable estimate of the overall burden
taxpayers incur to comply with the tax laws was not available.\3 As a
part of our study, we spoke with several business and tax
professionals, who told us that the complexity of the Internal
Revenue Code, compounded by the frequent changes made to the Code, is
part of what makes federal tax compliance so burdensome.
Recently, IRS initiated another effort to obtain a contractor to
develop an improved burden measurement model and is now in the
initial stages of determining contractor interest. In the short
term, IRS plans to expand its current measure of taxpayer burden to
include contact and enforcement burden, such as the burden taxpayers
incur when responding to IRS notices, telephone calls, and audits.
Despite recognizing the shortcomings in the current taxpayer burden
measure, IRS has set goals for reducing burden based on the measure.
It then rolls these goals up into its mission effectiveness
indicator. To show progress through this indicator, IRS must reduce
the number of lines on tax forms or worksheets, reduce the number of
words and pages in instructions, or take actions that affect the
variables in the Little model. However, the model does not
distinguish between lines on forms that add to burden and lines that
reduce the burden by making the calculation of tax liability easier.
Unless additional analysis is done to assess how eliminating
particular lines on forms affects burden, IRS could take actions to
meet its goals that actually increase taxpayer burden. Furthermore,
most IRS customer service programs have no effect on IRS' measure of
taxpayer burden.
Devising a comprehensive measure to gauge the costs taxpayers incur
to meet their federal tax obligations is a difficult task and offers
a significant challenge for IRS. First, IRS would need to devise a
means to capture the costs taxpayers incur after they file their
returns. This may be difficult to do, because the costs could vary
substantially depending on the circumstances of the different
taxpayers. For example, providing information to support a tax
return may not cost very much when compared to the cost of preparing
for and responding to an audit. Second, because of the limitations
of the Little model, IRS must decide whether to revise the model or
to devise another means to estimate the costs taxpayers incur to
prepare and file their tax returns. Third, IRS must measure the
elements of burden it can influence as opposed to the burden caused
by such things as changes in the tax code. A reliable taxpayer
burden measure would allow IRS to make decisions on how to allocate
resources to best reduce the burdens taxpayers face to meet their tax
obligations. Finally, as IRS refines its taxpayer burden measure, it
will be faced with devising an efficient means for collecting and
analyzing the data to measure burden over time. Otherwise, the cost
of measuring burden could exceed the benefits.
--------------------
\2 Also, IRS has acknowledged that its estimate of total tax
liability is based on outdated Taxpayer Compliance Measurement
Program (TCMP) data. TCMP is a program that measures compliance
nationwide for selected types of taxpayers--for example, individuals
and small corporations.
\3 Tax System Burden: Tax Compliance Burden Faced by Business
Taxpayers (GAO/T-GGD-95-42, Dec. 9, 1994).
DEVELOPING COMPARABLE
MEASURES FOR DIFFERENT
CUSTOMER SERVICE PROGRAMS
---------------------------------------------------------- Letter :5.2
Among other things, the Results Act requires agencies to develop and
implement measures that are useful in improving program performance
or in supporting policy decisionmaking. One way IRS can do this is
to develop measures that can be used to compare the effectiveness of
the different customer service programs. Our analysis of IRS' fiscal
year 1997 program-level measures for customer service points out the
need for such measures, but the history of the initial contact
resolution measure demonstrates the difficulty IRS faces in
implementing such measures. Although IRS has three new
strategic-level customer service measures for fiscal year 1998,
similar to the initial contact resolution measure, two are limited to
measuring telephone assistance.
When taxpayers need assistance from IRS, among other things, they can
call a customer service center, visit a walk-in site, call or visit a
problem resolution office, call or visit an outreach facility, or
access IRS' Internet web site. IRS' 1997 Annual Performance Plan had
27 program-level indicators for its customer service centers and 3
for its Problem Resolution Program. However, the plan had no
program-level indicators to measure the performance of the walk-in
sites, education and outreach programs, and the Internet web site,
even though these three sources of assistance provide a range of
services to help taxpayers file their returns and otherwise comply
with the tax laws and reporting requirements.
One of IRS' fiscal year 1997 strategic-level indicators for measuring
its progress in improving customer service was the initial contact
resolution rate. This measure is intended to gauge IRS' progress in
satisfactorily resolving all issues resulting from a taxpayer's first
inquiry to IRS--formerly known as the "one-stop service" concept.
Providing one-stop service would reduce taxpayer burden and the
demand for IRS services. However, since the August 1991
implementation of its one-stop service goal, IRS has often redefined
the goal and the types of contacts that are counted as successful and
plans to change the goal again.
Originally, IRS' measurement focused on account-related taxpayer
inquiries at district toll-free telephone sites. In our August 1994
report, we concluded that IRS was overstating its successes for
one-stop service because it was counting calls that did not fully
resolve the taxpayers' questions.\4 We recommended that IRS develop
better measures to exclude those instances where taxpayers would
likely need to contact IRS again about the same matter. We also
recommended that IRS measure all types of taxpayer inquiries,
including all telephone contacts, service center correspondence, and
walk-in inquiries. In March 1995, IRS changed the name of the
measure to "initial contact resolution" and incorporated our
recommendations to include correspondence and walk-in inquiries.
Officials told us that IRS was establishing a new definition for
fiscal year 1998 that would be limited to telephone operations, which
was recommended in a recent internal audit report.\5 The internal
audit report did not address the need for measuring other types of
IRS assistance, such as education and outreach and walk-in.
Essentially, the report concluded that IRS' initial contact
resolution measure should be limited to telephone operations, because
the inclusion of correspondence would add responses to notices that
had, in the past, taken up to 60 days to resolve. An IRS official
told us that the initial contact resolution measure would not include
walk-ins because (1) IRS does not have a system in place to measure
the rate; (2) it is very difficult to monitor walk-in contacts in a
valid way without standing over the individual customer service
representative; and (3) the volume is relatively small compared to
telephone contacts and paper correspondence and, as a result, would
not affect the measure very much. Because IRS' customer service
programs vary, without comparable measures, IRS is unable to compare
the performance and effectiveness of the different customer service
programs.
Comparable measures for the customer service programs would allow IRS
to monitor the performance and compare the effectiveness of the
different programs. Such comparisons would assist IRS in making
decisions on how to allocate resources among the different programs
to maximize results. However, developing comparable measures of
effectiveness will be difficult, primarily because of the range of
services and options taxpayers have when they need assistance from
IRS. Also, IRS would need to consider the costs of collecting and
analyzing the data to measure performance of the different programs.
IRS added three new strategic-level measures in its September 30,
1997, Strategic Plan: (1) toll-free telephone level of access, which
is intended to compare the number of calls attempted to the number of
calls answered; (2) tax law accuracy rate for taxpayer inquiries,
which is intended to measure the accuracy of tax law information
provided to taxpayers through the toll-free telephone assistance
program; and (3) customer satisfaction rates. Similar to the initial
contact resolution measure, the first two measures are also limited
to the telephone program, even though taxpayers have other sources,
such as walk-in sites and the Internet, to obtain answers to their
tax law questions. At the time of our review, IRS was in the process
of determining how to measure customer satisfaction.
--------------------
\4 Tax Administration: Better Measures Needed to Assess Progress of
IRS' One-Stop Service (GAO/GGD-94-131, Aug. 29, 1994).
\5 Achieving Initial Contact Resolution Rate (ICR) In Customer
Service, IRS Internal Audit (July 29, 1997).
REFINING MEASURES TO BETTER
GAUGE THE QUALITY OF
SERVICES PROVIDED
---------------------------------------------------------- Letter :5.3
One of IRS' new strategic-level productivity measures for its 24
customer service centers for fiscal year 1998 is "customers
successfully served per dollar expended." Our analysis of this
measure points out the need to better measure the quality of services
provided.
According to IRS' definition, successfully served means a taxpayer
received "an accurate response to a call or resolution of a case."
This definition does not consider other elements that would affect
what a taxpayer may consider as successful service, such as the
number of times the taxpayer called before being assisted, how long
the taxpayer had to wait before being served, and the courtesy and
professionalism of the assistor. As a result, the taxpayer, although
served, may not believe he or she was "successfully" served.
IRS' strategic-level customer service measures have similar
limitations. For example, the initial contact resolution measure is
intended to gauge IRS' performance in resolving issues resulting from
a taxpayer's first inquiry to IRS. The tax law accuracy rate measure
gauges the extent to which taxpayers are provided correct answers.
IRS does not measure such things as how long it took to resolve the
issues or how courteous and professional the assistors were when
interacting with the taxpayers or whether the need for the contact
could have been prevented.
Revising measures to better gauge the quality of assistance is a
major challenge for IRS. For example, developing measures of
timeliness will be very difficult because of the different programs
and the range of services they provide. Also, IRS would have to
devise a means to capture such data. As with other measures, IRS may
be faced with making trade-offs between how to refine the measures
and the cost of collecting the needed data.
OBTAINING STAKEHOLDERS'
INVOLVEMENT
------------------------------------------------------------ Letter :6
Although statutory requirements are to be the starting point for
agency mission statements, Congress, the executive branch, and other
interested parties may all disagree about a given agency's mission
and goals. The Results Act seeks to address such situations by
requiring federal agencies to consult with Congress and solicit the
views of other stakeholders in developing their strategic plans.
Stakeholder involvement is important to help agencies ensure that
their efforts and resources are targeted at the highest priorities.
Obtaining stakeholder involvement is especially important for IRS as
it seeks to balance its efforts and resources between assisting
taxpayers and enforcing compliance with the nation's tax laws.
Stakeholders could assist IRS in devising performance measures that
would enhance IRS' ability to make more informed decisions about how
to allocate its resources between the competing demands of assistance
and enforcement.
CONCLUSIONS
------------------------------------------------------------ Letter :7
IRS is striving to develop and implement a results-oriented
performance measurement system to meet the requirements of the
Results Act. However, IRS faces some difficult challenges as it
develops and implements its efforts to reduce taxpayer burden through
improved customer service. IRS will be faced with devising reliable
measures that are useful in improving agency and program performance,
improving accountability, or supporting policy decisionmaking. At
the same time, IRS will be faced with making decisions on how to
minimize the costs of collecting data and measuring results over
time.
IRS' taxpayer burden measure is not a useful guide to IRS performance
because it is based on flawed methodology that does not link to the
burdens IRS influences and the various services it provides.
Additionally, it does not measure burdens taxpayers face after they
file their tax returns. As a result, most IRS customer service
programs that IRS characterizes as customer service have no effect on
IRS' measure of taxpayer burden.
IRS does not have a comprehensive set of customer service indicators
that gauges the full range of taxpayer services. As a result, IRS is
unable to compare the performance of the different customer service
programs and make funding decisions based on the programs' costs and
benefits--a key goal of the Results Act. Developing comparable
measures for the different programs will be difficult, primarily
because of the range of assistance the different programs provide.
Similarly, IRS' customer service measures do not adequately measure
the quality of the services taxpayers receive from IRS. Although
some of the measures gauge the extent to which taxpayers' issues are
resolved or the accuracy of the information IRS provides, they do not
measure such things as how long it takes IRS to resolve the issues or
how courteous and professional the assistors are when interacting
with the taxpayers. Revising the measures to better gauge the
quality of assistance is a major challenge for IRS, primarily because
of the many different programs and the range of services they
provide. Also, IRS would have to devise a means to capture such
data.
Devising ways to measure taxpayer burden and overcoming the other
limitations we identified offer significant challenges for IRS as it
strives to meet the requirements of the Results Act. Not only will
IRS be faced with devising consistent, results-oriented measures for
a range of taxpayer services, it will also be faced with making
decisions on how to minimize the costs of collecting data and
measuring results over time. In doing so, IRS is also faced with
balancing competing priorities. To balance these competing
priorities, it is essential that IRS continue to involve those who
are served by IRS--the taxpayers--as well as other stakeholders, such
as Congress and the Office of Management and Budget.
RECOMMENDATIONS TO THE
COMMISSIONER OF INTERNAL
REVENUE
------------------------------------------------------------ Letter :8
As IRS refines its customer service performance measures, we believe
it is essential that IRS make the measures useful for managing the
different customer service programs, allocating resources, improving
accountability, and supporting policy decisions. Accordingly, as IRS
refines its customer service measures, we recommend that the
Commissioner of Internal Revenue direct the appropriate officials to
work to develop performance indicators that cover the full range of
IRS' customer service programs.
AGENCY COMMENTS
------------------------------------------------------------ Letter :9
We requested comments on a draft of this report from the Commissioner
of Internal Revenue or his designated representative. In a January
21, 1998, meeting responsible IRS officials, including the Chief
Management and Administration; the National Director, Strategic
Planning Division; and staff of the Executive Officer for Customer
Service provided oral comments and some factual clarifications, which
we incorporated in the report where appropriate. The Commissioner of
IRS provided us written comments on January 23, 1998 (see app. III).
He concurred with the report's findings and recommendation. He said
that IRS recognizes the critical importance of measuring customer
service and is working to improve its measures, including consulting
many stakeholders. He also said that IRS is working with a
contractor to develop customer satisfaction surveys for all business
lines that interact with the public.
On January 28, 1998, after our receipt of IRS' comments on the draft
of this report, the IRS Commissioner announced a conceptual framework
for a proposal to reorganize IRS to better align its activities into
organizational elements serving different types of taxpayers (e.g.,
individuals, large corporations). Although details of this proposed
reorganization are not available, and any IRS reorganization may be
affected by other proposals for IRS restructuring under consideration
by Congress, we note that the customer service measures discussed in
this report and any IRS plans to improve them may be affected by
these possible organizational changes.
---------------------------------------------------------- Letter :9.1
We are sending copies of this report to the Subcommittee's Ranking
Minority Member, the Chairmen and Ranking Minority Members of the
House Committee on Ways and Means and the Senate Committee on
Finance, various other congressional committees, the Secretary of the
Treasury, the Commissioner of Internal Revenue, the Director of the
Office of Management and Budget, and other interested parties.
Copies will also be made available to others upon request.
Major contributors to this report are listed in appendix IV. Please
contact me on (202) 512-9110 if you have any questions.
Sincerely yours,
Lynda D. Willis
Director, Tax Policy and
Administration Issues
DEFINITIONS OF MISSION-LEVEL,
STRATEGIC-LEVEL, AND SELECTED
CUSTOMER SERVICE PROGRAM-LEVEL
MEASURES
=========================================================== Appendix I
These definitions are as stated in the Internal Revenue Service's
(IRS) September 30, 1997, Strategic Plan for fiscal years 1997
through 2002, except for minor changes we made for clarity. We did
not validate these definitions.
MISSION LEVEL
--------------------------------------------------------- Appendix I:1
Mission Effectiveness Indicator: This compares the revenue IRS
expects to collect during a fiscal year, less the cost of collecting
that revenue, with the amount of revenue that IRS would collect if
all tax obligations were honored. The four components of this
measure are budget, total revenue, burden, and total tax liability.
-- Budget: This is the amount of money appropriated by Congress or
requested by IRS through Treasury and the Office of Management
and Budget (OMB).
-- Total revenue: This is all revenue collected by IRS, including
revenue resulting from enforcement activities, but excluding
refunds.
-- Burden: This is a "monetized" estimate of the number of burden
hours placed on taxpayers to meet their tax obligations. The
calculation is based on a methodology developed by Arthur D.
Little, Inc.
-- Total tax liability: This is an estimate of the amount of
individual income, corporate income, and employment taxes that
should have been paid in a given year, if all taxes that were
legally owed had been paid.
STRATEGIC LEVEL
--------------------------------------------------------- Appendix I:2
Improve Customer Service Objective: The purpose of this objective is
to better serve the public, reduce taxpayer burden, and increase
public confidence in the tax administration system. IRS seeks to
improve taxpayer access, resolve as many inquiries as possible on the
first contact, and increase customer satisfaction.
Taxpayer burden cost (in dollars) for IRS to collect $100: This
ratio measures the private sector costs compared to the cost for IRS
to collect $100 in net tax revenue. Net tax revenue includes all
revenue collected (income, employment, estate and gift, and excise
taxes) by IRS in a fiscal year, less refunds. Private sector costs
cover the paperwork burden imposed on the public as a result of the
federal tax reporting system administered by IRS. Private sector
costs of the paperwork burden are based on the estimated time
individual and business taxpayers spend keeping tax records, learning
about tax laws, preparing tax forms, and sending tax forms to IRS.
Taxpayer paperwork burden is converted from time to dollars by
multiplying total time by the estimated value of a taxpayer's hour.
-- Initial contact resolution rate: This measures the successful
resolution of all issues resulting from the taxpayer's first
inquiry, telephone only.
-- Toll-free telephone level of access: This is the percentage of
calls answered. The percentage is computed by comparing the
number of calls attempted (demand) to the number answered in all
components of the Customer Service function (Automated
Collection System, Customer Service Toll-free, and the
Centralized Inventory and Distribution System).
-- Tax law accuracy rate for taxpayer inquiries: This measures the
rate at which IRS' toll-free telephone assistance program
provides taxpayers accurate tax information.
-- Customer satisfaction rates: This measure was under development
at the time of our review.
Increase Compliance Objective: The purpose of this objective is to
encourage and assist taxpayers to voluntarily file timely and
accurate returns and to pay on time and, if taxpayers do not comply,
to take appropriate compliance actions.
-- Total collection percentage: This is the ratio of total
collections to total estimated true tax liability.
-- Total net revenue collected: This is all revenue collected by
IRS, including revenue resulting from enforcement activities,
but excluding refunds.
Increase Productivity Objective: The purpose of this objective is to
continually improve operations and the quality of products and
services provided to taxpayers, using systems management tools and a
highly skilled work force.
-- Budget cost to collect $100 in revenue: This ratio measures the
IRS budget cost of collecting $100 in net tax revenue. Net tax
revenue includes all revenue collected (income, employment,
estate and gift, and excise taxes) by IRS in a fiscal year, less
refunds.
-- Customers successfully served per dollars expended: This
measure calculates the average cost for IRS' 24 customer service
centers to accurately respond to a taxpayer's inquiry.
SELECTED CUSTOMER SERVICE
PROGRAM LEVEL\6
--------------------------------------------------------- Appendix I:3
Number of calls answered: This is the number of calls accepted by
the Automatic Call Distributor system, including calls where the
caller chooses Tele-Tax (an interactive, self-directed system) during
and after business hours.
Number of assistor calls answered: This is the number of calls
accepted by the Automatic Call Distributor system and answered by an
assistor.
Percentage of scheduled calls answered: This is the number of calls
answered as a percentage of the number of calls expected to be
answered by the call sites, considering the level of staffing.
Level of access: This is the number of taxpayers who receive
telephone assistance as a percentage of the total number of taxpayers
seeking assistance.
Number of calls answered per full-time equivalent (FTE) employee:
This is the number of calls accepted by the Automatic Call
Distributor System, minus calls abandoned, divided by the number of
FTE employees assigned to answer taxpayers' calls.
--------------------
\6 These selected customer service program-level definitions are
included in IRS' fiscal year 1997 Annual Performance Plan.
IRS' PROGRAM-LEVEL CUSTOMER
SERVICE MEASURES
========================================================== Appendix II
Number of Calls Answered - Includes Automated
Number of Assistor Calls Answered
Percent of Scheduled Calls Answered
Taxpayer Service (TPS) Level of Access
TPS Calls Answered per Full-time Equivalent Employee (FTE)
TPS Tax Law Accuracy Rate
TPS Account Accuracy Rate
Notice Accuracy - Service Center only
Adjustments - Work Days of Inventory - Service Center only
Customer Service Accuracy Rate - Service Center only
Customer Service Productivity - Service Center only
Customer Service Timeliness - Service Center only
Service Center Collection Branch Dollars Collected
Assessments, Substitute for Return
Percent of Problem Resolution Program (PRP Cases) Identified -
Service
Center only
PRP Average Processing Time To Close Cases
PRP Quality Customer Service Rate
Service Center Examination Cycle Time
Service Center Examination Dollars Recommended
Service Center Examination Dollars Recommended per FTE
Service Center Examination Audit Coverage
Automated Collection System (ACS) Level of Service
ACS Average Cycles per Taxpayer Delinquent Account (TDA)/Taxpayer
Delinquency Investigation (TDI) Disposition
Total Dollars Collected as a Percent of Current Year ACS Receivables
ACS Dollars Collected
ACS Dollars Collected per FTE
ACS TDA/TDI Average Hours per Entity
Underreporter Assessments
Underreporter Refunds
Document Matching Dollars Recommended
(See figure in printed edition.)APPENDIX III
COMMENTS FROM THE INTERNAL REVENUE
SERVICE
========================================================== Appendix II
(See figure in printed edition.)
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix IV
GENERAL GOVERNMENT DIVISION,
WASHINGTON, D.C.
Robert L. Giusti, Senior Evaluator
Monika R. Gomez, Evaluator
ATLANTA FIELD OFFICE
A. Carl Harris, Assistant Director
Catherine H. Myrick, Evaluator-in-Charge
Katherine P. Chenault, Senior Evaluator
Ronald W. Jones, Evaluator
*** End of document. ***