IRS Telephone Assistance: Quality of Service Mixed in the 2000
Filing Season and Below IRS' Long-Term Goal (06-APR-01,
GAO-01-189).
The Internal Revenue Service (IRS) must significantly improve
telephone assistance if it is to meet its long-term goal of
providing world-class customer service to the tens of millions of
taxpayers who call. Although IRS has tried to analyze its
performance and identify ways to improve, these efforts have been
incomplete. IRS' analyses did not cover all of the key management
decisions and other key factors that affect telephone
performance. Designing and conducting a comprehensive analysis of
the key management decisions and other key factors that affect
telephone performance in each filing season will be difficult
because the factors that affect performance are multiple and
interrelated. However, without a more comprehensive analysis of
the factors that affect performance, IRS lacks the information it
needs to make decisions to improve performance.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-189
ACCNO: A00805
TITLE: IRS Telephone Assistance: Quality of Service Mixed in the
2000 Filing Season and Below IRS' Long-Term Goal
DATE: 04/06/2001
SUBJECT: Customer service
Electronic forms
Human resources utilization
Performance measures
Personnel management
Tax administration systems
Tax returns
Taxpayers
Telephone
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-189
GAO- 01- 189
Report to the Chairman, Subcommittee on Oversight, Committee on Ways and
Means, House of Representatives
United States General Accounting Office
GAO
April 2001 IRS TELEPHONE ASSISTANCE Quality of Service Mixed in the 2000
Filing Season and Below IRS? Long- Term Goal
Page i GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season Letter 1
Results in Brief 2 Background 4 Scope, and Methodology 5 Telephone
Assistance Results Were Mixed and Fell Short of WorldClass Service 6
Interrelated Factors Influenced IRS? Telephone Performance 8 IRS? Analysis
of Its Performance Was Incomplete 15 IRS? Performance Management System
Restricts the Use of
Productivity Data 17 Conclusion 18 Recommendation for Executive Action 18
Agency Comments and Our Evaluation 18
Appendix I Comments From the Internal Revenue Service 21
Table
Table 1: IRS Filing Season Telephone Assistance Performance 7 Contents
Page 1 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
April 6, 2001 The Honorable Amo Houghton Chairman, Subcommittee on Oversight
Committee on Ways and Means House of Representatives
Dear Mr. Chairman: IRS receives tens of millions of telephone calls each
year from taxpayers needing assistance in understanding and meeting their
tax obligations. Taxpayers call with a variety of issues, including
questions about filing returns, tax law, refunds, and notices IRS sends when
it detects errors on returns. Taxpayers often have been unable to reach IRS,
especially during the filing season when demand for assistance is greatest.
Moreover, when taxpayers reached IRS, they may have received inaccurate
information. IRS has pledged to take steps to make its telephone operation a
?worldclass
customer service organization? that would provide taxpayer assistance that
is comparable to the best practices in the private and public sectors.
Because of your interest in the quality of IRS? telephone service, you asked
us to address various aspects of IRS? telephone assistance program and other
telephone customer service organizations. This is our third report in
response to your request. Our first report, issued in August 2000, presented
information on the human capital management practices of selected public and
private call centers. 1 Our second report, issued in January 2001, evaluated
IRS? human capital management practices in identifying human capital needs,
implementing human capital practices to achieve IRS? goals, and evaluating
those practices to continuously improve. 2 This report discusses the
performance of IRS? telephone assistance in the 2000 filing season and how
IRS identified lessons learned to improve performance. More specifically,
our objectives were to (1) assess IRS? performance in the 2000 filing season
using three telephone program performance
1 Customer Service: Human Capital Management at Selected Public and Private
Call Centers (GAO/ GGD- 00- 161, Aug. 22, 2000). 2 IRS Telephone Assistance:
Opportunities to Improve Human Capital Management (GAO01- 144, Jan. 30,
2001).
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
measures: level of service, tax law accuracy, and account accuracy; 3 (2)
identify the key factors and describe how they affected performance; and (3)
assess IRS? process for analyzing its performance in the 1999 and 2000
filing seasons in order to make improvements. In the course of our audit
work, we learned that IRS restricted supervisors from using productivity
data to evaluate assistors? performance. Some IRS officials we talked to
were uncertain about the basis for the restriction, for example, some
thought that it was mandated by the Internal Revenue Service Restructuring
and Reform Act of 1998. 4 After discussion with your staff, we agreed to
determine IRS? basis for this restriction, as a fourth objective.
IRS telephone assistance showed mixed results in the 2000 filing season. IRS
answered 59 percent of calls to the toll- free taxpayer assistance lines in
2000, better than in the 50 percent achieved in 1999, but below the 69
percent level of service achieved in the 1998 filing season. For tax law and
account questions, IRS estimated it provided accurate answers 73 percent and
59 percent of the time, respectively. 5 Tax law accuracy was comparable to
1999 performance, but lower than the 2000 target of 80 percent. Account
accuracy was slightly lower than the 2000 target of 63 percent. Although IRS
had not defined world- class service in terms of specific measures and
goals, IRS officials acknowledged performance fell short of its long- term
goal of providing assistance comparable to that provided by leading public
and private telephone customer service organizations.
A number of interrelated factors influenced IRS? performance in providing
telephone assistance, such as the demand for assistance, number of staff IRS
devoted to providing assistance, assistors? productivity, and their skill
level. IRS did not have complete information on how these and other factors
affected level of service and accuracy in the 2000 filing season. However,
according to IRS officials,
3 Level of service is the rate at which taxpayers who called IRS actually
got through and received service. Tax law and account accuracy rates are the
percentages of a sample of calls in which telephone assistors provided
accurate answers and fully adhered to IRS guidance for assisting taxpayers
with those types of questions.
4 P. L. 105- 206, July 22, 1998. 5 The sample of tax law and account calls
IRS monitored allowed IRS to be 90 percent confident that the telephone
assistors? true accuracy rate was between 72 and 74 percent in responding to
tax law questions, and between 58 and 60 in responding to account questions.
Results in Brief
Page 3 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
the increase in level of service was due primarily to a 25- percent
decrease in demand, partly because IRS sent taxpayers fewer notices and
processed returns more quickly;
the level of service might have been higher had IRS not reduced staffing
of telephone assistance by 8 percent compared to 1999 in an effort to better
balance overall workload and staffing among customer service programs;
assistor productivity, in terms of the average length of a call, declined
about 17 percent, partly because assistors received a greater percentage of
the types of calls that took longer to answer; and
the level of productivity continued to be affected by policy changes IRS
made before the 1999 filing season, such as discontinuing the automatic
routing of another call to assistors immediately upon their completion of a
call.
In addition, according to IRS officials, gaps between the skills that
assistors had and those IRS needed negatively affected the accuracy of IRS?
responses to taxpayer inquiries. Also, flaws in the guidance assistors used
and frequent changes to it caused inaccurate responses to taxpayers? account
questions.
IRS? analysis of its telephone assistance performance in the 1999 and 2000
filing seasons was incomplete. Although IRS collected various data and
conducted several analyses of performance, the approach either did not
assess or assessed incompletely some of the key management decisions and
other factors that affected performance. As a consequence, IRS management
did not have some significant information that could have been used to make
decisions intended to improve future performance. For example, in its
studies of productivity, IRS did not study all the segments of assistors?
time that would affect overall productivity.
Because IRS? analysis of performance in recent filing seasons was
incomplete, we recommend that the IRS Commissioner ensure that IRS analyzes
all the key management decisions and other key factors affecting telephone
performance each filing season to determine their impact on the quality of
service and to make improvements. The Commissioner agreed with our
assessment of IRS? telephone performance during the 2000 filing season and
with our recommendation.
IRS? ?balanced measures? performance management system, and not the IRS
Restructuring and Reform Act of 1998, was the basis for restricting the
Page 4 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
use of productivity data to evaluate employee performance. 6 When designing
and implementing the balanced measures system, IRS management decided to
prohibit telephone assistance supervisors from using productivity data when
evaluating all assistors. The prohibition was intended to promote a more
balanced focus by assistors on efficiency, quality, and service.
IRS? telephone assistors are located at 25 call sites around the country. In
the 1999 filing season, IRS made major changes to its telephone customer
service program. For example, IRS extended its hours of service to 24 hours
a day, 7 days a week. IRS officials said they believed around- theclock
assistance would improve the level of service by distributing demand more
evenly and support IRS? efforts to provide world- class service by making
assistance available anytime.
Also in 1999, IRS began managing its telephone operations centrally at the
Customer Service Operations Center in Atlanta by using new call- routing
technology. IRS? call router was designed to improve the overall level of
service, as well as lessen disparities in the level of service across sites
by sending each call to the first available assistor nationwide who had the
necessary skills to answer the taxpayer?s question. As part of this
centralized management, IRS developed its first national call schedule that
projected the volume of calls, for each half- hour, at each of IRS? 25 call
sites, and the staff resources necessary to handle that volume.
As in previous years, in the 2000 filing season, IRS had three toll- free
telephone numbers taxpayers could call with questions about tax law,
taxpayer accounts, and refunds. The three primary measures IRS used to
evaluate its telephone performance were level of service, tax law accuracy,
and account accuracy.
IRS measures its level of service by determining the rate at which taxpayers
that call IRS actually get through and receive assistance. Level of service
is calculated by dividing the number of calls answered by the total call
attempts. Calls answered is defined as calls that received service,
6 IRS? balanced measure system is its approach to evaluating employee and
organizational performance; it is designed to balance customer satisfaction,
employee satisfaction, and business results. Background
Page 5 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
either from assistors or telephone interactive applications. 7 Total call
attempts includes repeat calls and is the sum of calls answered, calls
abandoned by the caller before receiving assistance, and calls that received
a busy signal. IRS? tax law accuracy and account accuracy rates are based on
a sample of nationwide calls that quality assurance staff listen in on and
score for accuracy. Using IRS? Centralized Quality Review System, staff in
Philadelphia listen to sample calls from beginning to end and determine
whether the assistors provide accurate answers, follow procedural guidance
to ensure a complete response, and are courteous to the taxpayers. If the
assistors fail to adhere to any part of the guidance, or are not courteous
to the taxpayers, the calls are counted as inaccurate. IRS began centrally
monitoring calls to measure tax law accuracy in fiscal year1999 and account
accuracy in fiscal year 2000.
To address our objectives, we examined documents and interviewed IRS
officials. Specifically:
to assess IRS? performance in the three main telephone assistance toll-
free numbers, we compared its 2000 filing season level of service, tax law
accuracy, and account accuracy with its performance in the 1998 and 1999
filing seasons and its performance targets, and discussed with IRS officials
how its performance compared with world- class customer service; 8
to identify the key factors and describe how they affected performance in
the 1999 and 2000 filing seasons we interviewed IRS officials, including
executives, division chiefs, and first- line supervisors in Customer Service
7 Interactive applications are designed to allow taxpayers with certain
questions to obtain the information or service they need without speaking to
an assistor, such as determining the status of their refunds.
8 According to IRS officials, the filing season generally begins January 1
and ends around July 15 each year. In previous years, GAO's review of
telephone performance generally covered the period January 1 through the
April 15 filing deadline. However, demand for telephone assistance continues
past the deadline because taxpayers call about such issues as the status of
refunds or notices IRS sends to taxpayers related to returns they filed.
Level of service rates are based on comparable data for weeks beginning
January 1 and ending around July 15. Tax law and accounts accuracy rates are
based on data for the months of January through June, except for tax law
accuracy in 1999, which was based on data for January through April due to
database limitations. According to IRS, account accuracy in the 2000 filing
season should not be compared to accuracy rates in previous years, nor
should tax law accuracy in the 1999 or 2000 filing season be compared to the
accuracy rate in 1998, because IRS changed the way accuracy was measured.
Instead of conducting test calls that measured limited adherence to
guidance, IRS monitored actual calls and more strictly measured assistors?
adherence to guidance. Scope, and
Methodology
Page 6 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
Field Operations and at call sites; and analyzed documents, including
various reports that described and analyzed the factors that affected IRS?
performance;
to assess IRS? process for analyzing its performance in the 1999 and 2000
filing seasons in order to make improvements, we interviewed IRS officials,
including National Office and Customer Service Field Operations officials
responsible for collecting and analyzing data on IRS performance; and
analyzed documents, including various reports related to the process, such
as the 1999 National Office business review and statistical analyses of 2000
filing season performance; and
to determine the basis for restricting supervisors from using productivity
data to evaluate or discuss telephone assistor performance, we interviewed
IRS officials, including officials in the Organizational Performance
Division and Customer Service Field Operations; and analyzed documents
related to the restriction, including the Internal Revenue Manual and
materials used to train supervisors on the use of statistics.
We performed our work at IRS? National Office in Washington, D. C.; Office
of the Chief, Customer Service Field Operations, and Customer Service
Operations Center in Atlanta; and the telephone assistance call sites in
Atlanta, Dallas, and Kansas City, KS. We chose these three sites in order to
include sites of various sizes, hours of operation, and work. We did not
independently assess the accuracy of IRS? performance data, however, we
verified that IRS had procedures in place intended to ensure data
reliability. We did our work from January 2000 through February 2001 in
accordance with generally accepted government auditing standards.
We obtained written comments on a draft of this report from the Commissioner
of Internal Revenue in a letter dated April 2, 2001. The comments are
discussed at the end of this report and reprinted in appendix I.
IRS telephone assistance showed mixed results in the 2000 filing season.
Performance improved somewhat in the 2000 filing season as compared with
1999, but according to IRS officials, fell short of IRS? long- term goal to
provide world- class customer service. While IRS had not established
specific measures and goals for world- class service, it was considering
adopting some of those used by leading telephone customer service
organizations. Telephone Assistance
Results Were Mixed and Fell Short of World- Class Service
Page 7 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
In the 2000 filing season, IRS answered 36.1 million of the 61 million calls
taxpayers made, resulting in a 59- percent level of service- better than the
50 percent IRS achieved in the 1999 filing season and its target of 58
percent, but short of the 69 percent IRS achieved in the 1998 filing season.
IRS provided accurate responses in 73 percent of the tax law calls it
answered- unchanged from 1999 and lower than its 2000 target of 80 percent.
Account accuracy in the 2000 filing season was slightly lower than IRS?
target of 63 percent. Table 1 shows IRS? performance during the 1998- 2000
filing seasons.
Table 1: IRS Filing Season Telephone Assistance Performance 1998 1999 2000
Measure Actual Actual Target Actual
Level of service 69% 50% 58% 59% Tax law accuracy a 73% b 80% 73% b Accounts
accuracy a a 63% 59% b a Comparable data does not exist.
b Actual values are estimates, with 90 percent confidence that the true
value is never more than plus or minus 2 percentage points. Source: IRS
data.
IRS officials in National Office and Customer Service Field Operations
recognized that telephone performance in the 2000 filing season fell short
of its long- term goal of providing world- class customer service--
assistance comparable to that provided by leading public and private
telephone customer service organizations. IRS has not defined world- class
service in terms of specific measures and goals. 9 However, IRS officials
have acknowledged the need to change their performance measures to be more
consistent with leading telephone customer service organizations. IRS? level
of service measures the percentage of call attempts that receive assistance,
with no consideration of how long callers wait for it. Some leading
organizations measure service level as the percentage of calls answered
within a specified period of time, such as answering 90 percent of calls
within 30 seconds. 10 IRS was considering adopting a similar measure and
goal. However, IRS? performance in fiscal year 2000 fell
9 In responding to our earlier report (GAO- 01- 144, January 30, 2001), the
Commissioner said that IRS? short- term level- of- service goal for fiscal
year 2002 would be 74 percent, with a goal of reaching 85 to 90 percent by
fiscal year 2003.
10 GAO/ GGD- 00- 161, August 22, 2000.
Page 8 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
substantially short of this level, with only 31 percent of calls being
answered within 30 seconds.
A number of interrelated factors influenced IRS? telephone assistance
performance in the 2000 filing season. According to IRS, some of the key
factors were the demand for assistance, staffing levels, assistor
productivity, assistor skills, and IRS? guidance for assistors.
Additionally, many of the factors were interrelated- changes in one factor
could cause changes in others.
According to an analysis by Customer Service Field Operations officials, IRS
was able to answer a greater percentage of calls in the 2000 filing season
compared with 1999 because demand for service substantially decreased. IRS
measured demand in two ways: total call attempts and unique telephone number
attempts. Total call attempts includes repeat calls and is the sum of calls
answered, calls abandoned by the caller before receiving assistance, and
calls that received a busy signal. The unique telephone number measure is
designed to count the number of taxpayers who called, rather than the number
of calls. It measures the number of calls from identifiable telephone
numbers, and counts all call attempts from each telephone number as one call
until it reaches IRS and is served, or until a 1- week window expires. Total
call attempts decreased from 83.5 million in 1999 to 62.8 million, a 25-
percent decrease, while unique number attempts decreased from 33.2 million
to 25.9 million, a 22percent decrease. 11 According to IRS, demand declined
partly because IRS issued 1.8 million fewer notices to taxpayers asking them
to call IRS about such issues as math errors IRS detected while processing
returns. Also, fewer taxpayers called about the status of their refunds
because IRS processed returns more quickly.
Additionally, timing of notices IRS sends taxpayers influences demand for
assistance. For example, as we previously reported, in the 2000 filing
season, because of contract delays, a contractor mailed the bulk of over 1
million notices to taxpayers over a 2- week period, rather than over a 7-
11 Total call attempts are based on data from October 1, 1999, through April
22, 2000, for all of IRS? toll- free lines, which include, among others, IRS
telephone lines for collecting taxes and taking criminal investigation
calls. Unique call attempts are based on data from October 1, 1999, through
April 8, 2000, for the three assistance toll- free lines. Interrelated
Factors
Influenced IRS? Telephone Performance
Demand
Page 9 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
week period as intended. 12 When taxpayers called about the notices, IRS was
unprepared to answer the unexpected increase in the number of telephone
calls, which caused level of service to decline during this period.
According to IRS officials, a factor that may have prevented the level of
service from being higher in the 2000 filing season was IRS? decision to
reduce the staff dedicated to telephone assistance as compared with 1999.
Specifically, in the 2000 filing season, IRS dedicated 4,912 staff years to
telephone assistance as compared to 5,339 staff years in 1999, an 8- percent
decline. 13 According to IRS officials, IRS dedicated fewer resources to
telephone assistance to increase staffing in other programs, including the
telephone collection system, adjustments, and service center compliance. IRS
managers were concerned that in 1999, when IRS redirected resources from
these other programs to telephone assistance, the backlog in these programs
increased to unacceptable levels, causing uneven service and a decline in
collection revenues.
Assistor productivity is another factor that affects the level of service
taxpayers receive from IRS. According to IRS officials, the level of service
would have been higher had assistor productivity not declined in the 2000
filing season. This decline was in addition to a productivity decline that
occurred in the 1999 filing season.
According to analysts and officials in Customer Service Field Operations, a
key indicator of productivity is the average time for an assistor to handle
a call. Handle time is the total of the time
an assistor spends talking to the taxpayer,
the taxpayer is on hold, and 12 Tax Administration: IRS' 2000 Tax Filing
Season and 2001 Budget Request (GAO/ TGGD/ AIMD- 00- 133, Mar. 28, 2000). 13
A staff year is equivalent to one full- time employee, also known as a full-
time equivalent. It includes direct and overhead time (both regular and
overtime) expended by permanent and seasonal Customer Service and Compliance
employees related to answering telephone calls on the three assistance
lines, Criminal Investigation, and Problem Resolution Program toll- free
telephone lines. In October 2000, IRS had 9,865 customer service
representatives, with about 40 percent of them being seasonal employees who
worked less than 12 months per year. Staffing Level
Productivity
Page 10 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
the assistor spends in ?wrap status?, which is the time between hanging up
at the end of a call and indicating readiness to receive another call.
An IRS analysis showed that the average handle time increased from 318.5
seconds in the 1999 filing season to 371.5 seconds in the 2000 filing
season, or about a 17- percent decline in productivity.
According to a Treasury Inspector General for Tax Administration report, an
increase in the number of calls an assistor handles has a profound effect on
level of service. For example, if assistors had handled one more call per
hour, IRS would have answered more than 8.5 million additional calls during
the first 6 months of fiscal year 1999. 14
While IRS had not determined all the causes of the decline in productivity
since 1998, according to a July 2000 IRS study, approximately 58 percent of
the productivity decline from 1999 to 2000 was due to assistors? receiving a
greater percentage of calls that took longer to handle. For example,
screening calls, in which the assistor talked with the taxpayer for only a
short time to determine the taxpayer?s question and where the call should be
routed, decreased from 35 percent of the calls assistors handled in 1999 to
21 percent in 2000. The study concluded that assistors likely handled fewer
of these calls because IRS changed its telephone message to discourage
callers from posing as rotary dialers without a touch- tone telephone,
allowing them to bypass the menu system and go directly to an assistor. This
study did not identify what caused the remaining 42- percent decline in
productivity in 2000.
According to IRS officials, four policy changes that lowered productivity in
the 1999 filing season continued to adversely affect productivity in the
2000 filing season. Specifically, in 1999, IRS
discontinued automatically routing another call to an assistor immediately
upon completion of a call;
increased restrictions on using productivity data when evaluating
assistors? performance;
14 Treasury Inspector General for Tax Administration, Toll- Free Telephone
Service Levels Declined in 1999, Despite Costly Efforts to Achieve World-
Class Performance (Ref. No. 2000- 30- 062, March 2000), p. 29.
Page 11 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
disproportionately diverted staff from the peak demand shifts to shifts
when fewer taxpayers call when it implemented its 24- hour- a- day, 7- day-
aweek assistance; and
discontinued measuring productivity of individual call sites. First, as
part of its November 1998 agreement with the National Treasury Employees
Union, IRS discontinued using a call management tool-? autoavailable?- that
automatically routed another telephone call to an assistor as soon as a call
was completed. Instead, assistors were placed in ?wrap
status? after each call and were unavailable until they pressed a keyboard
button that made them available. Wrap status was designed to allow assistors
time to document the results of a call or to allow them to take a momentary
break after a stressful call. According to IRS officials, allowing assistors
to determine when they were ready to take another call added time to each
call, causing other callers to wait longer for service. With longer wait
times, many taxpayers hung up before reaching an assistor, thereby reducing
level of service. According to IRS statistics, for its tax law, account, and
refund assistance lines, the average wrap times increased 94, 204, and 176
percent, respectively, from 1998 to 1999.
Second, 1999 was the first filing season with increased restrictions on
supervisors using productivity data to evaluate assistors? performance or
discuss their performance. Some IRS studies of the 1999 filing season
concluded that the restrictions negatively affected productivity. For
example, one IRS study found that many site managers were concerned about
their inability to properly manage assistors? use of wrap time without using
productivity data. Five of the seven supervisors we spoke to about the 2000
filing season said they were dissatisfied with the restrictions. They said
assistors know supervisors are restricted from using productivity data to
evaluate employees? performance and that supervisors do not have adequate
time to devote to monitoring and physical observation. Therefore, they said
assistors are free to spend more time than necessary in wrap status.
Our conversations with IRS officials, including supervisors at call sites
and officials in the Organizational Performance Division, and review of
related documents indicated officials were uncertain about the basis for the
restriction, and some thought that it was mandated by the IRS Restructuring
and Reform Act. We discuss this issue near the end of this report.
Third, increasing the hours of telephone assistance to 24 hours a day, 7
days a week for the 1999 filing season may have decreased overall
Page 12 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
productivity because IRS disproportionately shifted staffing away from the
hours when most taxpayers call. According to an IRS review, the diversion of
staff away from hours when most taxpayers called resulted in a lower level
of service because taxpayers waited longer for assistance, more taxpayers
hung up while waiting, and demand increased because taxpayers redialed more.
Limited data from a week in the 2000 filing season indicated that IRS
continued to overstaff the night shift when compared to the other shifts.
For example, for the week of April 2, 2000, through April 8, 2000, assistors
working the night shift spent, on average, 44 percent of their time waiting
to receive a call, whereas assistors working the day and evening shift spent
15 percent of their time waiting to receive a call.
An IRS Customer Service Field Operations official responsible for scheduling
staff said assistors spent more time waiting for calls at night because,
when compared with the demand for assistance, IRS scheduled
disproportionately more assistors during the night shift than other shifts.
Assistors working nights generally had fewer skills, which required a
disproportionate level of staffing to ensure that all needed skills were
available. According to the official, IRS? attempts to attract more skilled
assistors to work off- peak hours were unsuccessful. To counter the negative
effects of staffing the extended hours, for fiscal year 2000, IRS limited
its staffing of tax law assistance to 16 hours a day, 6 days a week after
the filing deadline, when fewer taxpayers call with tax law questions.
Fourth, beginning in 1999, IRS no longer had a performance measure that held
sites accountable for productivity. Instead of measuring level of service as
it had in the past, IRS measured a site?s performance on the number of
assistors assigned to answer telephone calls each half- hour as compared to
the number of assistors specified in the site?s half- hour work schedule.
IRS made this change, in part, because the sites were no longer responsible
for predicting and meeting demand. According to an IRS assessment of the
1999 filing season, replacing the site level of service measure with the
measure of assistor presence diminished the focus on productivity and the
extent to which sites sought opportunities to improve productivity.
IRS Customer Service Field Operations officials added that, despite the
decline in productivity, taxpayers might have received better service
overall if assistors took the time needed to fully resolve each taxpayer?s
call, rather than being concerned about the number of calls answered.
However, IRS had not determined if the decline in productivity had improved
the quality of service.
Page 13 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
According to IRS officials, including the Commissioner, Customer Service
Field Operations officials, and supervisors at call sites, the accuracy
rates IRS achieved in the 2000 filing season continued to be adversely
affected by assistor skill gaps- the difference between the skills assistors
had and the skills needed by IRS. Skill gaps were caused, in part, when IRS
implemented its new call router in 1999.
With the call router, individual assistors were required to answer calls on
a broader range of topics, often without adequate training or experience.
Before the 1999 filing season, each call site decided how it would group
topics for routing and assistor specialization. According to a cognizant
official, the number of topic groups at sites ranged from 40 to 125, which
allowed assistors to typically specialize in only one or two topics. Because
the new call router could not handle differences in topic groups among call
sites, nor efficiently route calls to that many groups, the topic groups had
to be standardized and were reduced to 31. This increased the number of
topics in each group, which typically required an assistor to answer calls
on five or more tax law topics, creating a skill gap. IRS officials
recognized that assistors had struggled with the amount of information they
were required to know in 1999, so for the 2000 filing season IRS increased
the number of topic groups to 46, which decreased the number of topics in
each group. However, according to IRS officials, the loss of specialization
continued to affect accuracy in the 2000 filing season.
IRS officials said they were aware of how skill gaps had negatively affected
the accuracy of the assistance taxpayers received in 1999 and, in August
1999, IRS began to revise its training materials to better prepare assistors
to answer questions in their assigned topic groups. However, according to
IRS officials, much of the new training material was not developed in time
for the 2000 filing season. Furthermore, a cognizant IRS official said the
first attempt to revise the training did not separate each topic into a
self- contained course. For the 2001 filing season, IRS revised its training
material so that each course contained only one topic, enabling IRS to
provide assistors with just- in- time training on the specific topics they
were assigned to work.
IRS officials said organizational changes are needed to further reduce the
number of topics assistors are expected to know. In a May 2000 memo, the
Commissioner cited low accuracy scores and employee survey comments as
evidence that IRS was expecting its assistors and managers to have knowledge
in areas that are far too broad and that IRS was ?attempting the impossible?
by trying to fill skill gaps solely with training. IRS officials said IRS?
reorganization would allow specialization by taxpayer group, but that
Assistors? Skills
Page 14 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
even greater levels of specialization were needed. Accordingly, as part of
its restructuring efforts, in June 2000, IRS began long- term planning
efforts to create greater specialization at both the call site and assistor
levels.
The quality of the guidance assistors used also affected whether they
provided accurate assistance. IRS officials at National Office and call
sites said the guidance assistors used in the 2000 filing season to respond
to account questions was confusing and difficult to use, causing assistors
to make mistakes, thereby lowering the accuracy rate.
IRS officials said that over the years, the Internal Revenue Manual- the
assistors? guide for account questions- had grown from a collection of
handbooks to a large, unwieldy document with duplicative and erroneous
information. According to IRS officials, errors in the Manual had long been
a problem for which sites had developed local ?workaround? procedures. IRS
established a task force to correct these problems, and issued a new draft
version at the end of the 1999 filing season. While the draft Manual was
smaller and contained less duplicative and erroneous information, it was
missing some needed information and cross- references. However, IRS did not
realize the extent of the problems with the Manual until October 1999, when
it began holding assistors accountable for strictly adhering to the Manual
as part of its central monitoring of account accuracy. As a result, the
draft was recalled, and the task force continued to make corrections to the
Manual throughout the filing season. The task force issued two new versions
in February 2000 and May 2000. According to IRS officials, the frequent
changes in the Manual made it difficult for assistors to know which version
to use, sometimes leading to inaccurate answers.
According to IRS officials responsible for Manual revision, as October 1,
2000, the task force had corrected problems with the Manual and related
training material in time for the 2001 filing season. Additionally, IRS
officials said they implemented a new guide in October 2000 to make it
easier for assistors to follow the proper steps and provide accurate
assistance to taxpayers with account questions.
Determining how each factor affects level of service and accuracy is made
even more difficult because many of the factors are interrelated; changes in
one can affect another. For example, the demand for assistance, or the
number of call attempts, is influenced by the level of productivity. Fewer
incoming calls make it easier for a given number of assistors to answer a
greater percentage of incoming calls. Answering a greater percentage of
Assistors? Guidance
Interrelationships Make Identifying Key Factors Difficult
Page 15 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
incoming calls- a higher productivity level- reduces the number of repeat
calls, which reduces the number of calls overall. Similarly, the quality of
guidance assistors use affects not only accuracy, but also demand. While
step- by- step guidance on how to respond to questions would likely improve
accuracy levels and service for some taxpayers, it could also cause
assistors to take more time answering the call, lower productivity, and
increase the number of taxpayers who are unable to get through, causing them
to redial, and thereby increase demand.
IRS? analysis of its telephone assistance performance in the 1999 and 2000
filing seasons was incomplete. Although IRS collected various data and
conducted several analyses of performance, the approach either did not
assess or assessed incompletely some of the key management decisions and
other factors that affected performance. As a consequence, IRS management
had less information than it could have on which to make decisions intended
to improve future performance.
IRS undertook many efforts in 1999 and 2000 intended to identify factors
that affected performance. For example, IRS
conducted a best practices productivity study in 1999 to identify best
practices among IRS call sites and why productivity varied among them;
reviewed its implementation of 24- hour- a- day, 7- day- a- week
assistance to determine its effects on such things as costs and quality of
assistance;
conducted local and centralized monitoring of telephone calls to determine
what errors assistors made and why;
conducted a study in 2000 to determine why productivity had declined;
established a filing season critique program in 2000 to solicit
information from field staff about their problems and successes during the
filing season; and
conducted a 1999 fiscal year business review that addressed many of the
factors that affected telephone performance.
In some of its efforts, IRS began analyzing the data made available through
management information systems at its Customer Service Operations Center,
which opened in December 1998. For example, as a part of the 2000
productivity study noted above, IRS used statistical analysis to assess how
productivity was affected by such factors as the complexity of calls handled
and assistor experience and education. In a similar analysis, IRS assessed
how call demand was affected by such factors as returns filed, notices
issued, refunds issued, refund cycle times, and electronic filing IRS?
Analysis of Its
Performance Was Incomplete
Page 16 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
return rates. Although IRS now has better quantitative data to assess its
performance and make decisions about ways to improve performance, IRS
officials said much work still needs to be done to understand the factors
that affect performance.
Other leading telephone customer service organizations we studied see the
importance of continuous evaluation and incorporating evaluation results to
make improvements. 15 As we said in a recent report on management reform,
?an organization cannot improve performance and customer satisfaction if it
does not know what it does that causes current levels of performance and
customer satisfaction.? 16
IRS? efforts to evaluate the factors affecting telephone assistance were
incomplete and failed to provide IRS management with some significant
information that could have been used to improve performance. For example,
while IRS did several studies of productivity, the studies relied on handle
time as the measure of productivity. Other segments of assistors? time that
would affect overall productivity, including time spent waiting to receive a
call, time spent away from the telephone (in meetings, breaks, and
training), and time assistors were not assigned to answer calls, were not
studied. In another example, the most extensive single review of the factors
that affected performance- the 1999 National Office business review- did not
assess how extending the hours of service to 24 hours, 7 days a week
affected level of service. Earlier, we described how IRS? disproportionate
move of assistors to the night shift created differentials between shifts in
the time spent waiting for a call. Furthermore, while the National Office
review examined the effects of demand on service, it did not examine why
demand increased in 1999. Also, IRS did not evaluate the effectiveness of
its management decision not to automatically route calls to assistors as
soon as they completed a call, or the several other policy changes noted
above, even though they were intended to significantly improve overall
performance.
The gaps in IRS? information about the factors affecting past performance
impaired IRS? efforts to improve performance. An important example is the
decline in productivity, as measured by handle time. As discussed earlier,
some IRS officials believe that taxpayers may have received better
15 GAO/ GGD- 00- 161, August 22, 2000. 16 Management Reform: Using the
Results Act in Quality Management to Improve Federal Performance (GGD/ T-
99- 151, July 29, 1999).
Page 17 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
service overall if assistors took the time needed to fully resolve
taxpayers? calls. However, IRS had not determined whether overall service
improved as a result of increased handle time. Also discussed earlier was
the quality of guidance provided assistors. IRS did not realize until
October 1999 the extent of problems in the Internal Revenue Manual, too late
for fixes to be made for the 2000 filing season and sometimes leading to
inaccurate answers for taxpayers.
IRS? ?balanced measures? performance management system and not the IRS
Restructuring and Reform Act of 1998 was the basis for IRS restricting the
use of productivity data to evaluate employee performance. 17 The Act, and
subsequent regulation, prohibited supervisors from using records of tax
enforcement results, or other quantity measures, to impose production quotas
on or evaluate employees that make judgments about the enforcement of tax
laws. 18 When designing and implementing the balanced measures system, IRS
management decided to prohibit telephone assistance supervisors from using
productivity data when evaluating all assistors, even those that do not make
tax enforcement judgments. The prohibition was intended to promote a more
balanced focus by assistors on efficiency, quality, and service.
According to Organizational Performance Division officials, the balanced
measures system does not prohibit supervisors from using productivity data
to monitor employee performance. However, it requires supervisors to ?get
behind the numbers? and base discussions and evaluations of employee
performance solely on the direct review of employees? work. Officials said
IRS? design of the balanced measures system was heavily influenced by IRS?
environment in 1997 and 1998, during which IRS was under intense pressure
from Congress, the administration, and stakeholders to improve service to
taxpayers. The National Performance Review Customer Service Task Force and
National Commission on Restructuring the IRS had found that IRS? overall
environment and performance measurement focused on productivity to the
detriment of
17 IRS? balanced measure system is its approach to evaluating employee and
organizational performance; it is designed to balance customer satisfaction,
employee satisfaction, and business results.
18 Records of tax enforcement results include such things as number of
dollars collected or number of seizures conducted, while quantity measures
are outcome- neutral and include such things as number of cases closed, time
spent assisting taxpayers, or time spent per case. IRS? Performance
Management System Restricts the Use of Productivity Data
Page 18 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
service to taxpayers, making employees strive to meet short- term
performance and efficiency goals rather than have a balanced focus on
efficiency, quality, and taxpayer service. IRS officials said the
overemphasis on level of service and other productivity measures had
resulted in employees perceiving that productivity was more important than
quality, so assistors hurried through telephone calls and served taxpayers
poorly, rather than taking the time necessary to give the taxpayer full,
quality service. Also, officials said supervisors tended to consider
measures as ends in themselves, rather than determining the causes behind
employee performance and taking action to improve performance.
IRS must significantly improve telephone assistance if it is to meet its
longterm goal of providing world- class customer service to the tens of
millions of taxpayers that call. While IRS has undertaken efforts to analyze
its performance and identify ways to improve, these efforts have been
incomplete. IRS? analyses did not cover all of the key management decisions
and other key factors that affect telephone performance. Designing and
conducting a comprehensive analysis of the key management decisions and
other key factors that affect telephone performance in each filing season
will be a difficult task because the factors that affect performance are
multiple and interrelated. However, without a more comprehensive analysis of
the factors that affect performance, IRS management lacks the information it
needs to make decisions to improve performance.
We recommend that the IRS Commissioner ensure, as part of its analysis of
telephone assistance performance each filing season, that IRS take into
account all key management decisions and other key factors that can affect
performance, such as implementing 24- hour, 7- day assistance and the
decline in assistor productivity, to determine their impact on the quality
of service and to make improvements.
The Commissioner of Internal Revenue provided written comments on a draft of
this report in an April 2, 2001, letter, which is reprinted in appendix I.
The Commissioner agreed with our assessment of IRS? telephone performance
during the 2000 filing season and with our recommendation. The Commissioner
stated that the assessment of key management decisions and direction should
be fully integrated into both the planning process and performance review.
He recognized that IRS Conclusion
Recommendation for Executive Action
Agency Comments and Our Evaluation
Page 19 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
needed to improve its performance analysis to take into account all key
management decisions and other factors that can affect performance. He
stated that this would be done as a part of IRS? annual filing season
evaluation.
The Commissioner again expressed concern with our comparison of IRS'
performance in the 2000 filing season with its performance in the 1998
filing season, commenting that ?comparisons to 1998 are not valid due to the
changes made to accommodate our technological advance to a national
networked system.? As stated in our evaluation of the Commissioner?s
comments on our earlier report, 19 we believe it is appropriate to compare
IRS? performance before and after such operational changes. The changes made
after 1998 were intended to improve IRS? telephone service. The only way to
tell if service improved is to compare performance levels before and after
the changes.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from
the date of this letter. At that time, we will send copies to Representative
William J. Coyne, Ranking Minority Member of the Subcommittee;
Representative William Thomas, Chairman, and Representative Charles B.
Rangel, Ranking Minority Member, Committee on Ways and Means; the Honorable
Paul H. O?Neill, Secretary of the Treasury; the Honorable Charles O.
Rossotti, Commissioner of Internal Revenue; and the Honorable Mitchell E.
Daniels, Jr., Director, Office of Management and Budget. We will also make
copies available to others upon request.
19 Tax Administration: Assessment of IRS' 2000 Tax Filing Season (GAO- 01-
158, Dec. 22, 2000).
Page 20 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
If you have any questions or would like additional information, please call
James R. White at (202) 512- 9110 or Carl Harris at (404) 679- 1900. Key
contributors to this report are Ronald W. Jones, Julie Schneiberg, and Sally
Gilley.
Sincerely yours, James R. White Director, Tax Issues
Appendix I: Comments From the Internal Revenue Service
Page 21 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
Appendix I: Comments From the Internal Revenue Service
Appendix I: Comments From the Internal Revenue Service
Page 22 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season
Appendix I: Comments From the Internal Revenue Service
Page 23 GAO- 01- 189 Quality of Service Mixed in 2000 Filing Season (268906)
The first copy of each GAO report is free. Additional copies of reports are
$2 each. A check or money order should be made out to the Superintendent of
Documents. VISA and MasterCard credit cards are also accepted.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail:
U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013
Orders by visiting:
Room 1100 700 4 th St., NW (corner of 4 th and G Sts. NW) Washington, DC
20013
Orders by phone:
(202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet
For information on how to access GAO reports on the Internet, send an email
message with ?info? in the body to:
Info@ www. gao. gov or visit GAO?s World Wide Web home page at: http:// www.
gao. gov
Contact one:
Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
E- mail: fraudnet@ gao. gov
1- 800- 424- 5454 (automated answering system)
(268906)
Ordering Information To Report Fraud, Waste, and Abuse in Federal Programs
*** End of document. ***