Tax Systems Modernization: Management and Technical Weaknesses Must Be
Corrected If Modernization Is to Succeed (Chapter Report, 07/26/95,
GAO/AIMD-95-156).
Since 1986, the Internal Revenue Service (IRS) has spent $2.5 billion on
Tax Systems Modernization and expects to spent more than $8 billion on
the project through 2002. By any measure, this is a world-class
information systems development effort, much larger than most other
organizations will ever undertake. Tax Systems Modernization is the
centerpiece of IRS' vision of virtually paperless tax processing to
optimize operations and better serve taxpayers. This report critiques
the effectiveness of this effort. GAO discusses IRS' progress in
implementing its modernization and describes serious management and
technical weaknesses that must be overcome if tax systems modernization
is to success. GAO makes more than a dozen specific recommendation for
improving IRS' business management and information systems management
and development capabilities.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-95-156
TITLE: Tax Systems Modernization: Management and Technical
Weaknesses Must Be Corrected If Modernization Is to
Succeed
DATE: 07/26/95
SUBJECT: Tax administration
Electronic forms
Application software
Systems conversions
Accountability
Information resources management
Tax returns
Computer software verification and validation
IDENTIFIER: IRS Tax System Modernization Program
TSM
IRS Document Processing System
IRS Business Master Plan
IRS Corporate Accounts Processing System
IRS TeleFile Program
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to the Commissioner of the Internal Revenue Service
July 1995
TAX SYSTEMS MODERNIZATION -
MANAGEMENT AND TECHNICAL
WEAKNESSES MUST BE CORRECTED IF
MODERNIZATION IS TO SUCCEED
GAO/AIMD-95-156
Tax Systems Modernization
Abbreviations
=============================================================== ABBREV
CMM - Capability Maturity Model
GAO - General Accounting Office
IRS - Internal Revenue Service
TSM - Tax Systems Modernization
Letter
=============================================================== LETTER
B-260070
July 26, 1995
The Honorable Margaret Milner Richardson
Commissioner
Internal Revenue Service
Dear Ms. Richardson:
This report critiques the effectiveness of the Internal Revenue
Service's (IRS) efforts to modernize tax processing. The report,
which reflects our briefing to you in April 1995, discusses IRS's
progress to implement its modernization and describes serious
remaining management and technical weaknesses that must be corrected
if tax systems modernization is to succeed. It makes over a dozen
specific recommendations to you for improving IRS's business
management and information systems management and development
capabilities.
The head of a federal agency is required by 31 U.S.C. 720 to submit
a written statement on actions taken on these recommendations to the
Senate Committee on Governmental Affairs and the House Committee on
Government Reform and Oversight not later than 60 days after the date
of this report. A written statement must also be sent to the House
and Senate Committee on Appropriations with the agency's first
request for appropriations made more than 60 days after the date of
this report.
We are sending copies of this report to the Chairmen and the Ranking
Minority Members of (1) the Subcommittee on Treasury, Postal Service
and General Government, Senate and House Appropriations Committee,
(2) the Senate and House Committee on the Budget, (3) the
Subcommittee on Oversight, House Committee on Ways and Means, (4) the
Subcommittee on Taxation and IRS Oversight, Senate Committee on
Finance, (5) the Senate Committee on Governmental Affairs, and (6)
the House Committee on Government Reform and Oversight. We are also
sending copies to the Secretary of the Treasury and the Director of
the Office of Management and Budget, and will make copies available
to others upon request.
This work was performed under the direction of Dr. Rona B.
Stillman, Chief Scientist for Computers and Communications, who can
be reached at (202) 512-6412.
Sincerely yours,
Gene L. Dodaro
Assistant Comptroller General
EXECUTIVE SUMMARY
============================================================ Chapter 0
Since 1986, the Internal Revenue Service (IRS) has invested $2.5
billion in Tax Systems Modernization (TSM). In addition, it
requested another $1.1 billion for fiscal year 1996 for this effort
and, through 2001, expects to spend over $8 billion on TSM. By any
measure, this is a world-class information systems development
effort, much larger than most other organizations will ever
undertake. TSM is the centerpiece of IRS's vision of virtually
paperless tax processing to optimize operations and serve taxpayers
better.
Over the past decade, GAO has issued several reports and testified
before congressional committees on IRS's costs and difficulties in
modernizing its information systems. As a critical information
systems project that is vulnerable to schedule delays, cost
over-runs, and failure to meet mission goals, in February 1995, TSM
was added to GAO's list of high-risk areas.
PURPOSE
---------------------------------------------------------- Chapter 0:1
To identify needed improvements in IRS's TSM effort, GAO examined
business and technical practices that IRS has established to develop,
manage, and operate its information systems. These practices involve
IRS's business strategy to reduce reliance on paper, strategic
information management practices, software development capabilities,
technical infrastructures, and organizational controls. Assurance
that IRS has sound practices in these areas increases the likelihood
that TSM's objectives will be met cost-effectively and expeditiously.
BACKGROUND
---------------------------------------------------------- Chapter 0:2
One of IRS's most pressing problems is efficiently and effectively
processing the over 200 million tax returns it receives annually;
handling about 1 billion information documents, such as W2s and
1099s; and, when needed, retrieving tax returns from the over 1.2
billion tax returns in storage. IRS's labor-intensive tax return
processing, which uses concepts instituted in the late 1950s,
intensifies the need to meet this enormous information processing
demand by reengineering processes and using modern technology
effectively. TSM is key to IRS's vision of a virtually paper-free
work environment where taxpayer account updates are rapid and
taxpayer information is readily available to IRS employees to respond
to taxpayer inquiries.
In May 1994, GAO prepared a guide to the best practices used by
successful private and public sector organizations to improve mission
performance through strategic information management and technology.
Additionally, the Software Engineering Institute at Carnegie Mellon
University has developed a model, the Capability Maturity Model
(CMM), to evaluate an organization's software development capability.
Also, IRS has adopted a widely used systems development methodology,
known as Information Engineering, as a primary basis for developing
TSM systems. GAO's strategic information management best practices,
CMM's key software development process criteria, and the Information
Engineering methodology were collectively used to assess IRS's tax
system processing modernization efforts.
RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3
IRS recognizes the criticality to future efficient and effective
operations of attaining its vision of modernized tax processing, and
has worked for almost a decade, with substantial investment, to reach
this goal. In doing so, IRS has progressed in many actions that were
initiated to improve management of information systems; enhance its
software development capability; and better define, perform, and
manage TSM's technical activities.
Nevertheless, the government's investment of what could be more than
$8 billion and IRS's efforts to modernize tax processing are at
serious risk due to remaining pervasive management and technical
weaknesses that impede modernization efforts. In this regard, IRS
does not have a comprehensive business strategy to cost-effectively
reduce paper submissions, and it has not yet fully developed and put
in place the requisite management, software development, and
technical infrastructures necessary to successfully implement an
ambitious world-class modernization effort like TSM. Many management
and technical issues are unresolved, and promptly addressing them is
crucial to mitigate risks and better position IRS to achieve a
successful information systems modernization.
First, IRS's business strategy will not maximize electronic filings
because it primarily targets taxpayers who use a third party to
prepare and/or transmit simple returns, are willing to pay a fee to
file their returns electronically, and are expecting refunds.
Focusing on this limited taxpaying population overlooks most
taxpayers, including those who prepare their own tax returns using
personal computers, have more complicated returns, owe tax balances,
and/or are not willing to pay a fee to a third party to file a return
electronically. Without a strategy that also targets these
taxpayers, IRS will not meet its electronic filing goals or realize
its paperless tax processing vision. In addition, if, in the future,
taxpayers file more paper returns than IRS expects, added stress will
be placed on IRS's paper-based systems.
Next, IRS does not have the full range of management and technical
foundations in place to realize TSM objectives. To its credit, IRS
has (1) developed several types of plans to carry out its current and
future operations, (2) drafted criteria to review TSM projects, (3)
assessed its software development capability and initiated projects
to improve its ability to effectively develop software, and (4)
started to develop an integrated systems architecture\1 and made
progress in defining its security requirements and identifying
current systems' data weaknesses. However, despite activities such
as these, pervasive weaknesses remain to be addressed:
IRS's strategic information management practices are not fully in
place to guide systems modernization. For example, (1)
strategic planning is neither complete nor consistent, (2)
information systems are not managed as investments, (3) cost and
benefit analyses are inadequate, and (4) reengineering efforts
are not tied to systems development projects.
IRS's software development capability is immature and is weak in
key process areas. For instance, (1) a disciplined process to
manage system requirements is not applied to TSM systems, (2) a
software tool for planning and tracking development projects is
inconsistently used, (3) software quality assurance functions
are not well-defined or consistently implemented, (4) systems
and acceptance testing are neither well-defined nor required,
and (5) software configuration management\2 is incomplete.
IRS's systems architectures (including its security architecture
and data architecture), integration planning, and system testing
and test planning are incomplete. For example, (1) effective
systems configuration management practices are not established,
(2) integration plans are not developed and systems testing is
uncoordinated, and (3) standard software interfaces are not
defined.
Finally, IRS had not established an effective organizational
structure to consistently manage and control systems modernization
organizationwide. The accountability and responsibility for IRS's
systems development was spread among IRS's Modernization Executive,
Chief Information Officer, and research and development division. To
help address this concern, in May 1995, the Modernization Executive
was named Associate Commissioner. The Associate Commissioner will
manage and control modernization efforts previously conducted by the
Modernization Executive and the Chief Information Officer. The
research and development division will not, however, report to the
Associate Commissioner.
--------------------
\1 A system architecture is an evolving description of an approach to
achieving a desired mission. It describes (1) all functional
activities to be performed to achieve the desired mission, (2) the
system elements needed to perform the functions, (3) the designation
of performance levels of those system elements, and (4) the
technologies, interfaces, and location of functions.
\2 Configuration management involves selecting project baseline items
(e.g., specifications), systematically controlling these items and
changes to them, and recording their status and changes.
PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4
BUSINESS STRATEGY WILL NOT
MAXIMIZE ELECTRONIC FILING
-------------------------------------------------------- Chapter 0:4.1
IRS will not achieve the full benefits that electronic filing can
provide because it does not have a comprehensive business strategy to
reach its electronic filing goals or its vision of virtually
paperless tax processing. IRS's goal is to have electronic filings
for 70 million individual returns and 10 million business returns by
2001. This goal of 80 million electronically filed returns
represents 35 percent of all returns. On the basis of the current
rate of electronic filings from individuals, IRS estimates that by
2001, only about 29 million individuals will file electronically. If
10 million business returns are filed electronically as projected, a
total of about 39 million filings will be electronic. This is only
about 17.4 percent of the 224 million tax returns anticipated in
2001, less than half of IRS's goal.
IRS's electronic filing strategy focuses primarily on promoting
faster refunds to those taxpayers who use third parties to prepare
and transmit their simple tax returns and are willing to pay to file
their returns electronically. IRS has no comprehensive business
strategy to encourage other taxpayers to file electronically,
including, for example, taxpayers who (1) are unwilling to pay
preparers and transmitters to file electronically, (2) prepare their
own returns, (3) owe IRS for balances due, and (4) file complex tax
returns. Also not targeted are taxpayers who use personal computers
to prepare their tax returns. These taxpayers prepare their returns
electronically, print the returns on paper, and mail the paper to
IRS. IRS then expends effort to convert information on the paper
return back to electronic form.
Further, failure to meet or exceed electronic filing goals could
seriously impair IRS's future ability to efficiently process paper
tax returns. For example, IRS is developing the Document Processing
System to electronically capture all data from paper returns in five
submission processing centers. IRS is proceeding on the assumption
that at least 61 million of the 224 million returns estimated for
2001 will be filed electronically, which will leave 163 million paper
returns. However, since only 39 million returns may be filed
electronically by 2001, these processing centers and the Document
Processing System could be required to handle 185 million paper
returns, or 22 million more than is currently planned.
Without a strategy to maximize electronic filings, IRS, for the
foreseeable future, will continue to be inundated with paper filings
that must be processed using labor-intensive processes and
inefficient systems. Unless IRS revamps its approach, multi-billions
of dollars will be spent and the Service will still fall far short of
its paperless tax processing goal.
STRATEGIC INFORMATION
MANAGEMENT PRACTICES ARE NOT
EFFECTIVELY USED
-------------------------------------------------------- Chapter 0:4.2
IRS does not have an effective process for selecting, prioritizing,
controlling, and evaluating the progress and performance of major
information systems investments. IRS has actions underway to improve
its strategic information management practices, but many shortcomings
underscore the urgency of bolstering its use of the best practices
private and public sector organizations use in developing ambitious
and successful systems modernization efforts.
For example, IRS has developed several types of plans, such as the
IRS Future Concept of Operations, to carry out its current and future
operations. However, while critical parts of this plan have been
completed, it does not yet cover essential areas, including those
related to national and regional offices, workload distribution
management, area distribution centers, and process flows. Further,
the planning documents are not linked to each other, and there is not
a strong tie between TSM plans and IRS budgets. The absence of
complete and consistent strategic planning makes it difficult for IRS
to identify and effectively focus on completing priority TSM
projects.
IRS has also developed draft criteria, including factors such as
cost, project size, and mission benefit, to review TSM projects, but
these factors are not fully defined. For example, there are no
criteria by which to quantitatively assess a project's contribution
to achieving the business mission, or to measure its technical risk
and compare that to its cost and expected mission benefits.
Consequently, the draft review criteria do not provide a basis for
controlling and evaluating TSM information systems as investments
throughout their life cycles.\3 With this discipline, IRS could
identify early, and thus avoid investing resources in, high-risk
projects that have little potential to provide significant mission
benefits.
In addition, IRS has a process reengineering conceptual model, has
identified six core business areas, and is reengineering 3 of the 11
business processes that support these areas. However, IRS started
business process reengineering efforts after many automated systems
design and development efforts were already underway. As a result,
IRS's business process analysis is not driving TSM development and,
thus, there is no assurance that TSM will achieve the desired
objectives or support the improved business processes.
Further, IRS has done a cost and benefits analysis for TSM but this
analysis is flawed because it is based on outdated data and it
attributes some benefits to TSM that are actually attributable to
other initiatives and projects. As a result, the cost and benefits
analysis is unreliable and leaves IRS and the Congress without an
effective tool to know what investments in TSM are worthwhile.
Finally, IRS is currently assessing its skill and competency base,
but it does not have a completed process to upgrade skills or to
provide the training necessary to operate and maintain sophisticated
systems, such as those comprising TSM. More important, however,
because IRS has not yet completely identified the skills it will need
in the future, it cannot determine its current skills gap or develop
requisite training.
--------------------
\3 Life cycle is a term used to refer to the phases of a system's
evolution from beginning to end (i.e., from perceived need for a
system extending through systems design, implementation, operations,
and maintenance).
SOFTWARE DEVELOPMENT PROCESS
IS WEAK
-------------------------------------------------------- Chapter 0:4.3
In August 1993, using CMM, IRS rated its software development
capability as immature, the lowest level. This level of
maturity--CMM level 1--is described as ad hoc and, at times, chaotic,
and indicates significant weaknesses in software development
capability. Since that date, IRS's software development capability
has not improved significantly. IRS's software development
activities remain inconsistent and poorly controlled, with no
detailed procedures for systems engineers to follow in developing
software.
As a result, IRS faces a much greater exposure to extensive rework,
schedule slippage, and cost overruns in developing software. To
effectively develop software in-house, IRS must raise its in-house
software development maturity level. To oversee its contractors
effectively, IRS contract managers must understand the practices used
to develop software at CMM level 2.
To address software development weaknesses in key process areas, IRS
initiated several process action teams, which have made varying
progress. For instance, these teams have (1) studied and flow
charted the process for requesting information services, (2) adopted
a peer review process to assess software quality that is being
applied to selected projects, (3) selected a software tool for
planning and tracking the progress of software development projects,
and (4) issued guidance on unit testing.
Nonetheless, IRS's software development capability remains weak in
key process areas and the teams' actions have not yet significantly
improved IRS's software development capability. For example,
configuration management is incomplete, which means important
documentation to record and report the status and changes to
systems specifications is not tightly controlled;
a requirements management process, which defines, validates, and
prioritizes requirements, such as performance requirements and
delivery dates, is applied to only existing IRS systems, to the
exclusion of TSM systems; and
detailed procedures have not been defined for performing software
quality assurance functions, such as ensuring compliance of
software products and processes with defined standards and
independently verifying product quality.
Unless IRS makes substantial improvements in areas such as these, it
is unlikely to build TSM timely or economically and TSM is unlikely
to perform as intended. For instance, IRS could enhance software
quality assurance by using software metrics, which are numerical
measures presumed to predict an aspect of software quality, such as
the numbers of defects at various stages of software development and
the costs to repair defects.
GAO found, however, that IRS has not adequately defined a suite of
metrics. IRS's use of metrics is limited to only one type of metric,
collectively called function points, which is used to measure a
project's size, such as lines of code. IRS, however, is not
consistently or effectively applying even this limited metric to
software development projects throughout their life cycles.
SYSTEMS ARCHITECTURES,
TESTING, AND INTEGRATION ARE
NOT ADEQUATELY ADDRESSED
-------------------------------------------------------- Chapter 0:4.4
IRS has not adequately managed TSM technically. For example, while
systems architectures are necessary to provide detailed guidance to
systems designers and developers, the TSM integrated system
architecture, or blueprint, is being completed as modernization
progresses, and it is not driving TSM projects that are already
underway. Also, IRS has made progress in defining its security
requirements, issued an information security policy, and defined
preliminary security applications program interfaces, but it has not
completed security architecture in key areas, such as a security
concept of operations, disaster recovery and contingency plan, and
communications security plan. Further, IRS has analyzed its current
systems to identify data weaknesses, but its data architecture is
based on these existing processes, rather than on the improved
business processes that IRS is now developing.
In other technical areas, IRS
Has established a Configuration Control Board to consistently
manage and control all system changes, but the Board has focused
only on monitoring individual project costs and schedules.
Moreover, IRS has not established a process to manage systems
changes, which is necessary, for instance, to make engineering
and trade-off decisions, maintain up-to-date systems
descriptions, and track every system component.
Has no comprehensive integration strategy or programwide
integration plan that describes an approach and methodology to
integrate current and future initiatives into the TSM systems
architecture. IRS also performs systems testing in operational
environments, including its service centers or computer centers,
rather than in a controlled environment dedicated to thorough
testing. Although IRS recognizes the need for strong systems
integration and systems acceptance testing and is taking steps
to improve each of these areas, it has not yet completed the
requisite plans or established an integrated testing facility.
Has an effort underway to define and document standard application
program interfaces with TSM. These interfaces define how
applications software can access and use standard functions.
However, IRS is proceeding with TSM systems development projects
before this effort is complete. As a result, these systems will
require evaluations to determine what rework is needed to ensure
that their conformance with the standard interfaces IRS is
developing.
IRS recognizes the need to better define, oversee, and manage TSM
development in fundamental technical areas. However, until it
institutes stronger and more disciplined technical management, IRS
risks developing systems that do not satisfy mission objectives and
that require significant and costly redesign or replacement.
TSM ACCOUNTABILITY AND
AUTHORITY WERE FRAGMENTED
-------------------------------------------------------- Chapter 0:4.5
TSM is not a one-time, turnkey replacement of all current subsystems;
it is a target system IRS plans to reach by incrementally upgrading
or replacing operational systems over several years. Accordingly, it
is important that IRS maintain an organizationwide focus to manage
and control all new modernization systems and all upgrades and
replacements of operational systems.
However, no organizational structure existed below the Commissioner's
office with the accountability and authority needed to manage the tax
systems modernization. Historically, accountability and authority
for systems development and operation were fragmented among IRS's
Modernization Executive, Chief Information Officer, and research and
development division. In May 1995, the Modernization Executive was
named Associate Commissioner and given responsibility to manage and
control modernization efforts previously conducted by the
Modernization Executive and the Chief Information Officer. However,
the research and development division does not report to the
Associate Commissioner.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5
In a briefing to the IRS Commissioner on April 28, 1995, GAO made
several recommendations aimed at overcoming the management and
technical weaknesses impeding successful modernization efforts. In
this regard, GAO recommends that IRS's electronic filing business
strategy focus on a wider population of taxpayers, including all
taxpayers who can file electronically most cost beneficially.
In addition, GAO recommends improvements to IRS's strategic
information management, software development capability, and
technical activities. First, GAO recommends that the Commissioner
take immediate action to improve IRS's strategic information
management by implementing a process for selecting, prioritizing,
controlling, and evaluating the progress and performance of all major
information systems investments, both new and ongoing, including
explicit decision criteria. Using the best available information,
IRS needs to develop quantifiable decision criteria that consider
such factors as cost, mission benefits, and technical risk. By June
30, 1995, IRS should review all planned and ongoing systems for
fiscal year 1996 using these criteria. Through this review, IRS will
provide the Congress with insight, based on consistently applied and
well-defined factors, upon which to gauge IRS's priorities and
rationalization for TSM projects.
Next, GAO recommends that the Commissioner (1) immediately require
IRS's future software development contractors to have CMM level 2
maturity and (2) by December 31, 1995, take measures that will
improve IRS's software development capability. The specific measures
recommended are intended to move IRS to CMM level 2 and include
implementing consistent procedures for software requirements
management, quality assurance, configuration management, and project
planning and tracking.
Finally, GAO recommends that the Commissioner take several actions by
December 31, 1995, to improve key system development technical
activities. These specific actions include (1) completing an
integrated systems architecture and security and data architectures,
(2) institutionalizing formal configuration management for all new
systems development projects and upgrades and developing a plan to
bring ongoing projects under formal configuration management, and (3)
developing security concept of operations, disaster recovery, and
contingency plans.
GAO also recommends that the IRS Commissioner assign the Associate
Commissioner responsibility for managing and controlling all systems
development activities, including the research and development
division's systems development efforts.
The time frames that GAO is recommending coincide with congressional
deliberations on IRS's fiscal year 1996 and the fiscal year 1997
budget cycle. Meeting these time frames is necessary to provide the
Congress a sound basis for funding investments in system
modernization projects, overseeing TSM's progress in achieving
mission improvements, and analyzing TSM costs and benefits.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 0:6
In June 21, 1995, comments on a draft of this report, IRS agreed with
GAO's recommendations for improving TSM in areas such as electronic
filing, strategic information management, software development,
technical infrastructure, and accountability and responsibility.
Further, IRS said that steps have already been started to implement
several of GAO's recommendations, including (1) convening an
electronic filing strategy group to develop a comprehensive strategy
that will broaden public access to electronic filing and (2)
conducting a critical program review to rescope IRS program
objectives, set priorities, and adjust funding levels for TSM.
IRS also said that a detailed action plan was being developed to
implement all of GAO's recommendations, and IRS will make every
effort, within available resources, to implement them by December
1995. In addition, IRS said that it recently completed a
self-assessment of its practices compared to GAO's best practices for
strategic information management. According to IRS, its
self-assessment confirmed GAO's findings and will help strengthen
IRS's overall response to GAO's concerns.
GAO believes that the steps IRS has outlined will help to move toward
ensuring that the TSM effort is better focused to meet IRS's mission
needs.
INTRODUCTION
============================================================ Chapter 1
The receipt, processing, and retrieval of vast quantities of paper
forms and documents is one of IRS's most critical problems. IRS
annually receives
over 200 million tax returns with multiple attachments,
about 1 billion information documents (for example, W2s and 1099s),
and
several hundred million pieces of taxpayer correspondence.
To process this enormous volume of paperwork, IRS uses
labor-intensive processes and systems to (1) convert data from tax
returns into machine usable form, (2) maintain taxpayer accounts,
including current and historical data, (3) ensure refunds are prompt,
and (4) prepare bills for tax payments due. Retrieving paper forms
and documents involves over 1.2 billion tax returns stored in over 1
million square feet of space.
Also, IRS collects most of the government's revenue, currently over
$1.25 trillion annually, and it employs over 113,000 people, more
than any other civilian agency. IRS is headquartered in Washington,
D.C., and has 7 regional offices, 63 district offices, 10 service
centers, and 2 computer centers.
BACKGROUND
---------------------------------------------------------- Chapter 1:1
PROCESSING TAX RETURN
INFORMATION
-------------------------------------------------------- Chapter 1:1.1
Upon receipt at IRS's service centers, paper-based tax returns and
related supporting and information documents are manually extracted
from envelopes, sorted, batched, coded, and transcribed\1 into
electronic format. The service centers send electronically formatted
data to IRS's main computer center in Martinsburg, West Virginia.
IRS stores nearly all the paper supplied by taxpayers as part of, or
in support of, their tax filings.
Tax return processing at IRS service centers was designed in the late
1950s. Today, nearly 4 decades later, IRS still processes tax return
data using the processes instituted when automated systems were first
installed in the service centers.
In today's technological climate, taxpayers have come to expect
faster, better, more convenient service in virtually every facet of
their lives. To meet these expectations, IRS's outdated tax
processes and systems are being used to electronically capture and
provide more and more information. At the same time, the number of
tax-related documents is greatly expanding.
--------------------
\1 Transcribing is the process of keying specific data from paper tax
returns into IRS information systems and validating and verifying its
accuracy.
TAX SYSTEMS MODERNIZATION
-------------------------------------------------------- Chapter 1:1.2
Between the late 1960s and the early 1980s, IRS began several efforts
to modernize its operations. These efforts did not succeed, and on
numerous occasions the Congress expressed concern about the cost of
the redesign efforts, the inadequacy of security controls over
taxpayer information, the lack of clear management responsibility for
the programs, and the paucity of technical and managerial expertise.
In late 1986, IRS produced plans for a new modernization effort,
known today as Tax Systems Modernization (TSM). IRS estimates that
TSM could cost between $8 billion and $10 billion through 2001.
Through fiscal year 1995, IRS will have spent or obligated $2.5
billion for TSM, which comprises 36 systems development projects.
About $1.1 billion more has been requested for fiscal year 1996.
IRS has developed a business vision to guide its modernization
efforts. This vision calls for a work environment that is virtually
paper-free, where taxpayer account updates are rapid and taxpayer
information is readily available to IRS employees for purposes such
as customer service and compliance activities.
IRS's overall redesign of its tax processing system is key to
achieving this vision. An important component of the redesign is
maximizing the receipt of electronic information to reduce the
receipt of paper documents. IRS plans, for example, to expand the
electronic receipt of tax returns.
However, IRS believes the requirement to process large volumes of
paper documents will exist for the foreseeable future. As a result,
IRS is designing the Document Processing System to scan paper
documents and electronically capture data for subsequent processing
and retrieval at workstations. This system will require staff using
personal computers to correct and add data that the system cannot
accurately capture from paper documents. Like its electronic filing
system counterparts, the Document Processing System is to capture 100
percent of the numeric data submitted on tax returns, compared to
about 40 percent captured from paper returns today.
PREVIOUSLY REPORTED TSM
PROBLEMS
-------------------------------------------------------- Chapter 1:1.3
Throughout the modernization, we have reported on critical issues
related to
the need to build an effective organization structure for managing
technology;
problems in developing specific TSM systems and the reliability of
reported TSM costs;
weak internal, computer security, and fraud controls; and
antiquated systems that were not designed to provide the meaningful
and reliable financial information needed to effectively manage
and report on IRS's operations.
Because of problems such as these, in February 1995, we designated
TSM a high-risk systems modernization effort.\2 In general, these
major efforts experience cost over-runs, are prone to delays, and
often fail to meet intended mission objectives. Appendix II is a
list of our prior reports and testimonies pertinent to TSM.
--------------------
\2 An Overview, GAO's High-Risk Series (GAO/HR-95-1, February 1995).
OBJECTIVE, SCOPE, AND
METHODOLOGY
---------------------------------------------------------- Chapter 1:2
Our objective was to review the business and technical practices IRS
has established to develop, manage, and operate its information
systems and, in particular, the TSM initiative. We examined IRS's
business strategy for reducing paper tax return submissions,
strategic information management processes,
software development capability,
technical infrastructure, and
systems development accountability and responsibility.
To review IRS's business strategy for reducing paper tax return
submissions, we interviewed IRS officials who have responsibility for
submission processing and electronic filing. We analyzed various
task force studies on electronic filing and summaries of issues
compiled by an IRS task team charged with promoting electronic
filing.
In addition, we examined IRS internal audit reports on the
performance and development of systems designed to handle paper
returns, reports of problems from the service centers responsible for
processing tax returns, and a risk assessment and critical design
review of operational and developmental systems. Further, we
reviewed project plans and technical charters for paper processing
systems, and we discussed systems requirements and performance test
results with the contractor developing the Document Processing
System.
To assess IRS's strategic information management processes, we
interviewed IRS officials who have responsibility for systems
development. We also analyzed IRS planning documents, including
IRS's Business Master Plan, Future Concept of Operations, and
Integrated Transition Plan and Schedule. We obtained and analyzed
IRS documentation and task force studies related to (1) planning and
managing information technology, (2) analyzing systems development
costs and benefits, (3) reengineering business processes, and (4)
training staff in the use of new information technology.
In analyzing IRS's strategic information management practices, we
drew heavily from our research on the best practices of private and
public sector organizations that have been successful in improving
their performance through strategic information management and
technology. These fundamental best practices are discussed in our
report, Executive Guide: Improving Mission Performance Through
Strategic Information Management and Technology (GAO/AIMD-94-115, May
1994), and our Strategic Information Management (SIM) Self-Assessment
Toolkit (GAO/Version 1.0, October 28, 1994, exposure draft).
To evaluate IRS's software development capability, we validated IRS's
August 1993 assessment of its software development maturity based on
the Capability Maturity Model (CMM) developed in 1984 by the Software
Engineering Institute at Carnegie Mellon University. CMM establishes
standards in key software development processing areas and provides a
framework to evaluate a software organization's capability to
consistently and predictably produce high-quality products. We
discussed with IRS software development officials IRS's CMM rating
and actions initiated to improve it.
We also identified and assessed IRS's initiatives to improve software
development capability in key process areas, including (1)
requirements management,\3 (2) project planning, tracking, and
oversight, and (3) configuration management. In another key process
area, software quality assurance, we examined, in particular, IRS's
use of metrics to control software development projects.\4
To assess IRS's technical infrastructures, we discussed security and
data standards with systems architects and technical specialists. In
addition, we obtained and analyzed
integrated systems architecture documents;
systems development documents for security and data standards; and
project plans, quality measurement plans, and technical charters
for all TSM projects.
To assess accountability and responsibility for developing systems,
we identified the IRS organizational components involved in
developing and operating information systems. We discussed with
IRS's Modernization Executive, Chief Information Officer, and
research and development division officials their respective systems
development roles, responsibilities, and accountability.
We performed our work at IRS headquarters in Washington, D.C., and at
facilities in Cincinnati, Ohio, and Nashville, Tennessee. On April
28, 1995, we briefed the IRS Commissioner and other senior IRS
executives on the results of our review and made recommendations to
them for overcoming the management and technical problems impeding
successful systems modernization efforts.
Our work was performed between February 1995 and June 1995, in
accordance with generally accepted government auditing standards.
IRS provided written comments on a draft of this report, which are
included as appendix I.
--------------------
\3 Software requirements management involves defining, validating,
and prioritizing requirements, such as functions, performance, and
delivery dates.
\4 Software quality assurance involves monitoring the actions and
products of line organizations to ensure compliance with established
standards, and highlighting product or process inadequacies.
Metrics, which are numerical measures presumed to predict an aspect
of software quality, are useful quality indicators.
A STRATEGY TO MAXIMIZE ELECTRONIC
FILING IS KEY TO MODERNIZATION
============================================================ Chapter 2
IRS is currently drowning in paper--a serious problem IRS can
mitigate only through electronic tax filings. But IRS will not
achieve the full benefits that electronic filing can provide because
it does not have a comprehensive business strategy to reach or exceed
its electronic filing goal, which is 80 million electronic filings by
2001. Today, IRS's estimates and projections for individual and
business returns suggest that, by 2001, as few as 39 million returns
may be submitted electronically, less than half of IRS's goal.
Maximizing electronic filings is important because tax returns filed
electronically do not have to move through IRS's labor-intensive
operations. Paper filings have to be opened, sorted, reviewed,
transcribed, shipped and stored, and then physically retrieved if IRS
employees later need data on the returns that are not transcribed.
IRS recognizes that increasing the number of electronic filings is
essential to both improve its tax return processing and advance
toward the virtually paperless environment envisioned by IRS under
TSM.
Creating a paperless environment, though, will involve making
significant changes to improve IRS's information management and will
require new processes and new ways of doing business. Private and
public sector organizations that have successfully improved their
performance have found that to move away from the status quo, an
organization must recognize opportunities to change and improve its
fundamental business processes. Without well-conceived business
strategies to capitalize on opportunities, meaningful change may be
slow, the quality of service may not improve, and modernization may
be impossible.
Consequently, one of IRS's most pressing modernization issues is the
efficient processing of vast quantities of information received on
tax returns, which in 1994, amounted to about 205 million returns.
In 1995, IRS expects total tax returns from individuals and
businesses to increase by 2 million, and by 2001, to reach 224
million filings.
To help process its avalanche of paperwork more efficiently, in 1990,
IRS introduced nationwide electronic filing to selected groups of
taxpayers as a means of using modern technology to streamline its
business processes. Looking to the future, IRS set a goal to receive
80 million tax filings electronically by 2001. IRS based this goal,
which accounts for about 35.7 percent of all tax filings expected in
2001, on a projection of electronic filing of 70 million individual
returns and 10 million business returns.
In working toward this goal, in 1994, about 16 million tax returns,
or 7.8 percent of all returns, were filed electronically, with about
50 percent of these being 1040A forms.\1 In 1995, IRS expects that
electronic filings will decrease to about 15 million, or 7.2 percent
of all tax returns.
On the basis of the current rate of electronic filings for
individuals, IRS now estimates that in 2001 only about 29 million
electronic returns will by filed by individuals. Combined with the
projected 10 million electronic filings from businesses, IRS may
receive only 39 million electronic returns in 2001. This is only
about 17.4 percent of the 224 million tax returns anticipated in
2001, less than half of IRS's goal. Table 2.1 summarizes IRS's
electronic filing activity for 1994 and projections for the future.
Table 2.1
Electronic Filing Activity and
Projections
Total
filings Number Percen
(in (in t of
Year millions) millions) total
--------------------------- ----------- ---------- ------
1994 (Actual) 205 16 7.8
1995 (Estimate) 207 15 7.2
2001 (Goal) 224 80 35.7
2001 (Estimate) 224 39 17.4
------------------------------------------------------------
IRS's current business strategy focuses primarily on promoting faster
refunds to clients of businesses that prepare and electronically
transmit tax returns. Tax return preparers and transmitters do not
pay a fee to IRS for electronic filings, but they charge a fee to
taxpayers. Consequently, IRS's business strategy for promoting
electronic filing is directed primarily at taxpayers who file using
third parties, are willing to pay to file electronically, file simple
tax returns, and are due refunds.
IRS has no comprehensive business strategy for promoting the benefits
of electronic filing to other taxpayers. In doing this, IRS should
consider all segments of the taxpaying population, including those
who (1) are unwilling to pay for tax preparer and transmitter
services, (2) owe IRS for balances due, and (3) file complex tax
returns. These taxpayers represent considerable potential for making
substantially greater use of electronic filing.
Moreover, IRS is not taking advantage of opportunities afforded by
personal computers to increase electronic filings. In recent years,
these computers have become a common fixture in many households. In
this regard, when personal computers are used to prepare tax returns,
taxpayers who are not willing to pay commercial transmitting fees
must print their electronically produced returns on paper and mail
them to IRS to be manually processed. This results in the redundant,
counterproductive conversion of the same data by both taxpayers and
IRS: taxpayers convert electronic data to paper returns, IRS then
laboriously converts information on the paper returns back to
electronic data.
Unless IRS attracts all potential electronic filers, it will never
achieve its vision of virtually paperless processing and will be
forced to process increasingly large workloads of paper tax returns.
Further, IRS's paper processing systems are not planned to
accommodate the large volume of paper returns that will result if
taxpayers file fewer returns electronically. For example, IRS is
designing the Document Processing System for use at five service
centers based on the assumption that, by 2001, at least 61 million of
the 224 million returns will be filed electronically, that is, 163
million paper returns will be processed through the Document
Processing System.
As table 2.1 shows, by 2001, since only 39 million tax returns may be
filed electronically, 185 million taxpayers could submit paper
returns, or about 22 million more returns than IRS is designing the
Document Processing System to process. Thus, IRS's most recent
estimates on individual filings for 2001 indicate that IRS may fall
far short of its electronic filing goal, which will result in an
increasing struggle to process paper filings.
--------------------
\1 Form 1040A is a simplified version of Form 1040.
RECOMMENDATION
---------------------------------------------------------- Chapter 2:1
To better achieve its virtually paperless processing environment, we
recommend that IRS refocus its electronic filing business strategy to
target, through aggressive marketing and education, those sectors of
the taxpaying population that can file electronically most cost
beneficially.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 2:2
IRS agreed with our recommendation regarding its electronic filing
strategy. IRS said it has convened a working group, chaired by the
electronic filing executive, to develop a detailed, comprehensive
strategy to broaden public access to electronic filing, while also
providing more incentives for practitioners and the public to file
electronically.
IRS said the strategy will include approaches for taxpayers who are
unwilling to pay for tax preparer and transmitter services, who owe
IRS for balances due, and/or who file complex tax returns. IRS said
further that the strategy will also address that segment of the
taxpaying population that would prefer to file from home by personal
computers.
We believe that, by developing a more comprehensive electronic filing
strategy, IRS will help to maximize the benefits possible through
greater use by taxpayers of electronic filing. These benefits are
central to more efficiently processing the vast quantities of
information IRS receives on tax returns and, thus, to achieve the
virtually paperless tax processing environment IRS hopes to attain
through modernization.
IRS IS WORKING TO OVERCOME
INEFFECTIVE STRATEGIC INFORMATION
MANAGEMENT PRACTICES
============================================================ Chapter 3
IRS is not yet effectively using a strategic information management
process to plan, build, and operate its information systems. TSM has
been underway for almost a decade and will require years of further
development effort and substantial human and financial resources.
IRS, however, does not yet have in place an effective process for
selecting, prioritizing, controlling, and evaluating the progress and
performance of major information systems investments.
A sound strategic information management process involves several
fundamental practices: (1) applying strategic planning, (2) managing
information technology as investments, (3) analyzing costs and
benefits and measuring performance, (4) using business process
analysis, and (5) upgrading skills and training. This process
focuses on results and emphasizes simplifying and redesigning complex
mission processes, which is essential to meeting mission goals and
satisfying customers' needs.
IRS recognizes the importance to TSM's success of implementing a
sound strategic information management process and has assessed its
strategic information management using GAO's strategic information
management self-assessment toolkit.\1 IRS's self-assessment
identified improvements for better managing information systems. We
too found serious shortcomings that underscore the urgency of IRS
bolstering the strategic information management process it has begun.
We also identified IRS efforts to upgrade skills and training.
--------------------
\1 Strategic Information Management (SIM) Self-Assessment Toolkit
(GAO/Version 1.0, October 28, 1994 exposure draft).
STRATEGIC PLANNING IS
INCOMPLETE AND INCONSISTENT
---------------------------------------------------------- Chapter 3:1
Although IRS has developed several types of plans for carrying out
its current and future operations, these plans are neither complete
nor consistent. Moreover, IRS's various planning documents are not
linked to each other or to TSM budget requests. Even though TSM has
been underway for 10 years, complete, clear, and concise planning for
TSM and its multibillion dollar investment is not evident. As a
result, it is difficult for IRS to identify and effectively focus on
completing priority aspects of TSM.
Public and private sector organizations that have been successful in
developing major systems have found that, to be successful, once the
organization has made a serious commitment to change its management
of information and technology, it is paramount to adopt a strategic
planning approach. Their experience is that strategic business and
information system plans must have a tight link to mission goals and
must be predicated on satisfying explicit, high-priority customer
needs. This orientation helps to ensure that information technology
projects are delivered on time and within budget and that they
produce meaningful improvements in cost, quality, or timeliness of
service.
We identified several different efforts by IRS to prepare plans to
delineate a vision for the future and actions required to realize
that vision. These planning documents include
the Business Master Plan, which reflects the business priorities
set by IRS's top executives and links IRS's strategic objectives
and business vision with the tactical actions needed to
implement them;
the IRS Future Concept of Operations, which articulates IRS's
future business vision so that the Congress, IRS employees, and
the public can see and better understand IRS's plans for serving
the public; and
the Integrated Transition Plan and Schedule, which provides a
top-level view of the modernization program's tasks, activities,
and schedules and is the primary tool used for accountability
for delivering the products and services necessary to implement
modernization.
We found, however, that these documents are incomplete and
inconsistent. For example, as of May 1995, 4 volumes of the 10
volume IRS Future Concept of Operations had not been completed.
These volumes covered (1) national and regional offices, (2) workload
distribution management, (3) area distribution centers, and (4)
process flows. While the six completed volumes include critical
areas, the incomplete documents are necessary for a comprehensive
vision of IRS's future operations.
Also, of the 27 action items identified in the Business Master Plan
that relate to information systems, 15 could not be identified in the
Integration Transition Plan and Schedule. Further, the Business
Master Plan's actions and performance measures have not been changed
to reflect recent electronic filing trends, which indicate that IRS
will fall far short of its electronic filing goal.
We found other indications of weak planning processes as well.
Specifically, IRS did not have a fully integrated planning and
budgeting process for TSM, although the Office of Economic Analysis
is moving in that direction. For example, this office is developing
a new TSM cost model for IRS. Steps such as this are positive
because a strong tie between TSM plans and IRS budgets will be
especially important to ensure that information is available to IRS
managers and the Congress to show TSM's future funding needs and the
results of past investments.
While IRS has undertaken fundamental TSM planning, stronger overall
strategic planning for TSM is still needed. This would involve (1)
defining the information technology capabilities required to support
reengineered business processes, (2) identifying, assessing, and
mitigating the risks involved in developing both TSM as a whole, as
well as individual component projects, (3) formulating schedules and
milestones for development, and (4) allocating needed resources.
INFORMATION TECHNOLOGY IS NOT
MANAGED AS AN INVESTMENT
---------------------------------------------------------- Chapter 3:2
Currently, IRS does not have a process to manage TSM information
systems projects as investments, even though IRS expects the
government's past and future investment in TSM to exceed $8 billion.
Foremost, at the time of our review, IRS lacked comprehensive
decision criteria for controlling and evaluating TSM projects
throughout their life cycles.
When organizations use strategic information management best
practices, they manage information systems projects as investments,
rather than expenses. These organizations view projects as efforts
to improve mission performance, not as efforts to implement
information technology. For public and private sector organizations
that have been successful in developing major systems, the basis for
making decisions on information technology investments has been an
explicit set of criteria that are used to evaluate the expected
mission benefits, potential risks, and estimated cost of each
project. This investment focus systematically reduces inherent risks
while maximizing benefits of complex projects.
IRS maintains that all TSM projects have equal priority and must be
completed or the modernization will fail. An "all-or-nothing"
approach to large information technology projects is usually
unrealistic and generally unattainable. Instead, a reasoned and an
explicit framework for managing information technology investments is
essential.
IRS currently holds program control meetings to assess and control
information technology. However, these meetings have generally
focused on the costs and implementation schedules of individual
projects, rather than on comprehensively evaluating and prioritizing
risks and returns expected from these investments. Instead of using
explicit criteria to measure risks and returns, IRS evaluates each
project's progress using a time-line.
At the completion of our review, IRS had developed draft criteria for
TSM projects. These criteria included risk and return factors (e.g.,
cost, project size, and mission benefit), which it plans to use for
the first time during top management's review of the fiscal year 1997
budget. However, these factors were not defined so they could be
used consistently to assess projects. For instance, IRS
characterized project size as small, medium, large, and very large,
but did not quantify these terms. Similarly, IRS has not yet defined
decision criteria and quantifiable measures to assess mission
benefits, risk, and cost, all of which are important to enable IRS
managers to adequately select, control, and evaluate information
systems projects. IRS is currently developing better decision
criteria.
Managing TSM as an investment would require IRS to assess,
prioritize, control, and evaluate its investment in current and
planned TSM information technology projects based on explicit and
consistently applied decision criteria. By adopting this approach,
top management's attention would be drawn to assessing and managing
risk and making the tradeoffs between continued funding of existing
operations and developing new capabilities. Most important, with a
disciplined process, IRS could promptly identify, and thus avoid
investing in, higher-risk projects that have little potential to
provide significant mission benefits. Moreover, this would reenforce
accountability for improved performance.
ANALYZING COSTS AND BENEFITS IS
INADEQUATE
---------------------------------------------------------- Chapter 3:3
Contrary to best practices used by leading private and public
organizations, IRS's TSM costs and benefits analysis is inadequate.
As a result, IRS and the Congress do not know whether TSM information
systems projects will really make a difference. Until an adequate
analysis is performed and measures are defined, IRS will not know
whether investments in TSM are worthwhile.
In January 1995, IRS advised the House Budget Committee that,
including operating costs for the next 10 years, TSM will cost about
$13 billion and will provide over $32 billion in benefits. However,
IRS's overall cost projection is unreliable for several reasons. For
example, IRS based the projection on an October 1992 TSM cost model,
which IRS did not adequately update to reflect systems that have
since been added to TSM, IRS's more recent business visions, and
changes in TSM systems development methods.
The benefits estimate also had shortcomings. For instance, in some
cases, IRS attributed to TSM the savings associated with reducing
staff resources; in other cases, IRS computed benefits based on
additional revenues expected if staff were reassigned to tax
collection. Although a decision to use these staff for collections
may increase revenue, the additional staff--not the system--will
provide this benefit. This point becomes clear when the following
scenarios are considered: (1) IRS could assign additional staff to
collections independent of the information system and (2) if IRS
reassigns to other nonrevenue producing activities the staff years
saved, the revenue benefits would evaporate even though the
information system would not change. A convincing benefits analysis
for a system must compare operational costs with and without the
system, other variables being held constant.
IRS recognizes that it has not adequately assessed TSM costs and
benefits and is currently working with a contractor on an economic
analysis to better reflect the cost and benefits of TSM. IRS expects
another cost and benefit analysis to be completed by September 1995.
We will continue to monitor IRS's progress in analyzing TSM cost and
benefits.
REENGINEERING EFFORTS ARE NOT
TIED TO SYSTEMS DEVELOPMENT
PROJECTS
---------------------------------------------------------- Chapter 3:4
After many automated systems design and development efforts were
already underway, IRS started business process reengineering, which
involves critically reexamining core business processes and
redesigning them to achieve significantly better performance.
Compounding this problem is IRS's lack of a comprehensive plan and
schedule defining how and when to integrate these business
reengineering efforts with on-going TSM projects.
Organizations that successfully develop systems do so only after
analyzing and redesigning critical business processes. Information
systems projects that do not consider business process redesign
typically fail or reach only a fraction of their potential.
Accomplishing significant improvement in performance nearly always
requires streamlining or redesigning critical work processes.
IRS has identified six core business areas and defined 11 business
processes that support these areas. Of these 11, 3 were selected to
begin reengineering efforts. Those selected for initial redesign are
(1) processing returns, (2) responding to taxpayers, and (3)
enforcement actions.
Overall, we found IRS's reengineering methods to be consistent with
generally recognized business process reengineering principles. IRS
had, for example, assessed some existing data on customer values,
analyzed current processes, and designed target processes and plans
to validate the target designs. Further, IRS has a project
management structure consisting of process owners, an executive
steering committee, project managers, cross-functional teams, and
contractor support to ensure that all stakeholders can participate.
However, these efforts are not yet complete, and IRS did not assess
the actual steps needed to implement these efforts.
IRS officials acknowledge that reengineering efforts began after the
start of many TSM systems projects. Until reengineering is
sufficiently completed to drive TSM projects, there is no assurance
that the projects will achieve the desired business objectives and
result in improved operations.
IRS IS WORKING TO UPGRADE
SKILLS AND TRAINING
---------------------------------------------------------- Chapter 3:5
IRS is currently reassessing its skill and competency base to ensure
that its personnel and training programs will meet future needs.
Operating and maintaining progressively sophisticated systems, such
as those comprising TSM, requires continuously higher skill levels
and updated knowledge--an additional critical factor for success,
according to best-practice organizations. Antiquated skill bases can
inhibit an organization's ability to change.
IRS has several initiatives planned and underway to upgrade the
skills of its personnel. For example, IRS
has defined positions needing competency assessments;
plans to assess staff skills using competency assessment
instruments, which are currently being developed; and
is reorganizing and strengthening its training program by
establishing a Corporate Education unit.
We are currently assessing IRS's human resource planning for
modernization and will continue to monitor progress in this area.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 3:6
To address IRS's strategic information management weaknesses, we
recommend that the IRS Commissioner take immediate action to
implement a complete process for selecting, prioritizing,
controlling, and evaluating the progress and performance of all major
information systems investments, both new and ongoing, including
explicit decision criteria.
We also recommend that IRS use these criteria to review all planned
and ongoing systems investments by June 30, 1995. Meeting this time
frame is important so that the Congress has a sound basis for
determining IRS's fiscal year 1996 appropriations.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 3:7
IRS agreed with our recommendations to improve its strategic
information management. In addition, IRS said that it had recently
completed a self-assessment of its practices compared to GAO's best
practices for strategic information management. According to IRS,
its self-assessment confirmed GAO's findings and will help strengthen
IRS's overall response to GAO's concerns.
In response to our recommendations, IRS said that it
will continue to work on simplifying and ensuring the consistency
of all its key planning documents;
has initiated a priority setting process for meeting business needs
through information system investments;
has developed an initial set of investment evaluation criteria for
use as part of an ongoing process to evaluate spending plans for
information systems;
has completed a comprehensive review of the proposed fiscal year
1996 budget for TSM, which will enable IRS to rescope its
program objectives, set priorities, and adjust funding levels
for TSM;
will continue to refine the investment evaluation criteria and also
institutionalize a formal process based on the use of this
criteria; and
is developing and implementing the use of an information technology
investments alternative to select, prioritize, control, and
evaluate information technology investments to achieve
reengineered program missions.
Actions such as these could provide IRS the underpinnings it needs
for strategic information management. IRS indicated that progress
toward implementing these improvements will be monitored by the IRS's
Associate Commissioner. We believe that this is essential to ensure
prompt and effective implementation.
Regarding a cost and benefits analysis, IRS said that the September
1995 analysis will address the costs and benefits of TSM and allow
IRS to identify and focus on competing priorities. In particular,
IRS expects the new analysis to reflect a much more extensive benefit
estimate than IRS currently has available. We believe an adequate
cost and benefits analysis will help IRS to know whether investments
in TSM are worthwhile.
Regarding skills and training, IRS said that it is taking steps to
ensure that personnel and training programs meet future needs,
especially those relating to information systems. These steps
include (1) establishing a training steering committee to consolidate
all information systems training currently underway, with the goal of
increasing the skill level of IRS employees and (2) identifying job
requirements for information systems professionals, which IRS will
use in developing training and education programs that are directly
linked to mission needs and critical occupational performance goals.
Although IRS is in the process of identifying job requirements, we
believe that, until reengineering is complete, IRS can only
incorporate prototype job requirements into its training and
development efforts. In addition, IRS's current plans do not address
how job requirements created as a result of reengineering efforts
will be incorporated into its training environment.
SOFTWARE DEVELOPMENT PROCESS IS
WEAK BUT IMPROVEMENTS ARE UNDERWAY
============================================================ Chapter 4
IRS's software development activities are inconsistent and poorly
controlled because IRS has few detailed procedures for its engineers
to follow in developing software. IRS's software development
deficiencies can greatly affect the quality, timeliness, and
cost-effectiveness of TSM. Unless IRS improves its software
development capability, it is unlikely to build TSM timely or
economically and systems are unlikely to perform as intended.
To assess its software capability, in September 1993, IRS rated
itself against a Capability Maturity Model (CMM) designed by the
Software Engineering Institute, a nationally recognized authority in
the area. IRS found that, even though TSM is a world-class
undertaking, its software development capability is immature. IRS
placed its software development capability at the lowest level,
described as ad hoc and sometimes chaotic and indicating significant
weaknesses in software development capability.
Realizing that its software development capability needed
improvement, IRS initiated process action teams to address software
development weaknesses in key process areas. These teams have made
varying degrees of progress to improve IRS's software development
capability and define uniform procedures in the key process areas.
Their progress notwithstanding, substantial additional improvement is
necessary before IRS's software development capability can be
upgraded to at least the next CMM level, where its activities would
be more disciplined and considered to be repeatable. Whether
software development is done by IRS, which has nearly 2,000 people
working in the area, or by contractors, mature software development
capabilities are key to quality, timely, and cost-effective TSM
software development.
Closely associated with one key software development process area,
software quality assurance, is the use of software metrics, which are
numerical measures used to predict an aspect of software quality. In
this regard, we found that IRS has not adequately defined a suite of
metrics. Moreover, IRS is not consistently or effectively using even
its limited metrics for assessing the quality of software development
projects throughout their life cycles.
SOFTWARE DEVELOPMENT CAPABILITY
IS IMMATURE
---------------------------------------------------------- Chapter 4:1
The Software Engineering Institute was established at Carnegie Mellon
University in 1984 primarily to address the Defense Department's
software development problems. In 1991, the Institute developed CMM
for use by organizations to evaluate their capability to consistently
and predictably produce high-quality software. Table 4.1 describes
CMM's five maturity levels.
Table 4.1
CMM Levels and Descriptions
Level Name Description
------------------ ------------------ --------------------
5 Optimizing Continuous process
improvement is
enabled by
quantitative
feedback from the
process and from
testing innovative
ideas and
technologies.
4 Managed Detailed measures of
the software process
and product quality
are collected. Both
the software process
and products are
quantitatively
understood and
controlled using
detailed measures.
3 Defined The software process
for both management
and engineering
activities is
documented,
standardized, and
integrated into an
organizationwide
software process.
All projects use a
documented and
approved version of
the organization's
process for
developing and
maintaining
software.
2 Repeatable Basic project
management processes
are established to
track cost,
schedule, and
functionality. The
necessary process
discipline is in
place to repeat
earlier successes on
similar projects.
1 Initial The software process
is characterized as
ad hoc and
occasionally even
chaotic. Few
processes are
defined and success
depends on
individual efforts.
------------------------------------------------------------
IRS rated itself at CMM level 1 because its assessment showed
significant weaknesses in all key process areas prescribed for an
organization to attain a level 2 capability. The key process areas
designated by the Institute as necessary to reach CMM level 2 include
(1) requirements management, (2) software project planning, (3)
software project tracking and oversight, (4) software quality
assurance, and (5) software configuration management.
Further, the National Research Council also identified IRS's software
development weaknesses and, in its Fall 1994 report on TSM,\1 stated
that IRS needed to develop a mature software development
organization. The Council reported that, compared to accepted modern
standards, IRS's internal development capability is largely out of
date and rudimentary.
--------------------
\1 Continued Review of the Tax Systems Modernization of the Internal
Revenue Service (Interim Report), Computer Science and
Telecommunications Board, National Research Council.
SOFTWARE DEVELOPMENT
IMPROVEMENT INITIATIVES ARE
PROGRESSING
---------------------------------------------------------- Chapter 4:2
To improve its software development capability and attain a higher
CMM rating, the IRS Information Systems Organization's Applications
Design and Development Management group initiated five process action
teams to address the weaknesses identified by IRS's assessment and
the National Research Council's review. Table 4.2 identifies the
teams and describes the key process areas each was to address.
Table 4.2
IRS Teams Addressing Key Software
Development Process Areas
Team Areas to be addressed
------------------ ----------------------------------------
Requirements Defining, validating, and prioritizing
Management requirements, such as functions,
performance, and delivery dates.
Software Quality Monitoring the actions and products of
Assurance line organizations to ensure compliance
with established standards, and
highlighting product or process
inadequacies.
Project Planning Ensuring that project plans define what
and Tracking is to be done, at what cost, by whom,
and on what schedule, and establishing
criteria for tracking projects.
Testing Defining procedures for the testing of
software units and systems and for
acceptance testing.
Configuration Selecting project baseline items, such
Management as specifications; systematically
controlling these items and changes to
them; and recording and reporting status
and change activity for these items.
------------------------------------------------------------
The following discussion highlights the work of these teams, which we
found in various stages of completion. Although the teams have
generally made progress, IRS's software development capabilities
remain weak in each of the key process areas they were to address.
The Requirements Management team (1) studied and flow charted the
process for requesting information services and (2) generated
and is delivering related training materials. However, the
requirements management process developed by the team is
currently being applied to only legacy systems (i.e., existing
IRS systems). An equivalent requirements management process for
TSM systems was still under development. Also, customer
involvement with the team's requirements management process has
been limited.
The Software Quality Assurance team adopted the peer review portion
of a planning, review, and inspection process developed by IRS's
Quality Assurance Group. The team is applying this process to
selected projects and has developed training for using the
process, which IRS is giving to its systems engineers. However,
IRS has not yet decided whether to conduct the team's peer
review approach on all projects. Also, IRS has yet to define
detailed procedures for performing other software quality
assurance functions, such as (1) ensuring compliance of software
products and processes with defined standards, (2) independent
verification of product quality, (3) periodic audits and reviews
by the Software Quality Assurance group, and (4) feedback of the
software quality assurance activities and findings to facilitate
improvement of the process.
The Project Planning and Tracking team selected a software tool for
planning and tracking the progress of software development
projects. Because the team did not prepare guidelines
specifying the minimum planning and tracking elements to apply
to projects, project managers who use the software must define
the details to track. As a result, this tool is being
inconsistently used and, thus, IRS has been unable to
consistently track the progress of their projects.
The Testing team has issued guidance on unit testing. However,
there are no procedures for systems and acceptance testing.
The Configuration Management team is waiting for configuration
management of the corporate-level to be defined in order to
define lower-level processes and procedures. The only
configuration management in place is version control of
software.\2 As a result, important items are not yet under
configuration management, including documentation, and software
development folders.
Although the teams have made progress, their accomplishments have not
significantly improved IRS's software development capability.
Foremost, IRS has not developed and implemented consistent guidelines
and procedures in the key process areas essential at CMM level 2.
Unless IRS's weaknesses in software quality assurance and software
configuration management are corrected, IRS faces a much greater risk
of extensive rework, schedule slippage, and cost overruns in
developing software.
This risk is present whether IRS or a contractor develops TSM
software. In this regard, to effectively oversee a contractor's work
to develop software, and thereby help to ensure prompt and successful
completion of the software, it is important for IRS's software
project managers to understand the practices needed to develop
software at CMM level 2. To further mitigate the risk of potential
problems in developing software under contracts, it is critical that
IRS's software development contractors not be at CMM level 1. IRS
does not, however, require all of its software development
contractors to be at least at CMM level 2.
--------------------
\2 Version control is the process of certifying and releasing
improved versions of software and its documentation in a controlled
manner (as opposed to making ad hoc untested and unvalidated
changes).
METRICS ARE NOT CONSISTENTLY
USED
---------------------------------------------------------- Chapter 4:3
Although not a specific key process area for rating an organization's
software capabilities, it is nonetheless crucial that a set of
quality indicators, and their associated measures, called metrics, be
used to assess the quality of software development throughout a
project. IRS has not yet effectively established such a measurement
process.
Early detection and avoidance of problems and control of software
development projects are possible through the collection, validation,
and analysis of metrics, which are numerical measures presumed to
predict an aspect of software quality. Useful metrics include
numbers of defects found at various stages of development, costs to
repair defects, and the extent of test coverage.
Basically, metrics, such as the number and frequency of errors
associated with a specific section of software, are taken to analyze
the quality of software. Such analyses can identify situations where
quality is unacceptable or questionable. In this way, the metrics
are validated against quality factors throughout a software
development project.
According to IRS officials responsible for software development, IRS
has not yet defined a complete suite of metrics to be used in the
software development program to assess the on-going quality of TSM
projects. IRS's present use of metrics allows for only one type of
metric, collectively called function points.\3
Even so, IRS's use of function points for assessing all software
development projects is inconsistent, and IRS does not have a firm
schedule for full implementation throughout the agency. In addition
to function points, the following metrics, would, at a minimum, also
be necessary: (1) complexity, (2) personnel and effort, (3)
problems/defects by development phase, and (4) cost per defect.
Further, IRS's use of function points does not trace back to quality
improvement goals derived from IRS's business objectives. In this
regard, IRS could use the following metrics to measure software
attributes related to business goals:
Fewer product defects found by customers.
Earlier identification and correction of defects.
Fewer defects introduced during development.
Faster time to market.
Better predictability of project schedules and resources.
Without clearly establishing a suite of metrics that trace back to
business objectives through quality improvement goals, and that are
implemented organizationwide in a uniform and consistent manner, IRS
will be hampered in assessing the progress and quality of its
software projects. Moreover, the absence of a suite of metrics makes
it difficult for IRS to identify the reasons certain software
development practices perform well while others perform poorly.
Metrics, therefore, when used organizationwide in developing
software, would provide IRS a means to better ensure uniform software
development, thus avoiding the potential for repeating problems that
could be costly and time-consuming to correct.
--------------------
\3 Function points are used to measure a project's size, such as
lines of code.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 4:4
To address IRS's software development weaknesses, we recommend that
the IRS Commissioner immediately require that all future contractors
who develop software for the agency have a software development
capability rating of at least CMM level 2.
To further upgrade IRS's software development capability, we also
recommend that the Commissioner take action before December 31, 1995,
to
define, implement, and enforce a consistent set of requirements
management procedures for all TSM projects that goes beyond
IRS's current request for information services process, and for
software quality assurance, software configuration management,
and project planning and tracking and
define and implement a set of software development metrics to
measure software attributes related to business goals, such as
those outlined in this chapter.
Completing these actions by the end of 1995 is essential so that the
Congress, in monitoring TSM's progress and acting on TSM budget
requests, has assurance that IRS will be able to effectively develop,
and/or oversee contractors' development of, software associated with
systems modernization projects.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 4:5
IRS agreed with our recommendations for improving its software
development capability, and is taking steps to do so. IRS said that
it is committed to developing consistent procedures addressing
requirements management, software quality assurance, software
configuration management, and project planning and tracking.
Regarding metrics, IRS said that it is developing a comprehensive
measurement plan to link process outputs to external requirements,
corporate goals, and recognized industry standards. IRS said also
that it has "baselined" all legacy systems using an accepted Software
Engineering Institute metric.
We believe these steps, if implemented and institutionalized
effectively, would provide IRS the disciplined approach necessary to
improve its software development capability. Mature software
development capabilities are key to quality, timely, and
cost-effective TSM software development.
IRS also stated its belief that most government agencies and private
organizations are not far along in raising their software development
maturity profiles. We have identified several government
organizations that have adopted CMM and are moving toward higher CMM
levels. For example, the Department of the Army's Information
Systems Software Development Center in Virginia and the Department of
the Air Force's Sacramento Air Logistics Center were both assessed by
SEI authorized assessors as CMM level 3. The Air Force also has a
deadline for all its software activities to reach CMM level 3 by
1998. The software development capabilities of other organizations
notwithstanding, we believe that a complex and costly systems
development project, such as TSM, at a minimum, would warrant a CCM
level 2.
IRS RECOGNIZES IMPORTANCE OF
BETTER SYSTEMS ARCHITECTURES,
INTEGRATION, AND TESTING BUT THESE
ARE NOT YET ADEQUATE
============================================================ Chapter 5
IRS is not adequately performing and managing key TSM technical
activities critical to the success of a large and complex systems
modernization effort. In particular, IRS has not (1) defined and
completed a TSM architecture, (2) established effective processes for
configuration management, (3) defined the interfaces and standards
needed to ensure that TSM components successfully integrate and
interoperate, and (4) defined and completed TSM testing plans and
established a testing facility.
IRS recognizes that, for modernization to succeed, TSM's technical
activities must be better defined, performed, and managed. Until IRS
improves these areas, it is at increased risk of developing systems
that are unreliable, do not meet user needs, cannot work together
effectively, and require significant and costly redesign and
reprogramming to correct weaknesses.
TSM INTEGRATED SYSTEM
ARCHITECTURE IS INCOMPLETE
---------------------------------------------------------- Chapter 5:1
IRS has adopted a systems development methodology, known as
Information Engineering, which is a formal, structured system
development methodology widely used in the public and private sectors
to provide a disciplined approach to information systems development.
The principal deliverable of Information Engineering's first stage,
Information Strategic Planning, is an integrated systems
architecture.
An integrated systems architecture (1) guides and constrains system
design and development by providing a balanced, top-down view of the
system, which system designers need to build the system and (2)
organizes system functionality and defines relationships among those
functions. In establishing this guidance and functionality, it is
key to define security and data architectures and standard
application program interfaces.
In July 1993, IRS published an initial version of its integrated
system architecture. According to this document, the TSM integrated
systems architecture will be completed as other modernization work
progresses. This approach defeats the purpose of an integrated
systems architecture, which is to guide a system's development, not
to merely document its development without formal guidance. Further,
TSM security and data architectures and standard application program
interfaces are incomplete and, thus, designers and developers do not
have sufficient guidance to build individual TSM systems.
SECURITY ARCHITECTURE IS
INCOMPLETE
-------------------------------------------------------- Chapter 5:1.1
Because TSM's security architecture is incomplete, systems designers
do not have sufficient guidance on how to incorporate restricted
access to IRS systems and data. IRS has made progress in defining
its security requirements, but it continues to develop and implement
systems without first completing the necessary security architecture
and security applications.
In February 1994, IRS issued a risk assessment that identified
potential security risks, determined their severity, and identified
areas needing safeguards, and in October 1994, issued an information
security policy. Since then, IRS has completed security documents
relating to
high-level security requirements, including mission, management,
and technical security requirements;
functional security requirements, which specify user security
needs;
a preliminary data sensitivity analysis, which is used to determine
data sensitivity (e.g., sensitive but unclassified, etc.); and
a draft information system target security architecture, which
specifies TSM information security goals.
In addition, an IRS infrastructure and engineering task group has
defined a set of preliminary security applications program interfaces
that will guide application developers in requesting systems security
functions. IRS officials told us that once these interfaces have
been completed and thoroughly tested, IRS will mandate their use.
This progress notwithstanding, the TSM security architecture and
security applications interfaces remain incomplete and unavailable to
systems designers and developers. Without this crucial systems
security guidance, IRS has no assurance that taxpayer data will be
adequately protected.
Key security guidance that has not yet been developed includes
a disaster recovery and contingency plan, which would ensure that
information systems can restore operations and data in the case
of sabotage, natural disaster, or other operational disruption;
a security concept of operations, which would define IRS plans for
operating in TSM's new security environment;
a security test and evaluation plan, which would validate the
operational effectiveness of system security controls;
a security certification and accreditation plan, which would
provide IRS managers and system security officers adequate
assurance that the system will protect information as required
by the security policy;
a communications security plan, which would define how security
controls will be implemented when sending and receiving
sensitive information electronically between and among
distributed TSM subsystems and external agencies that must
provide tax-related information to IRS; and
an identification and authentication plan, which would define
processes to verify user identities when accessing sensitive tax
data.
Security has been a serious problem with IRS's current systems. Our
audits of IRS's financial statements under the Chief Financial
Officers Act (Public Law 101-576) have shown that IRS's controls do
not yet ensure that taxpayer data are adequately protected from
unauthorized access, change, disclosure, or disaster. Specifically,
IRS has not adequately (1) restricted access to taxpayer data to only
those employees who need it, (2) monitored the activities of
thousands of employees who were authorized to read and change
taxpayer files, and (3) limited the use of computer programs to only
those that have been authorized.
We have reported that,\1 as a result, IRS did not have reasonable
assurance that the confidentiality and accuracy of these data were
protected and that the data were not manipulated for purposes of
personal gain. IRS's own reviews have identified instances where IRS
employees (1) manipulated taxpayer records to generate unauthorized
refunds, (2) accessed taxpayer records to monitor the processing of
fraudulent returns, and (3) browsed taxpayer accounts that were
unrelated to their work, including those of friends, relatives, and
neighbors.
--------------------
\1 Financial Audit: Examination of IRS' Fiscal Year 1993 Financial
Statements (GAO/AIMD-94-120, June 15, 1994).
DATA ARCHITECTURE REFLECTS
EXISTING PROCESSES RATHER
THAN REENGINEERED PROCESSES
-------------------------------------------------------- Chapter 5:1.2
IRS is perpetuating its current data weaknesses by continuing to
build TSM systems without the guidance afforded by a data
architecture that reflects reengineered processes. An IRS analysis
of its current systems identified the following data weaknesses:
Updated data on one system are not immediately available to users
of other systems.
Master data files are updated once a week, and it can take up to 2
weeks for data in a taxpayer account to be changed.
Inconsistent and incomplete data on different systems can affect
fundamental computations and can result, for example, in
inconsistent calculations of interest and penalties.
Data are stored in unique formats on different systems and are
accessed using various techniques.
In 1994, to address data weaknesses, IRS initiated the Corporate
Accounts Processing System project. IRS is developing this project
in phases over 7 years, with each phase adding new TSM functionality.
Through the Corporate Accounts Processing System, IRS expects to
provide more efficient access to data, reduce data redundancy, and
improve data integrity.
Nonetheless, the success of the Corporate Accounts Processing System
project depends on improving current business processes through
reengineering. At the time of our review, however, the project was
modeling IRS's existing business processes because IRS had not
completed its reengineering.
To effectively correct existing data weaknesses that IRS identified
and that the Corporate Accounts Processing System project is to
address, IRS must first define how its business processes will be
reengineered. Only then will IRS be in a strong position to build
new systems based on a data architecture that reflects reengineered
business processes.
STANDARD APPLICATION PROGRAM
INTERFACES ARE NOT DEFINED
-------------------------------------------------------- Chapter 5:1.3
Standard application program interfaces are essential to guide
systems development because they define how applications software can
access and use standard functions and services (e.g., communications
services). These interfaces provide many systems development
benefits, including improved interoperability, consistent
implementation, less complex applications, standardized software
coding, and simplified maintenance.
Realizing the benefits of providing standard application program
interfaces for system development, IRS has established an interface
task group and initiated an effort to define, code, test, and
document standard application program interfaces for TSM. IRS has
drafted an infrastructure services manual to provide an explanation
of infrastructure services that will be available to systems
developers. IRS also expects to prepare a more comprehensive and
detailed manual describing application processing interfaces.
However, many TSM standard application program interfaces are not yet
defined, implemented, or documented. Nonetheless, IRS is continuing
to build TSM projects. As a result, these projects are likely to
require modification once standard application program interfaces are
defined and required.
EFFECTIVE CONFIGURATION
MANAGEMENT PRACTICES ARE NOT
ESTABLISHED
---------------------------------------------------------- Chapter 5:2
Systems change throughout their life cycle to (1) improve systems
designs and operations and facilitate maintenance, (2) reflect
changing mission requirements, and (3) respond to changes in the
budget and schedules. These changes must be controlled through
configuration management to ensure that they are cost-effective and
properly implemented, documented, and tested.
Configuration management ensures that the integrity and stability of
a system are preserved as it is designed, built, operated, and
changed. Configuration management is also important for making
engineering and trade-off decisions, maintaining up-to-date systems
descriptions, and tracking every system component.
In 1994, IRS established an Information System Configuration Control
Board to manage and control all systems changes. However, the Board
has focused on monitoring individual project costs and schedules and
developing configuration management guidance. A process has not yet
been established to manage systems changes. Further, IRS does not
have a configuration management plan that precisely defines the
processes to be implemented, how and when they will be implemented,
and who will be responsible for performing specific configuration
management functions.
SYSTEMS INTEGRATION PLANNING IS
INCOMPLETE
---------------------------------------------------------- Chapter 5:3
In 1992, IRS initiated an effort to design and develop both a
comprehensive integration strategy and a programwide integration plan
to help IRS successfully transition from its current environment to
one that meets TSM-defined objectives and capabilities. A
preliminary strategy described by IRS's Executive for Systems
Architecture was for (1) an integration approach that included a
methodology to integrate current and future initiatives into the TSM
systems architecture, (2) an associated problem detection and
resolution process, and (3) the analysis processes (e.g., testing and
quality assurance) required to ensure projects are being, and have
been, successfully integrated. The preliminary strategy addressed
both the integration of individual projects and the transition of all
projects to an integrated processing environment.
Since then, little has been done to complete a comprehensive
integration strategy or develop an integration plan that defines
implementation guidance and processes. In 1994, IRS planned to
perform further work on integration management, but did not fund this
effort in either fiscal year 1994 or fiscal year 1995. Until there
is an effective integration process and a completed integration plan
in place, IRS will have little assurance that its systems
modernization components will operate effectively together.
SYSTEM TESTING AND TEST
PLANNING ARE INADEQUATE
---------------------------------------------------------- Chapter 5:4
An organization performs system testing to detect system design and
development errors and correct them before putting a system into
operation. Inadequate testing increases the likelihood that errors
will be undetected, reduces the extent to which a system can provide
accurate and reliable processing services and information, and,
because the discovery of errors is likely to be delayed, increases
the cost of modifying the system.
A testing plan ensures that sufficient testing is done during system
development and prior to deployment. The plan defines, for example,
what is to be accomplished during testing, who is to do the testing,
where it is to be performed, and what constitutes success.
IRS acknowledges the importance of testing in the development of TSM
systems, but has not yet developed a complete and comprehensive
testing plan for TSM. In addition, individual TSM system development
projects are developing their own testing plans. IRS describes these
individual testing plans as rudimentary and inadequate. As a result,
IRS has no assurance that its individual systems will be thoroughly
and consistently tested or that systems will perform correctly or
effectively.
Currently, IRS performs system development testing in an operational
environment using taxpayer data at its service centers or computer
centers. Because tax processing production work at these facilities
has a higher priority than testing, the time, computer, and human
resources applied to testing, as well as the resulting depth of
testing, are limited. This limitation seriously affects testing
quality and completeness. This testing environment also introduces
the possibility that testing can, under unforeseen circumstances,
affect and disrupt production.
To help overcome this situation, IRS plans to establish an
Integration Test and Control Facility to provide an environment that
will more effectively support the testing and integration of legacy
and TSM systems. By establishing this testing facility, IRS expects
to (1) improve the quality of delivered software, (2) provide
information resources needed for testing and integration, and (3)
reduce risks in integrating and transitioning from current legacy
systems to TSM.
In September 1994, IRS developed a concept of operations for the
integrated testing facility, which describes its functions and
responsibilities. IRS has been working with a contractor to define
the facility's functions and responsibilities. IRS is also working
with the General Services Administration to select a facility site.
However, until IRS completes its testing plans, implements effective
testing processes, and establishes its Integration Test and Control
Facility, it has little assurance that systems will be adequately and
effectively tested.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 5:5
To address IRS's technical infrastructure weaknesses, we recommend
that, before December 31, 1995, the IRS Commissioner
complete an integrated systems architecture, including security,
telecommunications, network management, and data management;
institutionalize formal configuration management for all newly
approved projects and upgrades and develop a plan to bring
ongoing projects under formal configuration management;
develop security concept of operations, disaster recovery, and
contingency plans for the modernization vision and ensure that
these requirements are addressed when developing information
system projects;
develop a testing and evaluation master plan for the modernization;
establish an integration testing and control facility; and
complete the modernization integration plan and ensure that
projects are monitored for compliance with modernization
architectures.
Completion of these actions in 1995 is essential so that the
Congress, in carrying out its oversight role and making TSM funding
decisions, has assurance that the government's TSM investment is
adequately protected through effective management of the technical
aspects of tax processing modernization.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 5:6
IRS agreed with our recommendations to improve its systems
architectures, testing, and integration. IRS commented that it is
identifying the necessary actions to ensure that defined systems
development standards and architectures are enforced agencywide. IRS
said also that it
is planning for its 1996 IRS Information System Architecture to
reflect a total system view;
is reviewing existing documentation to determine how best to
incorporate our security architecture recommendation;
is in the process of improving its configuration management process
by implementing change control, as well as developing guidance;
has initiated a series of assessments for major TSM systems to
review and baseline existing requirements for each deliverable,
including documented interfaces;
will merge integration testing, systems testing, and other
testing-related personnel in one facility, and is planning to
establish an interim test and control capability; and
has developed a release engineering approach to transition from its
current environment to one meeting TSM-defined objectives and
capabilities.
We believe that actions to improve TSM's technical infrastructure,
such as those IRS has outlined in its comments, are necessary
prerequisites to adequately develop and implement new systems. In
addition, while release engineering can facilitate the transition
from IRS's current environment to one meeting TSM-defined objectives
and capabilities, to be successful, it must be closely coordinated
with requirements and configuration management.
IRS RECENTLY ACTED TO STRENGTHEN
ACCOUNTABILITY AND RESPONSIBILITY
FOR SYSTEMS MODERNIZATION
============================================================ Chapter 6
Effective overall systems modernization management is important
because TSM is not a one-time, turnkey replacement of all current
subsystems; rather, it is a target system that will be reached by
incrementally upgrading or replacing operational subsystems.
Consequently, to successfully implement IRS's systems modernization,
an organizational structure must be in place to consistently manage
and control all systems development efforts. This organizational
structure would provide accountability and responsibility for all
systems investments, including prioritizing new modernization systems
and upgrades and maintaining all operational systems.
However, below the Commissioner's Office, the management authority
and control needed to modernize tax processing has been fragmented.
Until recently, IRS's Modernization Executive was responsible for
developing TSM information systems until they became operational.
Under this executive, each TSM system was managed by a program
control group that was tasked with reviewing the project, making
milestone decisions, and mitigating project risks.
In addition, the Chief Information Officer was responsible for
developing non-TSM systems and for the operation of all IRS systems.
This included the TSM systems that were developed by the
Modernization Executive and that had been in operation for about 1
year.
In addition to systems development and operations being managed and
controlled by the Modernization Executive and the Chief Information
Officer, several systems development projects were managed and
controlled by IRS's research and development division. For example,
this division's staff of 30 information specialists developed both
Telefile\1 and the Filing Fraud system, which are TSM systems.
Neither the Modernization Executive nor the Chief Information Officer
had decision-making responsibility for these systems or the authority
to ensure compliance with IRS system development standards and
practices.
During our April 28, 1995, meeting with the IRS Commissioner, we
recommended that she establish consolidated, organizationwide control
over all information systems investments, including all new systems
in research and development and operational systems being upgraded
and replaced. In May 1995, the Modernization Executive was named
Associate Commissioner and given responsibility to manage and control
all system development efforts that had previously been the
responsibility of the Modernization Executive and the Chief
Information Officer. However, the research and development division
still does not report to the Associate Commissioner.
It is critical that the Associate Commissioner now establish
organizationwide system modernization accountability and address the
problems this report discusses. This entails
ensuring strategic planning documents are complete and consistent;
developing a comprehensive plan and schedule for linking
reengineering efforts to systems development projects;
exercising consolidated control over all information systems
investments, including all new systems in research and
development and operational systems being upgraded and replaced;
and
ensuring that defined systems development standards and
architectures are enforced.
--------------------
\1 Telefile is a system tax filers can use to input 1040EZ tax return
information through touch-tone phones.
RECOMMENDATION
---------------------------------------------------------- Chapter 6:1
To fully strengthen systems development accountability and
responsibility, we recommend that the IRS Commissioner give the
Associate Commissioner management and control responsibility for all
systems development activities, including those of IRS's research and
development division.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 6:2
In commenting on a draft of this report, IRS reiterated that the
Associate Commissioner is responsible for all aspects of
modernization program planning and management, budget formulation and
execution, and information systems development and management.
Further, IRS said that it was considering whether the Associate
Commissioner's systems development responsibilities are to include
those of the research and development division.
We strongly urge IRS to also place with the Associate Commissioner
accountability and responsibility for the research and development
division's systems development activities. By doing so, IRS will
help to ensure that systems development efforts are consistently
managed and controlled organizationwide.
(See figure in printed edition.)Appendix I
COMMENTS FROM THE INTERNAL REVENUE
SERVICE
============================================================ Chapter 6
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
RELATED GAO PRODUCTS
============================================================ Chapter 1
IRS Automation: Controlling Electronic Filing Fraud and Improper
Access to Taxpayer Data (GAO/T-AIMD/GGD-94-183, July 19, 1994).
Tax Systems Modernization: Automated Underreporter Project Shows
Need for Human Resource Planning (GAO/GGD-94-159, July 8, 1994).
Financial Audit: Examination of IRS' Fiscal Year 1993 Financial
Statements (GAO/AIMD-94-120, June 15, 1994).
Tax Administration: IRS' New Business Vision (GAO/T-GGD-94-58, Nov.
17, 1993).
IRS Information Systems: Weaknesses Increase Risk of Fraud and
Impair Reliability of Management Information (GAO/AIMD-93-34, Sept.
22, 1993).
Financial Management: IRS Lacks Accountability Over Its ADP
Resources (GAO/AIMD-93-24, Aug. 5, 1993).
Tax Systems Modernization: Comments on IRS' Fiscal Year 1994 Budget
Request (GAO/T-IMTEC-93-6, Apr. 27, 1993).
Tax Administration: Achieving Business and Technical Goals in Tax
Systems Modernization (GAO/T-GGD-93-24, Apr. 27, 1993).
Tax Systems Modernization: Program Status and Comments on IRS'
Portion of President's Request for Fiscal Year 1993 Supplemental
Funds (GAO/T-IMTEC-93-3, Mar. 30, 1993).
Tax Administration: Status of Tax Systems Modernization, Tax
Delinquencies, and the Tax Gap (GAO/T-GGD-93-4, Feb. 3, 1993).
Tax Administration: Opportunities to Increase the Use of Electronic
Filing (GAO/GGD-93-40, Jan. 22, 1993).
Tax Administration: IRS Can Improve Controls Over Electronic Filing
Fraud (GAO/GGD-93-27, Dec. 30, 1992).
Tax Systems Modernization: Concerns Over Security and Privacy
Elements of the Systems Architecture (GAO/IMTEC-92-63, Sept. 21,
1992).
TSM: IRS' Key Planning Components (GAO/IMTEC-92-55R, May 22, 1992).
Tax Systems Modernization: Update on Critical Issues Facing IRS
(GAO/T-IMTEC-92-18, May 13, 1992).
Tax Systems Modernization: Input Processing Strategy Is Risky and
Lacks a Sound Analytical Basis (GAO/T-IMTEC-92-15, Apr. 29, 1992).
Tax Systems Modernization: Progress Mixed in Addressing Critical
Success Factors (GAO/T-IMTEC-92-13, Apr. 2, 1992).
Tax Systems Modernization: Factors Critical to Success
(GAO/T-IMTEC-92-10, Mar. 10, 1992).
Tax Systems Modernization: Private Sector Modernization Efforts IRS
May Want to Examine (GAO/GGD-91-133FS, Sept. 24, 1991).
Tax System Modernization: Status of On-Line Files Initiative and
Telecommunications Planning (GAO/IMTEC-91-41FS, Aug. 14, 1991).
Identifying Options for Organizational and Business Changes at IRS
(GAO/T-GGD-91-54, July 9, 1991).
Tax System Modernization: Issues Facing IRS (GAO/T-IMTEC-91-18, July
9, 1991).
Tax System Modernization: An Assessment of IRS' Design Master Plan
(GAO/IMTEC-91-53BR, June 25, 1991).
Tax System Modernization: Attention to Critical Issues Can Bring
Success (GAO/T-IMTEC-91-8, June 25, 1991).
Tax System Modernization: Further Testing of IRS' Automated Taxpayer
Service Systems Is Needed (GAO/IMTEC-91-42, June 20, 1991).
Managing IRS: Important Strides Forward Since 1988 but More Needs to
Be Done (GAO/GGD-91-74, Apr. 29, 1991).
Uncertainties Surrounding IRS' Fiscal Year 1992 Budget Request for
Tax System Modernization (GAO/T-IMTEC-91-4, Mar. 20, 1991).
Tax System Modernization: Status of IRS' Input Processing Initiative
(GAO/IMTEC-91-9, Dec. 12, 1990).
Tax System Modernization: Management Mistakes Caused Delays in
Automated Underreporter System (GAO/IMTEC-90-51, July 10, 1990).
Tax System Modernization: IRS' Efforts to Improve Taxpayer
Correspondence (GAO/IMTEC-90-26, Mar. 22, 1990).
Tax System Modernization: IRS' Challenge for the 21st Century
(GAO/IMTEC-90-13, Feb. 8, 1990).
ADP Modernization: IRS' Automated Examination System--Troubled Past,
Uncertain Future (GAO/IMTEC-89-54, June 22, 1989).
ADP Modernization: IRS Needs to Assess Design Alternatives for Its
Electronic Filing System (GAO/IMTEC-89-33, May 5, 1989).
Managing IRS: Actions Needed to Assure Quality Service in the Future
(GAO/GGD-89-1, Oct. 14, 1988).
ADP Modernization: IRS' Tax System Redesign Progress and Plans for
the Future (GAO/IMTEC-88-23BR, Apr. 27, 1988).
ADP Modernization: IRS' Redesign of Its Tax Administration System
(GAO/IMTEC-88-5FS, Nov. 9, 1987).
*** End of document. ***