Tax Systems Modernization: Actions Underway But IRS Has Not Yet Corrected
Management and Technical Weaknesses (Letter Report, 06/07/96,
GAO/AIMD-96-106).
GAO reported last year that the government's $8 billion tax systems
modernization effort was plagued by pervasive management and technical
weaknesses. Although the Internal Revenue Service (IRS) has taken steps
to overcome these shortcomings, none of these actions, either
individually or collectively, fully satisfy GAO's recommendations, and
it is unclear when these actions will result in disciplined systems
development. The upshot is that IRS continues to spend hundreds of
millions of dollars on tax systems modernization even though fundamental
weaknesses jeopardize that investment. IRS now plans to used a prime
contractor and increase its use of software development contractors to
develop tax systems modernization. However, its plans and schedules in
this area are poorly defined and cannot be fully understood or assessed.
Moreover, as the experience with Cyberfile and the Document Processing
System projects makes clear, IRS does not have the mature processes
needed to acquire software and manage contractors effectively.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-96-106
TITLE: Tax Systems Modernization: Actions Underway But IRS Has Not
Yet Corrected Management and Technical Weaknesses
DATE: 06/07/96
SUBJECT: Tax administration systems
Strategic information systems planning
Computer software verification and validation
Information resources management
Systems conversions
Electronic forms
IDENTIFIER: IRS Tax System Modernization Program
IRS Cyberfile
IRS Service Center Recognition/Image Processing System
IRS Integrated Case Processing System
IRS Corporate Accounts Processing System
IRS Modernization Integration Plan
IRS Document Processing System
Software Capability Maturity Model
TSM
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
June 1996
TAX SYSTEMS MODERNIZATION -
ACTIONS UNDERWAY BUT IRS HAS NOT
YET CORRECTED MANAGEMENT AND
TECHNICAL WEAKNESSES
GAO/AIMD-96-106
Tax Systems Modernization
(511511)
Abbreviations
=============================================================== ABBREV
CAPS - Corporate Accounts Processing System
CMM - Capability Maturity Model
GPMO - Government Program Management Office
ICP - Integrated Case Processing
IRS - Internal Revenue Service
NTIS - National Technical Information Service
OMB - Office of Management and Budget
SLC - Systems Life Cycle
TIPSS - Treasury Information Processing Support Services
TSM - Tax Systems Modernization
Letter
=============================================================== LETTER
B-271523
June 7, 1996
Congressional Requesters
In April 1995 briefings to your Committees and the Internal Revenue
Service (IRS), we reported that the government's efforts to modernize
tax processing by investing what could be more than $8 billion in tax
systems modernization (TSM) were at serious risk due to pervasive
management and technical weaknesses. In our July 1995 report,\1 we
made several recommendations to IRS to correct these weaknesses. In
this regard, GAO recommended, inter alia, that the Commissioner:
-- Formulate a comprehensive business strategy for maximizing
electronic filings by targeting those sectors of the taxpayer
population that can file electronically most cost-beneficially.
-- Take immediate action to improve IRS' strategic information
management by implementing a process for selecting,
prioritizing, controlling, and evaluating the progress and
performance of all major information systems investments, using
explicit decision criteria. Further, by June 30, 1995, IRS
should review all planned and ongoing systems for fiscal year
1996 using these criteria.
-- By December 31, 1995, implement disciplined, consistent
procedures for software requirements management, quality
assurance, configuration management, and project planning and
tracking.
-- By December 31, 1995, complete an integrated systems
architecture and security and data architectures.
The time frames that we recommended coincided with congressional
deliberations on IRS' 1996 and 1997 budget cycles and were intended
to ensure that the Congress could have confidence that IRS could
effectively start managing its fiscal year 1996 TSM budget.
Reflecting continued congressional concern with TSM, the Treasury,
Postal Service, and General Government Appropriations Act of 1996\2
required that the Department of the Treasury provide a report to the
House and Senate Committees on Appropriations (1) identifying,
evaluating, and prioritizing all IRS systems investments planned for
fiscal year 1996, using explicit decision criteria, (2) providing a
schedule for successfully correcting weaknesses that we identified in
April 1995, (3) presenting a milestone schedule for developing and
implementing all projects included in the tax systems modernization
program, and (4) presenting a plan to expand the utilization of
external expertise for systems development and total program
integration. The conference report on the Act\3 directed that we
review the Department of the Treasury report on IRS and, within 30
days of receipt of the Treasury report, provide your Committees our
independent assessment of the actions taken by IRS to correct the
weaknesses identified in our July 1995 report.
In March 1996 testimony to the Subcommittee on the Treasury, Postal
Service, and General Government of the Committee on Appropriations,
House of Representatives, we provided a progress report on actions
taken by IRS to address our recommendations.\4 On May 9, 1996, we
received copies of the final report on TSM from the Department of the
Treasury.\5 As directed in the conference report on the Treasury,
Postal Service, and General Government Appropriations Act of 1996, we
have assessed IRS actions to correct its management and technical
weaknesses, as delineated in Treasury's report on TSM, and are
reporting our findings to you by this letter.
--------------------
\1 Tax Systems Modernization: Management and Technical Weaknesses
Must Be Corrected If Modernization Is To Succeed (GAO/AIMD-95-156,
July 26, 1995).
\2 Public Law 104-52, 11-19-95.
\3 H.R. Report No. 291, 104th Cong., 1st Session (1995).
\4 Status of Tax Systems Modernization, Tax Delinquencies, and the
Potential for Return-Free Filing (GAO/T-GGD/AIMD-96-88, March 14,
1996).
\5 Report to House and Senate Appropriations Committees: Progress
Report on IRS's Management and Implementation of Tax Systems
Modernization (Department of the Treasury, May 6, 1996).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
The May 1996 Treasury report delineates, and we verified, that IRS
has initiated a number of actions and is making some progress in
addressing our recommendations. For example, IRS (1) is preparing a
comprehensive strategy to maximize electronic filing; (2) has created
an investment review board to select, control, and evaluate its
information technology investments; (3) has updated its system
engineering process, is updating its systems life cycle methodology,
and is working across various IRS organizations to define disciplined
processes for software requirements management, quality assurance,
configuration management, and project planning and tracking; and (4)
has completed a descriptive overview of an integrated, three-tier,
distributed systems architecture.
However, many of these actions are still incomplete and do not
respond fully to any of our recommendations. For example,
-- the comprehensive business strategy for electronic filing is not
scheduled for completion until August 1996;
-- IRS does not yet have a repeatable process for selecting,
controlling, and evaluating its technology investments, and all
planned and ongoing systems have not been reviewed in a single
investment portfolio, and the basis for making investment
decisions is still unclear;
-- the procedures for requirements management, quality assurance,
configuration management, and project planning and tracking are
being developed, but are still incomplete; and
-- IRS has not completed its integrated systems architecture nor
its security and data architectures, and there is no schedule
for doing so.
As a result, IRS has not made adequate progress in correcting its
management and technical weaknesses, and none of our recommendations
have been fully implemented. Until IRS' weaknesses are corrected, we
believe the Congress should consider limiting TSM spending to only
cost-effective modernization efforts that (1) support ongoing
operations and maintenance, (2) correct IRS' pervasive management and
technical weaknesses, (3) are small, represent low technical risk,
and can be delivered in a relatively short time frame, and (4)
involve deploying already developed systems, only if these systems
have been fully tested, are not premature given the lack of a
completed architecture, and produce a proven, verifiable business
value.
In its report, Treasury states that because IRS does not currently
have the capability to develop and integrate TSM, it will obtain
additional contractual help to accomplish these tasks. While
effective use of additional contractors may, in the future,
strengthen IRS' capability to modernize, it is clear that IRS does
not now have the capability to manage all of its current contractors
successfully. For example, Cyberfile, which is being built by
contractors, has used many of the same undisciplined software
development practices we had criticized at IRS, and, as a result,
could not be piloted during the 1996 tax filing season as originally
planned.\6 Therefore, if IRS is to use additional contractors
effectively in the future, it will have to first strengthen and
improve its ability to manage software development contractors.
--------------------
\6 Tax Systems Modernization: Management and Technical Weaknesses
Must Be Overcome To Achieve Success (GAO/T-AIMD-96-75, March 26,
1996).
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :2
We reviewed the Department of the Treasury's report, interviewed
senior IRS officials responsible for the actions being taken to
correct the management and technical weaknesses, and reviewed
documentation. On June 4, 1996, we briefed senior Treasury and IRS
officials, including the Deputy Secretary of the Treasury and the
Commissioner of the IRS, on the results of our review.
We performed our work at IRS headquarters in Washington, D.C.,
between May 9, 1996 and June 4, 1996 in accordance with generally
accepted government auditing standards. The Department of the
Treasury and IRS provided comments on a draft of this report, which
are discussed in the "Agency Comments and Our Evaluation" section and
are reprinted in appendix I.
BACKGROUND
------------------------------------------------------------ Letter :3
IRS envisions a modernized tax processing environment which is
virtually paper free and in which taxpayer information is readily
available to IRS employees to update taxpayer accounts and respond to
taxpayer inquiries. In our July 1995 report, we emphasized the need
for IRS to have in place sound management and technical practices to
increase the likelihood that TSM's objectives will be
cost-effectively and expeditiously met.\7 A 1996 National Research
Council report on TSM had a similar message.\8 Its recommendations
parallel the over a dozen recommendations we made in July 1995 to
improve IRS' (1) business strategy to reduce reliance on paper, (2)
strategic information management practices, (3) software development
capabilities, (4) technical infrastructures, and (5) organizational
controls.
In the July 1995 report, we described our methodology for analyzing
IRS' strategic information management practices, drawing heavily from
our research on the best practices of private and public sector
organizations that have been successful in improving their
performance through strategic information management and technology.
These fundamental best practices are discussed in our report
Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology (GAO/AIMD-94-115, May 1994),
and our Strategic Information Management (SIM) Self-Assessment
Toolkit (GAO/Version 1.0, October 28, 1994, exposure draft).
To evaluate IRS' software development capability, we validated IRS'
September 1993 assessment of its software development maturity based
on the Capability Maturity Model (CMM) developed by Carnegie Mellon
University's Software Engineering Institute, a nationally recognized
authority in the area. This model establishes standards in key
software development process areas (i.e., requirements management,
project planning, project tracking and oversight, configuration
management, quality assurance, and subcontractor management) and
provides a framework to evaluate a software organization's capability
to consistently and predictably produce high-quality products.
When we briefed the IRS Commissioner in April 1995 and issued our
report documenting its weaknesses in July 1995, IRS agreed with our
recommendations to make corrections expeditiously. At that time, we
considered IRS' response to be a commitment to correct its management
and technical weaknesses.
In September 1995, IRS submitted an action plan to the Congress
explaining how it planned to address our recommendations. In our
March 1996 testimony\9 to the House Appropriation Committee's
Subcommittee on Treasury, Postal Service, and General Government, we
noted that this plan, follow-up meetings with senior IRS officials,
and other draft and "preliminary draft" documents received through
early March 1996 provided little tangible evidence that actions being
taken would correct the pervasive management and technical weaknesses
that continued to place TSM, and the huge investment it represents,
at risk. This interim status report on IRS' efforts to respond to
our July 1995 recommendations noted that IRS had initiated a number
of activities and made some progress in addressing our
recommendations to improve management of information systems; enhance
its software development capability; and better define, perform, and
manage TSM's technical activities. However, we reported that none of
these steps had fully satisfied any of our recommendations.
Consequently, IRS was not in an appreciably better position in March
1996 than it was in April 1995 to assure the Congress that it would
spend its fiscal year 1996 and future TSM appropriations judiciously
and effectively.
In a subsequent testimony before the Senate Committee on Governmental
Affairs,\10 we reiterated our concerns that IRS' effort to modernize
tax processing was jeopardized by persistent and pervasive management
and technical weaknesses, and that ongoing efforts did not include
milestones or provide enough evidence to conclude that weaknesses
will soon be corrected. We also addressed analogous technical
weaknesses in an electronic filing system project called Cyberfile
which substantiated our concerns that IRS was continuing to risk
millions of dollars in undisciplined systems development in fiscal
year 1996.\11 In addition, we identified physical security risks at
the planned Cyberfile data center.\12
--------------------
\7 GAO/AIMD-95-156, July 26, 1995.
\8 Continued Review of the Tax Systems Modernization of the Internal
Revenue Service--Final Report, Computer Science and
Telecommunications Board, National Research Council, 1996.
\9 GAO/T-GGD/AIMD-96-88, March 14, 1996.
\10 GAO/T-AIMD-96-75, March 26, 1996.
\11 Cyberfile, which is planned to allow taxpayers to submit their
returns electronically from personal computers, is being developed
for the IRS by the Department of Commerce's National Technical
Information Service.
\12 Security Weaknesses at IRS' Cyberfile Data Center
(GAO/AIMD-96-85R, May 9, 1996).
TREASURY DEPARTMENT'S TSM
REPORT ACKNOWLEDGES WEAKNESSES
AND DESCRIBES REDIRECTION
EFFORTS
------------------------------------------------------------ Letter :4
The Department of the Treasury, in its May 1996 report to the Senate
and House Appropriations Committees, provides a candid assessment of
TSM progress and future redirection, and a description of ongoing and
planned actions intended to respond to our recommendations to correct
management and technical weaknesses. It finds that despite some
qualified success, IRS has not made progress on TSM as planned
because systems development efforts have taken longer than expected,
cost more than originally estimated, and delivered less functionality
than originally envisioned. It concludes that significant changes
are needed in IRS' management approach, and that it is beyond the
scope of IRS' current ability to develop and integrate TSM without
expanded use of external expertise.
The report notes that work has been done to rethink, scale back, and
change the direction of TSM. Additional changes are still in
progress with actions underway to restructure the management of TSM
and expand the use of contractors. Agreeing that our July 1995
recommendations are valid, the report notes that more work has to be
done to respond to our recommendations. It states that progress in
IRS' management and technical areas can only be achieved by
institutionalizing improved practices and monitoring projects for
conformance to mandated standards and practices.
The report does not address the basic problem of continuing to invest
hundreds of millions of dollars in TSM before the requisite
management and technical disciplines are in place. Neither does it
address the risk inherent in shifting hundreds of millions of dollars
to additional contractual efforts when the evidence is clear that IRS
does not have the disciplined processes in place to manage all of its
current contractual efforts (e.g., Cyberfile) effectively.
FUNDAMENTAL MANAGEMENT AND
TECHNICAL WEAKNESSES ARE BEING
ADDRESSED, BUT NONE HAVE BEEN
FULLY CORRECTED
------------------------------------------------------------ Letter :5
IRS has initiated a number of actions to address management and
technical weaknesses that continue to impede successful systems
modernization. However, ongoing efforts do not correct the
weaknesses and do not provide enough evidence to determine when they
will be corrected and what steps if any are being taken in the
interim to mitigate the risks associated with ongoing TSM spending.
IRS DOES NOT YET HAVE A
COMPREHENSIVE STRATEGY TO
MAXIMIZE ELECTRONIC FILINGS
---------------------------------------------------------- Letter :5.1
IRS has identified increasing electronic filings as critical to
achieving its modernization vision. We noted that IRS did not have a
comprehensive business strategy to reach or exceed its electronic
filing goal, which was 80 million electronic filings by 2001. IRS'
estimates and projections for individual and business returns
suggested that, by 2001, as few as 39 million returns may be
submitted electronically, less than half of IRS' goal and only about
17 percent of all returns expected to be filed.
We reported that IRS' business strategy would not maximize electronic
filings because it primarily targeted taxpayers who use a third party
to prepare and/or transmit simple returns, are willing to pay a fee
to file their returns electronically, and are expecting refunds.
Focusing on this limited taxpaying population overlooked most
taxpayers, including those who prepare their own tax returns using
personal computers, have more complicated returns, owe tax balances,
and/or are unwilling to pay a fee to a third party to file a return
electronically.
We concluded that, without a strategy that also targets these
taxpayers, IRS would not meet its electronic filing goals. In
addition, if, in the future, taxpayers file more paper returns than
IRS expects, added stress will be placed on IRS' paper-based systems.
Accordingly, we recommended that IRS
refocus its electronic filing business strategy to target, through
aggressive marketing and education, those sectors of the taxpaying
population that can file electronically most cost-beneficially.
IRS agreed with this recommendation and said that it had convened a
working group to develop a detailed, comprehensive strategy to
broaden public access to electronic filing, while also providing more
incentives for practitioners and the public to file electronically.
It said that the strategy would include approaches for taxpayers who
are unwilling to pay for tax preparer and transmitter services, who
owe IRS for balances due, and/or who file complex tax returns. IRS
said further that the strategy would address that segment of the
taxpaying population that would prefer to file from home, using
personal computers.
To date, IRS has performed an electronic filing marketing analysis at
local levels; developed a marketing plan to promote electronic
filing; consolidated 21 electronic filing initiatives into its
Electronic Filing Strategies portfolio; and initiated a reengineering
project with a goal to reduce paper tax return filings to 20 percent
or less of the total volume by the year 2000. It plans to complete
its electronic filing strategy in August 1996. These initiatives
could result in future progress toward increasing electronic filings.
However, our review found that these initiatives are not far enough
along to determine whether they will culminate in a comprehensive
strategy that identifies how IRS plans to target those sectors of the
taxpaying population that can file electronically most
cost-beneficially. It also is not clear how the reengineering
project will impact the strategy or how these initiatives will impact
TSM systems that are being developed.
IRS' STRATEGIC INFORMATION
MANAGEMENT PRACTICES REMAIN
INEFFECTIVE
---------------------------------------------------------- Letter :5.2
We reported that IRS did not have strategic information management
practices in place. We found, for example, that, despite the
billions of dollars at stake, information systems were not managed as
investments. To overcome this, and provide the Congress with insight
needed to assess IRS' priorities and rationalization for TSM
projects, we recommended that the IRS Commissioner
take immediate action to implement a complete process for selecting,
prioritizing, controlling, and evaluating the progress and
performance of all major information systems investments, both new
and ongoing, including explicit decision criteria, and
using these criteria, to review all planned and ongoing systems
investments by June 30, 1995.
In agreeing with these recommendations, IRS said it would take a
number of actions to provide the underpinning it needs for strategic
information management. IRS said, for example, that it was
developing and implementing a process to select, prioritize, control,
and evaluate information technology investments to achieve
reengineered program missions.
Our assessment found that IRS has taken steps towards putting into
place a process for managing its extensive investments in information
systems. Following are examples of these steps.
-- IRS created the executive-level Investment Review Board, chaired
by the Associate Commissioner for Modernization, for selecting,
controlling and evaluating all of IRS' information technology
investments.
-- IRS developed initial and revised sets of decision criteria used
last summer and again in November 1995 as part of its Resource
Allocation and Investment Review to make additional changes in
information technology resource allocations for remaining fiscal
year 1996 funds and planned fiscal year 1997 spending. This
review included only TSM projects under development. It did not
address operational systems, infrastructure, or management and
technical support activities.
-- The Treasury Department created a Modernization Management Board
to review and validate high-risk, high-cost TSM investments and
to set policy and strategy for IRS modernization effort.
-- IRS is considering the use of a "project readiness review" as an
additional Investment Review Board control mechanism for gauging
project readiness to proceed with spending.
-- IRS developed the Business Case Handbook that includes decision
criteria on costs, benefits, and risks. It is reassessing the
business cases, which were developed on the TSM projects, using
the handbook. Eleven cases are scheduled for completion in June
1996, and IRS plans to have the remaining cases completed by
September 1996. Results are planned to be presented to the
Investment Review Board to assist in making funding decisions
for fiscal year 1997.
-- IRS has developed the Investment Evaluation Review Handbook
designed to assess projected costs and benefits against actual
results. The handbook has been used on four TSM projects and
five additional reviews are scheduled to be completed within the
next year. The completed reviews contain explicit descriptions
of problems encountered in developing these systems. The
reviews make specific recommendations for management and
technical process changes to improve future results. Specific
recommendations pertain to strengthening project direction and
decision-making. Many reflect concerns that we have raised in
past reviews. The investment evaluation reviews were presented
to the Investment Review Board and disseminated to other IRS
managers. IRS is defining roles, responsibilities, and
processes for incorporating Investment Evaluation Review
recommendations at the project and process levels.
These are positive steps and indicate a willingness to address many
of the weaknesses raised in our past reports and testimonies. But,
as noted in Treasury's report on TSM, the investment process is not
yet complete. According to Treasury, it is missing (1) specific
operating procedures, (2) defined reporting relationships between
different management boards and committees, and (3) updated business
cases for major TSM technology investments.
These concerns coincide with two central criticisms we have
repeatedly made about TSM. Because of the sheer size, scope, and
complexity of TSM, it is imperative that IRS institutionalize a
repeatable process for selecting, controlling, and evaluating its
technology investments, and that it make informed investment
decisions based on reliable qualitative and quantitative assessments
of costs, benefits, and risks. Although IRS is planning and in the
initial stages of implementing parts of such a process, a complete,
fully-integrated process does not yet exist. Specifically, IRS has
not provided us evidence to justify its claims that its decisions
were supported by acceptable data on project costs, benefits, and
risks. For example:
-- Our review found no evidence to suggest that IRS established
minimal data requirements for the decisions made as part of the
TSM Resource Allocation and Investment Review or the rescope
process in December 1995. For example, because IRS lacks the
basic capabilities for disciplined software development, it
cannot convincingly estimate systems development costs,
schedule, or performance. Subsequent to its rescope analysis,
IRS developed minimal data quality requirements for cost-benefit
and risk studies, proposed return on investment calculations,
and return on investment thresholds, or comparisons of expected
performance improvements with results to date. However, to
date, few, if any projects have met these criteria.
-- In deciding whether to accelerate, delay, or cancel specific TSM
projects, IRS did not use validated data on actual versus
projected costs, benefits, or risks as set forth by the Office
of Management and Budget (OMB).\13 Instead, IRS continues to
make its decisions based on spending whatever budgeted funding
ceiling amounts can be obtained through its annual budget and
appropriations cycles. As a result, IRS cannot convincingly
justify its TSM spending decisions.
-- All projects (i.e., proposed projects, projects under
development, operational systems, infrastructure, and management
and technical support activities) were not included in a single
systems investment portfolio. Instead, only TSM projects under
development were ranked. As a result, there is no compelling
rationale for determining how much to invest in these projects
compared to other projects, such as operational systems,
infrastructure, etc.
-- There is no defined process with prescribed roles and
responsibilities to ensure that the results of investment
evaluation reviews are being used to (1) modify project
direction and funding when appropriate and (2) assess and
improve existing investment selection and control processes and
procedures. As a result, there is no evidence that changes are
occurring based on the valuable lessons learned as in the
recently completed post implementation review of the Service
Center Recognition/Image Processing System. For example, IRS
found that because system requirements were not adequately
defined or documented, the system could not be quantifiably
tested properly which adversely affected the implementation of
the system. Moreover, with only four investment evaluation
reviews completed to date and five planned for the upcoming
year, this represents only a small fraction of the total IRS
annual investment in TSM. More must be done to confirm actual
results achieved from TSM expenditures.
--------------------
\13 Evaluating Information Technology Investments: A Practical Guide
(Executive Office of the President, Office of Management and Budget,
November 1995.)
REENGINEERING EFFORTS NOT
LINKED TO MODERNIZATION
---------------------------------------------------------- Letter :5.3
We noted in our July 1995 report that IRS' reengineering efforts were
not linked to its systems development efforts. As shown in our work
with leading organizations, information system development projects
that are not driven by a critical reexamination and redesign of
business processes achieve only a fraction of their potential to
improve performance, reduce costs, and enhance quality.
Since our July report, IRS' reengineering efforts have undergone a
redirection. Three reengineering projects--processing returns,
responding to taxpayers, and enforcement actions--were halted because
IRS decided to focus instead on an enterprise-level view of
reengineering.
Its new effort, entitled Tax Settlement Reengineering, was begun in
March 1996 and involves a comprehensive review of all the major
processes and activities that enable taxpayers to settle their tax
obligations, from educational activities through final settlement of
accounts. The reengineering project team, working with IRS'
Executive Committee, has identified 16 major processes involved in
tax settlement and is about to begin reengineering four of them.
High-level designs of the new processes are scheduled to be defined
by September 30, 1996, with work on detailed designs to start early
in fiscal year 1997, if approved by the Executive Committee.
Reengineering efforts on as many as eight other tax settlement
processes could be underway by the end of fiscal year 1997.
Although this effort could have substantial impact, IRS still faces
the same problem we reported on a year ago. Reengineering lags well
behind the development of TSM projects, whereas it should be ahead of
it--defining and directing the technology investments needed to
support new, more efficient business processes. Until the
reengineering effort is mature enough to drive TSM projects, there is
no assurance that ongoing systems development efforts will support
IRS' future business needs and objectives.
The reengineering team believes that by September 1996 they will have
a general idea of how the first four tax settlement reengineering
projects may impact current system development efforts. If
additional reengineering projects are started as planned in 1997, it
could be another year or more before most of the information and
systems requirements stemming from these projects are defined.
Meanwhile, investment continues in many TSM projects that may not
support the requirements resulting from these reengineering efforts.
IRS acknowledges that integration of reengineering and TSM must
occur, and has assigned responsibility for it to the Associate
Commissioner for Modernization, but has not yet specified how or when
the requisite integration will occur.
SOFTWARE DEVELOPMENT
ACTIVITIES ARE INCONSISTENT
AND POORLY CONTROLLED
---------------------------------------------------------- Letter :5.4
We reported that unless IRS improves its software development
capability, it is unlikely to build TSM timely or economically, and
systems are unlikely to perform as intended. To assess its software
capability, in September 1993, IRS rated itself using the Software
Engineering Institute's CMM. IRS placed its software development
capability at the lowest level, described as ad hoc and sometimes
chaotic and indicating significant weaknesses in its software
development capability. Our review confirmed that IRS' software
development capability was immature and was weak in key process
areas. For instance,
-- a disciplined process to manage system requirements was not
being applied to TSM systems,
-- a software tool for planning and tracking development projects
was not consistently used,
-- software quality assurance functions were not well defined or
consistently implemented,
-- systems and acceptance testing were neither well defined nor
required, and
-- software configuration management\14 was incomplete.
To address IRS' software development weaknesses and upgrade IRS'
software development capabilities, we recommended that the IRS
Commissioner
immediately require that all future contractors who develop software
for the agency have a software development capability rating of at
least CMM Level 2,\15 and
before December 31, 1995,
define, implement, and enforce a consistent set of requirements
management procedures for all TSM projects that goes beyond IRS'
current request for information services process, and for software
quality assurance, software configuration management, and project
planning and tracking; and
define and implement a set of software development metrics to measure
software attributes related to business goals.
IRS agreed with these recommendations and said that it was committed
to developing consistent procedures addressing requirements
management, software quality assurance, software configuration
management, and project planning and tracking. It also said that it
was developing a comprehensive measurement plan to link process
outputs to external requirements, corporate goals, and recognized
industry standards.
Specifically regarding the first recommendation, IRS has (1)
developed standard wording for use in new and existing contracts that
have a significant software development component, requiring that all
software development be done by an organization that is at CMM Level
2, (2) developed a plan for achieving CMM Level 2 capability on all
of its contracts, and (3) started to implement a plan to monitor
contractors' capabilities, which may include the use of CMM-based
software capability evaluations. The Department of the Treasury
report also noted that a schedule for conducting software capability
evaluations was developed. However, we found that IRS does not yet
have the disciplined processes in place to ensure that all
contractors are performing at CMM Level 2. For example, contractors
developing the Cyberfile electronic filing system were not using CMM
Level 2 processes, subsequent to our July 1995 recommendation.
Further, no schedule for conducting software capability evaluations
has yet been developed.
With respect to the second recommendation, IRS is updating its
systems life cycle (SLC) methodology. The SLC is planned to have
details for systems engineering and software development processes,
including all CMM key process areas. IRS has updated its systems
engineering process to include guidance for defining and analyzing
systems requirements and for preparing work packages. Furthermore,
IRS has drafted handbooks providing guidance to audit and verify
developmental processes. In addition, IRS has developed a
configuration management plan template, updated its requirements
management request for information services\16 document, and
developed and implemented a requirements management course. The
Department of the Treasury also reported that IRS is testing the SLC
on two TSM efforts, Integrated Case Processing (ICP) and Corporate
Accounts Processing System (CAPS). IRS also has a CMM process
improvement plan and work is being done across various IRS
organizations to define processes to meet CMM Level 2. Finally, IRS
is assessing its capabilities to manage contractors using the CMM
goals.
However, the procedures for requirements management, software quality
assurance, software configuration management, and project planning
and tracking are still not complete. A software development life
cycle implementation project, which is to include these procedures,
is not scheduled for completion until September 30, 1996. In
addition, software quality assurance and configuration management
plans for two ICP projects\17 were not being used, and the groups
developing software for CAPS do not have a software configuration
management plan or a schedule for its development. Furthermore, ICP
and CAPS development is continuing without the guidelines and
procedures for other process areas (e.g., requirements management,
project planning, and project tracking and oversight) required by CMM
Level 2.
Regarding the third recommendation, IRS has a three-phase process to
(1) identify data sources for metrics, (2) define metrics to be used,
and (3) implement the metrics. A partial set of metrics is currently
being identified. Initial use of these metrics--populated with real
data and in a preliminary format--is scheduled for use on a set of
identified projects beginning on June 30, 1996. Data sources for
these metrics have been identified and weaknesses (such as
difficulties in retrieving the data and inconsistencies in the data)
are being documented to provide feedback to various systems' owners.
However, this initial set of metrics is incomplete. It focuses on
areas such as time reporting, project sizing, and defect tracing and
analysis, but does not include measures for determining customer
satisfaction and cost estimation. Such measures are needed to
adequately track the needed functionality with associated costs
throughout systems development. Further, there is no schedule for
completing the definition of metrics or for institutionalizing the
processes needed to ensure their use. Finally, there is no mechanism
in place to correct identified data and data collection weaknesses.
In summary, although IRS has begun to act on our recommendations,
these actions are not yet complete or institutionalized, and, as a
result, systems are still being developed without the disciplined
practices and metrics needed to give management assurance that they
will perform as intended.
--------------------
\14 Configuration management involves selecting project baseline
items (e.g., specifications), systematically controlling these items
and changes to them, and recording their status and changes.
\15 The Software Engineering Institute at Carnegie Mellon University
has developed a model, the Software Capability Maturity Model (CMM),
to evaluate an organization's software development capability. CMM
Level 2 denotes that basic project management processes are
established to track cost, schedule, and functionality and the
necessary process discipline is in place to repeat earlier successes
on similar projects.
\16 A request for information services is a process to request
changes to IRS' computer systems. This process provides a way to
request, control, monitor, and track changes to IRS' computer
systems.
\17 The two projects are the Case Processing System and the Case
Inventory Management System.
SYSTEMS ARCHITECTURES,
INTEGRATION, AND TESTING ARE
INCOMPLETE
---------------------------------------------------------- Letter :5.5
We reported that IRS' systems architectures,\18 integration planning,
and system testing and test planning were incomplete.
To address IRS' technical infrastructure weaknesses, we recommended
that the IRS Commissioner before December 31, 1995,
complete an integrated systems architecture, including security,
telecommunications, network management, and data management;
institutionalize formal configuration management for all newly
approved projects and upgrades and develop a plan to bring ongoing
projects under formal configuration management;
develop security concept of operations, disaster recovery, and
contingency plans for the modernization vision and ensure that these
requirements are addressed when developing information system
projects;
develop a testing and evaluation master plan for the modernization;
establish an integration testing and control facility; and
complete the modernization integration plan and ensure that projects
are monitored for compliance with modernization architectures.
IRS agreed with these recommendations and said that it was
identifying the necessary actions to define and enforce systems
development standards and architectures agencywide. IRS' current
efforts in this area follow.
-- In April 1996, IRS completed a descriptive overview of its
integrated three-tier, distributed systems architecture to
provide management with a high-level view of TSM's
infrastructure and supporting systems. IRS has tasked the
integration support contractor to develop the data and security
architectures.
-- IRS has adopted an accepted industry standard for configuration
management. It developed and distributed its Configuration
Management Plan template, which identifies the elements needed
when constructing a configuration management plan. In April
1996, enterprisewide configuration management policies and
procedures were established. IRS also plans to obtain
contractor support to develop, implement, and maintain a
vigorous configuration management program.
-- IRS has prepared a security concept of operations and a disaster
recovery and contingency plan.
-- IRS has developed a test and evaluation master plan for TSM.
IRS plans to develop implementation and enforcement policies for
the plan.
-- IRS has established an interim integration testing and control
facility, which is currently being used to test new software
releases. It is also planning a permanent integration testing
and control facility, scheduled to be completed by December
1996.
-- IRS has completed drafts of its TSM Release Definition Document,
which is planned to provide definitions for new versions of TSM
software from 1997 to 1999, and Modernization Integration Plan,
which is planned to define IRS' process for integrating current
and future TSM initiatives.
These activities start to address our recommendations, but do not
fully satisfy any of them. Specifically:
-- IRS has not completed its integrated systems architecture (the
"blueprints" of TSM), and has not committed to a completion
date. Its completed high-level overview was not intended to,
and does not, provide the level of detail needed to provide
effective guidance to design and build systems. For example,
IRS' concept of a three-tier, distributed architecture has not
been delineated to the level needed to provide sufficient detail
to understand the security requirements and implications. It
does not, for instance, specify what security mechanisms are to
be implemented between and among the three tiers to ensure that
only properly authorized users are allowed to access tax
processing application software and taxpayer data. IRS is using
contractors to complete its security and data architectures, but
has not committed to a completion date. Meanwhile, IRS is
investing in building TSM systems without the "blueprints" that
are needed.
-- IRS has not yet brought its development, acceptance, and
production environments under configuration management control.
For example, there is no disciplined process for moving software
from the test to the production environment. Additionally,
although directives have been distributed to follow various TSM
systems development standards, no enforcement mechanisms are in
place.
-- Our review of the security concept of operations found that the
document does not identify selected security methods and
techniques. For example, it discusses two methods for providing
identification and authentication for controlling user access to
various systems without specifying which method should be used.
The security concept of operations is also sometimes
inconsistent with the security mechanisms currently being
implemented on systems now being developed and does not indicate
how, when, or if these inconsistencies will be resolved. The
specific methods and techniques are currently planned to be
provided in different versions of a planned technical concept of
operations. The first version is currently planned to be
completed in January 1997.
-- IRS' disaster recovery and contingency plan is a high-level
document for planning that presents basic tenets for information
technology disaster recovery but not the detail needed to
provide useful guidance in emergencies. For example, it does
not explain the steps that computing centers need to take to
absorb the workload of a center that suffers a disaster.
-- The test and evaluation master plan provides the guidance needed
to ensure sufficient developmental and operational testing of
TSM. However, it does not describe what security testing should
be performed, or how these tests should be conducted. Further,
it does not specify the responsibilities and processes for
documenting, monitoring, and correcting testing and integration
errors.
-- IRS is still working on plans for its integration testing and
control facility. In the interim, it has established a
temporary facility which is being used for limited testing. The
permanent facility is not currently being planned to simulate
the complete production environment, and will not, for example,
include mainframe computers. Instead, IRS plans to continue to
test mainframe computer software and systems which interface
with the mainframes in its production environment. To ensure
that IRS does not put operations and service to taxpayers at
risk, IRS should prepare a thorough assessment of its solution,
including an analysis of alternative testing approaches and
their costs, benefits, and risks.
-- IRS' draft TSM Release Definition Document and draft
Modernization Integration Plan (1) do not reflect TSM rescoping
and the information systems reorganization under the Associate
Commissioner, (2) do not provide clear and concise links\20 to
other key documents (e.g., its integrated systems architecture,
business master plan, concept of operations, and budget), and
(3) assume that IRS has critical processes in place that are not
implemented (e.g., effective quality assurance and disciplined
configuration management).
In summary, although IRS has taken actions to prepare a systems
architecture and improve its integration and system testing and test
planning, these efforts are not yet complete or institutionalized,
and, as a result, TSM systems continue to be developed without the
detailed architectures and discipline needed to ensure success.
--------------------
\18 A system architecture is an evolving description of an approach
to achieving a desired mission. It describes (1) all functional
activities to be performed to achieve the desired mission, (2) the
system elements needed to perform the functions, (3) the designation
of performance levels of those system elements, and (4) the
technologies, interfaces, and location of functions.
\20 For example, it is not clear how particular software releases are
tied to business master plan goals and objectives and to the
integrated transition plan and schedule's products and services.
Without these links, the documents do not provide important
information on how much will be done by each release, in what period
of time, and at what cost.
NO SINGLE IRS ENTITY
CONTROLS ALL INFORMATION
SYSTEMS EFFORTS
---------------------------------------------------------- Letter :5.6
We reported that IRS had not established an effective organizational
structure to consistently manage and control systems modernization
organizationwide. The accountability and responsibility for IRS'
systems development was spread among IRS' Modernization Executive,
Chief Information Officer, and research and development division. To
help address this concern, in May 1995, the Modernization Executive
was named Associate Commissioner. The Associate Commissioner was to
manage and control systems development efforts previously conducted
by the Modernization Executive and the Chief Information Officer.
In September 1995, the Associate Commissioner for Modernization
assumed responsibility for the formulation, allocation, and
management of all information systems resources for both TSM and
non-TSM expenditures. In February 1996, IRS issued a Memorandum of
Understanding providing guidance for initiating and conducting
technology research and for transitioning technology research
initiatives into system development projects.
It is important that IRS maintain an organizationwide focus to manage
and control all new modernization systems and all upgrades and
replacements of operational systems throughout IRS. To do so, we
recommended that the IRS Commissioner
give the Associate Commissioner management and control responsibility
for all systems development activities, including those of IRS'
research and development division.
Steps are being taken by the Associate Commissioner to establish
effective management and control of systems development activities
throughout IRS. For example, its SLC methodology is required for
information systems development, and information technology entities
throughout the agency have been directed to submit documentation on
all information technology projects for review. However, there is no
defined and effective mechanism for enforcing the standards or
ensuring that organizational entities cannot conduct systems
development activities outside the control of the Associate
Commissioner. Further, no timeframes have been established for
defining and implementing such control mechanisms. As a result,
systems development conducted by the research and development
division has now been redefined as technology research, keeping it
from the control of the Associate Commissioner.
In summary, although improvements have been made in consolidating
management control over systems development, the Associate
Commissioner still does not yet have control over all IRS' systems
development activities.
PLANS MUST BE DEFINED AND
CAPABILITIES STRENGTHENED
BEFORE OBTAINING ADDITIONAL
CONTRACTUAL SUPPORT
------------------------------------------------------------ Letter :6
IRS plans to increase its reliance on the private sector by (1)
preparing an acquisition plan and statement of work to conduct an
expedited competitive selection for a prime development and
integration contractor; (2) transferring responsibility for systems
engineering, design, prototyping, and integration for core elements
of TSM to its integration support contractor; and (3) making greater
use of software development contractors, including those available
under the Treasury Information Processing Support Services (TIPSS),
to develop and deliver major elements of production TSM systems. By
increasing its reliance on contractors, IRS expects to improve the
accountability for and probability of TSM success.
IRS plans to increase the use of private-sector integration and
development expertise by expanding the use of contractors to support
TSM. It outlined a three-track approach for transitioning over a
period of 2 years to the use of a prime contractor that would have,
according to IRS, overall authority and responsibility for the
development, delivery, and deployment of modernized information
systems.
To facilitate this strategy, IRS reported it would consolidate the
management of all TSM resources, including key TSM contractors, in
its Government Program Management Office (GPMO). Under the direct
control of the Chief Information Officer, GPMO will be delegated
authority for the management and control of the IRS staff and
contractors that plan, design, develop, test, and implement TSM
components. IRS plans to have GPMO fully staffed and operational by
October 1, 1996. IRS representatives told us the agency was
currently developing a detailed contract management plan and a
statement of work for acquiring its prime contractor, and believed it
could award a contract in about 2 years.
IRS' approach to expanding the use of contractors to build TSM is
still in the early planning stages. Because of this, IRS was unable
to provide us with formal plans, charters, schedules, or the
definitions of shared responsibilities between GPMO and the existing
program and project management staff.
At this point, it is unclear what these IRS planned actions entail,
or how they will work. For example, IRS has not specified how and
when it plans to transfer its development activities to contractors,
and to what extent contractors could be held responsible for existing
problems in these government-initiated systems. This is particularly
important because if IRS continues as planned, the principal TSM
systems will be in development and/or deployed before IRS plans to
select a prime contractor in about 2 years. Moreover, it is not
clear how the prime contractor would direct potential competitors
that are already under contract with IRS. Without further
explanation of and a schedule for transitioning specific
responsibilities from IRS to contractors, we cannot fully understand
or assess IRS' plans.
Further, plans to use additional contractors will succeed if, and
only if, IRS has the in-house capabilities to manage these
contractors effectively. In this regard, there is clear evidence
that IRS' capability to manage contractors has weaknesses. In August
1995, IRS acquired the services of the Department of Commerce's
National Technical Information Service (NTIS) to act as IRS' prime
contractor in developing Cyberfile. However, Cyberfile was not
developed using disciplined management and technical practices. As a
result, this project exhibited many of the same problems we have
repeatedly identified in other TSM systems, and, after providing $17
million to NTIS, it was not ready for planned testing during the 1996
tax filing season. Similarly, IRS contracted in 1994 to build the
Document Processing System. After expending over a quarter of a
billion dollars on the project, IRS has now suspended the effort and
is reexamining some of its basic requirements, including which and
how many forms should be processed, and which and how much data
should be read from the documents.
We recently initiated an assignment to evaluate in detail IRS'
software acquisition capabilities using the Software Engineering
Institute's Software Acquisition CMM. This assignment is scheduled
to be completed later this year. It is clear that unless IRS has
mature, disciplined processes for acquiring software systems through
contractors, it will be no more successful in buying software than it
has been in building software.
CONCLUSIONS
------------------------------------------------------------ Letter :7
IRS has initiated a number of actions and is making some progress in
addressing our recommendations to correct its pervasive management
and technical weaknesses. However, none of these actions, either
individually or in the aggregate, fully satisfy any of our July 1995
recommendations and it is not clear when these actions will result in
disciplined systems development. As a result, IRS continues to spend
hundreds of millions of dollars on TSM through fiscal year 1997,
while fundamental weaknesses jeopardize the investment.
Recognizing its internal weaknesses, IRS plans to use a prime
contractor and increase use of software development contractors to
develop TSM. However, in this area, its plans and schedules are not
well defined, and, therefore, cannot be completely understood or
assessed. Further, as the experience with Cyberfile and the Document
Processing System projects makes clear, IRS does not have the mature
processes needed to acquire software and manage contractors
effectively.
MATTERS FOR CONGRESSIONAL
CONSIDERATION
------------------------------------------------------------ Letter :8
Because IRS still does not have (1) effective strategic information
management practices needed to manage TSM as an investment, (2)
mature and disciplined software development processes needed to
assure that systems built will perform as intended, (3) a completed
systems architecture that is detailed enough to guide and control
systems development, and (4) a schedule for accomplishing any of the
above, the Congress could consider limiting TSM spending to only
cost-effective modernization efforts that (1) support ongoing
operations and maintenance, (2) correct IRS' pervasive management and
technical weaknesses, (3) are small, represent low technical risk,
and can be delivered in a relatively short time frame, and (4)
involve deploying already developed systems, only if these systems
have been fully tested, are not premature given the lack of a
completed architecture, and produce a proven, verifiable business
value. As the Congress gains confidence in IRS' ability to
successfully develop these smaller, cheaper, quicker projects, it
could consider approving larger, more complex, more expensive
projects in future years.
Because IRS does not manage all of its current contractual efforts
effectively, and because its plans to use a "prime" contractor and
transition much of its systems development to additional contractors
are not well defined, the Congress could consider requiring that IRS
institute disciplined systems acquisitions processes and develop
detailed plans and schedules before permitting IRS to increase its
reliance on contractors.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :9
On June 6, 1996, we met with Treasury and IRS officials to discuss a
draft of this report and we incorporated their comments as
appropriate in finalizing it. In addition, on June 6, 1996, we
received written comments from Treasury.
In his letter, the Deputy Secretary of the Treasury reiterates
Treasury's commitment to significantly increased oversight of TSM and
to making a sharp turn in the way TSM is managed. He also makes
clear Treasury's and IRS' understanding that additional improvements
are necessary to fully correct the management and technical
weaknesses delineated in our report. The Deputy Secretary of the
Treasury also says that he is reducing the fiscal year 1997 budget
request for TSM from $850 million to $664 million and will need to
ensure, at all times, solid stewardship for the dollars appropriated
and clear accountability for the investments undertaken.
Achieving sound management for the TSM program will require that IRS
(1) institutionalize effective strategic information management
practices, (2) institutionalize mature and disciplined software
development processes, and (3) complete systems, data, and security
architectures and use them to guide and control systems development,
before making major investments in TSM systems development. Until
these disciplined processes are in place and the requisite
architectures completed, the Congress could consider limiting IRS TSM
spending to only cost-effective modernization efforts that meet the
criteria outlined in our Matters for Congressional Consideration.
---------------------------------------------------------- Letter :9.1
We are sending copies of this report to the Chairmen and the Ranking
Minority Members of (1) the Senate and House Committees on the
Budget, (2) the Subcommittee on Taxation and IRS Oversight, Senate
Committee on Finance, (3) the Senate Committee on Governmental
Affairs, (4) the Subcommittee on Oversight, House Committee on Ways
and Means, and (5) the House Committee on Government Reform and
Oversight. We are also sending copies to the Secretary of the
Treasury, Commissioner of the Internal Revenue Service, and Director
of the Office of Management and Budget. Copies will be available to
others upon request.
This work was performed under the direction of Dr. Rona B.
Stillman, Chief Scientist for Computers and Telecommunications, who
can be reached at (202) 512-6412. Other major contributors are
listed in appendix II.
Gene L. Dodaro
Assistant Comptroller General
List of Requesters
The Honorable Mark O. Hatfield
Chairman
The Honorable Robert C. Byrd
Ranking Minority Member
Committee on Appropriations
United States Senate
The Honorable Richard C. Shelby
Chairman
The Honorable J. Robert Kerrey
Ranking Minority Member
Subcommittee on Treasury, Postal
Service, and General Government
Committee on Appropriations
United States Senate
The Honorable Bob Livingston
Chairman
The Honorable David Obey
Ranking Minority Member
Committee on Appropriations
House of Representatives
The Honorable Jim Lightfoot
Chairman
The Honorable Steny H. Hoyer
Ranking Minority Member
Subcommittee on Treasury, Postal
Service, and General Government
Committee on Appropriations
House of Representatives
(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
THE TREASURY
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
Leonard Baptiste, Jr., Senior Assistant Director
Ronald W. Beers, Assistant Director
John P. Finedore, Assistant Director
William D. Hadesty, Technical Assistant Director
Leonard J. Latham, Technical Assistant Director
David L. McClure, Assistant Director
K. Alan Merrill, Technical Assistant Director
Keith A. Rhodes, Technical Assistant Director
Kelly A. Wolslayer, Evaluator-in-Charge
Madhav S. Panwar, Senior ADP/Telecommunications Specialist
Ronald E. Famous, Senior Information Systems Analyst
Barbarol J. James, Senior ADP/Telecommunications Specialist
Sabine R. Paul, Senior Information Systems Analyst
Nancy M. Donnellan, Information Systems Analyst
GENERAL GOVERNMENT DIVISION,
WASHINGTON, D.C.
Sherrie Russ, Senior Evaluator
Christopher E. Hess, Evaluator
*** End of document. ***