Tax Systems Modernization: Actions Underway But Management and Technical
Weaknesses Not Yet Corrected (Testimony, 09/10/96, GAO/T-AIMD-96-165).
GAO discussed the Internal Revenue Service's (IRS) efforts to modernize
the tax processing system. GAO noted that: (1) it has recently made
numerous recommendations to IRS relating to its Tax Systems
Modernization (TSM) effort; (2) IRS is making progress in maximizing
electronic tax filing and controlling its systems and software
development efforts; (3) IRS action on some GAO recommendations is
incomplete; (4) IRS has not defined a process for selecting,
controlling, and evaluating its technology investments, completed
procedures for requirements management, quality assurance, configuration
management, and project planning and tracking, or defined its systems,
security, and data architectures; (5) a Department of the Treasury
report acknowledged that IRS does not have the capability to develop and
integrate TSM, and will obtain additional contractual help to do so; and
(6) while additional contracting may help, IRS does not have the
capability to successfully manage all of its current contractors.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-AIMD-96-165
TITLE: Tax Systems Modernization: Actions Underway But Management
and Technical Weaknesses Not Yet Corrected
DATE: 09/10/96
SUBJECT: Electronic forms
Systems conversions
Tax administration systems
Computer services contracts
Information resources management
Strategic information systems planning
Systems design
Computer security
Requirements definition
Computer software
IDENTIFIER: IRS Corporate Accounts Processing System
IRS Cyberfile
IRS Electronic Filing Strategies Portfolio
IRS Tax System Modernization Program
SBA 8(a) Program
IRS Service Center Recognition/Image Processing System
TSM
Software Capability Maturity Model
IRS Integrated Case Processing System
IRS Modernization Integration Plan
IRS TSM Release Definition Document
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Before the Committee on Governmental Affairs
United States Senate
For Release on Delivery
Expected at
10 a.m.
Tuesday,
September 10, 1996
TAX SYSTEMS MODERNIZATION -
ACTIONS UNDERWAY BUT MANAGEMENT
AND TECHNICAL WEAKNESSES NOT YET
CORRECTED
Statement of Dr. Rona B. Stillman
Chief Scientist, Computers and Telecommunications
Accounting and Information Management Division
GAO/T-AIMD-96-165
GAO/AIMD-96-165T
(511524)
Abbreviations
=============================================================== ABBREV
TSM - x
IRS - x
NTIS - x
SBA - x
ADP - x
OMB - x
CMM - x
SLC - x
ICP - x
CAPS - x
ICP - x
TIPSS - x
GPMO - x
============================================================ Chapter 0
Mr. Chairman and Members of the Committee:
We are pleased to be here today to discuss the Internal Revenue
Service's (IRS) efforts to modernize its tax processing system. In
July 1995, we reported that pervasive management and technical
weaknesses placed at serious risk the government's multibillion
dollar Tax Systems Modernization (TSM) effort. To correct these
weaknesses, we made more than a dozen recommendations to improve IRS'
(1) electronic filing strategy, (2) strategic information management,
(3) software development, (4) systems architecture, integration, and
testing, and (5) accountability and control of systems
modernization.\1
In March 1996 testimony\2 before the Committee, we reiterated our
concerns that IRS' effort to modernize tax processing was jeopardized
by persistent and pervasive management and technical weaknesses and
we advised the Committee that IRS' ongoing efforts did not include
milestones or provide enough evidence to conclude that weaknesses
will soon be corrected. We also noted that the development of IRS'
Cyberfile electronic filing system exhibited many of the technical
weaknesses that we identified with TSM. We recently completed a
review for the Committee on Cyberfile contractual issues and issued a
report which identifies severe acquisition and financial problems.\3
In a May 1996 report required by the Treasury, Postal Service, and
General Government Appropriations Act of 1996, the Department of the
Treasury delineated, and we verified, that IRS has initiated a number
of actions and is making some progress in addressing our July 1995
recommendations.\4 For example, IRS (1) is preparing a comprehensive
strategy to maximize electronic filing, (2) has created an investment
review board to select, control, and evaluate its information
technology investments, (3) has updated its system engineering
process, is updating its systems life cycle methodology, and is
working across various IRS organizations to define disciplined
processes for software requirements management, quality assurance,
configuration management, and project planning and tracking, and (4)
has completed a descriptive overview of an integrated, three-tier,
distributed systems architecture.
However, as we reported in June 1996,\5 many of these actions are
still incomplete and do not respond fully to any of our
recommendations. Examples include the following:
-- The comprehensive business strategy for electronic filing is not
scheduled for completion until October 1996.
-- IRS does not yet have a repeatable process for selecting,
controlling, and evaluating its technology investments; not all
planned and ongoing systems have been reviewed in a single
investment portfolio; and the basis for making investment
decisions is still unclear.
-- The procedures for requirements management, quality assurance,
configuration management, and project planning and tracking are
being developed but are still incomplete.
-- IRS has completed neither its integrated systems architecture
nor its security and data architectures and has not completed a
schedule for doing so.
As a result, IRS has not made adequate progress in correcting its
management and technical weaknesses, and none of our recommendations
has been fully implemented. Until IRS' weaknesses are corrected, we
believe the Congress should consider limiting TSM spending to only
cost-effective modernization efforts that (1) support ongoing
operations and maintenance, (2) correct IRS' pervasive management and
technical weaknesses, (3) are small, represent low technical risk,
and can be delivered in a relatively short time frame, and (4)
involve deploying already developed systems only if these systems
have been fully tested, are not premature given the lack of a
completed architecture, and produce a proven, verifiable business
value.
In its report, Treasury states that because IRS does not currently
have the capability to develop and integrate TSM, it will obtain
additional contractual help to accomplish these tasks. While
effective use of additional contractors may, in the future,
strengthen IRS' capability to modernize, IRS clearly does not now
have the capability to manage all of its current contractors
successfully. For example, Cyberfile, which is being built by
contractors, has used many of the same undisciplined software
development practices we had criticized at IRS and, as a result,
could not be piloted during the 1996 tax filing season as originally
planned. Therefore, if IRS is to use additional contractors
effectively in the future, it will have to first strengthen and
improve its ability to manage software development contractors.
I will now discuss the results of our Cyberfile review, the TSM
redirection efforts described in Treasury's report, IRS' progress in
addressing each of the recommendations we made in July 1995, and
actions that must be taken before IRS obtains additional contractual
support to develop TSM.
--------------------
\1 Tax Systems Modernization: Management and Technical Weaknesses
Must Be Corrected If Modernization Is To Succeed (GAO/AIMD-95-156,
July 26, 1995).
\2 Tax Systems Modernization: Management and Technical Weaknesses
Must Be Overcome To Achieve Success (GAO/T-AIMD-96-75, March 26,
1996).
\3 Tax Systems Modernization: Cyberfile Project Was Poorly Planned
and Managed (GAO/AIMD-96-140, August 26, 1996).
\4 Report to House and Senate Appropriations Committees: Progress
Report on IRS's Management and Implementation of Tax Systems
Modernization (Department of the Treasury, May 6, 1996).
\5 Tax Systems Modernization: Actions Underway But IRS Has Not Yet
Corrected Management and Technical Weaknesses (GAO/AIMD-96-106, June
7, 1996).
CYBERFILE PROJECT WAS POORLY
PLANNED AND MANAGED
---------------------------------------------------------- Chapter 0:1
We found that IRS' selection of the Department of the Commerce's
National Technical Information Service (NTIS) to develop Cyberfile
was not based on sound analysis. IRS did not adequately analyze
requirements, consider alternatives, or assess NTIS' capabilities to
develop and operate an electronic filing system, even though the need
for these critical prerequisites was brought to management's
attention as early as July 1995. Instead, IRS selected NTIS because
it was expedient and because NTIS promised IRS, without any objective
support, that it could develop Cyberfile in less than 6 months and
have it operating by February 1996.
In an attempt to meet these self-imposed time constraints, the
Cyberfile project was hastily initiated. Development and acquisition
were undisciplined, and the project was poorly managed and overseen.
As a result, it was not delivered on time, and after advancing $17.1
million to NTIS, IRS has suspended Cyberfile's development and is
reevaluating the project.
We also found that IRS and NTIS did not follow all applicable
procurement laws and regulations in developing Cyberfile. IRS cited
the Brooks ADP Act (40 U.S.C. 759) for its authority to procure
Cyberfile. However, IRS did not perform requirements and
alternatives analyses as required by the Federal Information
Resources Management Regulation which implemented the Brooks ADP Act.
NTIS also violated applicable procurement laws and regulations in
developing Cyberfile. To obtain contractor services quickly, NTIS
modified an existing sole source contract awarded through the Small
Business Administration's (SBA) small and disadvantaged businesses
program, referred to as the "Section 8(a)" program. The modification
provided $3.3 million for Cyberfile and increased the total contract
value to $4.3 million. NTIS did not submit this modification to SBA
for review as required under the Section 8(a) program. Further, this
modification circumvented SBA rules requiring that contracts over
$3.0 million be competed among eligible Section 8(a) firms rather
than being issued on a sole source basis. In addition, at the time
of modification, the contractor was not an eligible 8(a) firm under
SBA regulations, and had NTIS submitted the modification to SBA as
required, responsible officials at SBA said they would have rejected
it.
Our review of Cyberfile obligations and costs found that they were
not accounted for properly. IRS (1) significantly understated the
obligations related to the project and (2) improperly accounted for
the $17.1 million advanced to NTIS. In addition, NTIS did not
promptly and accurately account for Cyberfile obligations and costs.
Specifically, significant financial transactions were not properly
documented and obligations and costs were not recorded promptly and
accurately.
Finally, we found that adequate financial and program management
controls were not implemented to ensure that Cyberfile was acquired
cost-effectively. As a result, excess costs were incurred. For
example, the interagency agreement between IRS and NTIS was not
structured to minimize costs, and Cyberfile costs continued to be
incurred after the project was suspended. Specifically, the
agreement allowed NTIS to assess a 10 percent management fee for (1)
costs associated with NTIS' failure to follow preferred management
practices, such as late payment penalties and (2) items which IRS
could have readily obtained directly and provided to NTIS, such as
computer equipment acquired under existing government contracts. In
this regard, IRS incurred about $89,000 in NTIS management fees for
purchasing items costing over $886,000 under an existing Department
of the Treasury contract which is administered by IRS. If IRS had
purchased these items directly and provided them to NTIS, it could
have avoided NTIS' $89,000 management fee.
In light of the severity of acquisition and financial problems
identified during our review, we recommended, inter alia, that before
resuming work on Cyberfile, IRS and the Department of Commerce each
submit a report to the Committee detailing the (1) weaknesses in IRS'
and NTIS' acquisition and financial management processes and controls
that permitted Cyberfile mismanagement and (2) actions that have been
taken to ensure that these weaknesses in processes and controls have
been corrected and that resulting mismanagement does not recur.
In commenting on our Cyberfile report, Treasury agreed with our
findings and recommendations. It stated that it shared our concerns
regarding IRS' management of the Cyberfile project, and that the
experience with the project underscores the importance of IRS
implementing our recommendations. IRS did not disagree with any of
our findings or recommendations. It said that Cyberfile was not
successful and encountered problems and explained that it is
conducting an internal review of Cyberfile to identify a full range
of corrective action.
TREASURY'S TSM REPORT
ACKNOWLEDGES WEAKNESSES AND
DESCRIBES TSM REDIRECTION
EFFORTS
---------------------------------------------------------- Chapter 0:2
The Department of the Treasury, in its May 1996 report to the Senate
and House Appropriations Committees, provides a candid assessment of
TSM progress and future redirection, and a description of ongoing and
planned actions intended to respond to our July 1995 recommendations
to correct management and technical weaknesses. Treasury found that
despite some qualified success, IRS has not made progress on TSM as
planned because systems development efforts have taken longer than
expected, cost more than originally estimated, and delivered less
functionality than originally envisioned. It concluded that
significant changes are needed in IRS' management approach and that
it is beyond the scope of IRS' current ability to develop and
integrate TSM without expanded use of external expertise.
The report notes that work has been done to rethink, scale back, and
change the direction of TSM. Additional changes are still in
progress with actions underway to restructure the management of TSM
and expand the use of contractors. Agreeing that our July 1995
recommendations are valid, the report notes that more work has to be
done to respond effectively to our recommendations. It states that
progress in IRS' management and technical areas can only be achieved
by institutionalizing improved practices and monitoring projects for
conformance to mandated standards and practices.
However, the report does not address the basic problem of continuing
to invest hundreds of millions of dollars in TSM before the requisite
management and technical disciplines are in place. Nor does it
address the risk inherent in shifting hundreds of millions of dollars
to additional contractual efforts when the evidence, as demonstrated
with IRS' Cyberfile project, is clear that IRS does not have the
disciplined processes in place to manage all of its current
contractual efforts effectively.
FUNDAMENTAL MANAGEMENT AND
TECHNICAL WEAKNESSES ARE BEING
ADDRESSED, BUT NONE HAVE BEEN
CORRECTED
---------------------------------------------------------- Chapter 0:3
IRS has initiated a number of actions to address management and
technical weaknesses that continue to impede successful systems
modernization. However, ongoing efforts do not correct the
weaknesses and do not provide enough evidence to determine when they
will be corrected and what steps if any are being taken in the
interim to mitigate the risks associated with ongoing TSM spending.
IRS DOES NOT YET HAVE A
COMPREHENSIVE STRATEGY TO
MAXIMIZE ELECTRONIC FILINGS
-------------------------------------------------------- Chapter 0:3.1
IRS has identified increasing electronic filings as critical to
achieving its modernization vision. In our July 1995 report, we said
that IRS did not have a comprehensive business strategy to reach or
exceed its goal of 80 million electronic filings by 2001. IRS'
estimates and projections for individual and business returns
suggested that by 2001, as few as 39 million returns may be submitted
electronically, less than half of IRS' goal and only about 17 percent
of all returns expected to be filed.
We reported that IRS' business strategy would not maximize electronic
filings because it primarily targeted taxpayers who use a third party
to prepare and/or transmit simple returns, are willing to pay a fee
to file their returns electronically, and are expecting refunds.
Focusing on this limited taxpaying population overlooked most
taxpayers, including those who prepare their own tax returns using
personal computers, have more complicated returns, owe tax balances,
and/or are unwilling to pay a fee to a third party to file a return
electronically.
We concluded that without a strategy that also targets these
taxpayers, IRS would not meet its electronic filing goals. In
addition, if, in the future, taxpayers file more paper returns than
IRS expects, added stress will be placed on IRS' paper-based systems.
Accordingly, we recommended that IRS
refocus its electronic filing business strategy to target, through
aggressive marketing and education, those sectors of the taxpaying
population that can file electronically most cost-beneficially.
IRS agreed with this recommendation and said that it had convened a
working group to develop a detailed, comprehensive strategy to
broaden public access to electronic filing while also providing more
incentives for practitioners and the public to file electronically.
It said that the strategy would include approaches for taxpayers who
are unwilling to pay for tax preparer and transmitter services, who
owe IRS for balances due, and/or who file complex tax returns.
Further, IRS said that the strategy would address that segment of the
taxpaying population that would prefer to file from home using
personal computers.
To date, IRS has performed an electronic filing marketing analysis at
local levels; developed a marketing plan to promote electronic
filing; consolidated 21 electronic filing initiatives into its
Electronic Filing Strategies portfolio; and initiated a
re-engineering project with a goal to reduce paper tax return filings
to 20 percent or less of the total volume by the year 2000. It plans
to complete its electronic filing strategy in October 1996. These
initiatives could result in future progress toward increasing
electronic filings. However, our review found that these initiatives
are not far enough along to determine whether they will culminate in
a comprehensive strategy that identifies how IRS plans to target
those sectors of the taxpaying population that can file
electronically most cost-beneficially. It also is not clear how the
reengineering project will impact the strategy or how these
initiatives will impact TSM systems that are being developed.
IRS' STRATEGIC INFORMATION
MANAGEMENT PRACTICES REMAIN
INEFFECTIVE
-------------------------------------------------------- Chapter 0:3.2
In our July 1995 report, we said that IRS did not have strategic
information management practices in place. We found, for example,
that despite the billions of dollars at stake, information systems
were not managed as investments. To overcome this, and provide the
Congress with the insight needed to assess IRS' priorities and
rationalization for TSM projects, we recommended that the IRS
Commissioner
take immediate action to implement a complete process for selecting,
prioritizing, controlling, and evaluating the progress and
performance of all major information systems investments, both new
and ongoing, including explicit decision criteria, and
using these criteria, to review all planned and ongoing systems
investments by June 30, 1995.
In agreeing with these recommendations, IRS said it would take a
number of actions to provide the underpinning it needs for strategic
information management. IRS said, for example, that it was
developing and implementing a process to select, prioritize, control,
and evaluate information technology investments to achieve
reengineered program missions.
Our assessment found that IRS has taken steps towards putting into
place a process for managing its extensive investments in information
systems. The following are examples of these steps.
-- IRS created an executive-level Investment Review Board, chaired
by the Associate Commissioner for Modernization, for selecting,
controlling and evaluating all of IRS' information technology
investments.
-- IRS developed initial and revised sets of decision criteria used
in the summer and fall of 1995, as part of its Resource
Allocation and Investment Review to make additional changes in
information technology resource allocations for remaining fiscal
year 1996 funds and planned 1997 spending. This review included
only TSM projects under development. It did not address
operational systems, infrastructure, or management and technical
support activities.
-- The Treasury Department created a Modernization Management Board
to review and validate high-risk, high-cost TSM investments and
to set policy and strategy for IRS modernization effort.
-- IRS is considering the use of a "project readiness review" as an
additional Investment Review Board control mechanism for gauging
project readiness to proceed with spending.
-- IRS developed a business case handbook that includes decision
criteria on costs, benefits, and risks. It is reassessing the
business cases, which were developed on the TSM projects, using
the handbook. Results are planned to be used by the Investment
Review Board to assist in making funding decisions for fiscal
year 1997.
-- IRS has developed an investment evaluation review handbook
designed to assess projected costs and benefits against actual
results. The handbook has been used on four TSM projects and
five additional reviews are scheduled to be completed within the
next year. The completed reviews contain explicit descriptions
of problems encountered in developing these systems. The
reviews make specific recommendations for management and
technical process changes to improve future results. Specific
recommendations pertain to strengthening project direction and
decision-making. Many reflect concerns that we have raised in
past reviews. The investment evaluation reviews were presented
to the Investment Review Board and disseminated to other IRS
managers. IRS is defining roles, responsibilities, and
processes for incorporating Investment Evaluation Review
recommendations at the project and process levels.
These are positive steps and indicate a willingness to address many
of the weaknesses raised in our past reports and testimonies. But,
as noted in Treasury's report on TSM, the investment process is not
yet complete. According to Treasury, it is missing (1) specific
operating procedures, (2) defined reporting relationships between
different management boards and committees, and (3) updated business
cases for major TSM technology investments.
These concerns coincide with two central criticisms we have
repeatedly made about TSM. Because of the sheer size, scope, and
complexity of TSM, it is imperative that IRS institutionalize a
repeatable process for selecting, controlling, and evaluating its
technology investments, and that it make informed investment
decisions based on reliable qualitative and quantitative assessments
of costs, benefits, and risks. Although IRS is planning and is in
the initial stages of implementing parts of such a process, a
complete, fully integrated process does not yet exist. Specifically,
IRS has not provided us evidence to justify its claims that its
decisions were supported by acceptable data on project costs,
benefits, and risks. For example:
-- Our review found no evidence to suggest that IRS established
minimal data requirements for the decisions made as part of the
TSM Resource Allocation and Investment Review or the rescope
process in December 1995. For example, because IRS lacks the
basic capabilities for disciplined software development, it
cannot convincingly estimate systems development costs,
schedule, or performance. Subsequent to its rescope analysis,
IRS developed minimal data quality requirements for cost-benefit
and risk studies, proposed return-on-investment calculations,
and return-on-investment thresholds, or comparisons of expected
performance improvements with results to date. However, to date
few, if any, projects have met these criteria.
-- In deciding whether to accelerate, delay, or cancel specific TSM
projects, IRS did not use validated data on actual versus
projected costs, benefits, or risks as set forth by the Office
of Management and Budget (OMB).\6 Instead, IRS continues to make
its decisions based on spending whatever budgeted funding
ceiling amounts can be obtained through its annual budget and
appropriations cycles. As a result, IRS cannot convincingly
justify its TSM spending decisions.
-- All projects (i.e., proposed projects, projects under
development, operational systems, infrastructure, and management
and technical support activities) were not included in a single
systems investment portfolio. Instead, only TSM projects under
development were ranked. As a result, there is no compelling
rationale for determining how much to invest in these projects
compared to other projects, such as operational systems and
infrastructure.
-- There is no defined process with prescribed roles and
responsibilities to ensure that the results of investment
evaluation reviews are being used to (1) modify project
direction and funding when appropriate and (2) assess and
improve existing investment selection and control processes and
procedures. As a result, there is no evidence that changes are
occurring based on valuable lessons learned, as in the recently
completed post-implementation review of the Service Center
Recognition/Image Processing System. For example, IRS found
that because system requirements were not adequately defined or
documented, the system could not be quantifiably tested properly
which adversely affected the implementation of the system.
Moreover, with only four investment evaluation reviews completed
to date and five planned for the upcoming year, this represents
only a small fraction of the total IRS annual investment in TSM.
More must be done to confirm actual results achieved from TSM
expenditures.
--------------------
\6 Evaluating Information Technology Investments: A Practical Guide
(Executive Office of the President, Office of Management and Budget,
November 1995).
REENGINEERING EFFORTS NOT
LINKED TO MODERNIZATION
-------------------------------------------------------- Chapter 0:3.3
We noted in our July 1995 report that IRS' reengineering efforts were
not linked to its systems development efforts. As shown in our work
with leading organizations, information system development projects
that are not driven by a critical reexamination and redesign of
business processes achieve only a fraction of their potential to
improve performance, reduce costs, and enhance quality.
Since our July report, IRS' reengineering efforts have undergone a
redirection. Three reengineering projects--processing returns,
responding to taxpayers, and enforcement actions--were halted because
IRS decided to focus instead on an enterprise-level view of
reengineering.
Its new effort, entitled Tax Settlement Reengineering, was begun in
March 1996 and involves a comprehensive review of all the major
processes and activities that enable taxpayers to settle their tax
obligations, from educational activities through final settlement of
accounts. The reengineering project team, working with IRS'
Executive Committee, has identified 16 major processes involved in
tax settlement and is about to begin reengineering four of them.
High-level designs of the new processes are scheduled to be defined
by September 30, 1996, with work on detailed designs to start early
in fiscal year 1997, if approved by the Executive Committee.
Reengineering efforts on as many as eight other tax settlement
processes could be underway by the end of fiscal year 1997.
Although this effort could have substantial impact, IRS still faces
the same problem we reported on a year ago. Reengineering lags well
behind the development of TSM projects, whereas it should be ahead of
it--defining and directing the technology investments needed to
support new, more efficient business processes. Until the
reengineering effort is mature enough to drive TSM projects, there is
no assurance that ongoing systems development efforts will support
IRS' future business needs and objectives.
The reengineering team believes that by September 30, 1996, they will
have a general idea of how the first four tax settlement
reengineering projects may impact current systems development
efforts. If additional reengineering projects are started as planned
in 1997, it could be another year or more before most of the
information and systems requirements stemming from these projects are
defined. Meanwhile, investment continues in many TSM projects that
may not support the requirements resulting from these reengineering
efforts.
IRS acknowledges that integration of reengineering and TSM must occur
and has assigned responsibility for it to the Associate Commissioner
for Modernization. However, it has not yet specified how or when the
requisite integration will occur.
SOFTWARE DEVELOPMENT
ACTIVITIES ARE INCONSISTENT
AND POORLY CONTROLLED
-------------------------------------------------------- Chapter 0:3.4
In our July 1995 report, we said that unless IRS improves its
software development capability, it is unlikely to build TSM timely
or economically and systems are unlikely to perform as intended. To
assess its software capability, in September 1993, IRS rated itself
using the Software Engineering Institute's Capability Maturity Model
(CMM). IRS placed its software development capability at the lowest
level, described as ad hoc and sometimes chaotic and indicating
significant weaknesses in its software development capability. Our
review confirmed that IRS' software development capability was
immature and weak in key process areas. For instance,
-- a disciplined process to manage system requirements was not
being applied to TSM systems,
-- a software tool for planning and tracking development projects
was not consistently used,
-- software quality assurance functions were not well defined or
consistently implemented,
-- systems and acceptance testing were neither well defined nor
required, and
-- software configuration management\7 was incomplete. To address
IRS' software development weaknesses and upgrade IRS' software
development capabilities, we recommended that the IRS
Commissioner
immediately require that all future contractors who develop software
for the agency have a software development capability rating of at
least CMM Level 2;\8 and
before December 31, 1995,
define, implement, and enforce a consistent set of requirements
management procedures for all TSM projects that goes beyond IRS'
current request for information services process, and for software
quality assurance, software configuration management, and project
planning and tracking; and
define and implement a set of software development metrics to measure
software attributes related to business goals.
IRS agreed with these recommendations and said that it was committed
to developing consistent procedures addressing requirements
management, software quality assurance, software configuration
management, and project planning and tracking. It also said that it
was developing a comprehensive measurement plan to link process
outputs to external requirements, corporate goals, and recognized
industry standards.
Specifically regarding the first recommendation, IRS has (1)
developed standard wording for use in new and existing contracts that
have a significant software development component, requiring that all
software development be done by an organization that is at CMM Level
2, (2) developed a plan for achieving CMM Level 2 capability on all
of its contracts, and (3) started to implement a plan to monitor
contractors' capabilities, which may include the use of CMM-based
software capability evaluations. The Department of the Treasury
report also noted that a schedule for conducting software capability
evaluations was developed. However, we found that IRS does not yet
have the disciplined processes in place to ensure that all
contractors are performing at CMM Level 2. For example, contractors
developing the Cyberfile electronic filing system were not using CMM
Level 2 processes subsequent to our July 1995 recommendation.
Further, no schedule for conducting software capability evaluations
has yet been developed.
With respect to the second recommendation, IRS is updating its
systems life cycle (SLC) methodology. The SLC is planned to have
details for systems engineering and software development processes,
including all CMM key process areas. IRS has updated its systems
engineering process to include guidance for defining and analyzing
systems requirements and for preparing work packages. Further, IRS
has drafted handbooks providing guidance to audit and verify
developmental processes. In addition, IRS has developed a
configuration management plan template, updated its requirements
management request for information services\9 documents, and
developed and implemented a requirements management course. The
Treasury Department also reported that IRS is testing the SLC on two
TSM efforts, Integrated Case Processing (ICP) and Corporate Accounts
Processing System (CAPS). IRS also has a CMM process improvement
plan, and work is being done across various IRS organizations to
define processes to meet CMM Level 2. Finally, IRS is assessing its
capabilities to manage contractors using the CMM goals.
However, the procedures for requirements management, software quality
assurance, software configuration management, and project planning
and tracking are still not complete. A software development life
cycle implementation project, which is to include these procedures,
is not scheduled for completion until September 30, 1996. In
addition, software quality assurance and configuration management
plans for two ICP projects\10 were not being used, and the groups
developing software for CAPS do not have a software configuration
management plan or a schedule for its development. Further, ICP and
CAPS development is continuing without the guidelines and procedures
for other process areas (e.g., requirements management, project
planning, and project tracking and oversight) required by CMM Level
2.
Regarding the third recommendation, IRS has a three-phase process to
(1) identify data sources for metrics, (2) define metrics to be used,
and (3) implement the metrics. A partial set of metrics was
identified. Initial use of these metrics--populated with real data
and in a preliminary format--began in June 1996. Data sources for
these metrics have been identified and weaknesses (such as
difficulties in retrieving the data and inconsistencies in the data)
are being documented to provide feedback to various systems' owners.
However, this initial set of metrics is incomplete. It focuses on
areas such as time reporting, project sizing, and defect tracing and
analysis, but it does not include measures for determining customer
satisfaction and cost estimation. Such measures are needed to
adequately track the needed functionality with associated costs
throughout systems development. Further, there is no schedule for
completing the definition of metrics or for institutionalizing the
processes needed to ensure their use. Finally, there is no mechanism
in place to correct identified data and data collection weaknesses.
In summary, although IRS has begun to act on our recommendations,
these actions are not yet complete or institutionalized and, as a
result, systems are still being developed without the disciplined
practices and metrics needed to give management assurance that they
will perform as intended.
--------------------
\7 Configuration management involves selecting project baseline items
(e.g., specifications), systematically controlling these items and
changes to them, and recording their status and changes.
\8 The Software Engineering Institute at Carnegie Mellon University
has developed a model, the Software Capability Maturity Model (CMM),
to evaluate an organization's software development capability. CMM
Level 2 denotes that basic project management processes are
established to track cost, schedule, and functionality and that the
necessary process discipline is in place to repeat earlier successes
on similar projects.
\9 A request for information services is a process to request changes
to IRS' computer systems. This process provides a way to request,
control, monitor, and track changes to IRS' computer systems.
\10 The two projects are the Case Processing System and the Case
Inventory Management System.
SYSTEMS ARCHITECTURES,
INTEGRATION, AND TESTING ARE
INCOMPLETE
-------------------------------------------------------- Chapter 0:3.5
In our July 1995 report, we said that IRS' systems architectures,\11
integration planning, and system testing and test planning were
incomplete.
To address IRS' technical infrastructure weaknesses, we recommended
that the IRS Commissioner
before December 31, 1995,
complete an integrated systems architecture, including security,
telecommunications, network management, and data management;
institutionalize formal configuration management for all newly
approved projects and upgrades and develop a plan to bring ongoing
projects under formal configuration management;
develop security concept of operations, disaster recovery, and
contingency plans for the modernization vision and ensure that these
requirements are addressed when developing information system
projects;
develop a testing and evaluation master plan for the modernization;
establish an integration testing and control facility; and
complete the modernization integration plan and ensure that projects
are monitored for compliance with modernization architectures.
IRS agreed with these recommendations and said that it was
identifying the necessary actions to define and enforce systems
development standards and architectures agencywide. IRS' current
efforts in this area follow.
-- In April 1996, IRS completed a descriptive overview of its
integrated three-tier, distributed systems architecture to
provide management with a high-level view of TSM's
infrastructure and supporting systems. IRS has tasked the
integration support contractor to develop the data and security
architectures.
-- IRS has adopted an accepted industry standard for configuration
management. It developed and distributed its Configuration
Management Plan template, which identifies the elements needed
when constructing a configuration management plan. In April
1996, enterprisewide configuration management policies and
procedures were established. IRS also plans to obtain
contractor support to develop, implement, and maintain a
vigorous configuration management program.
-- IRS has prepared a security concept of operations and a disaster
recovery and contingency plan.
-- IRS has developed a test and evaluation master plan for TSM.
IRS plans to develop implementation and enforcement policies for
the plan.
-- IRS has established an interim integration testing and control
facility, which is currently being used to test new software
releases. It is also planning a permanent integration testing
and control facility, scheduled to be completed by December
1996.
-- IRS has completed drafts of its TSM Release Definition Document,
which is planned to provide definitions for new versions of TSM
software from 1997 to 1999, and Modernization Integration Plan,
which is planned to define IRS' process for integrating current
and future TSM initiatives.
These activities start to address our recommendations, but do not
satisfy any of them. Specifically:
-- IRS has not completed its integrated systems architecture (the
"blueprints" of TSM) and has not committed to a completion date.
Its completed high-level overview was not intended to, and does not
provide, the level of detail needed to provide effective guidance to
design and build systems. For example, IRS' concept of a three-tier,
distributed architecture has not been delineated to the level needed
to provide sufficient detail to understand the security requirements
and implications. It does not, for instance, specify what security
mechanisms are to be implemented between and among the three tiers to
ensure that only properly authorized users are allowed to access tax
processing application software and taxpayer data. IRS is using
contractors to complete its security and data architectures, but has
not committed to a completion date. Meanwhile, IRS is investing in
building TSM systems without the "blueprints" that are needed.
-- IRS has not yet brought its development, acceptance, and
production environments under configuration management control.
For example, there is no disciplined process for moving software
from the test to the production environment. Additionally,
although directives have been distributed to follow various TSM
systems development standards, no enforcement mechanisms are in
place.
-- Our review of the security concept of operations found that the
document does not identify selected security methods and
techniques. For example, it discusses two methods for providing
identification and authentication for controlling user access to
various systems without specifying which method should be used.
The security concept of operations is also sometimes
inconsistent with the security mechanisms currently being
implemented on systems now being developed and does not indicate
how, when, or if these inconsistencies will be resolved. The
specific methods and techniques are currently planned to be
provided in different versions of a planned technical concept of
operations. The first version is currently planned to be
completed in January 1997.
-- IRS' disaster recovery and contingency plan is a high-level
document for planning that presents basic tenets for information
technology disaster recovery but not the detail needed to
provide useful guidance in emergencies. For example, it does
not explain the steps that computing centers need to take to
absorb the workload of a center that suffers a disaster.
-- The test and evaluation master plan provides the guidance needed
to ensure sufficient developmental and operational testing of
TSM. However, it does not describe what security testing should
be performed, or how these tests should be conducted. Further,
it does not specify the responsibilities and processes for
documenting, monitoring, and correcting testing and integration
errors.
-- IRS is still working on plans for its integration testing and
control facility. In the interim, it has established a
temporary facility which is being used for limited testing. The
permanent facility is not currently being planned to simulate
the complete production environment and will not, for example,
include mainframe computers. Instead, IRS plans to continue to
test mainframe computer software and systems which interface
with the mainframes in its production environment. To ensure
that IRS does not put operations and service to the taxpayers at
risk, IRS should prepare a thorough assessment of its solution,
including an analysis of alternative testing approaches and
their costs, benefits, and risks.
-- IRS' draft TSM Release Definition Document and draft
Modernization Integration Plan (1) do not reflect TSM rescoping
and the information systems reorganization under the Associate
Commissioner, (2) do not provide clear and concise links\12 to
other key documents (e.g., its integrated systems architecture,
business master plan, concept of operations, and budget), and
(3) assume that IRS has critical processes in place that
actually have not been implemented (e.g., effective quality
assurance and disciplined configuration management).
In summary, although IRS has taken actions to prepare a systems
architecture and improve its integration and system testing and test
planning, these efforts are not yet complete or institutionalized,
and, as a result, TSM systems continue to be developed without the
detailed architectures and discipline needed to ensure success.
--------------------
\11 A systems architecture is an evolving description of an approach
to achieving a desired mission. It describes (1) all functional
activities to be performed to achieve the desired mission, (2) the
system elements needed to perform the functions, (3) the designation
of performance levels of those system elements, and (4) the
technologies, interfaces, and location of functions.
\12 For example, it is not clear how particular software releases are
tied to business master plan goals and objectives and to the
integrated transition plan and schedule's products and services.
Without these links, the documents do not provide important
information on how much will be done by each release, in what period
of time, and at what cost.
NO SINGLE IRS ENTITY
CONTROLS ALL INFORMATION
SYSTEMS EFFORTS
-------------------------------------------------------- Chapter 0:3.6
In our July 1995 report, we said that IRS had not established an
effective organizational structure to consistently manage and control
systems modernization organizationwide. The accountability and
responsibility for IRS' systems development was spread among IRS'
Modernization Executive, Chief Information Officer, and research and
development division. To help address this concern, in May 1995, the
Modernization Executive was named Associate Commissioner for
Modernization. The Associate Commissioner was to manage and control
systems development efforts previously conducted by the Modernization
Executive and the Chief Information Officer.
In September 1995, the Associate Commissioner assumed responsibility
for the formulation, allocation, and management of all information
systems resources for both TSM and non-TSM expenditures. In February
1996, IRS issued a Memorandum of Understanding providing guidance for
initiating and conducting technology research and for transitioning
technology research initiatives into system development projects.
It is important that IRS maintain an organizationwide focus to manage
and control all new modernization systems and all upgrades and
replacements of operational systems throughout IRS. To do so, we
recommended that the IRS Commissioner
give the Associate Commissioner management and control responsibility
for all systems development activities, including those of IRS'
research and development division.
Steps are being taken by the Associate Commissioner to establish
effective management and control of systems development activities
throughout IRS. For example, its SLC methodology is required for
information systems development, and information technology entities
throughout the agency have been directed to submit documentation on
all information technology projects for review. However, there is no
defined and effective mechanism for enforcing the standards or
ensuring that organizational entities cannot conduct systems
development activities outside the control of the Associate
Commissioner. Further, no time frames have been established for
defining and implementing such control mechanisms. As a result,
systems development conducted by the research and development
division has now been redefined as technology research, keeping it
from the control of the Associate Commissioner.
In summary, although IRS has made improvements in consolidating
management control over systems development, the Associate
Commissioner still does not yet have control over all IRS' systems
development activities.
PLANS MUST BE DEFINED AND
CAPABILITIES STRENGTHENED
BEFORE OBTAINING ADDITIONAL
CONTRACTUAL SUPPORT
---------------------------------------------------------- Chapter 0:4
IRS plans to increase its reliance on the private sector by (1)
preparing an acquisition plan and statement of work to conduct an
expedited competitive selection for a "prime" development and
integration contractor, (2) transferring responsibility for systems
engineering, design, prototyping, and integration for core elements
of TSM to its integration support contractor, and (3) making greater
use of software development contractors, including those available
under the Treasury Information Processing Support Services (TIPSS),
to develop and deliver major elements of production TSM systems. By
increasing its reliance on contractors, IRS expects to improve the
accountability for and probability of TSM success.
IRS plans to increase the use of private-sector integration and
development expertise by expanding the use of contractors to support
TSM. It outlined a three-track approach for transitioning over a
period of 2 years to the use of a "prime" contractor that would have,
according to IRS, overall authority and responsibility for the
development, delivery, and deployment of modernized information
systems.
To facilitate this strategy, IRS reported it would consolidate the
management of all TSM resources, including key TSM contractors, in
its Government Program Management Office (GPMO). Under the direct
control of the Chief Information Officer, the GPMO will be delegated
authority for the management and control of the IRS staff and
contractors that plan, design, develop, test, and implement TSM
components. IRS plans to have the GPMO fully staffed and operational
by October 1, 1996. IRS informed us that as of August 29, 1996, a
director had been appointed and over 40 staff had been assigned to
the office. IRS representatives also told us the agency was
currently developing a detailed contract management plan and a
statement of work for acquiring its "prime" contractor, and believed
it could award a contract in about 2 years.
IRS' approach to expanding the use of contractors to build TSM is
still in the early planning stages. Because of this, IRS was unable
to provide us with formal plans, charters, schedules or the
definitions of shared responsibilities between the GPMO and the
existing program and project management staff.
At this point, it is unclear what these IRS planned actions entail,
or how they will work. For example, IRS has not specified how and
when it plans to transfer its development activities to contractors,
and to what extent contractors could be held responsible for existing
problems in these government-initiated systems. This is particularly
important because if IRS continues as planned, the principal TSM
systems will be in development and/or deployed before IRS plans to
select a "prime" contractor in about 2 years. Moreover, it is not
clear how the "prime" contractor would direct potential competitors
that are already under contract with IRS. Without further
explanation of and a schedule for transitioning specific
responsibilities from IRS to contractors, we cannot fully understand
or assess IRS' plans.
Further, plans to use additional contractors will succeed if, and
only if, IRS has the in-house capabilities to manage these
contractors effectively. In this regard, there is clear evidence
that IRS' capability to manage contractors has weaknesses. In August
1995, IRS acquired the services of NTIS to act as IRS' "prime
contractor" for Cyberfile, i.e., NTIS was given the overall
responsibility for development, delivery, and deployment of the
system. However, Cyberfile was not developed using disciplined
management and technical practices. As a result, this project
exhibited many of the same problems we have repeatedly identified in
other TSM systems, and after providing $17 million to NTIS, Cyberfile
was not ready for planned testing during the 1996 tax filing season.
Similarly, IRS contracted in 1994 to build the Document Processing
System. After spending over a quarter of a billion dollars on the
project, IRS has now suspended the effort and is reexamining some of
its basic requirements, including which and how many forms should be
processed, and which and how many data should be read from the
documents.
It is clear that unless IRS has mature, disciplined processes for
acquiring software systems through contractors, it will be no more
successful in buying software than it has been in building software.
In conclusion, IRS still does not have (1) effective strategic
information management practices needed to manage TSM as an
investment, (2) mature and disciplined software development processes
needed to assure that systems built will perform as intended, (3) a
completed systems architecture that is detailed enough to guide and
control systems development, and (4) a schedule for accomplishing any
of the above. Accordingly, the Congress could consider limiting TSM
spending to only cost-effective modernization efforts that (1)
support ongoing operations and maintenance, (2) correct IRS'
pervasive management and technical weaknesses, (3) are small,
represent a low technical risk, and can be delivered in a relatively
short time frame, and (4) involve deploying already developed
systems, only if these systems have been fully tested, are not
premature--given the lack of a completed architecture--and produce a
proven, verifiable business value. As the Congress gains confidence
in IRS' ability to successfully develop these smaller, cheaper,
quicker projects, it could consider approving larger, more complex,
and more expensive projects in future years.
Further, IRS does not manage all of its current contractual efforts
effectively, and its plans to use a "prime" contractor and transition
much of its systems development to additional contractors are not
well defined. Accordingly, the Congress could consider requiring
that IRS institute disciplined systems acquisitions processes and
develop detailed plans and schedules before permitting IRS to
increase its reliance on contractors.
-------------------------------------------------------- Chapter 0:4.1
Mr. Chairman, this concludes my statement. I will be glad to answer
any questions.
*** End of document. ***