Tax Systems Modernization: Results of Review of IRS' March 7, 2000,
Expenditure Plan (Letter Report, 05/24/2000, GAO/AIMD-00-175).
Pursuant to a legislative requirement, GAO reviewed the Internal Revenue
Service's (IRS) second Information Technology Investment Account (ITIA)
expenditure plan, focusing on: (1) the progress IRS has made in meeting
the commitments in its first expenditure plan; (2) whether the plan
satisfies the conditions specified in the Department of the Treasury's
fiscal year 1998 and 1999 appropriations acts; (3) whether the plan is
consistent with GAO's open recommendations on IRS systems modernization;
and (4) whether GAO have any other observations about IRS systems
modernization efforts.
GAO noted that: (1) IRS met relatively few commitments in its $35
million first ITIA expenditure plan, even though the IRS later received
an additional $33 million and nearly 5 months of extra time to
accomplish the goals set forth in the plan; (2) notwithstanding IRS'
progress to date, GAO believes that its second expenditure plan
satisfies the legislative conditions placed on the use of ITIA funds,
and it is generally consistent with recommendations contained in GAO's
earlier reports for strengthening its modernization management
capability before building new systems; (3) in particular, the second
expenditure plan places appropriate emphasis and priority on
implementing and updating the modernization blueprint in light of recent
organizational restructuring and ongoing business process reengineering
as well as technology advances; (4) also, the plan provides for fully
implementing the Enterprise Life Cycle and related software acquisition
and investment management processes and slowing investments in new
systems until these management controls are established; (5) as was the
case with the first plan, the key to IRS' success will be whether it
effectively implements the second expenditure plan; (6) to improve on
its performance in implementing the first plan and to establish the much
needed management and technical foundation for modernizing its systems,
IRS will need to adhere to its stated commitment of first establishing
the institutional management and technical processes and the
architecture artifacts that are absolute prerequisites to building a
portfolio of interrelated systems that deliver promised functionality
and performance on time and within budget; (7) to establish its
modernization management and technical foundation capabilities and
refrain from building systems until it does so, IRS has recently
initiated actions, as described in the second expenditure plan, to
redirect and restructure its modernization effort; and (8) until it has
completed these actions, it will continue to lack key modernization and
technical controls, such as complete and enforced architecture, fully
implemented life cycle methodology, clearly defined contractor roles and
responsibilities, and fully implemented investment management controls.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-00-175
TITLE: Tax Systems Modernization: Results of Review of IRS' March
7, 2000, Expenditure Plan
DATE: 05/24/2000
SUBJECT: Tax administration systems
ADP procurement
Strategic information systems planning
Systems development life cycle
Information resources management
Financial management
Systems design
Systems conversions
IDENTIFIER: IRS Enterprise Life Cycle
IRS Tax System Modernization Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO/AIMD-00-175
Accounting and Information
Management Division
B-285317
May 24, 2000
The Honorable Ben Nighthorse Campbell
Chairman
The Honorable Byron L. Dorgan
Ranking Minority Member
Subcommittee on Treasury and
General Government
Committee on Appropriations
United States Senate
The Honorable Jim Kolbe
Chairman
The Honorable Steny H. Hoyer
Ranking Minority Member
Subcommittee on Treasury,
Postal Service, and General Government
Committee on Appropriations
House of Representatives
For over a decade, the Internal Revenue Service (IRS) has aimed to improve
service to our nation's taxpayers by attempting to modernize its antiquated
tax processing systems that IRS deems "fundamentally deficient." However,
this program to modernize systems has been plagued by management and
technical weaknesses that jeopardize its success. To correct these
weaknesses, replace the systems, and ultimately improve service to the
taxpayers, IRS requested, and the Congress established, a $506 million
Information Technology Investments Account (ITIA) via the Department of the
Treasury's fiscal year 1998 and 1999 appropriations acts.1 The acts limit
the obligation of ITIA funds until IRS submits to the Congress for approval,
an expenditure plan that (1) implements the IRS
Modernization Blueprint,2 (2) meets Office of Management and Budget (OMB)
investment guidelines for information systems, (3) is reviewed and approved
by IRS' Investment Review Board,3 OMB, and Treasury's IRS Management Board,
and is reviewed by GAO, (4) meets the requirements of IRS' system life cycle
management program,4 and (5) complies with acquisition rules, requirements,
guidelines, and system acquisition management practices of the federal
government. These legislative conditions are consistent with systems
modernization recommendations we have made to IRS but that IRS has not yet
fully implemented.
Pursuant to the appropriation acts, this report provides the results of our
review of IRS' second expenditure plan--entitled March 7, 2000, Information
Technology Investment Account (ITIA) Spending Plan--that includes reporting
on IRS progress against commitments made in its first expenditure plan
submitted in May 1999. As agreed with your offices, we reviewed IRS' second
plan to determine (1) what progress IRS has made in meeting the commitments
in its first expenditure plan, (2) whether the plan satisfies the conditions
specified in Treasury's fiscal year 1998 and 1999 appropriations acts, (3)
whether the plan is consistent with our open recommendations on IRS' systems
modernization, and (4) whether we have any other observations about IRS'
systems modernization efforts. We briefed your respective offices on the
results of our review on March 31, 2000, and, as agreed with your offices,
are issuing this report on those results after finalizing our analysis.
Prior to briefing your offices, we also briefed IRS on our results and
incorporated their comments where appropriate. IRS also provided us with
oral comments on a draft of this report, which are in the "Agency Comments"
section.
We performed our work from January through May 2000 in accordance with
generally accepted government auditing standards. (See appendix I for
details on our scope and methodology.)
IRS met relatively few commitments in its $35 million first ITIA expenditure
plan, even though the Service later received an additional
$33 million and nearly 5 months of extra time to accomplish the goals set
forth in the first plan. For example, the first plan stated that the systems
life cycle management methodology, known as the Enterprise Life Cycle (ELC),
was to be developed and implemented by October 31, 1999, at a cost of $8.22
million.5 As of the end of February 2000 and after investing about $9
million, the ELC had not yet been fully defined, staff had not yet been
trained on its use, and it was not being followed on most projects.
Notwithstanding IRS' progress to date, we believe that its second
expenditure plan satisfies the legislative conditions placed on the use of
ITIA funds, and it is generally consistent with recommendations contained in
our earlier reports6 for strengthening its modernization management
capability before building new systems. In particular, the second
expenditure plan places appropriate emphasis and priority on implementing
and updating the modernization blueprint (i.e., enterprise architecture and
sequencing plan) in light of recent organizational restructuring and ongoing
business process reengineering as well as technology advances. Also, the
plan provides for fully implementing the ELC and related software
acquisition and investment management processes and slowing investments in
new systems until these management controls are established.
As was the case with the first plan, the key to IRS' success will be whether
it effectively implements the second expenditure plan. To improve on its
performance in implementing the first plan and to establish the much needed
management and technical foundation for modernizing its systems, IRS will
need to adhere to its stated commitment of first establishing the
institutional management and technical processes and the architecture
artifacts that are absolute prerequisites to building a portfolio of
interrelated systems that deliver promised functionality and performance on
time and within budget.
To establish its modernization management and technical foundation
capabilities and refrain from building systems until it does so, IRS has
recently initiated actions, as described in the second expenditure plan, to
redirect and restructure its modernization effort. Until it has completed
these actions, it will continue to lack key modernization and technical
controls, such as a complete and enforced architecture, fully implemented
life cycle methodology, clearly defined contractor roles and
responsibilities, and fully implemented investment management controls.
Therefore, we reiterate our long-standing recommendations to the
Commissioner of Internal Revenue for correcting the above-noted weaknesses
and limiting IRS' investments in new systems until they are corrected. In
commenting on a draft of this report, IRS concurred with our findings and
conclusions and said that the initiatives in the second expenditure plan are
intended to address the identified management and technical weaknesses.
Over a decade ago, IRS began its systems modernization program, then called
Tax Systems Modernization (TSM), to establish a virtually paper-free tax
processing environment where taxpayer information would be readily available
to IRS employees for updating taxpayer accounts and responding to taxpayer
inquiries. In 1995, we identified serious management and technical
weaknesses with TSM that jeopardized its successful completion.7
Accordingly, we made over a dozen recommendations to fix the problems, such
as formulating a comprehensive business strategy, establishing information
technology (IT) investment management processes, and completing and
enforcing an integrated enterprise architecture. In addition, because of the
seriousness of the weaknesses, we designated TSM as a high-risk IT
initiative, placed the modernization on our list of high-risk federal
programs,8 and have continued to monitor this program.
Since then, IRS has taken some actions aimed at implementing our
recommendations. For example, in May 1997, IRS issued a modernization
blueprint to define, direct, and control investments in modernized systems
and related infrastructure. Also, in December 1998, IRS awarded its Prime
Systems Integration Services (PRIME) contract for systems modernization.
In 1998, the Congress established the ITIA and limited IRS' obligation of
ITIA funds until IRS submits to the Congress for approval an expenditure
plan that meets certain conditions.9 As mentioned earlier, the conditions
are that the plan should (1) implement the modernization blueprint,
(2) meet OMB's IT investment guidelines, (3) be reviewed and approved by
IRS' Investment Review Board, OMB, and Treasury's IRS Management Board and
be reviewed by GAO, (4) meet the requirements of IRS' life cycle program,
and (5) comply with acquisition rules, requirements, guidelines, and systems
acquisitions management practices of the federal government. To date, the
Congress has appropriated $506 million for the account via IRS' fiscal year
1998 and 1999 appropriations acts.10
In May 1999, IRS submitted its first or "initial" expenditure plan,
requesting about $35 million for modernization initiatives and commitments
to be delivered by October 31, 1999. As part of this plan, IRS also stated
its intention to modernize its systems incrementally and submit incremental
expenditure plans for release of ITIA funds. We reviewed the plan and
reported in June 199911 that this incremental approach was an industry best
practice, and if properly implemented, the plan was an appropriate first
step. However, to measure IRS' modernization performance and accountability
on this and future expenditure plans, we recommended that each plan fully
disclose IRS' progress against incremental goals, deliverables, and benefits
set forth in earlier plans. Based on our report, the House and Senate
Appropriations Subcommittees approved IRS'
$35 million expenditure plan in June 1999.
At that time, IRS planned to submit a second expenditure plan in October
1999. However, it was unable to do so on time, and in early December 1999,
submitted to the House and Senate appropriations subcommittees a "stopgap"
funding measure to obligate about $33 million from ITIA until the next plan
was submitted. We reviewed the "stopgap" funding measure and raised concerns
about projects that were scheduled to begin detailed design and software
development before, among other things, the enterprise architecture was
completed and the ELC was defined and implemented. Later that December, the
appropriations subcommittees approved IRS' $33 million "stopgap" funding
measure but in discussions and correspondence, directed IRS to (1) expedite
completion of the architecture and implementation of the ELC and (2) explain
in future expenditure plans how it plans to manage the risk of performing
detailed design or development work if the architecture is not completed or
the ELC is not implemented.
Subsequently, IRS reassessed its modernization program structure and plans
and restructured the program by scaling back its system development efforts
until it first put in place the requisite modernization management
capability. For example, it has efforts underway to update its modernization
blueprint and implement the ELC. The simplified diagram below graphically
depicts the ELC processes and major milestones that IRS is working to
implement.
On March 9, 2000, IRS submitted to the appropriations subcommittees its
second expenditure plan that reflected these restructuring decisions and
sought to obligate about $176 million from ITIA. Table 1 summarizes the
expenditure plan, including each initiative's cost, ELC milestone, and
milestone date.
Dollars in thousands
Proposed modernization Date to achieve
initiatives ELC milestone milestone Cost
Program-level activities
(category total) $66,556
Management Capabilities
Program Management Office N/A 9/30/2000 7,028
ELC Enhancements, Maintenance
and Full Deployment N/A 9/30/2000 5,113
Federally Funded Research and
Development Contractor N/A 9/30/2000 5,335
Management Reserve N/A N/A 10,507
Subtotal $27,983
Architecture Engineering
Enterprise Architecture and
Blueprint 2000 N/A 9/30/2000 8,667
Configuration Management
Policies and Procedures N/A 9/30/2000 1,411
Business Integration N/A 9/30/2000 4,012
Subtotal $14,090
Strategic Project Planning
Vision and Strategy, Tax
Administration 2 12/31/2000 20,508
Vision and Strategy, Internal
Management 2 11/30/2000 3,975
Subtotal $24,483
Project-level activities
(category total) $109,766
2001 Release −
Development
Customer Communications 3 5/05/2000 2,721
Customer Relationship
Management Exam 3 6/30/2000 1,359
Subtotal $4,080
2001 Release −
Implementation
Customer Communications 4, 5 5/31/2001 38,389
Customer Relationship
Management Exam 4, 5 6/30/2002 7,954
Management Reserve −
Filing Season 2001 Release 2,604
Subtotal $48,947
2002 Release − Design
Customer Communications 3 9/30/2000 3,509
e-Services 3 9/30/2000 3,854
Subtotal $7,363
Tax Account Replacement
Customer Account Data Engine
(CADE) 3 12/31/2000 15,312
Subtotal $15,312
Integrated Financial System
(IFS)
IFS Revenue Accounting
Tax Account Subledger
3 7/30/2000 1,175
Collections Subledger
3 7/30/2000 1,856
Subtotal
$3,031
Enabling Infrastructure
Security and Technology
Infrastructure Releases (STIR) 3 9/30/2000 8,549
Enterprise Systems Management
(ESM) 3 9/30/2000 3,448
Telecommunications Enterprise
Strategic Program (TESP) 2 9/30/2000 11,137
Solutions Demonstration Lab
(SDL) 3 9/30/2000 2,650
Virtual Development
Environment (VDE) 3 9/30/2000 4,400
Enterprise Integration and
Test Environment (EITE) 3 9/30/2000 849
Subtotal $31,033
Total $176,322
Short of Expectations
For its first expenditure plan, IRS requested about $35 million to initiate
selected modernization investments and produce specified deliverables by
October 31, 1999. This plan provided for three categories of investments:
(1) supporting business goals, (2) building management capability, and
(3) planning a modern IT infrastructure. The "supporting business goals"
initiatives included the early phases of selected systems acquisition
projects that are intended to improve service to taxpayers by the year 2001
tax filing season. The "building management capability" initiatives provided
for defining and beginning to institute at the program-level, mature
modernization management and systems engineering processes that are to
permit effective blueprint implementation. The "planning modern
infrastructure" initiatives refer to the first steps in establishing the
technology foundation−such as the networks, operating platforms, and
security systems−upon which to build, interconnect, and operate
modernized system applications.
In December 1999, the appropriations subcommittees approved IRS'
$33 million "stopgap" funding measure. In addition, by March 2000 when the
second expenditure plan was submitted, IRS had five additional months to
fulfill its incremental commitments in the first plan and to address three
new commitments added during IRS' execution of the first plan. Despite the
extra time and resources, IRS has not met, or does not know whether it has
met, 17 of its 19 commitments. For example, in the plan, IRS committed to
fully implementing the ELC by October 31, 1999. As noted earlier, the ELC is
to provide IRS with a disciplined and institutional approach−that is,
the policies, processes, and products−for managing IT investments
throughout their life cycles. As of February 29, 2000, IRS reported that it
had invested about $9 million in the ELC, but had not yet completely defined
and fully implemented it. In fact, until only recently, none of the
modernization initiatives were following the ELC because IRS and its
contractor staff had not been trained in its use. This is important because
failure to adhere to the management and engineering discipline embedded in
the ELC seriously jeopardizes a project's ability to deliver promised
capability on time and within budget.
As another example, IRS committed to defining the scope of its Customer
Relationship Management-Exam (CRM-Exam) project, which under the initial
plan, was to include redesigning IRS' examination processes and defining
system requirements to, among other things, enable faster exam case
resolution. By October 31, 1999, IRS had only begun to work on these tasks
and under its second expenditure plan, had significantly reduced the scope
of what the project would deliver in 2001, electing to postpone other parts
of the project to 2002 and beyond. Similarly, on its security and technology
infrastructure project, IRS committed to developing
(1) requirements and design specifications for the hardware and software
platforms and networks upon which its planned application projects
(e.g., CRM-Exam) would operate, (2) a sequencing plan or "roadmap" for
aligning the development and deployment of its security and technology
infrastructure with these application projects, (3) a detailed "tactical"
plan and schedule to guide the deployment of the infrastructure, and (4) the
costs and benefits of the infrastructure releases for use by the other
initiatives in developing their business case justifications. By October 31,
1999, IRS had not fully met these commitments. Table 2 summarizes IRS
progress against each of its 19 commitments.
IRS documents show IRS is in the process
Modernization initiative whether initiative met of determining
commitmentsa initiative statusb
Yes No
Business goals
Business Systems Planning
(BSP) X
Customer Account Data Engine
(CADE) X
Correspondence and Document
on Demand Imaging (CADDI) X
Customer Relationship
Management (CRM) Exam X
Customer Relationship
Management (CRM) Core X
Customer Relationship
Management (CRM) Collection X
Customer Communications X
e-Services: Near Term X
Integrated HR System:
Integrated Personnel System X
(IPS)
Integrated Financial System:
Revenue Accounting X
Enabling infrastructure
Infrastructure Program
Management and Integration X
Office (IPMO)
Security and Technology
Infrastructure Releases X
(STIR)
Enterprise Systems
Management (ESM) X
Telecommunications
Enterprise Strategic Program X
(TESP)
Solution Demonstration
Laboratory (SDL) and Virtual
Development Environment X
(VDE)c
Management capabilities
Enterprise Program
Management Office (EPMO) X
Process Partnership
Management Team (PPMT) X
Business Integration (BI)c X
Architecture Engineeringc X
Total 2 7 10
aMeeting its commitments means that IRS met the deliverables indicated in
the initial expenditure plan within 125 percent of estimates for time and
cost.
bIRS has not yet conducted reviews of these initiatives and therefore cannot
report at this time whether commitments were met.
cThese projects were added after submission of the initial expenditure plan.
IRS' second expenditure plan satisfies the conditions that the Congress
placed on the use of ITIA funds. Nonetheless, because the plan is to be
executed over many months, it does not provide that the IRS life cycle
program requirements, OMB investment guidelines, and federal acquisition
rules and management practices will be employed and adhered to on all
projects throughout the period covered by the plan because full
implementation of these requirements will not occur until September 2000.
Our assessment of the plan's satisfaction of each condition is detailed
below.
IRS' second expenditure plan provides for implementation of the
modernization blueprint, but the Service is still working to establish a
reliable date for completing the planned update of the blueprint. Developed
in May 1997, the blueprint includes, among other things, high-level
functional and technical architectures and a general transition plan
describing the timing of the introduction of new systems to establish the
modernized systems environment. We reviewed the May 1997 blueprint, and in
February 1998, reported that it was a good first step but that it lacked
sufficient precision and detail upon which to build modernized systems.
Accordingly, we made recommendations to complete the blueprint as part of a
longer-term implementation strategy, and IRS agreed to implement them.12
In mid-1998, following the enactment of the IRS Restructuring and Reform
Act13 and the announcement of IRS' plans to restructure its organization, we
told IRS' Chief Information Officer (CIO) that the blueprint still needed to
be completed, including being updated to reflect the act's requirements,
planned organizational changes and associated business process reengineering
efforts, and advances in technology. The CIO agreed, stating that IRS
already planned to do so. IRS' first expenditure plan reflected this
commitment. Specifically, it included blueprint implementation initiatives
aimed at updating the 1997 blueprint and beginning system acquisition
projects that were discussed in general terms in the 1997 blueprint, such as
establishing a modern corporate database for taxpayer accounts.
IRS' blueprint implementation strategy, as described in the second
expenditure plan, continues planning steps on various projects mentioned in
the blueprint, such as the taxpayer account database project, which IRS
calls the Customer Account Data Engine. The second expenditure plan also
provides for planning (i.e., requirements definition and preliminary design)
for certain new system investments cited in the 1997 blueprint. In addition,
the plan provides for updating the 1997 blueprint. For example, the plan
seeks about $8.7 million to develop an integrated set of about
50 architectural artifacts that are intended to describe in business and
technical terms the nature and characteristics of IRS future mode of
operations.
We reviewed the framework that IRS is using to construct this updated
blueprint and found it to be consistent with published government and
private-sector frameworks for enterprise architectures. We also found that
IRS' stated approach for updating the blueprint provides for integration of
efforts to reorganize, restructure, and reengineer IRS' business areas, with
efforts to specify the technology framework within which to modernize the
systems to support these business areas. Additionally, we found that IRS'
stated approach also provides for addressing business areas not covered by
the 1997 blueprint, such as IRS' internal management and administrative
business functions.
The second expenditure plan commits to completing the revised blueprint by
September 30, 2000. However, at the time of our review, IRS did not yet have
the detailed work breakdown structures, including the resource needs and
schedules, to support this milestone. IRS officials recognize the need to
develop this support and told us they were in the process of doing so.
Without this supporting analysis and decomposition of work steps, resource
needs, and time frames, there is limited basis for confidence in the
September 30, 2000, milestone.
IRS' second expenditure plan provides for meeting the requirements in IRS'
life cycle management program, which IRS refers to as its ELC. According to
the plan, the ELC is to be completed and implemented by September 30, 2000,
at a cost of $5.1 million. As of March 2000, IRS had developed a work
breakdown structure and supporting resource and schedule plans for
accomplishing this. Among other things, these detailed plans called for
creating a baseline version of the ELC that included such missing components
as life cycle management requirements for system security accreditation.
Additionally, the plans called for incorporating relevant components from
ongoing IRS efforts to define and implement OMB-required processes for IT
investment management (i.e., investment selection, control, and evaluation),
such as its recently developed procedures for preparing business case
justifications. The plans also called for ELC training for both IRS and
PRIME contractor staff. However, because the second expenditure plan
provides for investing in systems for several months before these planned
ELC efforts are completed, it does not anticipate that life cycle
requirements will be met on all projects during the period covered by the
plan.
IRS' expenditure plan provides for meeting OMB investment guidelines by
September 30, 2000.14 OMB's guidelines call for agencies to adopt a
data-driven, analytically based approach to selecting, controlling, and
evaluating IT investments. The overriding objective is to ensure that
investment decisions are made in a disciplined and rigorous manner based on
established criteria, such as return-on-investment and architectural
compliance, and to the extent possible, that system investments be broken
into a series of increments.
To complement its ELC and support management of the modernization, IRS
recently decided to incorporate its ongoing efforts to define and implement
investment management processes, which IRS refers to as its Investment
Decision Management (IDM) processes, into its ELC efforts. Thus far, IRS (1)
has developed business case procedures that are intended to ensure that
system investment decisions are based on compelling return-on-investment
justifications and (2) is working on completing other IT investment
management processes. IRS plans to complete and implement its IDM processes,
including their integration with the ELC, by September 2000. Consequently,
the second expenditure plan does not provide for adherence to OMB's guidance
until September 2000, when IRS' IT investment processes are scheduled to be
implemented.
Board, and OMB and Reviewed by GAO
IRS' Core Business Systems Executive Steering Committee (which replaced IRS'
Investment Review Board), Treasury's IRS Management Board, and OMB approved
the $176 million expenditure plan on March 9, 2000, and sent it to the
Congress that day. With its March 14, 2000, letter to us, IRS enclosed a
copy of the plan submitted to the Congress, and we reviewed it. We then
briefed the staffs of the House and Senate Appropriations Subcommittees on
our review, and the results are contained in this report.
Guidelines, and Management Practices
IRS' second expenditure plan provides for complying with federal acquisition
rules and management practices by September 30, 2000. According to federal
acquisition laws, rules, and regulations,15 agencies should, among other
things, use disciplined decision-making processes for planning, selecting,
managing, and controlling the acquisition of IT. By doing so, they mitigate
the risks of acquiring systems that are not delivered on time and on budget
and do not work as intended.
As mentioned earlier, IRS' expenditure plan calls for implementing the ELC,
which is to include mature software acquisition management practices and IT
investment management processes. In particular, IRS plans to implement
mature software/systems acquisition management practices within its Business
Systems Modernization Office, which is the IRS program office responsible
for managing the PRIME contractor and other modernization contractors. In
doing so, IRS intends to build the capability in accordance with the
Software Engineering Institute's (SEI) software acquisition capability
maturity model requirements.16 Among the maturity model's requirements are
disciplined and rigorous processes for acquisition planning, contractor
solicitation, contractor tracking and oversight, and acquisition risk
management.
While IRS' second expenditure plan provides for implementing these
acquisition capabilities, it does not anticipate this being done until
September 2000. Consequently, the second expenditure plan anticipates that
these capabilities will not be implemented until after the first six months
of plan execution. The absence of these capabilities, which we independently
verified to be the case on two modernization projects under the first
expenditure plan,17 can greatly reduce the chances of the projects
delivering promised system capabilities on time and within budget.
Recommendations
In 1995, we first made recommendations for correcting serious and pervasive
modernization management and technical weaknesses. Since then, IRS has taken
actions aimed at implementing our recommendations. We have been monitoring
IRS' actions and have made follow-up recommendations that recognize IRS'
progress and define residual steps that IRS needs to take to ensure that it
is ready to and capable of modernizing its systems effectively.
Our open recommendations fall into three categories: (1) completing the
modernization blueprint, (2) developing the management and engineering
capability to modernize systems effectively, and (3) until the first two
recommendations are implemented, limiting modernization spending to certain
small, cost-effective, low-risk efforts. In addition, based on our review of
IRS' first expenditure plan, we added the following two recommendations that
also remain open: (1) ensure that future expenditure plans fully disclose
IRS' progress against incremental goals, deliverables, and benefit
expectations specified in preceding plans and
(2) fully define and explain in the second expenditure plan the nature and
functioning of IRS' "partnership" with its contractors, including the
respective roles and responsibilities of IRS and its contractors.
IRS' second expenditure plan is generally consistent with these
recommendations. For example, the plan provides for updating the
modernization blueprint. Further, the plan provides for detailing
IRS/contractor roles and responsibilities and completing and implementing
system life cycle and investment management controls, including mature
software acquisition capabilities and other related controls. IRS plans to
have all of these capabilities and controls implemented by September 30,
2000.
In addition, the plan does not prematurely seek funds to build systems
(i.e., perform detailed system design and software development activities),
with two exceptions--Customer Communications 2001 Release and CRM-Exam 2001
Release. The purpose of the Customer Communications project is to improve
taxpayer telephone access to customer service representatives and other
sources by introducing new call routing capabilities. The purpose of the
CRM-Exam project is to provide revenue agents the tools needed to perform
complex tax calculations quickly during audits. We reviewed the extent of
detailed design and new software development associated with each of these
near-term projects and found both were more akin to system maintenance
projects than modernization projects, thus representing minimal technical
risk. More specifically, the Customer Communications project primarily
involves the purchase of more powerful and standard hardware and operating
platforms and commercial off-the-shelf (COTS) software. Similarly, the
CRM-Exam project is largely a COTS application software upgrade for agent
laptop computers. Equally important to these projects' technical risk is
whether they will produce a positive return on investment. While as of the
time of our review, IRS had yet to prepare business case justifications for
these projects to demonstrate that it had a cost-benefit basis for
proceeding with development and implementation, IRS officials stated that
such business cases, as specified in the ELC, are a prerequisite to
proceeding.
Our review disclosed three additional observations concerning IRS' second
expenditure plan and management of the modernization. First, IRS is in the
process of restructuring its modernization program management office, which
it calls its Business Systems Modernization Office (BSMO). This
restructuring is not included in IRS' second expenditure plan because the
BSMO is not funded from ITIA, but rather out of IRS' Information Systems
appropriation. Second, IRS is in the process of changing its approach to
defining task orders for the PRIME contractor to improve IRS/PRIME risk
allocation, leverage PRIME solution innovation, and clearly fix
accountability for contract deliverables. Third, the cost estimates provided
for most initiatives in the second expenditure plan were "rough estimates"
provided by the PRIME contractor and were neither based on detailed cost
analysis of initiatives' component tasks, deliverables, and time frames nor
were they subjected to IRS' independent analysis and negotiation.
Office
To effectively establish and implement the management and technical controls
previously discussed, such as an enterprise architecture and mature
acquisition processes, IRS must have a well-structured and fully operational
BSMO. IRS currently lacks such a BSMO, although it has established certain
immediate-need BSMO functions, and it is in the process of defining
structures, plans, and schedules for making this office fully operational.
For example, it has established a systems engineering and architecture
office to update, implement, and enforce the modernization blueprint;
created major project offices to manage and oversee PRIME contractor
progress on the projects; and established a process management group to
define and implement the ELC. However, it still needs, for example, to
appoint a full-time manager to head the program office and incorporate other
important functions essential to effectively managing the acquisition of
software-intensive systems, such as quality assurance, risk management, and
configuration management. IRS officials stated that they are in the process
of operationalizing the BSMO and addressing each of these areas.
IRS' plans for its BSMO are not discussed in its second expenditure plan
because according to IRS, the expenditure plan only covers contractor costs
and does not include IRS' internal costs of managing the modernization. That
is, IRS is only funding PRIME and support contractors (e.g., the Federally
Funded Research and Development Contractor) out of ITIA, and is funding
internal management costs out of its Information Systems and other
appropriations. For example, from October 1, 1998, through December 31,
1999, IRS reported that it spent about $17.0 million for internal
modernization management related costs.
Under IRS' contract with the PRIME contractor, IRS has been issuing task
orders to that contractor specifying activities to be performed, support to
be given, and difficult-to-measure deliverables to be produced. Further,
these task orders were generally not definitized, meaning that IRS and the
contractor never concluded negotiations to establish fixed prices and
milestones for the work and products delivered. Instead, IRS was largely
paying for the contractor's time and materials for work to be performed for
as long as it took to complete the work.
According to IRS officials, they decided to change their approach to
defining task orders because the current approach was not, among other
things, bringing the PRIME contractor's innovative expertise to bear at the
least possible cost and was not sharing modernization risks with the PRIME
contractor. IRS is now in the process of implementing performance-based task
orders. Under this approach, the contractor is given work objectives and the
discretion on how to best meet them. Additionally, the contractor agrees to
a fixed price cost for the work to be performed. To implement
performance-based task orders, IRS recently canceled the majority of its
existing task orders, trained its staff on performance-based contracting,
defined new task orders, and scheduled reviews to begin in mid-May 2000 for
task order approval and award.
Initiatives Is Unclear
IRS' second expenditure plan requests about $176 million to fund nine
initiatives (see table 1 for each initiative's respective funding amount).
While we did not independently validate the amounts requested for each
initiative, we did determine the extent to which the amounts were based on
verifiable analysis and rigorous estimation processes, and whether the
amounts were subject to government assessment and negotiation. With the
exception of three initiatives, we found that the reliability of the funding
estimates in IRS' second plan is unclear. Specifically, the estimates for
most of the planned initiatives were contractor-provided gross estimates
that were not based on a rigorous decomposition of the steps needed to
complete the initiatives (i.e., detailed work breakdown structures of tasks
and deliverables), and were not subjected to an IRS independent cost
assessment or negotiation with the PRIME contractor. This is not so much a
concern for the two near-term release initiatives (Customer Communications
2001 Release and CRM-Exam 2001 Release) because the primary cost drivers for
them are largely hardware and COTS software, the costs of which are easier
to estimate reliably. However, it is a concern for the other initiatives
because the cost driver for these is largely labor, which requires clear and
complete decomposition of work tasks to predict reliably. Of these other
initiatives, we found that only the Integrated Financial System initiative
had a detailed work breakdown structure and schedule.
IRS' performance in implementing the first expenditure plan has not met
expectations. A primary reason for this performance shortfall has been IRS'
inability to define and implement requisite modernization management and
technical controls, such as an enterprise architecture and a systems life
cycle methodology that includes mature software acquisition capabilities and
IT investment management processes. These control weaknesses are the same
ones that led to over $3 billion in wasteful spending on IRS' prior attempt
to modernize, and the same ones that our long-standing recommendations are
designed to address.
Notwithstanding IRS' performance to date, its second expenditure plan
includes initiatives that are intended to complete unfinished work started
under the first plan, define and implement missing management and technical
controls, and proceed in a manner that is generally in line with legislative
mandates and our open recommendations. Equally important, the plan does not
propose investing in new systems beyond ELC milestone 3, which is also
consistent with our open recommendations for not building systems (i.e.,
developing detailed system design and writing software-intensive
applications) until these controls are in place and functioning. In our
view, before any investment initiative proceeds past this milestone, it is
vital that it be justified by a thorough, compelling business case, and that
it employ full ELC rigor and discipline, including mature acquisition
processes and associated investment management processes. Additionally, for
any initiative proceeding past this milestone in advance of completing IRS'
enterprise architecture, it is critical that the technical risks of doing so
be defined and mitigated.
Because of the comprehensive nature of our current set of open
recommendations, we are not making additional recommendations at this time.
However, we wish to emphasize to the Commissioner of Internal Revenue that
these open recommendations remain operative and applicable, and they will
remain so until IRS (1) addresses the modernization technical and management
weaknesses discussed in this report by establishing the requisite controls,
such as a complete and enforced enterprise architecture/blueprint and a
complete and fully implemented ELC, and (2) does so before it begins
building new, software-intensive systems.
IRS provided oral comments on our briefing results and a draft of this
report, and we incorporated its comments where appropriate. In commenting on
the briefing and draft report, IRS concurred with our findings and
conclusions and said that the initiatives in the second expenditure plan are
intended to address the identified management and technical weaknesses.
We are sending copies of this report to Senator Max Baucus, Senator Robert
C. Byrd, Senator Orrin G. Hatch, Senator Joseph I. Lieberman, Senator Daniel
Patrick Moynihan, Senator William V. Roth, Jr., Senator Ted Stevens, and
Senator Fred Thompson, and to Representative Bill Archer, Representative Dan
Burton, Representative William J. Coyne, Representative Stephen Horn,
Representative Amo Houghton, Representative David R. Obey, Representative
Charles B. Rangel, Representative Jim Turner, Representative Henry A.
Waxman, and Representative C.W. Bill Young, in their capacities as Chairmen
or Ranking Minority Members of Senate and House Committees and
Subcommittees. We are also sending copies to the Honorable Charles O.
Rossotti, Commissioner of Internal Revenue; the Honorable Lawrence H.
Summers, Secretary of the Treasury; and the Honorable Jacob J. Lew, Director
of the Office of Management and Budget. Copies will also be made available
to others upon request.
If you or your staffs have any questions about this report please contact us
at (202) 512-6240 or by e-mail at [email protected] or [email protected]
. Key contributors to this report are listed in appendix II.
Randolph C. Hite
Associate Director
Governmentwide and Defense
Information Systems
Keith A. Rhodes
Director, Office of Computer
and Information Technology Assessment
Objectives, Scope, and Methodology
Pursuant to the Department of the Treasury's fiscal year 1998 and 1999
appropriations acts, the Congress limited IRS' obligation of ITIA funds
until IRS and Treasury submitted for its approval an expenditure plan that
per the acts, (1) implements the IRS Modernization Blueprint, (2) meets
OMB's investment guidelines for information systems, (3) is reviewed and
approved by IRS' Investment Review Board,18 OMB, and Treasury's IRS
Management Board and is reviewed by GAO, (4) meets the requirements of IRS'
system life cycle management program, and (5) is in compliance with
acquisition rules, requirements, guidelines, and system acquisition
management practices of the federal government. To satisfy this legislative
mandate, we reviewed IRS' March 7, 2000, expenditure plan. This was IRS'
second such plan submitted to the Congress. As agreed with the Senate and
House Appropriations Subcommittees, our objectives were to determine (1)
what progress IRS has made in meeting the commitments in its previous
expenditure plan, (2) whether the plan satisfies the conditions specified in
IRS' fiscal year 1998 and 1999 appropriations acts, (3) whether the plan is
consistent with our open recommendations on IRS' systems modernization, and
(4) whether we have any other observations about IRS' systems modernization
efforts.
To determine IRS' reported progress in implementing its first expenditure
plan, we identified in the initial plan the tasks and deliverables that IRS
had committed to deliver by October 31, 1999, with the $35 million approved
by the Congress as well as initiatives added during the execution of the
plan. We then compared this with IRS' progress as reported via its second
expenditure plan, management readiness reviews, and other progress tracking
documents. We also interviewed IRS and contractor officials responsible for
tracking modernization program progress.
To determine whether IRS' expenditure plan satisfied the conditions
specified in appropriations acts,19 we first identified and reviewed the
relevant IRS and federal documents referenced in the statutory conditions,
such as the modernization blueprint, OMB information systems investment
guidelines (e.g., Raines Rules), and the Federal Acquisition Regulation. We
then documented IRS' completed, ongoing, and planned modernization
initiatives. To do this, we reviewed IRS' March 7, 2000, second expenditure
plan, entitled the Information Technology Investment Account Spending Plan,
and other supporting documentation, such as the individual initiatives'
project plans and descriptions, briefings, the PRIME contract and associated
task orders, and Executive Steering Committee agendas and decision papers
proposing courses of action. We also interviewed IRS' Chief Information
Officer, other IRS officials, and PRIME contractor officials working on the
modernization program to gain an understanding of what IRS is doing to
satisfy the legislative conditions. This included receiving weekly briefings
and reports on how IRS and contractor teams were progressing on ongoing
initiatives, such as efforts to improve customer service, update the
modernization blueprint, build the capability to acquire systems
effectively, establish a new system development life cycle methodology
(i.e., ELC), define IRS and contractor roles and responsibilities, and
establish the systems modernization program office. We also reviewed the
business and systems development life cycle methodology that IRS is
modifying to develop its ELC and were briefed by IRS and its contractors
involved in this effort. We also attended IRS' Executive Steering Committee
meetings to determine how the modernization program was being managed,
including IRS' strategic modernization approach and progress. Last, we
analyzed each of IRS' modernization initiatives vis-�-vis the statutory
conditions to identify any variances or inconsistencies.
To determine whether IRS' second expenditure plan is consistent with our
past systems modernization recommendations, we extracted from our inventory
of open recommendations those pertaining to IRS' modernization and grouped
them into the following three categories: (1) completing the modernization
blueprint, (2) developing the management and engineering capability to
modernize systems effectively, and (3) limiting modernization spending to
certain small, cost-effective, low-risk efforts until the first two
recommendations are implemented. We added to these categories our recent
recommendations based on our review of IRS' first expenditure plan. We then
compared IRS' efforts on its completed, ongoing, and planned initiatives
with the intent of our open recommendations to identify any variances or
inconsistencies.
To determine other observations on IRS' management of the systems
modernization program and the second expenditure plan, we attended Executive
Steering Committee meetings to ascertain relevant program issues, such as
IRS' decision to redirect and restructure the program and the impetus for
this mid-course correction. To learn more about the underlying causes for
and IRS' progress in redirecting the program, we met with and interviewed
the Chief Information Officer and IRS program officials responsible for the
day-to-day management and control of the modernization. To assess the
readiness of the BSMO, we met with the Business Systems Modernization
Executive and other BSMO officials to understand IRS' plans and the steps it
is taking to make this office operational. In addition, to assess the
reliability of the cost estimates provided in IRS' second expenditure plan,
we reviewed the estimates and supporting documents provided by the
contractor and IRS for each of the initiatives in the plan to determine
whether there was an objective basis for each. We also interviewed IRS and
contractor officials to understand the processes that they followed in
developing the estimates used in the plan.
We performed our work at IRS headquarters in Washington, D.C., and its
facility in Lanham, Maryland, from January 2000 through May 2000 in
accordance with generally accepted government auditing standards.
GAO Contact and Staff Acknowledgments
Gary Mountjoy, (202) 512-6367
In addition to those named above, other key contributors were Bernard
Anderson, Lorne Dold, Timothy Hopkins, William Lew, Madhav Panwar, and Agnes
Spruill.
(511971)
1. The fiscal year 1998 Treasury and General Government Appropriations Act
(Public Law 105-61) and the fiscal year 1999 Omnibus Consolidated and
Emergency Supplemental Appropriations Act (Public Law 105-277).
2. In the conference report accompanying the fiscal year 1997 Omnibus
Appropriations Act (Public Law 104-208), the Congress directed Treasury and
IRS to develop a modernization blueprint. IRS submitted the IRS
Modernization Blueprint to the Congress on May 15, 1997. It consisted of
four principal components: (1) a systems life cycle, which defines the
policies, processes, and products for managing information technology
investments from conception, development, and deployment through maintenance
and support, (2) business requirements, (3) functional and technical
architectures, which define the critical attributes of an agency's
collection of information systems in both business/functional and
technical/physical terms, and (4) a sequencing plan that defines the actions
that must be taken and their schedules and costs to cost effectively evolve
from the current to the future systems operating environment.
3. According to IRS, the Investment Review Board has been replaced by the
Core Business Systems Executive Steering Committee, which is chaired by IRS'
Commissioner.
4. This program includes the policies, processes, and products for managing
information technology investments from conception, development, and
deployment through maintenance and support.
5. See the background section of this report for a simplified diagram
describing the major ELC phases and milestones.
6. For example, see Tax Systems Modernization: Blueprint Is a Good Start But
Not Yet Sufficiently Complete to Build or Acquire Systems
(GAO/AIMD/GGD-98-54 , February 24, 1998) and Tax Systems Modernization:
Management and Technical Weaknesses Must Be Corrected If Modernization Is to
Succeed (GAO/AIMD-95-156 , July 26, 1995).
7. GAO/AIMD-95-156 , July 26, 1995.
8. High-Risk Series: An Overview (GAO/HR-95-1 , February 1995), High-Risk
Series Information Management and Technology (GAO/HR-97-9 , February 1997),
and High-Risk Series: An Update (GAO/HR-99-1 , January 1999).
9. See footnote 1.
10. The Congress appropriated $325 million in fiscal year 1998 with the
funds set to expire on September 30, 2000. In May 1998, the Congress
rescinded $30 million of this ITIA appropriation for urgent Year 2000
requirements. In fiscal year 1999, the Congress appropriated $211 million
for ITIA with the funds set to expire on September 30, 2002.
11. Tax Systems Modernization: Results of Review of IRS' Initial Expenditure
Plan (GAO/AIMD/GGD-99-206 , June 15, 1999).
12. GAO/AIMD/GGD-98-54 , February 24, 1998.
13. Public Law 105-206, July 22, 1998.
14. Evaluating Information Technology Investments, A Practical Guide,
Version 1.0 (Executive Office of the President, OMB, November 1995) and OMB
Memorandum M-97-02, Funding Information Systems Investments (October 1996).
15. For example, see the Clinger-Cohen Act of 1996, OMB Circular A-109, and
the Federal Acquisition Regulation.
16. A model developed by the SEI at Carnegie Mellon University to evaluate
an organization's software acquisition capability.
17. We performed an SEI software acquisition capability maturity model
evaluation on two projects contained in the first expenditure plan (Customer
Communications and Electronic Services) and found that collectively IRS had
many weaknesses in the following key process areas: acquisition risk
management, contracting tracking and oversight, evaluation, project
management, requirements development and management, and software
acquisition planning. Because of the number and nature of weaknesses found,
IRS' current acquisition capability is unlikely to result in quality
software products being produced on time and within budget.
18. According to IRS, the Investment Review Board has been replaced by the
Core Business Systems Executive Steering Committee, which is chaired by IRS'
Commissioner.
19. Public Law 105-61, Treasury and General Government Appropriations Act,
1998, and Public Law 105-277, Omnibus Consolidated and Emergency
Supplemental Appropriations Act, 1999.
*** End of document. ***